Send data tab takes 22 minutes to load
Tab 2.3. takes about 22 minutes to load. My site may be pushing the limits of Woocommerce as a platform, but this long of a loading time started somewhere about version 21.4 or so.
Since wordfence won’t let me copy/paste sql statements here, here is a link to a text file containing three super-slow queries:
Also, the indexing itself is super-slow. I used to be able to index 65000 products in a matter of minutes, but now it takes like half an hour for each 300 chunk.
I will make stress tests.
The difference between taxonomy and post types indexing is that taxonomy terms does not have a publication date. Hence, no way to perform incremental indexing. So, the query that retrieves taxonomy terms to index is a join with a custom table that stores the last indexed date for the taxonomy terms. This join could be the reason of the slowness.
I made my tests with 1000 terms, and everything was good.
But with 10,000 terms, the SQL to retrieve terms is really slow. It appears a “sql inner join” between 10 thousands and 10 thousands terms is not viable.
I made some preliminary tests to replace it with a slightly modified ” sql union”, and it’s very fast for the same 10 thousand terms.
After verification of the speed, I’m now checking if all the other parts of the indexing code can be adapted too. Not trivial, because the code was custom built around the “sql inner join”.
Here is a new version with the fix: https://www.dropbox.com/s/r9zyw53xx2kat03/wpsolr-pro-taxonomies-indexing-fast.zip?dl=0
Tested with 10,000 terms indexed in Elasticsearch. At least as fast as with post types indexing.
Thanks for working on this so quickly. It looks like this table:
WordPress database error: [Table ‘woo.wpsolr_acceptance_wpsolr_index_history’ doesn’t exist]
Is not being created at any point. I tried removing/re-adding plugin, but it still gives that error on send data.
Sorry for that. Here is the fix: https://www.dropbox.com/s/hjs2j9am5pi7lom/wpsolr-pro-taxonomies-indexing-fast-1.zip?dl=0
You must be logged in to reply to this topic.