Optimizations to index a million WooCommerce products
-
pseudo visitorParticipant4 years, 8 months ago #6845
We have more than a million products, and so temporarily upgraded the RAM and CPU to 64 GB RAM and 16 CPU.
However, I need your help in defining the right or maximum values in PHP and FPM settings.
Please let me know if I need to optimize any value for a quick indexing on an Elasticsearch host.
pseudo visitorParticipant4 years, 8 months ago #6847It is not related to WPSOLR (Your Elasticsearch server being hosted elsewhere).
You should ask to WordPress performance specialists, like https://www.wpintense.com/.
pseudo visitorParticipant4 years, 8 months ago #6848We are using Runcloud to manage the website server.
Both the website and Elasticsearch are hosted on Digitalocean.
The synchronization speed is very slow and I believe entering the right PHP and FMP settings will help.
We will try with this first.
wpsolrKeymaster4 years, 8 months ago #6849If your performance is about indexing, you’ll have 2 factors to consider:
– WordPress
Indexing requires extracting data from WP SQL database. It requires essentially SQL performance.– Elasticsearch
As you are hosted, a few hundreds of seconds are lost each time you send a batch of documents to index.
You can consider increasing the batch size while indexing. But it can then drain more resources from WordPress.pseudo visitorParticipant4 years, 8 months ago #6850It worked.
I just increased the Memory limit to 60 GB on 64 GB RAM server and as of now, I am getting a speed of 1,000 per minute. I hope it will be completed by tomorrow.
Meanwhile, please let me know how to test the imports. I can see a total of 20,000 products imported.
pseudo visitorParticipant4 years, 8 months ago #6852Yes, the database is not optimized and this is why I am getting a slow speed. The batch size is also 1000.
You must be logged in to reply to this topic.