Optimizations to index a million WooCommerce products

  • pseudo visitor
    Participant
    # 9 months, 3 weeks ago

    We have more than a million products, and so temporarily upgraded the RAM and CPU to 64 GB RAM and 16 CPU.

    However, I need your help in defining the right or maximum values in PHP and FPM settings.

    Please let me know if I need to optimize any value for a quick indexing on an Elasticsearch host.

    pseudo visitor
    Participant
    # 9 months, 3 weeks ago

    It is not related to WPSOLR (Your Elasticsearch server being hosted elsewhere).

    You should ask to WordPress performance specialists, like https://www.wpintense.com/.

    pseudo visitor
    Participant
    # 9 months, 3 weeks ago

    We are using Runcloud to manage the website server.

    Both the website and Elasticsearch are hosted on Digitalocean.

    The synchronization speed is very slow and I believe entering the right PHP and FMP settings will help.

    We will try with this first.

    wpsolr
    Keymaster
    # 9 months, 3 weeks ago

    If your performance is about indexing, you’ll have 2 factors to consider:

    – WordPress
    Indexing requires extracting data from WP SQL database. It requires essentially SQL performance.

    – Elasticsearch
    As you are hosted, a few hundreds of seconds are lost each time you send a batch of documents to index.
    You can consider increasing the batch size while indexing. But it can then drain more resources from WordPress.

    pseudo visitor
    Participant
    # 9 months, 3 weeks ago

    It worked.

    I just increased the Memory limit to 60 GB on 64 GB RAM server and as of now, I am getting a speed of 1,000 per minute. I hope it will be completed by tomorrow.

    Meanwhile, please let me know how to test the imports. I can see a total of 20,000 products imported.

    wpsolr
    Keymaster
    # 9 months, 3 weeks ago

    I am getting a speed of 1,000 per minute

    1000 per minute seems slow. How much is your batch size ?

    pseudo visitor
    Participant
    # 9 months, 3 weeks ago

    Yes, the database is not optimized and this is why I am getting a slow speed. The batch size is also 1000.

Viewing 7 posts - 1 through 7 (of 7 total)

You must be logged in to reply to this topic.