Large number of indexed products are not shown

  • KumarG
    Participant
    # 1 year, 4 months ago

    Hey,
    iam new here and testing WpSolr with Aiven Elasticsearch.
    249797 products have been indexed, but when filtering, only 10,000 products are shown. I am using Aiven as Elasticsearch server.
    How to display all products?
    I can’t apply the solution here to Aiven or I don’t know how.
    https://www.wpsolr.com/fix-elasticsearch-error-result-window-is-too-large/
    Thanks in advance
    Best regards
    Kumar

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    Apply the cURL command described in https://www.wpsolr.com/fix-elasticsearch-error-result-window-is-too-large/ to increase your index window size.

    If you index is password protected, add parameter -u username:password

    KumarG
    Participant
    # 1 year, 4 months ago

    thanks
    it was successfull:
    xxxxx-MBP:~ xxxx$ curl -X PUT “https://avnadmin:xxxxxxx.aivencloud.com:14443/xxxxxxx/_settings” -H ‘Content-Type: application/json’ -d ‘{ “index” : { “max_result_window” : 1000000 } }’
    {“acknowledged”:true}xxxxx-MBP:~ xxxxx$

    But when I filter I get a total of 10K products displayed. I have 50 products per page so 200 pages are displayed. But should be shown more than 400 pages, because I have more than 100.000 in the category ladies.
    Thank ins advanced

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    Are you working with the same index in WPSOLR?
    (the same “my_index” name in WPSOLR and in /my_index/_settings)

    KumarG
    Participant
    # 1 year, 4 months ago

    Yes, it’s the same Index

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    Can you check your index settings with the same query with GET?

    (In Kibana: GET /my_index/_settings)

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    If the settings window size is set as expected, you should ask your Elasticsearch hosting provider support.
    Elasticsearch is recommending not to increase the default value, as it is very resource intensive. Perhaps updating the setting on your index is not authorized?

    KumarG
    Participant
    # 1 year, 4 months ago

    with GET /my_index/_settings i got this:
    hope this Help
    {“acceptance_random_674568745398rh87r943ihuf87”:{“settings”:{“index”:{“number_of_shards”:”5″,”provided_name”:”acceptance_random_674568745398rh87r943ihuf87″,”max_result_window”:”1000000″,”creation_date”:”1614092785098″,”analysis”:{“filter”:{“stemmer”:{“type”:”stemmer”,”language”:”english”},”autocompleteFilter”:{“max_shingle_size”:”4″,”min_shingle_size”:”2″,”type”:”shingle”},”stopwords”:{“type”:”stop”,”stopwords”:[“_english_”]}},”analyzer”:{“default_search”:{“type”:”english”},”default”:{“type”:”english”},”wpsolr_analyser_autocomplete”:{“filter”:[“lowercase”,”autocompleteFilter”],”char_filter”:[“html_strip”],”type”:”custom”,”tokenizer”:”standard”},”wpsolr_analyser_did_you_mean”:{“filter”:[“lowercase”],”char_filter”:[“html_strip”],”type”:”custom”,”tokenizer”:”standard”}}},”number_of_replicas”:”0″,”uuid”:”NN_lxMjuQu-VgCJ0jn0WIA”,”version”:{“created”:”7090399″}}}}}

    KumarG
    Participant
    # 1 year, 4 months ago

    I Have done some tests:
    if you go to the to (removed)
    you can see ate the Bottom 200 Pages.

    – first you Select Nike in filter “Marke”
    You can see at the Bottom 6 pages

    – second you Select Nike and Schuhegröße “32,5 EU”
    you see at the Bottom over 4000 Pages.

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    Which Elasticsearch version is it?

    KumarG
    Participant
    # 1 year, 4 months ago

    Elasticsearch version 7.9.3

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    This probably explains why PUT /my_index/_settings { "max_result_window" : 1000000 } } does not work. I tried it on my local Elasticsearch 7.x, and got the same issue.
    It worked though on an Elasticsearch 6.x as described on my documentation.

    Strangely, the 7.x documentation of the update settings API states that it should work on all versions.

    KumarG
    Participant
    # 1 year, 4 months ago

    I got the following feedback from Aiven, that the value has changed :
    2021-02-25T14:14:03.752507+0000 xxxxxx elasticsearch7: [2021-02-25T14:14:03,752][INFO ][o.e.c.s.IndexScopedSettings] [xxxxxxx] [acceptance_random_674568745398rh87r943ihuf87] updating [index.max_result_window] from [10000] to [1000000]

    wpsolr
    Keymaster
    # 1 year, 4 months ago

    Not sure it worked though.

    You could try to use an index template to add the max_result_window at index creation time instead: https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html

    You will need to create a new index in WPSOLR after.

    KumarG
    Participant
    # 1 year, 4 months ago

    I am not that techy, I will do a final test with Elastic Cloud and see if it works. Since I also just saw that the current version has not yet been tested with Aiven, but with Elastic Cloud it has.
    Testing the plugin only goes 7 days right?

Viewing 15 posts - 1 through 15 (of 21 total)

You must be logged in to reply to this topic.