Remove 10000 limit when using elasticsearch
-
wpsolrKeymaster3 years, 6 months ago #18819
@maxwatson Can you show me exactly the problem in a screenshot?
wpsolrKeymaster3 years, 6 months ago #18822Clients with the same WPSOLR as yours can display hundreds of thousands of results. No need of Elasticsearch cursors. Cursors are only useful for paginating above 10K results (displaying search page 1000, if a page contains 10 results). But you are showing page 1.
wpsolrKeymaster3 years, 6 months ago #18823It could be due to a breaking change for Elasticsearch 7, which returns 10,000 total documents maximum, unless track-total-hits is set on each query:
https://www.elastic.co/guide/en/elasticsearch/reference/current/breaking-changes-7.0.html#track-total-hits-10000-defaultmaxwatsonParticipant3 years, 6 months ago #18827Yes, it’s true, if I set track_total_hits=true, i get all the results
“hits” : {
“total” : {
“value” : 176275,
“relation” : “eq”
},
But how to add this parameter to each query?It isn’t possible to add it via configuration, i need to set it on every request.
wpsolrKeymaster3 years, 2 months ago #21347The detailed tutorial to fix the 10,000 results in Elasticsearch is on Fix Elasticsearch error “Result window is too large”
mneimneParticipant1 year, 5 months ago #29617Hello
I followed your guide on how to make Elasticsearch return more than 10,000 results, however it doesn’t seem to be working, always 10K results are returned.
Knowing that the settings API call on that index indicates that this value was set to more than 10K.
Elasticsearch version 7.6.2wpsolrKeymaster1 year, 5 months ago #29619Indeed, Elasticsearch settings as described in our documentation do not work anymore with the recent versions of Elasticsearch / OpenSearch.
Here is a fix, released in next version of WPSOLR) with a new option “Show all results” in screen 2.1: https://trello.com/c/RYsQ9hr8/189-add-option-to-show-all-results-with-elasticsearch: it sets the parameter ‘track_total_hits’ to ‘true’ for all queries.
(It has been tested on our Kinsta demo with 100,000 products: https://demo-woocommerce-flatsome-kinsta-100k-products.wpsolr.com/shop/ )
wpsolrKeymaster1 year, 1 month ago #30628The fix is now part of WPSOLR 22.3: you can override the Elasticsearch 10,000 default limit from the view’s configuration.
You must be logged in to reply to this topic.