Filter query a multivalued field with on multiple values

  • HaikoG
    Participant
    2 years, 9 months ago #18146

    Hi,

    We have to filter search results on a field with a ‘OR’ statement, but even though the SOLR webinterface gives me results, when I run the sam query through wpsolr I get no results.

    I’m searching in a field called ‘pricelist_ids’, which can contain multiple strings. In the Solr webinterface I use the following fq:
    pricelist_ids:(“131” OR “22”)

    This gives me items with pricelist_ids 131 or 22 or both.

    In WPSOLR I use the WPSOLR_FILTER_UPDATE_WPSOLR_QUERY filter to add the filter query

    $current_query = $wpsolr_query->get_filter_query_fields();
    $current_query[] = ‘pricelist_ids:(“1” OR “‘.$pricelist.'”)’;
    $wpsolr_query->set_filter_query_fields($current_query);

    When logging the resulting query, it shows a correct filter query (when I copy this to Solr I do get results)

    [wpsolr_filter_query:protected] => Array
    (
    [0] => pricelist_ids:(“1” OR “131”)
    )

    When I use either pricelist_ids:(“1”) or pricelist_ids:(“131”) I get results from WPSOLR, but together doesn’t.

    I tried multiple setups
    pricelist_ids:(“1”, “131”)
    pricelist_ids:(1, 131)
    pricelist_ids:+(1, 131)

    even tried 2 seperate fq entries; one with pricelist_ids:(1) and one with pricelist_ids:(131)

    I think I’m missing something.. Can you share your thoughts on this?

    wpsolr
    Keymaster
    2 years, 9 months ago #18147

    What would be good is to see the query url parameters somewhere in the Solr logs. Perhaps there are some encoding issues.

    HaikoG
    Participant
    2 years, 9 months ago #18154

    I had to enable to logging, which was a bit of a hassle but requests are logged now. Unfortunately POST requests are not logged, so I had to revert to GET requests but the search results were the same. Here goes:

    Single pricelist_ids:

    [27/Feb/2020:08:11:12 +0000] “GET /solr/xxx/select?omitHeader=true&wt=json&json.nl=flat&q=%28probook%29&start=0&rows=12&fl=id%2CPID%2Ctype%2Cmeta_type_s%2Ctitle%2Cnumcomments%2Ccomments%2Cdisplaydate%2Cdisplaymodified%2C%2Acategories_str%2Cauthor%2C%2Apost_thumbnail_href_str%2C%2Apost_href_str%2Csnippet_s%2C_stock_str%2Cshop_thumbnail_url%2C_thumbnail_id_i%2C_visibility_str%2Cpermalink&q.op=AND&sort=_stock_i+desc&fq=type%3A%28%22product%22%29&fq=-post_status_s%3A%28%22draft%22+OR+%22pending%22+OR+%22trash%22+OR+%22future%22+OR+%22private%22+OR+%22auto-draft%22%29&fq=%7B%21tag%3D%7Dpricelist_ids%3A%22156%22&fq=%7B%21tag%3D%7Dtype%3A%22product%22&fq=%7B%21tag%3D%7Dtype%3A%22product%22&fq=%28%28%2A%3A%2A+-product_visibility_str%3A%2A%29+OR+-product_visibility_str%3A%28exclude-from-search%29%29&fq=%28%2A%3A%2A+-is_excluded_s%3A%5B%2A+TO+%2A%5D%29+OR+is_excluded_s%3A%28n%29&fq=%7B%21tag%3D%7Dtype%3A%22product%22&defType=edismax&qf=product_cat_t%5E5+title%5E5+content%5E2+ean_code_str+_sku_s+pa_ean-code_str+pa_frequentie-van-processor_str+pa_geheugencapaciteit_str+pa_harde-schijf-omvang_str+pa_opslagcapaciteit_str+pa_compatibele-producten_str%5E0.1+_product_attributes_str%5E0.2+pa_beeldschermdiagonaal_str+pa_capaciteit_str+pa_intern-geheugen_str+pa_interne-opslagcapaciteit_str+pa_interne-ram_str+pa_maximale-beeldresolutie_str+pa_maximale-opslagcapaciteit_str+pa_maximale-resolutie_str+pa_maximum-resolutie_str+pa_opslag-schijfgrootte_str+pa_opslagcapaciteit-harde-schijf_str+pa_processorfamilie_str+pa_processormodel_str+pa_projector-native-resolution_str+pa_resolutie_str+pa_schermgrootte_str+pa_schermgrootte-display_str+pa_schermresolutie-tablet_str+pa_standaardcapaciteit_str+pa_totale-opslagcapaciteit_str+pa_type-processor_str&facet=true&facet.mincount=1&facet.limit=12&facet.field=%7B%21key%3Dproduct_cat_str+ex%3Dfct_ex_product_cat_str%7Dproduct_cat_str&facet.field=%7B%21key%3Dpa_merk_str+ex%3Dfct_ex_pa_merk_str%7Dpa_merk_str&facet.field=%7B%21key%3Dcategories_str+ex%3Dfct_ex_categories%7Dcategories_str&facet.field=%7B%21key%3Dpa_beeldschermdiagonaal_str+ex%3Dfct_ex_pa_beeldschermdiagonaal_str%7Dpa_beeldschermdiagonaal_str&facet.field=%7B%21key%3Dpa_inclusief-besturingssysteem_str+ex%3Dfct_ex_pa_inclusief-besturingssysteem_str%7Dpa_inclusief-besturingssysteem_str&facet.field=%7B%21key%3Dpa_ingebouwde-luidsprekers_str+ex%3Dfct_ex_pa_ingebouwde-luidsprekers_str%7Dpa_ingebouwde-luidsprekers_str&facet.field=%7B%21key%3Dpa_intern-geheugen_str+ex%3Dfct_ex_pa_intern-geheugen_str%7Dpa_intern-geheugen_str&facet.field=%7B%21key%3Dpa_kleur-van-het-product_str+ex%3Dfct_ex_pa_kleur-van-het-product_str%7Dpa_kleur-van-het-product_str&facet.field=%7B%21key%3Dpa_processorfamilie_str+ex%3Dfct_ex_pa_processorfamilie_str%7Dpa_processorfamilie_str&facet.field=%7B%21key%3Dpa_processormodel_str+ex%3Dfct_ex_pa_processormodel_str%7Dpa_processormodel_str&facet.field=%7B%21key%3Dpa_resolutie_str+ex%3Dfct_ex_pa_resolutie_str%7Dpa_resolutie_str&facet.field=%7B%21key%3Dpa_totale-opslagcapaciteit_str+ex%3Dfct_ex_pa_totale-opslagcapaciteit_str%7Dpa_totale-opslagcapaciteit_str&facet.field=%7B%21key%3Dpa_touchscreen_str+ex%3Dfct_ex_pa_touchscreen_str%7Dpa_touchscreen_str&facet.field=%7B%21key%3Dpa_ssd-interface_str+ex%3Dfct_ex_pa_ssd-interface_str%7Dpa_ssd-interface_str&f.pa_inclusief-besturingssysteem_str.facet.sort=index&stats=true&stats.field=%7B%21ex%3Dfct_ex__price_str%7D_price_f&hl=true&hl.fl=title%2Ccontent%2Ccomments&f.title.hl.fragsize=100&f.title.hl.simple.pre=%3Cb%3E&f.title.hl.simple.post=%3C%2Fb%3E&f.content.hl.fragsize=100&f.content.hl.simple.pre=%3Cb%3E&f.content.hl.simple.post=%3C%2Fb%3E&f.comments.hl.fragsize=100&f.comments.hl.simple.pre=%3Cb%3E&f.comments.hl.simple.post=%3C%2Fb%3E HTTP/1.1” 200 6082

    multiple pricelists_ids with OR statement:

    [27/Feb/2020:08:13:38 +0000] “GET /solr/xxx/select?omitHeader=true&wt=json&json.nl=flat&q=%28probook%29&start=0&rows=12&fl=id%2CPID%2Ctype%2Cmeta_type_s%2Ctitle%2Cnumcomments%2Ccomments%2Cdisplaydate%2Cdisplaymodified%2C%2Acategories_str%2Cauthor%2C%2Apost_thumbnail_href_str%2C%2Apost_href_str%2Csnippet_s%2C_stock_str%2Cshop_thumbnail_url%2C_thumbnail_id_i%2C_visibility_str%2Cpermalink&q.op=AND&sort=_stock_i+desc&fq=type%3A%28%22product%22%29&fq=-post_status_s%3A%28%22draft%22+OR+%22pending%22+OR+%22trash%22+OR+%22future%22+OR+%22private%22+OR+%22auto-draft%22%29&fq=%7B%21tag%3D%7Dpricelist_ids%3A%22%28%5C%221%5C%22+OR+%5C%22156%5C%22%29%22&fq=%7B%21tag%3D%7Dtype%3A%22product%22&fq=%7B%21tag%3D%7Dtype%3A%22product%22&fq=%28%28%2A%3A%2A+-product_visibility_str%3A%2A%29+OR+-product_visibility_str%3A%28exclude-from-search%29%29&fq=%28%2A%3A%2A+-is_excluded_s%3A%5B%2A+TO+%2A%5D%29+OR+is_excluded_s%3A%28n%29&fq=%7B%21tag%3D%7Dtype%3A%22product%22&defType=edismax&qf=product_cat_t%5E5+title%5E5+content%5E2+ean_code_str+_sku_s+pa_ean-code_str+pa_frequentie-van-processor_str+pa_geheugencapaciteit_str+pa_harde-schijf-omvang_str+pa_opslagcapaciteit_str+pa_compatibele-producten_str%5E0.1+_product_attributes_str%5E0.2+pa_beeldschermdiagonaal_str+pa_capaciteit_str+pa_intern-geheugen_str+pa_interne-opslagcapaciteit_str+pa_interne-ram_str+pa_maximale-beeldresolutie_str+pa_maximale-opslagcapaciteit_str+pa_maximale-resolutie_str+pa_maximum-resolutie_str+pa_opslag-schijfgrootte_str+pa_opslagcapaciteit-harde-schijf_str+pa_processorfamilie_str+pa_processormodel_str+pa_projector-native-resolution_str+pa_resolutie_str+pa_schermgrootte_str+pa_schermgrootte-display_str+pa_schermresolutie-tablet_str+pa_standaardcapaciteit_str+pa_totale-opslagcapaciteit_str+pa_type-processor_str&facet=true&facet.mincount=1&facet.limit=12&facet.field=%7B%21key%3Dproduct_cat_str+ex%3Dfct_ex_product_cat_str%7Dproduct_cat_str&facet.field=%7B%21key%3Dpa_merk_str+ex%3Dfct_ex_pa_merk_str%7Dpa_merk_str&facet.field=%7B%21key%3Dcategories_str+ex%3Dfct_ex_categories%7Dcategories_str&facet.field=%7B%21key%3Dpa_beeldschermdiagonaal_str+ex%3Dfct_ex_pa_beeldschermdiagonaal_str%7Dpa_beeldschermdiagonaal_str&facet.field=%7B%21key%3Dpa_inclusief-besturingssysteem_str+ex%3Dfct_ex_pa_inclusief-besturingssysteem_str%7Dpa_inclusief-besturingssysteem_str&facet.field=%7B%21key%3Dpa_ingebouwde-luidsprekers_str+ex%3Dfct_ex_pa_ingebouwde-luidsprekers_str%7Dpa_ingebouwde-luidsprekers_str&facet.field=%7B%21key%3Dpa_intern-geheugen_str+ex%3Dfct_ex_pa_intern-geheugen_str%7Dpa_intern-geheugen_str&facet.field=%7B%21key%3Dpa_kleur-van-het-product_str+ex%3Dfct_ex_pa_kleur-van-het-product_str%7Dpa_kleur-van-het-product_str&facet.field=%7B%21key%3Dpa_processorfamilie_str+ex%3Dfct_ex_pa_processorfamilie_str%7Dpa_processorfamilie_str&facet.field=%7B%21key%3Dpa_processormodel_str+ex%3Dfct_ex_pa_processormodel_str%7Dpa_processormodel_str&facet.field=%7B%21key%3Dpa_resolutie_str+ex%3Dfct_ex_pa_resolutie_str%7Dpa_resolutie_str&facet.field=%7B%21key%3Dpa_totale-opslagcapaciteit_str+ex%3Dfct_ex_pa_totale-opslagcapaciteit_str%7Dpa_totale-opslagcapaciteit_str&facet.field=%7B%21key%3Dpa_touchscreen_str+ex%3Dfct_ex_pa_touchscreen_str%7Dpa_touchscreen_str&facet.field=%7B%21key%3Dpa_ssd-interface_str+ex%3Dfct_ex_pa_ssd-interface_str%7Dpa_ssd-interface_str&f.pa_inclusief-besturingssysteem_str.facet.sort=index&stats=true&stats.field=%7B%21ex%3Dfct_ex__price_str%7D_price_f&hl=true&hl.fl=title%2Ccontent%2Ccomments&f.title.hl.fragsize=100&f.title.hl.simple.pre=%3Cb%3E&f.title.hl.simple.post=%3C%2Fb%3E&f.content.hl.fragsize=100&f.content.hl.simple.pre=%3Cb%3E&f.content.hl.simple.post=%3C%2Fb%3E&f.comments.hl.fragsize=100&f.comments.hl.simple.pre=%3Cb%3E&f.comments.hl.simple.post=%3C%2Fb%3E&spellcheck=true&spellcheck.q=probook&spellcheck.count=10&spellcheck.extendedResults=true&spellcheck.collate=true&spellcheck.collateExtendedResults=true HTTP/1.1” 200 1023

    wpsolr
    Keymaster
    2 years, 9 months ago #18155

    I cannot see pricelists_ids in your logs.

    HaikoG
    Participant
    2 years, 9 months ago #18156

    They are urlencoded.
    The first one is
    fq=%7B%21tag%3D%7Dpricelist_ids%3A%22156%22
    or decoded: fq={!tag=}pricelist_ids:”156″

    fq=%7B%21tag%3D%7Dpricelist_ids%3A%22%28%5C%221%5C%22+OR+%5C%22156%5C%22%29%22
    or decoded: fq={!tag=}pricelist_ids:”(\”1\”+OR+\”156\”)”

    I think the extra quotes around the brackets in the second one should not be there.

    wpsolr
    Keymaster
    2 years, 9 months ago #18157

    Yes, the extra quotes in the 2nd query are strange.

    HaikoG
    Participant
    2 years, 9 months ago #18251

    Any ideas how to get rid of these quotes?

    HaikoG
    Participant
    2 years, 9 months ago #18252

    Found it:

    class-wpsolr-searchsolariumclient.php
    replace line 298: $field_value_escaped = “\”$field_value\””;
    with:
    if (strpos($field_value, “(“, 0) === false){
    $field_value_escaped = “\”$field_value\””;
    }else{
    $field_value_escaped = $field_value;
    }

    class-wpsolr-abstractsearchclient.php
    replace line 1618: $filter_query_field_value_escaped = “\”$filter_query_field_value\””;
    with:
    // In case the facet contains white space, we enclose it with “”.
    if (strpos($filter_query_field_value, “(“, 0) === false){
    $filter_query_field_value_escaped = “\”$filter_query_field_value\””;
    }else{
    $filter_query_field_value_escaped = $filter_query_field_value;
    }

    wpsolr
    Keymaster
    2 years, 9 months ago #18253

    Thanks for the tip.

    I will integrate that fix in the next release.

Viewing 9 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic.