[sannin]

8 months, 3 weeks ago #35325

Hey,

As you probably know, woocommerce related products are mostly crap. I believe you could make a new feature to create related product recommendation when a vector database is used. This is my use case ( i have hardcoded some variables 😛 ):

— Get the similar products for a single product id. I filter only the instock products in same category. I tried the new rerank module with not much success, maybe because it is trained on english data.

function get_weaviate_related_products( $pid ) {
	if ( ! $pid ) {
		return false;
	}

	$uuid = get_post_meta( $pid, 'wpsolr_weaviate_uuid4', true );

	if ( ! $uuid ) {
		return false;
	}

        $product_cats = get_the_terms( $pid, 'product_cat' );

	if ( ! $product_cats || is_wp_error( $product_cats ) ) {
        	return false;
	}

        $cat_name = $product_cats[0]->name;

	$weaviate_url = 'https://localhost:8080/v1/graphql';

        $query = sprintf('
        {
          Get {
            Mc_index (
              nearObject: {
                id: "%s"
              }
              autocut: 6
              where: {
                operator: And,
                operands: [{
                        path:["wpsolr__stock_status_str"],
                        operator: Like,
                        valueText: "instock"
                }, {
                        path:["wpsolr_non_flat_hierarchy_product_cat_str"],
                        operator: Like,
                        valueText: "%s"
                }]
              }) {
              wpsolr_pid
              _additional {
                distance
              }
            }
          }
        }',
        $uuid,
	$cat_name,
        );

        // Define the request parameters
        $args = array(
            'method' => 'POST',
            'headers' => array(
                'Content-Type' => 'application/json',
                'Accept' => 'application/json',
            ),
            'body' => json_encode(array(
                'query' => $query,
            )),
            'timeout' => 30,
        );

        // Make the request
        $response = wp_remote_post( $weaviate_url, $args );

        // Check for errors
        if ( is_wp_error( $response ) ) {
            //$error_message = $response->get_error_message();
            //echo "Something went wrong: $error_message";

	    return false;
        } else {
            // Decode the response body
            $body = json_decode($response['body'], true);
        }

	$related_ids = array();

	if ( ! $body['data']['Get']['Mc_index'] ) {
		return false;
	}

	foreach ( $body['data']['Get']['Mc_index'] as $doc ) {
		$related_ids[] = $doc['wpsolr_pid'];
	}

	if ( $related_ids && count( $related_ids ) >= 9 ) {
		$related_ids = array_slice( $related_ids, 1, 21 ); // Remove first result keep 20 at most

		$result = update_post_meta( $pid, '_weaviate_related_products', $related_ids );
		return $result;
	}

	return false;
}
add_action( 'woocommerce_update_product', 'get_weaviate_related_products', 10, 1 );

— Query all products once per day:

function get_weaviate_related_all( $force = true ) {
        $args = array(
                'limit'  => -1,
                'return' => 'ids',
                'status' => 'publish',
        );

	$product_ids = wc_get_products( $args );

        foreach ( $product_ids as $product_id ) {
                if ( ! $force && get_post_meta( $product_id, '_weaviate_related_products', true ) ) {
                        continue;
                }

                $result = get_weaviate_related_products( $product_id );
        }
}

— Add filters to woocommerce to display the related products:

function mc_related_products_args( $args ) {
        global $product;
        $args['posts_per_page'] = 8;

        return $args;
}
add_filter( 'woocommerce_output_related_products_args', 'mc_related_products_args', 20, 1 );

function mc_weaviate_related_products( $related_posts, $product_id, $args ) {
      $weaviate_related = get_post_meta( $product_id, '_weaviate_related_products', true );
      if ( $weaviate_related ) {
             return $weaviate_related;
      }

      return $related_posts;
}
add_filter( 'woocommerce_related_products', 'mc_weaviate_related_products', 99, 3 );

// Dont suffle related products
add_filter( 'woocommerce_product_related_posts_shuffle', '__return_false' );

// Remove out of stock products from related products
function mc_product_related_posts_query( $query, $product_id, $args ){
    global $wpdb;

    $query['join']  .= " INNER JOIN {$wpdb->postmeta} as pm ON p.ID = pm.post_id ";
    $query['where'] .= " AND pm.meta_key = '_stock_status' AND meta_value != 'outofstock' ";

    return $query;
}
add_filter( 'woocommerce_product_related_posts_query', 'mc_product_related_posts_query', 10, 3 );

Do you think its possible to add a similar feature to wp-solr?

[wpsolr]

8 months, 3 weeks ago #35358

Nice! 👌

Can I ask why you do not prefer real-time retrieval of similar products?
Pre-calculating related products over all products every night seems a bit extrem.

[sannin]

8 months, 3 weeks ago #35363

Do you mean retrieve similar products on every page view? I am concerned with performance and caching issues.

I could maybe save the result in a transient for a hour, but i prefer to do it overnight. I have a cron at 3am, and it doesn’t take more than an hour for 250k products.

[wpsolr]

8 months, 3 weeks ago #35364

I’m reluctant to add that kind of recommendations without the help of a real external recommender system.

This is why:

– Recommenders provide complex filters and boosts.
Check out https://docs.recombee.com/reql_filtering_and_boosting.html.
In your example, you use a category filter, but it should be possible to build something more complex.

– Recommenders constantly train their models on your data and user events to prevent drift and anticipate trends.
Similarity matching with vectors is already a plus, but certainly not a state of the art.
After all, the purpose of recommendations is to increase conversions (clicks, views, add to basket, orders …). And conversions are based on user events, not similarity (at least not only similarity).

– Recommenders provide analytics
We need reports to verify recommendations are effective. And even better, a way to implement A/B testing.

How good is your vector similarity recommendation engine built on Weaviate?

[sannin]

8 months, 3 weeks ago #35366

I consider this solution -as the title says- just a better related products, that displays true related products compared to the woocommerce implementation. The cosine distance of the vectors, combined with a basic filtering should be enough for that task.

Of course it can be much better when for example combined with metarank, to rerank based on ctr etc, but as it is now it has good performance and it is quite easy to implement imho.

[wpsolr]

8 months, 3 weeks ago #35367

This is something I had considered with Weaviate and Vespa, indeed: using content-only vector search for recommendations.

The Weaviate centroid can even go a bit further with user events.

But I’ve not decided to implement any kind of recommendations yet. Let see if others show an interest.