As a developer without real experience with ML, fine-tuning or distilling a SBERT model is a huge leap.
We’ve all heard about learn to rank for information retrieval’s tuning, but it requires lots of labeled data (hundreds to thousands).
(Which small to medium WooCommerce shop owner cannot provide, most probably)
Jo Kristian Bergum‘s tutorial is a breakthrough for me, because it also provides 3 notebooks to start playing with the following concepts on a WooCommerce shop:
1. Create questions (the labels), from a generative model (T5), and the WooCommerce product titles, descriptions, categories and attributes.
2. Classify questions/results as positive/negative, with a simple search on the actual vector database containing the already indexed WooCommerce products (embeddings already built with a SBERT bi-encoder model)
3. Train a large cross-encoder (“distillation”) from the positive/negative questions/results, to generate a much lighter tuned model.
4. Use the tuned model as a 2nd phase re-ranking, after the bi-encoder initial retrieval.
It looks great!
But is it really worth it, improvements vs pricing, on a small/medium WooCommerce shop search?
And can we automate the whole process, by batch, and by streaming?
(For instance during full or incremental indexing)
The future will tell us: https://wpsolr.com
#wpsolr #vectorsearch #generativeai #distillationai #vespasearch #weaviate #ml