News guides for WordPress & WooCommerce

Announcing Hugging Face Endpoints API integration to WPSOLR’s SeMI Technologies Weaviate search add-on for WordPress and WooCommerce . Let the Hugging Face Endpoints https://huggingface.co/inference-endpoints API do the GPUs’ intensive work for your data indexing. Keep your lightweight CPU-base Weaviate search, self-hosted or hosted at https://weaviate.io/pricing.html More informations on Weaviate, Hugging Face Enpoints, WordPress and WooCommerce: / #wordpress #woocommerce #search #weaviate #wpsolr #huggingface #gpu #ai #ml #nlp #enterprisesearch

Based on the current demo https://demo-woocommerce-flatsome-cloudways.wpsolr.com/shop/: – WooCommerce – Flatsome theme – Cloudways hosting – 2000 realistic products But instead of Elasticsearch, we’ll use: – Weaviate hosted by SeMI Technologies (sandbox tier https://weaviate.io/developers/weaviate/current/getting-started/installation.html#weaviate-cloud-service) – Vectorizer module for OpenAI embeddings API: https://beta.openai.com/docs/guides/embeddings/what-are-embeddings – Vectorizer module for Hugging Face Endpoints API: https://huggingface.co/inference-endpoints Features presented: – WooCommerce shop and product search powered with vector search, including: facets, filters, sort, pagination – Questions answering in search bar suggestions More: / #search #woocommerce #demo #weaviate #openAI #embeddings #hugginface #endpoints #ml

The current state of the art hybrid search is a two or three phases sequential approach: efficient candidates retrieval (BM25), followed by a vector reranking. But here, phases are not consecutive, but instead combined with a proprietary hybrid query. The explanation: https://www.pinecone.io/learn/hybrid-search-intro/ #search #searchengines #ml #neuralnetwork

WordPress & WooCommerce owners who wants to self-host Weaviate without costly/challenging GPU(s) can outsource the vectorization of their content to the on-demand https://huggingface.co/ inference API. The search itself can be powered by Weaviate on cost effective CPU(s).   #huggingface #weaviate #ml #search #searchengines #wpsolr #woocommerce #wordpress

Following recent experience with a well-known recommendations API, and another one for vector embeddings, I’m beginning to doubt this. Even on a small scale, prices are rising very sharply. But does the effect on the business follow? The promise of a marginal increase in the conversion rate of a few percent is certainly attractive for a multinational, but not necessarily for a small player. #ai

WPSOLR will integrate Algolia Recommend https://www.algolia.com/products/recommendations/ as its first recommendation brick.   #algolia, being already part of the #searchengines supported by WPSOLR, is the most logical decision to start our next journey with #recommendersystems.   Also, we add several feature requests from existing customers/forums/topic/algolia-insights-tracking/   The first step is to send user events, which will not only allow adding content and collaborative recommendations, but also enrich the whole #ai stack: – Personalization https://www.algolia.com/products/search-and-discovery/personalization/ – Personalized query suggestions https://www.algolia.com/products/search-and-discovery/search-autocomplete/ – Analytics https://www.algolia.com/products/search-and-discovery/analytics/ – Dynamic re-ranking https://www.algolia.com/products/ai-search/dynamic-reranking/ – Dynamic synonyms suggestions https://www.algolia.com/products/ai-search/dynamic-synonym-suggestions/    

Thanks to SeMI Technologies #weaviate dedicated module, both indexed documents and queries are transparently vectorized calling the OpenAI‘s embeddings API. Results “close” to the query are then retrieved from the #weaviate vector database with the #graphql operators “nearText” or “ask”. For more details: – WPSOLR 22.4: /forums/topic/release-22-4/ – WPSOLR & Weaviate: /feature-weaviate/ – OpenAI embeddings API: https://beta.openai.com/docs/guides/embeddings – Weaviate OpenAI module: https://weaviate.io/developers/weaviate/current/retriever-vectorizer-modules/text2vec-openai.html#introduction

This is an exciting moment, where #vectorsearch is getting to the rich #wordpress and #woocommerce community: site and shop owners, agencies, or even specialised hosting companies like Kinsta® Cloudways WP Engine Pantheon Platform All current modules of #weaviate are implemented within WPSOLR (CLIP, text transformers, Question answering), while keeping the best of inverted search such as filters, facets, sort, pagination, suggestions. With the compatibility of #weaviate with Hugging Face models, and OpenAI embeddings API, sky is the limit to what can be implemented. If you are still reluctant, you could just add some #weaviate features to your current WordPress / WooCommerce implementation. For instance Questions answering in an autocomplete search box. Please follow /feature-weaviate/ to learn more about the integration.

Your dream comes true, as #WPSOLR 22.5 will integrate the Google Retail API ! We always wanted to integrate Google search, but until now the right conditions were not spot on. But with the recent announcement of the Google #retail API fusioning with the Google Recommendation AI API, everything seems to click perfectly https://cloud.google.com/solutions/retail-product-discovery Just look at the features, under a single integrated API: – Power your WooCommerce with Google-quality #search, including filtering, faceting and boosting – #personalized #search and #recommendations, based on #machinelearning models – #multimodal #search for #products with an image thanks to Google Vision AI – Relevant #recommendations at scale with Recommandation AI at any scale Feel free to describe your requirements in comments, or create a new feature request at /forums/forum/technical-configuration-issues/new-features-requests/

Feel free to describe your requirements in comments, or create a new feature request at /forums/forum/technical-configuration-issues/new-features-requests/ For exemple: – Do you prefer to build your custom UI, or integrate to recommendation plugins or page builder like #elementor related posts widget ? – Which #recommendations plugins are you already using ? – Which filters to narrow down recommendations ? – With SeMI Technologies #weaviate vector similarity (and optional OpenAI ‘s embeddings) ? – With Algolia Recommendations ? – With #solr, Elastic #elasticsearch, OpenSearch Project “More like this” ? – With external APIs, like Amazon Web Services (AWS) Personalize or Google Recommendations AI ?

WPSOLR 22.4 will introduce a brand new feature: content #recommendations (also named “More like this”, “Similar items”, …) This feature was asked for a long time, but is now almost trivial with the help of SeMI Technologies #weaviate near* filters. Several content recommendation scenarios will be implemented, starting with the following (behaviour recommendations are out of scope at the moment): – Supplement symbolic text search (Weaviate’s “where” filter or Elastic/OpenSearch Project/#solr/Algolia search) with semantic text search recommendations (nearText or ask results from the same query to show related texts/images/Answers) – Show Vector space’s closest neighbours of a specific document (nearVector or nearObject search on a text/image document’s stored vector/id). For instance on the detail page of a WooCommerce product. And with the help of “Views”

WPSOLR suggestions have been integrated with SeMI Technologies  Weaviate qna-transformer reader.

Would it be nice to retrieve your puppies and kittens images among tens of thousands of untagged images from the WordPress media library search bar? This would be a game changer for your back-office team, wasting hours every single day browsing and scrolling for images without titles or descriptions or even tags… And without the tedious task of tagging them all? And in 50+ languages! Yes indeed, this is now possible ! As promised, here is the first WPSOLR implementation of SeMI Technologies Weaviate Hugging Face multi2vec-clip vector image search inside the WordPress media library. In the video, you can watch the upload of 4 images in the media library. Each image is indexed in real-time in a Weaviate multi2vec-clip docker instance. After that, just some fun playing with

We’ve built a new Flatsome WooCommerce demo with 100,000 products. Suggestions from WPSOLR are displayed in 250ms. You can try the live demo here: (demo removed) A short video showing suggestions:  

When you install WordPress, you’ll get an almost perfect page speed score. But suddenly your score gets lower and lower. Why? Well, probably because of the tens of plugins you installed. And you cannot blame one in particular: each plugin adds a few hundred of ms to your score. Solution: deactivate as many as possible with specialized plugins like WP Asset Cleanup

Vespa.ai is under consideration as the second semantic search engine API integrated with WPSOLR, after Weaviate. Vespa is a big data, production-ready, hybrid semantic and symbolic search solution, with many advanced ranking functions (BM25, ML models, …).

In an earlier post, I mentioned how quickly vector search was innovating. Here is an exemple, with Weaviate’s new module to manage vectorization of both documents and queries from OpenAI’s new Embedding API. I know some WordPress owners will be exited to get a GPT-3-powered search!

The great thing about Vector search is its almost infinite range: every day new ML models come out with amazing new capabilities. Semantic text search, questioning and answering, image similarity, CLIP image search, multi modal search, code search. It never stops. Just watch Weaviate and Vespa exponential innovations. Amazing.

With the question “what color are the leashes ?“, Weaviate Q&A module suggests all colors for all products in our WooCommerce. The magic is that the word “color” does not appear anywhere ! The Q&A model is able to infer that “pink” is a color.   Even more crazy, the question “what accessory for canines ?” returns the answer “a pink leash“. Which means that the model matched “dog” to “canines” and “” leash” to “accessory”. Just remember that there are not synonyms set on this index!

14 million monthly visits with a mobile PageSpeed score of 30 ! Who said Core Web vitals were fundamental in 2022 ? I don’t know for you, but the content remains #1 for me… What do you think?

WooCommerce search requires so many crafts! (recommendation, personnalisation, trending, multi-modal, BERT NLP …) Does it mean non-opinionated OSS e-commerce search solutions are reserved to big companies? (It could explain the rise of SaaS like @recombee @vue_ai @KlevuAI)

While Data Lakes rose on cheap limitless scalable block storage (S3, GCP, Azure), the Lucene search ecosystem is still working on traditional file storage No more, with the arrival of new players: https://quickwit.io/ and https://www.chaossearch.io/

I always thought it was reserved to elite users of  Google Analytics 360 But in fact, you can synchronize “normal” Google Analytics 4 properties to BigQuery, by batch and streaming! You can now use your analytics data with BigQuery ML, or send it to elsewhere (Amazon S3, pub/sub …).

After looking for a way to re-rank WPSOLR search results based on users interaction history, we are currently thinking of Amazon Personalize Ranking solution. It could magically create a ML recommendation layer for all our engines: Elasticsearch, OpenSearch, Solr, Algolia, Weaviate.

I liked this nice presentation of Conversion Rate Optimisation (CRO) from Vaimo. It shows that both Klevu and GA are required to track the customers’ journey. Klevu adds specific events tracking, while GA relates those events to the full history of the customer, from first visit to final checkout.

Blown away by the quality of Dataiku Academy 101! 100x better than tutorials, with a systematic presentation of both concepts and their screens. We all should take inspiration from that. https://academy.dataiku.com/basics-101

I stumbled upon this Google blog post about Vertex Matching Engine. It is the current planet-scale Vector search technology used  at Google search, Youtube recommendations, Google play store… https://cloud.google.com/blog/products/ai-machine-learning/vertex-matching-engine-blazing-fast-and-massively-scalable-nearest-neighbor-search

Download WPSOLR Install WPSOLR on a Kinsta internal’s *.kinsta.cloud url Choose a hosting for Solr, Elasticsearch, OpenSearch, Weaviate or Algolia That’s it. No license, no registration, no payment method required

The OpenSearch indices are now managed with the official OpenSearch PHP client https://github.com/opensearch-project/opensearch-php. All tests, including Apache Tika ingestion, are green. Will be released with WPSOLR 22.3. Trello roadmap:  https://trello.com/c/4RlQpdHA/184-use-official-opensearch-php-client  

There are many excellent search APIs to start your integration: symbolic (Elasticsearch, Solr, Algolia), semantic (Weaviate, Pinecone, Milvus, Vespa). But in all projects, there is some heavy lifting: create and setup indices, cleanse and send data, generate queries, display results… This is where plugins like WPSOLR are really useful, they manage for you the submerged part of the iceberg.

“You see, in this world, there is two kinds of search engines, my friend: those with both ANN and filters and those with either”

Software eats the world, ML eats software …. but is Vector database eating ML now?

Train a huge ML model Convert your data to vectors Store your vectors Convert your inference input to vector Return top-n results with an ANN algorithm 2-3-4-5 can be done easily with a vector search database. And you get inference on your data with a general purpose algorithm.

Download WPSOLR Install WPSOLR on a WPEngine internal’s *.wpengine.com url Choose a hosting for Solr, Elasticsearch, OpenSearch, Weaviate or Algolia That’s it. No license, no registration, no payment method required

The famous mantra “machine learning is 5% about models and 95% about data” is true with vector databases. Before querying your vector space, you need to get your data vectorised! You can see that clearly in all Jupyter notebooks demos with Tensor libraries:  the search itself is the last line of tens and tens of lines of python code. This is why WPSOLR is integrating Weaviate: it sets up the schema, ingest data into a vectoriser of your choice, and generates queries automatically worth hundred lines of GraphQL.

Search is dominated today by Elasticsearch and Solr, thanks to Lucene’s inverted index incredible speed and versatility. We all want to keep those nice search features (speed, filters and facets) but also add a touch of intelligence, or semantic. How? By using a vector database compatible with an inverted index! The following search engines provide traditional search and semantic search: Weaviate (Open source) Vespa (Open source) Algolia (Commercial)

Weaviate currently being integrated to WooCommerce, we are looking for real uses cases. Do you have a WooCommerce, and seeking a new solution using machine learning out of the obvious Q&A, NER ? Please let me know here, so we can investigate.

Just added this task of  using the OpenSearch PHP client on WPSOLR release 22.3 to the WPSOLR’s roadmap https://trello.com/c/4RlQpdHA/184-use-official-opensearch-php-client Yeah, opensearch-php is young, but it is official. Also, FYI, There is a PR right now to add it to the docs (active as of ~13 hrs ago). https://t.co/XFEZhNc1Dr — Kyle (@stockholmux) December 6, 2021

Moving from Jenkins to Github action brought me a great deal of stability. But as a developer I miss the Jenkins Groovy scripting language. It is so more powerful than static configuration files. A bit like Maven vs Gradle.

It is never too soon to preview new game-changing features. The new WPSOLR’s home page is now reflecting that Vector search engines have gained enough momentum to be integrated to WordPress and WooCommerce.

As we can see, the most difficult Trello cards are already done: The PHP framework is built and tested (managing schema simple types and array types, adding/deleting a Weaviate index, indexing data, searching, showing facets, filtering results) Weaviate GraphQL and REST APIs are integrated PHPUnit and Selenium tests for Github actions are ready Missing  features: Sorting Pagination Aggregation queries with statistics on numerical values Generating a custom Weaviate docker compose file Connecting to Weaviate Cloud Service with OpenIDs … (to be discovered soon)

Here is the Trello public roadmap for release 22.3 with Weaviate: https://trello.com/c/ykWgUCbh/171-add-weaviate-as-a-semantic-nlp-search-engine

The GraphQL’s console prettifier is rendering long text on single lines, which makes them unreadable.   It is far better readable with https://jsonformatter.org/graphql-formatter

42 hours sequential tests run now in parallel in 2 hours with no infrastructure errors. Before that, tests were launched on Jenkins + Kubernetes plugin, but 20-50% failed on some kind of infrastructure mismanagement.

80% done already. Bringing all Hugging Face Transformers models to WP