Search: AI, Hybrid, Classic

  • Pricing
  • Help
    • My Licenses
      • Login
      • Download
      • Why us
      • Contact us
    • Features
      • Google Retail Search
      • Weaviate
      • Elasticsearch
      • OpenSearch
      • Solr
      • Algolia
      • Roadmap
      • Artificial Intelligence
      • Hosting
      • Releases
    • Documentation
    • Support
    • Testimonials
    • Live Demos
    • Blog
Menu
  • Pricing
  • Help
    • My Licenses
      • Login
      • Download
      • Why us
      • Contact us
    • Features
      • Google Retail Search
      • Weaviate
      • Elasticsearch
      • OpenSearch
      • Solr
      • Algolia
      • Roadmap
      • Artificial Intelligence
      • Hosting
      • Releases
    • Documentation
    • Support
    • Testimonials
    • Live Demos
    • Blog
Search
Search
Search
Search

Home » Your configuration journey, step by step » Configure your indexes » Create a Weaviate index » HuggingFace

HuggingFace

960 views 0 December 22, 2022 Updated on December 23, 2022

1. HuggingFace vectorizer

HuggingFace vectorizer uses the HuggingFace API to vectorize data. Vectorizing from transformers models is CPU intensive, often requires GPUs, and can be very ineffective without proper optimisations. This is the purpose of this module, which calls your selected transformer model on HuggingFace servers with their inference endpoints API.

 

Create a docker compose file from Weaviate wizzard : select the Text transformers vectorizer, with a specific transformer model among the list. Then start the docker container.

Image wpsolr-weaviate-configurator-text2vec-huggingface-docker.png of HuggingFace

  • (1) Select the “Text” vectorizer type
  • (2) Select the “HuggingFace” vectorizer
  • (3) Leave it. The API key will be set later, in WPSOLR’s settings.

Download and execute the docker-compose file generated by the wizard (docker-compose up -d):
Image wpsolr-weaviate-configurator-text2vec-huggingface-docker-file.png of HuggingFace

Image wpsolr-weaviate-docker-autoschema-enabled-false-1.png of HuggingFace

 

Let’s setup our model on HuggingFace dashboard:

(Please check the HuggingFace endpoints pricing first)

a) Login: https://huggingface.co/login (signup first if necessary)

Image wpsolr-weaviate-huggingface-login.png of HuggingFace

b) Select the Inference Endpoints menu

Image wpsolr-weaviate-huggingface-endpoints-menu.png of HuggingFace

 

Image wpsolr-weaviate-huggingface-endpoints-menu-1.png of HuggingFace

c) Create a protected endpoint

Image wpsolr-weaviate-huggingface-endpoints-create.png of HuggingFace

  • (1) Select a sentence transformer model among Sentence Similarity models
  • (2) Select a hosting
  • (3) Select a datacenter
  • (4) Select a VM sizing (CPUs, GPUs …)
  • (5) Set advanced VM settings, like autoloading
  • (6) Use protected, or private if you own a VPC
  • (7) Create the endpoint
  • (8) Wait a few minutes for your Endpoint creation and initialisation

Image wpsolr-weaviate-huggingface-endpoints-created.png of HuggingFace

  • (1) Copy your Endpoint URL for later during your WPSOLR index configuration

 

Retrieve your HuggingFace API keys:

Image wpsolr-weaviate-huggingface-api-key-menu.png of HuggingFace

 

Image wpsolr-weaviate-huggingface-api-key-copy.png of HuggingFace

  • (2) Copy your HuggingFace API key for later during your WPSOLR index configuration

 

Now, let’s create our index in WPSOLR:

Image wpsolr-weaviate-huggingface-index-createpng.png of HuggingFace

  • (1) Select the text2vec-huggingface vectorizer module
  • (2) Copy your HuggingFace API access token described earlier
  • (3) Set a name for you index, visible in WPSOLR admin
  • (4) Set a name for your Weaviate class (index)
  • (5) Set the url of your Weaviate docker container
  • (6) Copy your endpoint URL
  • (5) Create the index. Done!

 

Connect to the Weaviate GraphQL console at https://console.semi.technology/console with url https://localhost:8080, to check our new index (class):

Image wpsolr-weaviate-huggingface-console-1.png of HuggingFace

2 Select your data

Image wpsolr-weaviate-new-index-text2vec-transormers-select.png of HuggingFace

  • (1) (2) (3) select the index you just created
  • (4) Choose a filter: “Near Text” to perform a vector search (search on concepts), or “Where” to perform a keywords search (classic search that works with words)
  • (5) Set a similarity for your “Near Text” search. The closer to “1”, the more precise is the vector search.

 

3 Index your data

Image wpsolr-weaviate-indexed.png of HuggingFace

 

Was this helpful?

Yes  No
Related Articles
  • Hybrid search
  • Question Answering
  • Cohere
  • OpenAI
  • CLIP
  • Transformers

Didn't find your answer? Contact Us

Previously
CLIP
Up Next
OpenAI
Expand All Collapse All
  • What is it WPSOLR ?
    • The standard WordPress SQL search
    • The WPSOLR search with Elasticsearch, Apache Solr, or Algolia
  • Your configuration journey, step by step
    • Install Apache Solr
    • Hosted Apache Solr and SolrCloud Services
    • Install Elasticsearch
    • Elasticsearch Hosting
    • Configure your indexes
      • Create a Google Retail search index
      • Create a Weaviate index
        • Transformers
        • CLIP
        • HuggingFace
        • OpenAI
        • Cohere
        • Question Answering
        • Hybrid search
      • Generate a test Apache Solr index
      • Generate a test Elasticsearch index
      • Create an Apache Solr index
      • Create a SearchStax SolrCloud index
      • Create an Elastic Elasticsearch index
      • Create an Amazon AWS OpenSearch index
      • Create an Algolia index
      • Create an Elasticsearch index
      • Create an Opensolr Solr index
      • Create a Qbox Elasticsearch index
      • Create an ElasticPress Elasticsearch index
      • Create an Aiven Elasticsearch or OpenSearch index
      • Create a Bonsai Elasticsearch index
      • Create an Amazon AWS Elasticsearch index
      • Create an ObjectRocket Elasticsearch index
      • Create a Cloudways Elasticsearch index
      • Create an Alibaba Cloud Elasticsearch index
      • Create a Compose Elasticsearch index
      • Connect to an Apache Solr index
      • Connect to an Elasticsearch index
    • Add-ons
      • bbPress add-on
      • YITH WooCommerce Ajax Search add-on
      • SEO add-ons
        • Yoast SEO add-on
        • All in One SEO add-on
      • Listable add-on
      • ACF add-on
      • Advanced Scoring add-on
      • Cron Scheduling add-on
      • Theme add-on
        • Filters layouts
          • Radiobox & Checkbox Layout
          • Numeric Range Layout
          • Colour Picker Layout
          • Range Slider Layout
        • Add Ajax to the current Theme
        • Collapse taxonomy hierarchies
        • Custom Facets CSS
      • WPML add-on
      • Premium add-on
        • Manage more than one Elasticsearch or Solr index
      • PDF Embedder add-on
      • Geolocation add-on
      • AI Natural Language APIs add-on
        • Amazon Comprehend
        • Google Natural Language
        • Aylien Text Analysis
        • MeaningCloud
        • Qwam Text Analytics
      • Toolset Types add-on
      • AI Image and OCR APIs add-on
        • Google Vision
        • Amazon Rekognition
      • Embed Any Document add-on
      • MyListing add-on
      • Polylang add-on
      • WooCommerce add-on
      • Cross-domain search add-on
      • Directory+ add-on
      • Toolset Views add-on
      • Listify add-on
      • Jobify add-on
      • Query Monitor add-on
  • Quick start
Login
Support
Privacy policy
  • Terms and Conditions
  • © 2020 wpsolr.com. All Rights Reserved.
Youtube
Roadmap