In the past, websites aimed to boost click-through and conversion rates by improving search features. Now, a new technology called recommender systems has emerged. These systems work subtly in the background, requiring little user interaction. They personalize recommendations, closely linking with personalized search to tailor the user experience. But how do these recommendation systems work ? They use algorithms to filter the results that will be returned to the user. These algorithms are : content-based filtering, collaborative filtering, hybrid filtering and knowledge-based filtering.
Diagram explaining how collaborative filtering works
Collaborative filtering
We will start of with the most popular type of filtering algorithm today : collaborative filtering. Collaborative filtering makes use of machine learning to quickly filter (recommend) the most relevant items to users. It consists of analyzing large groups of users to find similarities between them and then once a trend is determined suggest similar content to the most similar users.
For example, a first user may have bought the books Dune and Foundation, which are both Science fiction. If a second user buys or views Dune, they could be interested in Science fiction books too like the first user. The recommender has then found similarities between them and will suggest Foundation as well to the second user since the first user bought it.
This is a rather simple example but it can become complex fast which then requires machine learning (or AI) :
For one, this example only takes into account two users’ actions. There could be thousands more that the recommender needs to take into account. This increases difficulty tenfold since the differences the recommender needs to determine become more and more subtle (otherwise the recommendations are not personalized enough).
The second reason why is that some similarities may not be as visible. Some items could be in two separate categories but the users that are interested and interact with both items overlap. It is important to remember that the objective of recommendations is not necessarily to get the most relevant item overall or to recommend the most similar item but to drive user engagement by recommending the content that the user is most likely to view.
This type of filtering algorithm has one major downside. Since the AI needs to learn about your users beforehand, it will need a lot of user events/interactions. This is what we call a cold-start. Similar to a car on a snowy day, the recommender will struggle to start without the required data to be trained on. This can be solved using hybrid filtering, which replaces collaborative filtering when not enough data is collected.
Diagram explaining how content-based filtering works
Some of the recommenders that use collaborative filtering are Algolia, Recombee and Google Retail.
Content-based filtering
Content-based filtering is older and simpler than collaborative filtering. It will instead base it’s decision-making on the data associated to the items or users.
There are three major types of content-based filtering :
- Based on user history : this type of recommendation will take into account the users’ personal history of browsing, adding to carts, purchases, etc… to suggest content or items with similar attributes. These attributes could be defined using data or metadata.
- Based on item properties : this type of recommendation will recommend items similar to the one the user is viewing.
- Trending items : This is a very simple form of recommendations where the items that are recommended are the ones that are the most viewed, bought, etc…
Hybrid filtering
Hybrid filtering uses a mix of algorithms to function properly.
It can be used in multiple ways :
- to have more accurate or diverse recommendations. This way a recommendation is not limited to a single purpose : it could recommend the most similar items to the one the user is currently viewing, as well as different items that could interest the user based on their browsing history.
- as a way to have a content-based filtering algorthm as a fallback when data/user events are too few for collaborative filtering. So to avoid cold-start.
Knowledge-based filtering
Knowledge-based recommenders, or rule-based recommenders, make use of explicit domain knowledge to generate recommendations. This type of recommendations is used whenever the recommendations need to be precise (for large purchases for example) and the amount of data is not sufficient for collaborative filtering.
