WPSolr logo
Close this search box.

Table of contents :

What is the next step after finetuning?

DALL·E 2023-03-06 11.22.04 - funny cartoon of trying to fit a square in a round hole with a huge hammer

Table of contents :

After training LLMs on the whole world, prompting them for zero-shot, finetuning them on specialized domains, what is the next stage?

The goal is clear: adapting models further and further.

I can see two competitors there: website finetuning and recommenders.

– Recommenders –
Recommenders are basically finetuned on users. They already use embeddings, are effective, and do not need labeling thanks to user events. But privacy is a problem.

– Website finetuning –
On the other hand, Website finetuning is performed on the site content, especially vocabulary. But labeling is a problem. We can use LLMs to generate labels (questions), but is it not a chiken/eggs loop?

Trending posts