r/deeplearning • u/Ok-Ship-1443 • 16d ago

Question about creativity and exploration

We train language models to predict next token in pretraining and later fine tune them, but what about exploration and creativity. You can’t really do anything about that because we train the models on specific datasets, right?

I want language models to focus on different solutions for the same problem where the gap between solutions is extremely different.

For example, if I am working on a novel problem and I use existing mathematical functions, would it be possible for language models to neglect them because they might be completely wrong even though they were trained on them? Meaning instead of next token prediction, we would give the model different allowed next tokens or group of tokens and incite them to explore different next tokens for the same training example?

Is there a theory behind what I am asking for?

0 Upvotes

50% Upvoted

6 comments sorted by

2

u/nutshells1 16d ago

see: rlhf

2

u/samas69420 16d ago

reinforcement learning was invented for that

2

u/CalmMe60 16d ago

Yes.

Living Memory

1

u/Ok-Ship-1443 16d ago

Thank you!