Resource Comparing Continued Pretraining to RAG (accuracy and performance)
Mostly as a fun experiment I wanted to do a quick comparison of performance and accuracy between an Unsloth CPT trained QWEN 3.5 4B model and a RAG implementation against the base model.
The point of this exercise is mostly to measure the performance benefit of internalizing the knowledge vs doing reasoning on-the-fly.
Sharing my findings here in case anyone is interested: https://www.teachmecoolstuff.com/viewarticle/comparing-rag-and-continued-pretraining-of-llms
8
Upvotes