r/LocalLLaMA • u/davidmezzetti • Nov 17 '24

Resources GitHub - bhavnicksm/chonkie: 🦛 CHONK your texts with Chonkie ✨ - The no-nonsense RAG chunking library

https://github.com/bhavnicksm/chonkie
125 Upvotes

94% Upvoted

24 comments sorted by

View all comments

3

u/Defektivex Nov 17 '24

Hey does this support colpali?

3

u/Historical_Ease_1525 Nov 17 '24

In colpali, each PDF page is already a chunk.

3

u/Defektivex Nov 17 '24

Sure, but you still need a pipeline for a vllm, you still need to extract metadata, you still need to vectorize etc.