-
Notifications
You must be signed in to change notification settings - Fork 35
Closed
Description
When running the StarCoder model quantized using "q5_1" with a medium-sized context (3500 tokens) I run into this error:
ggml_new_tensor_impl: not enough space in the context's memory pool (needed 412241472, available 411791504)
Segmentation fault (core dumped)
How can I increase the context memory pool size?
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels