Oh and I typically get 16-20 tok/s running a 32b model on Ollama using Open WebUI. Also I have experienced issues with 4-bit quantization for the K/V cache on some models myself so just FYI
Oh and I typically get 16-20 tok/s running a 32b model on Ollama using Open WebUI. Also I have experienced issues with 4-bit quantization for the K/V cache on some models myself so just FYI
It really depends on how you quantize the model and the K/V cache as well. This is a useful calculator. https://smcleod.net/vram-estimator/ I can comfortably fit most 32b models quantized to 4-bit (usually KVM or IQ4XS) on my 3090’s 24 GB of VRAM with a reasonable context size. If you’re going to be needing a much larger context window to input large documents etc then you’d need to go smaller with the model size (14b, 27b etc) or get a multi GPU set up or something with unified memory and a lot of ram (like the Mac Minis others are mentioning).
It would be more interesting to see this with a cost of living figure for each state as well.
Is this what the Leonard Cohen song is about?
Anyone have more context on this? These are some pretty massive increases if the games in the table are in any way representative of all games.
Been having fun and happy to see a gameplay balance patch so soon. That said, the technical side of this game is what really needs work and according to everything I’ve been seeing from Digital Foundry and others there’s some serious low hanging fruit that could improve the frame rate and pacing that is still pretty poor on all systems. Hopefully they bring some attention to that side of things soon. Game is certainly playable in the current state in my opinion but would be much more enjoyable if it actually stuck to something close to 60 FPS in most situations on XSX/PS5.
Maybe 1-3 times a day. I find that the newest version of ChatGPT (4o) typically returns answers that are faster and better quality than a search engine inquiry, especially for inquiries that have a bit more conceptualization required or are more bespoke (i.e give me recipes to use up these 3 ingredients etc) so it has replaced search engines for me in those cases.
Looks like it now has Docling Content Extraction Support for RAG. Has anyone used Docling much?