• 7 Posts
  • 9 Comments
Joined 5 years ago
cake
Cake day: June 30th, 2020

help-circle


  • It really depends on how you quantize the model and the K/V cache as well. This is a useful calculator. https://smcleod.net/vram-estimator/ I can comfortably fit most 32b models quantized to 4-bit (usually KVM or IQ4XS) on my 3090’s 24 GB of VRAM with a reasonable context size. If you’re going to be needing a much larger context window to input large documents etc then you’d need to go smaller with the model size (14b, 27b etc) or get a multi GPU set up or something with unified memory and a lot of ram (like the Mac Minis others are mentioning).










  • Been having fun and happy to see a gameplay balance patch so soon. That said, the technical side of this game is what really needs work and according to everything I’ve been seeing from Digital Foundry and others there’s some serious low hanging fruit that could improve the frame rate and pacing that is still pretty poor on all systems. Hopefully they bring some attention to that side of things soon. Game is certainly playable in the current state in my opinion but would be much more enjoyable if it actually stuck to something close to 60 FPS in most situations on XSX/PS5.