Nvidia, the tech giant that’s been making your gaming experience smoother than a greased weasel on a water slide, has released a new tool that lets you run GenAI models on a PC. It’s called Chat with RTX, and it’s like having your very own AI-powered chatbot that can sift through your documents, files, and notes faster than a caffeinated squirrel on a nut hunt.
The tool is designed to work with GeForce RTX 30 Series and 40 Series cards and operates offline on a Windows PC. It’s like having a personal assistant who doesn’t need coffee breaks or sleep. You can ask it questions like, ‘What was the restaurant my partner recommended while in Las Vegas?’ and it will scan local files and provide the answer with context. It’s like having your very own Sherlock Holmes, minus the deerstalker hat and the violin.
Now, before you get too excited, there are a few caveats. For one, Chat with RTX can’t remember context. So, if you ask it about a common bird in North America and then ask about its colors, it won’t know you’re still talking about birds. It’s like having a conversation with a goldfish.
Also, the relevance of the app’s responses can be affected by a range of factors, including the question phrasing, the performance of the selected model, and the size of the fine-tuning dataset. It’s a bit like asking a toddler to explain quantum physics; the results may vary.
Despite these limitations, there’s something to be said for apps that make it easier to run AI models locally. The World Economic Forum predicts a ‘dramatic’ growth in affordable devices that can run GenAI models offline, including PCs, smartphones, and Internet of Things devices. The benefits are clear: offline models are more private, lower latency, and more cost-effective than cloud-hosted models.
So, while Chat with RTX might be more of a toy than a tool for production, it’s a step in the right direction. It’s a glimpse into a future where AI is as commonplace as smartphones and coffee machines. And who knows? Maybe one day, we’ll all have our very own AI-powered chatbots to help us remember where we left our keys.