NVIDIA's new open models put AI agents on your own hardware
NVIDIA has released Nemotron 3.5, a new family of open AI models built for business agents. One is built to run on a single desktop GPU, and another on edge devices. For Australian businesses that want AI agents without sending their data offshore, that changes what is practical.
What happened
In mid-October, NVIDIA released two new Nemotron 3.5 models, Super VL and Bolt 74B-A9B, and said a third, Bolt E4B, would follow on 23 October. All three are open-weight models. You can download them, run them on your own hardware and retrain them on your own data. NVIDIA has also published the datasets and training recipes behind them.
They are aimed at "always-on" agents: AI that works through specialised business tasks, such as reading documents, following instructions, using tools and completing multi-step workflows.
What's involved
- Nemotron 3.5 Super VL. The largest of the three, with 120 billion parameters, of which about 12 billion are active at a time. It reads text, images, PDFs, tables, charts and web pages. NVIDIA lists it for DGX Spark, DGX Station, RTX PRO workstations and data centre systems, and early users are customising it for work such as financial document analysis and medical claims.
- Nemotron 3.5 Bolt 74B-A9B. A 74-billion-parameter model compressed to about 24GB, so it fits on a single workstation GPU or a DGX Spark. It works with common local tools including Ollama, LM Studio, llama.cpp and vLLM.
- Nemotron 3.5 Bolt E4B. A small model of about 4GB for voice assistants, conversational agents and robots, built to run on Jetson edge devices and RTX PCs.
Why it matters here
Until recently, a capable AI agent usually meant sending customer records, contracts or financials to a cloud provider overseas. Many Australian businesses have held back for exactly that reason.
Models like these narrow the gap. When the model runs on hardware you own, your data stays on your premises and you decide what the agent can reach. You also get a record of what it did. That matters more as Privacy Act changes and new national AI standards raise the bar for showing how AI is used.
There is a cost angle too. Renting cloud GPUs has become more expensive this year, and agents that run all day are exactly the steady workload where owning hardware tends to pay off.
Running AI in-house does not make a business compliant by itself. You still need clear policies, access controls and human review. What it gives you is control and visibility, which make good governance easier to put in place.
Where your hardware fits
- Trying Bolt on one desk. A DGX Spark has 128GB of memory, so Bolt runs with plenty of room for long documents. On a workstation, the RTX PRO 4500 (32GB) or RTX PRO 5000 (48GB) gives headroom above Bolt's 24GB. The RTX PRO 4000 (24GB) matches its size but leaves little room to spare.
- Running Super VL. DGX Spark is on NVIDIA's supported list. For more speed or several users, look at the RTX PRO 6000 (96GB, available by enquiry) or a DGX Station.
- Edge and voice agents with Bolt E4B. The Jetson Orin Nano Super Developer Kit is an affordable way to start, and the Jetson AGX Orin 64GB Developer Kit gives more room for vision and robotics.
- Keeping it running. Agents that run all day need reliable storage and power. Synology NAS and Vertiv UPS cover both.
Not sure what will run on what? Our guide Will it run? Models by GPU memory is a quick reference, and DGX Spark or a workstation GPU? helps you choose between the two.
Where to start
- Pick one task. Choose a job you would not hand to a cloud service today, such as summarising contracts, checking invoices or answering staff questions from internal documents.
- Trial Bolt locally. Run it on one machine with a tool such as Ollama or LM Studio and test it on real, non-sensitive examples first.
- Measure before you scale. Check accuracy, speed and how often a person has to step in. Then decide whether you need Super VL, more memory or more users.
- Set the rules early. Decide what data the agent can see, what it is allowed to do, and who reviews its work.
If you'd like help matching a model to the right hardware, book a free consultation.
General information only, not legal or compliance advice.
General guidance only. Figures are approximate and depend on your model, software and settings. Book a consultation before you buy.