Cards
Eight Tesla V100-SXM2-32GB
Data-center cards from 2018. Two boards of four cards each, 256 GB of card memory in total. The first half of the model’s layers runs on one board and the second half on the other.
- Home
- Local models
Local models
An open model, running in your office.
For files you decide should not go to a cloud provider, the model can run on a machine you own. My server runs DeepSeek V4.1 Flash, an open-weight model, on eight used Nvidia V100 cards from 2018. Its speed is measured and the measurements are public.
Measured November 5, 2026
What eight cards from 2018 actually do.
A token is roughly three quarters of an English word. One request runs at a time.
215tok/swriting code, after a 1,000-token prompt
118tok/swriting legal text, after a 1,000-token prompt
59secondsto the first token of an answer about a 128,000-token file
524Ktokensof context accepted in one request
| Prompt length | First token | Code | Structured (JSON) | Legal | Prose |
|---|---|---|---|---|---|
| 1K tokens | 1.2 s | 215 | 203 | 118 | 89 |
| 32K tokens | 17.2 s | 200 | 207 | 117 | 90 |
| 128K tokens | 59.2 s | 183 | 197 | 111 | 91 |
| 256K tokens | 118.7 s | 160 | 180 | 103 | 84 |
Build: Volta sPLIt. One request at a time, thinking off, greedy decoding, 1,024 tokens out, mean of two runs. Decode speed in tokens per second. Every length from 1K to 256K, the thinking-on table and the method are in the repository.
In the press · Tom’s Hardware · November 9, 2026
Lawyer builds private AI server with eight used Nvidia V100 GPUs — up to 215 tokens per second to keep legal files offline, costing about $2,400 per four-GPU board
“Performance is surprisingly good for a model this size.”
By Shane Downing. The article walks through the hardware, the speeds and the problems along the way, including two host resets and a burnt power connector. It reports the builder’s own measurements, which are the ones in the table above.
What it does in a law practice
First-pass work on case files, reviewed before it leaves.
This is my own use, as I described it publicly and as Tom’s Hardware reported it.
- Reads a folder of records. It pulls out facts, dates and numbers and summarizes them.
- Takes a first pass at routine drafting. A lawyer reviews everything before it leaves the office.
- Searches and calls tools. Document search and tool calls are wired in.
- Carries about half the load. By my estimate, 50 to 60 percent of the model work in my practice can run on this server. The rest still goes to frontier models.
- Not yet self-contained. A setup where no client text leaves the office is the goal. It is not the present state, and I will not describe it as one.
What is in the box
Used data-center cards, two office circuits.
Model
DeepSeek V4.1 Flash
Open weights, compressed to mostly 2 bits per weight for this build. The parts of the model that do most of the work for each word sit entirely in card memory.
Power
150 watts per card
Each card is capped at half its rating. Two power supplies sit on two ordinary office circuits.
Cost, as reported
About $2,400 per four-card board
That is what the boards cost when they were bought, according to Tom’s Hardware. The same article says used cards now list for about $900 each and prices are rising.
Software
Published under Apache-2.0
The patches, launch settings, measurement scripts and results are in the public repositories, so the numbers on this page can be checked or rerun.
Interface
Chat, search and tools
It is driven through a chat interface, with document search and tool calls wired in. In a firm setup, the firm’s own rule decides which work goes to it and which goes to a frontier model.
Limits
What it does not do, stated plainly.
- It does not replace a frontier model. It is not faster than a cloud service, and about half of my model work still goes to one.
- It took work. As Tom’s Hardware reported, the host reset twice under load before the clock limits were set, and one board was pulled for a burnt power connector.
- The cards are old. Nvidia’s R580 driver branch is the last to support this generation, as Tom’s Hardware notes.
- One request at a time on the published build. Serving two or three at once is the next step.
- It does not settle your duties for you. Running a model on your own machine removes one outside party. Whether that satisfies your confidentiality and privilege obligations is a judgment for your lawyers.
Public builds
Download it, read it, run it.
- GitHub · the fast build
Volta sPLIt
DeepSeek V4.1 Flash on eight V100 cards, split across two boards. The numbers on this page come from it.
- Hugging Face · model card
Volta sPLIt on Hugging Face
The model card for the same build.
- GitHub · work in progress
Volta Max uPLIft FP4
The same model with most of its routed experts at the precision of the official weights. Slower today: 57 to 58 tokens per second on legal-style text and 51 to 52 on general text.
- PLI Labs
Pruned legal checkpoints and research notes
Open models cut down to the experts legal work uses, plus serving notes for other hardware.
What I do for a firm
Size it, build it, measure it on your documents.
- 01SizeWhich model fits which hardware, new or used, and what it costs to buy and to power.
- 02BuildThe server, the serving software and the settings, assembled and tested under load.
- 03MeasureSpeed and output quality on your own kinds of documents, with the method written down.
- 04Hand overLaunch scripts, a runbook and the measurement kit, so your staff can run it.
Next step
Tell me what must stay in the building.
Describe the kind of files and how many people would use the server. I will tell you what hardware that takes. Fees and how it starts.
nick@proprietarylegal.com