How to run a 120B model on a 64 GB PC in 5 min and protect billable hours
Every proposal you write from scratch is an hour you're not billing. A single on-premise workstation running gpt-oss-120b can change that math permanently. No SaaS subscription. No per-token bill. No client data leaving your building. By the end of this guide you'll know exactly what hardware you need, how to get the model running in LM Studio or Ollama in about five minutes, and you'll walk away with three ready-to-run prompt templates that produce a bid letter, a business proposal, and a contract draft on demand.
Who made it and why
model: gpt-oss-120b · maker: OpenAI · license: Apache-2.0
OpenAI published this model as a fully open release under the Apache-2.0 license. That matters more than people realize. Apache-2.0 means you can run it on your own hardware, use it commercially, and never pay a per-token fee. No usage clause, no attribution requirement that blocks commercial work, no "check back with the API vendor" when pricing changes.
The reason OpenAI released a 120B open-weight model is straightforward: enterprises want a high-quality long-form generation path that never touches an external API. Regulated industries, law firms, contractors working on government bids, anyone with NDAs baked into their client contracts. The model gives them that path.
What that means for you: the first time you generate a 2,500-word proposal on your own box, the cost is $0. The hundredth time, still $0.
The difference between a vague request and a usable document comes down to the prompt. Compare:
Before:write a proposal
After:
You are a senior consultant drafting a 2,500-word proposal for a municipal water-treatment RFP. Use the following structure: Executive Summary → Scope of Work → Timeline → Pricing → Risks & Mitigations Keep each section under 400 tokens.
The second prompt gets a structured, client-ready document. The first gets something you'll spend 40 minutes fixing. That gap is entirely about prompt structure, not model capability.
What it is good at, and what it is not
This is a 117B total parameter mixture-of-experts model with 5.1B active parameters per forward pass. In plain terms: it thinks like a much larger model, but only activates the relevant part of its brain for each token. That's why it fits in less memory than a 120B dense model would require, and why it holds together across long documents without the coherence drift you see in smaller models.
Where it earns its keep:
- 128k token context window. A full RFP package, your company boilerplate, and a draft all fit in one session.
- Consistent structure across multi-section documents. Scope of Work doesn't bleed into Pricing. Termination clauses don't contradict Payment Terms.
- Plain-English legal style when you ask for it. Good enough for a first draft that goes to your attorney, not a paralegal starting from scratch.
Where it falls short:
- Short, vague prompts produce verbose, wandering output. The model has a lot to say; you have to give it somewhere to go.
- Single-stream speed on consumer hardware is not measured on our bench. On the DGX Spark, single-stream runs at roughly 50 tok/s. Budget accordingly.
- It is not a lawyer. Drafts need review before signing. The model gets the structure right; a human confirms the terms.
A prompt that works:
Generate a 3-page contract for a freelance graphic-design retainer. Include: Parties, Services, Payment Terms, Confidentiality, and Termination clauses. Use plain-English legal style.
A prompt that doesn't: write a contract. The model will produce something, but it will wander. Feed it an outline and it stays on target every time.
If you're used to writing bids for trade work, the same discipline applies here. The post on writing roofing bids that win on paper covers the structure problem from the other side: what belongs in a bid before AI ever touches it.
Will it run on mine
The short answer: yes, if you have 64 GB or more of GPU memory. Here's the exact picture.
In its MXFP4 format as shipped, the model weighs 60.8 GB. The model card at huggingface.co/openai/gpt-oss-120b lists it as running on a single 80 GB GPU. That's the DGX Spark's target configuration.
DGX Spark (128 GB unified memory): Fits with headroom. On our bench (a DGX Spark), the model runs at roughly 125 tok/s aggregate batched and roughly 50 tok/s single-stream. That's the only measured number we'll quote here. Everything else for consumer machines is not measured on our bench.
True 64 GB machine (e.g., Mac Studio M3 Ultra with 64 GB, or a workstation with a single 64 GB GPU): The model loads with device_map="balanced", offloading layers between VRAM and system RAM as needed. Speed is not measured on our bench for these configurations, but the model does run. Expect meaningful slowdown versus the Spark, which affects daily throughput.
A quick check before you download 60+ GB:
python -c "from transformers import AutoModelForCausalLM; \
model = AutoModelForCausalLM.from_pretrained('openai/gpt-oss-120b', \
device_map='auto'); print('Loaded on devices:', model.device)"
If it loads without OOM errors, you're good. If it throws a memory error, try device_map="balanced" and let it spill to CPU RAM. Slower, but functional for low-volume use.
One honest note: if your GPU sits below 64 GB and you're hoping software tricks will close the gap, they mostly won't. You can quantize further, but you'll lose document quality on long-form outputs, which is exactly what you're running this model for. A 64 GB minimum is the practical floor for proposal-quality work.
Set it up in LM Studio and Ollama
You don't need Docker, don't need a custom Python environment, and don't need to read a research paper. Two paths, both fast.
LM Studio
Open LM Studio and search gpt-oss-120b. Click Install. Before you load the model, set max_memory to 60 GB and enable torch_dtype=bfloat16 in the model settings panel. That keeps you inside the MXFP4 footprint without blowing past your VRAM ceiling.
Once it loads, paste this into the system prompt field to test:
[System] You are a professional bid writer. [User] Draft a $25,000 bid for a local HVAC maintenance contract, following the outline below.
If you get a structured response in under two minutes, you're running. If you want to serve the model to other machines on your network, that's a two-minute config change covered in the guide on running LM Studio on your LAN.
Ollama
The Ollama tag is gpt-oss:120b. Two commands:
ollama pull gpt-oss:120b ollama serve
If your GPU has less than 80 GB VRAM, add the --cpu flag to force CPU offloading. It's slower, but the model runs. On the Spark, skip the flag and let Ollama use the full unified memory pool.
Both tools get you generating in under five minutes from a cold start. No custom Docker container, no dependency hell. That's the point.
Three things to get done this week
These are not generic examples. Each uses an exact Ultra Prompt template by name. Paste the template call and the variables into your local model; it fills the rest.
1. A personal bid (Project Proposal / Bid Letter)
Use this when you're quoting a job and want a clean, professional letter out in two minutes instead of thirty.
<template: Project Proposal / Bid Letter> Client: Greenfield Construction Scope: Electrical wiring for new office building (5,000 sq ft) Deadline: 30 days Budget cap: $45k
2. A business proposal (Business Proposal Prompt)
Use this when you're pitching a service engagement and need an executive-level document, not a one-pager.
<template: Business Proposal Prompt> Company: Apex Marketing Service: Full-funnel digital ad campaign Target KPI: 20% lift in qualified leads within 90 days
3. Something you won't send to the cloud (Contract Draft Prompt)
Contractor agreements, NDA-covered work, anything with payment terms tied to a real relationship. This stays on your box. Period.
<template: Contract Draft Prompt> Parties: Freelance Developer (Alice) & Startup XYZ Deliverable: Custom CRM integration Payment: 40% upfront, 60% on acceptance Confidentiality clause required.
Run each template once this week. On a Spark, you'll have a complete draft in under two minutes. On a 64 GB workstation, budget three to four minutes. Either way, you're not writing from scratch.
The same principle applies once you're past the draft stage. When a client pushes back on price, you need a response that's honest and doesn't lose the job. The post on what to say when the final bill is higher than the estimate covers that conversation with the same prompt-first approach.
Run it now
The recipe is free. Each link opens the finished prompt with the blanks already named, so you fill in your job and paste the result into the AI you use.
More like this, for whatever you are working on: open Ultra Prompt.