NVIDIA power limit for local LLMs: SW Power Cap guide

Your GPU can work hard without spending every watt it is allowed. On our RTX 3060 Ti, a 150 W cap kept 97% of the stock token speed. Here is the measured curve, the exact commands and the boot-time fix—then a wall-meter check to turn a GPU setting into a useful electricity number.

Measured on our bench; NVIDIA, systemd and P3 documentation support the setup instructions.

Key takeaway: At 150 W our 3060 Ti produced 75.5 tok/s against 77.8 tok/s at stock 200 W. That is 3% less speed for 25% less draw (bench notes).

Pick 150 W and you keep 97% of the speed

The first 50 W you take off the card cost almost nothing. Measured September 25, 2026: our method was qwen2.5:7b Q4_K_M on Ollama 0.34.2, with three 512-token runs per cap at seed 1 and temperature 0, a warm-up first, and driver telemetry sampled every 250 ms (bench notes). Speed is the mean of all three runs; draw averages samples above 40 W in runs two and three, and temperature is the peak. Caps ran lowest to highest back to back, so later steps started warmer. Compare the 3060 Ti and RTX 2060 results by GPU residency and offload before carrying numbers between configurations.

Cap	tok/s	Avg draw	tok/s per W	Core clock	Peak temp
100 W	58.8	99.6 W	0.59	1,092 MHz	50 C
125 W	72.3	122.5 W	0.59	1,516 MHz	53 C
150 W	75.5	145.4 W	0.52	1,675 MHz	57 C
175 W	76.9	172.0 W	0.45	1,789 MHz	61 C
200 W (stock)	77.8	193.4 W	0.40	1,855 MHz	66 C

The memory clock held at 6,801 MHz at every cap, and only the core clock moved. Our read: token generation on a 7B model is memory-bound, so an untouched memory clock is why the curve stays flat down to 125 W. An earlier run the same day at 150 W measured 76.3 tok/s on qwen2.5:7b and 73.9 tok/s on llama3.1:8b, so a second 8B-class model landed in the same range (bench notes).

To interpret those rates alongside an application’s wait time, use the Ollama API timing guide. It separates output-generation speed, loading, prompt work and client-observed response timing.

Drop to 125 W when a cooler card matters more than the last 7%

We consider 125 W the efficiency floor. It matched 100 W at 0.59 tok/s per watt and ran 13.5 tok/s faster. Against stock, it gave up 7% of speed for 37% less draw, and peak temperature fell from 66 C to 53 C (bench notes).

Going below 125 W is a bad trade on this card. At 100 W you lose 24% of speed for 49% less draw, and efficiency does not improve at all.

Example (illustrative): Say a batch job generates 100,000 tokens. At stock it takes about 21 minutes at 193 W. At 150 W it takes about 22 minutes at 145 W. That works out to roughly 0.069 kWh versus 0.053 kWh, calculated from our measured tok/s and average draw.

If the decision is still which card to buy, start with the budget GPU guide to match model size to VRAM. This power sweep tests a cap on an existing card; it does not make a model fit in less memory.

For workload memory planning, the Ollama context and parallel-request guide explains why a successful model load at one context changes the memory needed at another. Check that configuration before comparing power-cap runs.

Set an NVIDIA power limit with nvidia-smi

On Linux, first identify the GPU and read its allowed range. Use its UUID so a multi-GPU machine keeps targeting the same card after a reboot; NVIDIA recommends UUID or PCI bus ID because numeric GPU ordering can change (NVIDIA documentation).

nvidia-smi --query-gpu=name,uuid,power.min_limit,power.max_limit,power.default_limit,power.limit --format=csv

Copy your chosen GPU’s UUID into the placeholder below. Our 3060 Ti allows 100–220 W; our RTX 2060 allows only 125–170 W. Choose a cap inside your card’s range. Setting the limit requires root.

sudo nvidia-smi -i GPU-REPLACE-WITH-YOUR-UUID -pl 150
nvidia-smi -i GPU-REPLACE-WITH-YOUR-UUID -q -d POWER

Read back the power-limit fields, including the enforced limit where reported, then run your usual model. To undo the change, set -pl to the default wattage you recorded in the first command. Unsupported fields or controls can return N/A or a not-supported response; use the controls your card exposes.

What “SW Power Cap” means for local LLMs

SW Power Cap means NVIDIA’s software power algorithm is reducing clocks to stay inside the power ceiling. After you deliberately lower that ceiling, seeing the reason during generation is expected. Thermal slowdown is a different reason and calls for a cooling check.

To read the accumulated counters on a supported driver:

nvidia-smi -i GPU-REPLACE-WITH-YOUR-UUID --query-gpu=clocks_event_reasons_counters.sw_power_cap,clocks_event_reasons_counters.sw_thermal_slowdown --format=csv

Compare counter readings before and after your workload. For current flags, counter deltas and external power-brake signals, use the SW Power Cap diagnostic guide. Judge the cap by token speed and energy together: the counter identifies a cause, while the benchmark tells you what it costs.

Keep the power limit across reboots on Linux

A manual cap resets after driver unload. Persistence mode keeps the driver loaded between clients, but persistence mode also resets at reboot. You need a startup action to reapply both settings, rather than treating -pm 1 as a saved power profile (NVIDIA documentation).

On a systemd-based Linux host, use a oneshot service. Run this in the operating system that owns the GPU: the host for an ordinary Docker setup, or the guest after full GPU passthrough. Our Proxmox versus direct Docker guide explains that ownership boundary.

First run command -v nvidia-smi and use that executable path in the unit. Save the following as /etc/systemd/system/nvidia-power-limit.service, replacing both UUID placeholders and choosing your supported wattage. This example uses /usr/bin/nvidia-smi and a 150 W cap:

[Unit]
Description=Apply NVIDIA GPU power limit
After=systemd-modules-load.service
Before=ollama.service

[Service]
Type=oneshot
ExecStart=/usr/bin/nvidia-smi -i GPU-REPLACE-WITH-YOUR-UUID -pm 1
ExecStart=/usr/bin/nvidia-smi -i GPU-REPLACE-WITH-YOUR-UUID -pl 150
RemainAfterExit=yes
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

The two commands run in order. Before=ollama.service orders this unit before the native Ollama service when both start; for Docker or another launcher, order the workload behind this unit in that launcher’s configuration. The retry handles an initial failure while the driver becomes available. These are standard systemd oneshot settings.

Load the unit, enable it at boot and start it now:

sudo systemctl daemon-reload
sudo systemctl enable --now nvidia-power-limit.service
systemctl status nvidia-power-limit.service

Reboot, then read back the GPU power limits with nvidia-smi -i GPU-REPLACE-WITH-YOUR-UUID -q -d POWER. If you unload/reload the driver later, run sudo systemctl restart nvidia-power-limit.service to apply the cap again. A completed oneshot stays marked active; that status alone does not continually reassert the GPU setting. The systemctl manual covers enabling, restarting and inspecting the unit.

Run this checklist before you commit to a cap

1. Ask nvidia-smi for the Min, Max and Default Power Limit for your card and note them.
2. Warm up the model, then run the same prompt three times at stock, with a fixed seed and temperature 0.
3. Set 150 W, repeat the runs, and compare tok/s and average draw.
4. Try 125 W. Keep it if the speed loss is acceptable to you.
5. Check the sw_power_cap and thermal counters to confirm what is limiting clocks.
6. Add the boot-time reapply, reboot once, and confirm the Enforced Power Limit.

Keep this measured curve with its workload: RTX 3060 Ti, 7B inference and 512-token outputs. Repeat the sweep for a different card, larger model or long-context workload, and measure fan noise separately if quiet operation is your goal.

Speech synthesis needs its own measurements

These power-limit results describe LLM inference. For a voice-assistant bench, use our ElevenLabs, Kokoro and Piper decision guide to budget speech output and plan a separate latency and energy check; do not treat the LLM results above as TTS benchmarks.

Choosing a whole machine rather than tuning a card you own? Compare a GPU desktop with a unified-memory mini PC by the model you want to run.

Choose the PSU separately from the runtime cap

A software cap does not validate the complete machine’s power supply. The two-GPU PSU selection guide compares actual MSI, Seasonic and Corsair options using worked budgets, independent cable runs and physical-fit checks; choose PSU capacity and cables for the hardware, including the uncapped startup state.

Measure it at the wall with a Kill A Watt P4400

The Kill A Watt P4400 is the useful extra tool here: it shows active watts plus accumulated kWh and elapsed time on a local display. Plug your PC into it and measure the whole machine, including the CPU, fans and power-supply losses. Follow the meter’s label and connected-load ratings; the P4400 is a US-outlet meter (P3 specifications, operation manual).

Run the same warmed-up workload at stock and at your chosen cap. Record wall watts for a quick comparison; for a repeated batch, record the meter’s kWh and elapsed time for the same completed work. Keep the monitor and other equipment outside that measurement unless you deliberately want their energy included. Write down the totals before unplugging the P4400, because removing power resets its counters.

Choose it for the stand-alone readout. If you want retained totals or app history instead, the power-meter buying guide compares those workflows. Then use the monthly homelab electricity guide to combine your measured active and idle energy with your tariff.

What you save per month

Here is a GPU-board energy estimate from our measured sweep, with an assumed 30-day month and electricity at $0.15/kWh. Each row assumes the same active hours at stock and at the lower cap; idle periods are excluded. It estimates the GPU portion, not the whole-PC bill—use the wall meter above for that.

Cap	Assumed active hours/day	Measured GPU watts saved vs stock	GPU kWh saved/month	Estimated saving at $0.15/kWh
150 W	1	48.0 W	1.440	$0.22
150 W	4	48.0 W	5.760	$0.86
150 W	8	48.0 W	11.520	$1.73
125 W	1	70.9 W	2.127	$0.32
125 W	4	70.9 W	8.508	$1.28
125 W	8	70.9 W	17.016	$2.55

The formula is (stock watts − capped watts) × active hours/day × 30 ÷ 1,000 × tariff. Costs are rounded half-up to cents. A fixed-size batch takes longer at the lower speed, so compare kWh per completed job for that workload; the 100,000-token example above includes that extra runtime.

Start at 150 W on the measured 3060 Ti, keep the model and prompt constant, and read back the cap after a reboot. You keep nearly all the speed, reduce the card’s heat output and get an electricity comparison grounded in your own wall readings.