Qwen3.5-2B on an old Vostro: local setup and real timings

My Dell Vostro 3360 has a Core i3-3217U and 8 GB of RAM. It now runs Qwen3.5-2B with a browser chat, photo input, optional web search and no token billing. A parking-photo description finished in 3 minutes 43 seconds while the browser was closed, then returned to the same chat after reopening. This guide includes the current setup files, the fixes that made that work and the actual resource and timing records.

The laptop also runs home and media services, so inference uses two threads and low process priority. Text and image inference work offline after installation. Web search needs internet access and sends queries to an external provider. Long prompts and image processing still take minutes on this CPU.

What runs on the laptop

Qwen3.5-2B supplies the model; its card lists the Apache 2.0 license. llama.cpp loads a quantized GGUF file and serves the browser interface and local API. I used the Unsloth Q4_K_M conversion. No GPU, paid API, API key or subscription is involved in inference.

Internet access is needed to download the source, weights and browser assets. The running server uses a local model path and --offline. Electricity and storage still have costs; there is no per-token charge.

The 3 GiB limit is a service setting. The measured resident memory was about 1.41 GiB. I did not record average CPU utilization or power consumption, so I cannot give a measured wattage or electricity cost. I would allow several gigabytes of free disk space for the build and download caches.

1. Prepare Ubuntu and the configuration files

Use Bash on Ubuntu with a working systemd user manager. These steps target a fresh $HOME/local-llm installation; the upgrade section below covers an earlier setup. Keep the same shell open so the path variables remain defined. Download the 4 November companion package, then open a terminal in its download directory. The package contains configuration, the tested UI adapter, the small runtime patch and measurement records; weights and the original UI assets are downloaded separately.

sudo apt-get update
sudo apt-get install -y build-essential git curl python3 python3-pip unzip

# Check the versioned download before extraction.
printf '%s  %s\n' \
  958f0d73ea21e722d93c8569b3467ff7885365db7912ce2ef517bed7fa2d3763 \
  qwen-local-files-20261104.zip | sha256sum --check - &&
unzip -n qwen-local-files-20261104.zip

llm_root="$HOME/local-llm"
bundle_dir="$PWD/qwen-local-files-20261104"
(cd "$bundle_dir" && sha256sum --check SHA256SUMS)

mkdir -p "$llm_root"/{models,runtime,config}
cp "$bundle_dir/ui.json" "$llm_root/config/ui.json"
cp "$bundle_dir/measure.py" "$llm_root/measure.py"
cp -a "$bundle_dir/web-search" "$llm_root/"
cp -a "$bundle_dir/reliability" "$llm_root/"

Check the actual machine before choosing a build. A laptop's model name alone is not enough to identify its CPU instructions or available memory.

lscpu
free -h
df -h "$HOME"
grep -m 1 '^flags' /proc/cpuinfo

I used an isolated CMake 4.1.2 wheel. This also avoids depending on Python's virtual-environment bootstrap, which was unavailable on this installation.

python3 -m pip install --only-binary=:all: --no-compile \
  --target "$llm_root/toolchain" 'cmake==4.1.2'
cmake_bin="$llm_root/toolchain/cmake/data/bin/cmake"
"$cmake_bin" --version

2. Download and verify the model

The URL below pins the conversion to the revision used here. Download to a temporary filename and verify the SHA-256 before giving the server the final file. The pinned model repository records the file and revision.

curl --fail --location --retry 3 \
  --output "$llm_root/models/Qwen3.5-2B-Q4_K_M.gguf.part" \
  https://huggingface.co/unsloth/Qwen3.5-2B-GGUF/resolve/f6d5376be1edb4d416d56da11e5397a961aca8ae/Qwen3.5-2B-Q4_K_M.gguf &&
printf '%s  %s\n' \
  aaf42c8b7c3cab2bf3d69c355048d4a0ee9973d48f16c731c0520ee914699223 \
  "$llm_root/models/Qwen3.5-2B-Q4_K_M.gguf.part" | sha256sum --check - &&
mv "$llm_root/models/Qwen3.5-2B-Q4_K_M.gguf.part" \
   "$llm_root/models/Qwen3.5-2B-Q4_K_M.gguf"

The expected language-model size is 1,280,835,840 bytes. The current image-capable service also needs mmproj-F16.gguf from that exact conversion revision. Its size is 668,227,264 bytes; the expected checksum is verified below. The pinned conversion files provide both weights and the projector.

curl --fail --location --retry 3 \
  --output "$llm_root/models/mmproj-F16.gguf.part" \
  https://huggingface.co/unsloth/Qwen3.5-2B-GGUF/resolve/f6d5376be1edb4d416d56da11e5397a961aca8ae/mmproj-F16.gguf &&
printf '%s  %s\n' \
  7035e9cb8d7c6a9681d07eef9a364783e86ea4cd73faab2eabb4f43a101830c7 \
  "$llm_root/models/mmproj-F16.gguf.part" | sha256sum --check - &&
mv "$llm_root/models/mmproj-F16.gguf.part" \
   "$llm_root/models/mmproj-F16.gguf"

A checksum failure means stop and inspect the download before starting the server. The two weight files total about 1.95 GB in decimal units, before source, build files and caches.

3. Build llama.cpp for this CPU

I built on the Vostro itself, using GGML_NATIVE=ON so the compiler selects instructions the host supports. This matters on an i3 without AVX2. The upstream CPU build instructions are the reference; the commands here pin the tested source.

git clone --depth 1 --branch v0.8.0 \
  https://github.com/ggml-org/llama.cpp.git \
  "$llm_root/runtime/llama.cpp" &&
test "$(git -C "$llm_root/runtime/llama.cpp" rev-parse HEAD)" = \
  8642541b255bfb5dc39cd351a178be09013a55d6

The commit comparison must succeed before continuing. This setup adds one private change to that base: the native completed-stream buffer lasts 24 hours instead of five minutes. Apply the supplied patch before building:

git -C "$llm_root/runtime/llama.cpp" apply --check \
  "$bundle_dir/reliability/stream-retention.patch" &&
git -C "$llm_root/runtime/llama.cpp" apply \
  "$bundle_dir/reliability/stream-retention.patch"

This is a local source patch, not an upstream release setting. The shallow checkout identifies the base as 0.8.0-dev (build 1, commit 8642541); the patch is separately included in the download. If the patch check fails on an existing installation, inspect whether it is already applied rather than forcing it.

The browser interface is also an installation dependency. The original build fetched official prebuilt UI assets and I recorded version b12032 and its archive hash. The commands below verify that same archive, supply it as the build input and disable automatic UI downloads. This also prevents an upstream fallback to the latest alias.

mkdir -p "$llm_root/runtime/assets" &&
curl --fail --location --retry 3 \
  --output "$llm_root/runtime/assets/llama-ui-b12032.tar.gz" \
  'https://huggingface.co/buckets/ggml-org/llama-ui/resolve/b12032/dist.tar.gz?download=true' &&
printf '%s  %s\n' \
  17ce6362156b01ec7b94f485baba4e5d3c799046e660ca4301aa245781b32405 \
  "$llm_root/runtime/assets/llama-ui-b12032.tar.gz" | sha256sum --check - &&
mkdir -p "$llm_root/runtime/llama.cpp/tools/ui/dist" &&
tar -xzf "$llm_root/runtime/assets/llama-ui-b12032.tar.gz" \
  -C "$llm_root/runtime/llama.cpp/tools/ui/dist"

The verification and extraction are one guarded command chain. If it fails, stop before building. Configure the CPU server with both UI building and UI downloading disabled; it embeds the verified files supplied above. The build uses two workers and a low priority, leaving the existing services able to run.

"$cmake_bin" \
  -S "$llm_root/runtime/llama.cpp" \
  -B "$llm_root/runtime/build" \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_NATIVE=ON -DGGML_BACKEND_DL=OFF \
  -DLLAMA_BUILD_TESTS=OFF -DLLAMA_BUILD_EXAMPLES=OFF \
  -DLLAMA_BUILD_APP=OFF -DLLAMA_BUILD_SERVER=ON \
  -DLLAMA_BUILD_TOOLS=ON -DLLAMA_BUILD_MTMD=OFF \
  -DLLAMA_BUILD_UI=OFF -DLLAMA_USE_PREBUILT_UI=OFF \
  -DLLAMA_OPENSSL=OFF

nice -n 10 "$cmake_bin" --build "$llm_root/runtime/build" \
  --parallel 2 --target llama-server

"$llm_root/runtime/build/bin/llama-server" --version

The initial full build took about 32 minutes while the laptop also ran its existing workloads. A normal request does not rebuild anything. The server links the image-processing library; LLAMA_BUILD_MTMD=OFF disables the separate command-line target, not server image support. Node.js and a separate frontend service are not required. The service will serve an adapted copy of the verified browser files, prepared next.

4. Prepare the local browser UI

The helper reads an immutable copy of the verified b12032 assets and writes a separate served bundle. Copy the supplied source assets to the locations used by the tested helper, remove only the copied service worker, then add the local adapter before the app starts:

mkdir -p "$llm_root/runtime/build/tools/ui/dist" "$llm_root/runtime/webui"
cp -a "$llm_root/runtime/llama.cpp/tools/ui/dist/." \
  "$llm_root/runtime/build/tools/ui/dist/"
cp -a "$llm_root/runtime/build/tools/ui/dist/." \
  "$llm_root/runtime/webui/"
rm -f "$llm_root/runtime/webui/sw.js"

python3 - "$llm_root/runtime/webui/index.html" <<'PY'
from pathlib import Path
import sys
path = Path(sys.argv[1])
text = path.read_text()
loader = '<script src="/local-qwen-web-search.js?v=10"></script>'
if 'local-qwen-web-search.js' not in text:
    if text.count('<head>') != 1:
        raise SystemExit('Unexpected UI index; inspect it before continuing')
    path.write_text(text.replace('<head>', '<head>\n' + loader, 1))
PY

python3 "$llm_root/reliability/prepare_ui.py"

The UI helper refuses an unexpected upstream budget mapping. It creates bundle.BWYoSFMS.local-reliability.js and installs the tested adapter, loaded with the versioned URL ?v=10. The original assets stay unchanged. Removing the copied worker lets the server's retiring worker release an old UI cache; it does not clear chat history.

5. Start the local user service

The complete service file uses systemd's %h home-directory expansion. It points to the layout created above and listens on 127.0.0.1:8092. It includes the matching projector, 64–512 image tokens, two CPU threads, one slot, a 4096-token context, reasoning off, offline mode and the served UI path.

mkdir -p "$HOME/.config/systemd/user"
install -m 600 "$bundle_dir/qwen-local.service" \
  "$HOME/.config/systemd/user/qwen-local.service"

systemd-analyze --user verify \
  "$HOME/.config/systemd/user/qwen-local.service"
systemctl --user daemon-reload
systemctl --user enable --now qwen-local.service

systemctl --user status qwen-local.service
curl --fail http://127.0.0.1:8092/health

The health request returns {"status":"ok"} once model loading finishes. Wait and retry if it reports model loading. The original text-only first start took 11.1 seconds and a controlled restart about 4.0 seconds. With the projector, a later healthy restart took 5.028 seconds. Startup timing is separate from answer latency. Open http://127.0.0.1:8092/ in a browser on the laptop. Other devices cannot connect to this loopback listener.

The service has Nice=10 and a 3 GiB memory hard limit. Check which limits your user manager actually supports:

systemctl --user show qwen-local.service \
  -p MainPID -p MemoryCurrent -p MemoryHigh -p MemoryMax \
  -p MemorySwapMax -p Nice -p NRestarts

llm_pid=$(systemctl --user show qwen-local.service -p MainPID --value)
ps -p "$llm_pid" -o pid,ni,pcpu,rss,etime,comm

# Show which controllers are delegated to this user's manager.
systemctl show "user@$(id -u).service" -p DelegateControllers

This host delegated memory and PID controllers to user services, but not CPU or I/O controllers. I therefore did not claim a working CPU quota. Two inference threads and low process priority are the CPU policy used here. The MemoryCurrent cgroup value can also differ from process RSS, especially after restarts and shared file-cache accounting; compare the same metric across runs.

6. Use native tool calling and free keyless search

The native browser MCP client connects to Exa at https://mcp.exa.ai/mcp?tools=web_search_exa. Exa documents keyless search with a free rate limit. This installation has no search API key, subscription or paid fallback. Availability and anonymous limits belong to the provider; search is not offline.

The model generates the tool call, the browser executes the search, and the result returns to the model for an answer. The supplied adapter compacts the schema and two source excerpts to fit this small context. Only the readonly search tool gets a saved permission. Browser date/info tools are disabled by default to avoid unrelated prompt overhead.

Start a message with /search for a reliable search request:

/search Find the official llama.cpp repository and give its URL.

Native switches under + > Tools > Web Search still control availability. The adapter does not force a disabled tool back on. Plain text chat leaves tool selection automatic; the 2B model can skip a useful tool or invent a fact, so an explicit search command is better when sources matter. If the model omits source URLs, the browser adds a Sources list from the actual matching search result. That footer is separate from generated model tokens.

The current ui.json uses customJson: "{}". The earlier text-only guide forced an empty tool list, which would disable this search flow. The local bootstrap removes that old override once and preserves unrelated preferences and conversations. Refresh an already-open page; use a new conversation if an old one retains a different tool policy. Do not clear site storage or IndexedDB.

7. Attach a photo without exhausting the answer budget

Use + > Add files, attach a JPEG and ask what is visible. The matching --mmproj enables the vision path; without it, the original UI rejected images as unsupported. Check /props for modalities.vision: true. The model card describes image-and-text input with text output. This setup does not generate video.

A plain question with a newly attached photo suppresses automatic Exa search, so the model describes the image locally. An explicit /search request can still ask for web sources. Other tools and later text-only questions keep their normal policy.

For a simple description, Reasoning > Off is fastest. A saved High setting originally requested 8192 thinking tokens inside a total 512-token output limit. A reproduced request used all 512 tokens on reasoning and returned no visible answer. The local UI now budgets Low 64, Medium 96 and High/Max 128 thinking tokens. The adapter also caps thinking to a quarter of a smaller positive output limit. Search planning skips thinking.

Images use 64–512 tokens to fit the CPU and context. That reduces detail in large photos; the runtime recommends at least 1024 for precise visual grounding. The successful test recognized a nighttime parking scene, but the 2B model still guessed car makes and other details incorrectly in another run. Do not treat a broad scene description as reliable identification or precise positioning.

8. Recover a completion after closing the browser

The native UI tags a streaming request with X-Conversation-Id. The server can then continue that completion after its HTTP client disconnects. The UI reattaches to /v1/stream when the same chat is reopened. See the pinned resumable-streaming design.

The upstream completed buffer lasts five minutes. That was too short: a controlled request's replay returned HTTP 404 after a 365-second wait. The supplied one-line source patch changes retention to 24 hours, while the service stays running. The existing ring remains capped at 4 MiB per session; there is no additional aggregate-session cap. The service's 3 GiB memory limit remains in effect.

Reopen the same conversation in the same browser and origin. Chat history lives in browser IndexedDB, so this is not cross-device history sync. Pressing Stop explicitly cancels a request. A service restart, expiry or a newer completion for that conversation also removes the temporary replay. Saved replies already in browser history remain after a restart.

This covers an already-submitted model completion. MCP tools run in the browser, so a multi-step search workflow still needs an open browser to execute its next tool. The measurement client below is a foreground text benchmark and does not opt into resumable streaming.

In the accepted test, the browser closed 3.520 seconds after submitting a 300×238 parking image. The server completed a visible description at 222.720 seconds. After another 365 seconds with the browser still closed, native replay restored the exact answer without another generation. Recovery plus a further history-preserving reload finished at 603.345 seconds from submission. This tests the former expiry failure; 24 hours is the configured retention, not a completed 24-hour soak test.

9. Keep optional remote access private

The unit binds only to 127.0.0.1:8092. For my remote setup, an existing HTTPS tunnel forwards a protected hostname to that loopback origin. Authentication covers the entire hostname, including assets and API endpoints, with an owner allow policy and MFA. No model port is exposed directly and no management token is stored in the UI.

A remote hostname has its own browser origin and history. Keep using the same address when returning to a chat. The recovery test above used the local origin; separate checks verified the public TLS/login boundary and origin compatibility, without automating a personal MFA session.

10. Upgrade an earlier text-only installation

Download the versioned package into a fresh folder. Keep the pinned language-model file, verify and add the projector, then wait for the current request to finish. Stop only qwen-local.service before patching or rebuilding its mapped runtime files. Check the base commit, apply the retention patch once, rebuild, prepare the served UI, install the updated unit and ui.json, then reload the user manager and start Qwen.

git apply --check checks an unapplied patch; git apply --reverse --check checks whether it is already present. Inspect an unexpected result rather than resetting source changes. The companion README records the exact layout and upgrade order. A previously expired empty reply needs Regenerate after the page reloads; an old deleted buffer cannot be reconstructed by the new settings.

The original France benchmark: 1 November 2026

I sent a question asking the model, in Russian, to name the capital of France. The English translation of its opening sentence was: “The capital of France is Paris.” It then added an unrequested historical aside. The full response is retained as model output in the measurement record; it is not a verified historical source. The total time below includes that longer answer.

The timer started immediately before the HTTP request. “First answer text” means the first non-empty assistant content chunk, excluding response headers, empty deltas and progress events. “Complete response stream” ends at the SSE completion marker. Decode speed is llama.cpp's own generation metric; it excludes prompt processing and should not be read as the rate over the whole 26.554 seconds.

The model was already loaded. Part of the prompt was cached, the laptop's other home and media services were running, and this was one observation rather than repeated trials on an idle machine. It is useful evidence of what this setup did, but not a promise that every request will finish in 26 seconds.

The capital was correct. The unsolicited extra text is a reminder to check the model's explanations before using them. A small local model is useful for experiments, short drafts and simple questions; a successful timing test does not establish accuracy on complex tasks.

Photo and search measurements: 4 November 2026

These are single observations with the model loaded and other home/media workloads running. Prompts, image size, tool context and cache state differ between runs. They show what happened on this laptop; they are not averages or a controlled Off-versus-High comparison. Decode rate excludes prompt processing. The High token count includes reasoning.

An earlier complete search check took 123.900 seconds, including a repeated provider query of 0.557 seconds, with a cached 313-token prompt prefix. The later 311.770-second run is also part of the record. Neither should be presented as a guaranteed search latency. Image processing and prompt evaluation explain why a reply with only four output tokens can still take almost five minutes.

The updated measurement record includes cache counts, recorded server timings, failure observations and recovery checks. It excludes private images, chat identifiers and infrastructure addresses. The photos are not redistributed.

RSS and cgroup memory are different counters, partly because of shared file-backed pages. The 3 GiB limit is not a measured working-set requirement. No average CPU utilization, wattage or electricity cost was measured.

Measure your own request

The downloadable Python measurement client uses only the standard library and talks directly to the local endpoint. It prints the streamed answer followed by first-text time, full-stream time, usage and server timings.

python3 "$HOME/local-llm/measure.py" "Name the capital of France"

This command asks a new question in English. The table above describes the recorded Russian question; wording, language and cache state affect tokenization and speed. The companion JSON preserves the original prompt and answer so that translation does not silently change the evidence.

Operate it after installation

A systemd user service normally starts with the user manager. Lingering was already enabled on this host; enable it explicitly if you want the service to start at boot and continue after logout. Useful commands are:

journalctl --user -u qwen-local.service -n 60 --no-pager
systemctl --user restart qwen-local.service
systemctl --user stop qwen-local.service
systemctl --user start qwen-local.service

# Optional: keep the user manager running after logout and start it at boot.
sudo loginctl enable-linger "$USER"

# To disable automatic startup and stop the model:
systemctl --user disable --now qwen-local.service

The initial offline browser check blocked external requests and still completed a chat. Service startup was later observed after a real host reboot on 2 November. On 4 November, image upload, browser-close replay, native tool controls, actual Exa search, explicit Stop and history retention all passed. The adapter has 18 passing regression checks, plus a real native-menu/request check. Other services were kept independent.

For loading failures, check the service journal, both weight checksums and available memory. If upload reports that vision is unavailable, verify --mmproj, check /props and reload the page. If a previous reply is empty, reload the updated UI and choose Regenerate; Off avoids thinking overhead for simple questions. Keep browser history intact.

The server documentation for the pinned revision explains the flags and API. Keep the source commit, model revision and checksums with your setup. That makes a working local installation much easier to reproduce later.