Building a Local AI Coding Server – Part 4: Giving an AI Agent Its Own Account

Why the agent gets its own account

By the end of Part 3, the server had a fast coding model running around the clock, with an OpenAI-compatible API at 127.0.0.1:8080. (An API is the interface programs use to talk to the model. OpenAI-compatible means it speaks the same format as OpenAI's service, so most tools can use it.) The next goal was the reason I built this machine: an autonomous agent, a program that takes a task, writes code, runs commands, checks its own work and keeps going without me approving each step.

I used Cline CLI, the command-line version of the Cline coding assistant. Its -y flag ("yes to everything") lets it run file edits and shell commands without asking. That's what makes it autonomous, and it's also why it can't run as my normal user. An unattended process that executes whatever commands a language model comes up with should get the least access possible. If it misreads a task, it should damage a sandbox, not my home directory.

So the design was simple:

• A dedicated Linux user, agent, with no sudo (it can't run commands as administrator) and no password, so you can only reach it from the admin account.
• Everything it works on lives in Git repositories (Git tracks every change to a set of files), so any change can be inspected or undone.
• No secrets within its reach: no SSH keys, no tokens, nothing in its home directory worth stealing.

Setting up the user and Cline CLI

Creating the user is standard Linux account management:

sudo adduser --disabled-password --gecos "" agent   # new user, no password, no sudo
groups agent                                          # should list only: agent users
sudo chmod 750 /home/<your-user>                      # keep your own home private

The last line matters. On Ubuntu, home directories can be readable by other users depending on how they were created. With 750, only you and your group can look inside, so agent gets "Permission denied" on your files.

To switch into the account, run sudo -iu agent (and exit to leave). One habit I picked up: run those two commands on their own, never inside a pasted block. If you paste sudo -iu agent followed by more lines, the shell switches users partway through and the remaining lines run in whichever shell happens to be reading input. It's confusing and can run commands as the wrong user.

Cline CLI is an npm package (npm is Node.js's package manager). I installed it system-wide as the admin so the agent can use it but can't modify it:

sudo npm install -g cline

Then, as agent, I pointed it at the local model instead of a cloud provider:

cline auth -p openai -k local-no-key -b http://127.0.0.1:8080/v1 -m local-model

• -p openai uses the OpenAI-compatible provider.
• -k local-no-key is a dummy key. My server is bound to localhost and has no key, but the CLI wants a value.
• -b is the base URL of my llama.cpp server, and -m local-model is the alias I gave the model in Part 3, so swapping models later never touches the agent config.

I also gave the agent its own Git identity ("Cline Agent") so every commit in history clearly shows whether I or the agent made it. The first test task in a scratch sandbox repo worked: it wrote a small JavaScript file and ran it. It also didn't commit anything, which is where the real work started.

Rules, and how to prove they're loaded

Cline supports rules files: Markdown files whose contents get added to the model's instructions on every task. Global rules for a user live in ~/.cline/rules/. Mine is 01-autonomous-agent.md, and it covers scope (stay inside the project folder), a "never" list (no rewriting Git history, no reading or committing secrets, no sudo, no commands that run forever), "stop after three failed attempts at the same fix," and some JavaScript style preferences.

The hard part was proving the rules actually reached the model. My first idea was to ask the agent directly: "Quote any rules you were given, or say NO RULES." It answered NO RULES, with total confidence. I spent a while assuming Cline wasn't reading the file, moved it around, checked versions, and got the same answer.

The file was being loaded the whole time. Models are bad at describing their own instructions: the rules get blended into a large system prompt, and "what rules do you have?" is an ambiguous question, so the model just guessed. Asking a model about its own context is a weak test.

The reliable test is a canary: an instruction with an effect you can check from the outside and that the model would never produce by chance. I added this as the first rule:

Your first message in every task must begin with the word RULESLOADED.

Now a harmless task proves it in one line:

cline -y 'Do not change any files. Just say hello.'
# reply must start with RULESLOADED

If the reply starts with RULESLOADED, the rules are in. If not, they aren't, no interpretation required. I re-run this after every Cline update, because rule file locations have changed between versions before.

Side note on quoting: wrap agent tasks in single quotes. In bash, a ! inside double quotes triggers history expansion, and a cheerful "Say hello!" fails with event not found before Cline ever sees it.

Checklists beat guidelines, and a safety net beats both

With the rules confirmed loaded, the agent still skipped commits. My first version said things like "commit your work when you're done," which is advice. The model treated it as optional.

What worked better was rewriting the important parts as numbered, mandatory checklists with exact commands:

## START CHECKLIST (mandatory, in order)
1. Run `git status`.
2. If there are uncommitted changes, commit them first: `git add -A && git commit -m "Save leftover changes"`.

## FINISH CHECKLIST (mandatory, in order; the task is not done until all steps pass)
1. Run the code or its tests and confirm it works.
2. Run `git add -A`.
3. Run `git commit -m "<short description>"`.
4. Run `git status` and confirm the tree is clean.

The next run followed it exactly. The run after that skipped the commit again. Progress, sort of. A smaller local model follows concrete steps far more reliably than general guidance, but "far more reliably" still isn't "always." Anything that really has to happen shouldn't depend on the model.

So I wrapped Cline in a small launcher script, agent-run:

agent-run ~/projects/sandbox 'Add a function that ... and test it'

It does four things:

1. Runs cline -y on the task inside the project folder, with timeout 4h so a stuck run can't go on forever.
2. Saves the full output to a timestamped log in ~/logs/.
3. Afterwards, if the working tree has uncommitted changes, makes a safety-net commit ("Auto-commit leftover changes after agent run …"). Nothing the agent did is ever lost, and it's easy to spot runs where it forgot.
4. Prints the last five commits so I can see at a glance what happened.

On the very next test, the agent forgot its commit again, and the safety net caught it. That's the pattern I'd recommend for any autonomous setup: tell the model what to do, then enforce the parts that matter in plain code outside the model.

For long jobs I run the launcher inside tmux, a terminal multiplexer that keeps sessions alive after you disconnect. As agent: tmux new -s agent, start the job, detach with Ctrl+B then D, and close SSH. Later, sudo -iu agent and tmux attach -t agent puts me right back in the session.

What I learned, and what's next

• Contain first, then trust. A separate user with no sudo, a private home directory for yourself, and no secrets in reach costs ten minutes and removes most of the risk of -y.
• Test the rules from the outside. A canary like RULESLOADED beats asking the model what it was told.
• Write instructions as checklists. Numbered, mandatory steps with exact commands work much better than advice with a local model.
• Enforce what matters in code. Timeouts, logs and the safety-net commit live in the launcher, where they always run.
• Git is the undo button. Since every change ends up in a commit, I can review the agent's work with git log -p and roll back anything I don't like.

That wraps up the server build. Over four posts, it went from a pile of hardware to a machine where two Radeon AI PRO R9700s serve an 80B coding model at about 56 tokens per second, around the clock, with an autonomous agent working in its own sandbox. Multi-GPU vLLM is still blocked by the RCCL bug from Part 2, which is reported to AMD, but llama.cpp's layer split means I'm not waiting on it.

Next up is the reason I started this project: the programming game Screeps, where you control your units by writing JavaScript that runs every game tick. The plan is a private Screeps server for fast testing, a project repo the agent can build and check by itself, and Screeps-specific rules so the model writes code that respects the game's CPU and memory limits. That gets its own series, starting with Introducing ScreepsAiSLP: Screeps + AI Slop.