Auto-generated transcript. Minor errors may exist. The audio is the authoritative version.
Build Log. I'm Nick.
Here's what I shipped this week and what it taught me.
The Security Hole Nobody Talks About
What if the biggest AI security risk for your business isn't a shadowy hacker in a foreign country… but the laptop sitting on your developer's desk?
I'm talking about local AI security risks. The unsexy, overlooked vulnerabilities that are shipping in production right now while everyone argues about prompt injection and model poisoning.
Most AI security checklists are missing the three most common local attack vectors. I've seen them deployed and exploited in the wild. And I'm going to walk you through exactly what happens when your “secure” local setup becomes your biggest liability.
Why Your Local AI Isn't As Safe As You Think
Three months ago, I was consulting for a fintech startup. They'd made the smart choice, or so they thought. No OpenAI API calls. No cloud dependencies. Everything running locally on their infrastructure.
[BED: DUCK]
“We're keeping everything in-house,” the CTO told me. “Maximum security.”
They were running a local Llama instance for document classification. Processing loan applications, extracting key data points, routing them to the right underwriters. Clean setup. Docker containers. Proper network segmentation.
Except they had a twenty-gigabyte text file sitting in their temp directory with every single loan application from the past six months.
[BED: SWELL]
Teams are shipping these tools to “keep data in-house,” but they're deploying them on unvetted consumer hardware, with default configs, and calling it a day. The rush to deploy local models like Llama, Mistral, even local Claude instances creates this false sense of security.
You've locked the front door, but the windows are wide open.
This matters because a breach here isn't about sophisticated model attacks. It's about physical access, data exfiltration, and simple human error. The kind that costs real money and destroys real trust.
The Three Attack Surfaces Everyone Misses
You've probably heard that local AI is inherently more secure.
Here's what actually happens when you run it.
It's not the model. It's everything touching the model.
**Attack Surface One: The Data Pipeline's Memory.**
Where does your prompt context live before it hits the model? I've audited seventeen local AI deployments this year. In fourteen of them, the answer was “somewhere I didn't expect.”
Often, it's in a plaintext log file. Sometimes it's an unsecured Redis instance on the same machine. I've seen it in browser local storage when teams are using web interfaces. A simple local privilege escalation can dump it all.
And this is where it gets interesting from an operations standpoint.
I tested a setup for that fintech client: local Claude Haiku for classifying support tickets. The webhook fired, the pipeline caught it, but the classification prompts and outputs were being written to `/tmp/claude_logs.csv` for debugging purposes.
The logs were never cleaned up. Any process on that box could read them. Any user with local access could copy twenty thousand customer support conversations to a USB drive.
**Attack Surface Two: The Model Cache.**
Those local models create vector caches, weight caches, conversation history on disk. If they're not encrypted, a stolen laptop or a decommissioned server becomes a goldmine.
I was helping a law firm set up local document review last month. They were using a fine-tuned Mistral instance to identify privileged communications. Smart approach. Except the model was caching every document it had ever seen in an unencrypted SQLite database.
When they decomissioned the server six weeks later, they wiped the main drive but forgot about the cache volume. That volume had fragments of attorney-client privileged documents from four hundred cases.
**Attack Surface Three: The “Safe” Output.**
You think output is safe because it's just text. But if that local AI is summarizing sensitive emails or anonymizing data, and its output is auto-pasted into a Slack channel or a shared Google Doc, you've just created a new data leak path.
I watched this happen real-time during a security audit. Local AI was anonymizing patient records for research. Working perfectly. Names became Patient A, Patient B. Addresses became City X, City Y.
But the AI output was being automatically posted to a Slack channel that included contractors, interns, and a marketing consultant who definitely shouldn't have access to medical data, even anonymized.
The tool worked exactly as designed. The humans messed up the permissions.
Everyone says “run it locally, it's secure.”
Here's what actually happens when you run it: you create three new local data stores you probably aren't monitoring.
Your 15-Minute Local AI Security Audit
Here's your practical audit. Go run it today.
**Step One: Audit the Data Trail.**
SSH into whatever machine is running your local AI. Find the process ID of your inference engine. Run `lsof -p [PID]` to see every file it has open.
Then search for files you didn't expect: `find /tmp -name “*.csv” -o -name “*.log” -o -name “*.json”` and do the same in `/var/tmp` and the user home directory.
I guarantee you'll find something that surprises you.
At that fintech startup, this command showed me the twenty-gig log file. At the law firm, it revealed the SQLite cache. Five minutes of looking, months of cleanup avoided.
**Step Two: Sandbox the Execution.**
Don't run your model with user or admin privileges. This is basic security that somehow gets forgotten when AI is involved.
Use Docker with a specific user ID, not root. Create a systemd service file with defined, minimal permissions. Use private tmpfs mounts so temporary files never touch the real filesystem.
This is plain-English architecture: the pipeline catches the webhook inside a fenced yard, not running loose in your data center.
Here's the Docker command I use for local Claude runs: `docker run –user 1001:1001 –tmpfs /tmp –read-only –memory=2g –cpus=1.0`. The model gets exactly what it needs, nothing more.
**Step Three: Segment the Output.**
The process generating AI output should never have direct write access to your CMS, your Slack workspace, your shared drives. Use a queue.
Your local Llama run writes its result to a local database row. A separate process, with different credentials and different permissions, handles posting to external systems.
If the AI process gets compromised, the attacker can't automatically push malicious content to your customer-facing channels.
Mid-Roll Check-In
Stopping data leaks at the local level is just one piece. I've put together a one-page checklist called “The Local AI Deployment Hardening Guide.”
It has the exact commands for the audit I just walked through, sample Docker run commands, and a systemd service file template you can adapt for your setup.
Download it for free at buildlog.show/localchecklist. That's buildlog.show/localchecklist.
The Contrarian Take – Sometimes Cloud Is More Secure
Here's my most controversial bet: for teams under fifty people, a properly configured cloud AI API is often more secure than your local setup.
I'll bet with eighty percent confidence that a small team using the OpenAI API with strict project-level API keys, zero data retention turned on, and a gateway like Helicone for audit logging has a smaller attack surface than a dozen poorly managed local Llama instances.
The reasoning is simple. OpenAI has an entire security team you can't possibly match. Microsoft has compliance certifications that would cost you millions to achieve independently. Your risk shifts from local physical and data hygiene to key management and network security.
And key management is often easier to centrally control and audit.
Let's talk numbers. GPT-4-Turbo costs about seven cents per classification task. Sounds expensive until you consider that the compliance and security overhead of managing your own local setup easily eats four hours a week of senior developer time.
At a loaded cost of sixty dollars an hour, that's two hundred forty dollars a week. Twelve thousand five hundred dollars a year. How many API calls does that buy you?
I run thirteen WordPress sites. The ones using Claude Haiku for comment moderation through the API have had zero security incidents in eight months. The one site where I tried a local setup had a prompt injection attack in week three because I'd misconfigured the input sanitization.
The practical approach isn't “all local” or “all cloud.” It's hybrid.
Use Claude Haiku in the cloud for high-volume, low-risk classification with data retention controls turned on. Reserve local Claude Opus for heavy reasoning on truly sensitive documents, but only after you've done the hardening I outlined earlier.
Match the security model to the actual risk, not to your assumptions about what's theoretically safer.
What You're Going to Do Today
Today, pick one local AI tool your team is running or experimenting with. Could be an Ollama instance, a GPT4All model, a local fine-tune, anything.
First, SSH into that machine and run `lsof -p [PID]` to see every file your AI process has open. Write down what you find.
Second, check the directory where that model lives for any log or cache files you didn't expect. Use the find commands I gave you earlier.
If you find unsecured logs or a process running as root, you've just found your biggest local AI security risk. Go fix that one thing. Use a container, change the log directory, lock down the permissions.
Don't try to fix everything at once. Fix the one thing that would hurt most if it leaked tomorrow.
Then head to buildlog.show/localchecklist to do the full audit when you have thirty minutes to spare.
The cost breakdowns I mention are always in the show notes. Every API call, every subscription, every shortcut — documented.
If you're thinking about what to automate with AI, not just the how, check out Automation Logic. We break down real workflows, like the audit system that saved a fintech startup twenty hours a week. Search for it wherever you get your podcasts.
Outro
That's the build log for this week.
Ship something. Measure it. Tell me what happened.
Remember: security isn't just about the big, scary external threats. It's often about the quiet, local process you shipped and forgot to monitor.
The laptop on the desk. The temp directory that never gets cleaned. The cache file that outlives the project.
I'm Nick. Thanks for building with me.
