Back to blog
November 12, 2026•20 views

Linux 0day Patching Baseline: A Practical Runbook for Client Apps You Host

linux patching strategyubuntu security patchingvulnerability scanning client appssbom devopsdevopssecurity
Linux 0day Patching Baseline: A Practical Runbook for Client Apps You Host

Most teams I audit do not have a patching problem. They have a baseline problem. Nobody wrote down what "patched" actually means, so every CVE turns into an argument about whether it is urgent, who owns the box, and whether rebooting on a Tuesday is allowed.

Then a Linux 0day lands. Some researcher posts a writeup, the exploit goes public within hours, and suddenly everyone wants a patching process that should have existed six months ago.

I run Web Engineering and AI Automation at AKIJ iBOS and I have shipped 300+ sites and apps over six years, a lot of them hosted on Ubuntu for clients in Bangladesh, the US, the UK, and Australia. This is the Linux 0day patching baseline I actually use. Not a framework PDF. A working set of SLAs, commands, and checks.

What a Linux 0day patching baseline actually is

A patching baseline is three things written down and enforced:

  1. Definitions. Which packages count, what "patched" means, and how you verify it.
  2. SLAs. How fast you must patch, by severity, including out-of-band emergency patches.
  3. Evidence. Logs, SBOMs, and scan output you can hand to a client or auditor.

That is it. If your baseline is a 40-page security policy with no commands in it, you do not have a baseline. You have a wish.

A quick note on terms, because the SERP results out there are mostly news pieces. A 0day is a vulnerability exploited before a patch exists. Once Canonical ships a fix, it is no longer a 0day, it is an Nday and your clock is running. The baseline is what you do in both phases: mitigate while unpatched, patch fast when the fix lands, and verify after.

And one hard truth from the incident writeups: patching is not the same as cleaning. If an attacker was already in, closing the CVE closes the door but leaves the intruder inside. Plan for that. The same lesson shows up in the LiteLLM PyPI compromise and PHP2Shell RCE, where the fix and the cleanup were two separate jobs.

The severity-to-SLA table I hand to clients

This is the core of the baseline. Everything else is plumbing. I put this table in the hosting agreement so nobody has to negotiate during an incident.

Severity Example Patch SLA (prod) Mitigation if no patch Approval needed
Critical (CVSS 9.0+, remote, exploited) Kernel LPE, OpenSSH RCE 24 hours, emergency window WAF rule, network ACL, disable feature None, on-call can act
High (CVSS 7.0-8.9) Priv escalation, auth bypass 7 days Compensating control documented Tech lead
Medium (CVSS 4.0-6.9) Local DoS, info leak 30 days None Normal change process
Low (CVSS <4.0) Minor info disclosure 90 days, next cycle None Normal change process
Kernel (any) Livepatch or reboot 24h to livepatch, 30d to reboot Livepatch / kpatch None for livepatch

Two rules I add below the table:

  • Emergency patches skip the change freeze. If a client insists on a freeze window, we agree in writing that security patches are exempt. Otherwise you are one 0day away from a very bad week.
  • The SLA clock starts at disclosure, not at your convenience. If you only check for updates on the first Monday of the month, your 24-hour SLA is fiction.

I also track a patch compliance percentage per client environment: patched_hosts / total_hosts for critical and high CVEs. That number goes in the monthly report. It is the single most useful security metric I have found for non-technical stakeholders, because it is one number and it moves. If you want the wider set of controls a founder should demand from a developer, my NIST cybersecurity guide for solopreneurs covers the rest.

Step 1: Inventory before you patch anything

You cannot patch what you do not know exists. Before touching a single apt command, get a real inventory. In practice that means:

  • Every host, VM, and container image, with owner and environment (dev, staging, prod).
  • Which packages are installed and from where (Ubuntu repos, PPAs, third-party apt keys, manual .deb installs).
  • What is exposed to the internet (SSH, HTTP, anything listening on 0.0.0.0).
  • Which hosts are stateful and which are cattle (rebuildable).

A quick exposure check that has caught real problems for me:

# What is actually listening on the public interface?
sudo ss -tulpn | grep -v '127.0.0.1'

# Installed packages from non-Ubuntu sources (the usual suspects for stale CVEs)
apt list --installed 2>/dev/null | grep -v ubuntu

# Third-party apt keys and repos
ls /etc/apt/sources.list.d/

Anything installed from a random PPA or a .deb you downloaded by hand is a patching liability. It does not get updates from unattended-upgrades unless you configured it, and nobody remembers it exists until it breaks.

For containers, the inventory is your image registry. If you are running node:latest in production, stop. Pin a digest and rebuild on a schedule.

Step 2: Turn on Ubuntu security patching properly

Most Ubuntu boxes I see have unattended-upgrades installed and misconfigured. It patches some things, silently, at a random time, and nobody knows what changed.

Here is a sane config for a hosted client app. Edit /etc/apt/apt.conf.d/50unattended-upgrades:

Unattended-Upgrade::Allowed-Origins {
    "${distro_id}:${distro_codename}-security";
    "${distro_id}ESMApps:${distro_codename}-apps-security";
    "${distro_id}ESM:${distro_codename}-infra-security";
};

// Do NOT auto-reboot prod without a window
Unattended-Upgrade::Automatic-Reboot "false";

// But do reboot staging freely
// Unattended-Upgrade::Automatic-Reboot "true";
// Unattended-Upgrade::Automatic-Reboot-Time "03:00";

Unattended-Upgrade::Remove-Unused-Kernel-Packages "true";
Unattended-Upgrade::Remove-Unused-Dependencies "true";

Then in /etc/apt/apt.conf.d/20auto-upgrades:

APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Unattended-Upgrade "1";

Key decisions in that config:

  • Security only, not everything. Auto-upgrading feature packages on prod is how you get a surprise PHP minor version at 4am.
  • No auto-reboot on prod. Kernel updates need a reboot (or livepatch). You want that to happen in a controlled window, not mid-checkout.
  • Auto-reboot on staging is fine and encouraged. Staging exists to eat the reboot so prod does not have to.

You can read the full option reference in the Ubuntu unattended-upgrades documentation and the Ubuntu Server automatic updates guide.

Kernel updates and livepatch

Kernel CVEs are the ones that scare people, and for good reason: a local privilege escalation in the kernel means anyone with a low-priv shell gets root. The fix usually requires a reboot, which is exactly what you cannot do on a busy host.

Ubuntu Pro gives you Livepatch, which applies kernel security fixes without a reboot. If you are hosting client apps on Ubuntu LTS, the Pro subscription is cheap relative to the cost of an unplanned reboot. My rule: livepatch for the 24-hour window, scheduled reboot within 30 days to actually land the kernel. See the Canonical Livepatch documentation for setup and status commands.

# Check livepatch status
canonical-livepatch status

# List pending kernel updates
apt list --upgradable 2>/dev/null | grep -i linux-image

# Check if a reboot is required
[ -f /var/run/reboot-required ] && cat /var/run/reboot-required

That last command is the one I put in every monitoring check. If /var/run/reboot-required exists for more than 30 days, it fires an alert.

Step 3: The emergency 0day runbook

When a critical 0day drops and a patch is available, you do not want to be reading docs. You want a numbered list. This is mine.

  1. Confirm the CVE and the fixed version. Read the Ubuntu security notice. Note the exact package version that fixes it.
  2. Assess exposure. Is the affected service internet-facing? Is the vulnerable code path reachable? A kernel LPE on a host with no untrusted users is lower risk than an OpenSSH RCE on a public box.
  3. Apply the mitigation if patching is not instant. WAF rule, network ACL, disable the feature, or take the service offline. Document it.
  4. Patch canary first. One non-critical host or the staging environment. Verify the app still works.
  5. Roll to prod in rings. Ring 1: one prod host. Ring 2: 50%. Ring 3: everything. Watch error rates between rings.
  6. Verify the patch landed. Do not trust the exit code. Check the actual package version.
  7. Assess for compromise. If the 0day was exploited in the wild before the patch, assume breach until proven otherwise. Hunt for IOCs.
  8. Write it down. What happened, what you did, how long it took. This becomes next quarter's baseline update.

The canary step is the one people skip under pressure. Do not skip it. I have seen a "critical security patch" break a PHP extension and take down a client's checkout for two hours because someone pushed it to all hosts at once. If you want that canary ring to catch regressions automatically rather than relying on someone watching a dashboard, agentic QA agents are a good fit for post-patch smoke tests.

Verification commands

Never trust "upgraded successfully". Verify:

# Confirm a specific package version
dpkg -l | grep -E 'openssh-server|linux-image'

# Confirm the kernel actually running (post-reboot)
uname -r

# Confirm no held packages are blocking updates
apt-mark showhold

# Confirm the service restarted with the new binary
systemctl status ssh --no-pager

If uname -r does not match the installed kernel after a reboot, you have a bootloader problem, not a patching problem, and you need to know that before you tell the client it is fixed.

Step 4: Vulnerability scanning for client apps

Patching on a schedule is necessary but not sufficient. You need independent verification that nothing slipped through. That is where vulnerability scanning for hosted apps earns its keep.

My stack, in layers:

Layer Tool examples What it catches Cadence
OS packages apt, unattended-upgrades Missing security updates Daily
Host scan OpenSCAP, Lynis, Trivy (host) Config drift, missing patches, CIS gaps Weekly
Container images Trivy, Grype Vulnerable base images, stale deps Every build
App dependencies npm audit, Snyk, Dependabot Node/React package CVEs Every PR
External Authenticated network scan Exposed services, TLS issues Monthly

Two things I insist on:

  • Scan the running host, not just the repo. A green CI pipeline means nothing if the production box is three kernel versions behind.
  • Fail the build on criticals. A scan that only warns is a scan nobody reads. trivy image --exit-code 1 --severity CRITICAL in CI.

For a client-facing report, I collapse all of this into one table: host, critical count, high count, days since last patch, compliance status. Non-technical founders read that in ten seconds and understand it. The same principle of verifying what is actually deployed applies to your dependencies, see my CI checklist for AI-hallucinated dependencies for the app-layer version of this check.

Step 5: SBOMs, because "which version?" is a question you will be asked

An SBOM (software bill of materials) is a machine-readable list of every component in your app. When the next Log4Shell-style 0day hits, the difference between a two-hour response and a two-week response is whether you can answer "do we use this?" with a query instead of a Slack thread.

In a DevOps pipeline, generating one is cheap:

# Generate an SBOM for a container image with Syft
syft packages registry.example.com/client-app:1.4.2 -o spdx-json > sbom.json

# Scan that SBOM for known CVEs with Grype
grype sbom:sbom.json --fail-on high

# For a Node project, CycloneDX works well
npx @cyclonedx/cyclonedx-npm --output-file sbom.cdx.json

Store the SBOM as a build artifact, tagged with the image digest. Now when a CVE drops, you grep your SBOMs instead of guessing. This is the part of SBOM DevOps that actually pays off: fast, accurate blast-radius queries during an incident. The SPDX and CycloneDX specs are the two formats worth standardizing on.

Bangladesh, bandwidth, and the change-freeze reality

A note for teams hosting from Dhaka or Chattogram, because the SERP advice ignores it.

Mirror bandwidth and latency matter. Pulling a large kernel update across a slow international link during an incident is its own failure mode. I keep a local apt caching proxy (apt-cacher-ng) in front of the fleet, and I sync security indexes ahead of time. When a 0day drops, the packages are already local and the patch takes minutes, not an hour of waiting on a mirror.

The other regional reality is change freezes. Many clients here freeze changes for weeks around Eid or end-of-quarter. That is a business constraint, and the honest move is to write the exemption into the contract: security patches are never frozen. If a client will not accept that, put the residual risk in writing and move on. It is their call, but it is your name on the incident report. The same ownership mindset is what I argue for in the WordPress backup and data ownership playbook, the client owns the site, but you own the recovery story.

Common mistakes I keep seeing

  • Patching only when something breaks. That is reactive, not a baseline.
  • Auto-upgrading everything. Feature updates on prod are a great way to break a client app on a Friday.
  • No reboot window. Kernel patches sit pending for months because nobody scheduled the reboot.
  • Trusting exit codes. Verify versions with dpkg -l and uname -r.
  • Confusing patching with cleaning. If it was exploited, hunt for IOCs after you patch.
  • No inventory. You cannot protect hosts you forgot you own.

The baseline in one page

If you take nothing else from this, take the shape of it: inventory, SLA table, automated Ubuntu security patching, a canary-first emergency runbook, independent scanning, and SBOMs as build artifacts. That is a Linux patching strategy that survives contact with a real 0day, and the whole thing fits on a page you can hand a client.

The teams that handle 0days well are not the ones with the fanciest tooling. They are the ones who decided, in advance, how fast they would patch and how they would prove it. That decision is the baseline.

If you are hosting client apps and want a second pair of eyes on your patching and automation setup, I do this for a living. You can reach me through the contact page on this site.

Frequently asked questions

What is a Linux 0day patching baseline?

It is a written, enforced set of rules for how fast you patch, how you verify it, and what evidence you keep. Concretely, it has three parts: a severity-to-SLA table, automated Ubuntu security patching with a defined reboot window, and independent scanning plus SBOMs so you can prove what is actually running.

How fast should I patch a critical Linux 0day?

For an internet-facing host with an exploited remote vulnerability, 24 hours is the target I use, with mitigation (WAF rule, ACL, disabling the feature) applied immediately while the patch rolls out. Kernel fixes get livepatch within 24 hours and a scheduled reboot within 30 days. Anything slower and you are gambling.

Do I need Ubuntu Pro for Livepatch?

Not strictly, but if you host client apps on Ubuntu LTS and cannot reboot on demand, it is the cheapest insurance you will buy. It applies kernel security fixes without a reboot, which lets you meet a 24-hour kernel SLA without taking a production app down mid-day.

Is patching enough after a 0day was exploited in the wild?

No. Patching closes the door but does not remove an attacker who was already inside. If the vulnerability was being exploited before the fix shipped, treat it as a possible breach: hunt for indicators of compromise, check for persistence, and rotate credentials before you declare it resolved.

Share this post

Native Web Share is available on mobile, so that share sheet includes Instagram and any other app you have installed.

Let's Connect

Get In Touch

Let's build something premium together.

Contact info

Follow me
Chat on WhatsApp