<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en-US"><generator uri="https://jekyllrb.com/" version="3.10.0">Jekyll</generator><link href="https://kennethjbrandt.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://kennethjbrandt.com/" rel="alternate" type="text/html" hreflang="en-US" /><updated>2026-06-19T22:06:53+00:00</updated><id>https://kennethjbrandt.com/feed.xml</id><title type="html">Kenneth Brandt</title><subtitle>Building systems people can actually run across quality, governance, technology, and execution.</subtitle><author><name>Kenneth Brandt</name></author><entry><title type="html">Freedom, Delayed: Juneteenth, Isaac Woodard, and the Voice That Would Not Let It Be Buried</title><link href="https://kennethjbrandt.com/essays/2026/06/19/freedom-delayed-juneteenth-isaac-woodard-and-the-voice-that-would-not-let-it-be-buried.html" rel="alternate" type="text/html" title="Freedom, Delayed: Juneteenth, Isaac Woodard, and the Voice That Would Not Let It Be Buried" /><published>2026-06-19T00:00:00+00:00</published><updated>2026-06-19T00:00:00+00:00</updated><id>https://kennethjbrandt.com/essays/2026/06/19/freedom-delayed-juneteenth-isaac-woodard-and-the-voice-that-would-not-let-it-be-buried</id><content type="html" xml:base="https://kennethjbrandt.com/essays/2026/06/19/freedom-delayed-juneteenth-isaac-woodard-and-the-voice-that-would-not-let-it-be-buried.html"><![CDATA[<p>Juneteenth is often described as a celebration of freedom.</p>

<p>It is that.</p>

<p>But it is also something more difficult: a reminder that freedom can be declared before it is delivered.</p>

<p>That distinction matters. A proclamation is not the same thing as lived reality. A law is not the same thing as enforcement. A principle is not the same thing as protection. Between the announcement and the human being stands the system: the messenger, the court, the sheriff, the army, the town, the bus driver, the badge, the newspaper, the microphone, the memory.</p>

<p>And sometimes the system arrives late.</p>

<p>Sometimes it refuses to arrive at all.</p>

<p>That is why I have been thinking about Isaac Woodard.</p>

<p>Woodard was a Black World War II veteran. He served his country, came home in uniform, and was beaten and permanently blinded by police in South Carolina in 1946. He had survived war abroad and returned to a country still willing to deny him the basic dignity of citizenship at home.</p>

<p>That sentence should not become easy to read.</p>

<p>It should remain heavy.</p>

<p>Orson Welles understood that.</p>

<p>When Welles heard about Woodard’s beating, he used his radio program not merely to comment, but to witness. There is a difference. Commentary explains. Witness refuses to let the fact disappear.</p>

<p>Welles did not hide behind balance where balance would have been cowardice. He did not soften the matter into respectable distance. He spoke directly, morally, and publicly. He took a local atrocity and made it harder for the country to bury it.</p>

<p>The Radio Diaries series, <em>Orson Welles and the Blind Soldier</em>, tells this story with the care it deserves. It is the piece I would point people to first.</p>

<p>Welles’s words still cut through the years.</p>

<blockquote>
  <p>“The blind soldier fought for me.”</p>
</blockquote>

<blockquote>
  <p>“I have eyes. He hasn’t.”</p>
</blockquote>

<blockquote>
  <p>“Officer X, I’m talking to you.”</p>
</blockquote>

<blockquote>
  <p>“Wash your hands, Officer X.”</p>
</blockquote>

<p>There is no mistaking the purpose of that voice. It is not ornamental. It is not theatrical in the empty sense. It is theatrical in the ancient sense: a voice raised before the public so that the public must decide what sort of people they mean to be.</p>

<p>That is the part that stays with me.</p>

<p>Welles had a microphone. Woodard had been robbed of his sight. So Welles used what he had.</p>

<p>That is not a small thing.</p>

<p>A society often reveals itself by what it allows to remain local. Local silence is one of the oldest hiding places for national shame. A man is beaten. A town looks away. An officer is unnamed. A file is mishandled. A story becomes rumor. Time passes. Respectable people move on.</p>

<p>Welles refused to move on.</p>

<p>This is why Woodard belongs in a Juneteenth reflection.</p>

<p>Juneteenth is not only about the end of slavery finally being announced in Texas. It is about the distance between national promise and human delivery. It is about the terrible fact that freedom can be real in one document and absent in one life. It is about the machinery required to make a principle true.</p>

<p>Isaac Woodard’s story comes later, after emancipation, after Reconstruction, after service in uniform, after a war fought against fascism. And still the question remained: would the country’s declared principles reach the man standing at the bus stop, the jailhouse, the courtroom, the hospital bed?</p>

<p>In Woodard’s case, the answer was no.</p>

<p>That is why memory matters.</p>

<p>Memory is not nostalgia. It is not decoration. It is not a wreath placed politely on history so that everyone may return to lunch undisturbed. Memory is a form of accountability. It is how the present admits that the past is not finished merely because it is past.</p>

<p>I first encountered part of that idea not as abstraction, but on a wall.</p>

<p>When I worked at the Jewish Community Center of San Francisco, one of the values on the wall was <strong>Tzedek - Justice</strong>. Beneath it was a line from Emma Lazarus:</p>

<blockquote>
  <p>Until we are all free, we are none of us free.</p>
</blockquote>

<p>That sentence has stayed with me.</p>

<p>It belongs in this reflection because it refuses the cheap version of freedom, the version that treats liberty as a private possession rather than a shared condition. It says freedom is not complete while another person remains excluded from its protection. It says justice is not a decorative virtue. It is a public obligation.</p>

<p>That is the spirit I hear in Juneteenth.</p>

<p>It is also the spirit I hear in Welles’s broadcasts about Isaac Woodard.</p>

<p>Woodard’s blinding was not only an attack on one man. It was an attack on the meaning of citizenship itself. If a decorated veteran could return from war, still in the shadow of his service, and be beaten blind by the machinery of local authority, then the country had not merely failed him. It had exposed the distance between its language and its delivery.</p>

<p>Until we are all free, we are none of us free.</p>

<p>That is not sentiment. It is systems thinking with a moral spine.</p>

<p>I also love that Welles’s voice traveled forward.</p>

<p>Logic sampled Welles’s commentary on <em>No Pressure</em>, including in “Obediently Yours.” That matters to me. Not because a sample fixes history. It does not. But because it proves that witness can move. A radio voice from 1946 can find its way into hip-hop decades later, and someone who never went looking for Isaac Woodard may suddenly hear Welles speaking across time.</p>

<p>That is how memory survives. Not always through monuments. Sometimes through archives. Sometimes through a podcast. Sometimes through a record. Sometimes through a voice that refuses, even now, to become quiet.</p>

<p>There is a lesson here for institutions, too.</p>

<p>Freedom is not self-executing.</p>

<p>Neither is citizenship. Neither is justice. Neither is equality before the law.</p>

<p>A country may declare its principles in magnificent language and still fail to build systems worthy of them. The gap between the two is where people are harmed. It is where Isaac Woodard was harmed. It is where countless others were harmed. It is where the flattering story a nation tells about itself meets the lived experience of those forced to test whether the story is true.</p>

<p>Juneteenth asks us to remember delayed freedom.</p>

<p>Isaac Woodard asks us to remember denied citizenship.</p>

<p>Orson Welles asks us to remember the duty of witness.</p>

<p>Emma Lazarus asks us to remember that freedom is not complete while others remain excluded from its protection.</p>

<p>And Logic, in his own way, reminds us that witness can travel farther than the moment that first required it.</p>

<p>So this Juneteenth, I am thinking less about celebration as a finish line and more about delivery as a moral obligation.</p>

<p>Freedom announced is not enough.</p>

<p>Freedom must arrive.</p>

<p>And when it does not, someone has to say so clearly enough that the silence cannot survive.</p>

<h2 id="sources-and-listening">Sources and Listening</h2>

<ul>
  <li>
    <p><a href="https://www.radiodiaries.org/post/orson-welles-and-the-blind-soldier">Radio Diaries — <em>Orson Welles and the Blind Soldier</em></a>
A three-part series on Isaac Woodard, Orson Welles’s radio campaign, and the case’s role in the broader civil rights struggle.</p>
  </li>
  <li>
    <p><a href="https://orsonwelles.indiana.edu/items/show/2169">Indiana University — <em>Orson Welles on the Air, 1938–1946</em>: <em>Orson Welles Commentaries</em></a>
Indiana University’s archive entry for the July 28, 1946 episode in which Welles speaks about Isaac Woodard, Jr., who was beaten by police in South Carolina and blinded as a result.</p>
  </li>
  <li>
    <p><a href="https://archive.org/details/1946OrsonWellesCommentaries">Internet Archive — <em>1946 Orson Welles Commentaries</em></a>
A downloadable collection of Welles’s 1946 commentaries, including broadcasts related to Isaac Woodard, the NAACP, and postwar civil rights.</p>
  </li>
  <li>
    <p><a href="https://www.jccsf.org/about/">JCCSF — About</a>
Background on the Jewish Community Center of San Francisco and its Sheva Middot, including <strong>Tzedek - Justice</strong> and the Emma Lazarus line, “Until we are all free, we are none of us free.”</p>
  </li>
  <li>
    <p><a href="https://www.nopressurecredits.com/">Logic — <em>No Pressure</em> Credits</a>
Official album credits noting the use of excerpts from “Orson Welles Commentary (Born To Be Free)” performed by Orson Welles.</p>
  </li>
  <li>
    <p><a href="https://wellesnet.com/logic-orson-welles/">Wellesnet — Logic samples Orson Welles on “Obediently Yours”</a>
Background on Logic’s use of Welles commentary on <em>No Pressure</em>, including the connection to Welles’s anti-racist broadcasts and the Isaac Woodard case.</p>
  </li>
</ul>]]></content><author><name>Kenneth Brandt</name></author><category term="essays" /><category term="juneteenth" /><category term="history" /><category term="civil-rights" /><category term="isaac-woodard" /><category term="orson-welles" /><category term="memory" /><category term="citizenship" /><category term="witness" /><category term="justice" /><summary type="html"><![CDATA[Juneteenth reminds us that freedom is not real when it is merely announced. Isaac Woodard reminds us that citizenship is not real when the systems charged with protecting it refuse to deliver it.]]></summary></entry><entry><title type="html">Canonical, Snap, and the Infrastructure We Forgot to Question</title><link href="https://kennethjbrandt.com/essays/2026/05/06/canonical-snap-and-the-infrastructure-we-forgot-to-question.html" rel="alternate" type="text/html" title="Canonical, Snap, and the Infrastructure We Forgot to Question" /><published>2026-05-06T00:00:00+00:00</published><updated>2026-05-06T00:00:00+00:00</updated><id>https://kennethjbrandt.com/essays/2026/05/06/canonical-snap-and-the-infrastructure-we-forgot-to-question</id><content type="html" xml:base="https://kennethjbrandt.com/essays/2026/05/06/canonical-snap-and-the-infrastructure-we-forgot-to-question.html"><![CDATA[<p>There are few things more clarifying than watching the operating system you trust become difficult to reach because somebody, somewhere, has decided to turn infrastructure into a geopolitical piñata.</p>

<p>Canonical, the company behind Ubuntu, recently acknowledged that its web infrastructure was under what it called a “sustained, cross-border attack.” Reports described the event as a Distributed Denial of Service attack affecting Ubuntu and Canonical services, with a group known as 313 Team, also called the Islamic Cyber Resistance in Iraq, claiming responsibility. As with most things involving attribution, Telegram channels, packet floods, and people with dramatic names, that claim should be handled with care. Claiming responsibility is not the same thing as proving responsibility.</p>

<p>Still, the incident matters.</p>

<p>It matters because Ubuntu is not some obscure hobby project hiding in the back room of computing history. Ubuntu is a major Linux distribution used by developers, enterprises, cloud providers, hobbyists, schools, labs, and people like me who want their machines to work without first negotiating with the spirits of seventeen forgotten dependencies.</p>

<p>I have used Ubuntu for years. I understand the appeal. It is practical, well-documented, widely supported, and familiar enough that most problems have already been suffered by someone else in public, which is one of the quiet blessings of civilization.</p>

<p>But this incident made something visible.</p>

<p>Not because it proved Ubuntu is bad. It did not. Not because it proved Snap caused the outage. It did not. Not because it proved Canonical is malicious. It did not.</p>

<p>It made visible the assumptions underneath the system.</p>

<p>The Marine Corps has a saying about assumptions that is not fit for a corporate slide deck, which is one reason it is useful. The polite version is this: assumptions are where plans go to die.</p>

<p>That is what this incident exposed for me. What had I assumed about Ubuntu’s resilience? What had I assumed about centralized stores? What had I assumed about open source infrastructure simply because the source code was open?</p>

<p>The answer, uncomfortably, is that I had assumed more than I had inspected.</p>

<h2 id="cross-border-attack-and-the-language-of-seriousness">“Cross-Border Attack” and the Language of Seriousness</h2>

<p>Canonical’s initial public language described the event as a “sustained, cross-border attack.” Later reporting quoted Canonical confirming a sustained, cross-border DDoS attack.</p>

<p>That phrase is not necessarily false. A DDoS is often cross-border in the practical sense. Compromised machines, botnets, proxies, rented infrastructure, and hostile traffic do not pause politely at customs checkpoints. Packets are famously indifferent to passports.</p>

<p>But “cross-border attack” is one of those phrases that sounds more clarifying than it is. It gestures toward seriousness without giving users much operational clarity.</p>

<p>What users needed was not a geopolitical adjective. They needed plain information:
What services are affected?
Are package repositories affected?
Are ISO images safe?
Are security advisories reachable?
Are Snap services degraded?
Are APT mirrors functioning?
What should administrators do if normal update paths fail?</p>

<p>To Canonical’s credit, it later stated that mitigations had been implemented and services affected by the DDoS attack had been restored, though some services might remain partially degraded. That is useful. But the earlier phrase still matters because crisis communication is part of infrastructure trust.</p>

<p>In an incident, clarity is not decoration. It is a control.</p>

<p>A good incident statement does not need to satisfy everyone’s curiosity. It does need to tell users what is known, what is not known, what is affected, what is not affected, and what they should do next. Otherwise, people fill the gaps themselves. The internet, being a generous institution, will fill them with panic, speculation, memes, and four men in a forum explaining why this would not have happened on Gentoo.</p>

<h2 id="availability-is-security">Availability Is Security</h2>

<p>It is tempting to think of DDoS attacks as crude. In many ways, they are. A DDoS is not a subtle jewel thief descending through a skylight. It is more like a mob blocking every road into town while shouting.</p>

<p>But crude does not mean harmless.</p>

<p>Availability is part of security. That is especially true when the affected infrastructure supports operating system downloads, authentication, developer services, package access, security guidance, or update workflows.</p>

<p>If users cannot reliably reach the services they need during a security event, the system is weaker than its design documents suggest. If administrators cannot easily find guidance, confirm integrity, or route around failure, then the outage becomes more than inconvenience. It becomes friction at precisely the moment friction is most expensive.</p>

<p>That is why this was more than a bad day for a website.</p>

<p>It was a stress test.</p>

<p>And systems reveal themselves under stress.</p>

<h2 id="the-copy-fail-timing-problem">The Copy Fail Timing Problem</h2>

<p>There is another wrinkle in the story, and it is precisely the sort of wrinkle that makes infrastructure people age in dog years.</p>

<p>Around the same time Canonical was dealing with the DDoS, the Linux ecosystem was also responding to Copy Fail, CVE-2026-31431, a high-severity local privilege escalation vulnerability in the Linux kernel. Canonical described it as affecting Ubuntu releases before 26.04, with mitigations released through <code class="language-plaintext highlighter-rouge">kmod</code> and kernel fixes following through normal update channels.</p>

<p>Some people speculated that the DDoS could be related to Copy Fail. Tom’s Hardware noted that speculation, while also calling the premise “a little shaky.” That caution is warranted.</p>

<p>This does not prove Copy Fail caused the DDoS.
It does not prove the attackers were trying to prevent patching.
It does not prove Ubuntu packages were compromised.
It does not prove that anyone should run screaming into the woods holding a Debian netinst ISO like a sacred object.</p>

<p>Those would be dramatic claims, and dramatic claims are best handled with tongs.</p>

<p>But the timing matters anyway.</p>

<p>Copy Fail is not remotely exploitable by itself. An attacker needs some sort of local foothold first: an account, a workload, a CI job, a container context, or another way to execute code on the system. Once that foothold exists, however, the vulnerability becomes a path to root. In plain English, it can turn a bad day into a much worse one.</p>

<p>That makes availability part of the security story.</p>

<p>During a vulnerability response, users need clear advisories, reachable update infrastructure, working mirrors, automation that does not fail mysteriously, and communication that tells them what is affected and what is not. If the front door is being pelted with packets while the building is also trying to distribute fire extinguishers, the fire extinguishers may still exist. That is good. But one is entitled to ask why everyone was expected to find them through a doorway currently occupied by vandals.</p>

<p>The DDoS did not need to be about Copy Fail for Copy Fail to matter.</p>

<h2 id="the-mirrors-held">The Mirrors Held</h2>

<p>This is where the APT mirror network deserves credit.</p>

<p>The detail that keeps tugging at me is not only what failed. It is what did not.</p>

<p>During the packet storm, the older APT mirror model appears to have done what distributed infrastructure is supposed to do. Not perfectly, not romantically, and certainly not with the sort of glossy product language that makes executives feel they have purchased the future in convenient quarterly installments. But it held well enough to remind us why distributed systems exist.</p>

<p>APT mirrors are not magic. They are ordinary infrastructure, maintained by people, universities, providers, institutions, and mirror operators who accept the unglamorous responsibility of keeping packages available. Ubuntu’s own documentation describes archive mirrors, release mirrors, country mirrors, Launchpad registration, sync requirements, and regular update expectations. Archive mirrors are expected to update every six hours. Release mirrors update every four hours or use push mirroring.</p>

<p>This is not glamorous architecture.</p>

<p>It is better than glamorous.</p>

<p>It is legible. It is boring. It is distributed. It is the sort of thing that survives trouble precisely because no single door has to remain open for the whole town to eat.</p>

<p>There is a quiet genius in boring infrastructure. Mirrors, sync jobs, package indexes, signatures, fallback hosts, and the ability to change sources are not the stuff of keynote demos. They are not sleek. They are not especially charismatic. But when the weather turns hostile, charisma is a poor substitute for redundancy.</p>

<p>The old mirrors behaved like infrastructure.</p>

<p>That matters.</p>

<h2 id="the-store-becomes-the-chokepoint">The Store Becomes the Chokepoint</h2>

<p>This is where Snap comes back into the conversation.</p>

<p>The criticism of Snap has often been treated as ordinary Linux tribalism, and to be fair, Linux tribalism is one of the world’s most renewable energy sources. But beneath the shouting there has always been a serious architecture question.</p>

<p>A software distribution model is not only a packaging choice. It is a trust model. It is an availability model. It is a governance model.</p>

<p>Who controls the store?
Who reviews submissions?
Who decides what is verified?
Who operates the infrastructure?
What happens when that infrastructure is unavailable?
What alternatives exist for users, enterprises, or distributions that do not want their operating model tied so tightly to one company’s services?</p>

<p>Snapcraft’s own documentation describes the official Snap Store as hosted and curated by Canonical. That may be perfectly sensible for many use cases. Centralization can bring benefits: consistency, review processes, publisher identity, simplified distribution, automatic updates, and a more controlled user experience.</p>

<p>But convenience has a habit of putting on a nicer jacket and returning as dependency.</p>

<p>That is the uncomfortable comparison with APT mirrors.</p>

<p>APT’s older model assumes alternate paths. Snap’s public-store model assumes a central service. Those are different philosophies, not just different technologies.</p>

<p>The DDoS did not create that difference. It merely removed some of the polite fog around it.</p>

<h2 id="open-source-is-not-automatically-resilient">Open Source Is Not Automatically Resilient</h2>

<p>One of the lazier arguments in technology is that open source is automatically resilient because the code is open.</p>

<p>I like open source. I prefer it whenever practical. I think it is one of the great achievements of modern technical culture. It gives users, developers, researchers, and institutions an ability to inspect, modify, preserve, and improve software in ways closed ecosystems often resist.</p>

<p>But openness of source code is not the same thing as resilience of the operating model.</p>

<p>A system can be open and still centralized.
It can be transparent in one layer and opaque in another.
It can publish code while concentrating distribution through a small number of chokepoints.
It can call itself community-oriented while quietly making the community dependent on infrastructure it does not control.</p>

<p>The question is not only, “Can I read the code?”</p>

<p>The question is also:
Can I get the software?
Can I verify it?
Can I update it?
Can I route around failure?
Can I mirror what matters?
Can the system degrade gracefully when the world becomes impolite?</p>

<p>Because the world will become impolite. It has a schedule.</p>

<h2 id="open-source-in-a-political-world">Open Source in a Political World</h2>

<p>There is another piece here, and it is easy to miss if one thinks of Linux distributions as mere technical artifacts.</p>

<p>Open source does not live in a monastery.</p>

<p>It lives in the world. And the world contains botnets, proxies, criminal markets, state-aligned actors, hacktivists, bored teenagers, grievance politics, sanctions, wars, extortion attempts, and people who believe civilization is best improved by sending junk traffic at package infrastructure.</p>

<p>Ubuntu, Debian, Fedora, Arch, Alpine, Red Hat, SUSE, and the rest are not just operating systems in the abstract. They are parts of the modern technical commons. They support cloud workloads, developer machines, CI/CD systems, laboratories, classrooms, factories, home servers, small businesses, and governments. They are infrastructure in the broad civic sense.</p>

<p>That means they are now part of the geopolitical weather.</p>

<p>A DDoS against Canonical is not just a nuisance. It is a reminder that open-source infrastructure is also strategic infrastructure, and strategic infrastructure attracts strategic stupidity.</p>

<p>Sometimes the stupidity wears a uniform. Sometimes it wears an ideology. Sometimes it wears a Discord handle and sells DDoS-as-a-service to people whose moral development peaked somewhere between “found a booter panel” and “learned to spell operational security incorrectly.”</p>

<p>But the effect is the same. Systems we rely on are placed under pressure, and their hidden assumptions become visible.</p>

<h2 id="why-this-pushes-me-toward-fedora-or-debian">Why This Pushes Me Toward Fedora or Debian</h2>

<p>This incident pushes me, at least emotionally, a little farther from Ubuntu and a little closer to Fedora or Debian.</p>

<p>Not because Fedora or Debian are magic kingdoms where packets arrive on horseback bearing constitutional rights. They have their own tradeoffs, politics, rough edges, governance disputes, maintenance burdens, and software choices capable of ruining a Saturday.</p>

<p>But they feel different in the areas that now matter more to me.</p>

<p>Debian appeals to the part of me that wants infrastructure to be boring in the way a good bridge is boring. It is conservative, widely mirrored, deeply rooted in the free software tradition, and less inclined to make the user feel gradually enrolled into a corporate product strategy.</p>

<p>Fedora appeals to the part of me that wants modern Linux, strong engineering, upstream alignment, and a desktop experience that does not revolve around Canonical’s preferred packaging future. Fedora is not neutral ground. It has Red Hat’s shadow over it, and anyone pretending otherwise is selling something. But the tradeoffs feel different.</p>

<p>Ubuntu still has strengths. It remains useful. It remains familiar. It remains one of the easiest distributions to recommend to many people. I am not throwing it into the sea.</p>

<p>But trust is not only built by being useful.</p>

<p>Trust is built by making the user comfortable with the tradeoffs.</p>

<p>And this incident made me less comfortable.</p>

<h2 id="the-infrastructure-we-forgot-to-question">The Infrastructure We Forgot to Question</h2>

<p>The lesson here is not that Ubuntu is doomed, or that Snap is evil, or that everyone should reinstall their laptops before dinner.</p>

<p>The lesson is simpler and more durable: open-source resilience is not guaranteed by open code alone. It depends on the design of the whole system around the code.</p>

<p>It depends on distribution.
It depends on governance.
It depends on mirrors.
It depends on update paths.
It depends on incident communication.
It depends on whether users and administrators have alternate routes when the main road is blocked by a political parade of garbage packets.</p>

<p>A system can be open and still centralized. It can be convenient and still brittle. It can be widely trusted and still depend on infrastructure whose failure reveals assumptions nobody wanted to inspect too closely.</p>

<p>That is the part worth taking seriously.</p>

<p>Canonical’s DDoS did not prove that Ubuntu is broken. It did not prove that Snap is the end of civilization, though I recognize that some corners of the Linux internet will continue their liturgical dance on that subject with or without evidence.</p>

<p>What it did prove is more useful.</p>

<p>It proved that architecture has a memory. The older APT mirror model remembered a world where distribution needed redundancy. The newer centralized services remembered a world where convenience, identity, review, and control could be concentrated in a store.</p>

<p>Both worlds have lessons.</p>

<p>But when the weather turned political, the old mirrors looked wiser than they had the day before.</p>

<p>And that is enough to make a man reconsider his assumptions.</p>

<h2 id="sources">Sources</h2>]]></content><author><name>Kenneth Brandt</name></author><category term="essays" /><category term="ubuntu" /><category term="canonical" /><category term="snap" /><category term="linux" /><category term="cybersecurity" /><category term="infrastructure" /><category term="open-source" /><category term="resilience" /><category term="governance" /><summary type="html"><![CDATA[A DDoS against Canonical did not just interrupt services. It exposed an uncomfortable question about Ubuntu’s direction: how much of an open-source operating system should depend on centralized infrastructure?]]></summary></entry><entry><title type="html">Nortel and the Cost of Failing to Act on What You Already Know</title><link href="https://kennethjbrandt.com/essays/2026/03/30/nortel-and-the-cost-of-failing-to-act-on-what-you-already-know.html" rel="alternate" type="text/html" title="Nortel and the Cost of Failing to Act on What You Already Know" /><published>2026-03-30T00:00:00+00:00</published><updated>2026-03-30T00:00:00+00:00</updated><id>https://kennethjbrandt.com/essays/2026/03/30/nortel-and-the-cost-of-failing-to-act-on-what-you-already-know</id><content type="html" xml:base="https://kennethjbrandt.com/essays/2026/03/30/nortel-and-the-cost-of-failing-to-act-on-what-you-already-know.html"><![CDATA[<p>There are, broadly speaking, two ways for a serious institution to die.</p>

<p>The first is by misfortune. A market turns, a technology shifts, a rival gets there first, and the splendid machine discovers that history has no respect for strategic plans.</p>

<p>The second is more humiliating. It is to receive warning, possess intelligence, hold meetings, circulate memos, nod gravely, and then continue on as though knowledge itself were a form of action. This second method is especially popular among large organizations, partly because it feels responsible right up to the moment it becomes fatal.</p>

<p>Nortel remains interesting because it appears to have suffered from both.</p>

<p>For a time, Nortel was not merely successful. It was the sort of company a nation points to when it wants proof that its future has arrived. It had scale, laboratories, engineers, patents, fiber optics, ambition, and the confident air of an institution that had begun to suspect the future might arrive wearing its logo. It was, in other words, exactly the sort of enterprise one imagines would know how to protect what it had built.</p>

<p>This is where the story acquires its sting.</p>

<p>The familiar account of Nortel’s fall includes the usual cast of modern corporate tragedy: the telecom bubble, acquisitions, strategic overreach, accounting scandal, job cuts, deteriorating confidence, and the long dreary procession by which an institution discovers that market capitalization is not the same thing as immortality. All of that matters. One should be suspicious of any story that explains a collapse of that scale with a single villain and a dramatic soundtrack.</p>

<p>But there is another thread in the Nortel story that lingers in the mind because it is so painfully modern. Warnings reportedly surfaced about intrusions into executive accounts and access to sensitive internal material. The important fact is not merely that someone may have been in the walls. The important fact is what happened after that became known.</p>

<p>Or rather, what did not.</p>

<p>That is the part worth dwelling on, because it reveals something larger than a breach. It reveals the difference between information and consequence.</p>

<p>Modern organizations are astonishingly good at gathering signals. They monitor, detect, classify, flag, escalate, review, and distribute. They produce dashboards with little colors on them, which is always comforting. They generate alerts with the crisp procedural dignity of a ship’s bell in a well-run disaster. They are, in many cases, rich in information. What they are not always rich in is the willingness to let that information disrupt hierarchy, convenience, optimism, or quarterly rhythm.</p>

<p>And so the signal arrives, and instead of becoming action, it enters the digestive tract of the institution.</p>

<p>There it is processed. It is discussed, contextualized, translated into language suitable for people who dislike hearing from specialists, and finally rendered into something so balanced, so mature, and so operationally respectful that whatever urgency it once possessed has been carefully removed for everyone’s comfort. By the end of this process, what began as danger has become governance, and governance, alas, is often the method by which danger is taught to sit quietly in the corner until it grows teeth.</p>

<p>This is not only a cybersecurity problem. It is an organizational problem. It appears in quality systems, in compliance functions, in safety programs, in vendor oversight, in military planning, in public administration, and in every other arena where human beings confuse the existence of a process with the existence of a capability.</p>

<p>A real capability changes events.</p>

<p>That is the test.</p>

<p>A control is not real because it exists in a binder, or a slide deck, or a beautifully formatted policy document written in the sorrowful dialect of corporate reassurance. A control is real if it changes behavior, assigns ownership, creates thresholds, triggers decisions, and produces action before the matter graduates into catastrophe.</p>

<p>Otherwise, it is not a control. It is a decorative belief.</p>

<p>Institutions are full of decorative beliefs. They believe that because a risk has been identified, it has been addressed. They believe that because a responsibility has been written down, it has an owner. They believe that because everyone in the room agrees something is important, somebody else will deal with it. They believe that awareness is adjacent to action, that reporting is adjacent to remedy, that seriousness of tone is adjacent to seriousness of response. These are charming beliefs. They are also how very intelligent organizations end up holding autopsies for futures they once assumed were secure.</p>

<p>This, to me, is the enduring lesson in Nortel.</p>

<p>The company did not become a cautionary tale simply because risk existed. Risk is the admission price for doing anything worthwhile. Nor did it become a cautionary tale merely because hostile actors may have found their way into places they did not belong. In a world of states, spies, rivals, and opportunists, that is not shocking. What remains shocking, because it remains common, is the possibility that an institution can know something serious and still lack the internal machinery required to make that knowledge operationally decisive.</p>

<p>That is a systems failure.</p>

<p>It is also a human one.</p>

<p>Large organizations do not generally fail for lack of intelligence. They fail because intelligence collides with vanity, inconvenience, diffusion of responsibility, and the ancient managerial hope that a problem noticed today might somehow become a lesser problem tomorrow if everyone behaves calmly enough around it. Human beings are brilliant at this sort of evasion. We can convert warning into process, process into language, language into delay, and delay into an atmosphere of mature prudence. It is one of our more polished skills.</p>

<p>Unfortunately, the world does not always reward polish.</p>

<p>There is a certain kind of executive mind that treats security, quality, governance, and operational discipline as supporting functions, regrettably necessary but faintly separate from the real business of strategy. This is a mistake. In a serious institution, those things are strategy. The protection of critical knowledge is strategy. The ability to act on credible warning is strategy. The existence of clear decision rights before a crisis, rather than improvised authority during one, is strategy. An organization that cannot convert knowledge into action does not have an information problem. It has a command problem.</p>

<p>That distinction matters.</p>

<p>We like to flatter ourselves that collapse arrives with trumpets. Usually it arrives administratively. It is minuted, deferred, reviewed next quarter, and assigned to a workstream. By the time it becomes dramatic, the important opportunities have already been missed in rooms full of competent people saying reasonable things.</p>

<p>Which brings us back to Nortel.</p>

<p>The final lesson is not that innovation failed. Nortel had innovation. Nor is it that talent failed. It had talent in abundance. The darker possibility is that talent and innovation existed inside an institution whose operating system was not equal to the risks attached to its own importance. It may have known more than it could effectively act upon. That is a dangerous condition for any organization, and a terminal one for an organization whose value lies in what it knows.</p>

<p>There is something almost classical about that. Tragedy, after all, is not ignorance meeting fate. More often it is knowledge meeting character.</p>

<p>The warning was there, or near enough. The larger question was whether the institution had built a structure capable of hearing bad news in time to do something expensive, disruptive, and necessary about it. That is the sort of question every modern organization ought to ask itself, preferably before the answer is supplied by events.</p>

<p>Because the great danger is not always the threat you failed to imagine.</p>

<p>Sometimes it is the threat you recognized, discussed, documented, and politely declined to operationalize until the bill arrived.</p>

<p>That, in the end, is the more unnerving story. Not that the storm existed, but that the ship may have received reports about the weather and chosen, with all due professionalism, to continue arranging the deck furniture.</p>]]></content><author><name>Kenneth Brandt</name></author><category term="essays" /><category term="cybersecurity" /><category term="governance" /><category term="systems" /><category term="operational-resilience" /><category term="leadership" /><category term="risk" /><summary type="html"><![CDATA[Nortel is not only a story about cyber intrusion. It is a story about what happens when an institution can detect danger, discuss danger, and still fail to build the machinery required to act on what it already knows.]]></summary></entry><entry><title type="html">IoT Botnets, DDoS, and the Device Governance Problem</title><link href="https://kennethjbrandt.com/2026/03/19/iot-botnets-ddos-device-governance.html" rel="alternate" type="text/html" title="IoT Botnets, DDoS, and the Device Governance Problem" /><published>2026-03-19T00:00:00+00:00</published><updated>2026-03-19T00:00:00+00:00</updated><id>https://kennethjbrandt.com/2026/03/19/iot-botnets-ddos-device-governance</id><content type="html" xml:base="https://kennethjbrandt.com/2026/03/19/iot-botnets-ddos-device-governance.html"><![CDATA[<p>The recent takedown of four large IoT botnets is a useful reminder of an old truth: a machine does not care whose side it is on. It will obey whoever holds the leash.</p>

<p>Authorities in the United States, Canada, and Germany disrupted infrastructure tied to four botnets called Aisuru, Kimwolf, JackSkid, and Mossad. The official story says those botnets compromised more than three million connected devices and were used to drive DDoS attacks of extraordinary scale.</p>

<p>That sounds like a cyber story. It is not. Not really.</p>

<p>It is a governance story.</p>

<p>No sorcery was required. No death ray from Mars. The machinery was already here, plugged in, trusted, and humming along in offices, homes, field sites, and closets no one has opened in months. Routers. Cameras. Boxes with blinking lights. Cheap, connected, and forgotten. The attackers did not invent an army. They conscripted one.</p>

<p>That is the part worth your attention.</p>

<h2 id="what-matters-here">What matters here</h2>

<p>The interesting question is not how many devices were taken over. The interesting question is how many devices in your own environment could be.</p>

<h3 id="1-do-you-know-which-devices-can-become-someone-elses-infrastructure">1. Do you know which devices can become someone else’s infrastructure?</h3>

<p>Most organizations think in categories that make them comfortable. Servers matter. Laptops matter. The little appliance in the corner does not matter until it does.</p>

<p>But if a device is connected, reachable, weakly managed, and rarely patched, then it is not a harmless convenience. It is a latent liability. It may sit quietly for months, doing its small assigned duty, until someone else decides it has a more interesting purpose.</p>

<p>A machine with no owner is still owned. Just not by you.</p>

<h3 id="2-are-iot-and-edge-devices-governed-like-real-production-assets">2. Are IoT and edge devices governed like real production assets?</h3>

<p>There is a common superstition in operations: if something is small, cheap, or boring, it must also be low-risk. That is nonsense.</p>

<p>A camera does not become safe because it is boring. A router does not become harmless because nobody talks about it in steering committee meetings. An Android box does not become trustworthy because Finance never heard of it.</p>

<p>If the device is connected, it needs the same old unglamorous disciplines that keep everything else from becoming a fire: ownership, lifecycle control, patching, visibility, retirement. If those are absent, then you are not running a system. You are running a wish.</p>

<p>Luck is not a control.</p>

<h3 id="3-can-you-tell-when-small-devices-start-behaving-like-attack-infrastructure">3. Can you tell when “small devices” start behaving like attack infrastructure?</h3>

<p>The official account says these botnets launched hundreds of thousands of attacks and caused real financial harm. That means the signal was there. The question is whether anyone who was responsible could see it in time, and whether they knew what they were looking at when they did.</p>

<p>For most teams, that comes down to a few unromantic questions:</p>

<ul>
  <li>Can you see internet-exposed devices clearly?</li>
  <li>Do you know what normal outbound behavior looks like?</li>
  <li>Will anyone notice when one of these things starts speaking in a foreign tongue at machine speed?</li>
  <li>Is there a named human being who owns the problem when that happens?</li>
</ul>

<p>Most failures of this kind are not failures of intelligence. They are failures of housekeeping.</p>

<h3 id="4-is-ddos-resilience-treated-as-a-business-continuity-issue">4. Is DDoS resilience treated as a business-continuity issue?</h3>

<p>Another common mistake is to hand DDoS off to the network team and call it a day, as though a flood of malicious traffic were only a technical inconvenience.</p>

<p>It is not. It is a continuity problem.</p>

<p>When systems are overwhelmed, the real questions are not abstract:</p>

<ul>
  <li>Which services must remain reachable?</li>
  <li>Which dependencies break first?</li>
  <li>What can be degraded safely?</li>
  <li>Who talks to customers, partners, and internal teams when normal channels are under strain?</li>
</ul>

<p>At sufficient scale, DDoS stops being a nuisance and starts becoming a test of whether leadership understands its own business.</p>

<h2 id="the-practical-takeaway">The practical takeaway</h2>

<p>The recent disruption of these botnets is good news. It is also temporary news.</p>

<p>The underlying condition remains the same: the world is full of insecure, weakly governed connected devices, and adversaries have shown again and again that they are happy to put them to work.</p>

<p>So the useful question is not whether the government disrupted this batch.</p>

<p>The useful question is this:</p>

<p><strong>Which connected devices in your environment are important enough to inventory, patch, monitor, and retire deliberately, and which ones are still running on neglect and optimism?</strong></p>

<p>If the answer is vague, then the danger is not hypothetical. It is merely waiting its turn.</p>

<h2 id="a-few-useful-checks">A few useful checks</h2>

<ul>
  <li>Identify every internet-facing and remotely manageable IoT or edge device</li>
  <li>Assign a real owner to each device class</li>
  <li>Confirm patch responsibility and firmware hygiene</li>
  <li>Remove default credentials and weak remote access paths</li>
  <li>Baseline outbound behavior for non-traditional devices</li>
  <li>Alert on unusual traffic, scanning, or command patterns</li>
  <li>Tie DDoS response to business priorities, not just technical procedures</li>
  <li>Hold third-party and field-deployed devices to the same standards as internal ones</li>
</ul>

<p>The larger point is simple enough for a child, which is one reason adults tend to miss it:</p>

<p><strong>A device fleet becomes a security problem long before it becomes a headline.</strong></p>]]></content><author><name>Kenneth Brandt</name></author><summary type="html"><![CDATA[The recent takedown of four large IoT botnets is a useful reminder of an old truth: a machine does not care whose side it is on. It will obey whoever holds the leash.]]></summary></entry><entry><title type="html">Stryker and the Security Boundary Hidden in Device Management</title><link href="https://kennethjbrandt.com/2026/03/18/stryker-device-governance-operational-risk.html" rel="alternate" type="text/html" title="Stryker and the Security Boundary Hidden in Device Management" /><published>2026-03-18T00:00:00+00:00</published><updated>2026-03-18T00:00:00+00:00</updated><id>https://kennethjbrandt.com/2026/03/18/stryker-device-governance-operational-risk</id><content type="html" xml:base="https://kennethjbrandt.com/2026/03/18/stryker-device-governance-operational-risk.html"><![CDATA[<p>Stryker’s recent cyberattack is a reminder that endpoint management is not just an IT utility. It is a security boundary.</p>

<p>Public reporting indicates the incident disrupted Stryker’s Microsoft environment and may have involved remote wiping of managed Windows devices after attackers gained privileged access. Even without a classic ransomware pattern, the operational impact was severe.</p>

<p>That is the part leaders should pay attention to.</p>

<p>If an attacker can abuse identity, endpoint management, or administrative tooling, the blast radius can be just as serious as a traditional malware event. The lesson is not only cyber. It is governance, operations, and continuity.</p>

<h2 id="what-matters-here">What matters here</h2>

<p>A few practical questions matter more than the headlines:</p>

<h3 id="1-is-device-management-treated-like-a-critical-control-plane">1. Is device management treated like a critical control plane?</h3>

<p>If an attacker gained privileged access to your identity and endpoint-management stack, could they push destructive actions across the fleet?</p>

<p>If the answer is unclear, that is already a problem.</p>

<h3 id="2-are-destructive-actions-tightly-governed">2. Are destructive actions tightly governed?</h3>

<p>Remote wipe, reset, unenroll, and policy enforcement all have legitimate uses.</p>

<p>The question is whether those capabilities are protected by:</p>
<ul>
  <li>strong approval paths</li>
  <li>segmentation</li>
  <li>break-glass controls</li>
  <li>alerting for unusual use</li>
</ul>

<p>A capability that is safe in normal operations can still become an enterprise-wide risk if privilege and oversight fail at the same time.</p>

<h3 id="3-can-the-organization-detect-mass-destructive-actions-fast-enough">3. Can the organization detect mass destructive actions fast enough?</h3>

<p>A mature environment should be able to detect unusual wipe, reset, unenroll, or large-scale policy events quickly.</p>

<p>It should also be clear:</p>
<ul>
  <li>who gets alerted</li>
  <li>who can respond</li>
  <li>how the action gets interrupted</li>
  <li>how the organization verifies whether the event is legitimate or malicious</li>
</ul>

<h3 id="4-does-continuity-planning-assume-simultaneous-loss-of-endpoints-and-collaboration-systems">4. Does continuity planning assume simultaneous loss of endpoints and collaboration systems?</h3>

<p>Many organizations plan for an application outage or a site outage.</p>

<p>Fewer plan for a scenario where:</p>
<ul>
  <li>managed endpoints are disrupted</li>
  <li>internal communications are degraded</li>
  <li>administrative access is limited</li>
  <li>teams cannot coordinate normally</li>
</ul>

<p>That gap matters.</p>

<h2 id="the-practical-takeaway">The practical takeaway</h2>

<p>Endpoint management, identity, and administrative privilege are operational-risk issues, not just technical issues.</p>

<p>A simple internal stress test is this:</p>

<p><strong>Could an attacker with privileged access use your endpoint-management and identity systems to push destructive actions across your fleet before anyone stopped it?</strong></p>

<p>If the answer is unclear, the next step is not more discussion. It is control review.</p>

<h2 id="a-few-useful-checks">A few useful checks</h2>

<ul>
  <li>Identify which administrative systems can affect the largest number of endpoints at once</li>
  <li>Review what approvals are required for destructive or high-impact actions</li>
  <li>Confirm what alerts would fire if a mass wipe or reset were attempted</li>
  <li>Test how quickly the team could validate whether the event was malicious</li>
  <li>Review which business processes would fail first if endpoints and collaboration tools disappeared at the same time</li>
</ul>

<p>The broader point is simple:</p>

<p><strong>A control plane is still a control plane when an attacker is the one using it.</strong></p>]]></content><author><name>Kenneth Brandt</name></author><summary type="html"><![CDATA[Stryker’s recent cyberattack is a reminder that endpoint management is not just an IT utility. It is a security boundary.]]></summary></entry><entry><title type="html">When Cyber Incidents Break Patient-Critical Workflows: Lessons from the Stryker Disruption</title><link href="https://kennethjbrandt.com/insights/2026/03/11/stryker-cyber-disruption-gxp-workflows.html" rel="alternate" type="text/html" title="When Cyber Incidents Break Patient-Critical Workflows: Lessons from the Stryker Disruption" /><published>2026-03-11T00:00:00+00:00</published><updated>2026-03-11T00:00:00+00:00</updated><id>https://kennethjbrandt.com/insights/2026/03/11/stryker-cyber-disruption-gxp-workflows</id><content type="html" xml:base="https://kennethjbrandt.com/insights/2026/03/11/stryker-cyber-disruption-gxp-workflows.html"><![CDATA[<p>Cyber incidents are not “IT problems” when they interrupt patient-critical workflows.</p>

<p>The Stryker situation is a useful case study because it shows what happens when a widely used vendor ecosystem experiences a broad disruption. The technical event is only half the story. The other half is operational: how teams adapt, what workarounds emerge, and whether organizations can stay safe and compliant while systems are degraded.</p>

<p>This post is written for a mixed audience: quality leaders operating in regulated environments and cybersecurity professionals who want to understand how a cyber event becomes a safety and governance event.</p>

<h2 id="what-happened-facts-first">What happened (facts first)</h2>

<p>Stryker reported a cybersecurity incident that led to a global disruption impacting its Microsoft environment, and said it was working to restore systems and operations.</p>

<p>The company also filed an SEC 8-K describing the incident and the operational impact from the disruption.</p>

<p>Separately, reporting described how downstream healthcare workflows were affected, including reliance on fallbacks when connectivity or integrations were unavailable.</p>

<blockquote>
  <p>Note: This post focuses on operational and governance lessons rather than attributing actors. Attribution can change as investigations evolve.</p>
</blockquote>

<h2 id="the-point-for-both-gxp-and-cyber-teams">The point for both GxP and cyber teams</h2>

<p>In regulated environments, “continuity” is not just uptime. It is the ability to keep decisions safe, traceable, and governed while operating in degraded mode.</p>

<p>When a system goes down and teams fall back to manual steps, three things tend to happen quickly:</p>

<p>1) Controls weaken (informal workarounds become the workflow)<br />
2) Traceability degrades (decisions move to phone calls, chat, and memory)<br />
3) Risk increases (detection and escalation paths become unclear)</p>

<p>That is why cyber resilience is a quality capability, not just a security capability.</p>

<h2 id="what-quality-leaders-should-take-seriously">What quality leaders should take seriously</h2>

<h3 id="1-integrations-are-patient-safety-dependencies">1) Integrations are patient-safety dependencies</h3>

<p>Most “patient-critical” workflows are not a single system. They are a chain: device, connectivity, identity, routing, receiving system, storage, and retrieval. When any link breaks, teams substitute human memory and manual communication.</p>

<p>Your continuity plan should be written at the workflow level, not the application level.</p>

<h3 id="2-degraded-mode-needs-predefined-controls">2) Degraded mode needs predefined controls</h3>

<p>If you have no defined degraded mode, you will still operate. You will just operate without consistent controls.</p>

<p>A strong degraded-mode design answers:</p>

<ul>
  <li>What is allowed to continue?</li>
  <li>What must stop?</li>
  <li>Who can authorize exceptions?</li>
  <li>What is the minimum documentation required?</li>
  <li>How do we reconcile back into the system later?</li>
</ul>

<h3 id="3-cyber-incidents-can-become-data-integrity-incidents">3) Cyber incidents can become data integrity incidents</h3>

<p>Even when the initial impact looks like availability, degraded operations commonly create integrity issues:</p>

<ul>
  <li>delayed entry and backdating risk</li>
  <li>incomplete audit trails</li>
  <li>conflicting “source of truth” across systems</li>
  <li>missing reconciliation evidence after restoration</li>
</ul>

<p>Quality systems should anticipate this and include reconciliation steps as part of the incident playbook.</p>

<h2 id="a-practical-checklist-24-to-72-hour-survivability">A practical checklist (24 to 72 hour survivability)</h2>

<p>Use this as a fast self-assessment for any critical vendor platform or integration.</p>

<h3 id="workflow-continuity">Workflow continuity</h3>
<ul>
  <li>Do we have a documented way to operate safely for 24 to 72 hours if the system is offline?</li>
  <li>Are the fallback steps role-based and trained, or tribal knowledge?</li>
</ul>

<h3 id="ownership-and-escalation">Ownership and escalation</h3>
<ul>
  <li>Is there a named owner for the workflow end-to-end (not just the application owner)?</li>
  <li>Are severity tiers and stop rules defined ahead of time?</li>
</ul>

<h3 id="documentation-and-traceability">Documentation and traceability</h3>
<ul>
  <li>Do we have a minimal degraded-mode record template (who, what, when, why, and risk decision)?</li>
  <li>Do we have a reconciliation plan that is auditable (what gets reconciled, how, and who signs off)?</li>
</ul>

<h3 id="vendor-and-evidence-expectations">Vendor and evidence expectations</h3>
<ul>
  <li>Do vendor contracts define incident notification timing and required details?</li>
  <li>Do we have access to the logs and evidence we would need to defend decisions later?</li>
</ul>

<h3 id="testing-and-readiness">Testing and readiness</h3>
<ul>
  <li>Have we run a tabletop exercise on this workflow in the last 12 months?</li>
  <li>Have we tested the manual workaround, not just written it?</li>
</ul>

<p>If you cannot answer these quickly, you likely have a survivability gap.</p>

<h2 id="why-cyber-teams-should-care-about-the-gxp-angle">Why cyber teams should care about the GxP angle</h2>

<p>In healthcare and life sciences, downtime is not just lost productivity. It can alter clinical decisions, change what information is available, and introduce integrity and documentation risks that persist after systems return.</p>

<p>Cyber response is stronger when it includes quality and operational leadership early, because those teams know:</p>

<ul>
  <li>where the actual risk pathways are</li>
  <li>which workflows are safety-critical</li>
  <li>what evidence will be required later</li>
  <li>which controls must remain in place even during disruption</li>
</ul>

<p>Events like this are a reminder to treat vendor platforms and integrations as controlled systems with defined boundaries, fallback modes, and auditable recovery.</p>

<p>If you want a lightweight, inspection-ready baseline for continuity planning that connects cyber response to quality controls, I can share a starter template you can adapt.</p>

<p>Contact: contact@kennethjbrandt.com</p>

<h2 id="sources">Sources</h2>

<ul>
  <li>
    <p>KrebsOnSecurity: Iran-backed hackers claim wiper attack on medtech firm Stryker<br />
https://krebsonsecurity.com/2026/03/iran-backed-hackers-claim-wiper-attack-on-medtech-firm-stryker/</p>
  </li>
  <li>
    <p>Stryker: Message to customers (March 2026 update)<br />
https://www.stryker.com/us/en/about/news/2026/a-message-to-our-customers-03-2026.html</p>
  </li>
  <li>
    <p>U.S. SEC: Stryker 8-K filing (March 2026)<br />
https://www.sec.gov/Archives/edgar/data/310764/000119312526102460/d76279d8k.htm</p>
  </li>
  <li>
    <p>The Record (Recorded Future News): Stryker recovery timeline / incident reporting<br />
https://therecord.media/stryker-tells-sec-unknown-timeline-recovery</p>
  </li>
  <li>
    <p>TechCrunch: Stryker hack claim coverage<br />
https://techcrunch.com/2026/03/11/stryker-hack-pro-iran-hacktivist-group-handala-says-it-is-behind-attack/</p>
  </li>
</ul>]]></content><author><name>Kenneth Brandt</name></author><category term="insights" /><category term="GxP" /><category term="Cybersecurity" /><category term="Quality Systems" /><category term="Business Continuity" /><category term="Risk Management" /><category term="Vendor Management" /><category term="Data Integrity" /><category term="Clinical Operations" /><summary type="html"><![CDATA[Cyber incidents are not “IT problems” when they interrupt patient-critical workflows.]]></summary></entry><entry><title type="html">AI Agents and Distillation: What Quality Leaders Should Take Seriously</title><link href="https://kennethjbrandt.com/insights/2026/03/10/ai-agents-distillation-gxp-quality.html" rel="alternate" type="text/html" title="AI Agents and Distillation: What Quality Leaders Should Take Seriously" /><published>2026-03-10T00:00:00+00:00</published><updated>2026-03-10T00:00:00+00:00</updated><id>https://kennethjbrandt.com/insights/2026/03/10/ai-agents-distillation-gxp-quality</id><content type="html" xml:base="https://kennethjbrandt.com/insights/2026/03/10/ai-agents-distillation-gxp-quality.html"><![CDATA[<p>AI risk is not just about prompts anymore.</p>

<p>Two patterns are worth paying attention to if you operate in regulated environments.</p>

<h2 id="1-distillation-at-industrial-scale">1) Distillation at industrial scale</h2>
<p>Anthropic reported “distillation” campaigns where organizations created large volumes of Claude interactions through fraudulent accounts to extract model behavior for training.</p>

<p>In plain terms: if a model is accessible, someone will try to siphon capability at scale.</p>

<h2 id="2-ai-used-to-orchestrate-cyber-activity">2) AI used to orchestrate cyber activity</h2>
<p>Anthropic also published a report describing what it called the first reported AI-orchestrated cyber espionage campaign, and AP covered the same development.</p>

<p>Whether you are running clinical systems, manufacturing systems, or vendor platforms, cyber events can become quality events fast.</p>

<h2 id="why-this-matters-for-gxp-teams">Why this matters for GxP teams</h2>
<p>In GxP, your job is predictable outcomes under uncertainty. AI adds new uncertainty:</p>
<ul>
  <li>Hidden workflow drift</li>
  <li>Untracked “configuration” changes (prompts, retrieval sources, thresholds)</li>
  <li>Weak provenance for data and decisions</li>
  <li>Vendor tools that behave like black boxes</li>
</ul>

<p>The fix is not panic. It is control design.</p>

<h2 id="the-practical-move-treat-ai-like-a-controlled-system">The practical move: treat AI like a controlled system</h2>
<p>If you want inspection-ready AI operations, start with five basics:</p>

<p>1) Define intended use. Write down what the AI is allowed to do and what it is not allowed to do. If it touches a GxP decision, define the boundary and escalation path.<br />
2) Treat prompts and workflow logic as configuration. If people can quietly change prompts, retrieval sources, or thresholds, you have an untracked change. That is a control gap.<br />
3) Make audit trails answer inspection questions. Not just what happened, but who approved it, what evidence supported it, and what monitoring confirms it still works.<br />
4) Put vendors on the hook for evidence. If an AI feature is vendor-provided, insist on logging, retention, access control design, incident notification expectations, and rollback options.<br />
5) Pre-write incident response. Retrain vs rollback vs suppress should never be improvised. Your corrective action must match the failure mode, and the rationale must tie back to risk.</p>

<h2 id="a-quick-self-check-15-minutes">A quick self-check (15 minutes)</h2>
<ul>
  <li>Do I know every place AI is used in the workflow today?</li>
  <li>Do I have an owner for each AI-enabled capability?</li>
  <li>Can I show what changed, when, and why?</li>
  <li>Can I roll back quickly if outputs regress?</li>
  <li>Do I have a simple monitoring pack with thresholds, actions, and review cadence?</li>
  <li>Do vendors provide logs and evidence I can defend?</li>
</ul>

<p>If any of those are “no,” the fix is usually straightforward: scope, ownership, logging, change control, and response playbooks.</p>

<p>If you want a lightweight, inspection-ready baseline for AI governance in a GxP environment, I am happy to share a starter checklist.</p>

<p>Contact: contact@kennethjbrandt.com</p>

<p>Sources:</p>
<ul>
  <li>Anthropic: Detecting and preventing distillation attacks</li>
  <li>Anthropic: Disrupting the first reported AI-orchestrated cyber espionage campaign (full report)</li>
  <li>AP: Anthropic warns of AI-driven hacking campaign linked to China</li>
</ul>]]></content><author><name>Kenneth Brandt</name></author><category term="insights" /><category term="GxP" /><category term="Quality Systems" /><category term="Governance" /><category term="Data Integrity" /><category term="AI" /><category term="Cybersecurity" /><summary type="html"><![CDATA[AI risk is not just about prompts anymore.]]></summary></entry><entry><title type="html">Alert+Action Limits: A Starter Checklist for GxP Monitoring</title><link href="https://kennethjbrandt.com/gxp/ai/quality/2026/01/06/alert+action-limits-starter-checklist.html" rel="alternate" type="text/html" title="Alert+Action Limits: A Starter Checklist for GxP Monitoring" /><published>2026-01-06T00:00:00+00:00</published><updated>2026-01-06T00:00:00+00:00</updated><id>https://kennethjbrandt.com/gxp/ai/quality/2026/01/06/alert+action-limits-starter-checklist</id><content type="html" xml:base="https://kennethjbrandt.com/gxp/ai/quality/2026/01/06/alert+action-limits-starter-checklist.html"><![CDATA[<p>Alert limits and action limits are one of the fastest ways to make a monitoring program defendable. They also prevent the two failure modes auditors see all the time: dashboards with no decisions, and decisions with no thresholds.</p>

<h2 id="what-good-looks-like">What good looks like</h2>
<ul>
  <li>Alert limit = investigate signal and document assessment</li>
  <li>Action limit = execute a defined response (change control,CAPA,rollback,process controls)</li>
</ul>

<h2 id="starter-checklist">Starter checklist</h2>
<p>1) Define the baseline (validated reference)
2) Pick acceptance metrics (what “acceptable” means)
3) Separate signals (data drift,concept drift,performance drift)
4) Set limits by risk (patient impact drives tightness+SLA)
5) Assign owner+backup (named accountability)
6) Define response SLAs (review,escalation,closure)
7) Document decisions (review minutes matter)
8) Link actions to change control (threshold updates,retraining,rollback)</p>

<h2 id="visual-checklist">Visual checklist</h2>
<p><img src="/assets/img/checklists/alert-action-limits-a-starter-checklist-02.png" alt="Alert+Action limits starter checklist" /></p>

<p>If you want an editable template version,reach out via the contact page.</p>]]></content><author><name>Kenneth Brandt</name></author><category term="gxp" /><category term="ai" /><category term="quality" /><category term="Monitoring" /><category term="Alert Limits" /><category term="Action Limits" /><category term="CPV" /><category term="Validation" /><category term="Drift" /><summary type="html"><![CDATA[A practical checklist for setting alert+action limits that are risk-based, operational, and audit-friendly.]]></summary></entry><entry><title type="html">Audit Trails+Log Retention: A Starter Checklist for Inspection Readiness</title><link href="https://kennethjbrandt.com/gxp/quality/data-integrity/2026/01/06/audit-trails+log-retention-starter-checklist.html" rel="alternate" type="text/html" title="Audit Trails+Log Retention: A Starter Checklist for Inspection Readiness" /><published>2026-01-06T00:00:00+00:00</published><updated>2026-01-06T00:00:00+00:00</updated><id>https://kennethjbrandt.com/gxp/quality/data-integrity/2026/01/06/audit-trails+log-retention-starter-checklist</id><content type="html" xml:base="https://kennethjbrandt.com/gxp/quality/data-integrity/2026/01/06/audit-trails+log-retention-starter-checklist.html"><![CDATA[<p>Most inspection pain around audit trails is not about the audit trail feature. It’s about governance: what is captured,who reviews it,where evidence lives,and how long you can produce it on demand.</p>

<h2 id="starter-checklist">Starter checklist</h2>
<p>1) Define scope (systems and records that support GxP decisions)
2) Confirm event coverage (create,modify,delete,override,access)
3) Set review requirements (who reviews,how often,what triggers review)
4) Define retention (records+metadata+raw logs+reports)
5) Validate retrieval (can you produce it quickly,consistently,completely)
6) Ensure traceability (signal-&gt;review-&gt;decision-&gt;action)
7) Tie to SOPs (roles,escalation,investigation paths)</p>

<h2 id="visual-checklist">Visual checklist</h2>
<p><img src="/assets/img/checklists/audit-trails-log-retention-a-starter-checklist-02.png" alt="Audit trails and log retention starter checklist" /></p>

<p>A simple rule: if you can’t retrieve the audit trail record package quickly,you don’t have audit trails as a control. You have a hope.</p>]]></content><author><name>Kenneth Brandt</name></author><category term="gxp" /><category term="quality" /><category term="data-integrity" /><category term="Data Integrity" /><category term="Audit Trails" /><category term="Log Retention" /><category term="ALCOA+" /><category term="Inspection Readiness" /><summary type="html"><![CDATA[A practical checklist for audit trails and log retention that supports defensible decisions and faster inspections.]]></summary></entry><entry><title type="html">Control Strategy Without the Jargon: Control Layers That Survive an Audit</title><link href="https://kennethjbrandt.com/gxp/quality/qbd/2026/01/02/control-strategy-without-the-jargon.html" rel="alternate" type="text/html" title="Control Strategy Without the Jargon: Control Layers That Survive an Audit" /><published>2026-01-02T00:00:00+00:00</published><updated>2026-01-02T00:00:00+00:00</updated><id>https://kennethjbrandt.com/gxp/quality/qbd/2026/01/02/control-strategy-without-the-jargon</id><content type="html" xml:base="https://kennethjbrandt.com/gxp/quality/qbd/2026/01/02/control-strategy-without-the-jargon.html"><![CDATA[<p>Control strategies get overcomplicated fast. Teams drown in jargon, giant trace matrices, and “one more spreadsheet.”
But auditors (and operators) care about something much simpler:</p>

<p><strong>Can you show quickly and consistently what you control, who owns it, and where the evidence lives?</strong></p>

<p>This post lays out a practical approach that scales from development into commercial manufacturing.</p>

<h2 id="the-core-idea-in-one-line">The core idea (in one line)</h2>
<p><strong>CQA → CPP/CMA → control layers → owner/approvals → evidence</strong></p>

<p>If you can’t trace your strategy end-to-end, it won’t scale—and it won’t hold up under scrutiny.</p>

<hr />

<h2 id="step-1-start-with-cqas-what-must-be-true-for-the-patient">Step 1: Start with CQAs (what must be true for the patient)</h2>
<p>CQAs are the outcomes you’re protecting: identity, strength, purity, sterility assurance, content uniformity, etc.</p>

<p><strong>Practical rule:</strong> write each CQA in plain language:</p>
<ul>
  <li>What does “good” look like?</li>
  <li>What would “bad” mean for patient impact?</li>
</ul>

<p>If you can’t explain a CQA without acronyms, the map will turn into paperwork instead of control.</p>

<hr />

<h2 id="step-2-link-cppscmas-what-drives-those-cqas">Step 2: Link CPPs/CMAs (what drives those CQAs)</h2>
<p>CPPs and CMAs are the levers that meaningfully move CQAs.</p>

<p><strong>Keep it honest:</strong></p>
<ul>
  <li>If it doesn’t move a CQA, it’s not a CPP/CMA—it’s noise.</li>
  <li>If it moves a CQA, it needs a control layer you can defend.</li>
</ul>

<hr />

<h2 id="step-3-define-control-layers-controls-exist-in-layers-not-one-test">Step 3: Define control layers (controls exist in layers, not one test)</h2>
<p>Most failures come from “single-point control thinking”:</p>
<blockquote>
  <p>“We test it at the end, so we’re fine.”</p>
</blockquote>

<p>That doesn’t scale. Robust control strategies use <strong>layers</strong>, for example:</p>
<ul>
  <li><strong>Material controls</strong> (supplier qualification, incoming acceptance)</li>
  <li><strong>Process controls</strong> (setpoints, alarms, in-process checks)</li>
  <li><strong>Procedural controls</strong> (SOPs, training, line clearance)</li>
  <li><strong>Analytical controls</strong> (IPC testing, release testing)</li>
  <li><strong>System controls</strong> (access, audit trails, data integrity)</li>
</ul>

<p><strong>Goal:</strong> multiple independent ways to prevent/detect a bad outcome.</p>

<hr />

<h2 id="step-4-assign-owners--approvals-raci-that-isnt-theater">Step 4: Assign owners + approvals (RACI that isn’t theater)</h2>
<p>A control strategy fails when ownership is ambiguous.</p>

<p>At minimum, capture:</p>
<ul>
  <li><strong>Owner:</strong> accountable for control performance &amp; upkeep</li>
  <li><strong>Approvers:</strong> who approves changes (QA, MSAT, Validation, etc.)</li>
  <li><strong>Escalation path:</strong> who gets called when limits are breached</li>
</ul>

<p>If an auditor asks “who owns this control?” your answer should be immediate.</p>

<hr />

<h2 id="step-5-link-evidence-where-it-lives-how-to-retrieve-it">Step 5: Link evidence (where it lives, how to retrieve it)</h2>
<p>Controls don’t exist unless you can produce evidence.</p>

<p>For each control layer, link:</p>
<ul>
  <li>The <strong>record type</strong> (batch record section, log, system report)</li>
  <li>The <strong>system of record</strong> (eQMS, MES, LIMS, historian, DCS, etc.)</li>
  <li>The <strong>retention expectation</strong> (as applicable)</li>
  <li>The <strong>retrieval path</strong> (who pulls it, how long it takes)</li>
</ul>

<p><strong>Fast test:</strong> can your team retrieve a representative record in under 10 minutes?</p>

<hr />

<h2 id="step-6-review-cadence--triggers-change-control-is-your-friend">Step 6: Review cadence + triggers (change control is your friend)</h2>
<p>A control strategy should be reviewed on purpose, not “when someone remembers.”</p>

<p>Use:</p>
<ul>
  <li><strong>Periodic review</strong> (quarterly/ annually depending on risk)</li>
  <li><strong>Triggered review</strong> at change control events:
    <ul>
      <li>material/supplier change</li>
      <li>parameter range change</li>
      <li>equipment/software change</li>
      <li>deviation trend/complaint trend</li>
      <li>method change</li>
    </ul>
  </li>
</ul>

<p>This is how you keep the strategy aligned to the validated state.</p>

<hr />

<h2 id="common-failure-modes-what-breaks-first">Common failure modes (what breaks first)</h2>
<p>In practice, traceability usually breaks at:
1) <strong>Handoffs</strong> (development → tech transfer → commercial)
2) <strong>Ownership gaps</strong> (everyone is involved, nobody is accountable)
3) <strong>Evidence sprawl</strong> (records scattered across systems)
4) <strong>“One test” thinking</strong> (weak layering)
5) <strong>Unmanaged change</strong> (strategy doesn’t update when reality changes)</p>

<hr />

<h2 id="a-simple-audit-question-to-pre-answer">A simple audit question to pre-answer</h2>
<p><strong>“Show me how this CQA is controlled, and prove the controls are working.”</strong></p>

<p>If you can answer that with:</p>
<ul>
  <li>the map</li>
  <li>named owners</li>
  <li>linked evidence</li>
  <li>review history</li>
</ul>

<p>…you’re in a strong position.</p>

<hr />

<h2 id="question-for-you">Question for you</h2>
<p>Where does your traceability usually break first, <strong>process controls, test strategy, or documentation handoffs</strong>?</p>]]></content><author><name>Kenneth Brandt</name></author><category term="gxp" /><category term="quality" /><category term="qbd" /><category term="QbD" /><category term="Control Strategy" /><category term="CQA" /><category term="CPP" /><category term="CMA" /><category term="GMP" /><category term="Data Integrity" /><summary type="html"><![CDATA[A practical way to map CQAs to CPPs/CMAs, define layered controls, assign ownership, and link evidence so the strategy scales—and stands up in audits.]]></summary></entry></feed>