What breaks when you replace an AS/400 while the plant keeps running

The system is thirty years old, it has never lost a day of production, and the two people who understand it are within a few years of retirement. Every manufacturer we meet with an IBM midrange system in the plant, and the platform is IBM i now whatever the sign on the box said in 1996, is describing some version of that sentence, and it is usually delivered as a technical problem. It is not. The code is legible, the database is relational, and the business rules are all in there somewhere. The problem is that the plant cannot stop.

That constraint is what separates this from a normal modernization. Public-sector experience with the same class of system is instructive here: GAO’s work on federal legacy systems finds the same pattern of aging critical systems, unsupported components and staff scarcity that a manufacturer recognizes immediately. What GAO cannot capture is the specific thing that makes a shop-floor replacement harder than an office one, which is that the consequence of a failed cutover is a line that is not making product.

What is actually in there¶

Before anything is estimated, someone has to inventory what the system does rather than what it stores. In our experience the inventory is always larger than the team’s mental model, and it contains five categories.

• Interactive programs behind the green screens, which hold the business rules that were never written down anywhere else.
• A batch chain, usually scheduled, usually with dependencies that exist only in the order a CL program submits them, and in one person’s memory.
• Physical file definitions and the logical files layered over them, which encode assumptions about keys and uniqueness that no modern schema will reproduce by accident.
• Device integrations at the edges: label printers, scales, scanners, a CNC handoff, a customer EDI feed. These are the ones that get discovered late, because they are not in any repository.
• Reports that leave as spool files and end up as a customer document, an auditor’s evidence or a regulatory submission. Somebody downstream depends on the exact layout.

Four things that break, that nobody budgets for¶

1. The new screen is slower for the person using it¶

A 5250 screen is a keyboard-driven state machine. An operator who has run it for a decade completes a transaction without looking at the display, using a fixed sequence of field entries and function keys, and can do it wearing gloves. A modern replacement that requires a mouse, a page transition and a confirmation dialog can easily double the time per transaction. Multiply by shift volume and the plant has been slowed down by a project sold as an improvement.

This is preventable, and prevention is unglamorous: stand at the terminal, count the keystrokes for the ten highest-volume transactions, and make the keystroke count a design requirement with a number attached. Keyboard-first entry, no focus traps, function-key equivalents, and no dialog between the operator and the next scan. Aesthetics that cost the plant three seconds per unit are not aesthetics, they are a cost.

2. The batch chain is a dependency graph nobody drew¶

The nightly sequence usually accumulated over decades. Job order matters, some jobs are safe to rerun and some are not, and the ones you find first are the ones that fail loudly. The dangerous ones fail quietly: a job that completes with a non-zero condition, or that silently processes zero records because an earlier step did not deliver a file, and whose absence surfaces four days later as an inventory discrepancy.

Instrument the existing chain before replacing any of it. Record start and end times, record counts, and the completion and escape messages each job actually produced, for a full month. That log is the specification, and it also gives you the only credible before-and-after comparison you will get.

3. The data types carry meaning your new schema drops¶

Packed decimal fields, six-digit dates, zero and blank used as distinct sentinel values, alphanumeric fields with positional meaning in the first two characters, and a customer number that is right-justified and zero-padded in one file and not in another. Every one of these is a small correctness problem, and together they are the reason data migration is the phase that slips.

Treat migration as a reconciliation exercise rather than a copy. Move the data, then prove equivalence by running the same queries against both systems and comparing totals per period, per plant, per part. The differences are the specification for the next migration pass. A migration that has not been reconciled has not been tested, it has merely been executed.

4. The edges are physical¶

A label that prints two millimeters off is a customer receiving failure. A scale that returns weight in a fixed-width field with an implied decimal will silently be off by a factor of ten. These integrations are cheap to build and expensive to discover, and they cannot be tested from a laptop. Budget bench time with the actual hardware, on the actual floor, in the actual light.

The pattern that makes it survivable¶

Nothing here argues for a big-bang replacement, and a plant will not accept one anyway. The strangler fig approach, taking one process at a time and routing it to the new system while the old one continues, is the only shape we have seen work at this risk level. The difficulty is not the concept, it is choosing the seam.

Take reads before writes. A new reporting or visibility surface that reads from the existing data changes nothing operationally and delivers value in weeks, and it forces the team to learn the data model under real scrutiny. Writes come later, one process at a time, behind an anti-corruption layer that translates between the old vocabulary and the new one so the new system does not inherit thirty years of encoding decisions on day one.

That layer is temporary by design and permanent in practice, so give it an owner and a test suite. The version of this project that fails is the one where the new system quietly adopts the old field semantics to make an integration easier, because at that point you have paid for a rewrite and bought a copy.

Journaling is the asset nobody mentions¶

Running two systems side by side requires the new one to see changes as they happen, and the obvious approach, editing the existing programs to publish events, is the one that puts production at risk. It is usually unnecessary. Database journaling on the platform already records committed changes, and a reader over the journal gives a change-data-capture feed without touching a line of RPG.

That single decision is what makes the parallel period affordable. It lets the new system be fed continuously, compared against the old one on real traffic, and corrected before anything depends on it, all without a change window.

Cutover is a schedule problem¶

The last mile is where these projects are won or lost, and it is scheduled with the plant manager, not with the architect. Three rules keep it survivable, and all three are commitments to make before the first phase is designed rather than during the week it happens.

1. 1.Cut over per process, never per system. Receiving can move while production reporting stays. A single date on which everything changes is a date on which everything can fail.
2. 2.Every cutover is reversible until the next business day. If the answer to can we roll this back by first shift tomorrow is no, it is not ready, regardless of how the tests look.
3. 3.Cut over into a low-volume window with the old system still warm, and keep it warm longer than feels necessary. The cost of an extra month of parallel running is trivial against one shift of lost production.

The last thing worth saying is about ambition. The temptation on these projects is to fix the process while replacing the system, because everyone can see that the process is bad too. Resist it for the first phases. A replacement that changes behavior and technology at the same time cannot be debugged, because no one can tell whether a discrepancy is a defect or the new intent. Move the process first, improve it second, and you get two projects that can each be judged.