Uncategorized

Inherited an untested codebase What QA advice is obsolete now

You inherited an untested codebase at the worst possible moment: AI-assisted changes, dependency churn, and feature flag sprawl have made “just add tests later” more expensive than it was two years ago. My position is blunt: a QA engineer should stop chasing coverage first and should instead control change paths, because unverified behavior is safer when it is deliberately fenced than when it is broadly “improved.”

Your first tests should preserve ugly behavior, not prove good design

The advice that aged badly is “raise coverage before touching legacy code,” because line coverage rises fastest around easy code while the defects in an inherited system usually hide in old payment rules, date handling, permissions, retries, and data migrations. Coverage is still useful, but only after you know which behavior the business already depends on.

Over the last 24 months, the QA problem changed because AI coding assistants made plausible-looking edits cheaper, Renovate and Dependabot made dependency changes more frequent, and teams became more willing to ship small diffs behind flags. That means your inherited system is no longer a frozen mess; it is a moving mess with more automated pressure on it.

I would start with characterization tests, because they catch accidental behavior changes even when nobody can explain why the old behavior exists. In Java, ApprovalTests 24.22.0 and jqwik 1.9.1 help capture outputs and edge cases; in Python, pytest 8.3 with Hypothesis 6.x gives you a fast way to pin weird rules; in JavaScript, Jest 29.7 or Vitest 2.x can do the same without forcing a browser into the loop.

An initial quota that I would tune after the first week is 20 characterization tests around the riskiest flows, because 20 is small enough to finish before stakeholders lose patience and large enough to reveal whether the system is deterministic. Do not call that a benchmark; call it a starting constraint for a codebase with no trust budget.

def legacy_total(lines, vip=False):
    subtotal = sum(lines)
    if vip and subtotal > 100:
        return round(subtotal * 0.9, 2)
    if subtotal < 50:
        return subtotal + 5
    return subtotal

def test_legacy_total_characterization():
    assert legacy_total([20, 10], False) == 35
    assert legacy_total([60, 50], True) == 99.0

That tiny pytest example is not “good design,” and that is the point: it freezes behavior before refactoring pressure distorts it. Mutation testing tools such as PIT 1.16 for JVM projects and StrykerJS 8.x for TypeScript are valuable after this step, because they tell you whether tests actually fail when logic changes; they are frustrating before this step, because they punish you for not understanding the old logic yet.

Software Development Trends, Case Studies and Insights names real shifts, but a QA engineer inheriting an untested system should treat trends as constraints on triage rather than as a shopping list. AI, platform engineering, and observability matter here only when they help you answer one question: which change can corrupt production before anyone notices?

Feature flags became risk controls, and that makes old flag advice dangerous

Feature flags used to be sold mainly as release decoupling, but in legacy QA work they are now closer to operational risk controls because they decide who can experience unproven behavior. The outdated advice is “flag every risky change,” because unmanaged flags create hidden combinations that no QA engineer can exhaustively test.

Add feature flags to legacy code without rewriting everything is useful as a migration tactic, but it becomes bad QA strategy when every unknown path is hidden behind a permanent boolean. A flag without an owner, expiry date, default state, and rollback instruction is not a safety mechanism; it is another branch in an already untested system.

The last two years changed the flag conversation because OpenFeature 1.14 made vendor-neutral flag evaluation more realistic, LaunchDarkly expanded governance and audit workflows, and Unleash 5.x matured as a self-hosted option. That means “just add an if statement” is now outdated advice, because the hard part is no longer the conditional; the hard part is proving which users, jobs, queues, and data writes are affected.

I would NOT wrap every legacy function in a feature flag, because that multiplies execution paths faster than your tests can model them. I would flag only changes that have a reversible runtime decision, because database schema rewrites, one-way migrations, and background data repair jobs need migration gates and backups rather than user-facing toggles.

Use 14 days as a tunable expiry for high-risk temporary flags, because two sprints is usually enough to validate behavior while still short enough to make cleanup visible. Use 1% as a tunable first-cohort rollout for user-facing changes, because a small blast radius gives QA and support a chance to inspect real failures before cached state, queues, and downstream systems amplify them.

There is a practical comparison worth making: LaunchDarkly wins when audit trails, role-based approval, SDK maturity, and non-engineer targeting matter, and its cost is vendor dependency plus commercial pricing that can grow with environments, seats, or contexts. Unleash wins when self-hosting, data residency, and GitOps-style control matter, and its cost is operational ownership of services, storage, upgrades, and availability. Neither wins by default, because a QA engineer inherits the failure modes of the flag platform as soon as production behavior depends on it.

For config, I would require flag metadata in code review: owner, creation date, intended removal date, default, metric, and rollback note. OpenFeature helps standardize evaluation, but it does not remove QA responsibility because a standardized wrong default is still a wrong default. If your team uses YAML, JSON, or Terraform to manage flags, validate that config in CI with JSON Schema 2020-12 or Open Policy Agent 0.67, because typo-driven rollouts are still rollouts.

Screen automation is now the expensive default, so test seams first

Browser automation improved, but that does not make it the right first move for an untested codebase. Playwright 1.49, Cypress 13.16, and Selenium 4.27 are better than the tools many legacy systems grew up with, yet end-to-end tests still fail for reasons unrelated to product behavior: timing, test data, third-party scripts, browser updates, and environment drift.

A documented Playwright 1.49 assertion default is 5,000 ms, and that number matters because inherited systems often pass locally while hiding race conditions behind generous waits. Raising timeouts can reduce red builds, but it also delays feedback, so I would treat timeout increases as defects with names rather than as harmless stabilization.

The seam-first approach wins because it gives QA leverage before the UI is stable. Contract tests with Pact 4.x catch provider-consumer drift; WireMock 3.9 simulates ugly dependencies; Testcontainers 1.20 lets you start PostgreSQL 16, Redis 7, or Kafka 3.8 in repeatable tests; and JaCoCo 0.8.12 or Istanbul/nyc can show whether risky modules are even touched. These tools are not a stack to adopt wholesale, because each one adds maintenance, but each can isolate behavior that browser tests would expose too late.

Measure your repository’s 7-day flake rate before deleting or rewriting tests, because a flaky test suite with a 6% local failure rate has a different problem from a suite with one deterministic failing scenario. A release-gate threshold I would tune carefully is 2% flake rate for quarantine review, because below that level teams can still investigate individual failures and above it they usually start ignoring CI.

The old QA advice “automate the critical user journeys first” is incomplete now, because critical journeys often cross the exact legacy seams you do not control. If login, billing, and account changes all depend on the same brittle authorization helper, three browser tests give less protection than one focused test around that helper plus one smoke journey through the UI.

SonarQube 10.6, CodeQL 2.19, and Semgrep 1.x are useful here, but only as routing signals because static findings in legacy systems can drown a QA engineer in old problems that nobody plans to fix. I would configure new-code rules more aggressively than old-code rules, because stopping fresh risk is realistic while clearing a decade of warnings during onboarding is usually theater.

Production telemetry is evidence only after it has an owner and a rollback path

Another piece of outdated advice is “production monitoring will tell us if the change is bad,” because monitoring without an owner merely records the accident. The recent shift is that telemetry is easier to standardize: OpenTelemetry 1.30, Prometheus 2.55, Grafana 11, Sentry, Datadog, Honeycomb, and Elastic APM can connect traces, logs, metrics, and releases with less custom glue than older stacks required.

That does not mean telemetry replaces pre-release testing, because production evidence arrives after users or jobs have already touched the changed path. It does mean QA should define observable acceptance criteria for flagged changes: error rate, latency, event count, queue depth, retry volume, and rollback trigger. DORA metrics such as change failure rate and mean time to restore are useful at team level, but they are too coarse to prove that one legacy branch is safe.

For a risky legacy change, require an observed 24 hours of telemetry before expanding rollout beyond the first cohort, because daytime traffic, scheduled jobs, and cache expiry can expose different failure modes. For a batch process, use completed job count and dead-letter queue growth instead of page views, because no browser session will reveal a silent data repair failure.

I would sample the last 30 deploys to establish a local change failure baseline, because your inherited codebase has its own risk profile and vendor maturity reports will not tell you how often your rollback script actually works. If the last 30 deploys show that failures cluster around configuration, then QA should test config loading and defaults before adding another UI suite.

OpenAPI 3.1 contracts and AsyncAPI 3.0 specifications now deserve more QA attention than they used to, because many legacy systems are being wrapped by new services rather than replaced. A stale API contract is dangerous because it lets tests pass against a fantasy interface, so contract verification belongs in CI with the same seriousness as unit tests.

The disagreeable position is that QA should sometimes slow a “safe” flagged release, because reversible code can still write irreversible data. A feature flag can turn off a new button, but it cannot automatically undo duplicated invoices, malformed permissions, or corrupted search indexes. For those paths, I would demand backups, replay tests, idempotency checks, and a dry run, because rollback without data recovery is only a user-interface illusion.

Start tomorrow by choosing one risky production path and writing three artifacts for it: a characterization test, a flag or migration decision with an expiry, and one telemetry query that would prove harm quickly. Do not open a coverage dashboard first; open the code path that scares the team, because inherited QA earns trust by reducing the next release’s blast radius.