How to parse multi-language LOC CSV exports to isolate code smells in legacy files

The Misconception: Sorting by LOC Is Not Code Smell Detection

The first time I audited a 2.4-million-line polyglot estate for a migration readiness report, I did what most engineers do with a fresh LOC CSV export: I opened it in a spreadsheet, sorted descending by line count, and circled the top twenty files. Those, I told the steering committee, were our code smell hotspots. I was wrong, and the mistake cost us a sprint.

The assumption baked into that move is seductive and widespread — that large files are inherently smelly files. They often are. But often is not a metric, and conflating size with smell density is the single most common error I see in code metrics work today, especially on legacy systems where the export itself is a patchwork of multiple languages counted by multiple tools.

Here is what actually happened. The top of my sorted list was dominated by a 14,800-line COBOL copybook — a shared data definition file with almost zero executable logic. It was huge, yes, but it carried fewer smells than a 940-line PL/SQL package body further down the list that had cyclomatic complexity north of 280 and eleven distinct God-class symptoms. By trusting raw LOC as my lens, I buried the real risk surface under a pile of declarative boilerplate.

This article is the corrective. It walks through, step by step, how to parse a multi-language LOC CSV export — the kind you get when you run SonarQube on the Java tier, cloc on the scripts, and a mainframe-specific counter on the COBOL — and isolate the files where code smells actually live. The audience I am writing for is specific: the tech lead or architect performing a pre-migration code audit on a polyglot legacy estate, working from exported metrics rather than live tooling, because the live tooling cannot reach half the codebase.

Step 1 — Normalize the Export Before You Trust a Single Number

Every multi-language LOC export I have ever handled arrives broken in at least three quiet ways. The numbers look fine. The structure is not.

Strip the Header and Dialect Noise

SonarQube exports one schema. cloc exports another. Your mainframe LOC counter probably emits a fixed-width file someone renamed to .csv in 2009. Before you parse anything, you need a single normalized table with consistent columns: file_path, language, blank_lines, comment_lines, code_lines, tool_source.

The trap here is language labeling. In one audit, the same codebase contained files tagged as COBOL, Cobol, COBOL85, and Enterprise COBOL depending on which tool counted them. If you do not collapse those into a single canonical label, your downstream language buckets will fragment and your cross-language comparisons will silently undercount. I normalize every label to lowercase and strip version suffixes in a single pass before anything else happens.

Reconcile Path Roots

The Java metrics referenced paths like src/com/acme/.... The COBOL counter emitted PROD.COBOL.SRC(MODNAME) — a PDS member reference, not a file path at all. If your goal is to join LOC data with smell findings later, you need a join key. Pick one canonical path scheme early. I use a synthetic module_id column built from language + relative_path, and I document the mapping in a side file. This is unglamorous work, but it is the foundation everything else rests on.

Step 2 — Build a Smell-to-LOC Ratio, Not a Raw Count

This is the step that separates a useful triage from a sorted spreadsheet. Raw LOC tells you where the code is. It does not tell you where the smells are. For that, you need a density metric.

Choose Your Denominator Wisely

If you have smell counts per file (from SonarQube, for instance, or from a static analyzer that reached part of the estate), the ratio that matters is smells_per_kloc — code smells divided by thousands of lines of executable code. Note the word executable. Using total lines including comments and blanks dilutes the ratio and lets files with heavy documentation headers look cleaner than they are.

For the files your static analyzer could not reach — the COBOL, the PL/SQL, the VB6 — you will not have direct smell counts. This is where most auditors give up and fall back to raw LOC. Do not. Instead, build a proxy density from the structural signals you do have: cyclomatic complexity where available, nesting depth, function length, and the ratio of comment lines to code lines (a sudden drop in comment density on an old file is a strong smell indicator in legacy code).

A Worked Example from the 2.4M-Line Estate

On the migration audit I mentioned, the Java tier had full SonarQube coverage. The PL/SQL tier had complexity metrics from a database analyzer. The COBOL tier had nothing but raw LOC and a file-age stamp.

For the Java files, I computed smells_per_kloc directly. The median was 12.4. The 90th percentile was 41.7. One file — a 940-line billing calculation class — sat at 78.2 smells per KLOC, nearly double the 90th percentile. That file was 38th on the raw LOC sort. It was first on the density sort. That reordering changed the migration plan.

For the PL/SQL tier, I used complexity_per_kloc as a proxy. The same billing logic, mirrored in a PL/SQL package, scored 312 complexity per KLOC against a tier median of 47. Same business logic, same risk, different language, different tool — but the normalized ratio made them comparable.

Step 3 — Cross-Reference Language Buckets to Find Polyglot Hotspots

Legacy estates do not have code smells in one language at a time. They have them in clusters that span language boundaries, because the same business logic is often implemented in two or three languages — a Java service layer calling a PL/SQL procedure that mirrors a COBOL program — and the smells propagate across all three.

Why Single-Language Views Lie

If you parse your CSV export one language at a time, you will find the worst Java file, the worst PL/SQL file, and the worst COBOL file. You will miss the worst cluster: the set of files across three languages that all implement the same decrepit business rule and must be migrated together. These clusters are where migration risk concentrates, and they are invisible unless you cross-reference.

Joining CSV Slices with a Simple Script

The technique is straightforward once your paths are normalized. Load your CSV into a dataframe — pandas, polars, or even a well-structured set of dictionary comprehensions if you are keeping it minimal. Group by your synthetic module_id or by a business-domain tag if you have one. Then compute a cluster-level density: the sum of smells across all files in the cluster, divided by the sum of executable LOC across the same files.

On the estate I was auditing, this surfaced a five-file cluster — two Java classes, two PL/SQL packages, one COBOL program — totaling 6,200 executable lines and carrying a combined 311 code smells. That is 50.2 smells per KLOC at the cluster level, worse than any single file in the Java tier alone. No single-language view would have found it. The migration team re-sequenced their work to tackle that cluster first, as a unit, rather than migrating the Java layer and leaving the PL/SQL and COBOL mirrors to drift further.

Step 4 — Filter for Legacy File Signatures

Density tells you where smells are concentrated. It does not tell you which of those files are actually legacy in the sense that matters for migration planning — files that are old, lightly touched, and owned by too few people. A file with high smell density but active maintenance and a healthy author count is a different risk than a high-density file last modified in 2011 by someone who retired three years ago.

Age, Author Count, and Change Frequency

If your CSV export includes any of these signals — and most do, if you merged in git history or change logs — layer them as filters on top of your density ranking. I use a simple three-factor legacy score: file age in years, inverse author count, and inverse commit frequency over the trailing twelve months. Each factor is normalized to a 0–1 scale and averaged.

A file with a legacy score above 0.7 and a smell density above the 75th percentile is a priority-one migration target. A file with the same density but a legacy score below 0.3 is a refactor-in-place candidate. The distinction matters because migration budgets are finite, and treating every smelly file as a migration target burns through that budget on files that could be fixed where they stand.

Tagging the Real Risk Surface

On the 2.4M-line estate, applying this filter reduced the priority migration target list from 187 files — the raw high-density set — to 52 files. Those 52 files represented 11% of the executable LOC but carried an estimated 63% of the code smell mass and 71% of the cluster-level complexity. That is the number I brought to the steering committee. Not 14,800 lines of copybook. Not a sorted spreadsheet. A focused, defensible risk surface that mapped directly to migration sequencing.

Step 5 — Export a Triage List, Not a Dashboard

The final step is the one most metrics articles skip, because it is not about parsing at all. It is about what you hand to the people who have to act on your findings.

A dashboard is the wrong output for a migration audit. Dashboards invite browsing. A migration team does not need to browse. They need a triage list: a CSV or markdown table with file path, language, cluster ID, smell density, legacy score, and a single recommended action — migrate, refactor, or monitor. Sorted by cluster, not by file, so the team sees the work in the order they will actually do it.

I include a one-line rationale per file, generated from the signals that drove its classification. "High density (78.2/KLOC), low author count (2), last modified 2019-Q3, part of billing cluster #4." That sentence does more for a migration lead than any chart I could render.

What This Approach Buys You — and What It Does Not

Parsing a multi-language LOC CSV this way buys you three things a raw sort cannot. It buys comparability across languages that no single tool covers. It buys a density-aware view that separates declarative bulk from executable risk. And it buys a cluster-level lens that matches how legacy systems actually fail — in cross-language business-rule knots, not in isolated files.

What it does not buy you is precision. Proxy density metrics are estimates. Cluster joins depend on clean path normalization, which depends on tools that were never designed to talk to each other. The legacy score is a heuristic, not a measurement. Treat the output as a prioritization instrument, not a verdict. The goal is to point a finite migration team at the right 52 files out of 2.4 million lines. If the triage list gets them there within a sprint of the optimal sequence, the parsing work has done its job.

The lesson I carry from that first audit — the one where I sorted by LOC and circled the top twenty — is simple and I repeat it to every tech lead who inherits a metrics export: size is where the code sits. Density is where the code rots. Parse accordingly.