Why does JSON.parse silently drop duplicate keys?

RFC 8259 (the JSON spec) doesn't require parsers to reject duplicates — it leaves the behaviour up to the implementation. JavaScript's JSON.parse takes the permissive path: when it encounters two keys with the same name in one object, the later one overwrites the earlier one and no warning is emitted. The data is silently lost.

How does PDFFlare's tool find duplicates if JSON.parse can't?

The tool doesn't use JSON.parse — it would be useless since the parser is what drops the data. Instead, PDFFlare runs a raw-text scan, tracks every key seen inside each object scope, and reports any key that appears more than once at the same level. You get the line number of each duplicate so you can fix them in your source file.

Will the detector also catch case-sensitive 'duplicates'?

No. JSON keys are case-sensitive by spec — "foo" and "Foo" are different keys, not duplicates. If your data treats them as semantically equivalent (a common bug in mixed-case configs), you need to normalise case at the source. The detector follows the spec.

Do duplicate keys matter at every nesting level?

Yes. Duplicates can occur at any object scope — top-level, deeply nested, or inside an array element. PDFFlare's detector checks every object scope independently and reports the path so you know which nested object the duplicate lives in.

Is my JSON sent to a server when checking for duplicates?

No. The duplicate-keys tool runs entirely in your browser via a raw-text scan. Your JSON never uploads anywhere — safe for production configs, webhook payloads, customer data, or anything containing secrets.

May 1, 20267 min read

How to Find Duplicate Keys in JSON (Catch the Silent Data-Loss Bug)

A teammate hands you a JSON config from a third-party export. You run it through JSON.parse, ship it, and a week later someone notices a setting you definitely set is missing from the live app. The setting was there in the source file. So why is it gone? Because the file had duplicate keys with the same name and JSON.parse silently kept only the last one.

In this guide you'll learn how to find duplicate keys in JSON using PDFFlare's JSON Duplicate Keys detector — why JSON parsers silently drop duplicates, where this bug shows up in real systems, and how to fix or prevent it.

Why JSON.parse Silently Drops Duplicate Keys

The JSON spec (RFC 8259) is permissive on duplicates: implementations MAY accept them, MAY reject them, MAY warn. There's no required behaviour. JavaScript's JSON.parse chose the permissive path: take both keys, the later one wins, the earlier one is silently overwritten.

This is bad for two reasons:

Silent. No error. No warning. Your code runs normally with the wrong data.
Order-dependent. The bug behaviour depends on key order in the source file. Round-tripping through JSON.stringifychanges the ordering, which can flip which value “wins.”

The fix isn't to change JSON.parse(you can't). It's to detect duplicates upstream — before the parse drops half your data.

How PDFFlare's Detector Works

PDFFlare's tool doesn't use JSON.parse— that would be useless, since the parser is exactly what drops the data we're trying to detect. Instead, the tool runs a raw-text scan of the JSON, tracks every key it sees inside each object scope, and flags any key that appears more than once at the same level. You get the line number of each duplicate so you can fix it directly in your source.

How to Find Duplicate Keys in JSON (Step by Step)

Open PDFFlare's Duplicate Keys tool — no signup needed.
Paste your JSON into the input pane. Could be a config file, an API response, an export — anything you suspect might have duplicates.
Click Find Duplicates. The tool scans the raw text and lists every duplicate key with its line number, the value it had, and which object scope it appeared in.
Inspect the results.If duplicates are listed, decide what to do (merge, rename, delete) — see the next section. If “No duplicates found,” you can safely JSON.parse without data loss.

How to Fix Duplicate Keys

What to do depends on why the duplicate exists in the first place:

Real data conflict (two values for the same key): One of them is “canonical.” Decide which, delete the other. If both matter, restructure into an array (e.g. {"tags": ["a", "b"]} instead of two tags keys).
Manual edit gone wrong: Someone added a new key without realizing one already existed. Pick the correct value, delete the duplicate.
Generated by buggy code: Some serializer (especially in older or hand-rolled tools) emits duplicates. Fix the source, regenerate.
Merged from multiple sources: Two configs were naively concatenated. The fix is to merge properly with logic (e.g. last-wins explicitly, or rename one side).

Real-World Cases Where This Bites

Hand-Edited Configuration Files

A config file gets edited by multiple people over months. Someone adds a new timeout field at the top of the file, unaware that another timeout already exists 200 lines down. Both values are present in the source, only the bottom one ever takes effect. Until the file is reordered, at which point behaviour silently flips.

Log Aggregation

Some log pipelines emit one JSON object per line, with shared field names (e.g. request_id, user_id). When the pipeline merges multiple sources into a single document, duplicate keys can appear at the same level — and the merge step silently keeps only one.

Generated JSON From Templates

Template engines (Mustache, Handlebars, Jinja) producing JSON can emit duplicates if the same template variable is referenced twice. The output looks fine in the file, parses fine in JS, but produces mysterious data loss.

Webhook Payloads From Older APIs

Older or hand-rolled APIs occasionally emit duplicate keys (especially for arrays-as-multiple-keys patterns from XML legacy systems). Your handler picks one value silently.

Manually Built JSON in Tests

A developer builds a JSON test fixture by copy-pasting from real data. They duplicate a key by accident. The test runs, asserts on the “wrong” value, and either passes deceptively or fails with a confusing message.

Common Mistakes

Trusting JSON.parse to flag duplicates. It won't. Strict parsers exist (e.g. simple-json-parser with strict mode), but the JS built-in is permissive.
Assuming duplicates only happen at the top level. They can occur at any nesting level. PDFFlare's tool checks every object scope independently.
Treating two keys with different cases as different. JSON keys are case-sensitive — "Foo" and "foo" are different keys, not duplicates. Your data may treat them as duplicates semantically; clean that up at the source.
Thinking arrays of objects can have “duplicate keys.”An array of identical-shape objects has the same key in each element — that's normal, not a duplicate. Duplicates only occur within a single object.

Preventing Duplicates at the Source

Detecting duplicates after the fact is essential, but preventing them upstream is even better. Three patterns:

Build JSON from typed structures, not strings. JSON.stringify(obj)from a JS object can't produce duplicates because objects can't have duplicate keys. Avoid string concatenation when building JSON — that's where duplicates sneak in.
Use a JSON Schema with additionalProperties: falseat the API boundary. While the schema doesn't directly catch duplicates (parsers drop them before the schema sees), the discipline of writing a strict schema forces you to think about which keys are allowed.
Run the duplicate-key check as a CI step. If your repo contains JSON fixtures, configs, or test data, run PDFFlare-style detection on every PR. Block merges that introduce duplicates.

What Counts as a “Duplicate” Anyway?

The detector looks for keys that appear more than once at the same object level. Some borderline cases worth clarifying:

Same key, different nesting: {"user": {"name": "alice"}, "name": "bob"} has name twice but at different scopes — not a duplicate. JSON parsers handle both fine.
Same key inside array elements: [{"id": 1}, {"id": 2}] has two id keys but each is in its own object scope — not a duplicate. Standard array-of-objects pattern.
Whitespace differences: {"foo": 1, "foo ": 2} (trailing space) has two different keys (whitespace counts). Not a duplicate by the spec. But in practice, you almost certainly meant one — fix the trailing space.
Unicode normalization:Two visually identical keys can have different Unicode codepoints (e.g. composed vs decomposed accents). Strict spec says they're different. In practice, normalize to NFC before parsing.
Numeric keys: Object keys are always strings in JSON. {"1": "a", "01": "b"}has two keys (different strings, different values). They'd be the same number, but JSON doesn't care.

Best Practices

Run the duplicate check on every external JSON input. Configs, webhook payloads, third-party exports — anywhere you don't fully control the source.
Validate JSON with a strict parser in CI. Several JSON validators (e.g. jsonlint --quiet --no-comments) will fail on duplicates. Add to your CI pipeline so duplicates can't reach production.
Combine with a JSON Schema check. Schemas with additionalProperties: falsecatch a related class of bug — unexpected keys. PDFFlare's JSON Schema tool generates schemas you can use as guards.
Format your JSON before checking. PDFFlare's JSON Formatter gives you stable line numbers for the duplicate report.
Use ordered objects when order matters. JSON itself doesn't guarantee key order, but most implementations preserve insertion order. If you have to handle duplicates by “first wins,” document that explicitly and use a parser that supports it (some Python and Java parsers offer strict modes).

How Different Languages Handle JSON Duplicates

Behaviour varies wildly across implementations. Knowing yours matters:

JavaScript (JSON.parse): Last wins, silent. The most permissive of the major implementations.
Python (json.loads): Last wins by default — same as JavaScript. Pass object_pairs_hook=collections.OrderedDict with custom logic to detect duplicates, or use a strict wrapper.
Go (encoding/json): Decoding into a struct silently keeps the last duplicate key. Decoding into map[string]interface{} also keeps the last.
Rust (serde_json): By default, same behaviour. Set features = ["preserve_order"] and you can detect duplicates manually.
Java (Jackson): Has a strict-mode flag (FAIL_ON_READING_DUP_TREE_KEY) that throws on duplicates — disabled by default, but worth enabling.
jq: Last wins, silent. Same JS-style behaviour.

The takeaway: everypopular JSON parser silently drops duplicates by default. Fixing this requires either a strict-mode flag (where available) or a separate duplicate-check pass — which is what PDFFlare's tool gives you for free.

Strict JSON: When the Spec Says “Don't”

RFC 8259 (Section 4) says: “The names within an object SHOULD be unique.” That's SHOULD, not MUST — meaning compliant implementations are encouraged but not required to reject duplicates. That permissiveness is the source of every duplicate-key bug in the wild. If you can mandate strict mode in your tooling pipeline, do it. Otherwise add a check pass before parsing — exactly what PDFFlare's detector does.

Privacy: Your JSON Stays on Your Device

PDFFlare's Duplicate Keys tool runs entirely in your browser. The raw-text scan happens locally — your JSON never uploads to any server. Safe for production configs, internal payloads, JSON containing secrets or PII.

Wrapping Up

Duplicate keys are the canonical example of a bug that's obvious once you know to look for it, and invisible until you do. A 30-second scan with PDFFlare's detector catches the entire class of silent data-loss bugs that JSON.parse hides from you.

Got a JSON file you're about to feed to JSON.parse? Open PDFFlare's Duplicate Keys tool first. Two seconds for confidence that nothing's about to silently disappear.

Related Tools

JSON Formatter — format with line numbers for clear duplicate reports
JSON Schema — validate input against a schema with strict additional properties
JSON Stats — see total vs unique key counts at a glance
JSON Viewer — browse the parsed tree to see what survived parsing