You often need to compare data stored as JSON, such as API responses, configuration files, and translation files. But if you run a plain text diff on two JSON documents, you frequently get lots of differences even though the data is the same. This guide explains why and how to fix it.
Why false differences appear
The JSON standard (RFC 8259) defines an object as "an unordered collection of name/value pairs". In other words, {"a":1,"b":2} and {"b":2,"a":1} represent the same data. As text, however, the lines are in a different order, so diff sees a difference. The following differences also arise regardless of the data.
- Indentation width (2 spaces, 4 spaces, tabs) versus JSON compressed onto one line
- Whether there is a space after the colon (
"a":1versus"a": 1) - Number notation (
1.0versus1,1e2versus100) - String escapes (
"\u00e9"versus"é")
The fix: parse, then rewrite in the same shape
The most reliable method is to parse both JSON documents into data and then serialize them again with the same rules.
- Parse both texts as JSON. If there is a syntax error, fix it first.
- Sort the keys of every object recursively.
- Write them out again with the same indentation (e.g. 2 spaces).
- Compare the results line by line.
For example, the following two JSON documents differ in key order and in array order.
{"name":"kim","age":30,"tags":["a","b"]}{"age":30,"name":"kim","tags":["b","a"]}After sorting and rewriting, the key-order difference disappears and only the real difference, the array order, remains.
@@ -2,7 +2,7 @@ "age": 30, "name": "kim", "tags": [- "a",- "b"+ "b",+ "a" ] }Do not sort arrays
Unlike objects, arrays carry meaning in their order. ["a","b"] and ["b","a"] are different data. So as a rule, normalization does not sort arrays. That said, for arrays whose order does not matter in practice, such as a list of tags, sorting them yourself before comparing makes the result easier to read. Which is right depends on what the data means.
What parsing changes
Parsing and rewriting has side effects worth knowing about.
- Number notation is unified.
1.0and1both become1and look the same. That is usually desirable, but if the notation itself matters, compare the original text too. - Precision of large integers. JavaScript handles numbers as 64-bit floating point, so integers larger than
Number.MAX_SAFE_INTEGER(2^53 − 1 = 9007199254740991) cannot be represented exactly. For example,9007199254740993becomes9007199254740992after parsing. Watch out for this with JSON that stores long IDs as numbers. - Duplicate keys. RFC 8259 only recommends (SHOULD) that names within an object be unique, and how duplicate keys are handled varies between implementations. JavaScript's
JSON.parsekeeps the last value. - Comments and trailing commas. Comments like
// commentand trailing commas like[1, 2,]are not standard JSON (extended formats such as JSON5 and JSONC allow them). Standard parsers report an error.
On the command line
If jq is installed, you can print JSON with sorted keys using the -S (--sort-keys) option.
jq -S . before.json > a.json
jq -S . after.json > b.json
diff -u a.json b.jsonHow this differs from JSON Patch
A diff result is a line-based list of changes meant for people to read. JSON Patch (RFC 6902), by contrast, is a standard that expresses changes as paths in the JSON structure, such as {"op":"replace","path":"/age","value":31}, and JSON Merge Patch (RFC 7396) works by overlaying a JSON document containing only the changed parts onto the original. When programs exchange changes, these standards are the right tool; when people review changes, a sorted text diff is.
Try it in this tool
Turn on Sort JSON keys in the options of the Text Diff Checker, and both sides are parsed, their keys sorted, and rewritten with 2-space indentation before comparison. Array order is not changed, and if parsing fails, it tells you which side and at which character the error occurred. You can also drop two .json files at once. To read the result as a patch, see How to read a unified diff.