Skip to content

bpo-27413: add --no-ensure-ascii argument to json.tool #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 13 commits into from
Closed

bpo-27413: add --no-ensure-ascii argument to json.tool #201

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
a47f396
Add indent support for json.tool
dhimmel Feb 20, 2017
fb5d984
Add support for disabling ensure_ascii in json.tool
dhimmel Feb 20, 2017
4fb3bb3
Add test for no_escape arg of json.tool
dhimmel Feb 20, 2017
29d01fe
bpo-27413: --no-ensure-ascii argument in json.tool
dhimmel Feb 23, 2017
a3e2b23
Add non-ascii characters to TestTool.data
dhimmel Feb 23, 2017
97152e9
Add --no-indent argument
dhimmel Feb 23, 2017
d699070
Escape non-ascii characters in json.tool tests
dhimmel Feb 23, 2017
0f38c18
Switch --no-indent to use store_const
dhimmel Feb 26, 2017
66a173a
Improve argparse help messages
dhimmel Feb 26, 2017
dd25cf3
Add unicode character in the U+0080-U+00FF range
dhimmel Feb 26, 2017
3413f4f
Format unicode output for JSON
dhimmel Feb 26, 2017
52bafb0
Test indent
dhimmel Feb 26, 2017
1306518
Move unicode tests outside of self.data
dhimmel Feb 26, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 17 additions & 4 deletions Lib/json/tool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,24 +25,37 @@ def main():
help='a JSON file to be validated or pretty-printed')
parser.add_argument('outfile', nargs='?', type=argparse.FileType('w'),
help='write the output of infile to outfile')
parser.add_argument('--no-ensure-ascii', action='store_true', default=False,
help='Do not set ensure_ascii to escape non-ASCII characters')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This option name should be --ensure_ascii, (and default should be True).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@methane are you suggesting:

parser.add_argument('--ensure_ascii', action='store_false', default=True)
# And then when calling `json.dump`:
ensure_ascii=options.ensure_ascii

The problem here is that it's counterintuitive for --ensure_ascii to result in ensure_ascii=False. In https://bugs.python.org/issue27413, the discussion settled on --no-ensure-ascii, which I'm happy to switch to:

parser.add_argument('--no-ensure-ascii', action='store_true', default=False)
# And then when calling `json.dump`:
ensure_ascii=not options.no_ensure_ascii

Just let me know what's preferred.

group = parser.add_mutually_exclusive_group()
group.add_argument('--indent', default=4, type=int,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed tests for --indent and --no-indent.

help='Indent level for pretty-printing.')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Document default value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 66a173a.

group.add_argument('--no-indent', action='store_true', default=False,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe use action='store_const', dest='indent', const=None?

Copy link
Contributor Author

@dhimmel dhimmel Feb 26, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice idea. Implemented in 0f38c18.

As an aside, argparse in Python 3.6 seems to have a bug where mutual exclusivity is not checked if you don't change the default value. So for example:

# error: argument --no-indent: not allowed with argument --indent
echo "[1,2,3]" | python Lib/json/tool.py --indent=5 --no-indent

# No error, compact mode
echo "[1,2,3]" | python Lib/json/tool.py --indent=4 --no-indent

# No error, indented
echo "[1,2,3]" | python Lib/json/tool.py --no-indent --indent=4

Something to keep in mind... assuming this will be fixed.

help='Use compact mode.')
parser.add_argument('--sort-keys', action='store_true', default=False,
help='sort the output of dictionaries alphabetically by key')
options = parser.parse_args()

# Read input JSON
infile = options.infile or sys.stdin
outfile = options.outfile or sys.stdout
sort_keys = options.sort_keys
with infile:
try:
if sort_keys:
if options.sort_keys:
obj = json.load(infile)
else:
obj = json.load(infile,
object_pairs_hook=collections.OrderedDict)
except ValueError as e:
raise SystemExit(e)

# Export JSON
outfile = options.outfile or sys.stdout
with outfile:
json.dump(obj, outfile, sort_keys=sort_keys, indent=4)
json.dump(obj, outfile,
indent=None if options.no_indent else options.indent,
ensure_ascii=not options.no_ensure_ascii,
sort_keys=options.sort_keys,
)
outfile.write('\n')


Expand Down
14 changes: 11 additions & 3 deletions Lib/test/test_json/test_tool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ class TestTool(unittest.TestCase):
data = """

[["blorpie"],[ "whoops" ] , [
],\t"d-shtaeou",\r"d-nthiouh",
],\t"d-shtaeou",\r"\N{snake} \u03B4 and \U0001D037",
"i-vhbjkhnth", {"nifty":87}, {"morefield" :\tfalse,"field"
:"yes"} ]
"""
Expand All @@ -26,7 +26,7 @@ class TestTool(unittest.TestCase):
],
[],
"d-shtaeou",
"d-nthiouh",
"\N{snake} \u03B4 and \U0001D037",
"i-vhbjkhnth",
{
"nifty": 87
Expand All @@ -48,7 +48,7 @@ class TestTool(unittest.TestCase):
],
[],
"d-shtaeou",
"d-nthiouh",
"\N{snake} \u03B4 and \U0001D037",
"i-vhbjkhnth",
{
"nifty": 87
Expand Down Expand Up @@ -106,3 +106,11 @@ def test_sort_keys_flag(self):
self.assertEqual(out.splitlines(),
self.expect_without_sort_keys.encode().splitlines())
self.assertEqual(err, b'')

def test_no_ensure_ascii_flag(self):
infile = self._create_infile()
rc, out, err = assert_python_ok('-m', 'json.tool', '--no-ensure-ascii', infile)
self.assertEqual(rc, 0)
self.assertEqual(out.splitlines(),
self.expect_without_sort_keys.encode().splitlines())
self.assertEqual(err, b'')