Repository navigation
Add an option to json.tool to bypass non-ASCII characters. #71600
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-12-06.06:44:23.432>
created_at = <Date 2016-06-29.11:49:29.439>
labels = ['type-feature', 'library', '3.9']
title = 'Add an option to json.tool to bypass non-ASCII characters.'
updated_at = <Date 2019-12-06.06:44:23.427>
user = 'https://github.com/legnaleurc'bugs.python.org fields:
activity = <Date 2019-12-06.06:44:23.427>
actor = 'methane'
assignee = 'none'
closed = True
closed_date = <Date 2019-12-06.06:44:23.432>
closer = 'methane'
components = ['Library (Lib)']
creation = <Date 2016-06-29.11:49:29.439>
creator = 'legnaleurc'
dependencies = []
files = ['44246']
hgrepos = []
issue_num = 27413
keywords = ['patch']
message_count = 13.0
messages = ['269479', '269498', '269559', '269584', '269641', '269648', '269667', '273803', '273814', '273817', '273819', '273911', '357902']
nosy_count = 7.0
nosy_names = ['rhettinger', 'r.david.murray', 'methane', 'berker.peksag', 'serhiy.storchaka', 'legnaleurc', 'qingyunha']
pr_nums = ['201', '2720', '17472']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27413'
versions = ['Python 3.9']
Activity
legnaleurc commentedon Jun 29, 2016
This patch adds a command line option "--no-escape" that allows json.tool to display non-ASCII characters.
e.g.:
bitdancer commentedon Jun 29, 2016
Maybe name it --no-insure-ascii? Or --insure-ascii=no/yes. (or true/false).
legnaleurc commentedon Jun 30, 2016
If the arguments should be aligned with those in dump/load, then maybe "--no-ensure-ascii" is an option?
bitdancer commentedon Jun 30, 2016
Sorry, yes, that's what I meant. I think it will make it easier to understand and remember if the option uses the same terminology as the function.
legnaleurc commentedon Jul 1, 2016
Use "--no-ensure-ascii" instead.
berkerpeksag commentedon Jul 1, 2016
The patch needs tests and documentation.
I'd go with
action='store_false', default=True.legnaleurc commentedon Jul 1, 2016
Ok, I'll update it later.
If I'm not misreading your comment, this will change the original behavior, right? (because options.no_ensure_ascii will be set to True by default)
legnaleurc commentedon Aug 28, 2016
Added doc and test.
serhiy-storchaka commentedon Aug 28, 2016
Test fails on non-utf8 locale.
======================================================================
FAIL: test_no_ensure_ascii_flag (test.test_json.test_tool.TestTool)
----------------------------------------------------------------------
First differing element 0:
b'"\\u00e6\\u00b8\\u00ac\\u00e8\\u00a9\\u00a6"'
b'"\\u6e2c\\u8a66"'
+ [b'"\\u6e2c\\u8a66"']
----------------------------------------------------------------------
legnaleurc commentedon Aug 28, 2016
Test passed with LC_ALL=en_US and LC_ALL=en_US.UTF-8 .
I've tried to use locale.getdefaultlocale(), but seems the output string will vary in different locales.
berkerpeksag commentedon Aug 28, 2016
Assuming you also change
to
no, it won't change the original behavior. That way you won't need to invert the value of
options.no_ensure_asciiin line 37.serhiy-storchaka commentedon Aug 30, 2016
The last change just sweeps a problem under a rug.
For now json.tool never fails with valid data. But with the --no-ensure-ascii option it can fail when output a string not encodable with the locale encoding. All can work with common cases on common UTF-8 environment, but unexpectedly fail on nonstandard environment. It would be better to output encodable characters as is and represent unencodable characters with \uXXXX encoding.
methane commentedon Dec 6, 2019
New changeset efefe25 by Inada Naoki (wim glenn) in branch 'master':
bpo-27413: json.tool: Add --no-ensure-ascii option. (GH-17472)
efefe25