Skip to content

bpo-31505: Fix an assertion failure in json, in case _json.make_encoder() received a bad encoder() argument #3643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
serhiy-storchaka merged 5 commits into from
Sep 24, 2017
Merged

bpo-31505: Fix an assertion failure in json, in case _json.make_encoder() received a bad encoder() argument #3643

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a043ba6
init commit
orenmn Sep 18, 2017
fb91a87
fix test name
orenmn Sep 18, 2017
99244b7
improve patch C code
orenmn Sep 18, 2017
da90d1e
various fixes according to Serhiy's advice
orenmn Sep 18, 2017
014db74
remove the redundant import
orenmn Sep 19, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions Lib/test/test_json/test_speedups.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,27 @@ def test_make_encoder(self):
b"\xCD\x7D\x3D\x4E\x12\x4C\xF9\x79\xD7\x52\xBA\x82\xF2\x27\x4A\x7D\xA0\xCA\x75",
Copy link
Copy Markdown
Member

@serhiy-storchaka serhiy-storchaka Sep 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't related to this issue, but it seems to me that TypeError is raised for different cause than was originally in this test. This test doesn't work as intended.

Copy link
Copy Markdown
Contributor Author

@orenmn orenmn Sep 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, Victor wrote this test to verify that the interpreter doesn't crash (as part of https://bugs.python.org/issue6986, as originally PyArg_ParseTupleAndKeywords() was given actual fields of the new PyEncoderObject, instead of temp vars), so raising a TypeError is the wanted outcome here.
maybe we can just change the name of the test to 'test_issue6986' (as it tests only this specific issue)?
and maybe do the same for test_make_scanner()?
anyway, I guess any such change should be in a PR referring to bpo-6986..

Copy link
Copy Markdown
Member

@serhiy-storchaka serhiy-storchaka Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems your are right. Thank you for a reference.

None)

def test_bad_str_encoder(self):
# Issue #31505: There shouldn't be an assertion failure in case
# c_make_encoder() receives a bad encoder() argument.
def bad_encoder1(*args):
return None
enc = self.json.encoder.c_make_encoder(None, lambda obj: str(obj),
bad_encoder1, None, ': ', ', ',
False, False, False)
with self.assertRaises(TypeError):
enc('spam', 4)
Copy link
Copy Markdown
Member

@serhiy-storchaka serhiy-storchaka Sep 19, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be written without context manager:

self.assertRaises(TypeError, enc, 'spam', 4)

I prefer this form for simple function calls.

with self.assertRaises(TypeError):
enc({'spam': 42}, 4)

def bad_encoder2(*args):
1/0
enc = self.json.encoder.c_make_encoder(None, lambda obj: str(obj),
bad_encoder2, None, ': ', ', ',
False, False, False)
with self.assertRaises(ZeroDivisionError):
enc('spam', 4)

def test_bad_bool_args(self):
def test(name):
self.json.encoder.JSONEncoder(**{name: BadBool()}).encode({'a': 1})
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core and Builtins/2017-09-18-12-07-39.bpo-31505.VomaFa.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix an assertion failure in `json`, in case `_json.make_encoder()` received
a bad `encoder()` argument. Patch by Oren Milman.
16 changes: 13 additions & 3 deletions Modules/_json.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1429,10 +1429,20 @@ static PyObject *
encoder_encode_string(PyEncoderObject *s, PyObject *obj)
{
/* Return the JSON representation of a string */
if (s->fast_encode)
PyObject *encoded;

if (s->fast_encode) {
return s->fast_encode(NULL, obj);
else
return PyObject_CallFunctionObjArgs(s->encoder, obj, NULL);
}
encoded = PyObject_CallFunctionObjArgs(s->encoder, obj, NULL);
if (encoded != NULL && !PyUnicode_Check(encoded)) {
PyErr_Format(PyExc_TypeError,
"encoder() must return a string, not %.80s",
Py_TYPE(encoded)->tp_name);
Py_DECREF(encoded);
return NULL;
}
return encoded;
}

static int
Expand Down