Proper or custom JSON serialization of non-finite float values

@mdickinson

Feature or enhancement

The goal of this feature is to allow the JSON serializer in stdlib to serialize non-finite values (NaN, Inf, -Inf) according to the JSON specification. Going beyond just conforming to the spec, we could allow for custom serialization behavior.

Previous discussion

This problem was previously discussed in #84813, and a related PR was submitted in 2019, however, @mdickinson suggested I open a fresh issue where we can discuss the implementation in-depth.

Pitch

Currently, python's default JSON serializer encodes values like NaN as-is, with the explanation being that many JS-based JSON libraries also do this, and that the corresponding parsers can handle such non-conforming input.
In reality, most major browsers do not support this type of encoding and even NodeJS(v14.16.0) acts according to the previously-linked JSON spec.
The keyword argument allow_nan makes the serializer throw when encountering non-finite values when set to true, but I'd argue it is paramount to ensure compatibility with the spec and modern browsers. Changing the default behavior is of course not needed or possible at this point.

When implementing this feature, there are two main decisions to make.

Firstly, it has to be decided if allow_nan should be extended to take more datatypes like strings and callables, or if we should create a separate argument for this functionality.
Re-purposing allow_nan would make the control over such behavior centralized, however the name is very limiting.
It doesn't say anything about other non-finite values, and without looking at the docs, one would think it only takes bool values.

Secondly, it has to be decided how far we want to take this feature.
Do we want to have pre-defined cases like as_is, throw, and to_null, or do we want to allow the user to pass their own callable? The latter is implemented by the linked PR. Having both options is also a possibility.

Overall, each combination of decisions has its advantages and drawbacks. Since I wasn't a part of such discussions before, I don't have a preference.
All I want is to see this feature get implemented, and I can create a PR once consensus is reached.

Linked PRs

gh-98306: Support JSON encoding of NaNs and infinities as null #115246

@Dzeri96

@Dzeri96 Thanks for opening the issue!

There was a fair bit of confusion in #84813; it would be good to eliminate that confusion up front here. To clarify, when you say:

to serialize non-finite values (NaN, Inf, -Inf) according to the JSON specification

An option for this already exists, via allow_nan = False. I think you (and others) were asking for something different, namely an option to convert NaNs and Infinities to null. Is that correct?

An option for this already exists, via allow_nan = False.

After taking a second look at the spec I guess throwing an exception instead of producing any output could be considered conform.
In any case, yes, I'm suggesting we add options for conversion to null or even custom behavior as explained above.

@mdickinson

I know the last week has probably been very hectic with the release of 3.11, but I'd hate this issue to be buried like the MR from 2019. Any tips of how to promote discussion and push this to get reviewed? @mdickinson

@Dzeri96

@Dzeri96 Maybe a ping on https://discuss.python.org? I'm afraid that I'm not personally likely to have time to look at this this side of Christmas.

A thread has been created here. Let's close this issue and continue discussion there.

pythongh-98306: Support JSON encoding of NaNs and infinities as null

Re-opening this issue, since I think the discussion in the thread linked above pointed to a reasonable path forward.

There's a half-solution in #115246: it adds the allow_nan="null" support, but not the proposed DeprecationWarning.

@mdickinson

@mdickinson I know I promised that I'd deliver a fix for this a year ago, but some things came up... You know how it goes. I guess you took over and implemented everything needed?

@Dzeri96

@Dzeri96 That's fine, and yes, I absolutely know how it goes. :-)

If you have any cycles to spare, it would be great if you could take a look at #115246 and let me know whether it seems like a reasonable solution to you.

@mdickinson

@mdickinson just posted a comment on the MR. Since you own the feature branch, it's best you implement my proposed changes, in case you agree with me. Let me know if I should get involved in any other way.

There's a half-solution in #115246: it adds the allow_nan="null" support, but not the proposed DeprecationWarning.

The solution in #115246 is now a full solution.

@serhiy-storchaka

@serhiy-storchaka could you remove this from "Done" or mark this as a duplicate?

I followed links trying to understand the current status here so I thought I would pull them together in case it is helpful for others:

Issue for general float customization (carried over from bpo): Supporting customization of float encoding in JSON #81022
General float customization PR: gh-81022: Supporting customization of float encoding in JSON #13233
Preceding issue to this one (Proper or custom JSON serialization of non-finite float values #98306): json.dumps() should encode float number NaN to null #84813
Breakout thread on discuss.python.org related to this issue (Proper or custom JSON serialization of non-finite float values #98306)
Fallback to default for unsupported floats PR: gh-81022: JSONEncoder call self.default for unsupported floats #28648
PR with option to convert non-finite to null: gh-98306: Support JSON encoding of NaNs and infinities as null #115246
Duplicate issue on non-finite float customization with some unique comments: Allow customization of NaN and Infinity serialization in json module #134717
Another duplicate (with less unique information): Python JSON module doesn't actually produce JSON #70293

One other previous work to reference is simplejson which has had an ignore_nan option that converts non-finite values to None since 2013. Its readme still describes the project as the development version of the standard library json but I suppose features don't flow easily from it. In the previous discussion, it was mentioned that "just use simplejson" could be a solution. In the previous discussion, it was also mentioned that simplejson is the only popular json library with a feature for handling non-finite floats easily. ¹

To me it seems best just to copy what has been working for simplejson (just a separate ignore_nan option rather than adding more values to the allow_nan option).

Also, while summarizing things, I will give what I think is the main motivation for adding something like ignore_nan (and presented most succinctly in #28648):

The current default behavior of json.dump can produce output that does not meet the JSON specification such that it can be loaded by most other tools (note -- this is a separate concern from whether converting non-finite floats to null is recommended by the specification, which came up in previous discussion). The main mechanism for extending the serialization to cover other types is to pass a default function but that only gets passed unsupported types. Since float is generally supported, non-finite floats never get passed to default and there is no way to avoid getting "NaN" output other than walking the object before hand and replacing all the nan values.

I am opened myself to suggestions with the "only popular" label. I will mention any other json libraries with support for customizing non-finite float output here: jsonyx through its hook feature. ↩

In jsonyx you can fully customize this with a preprocessing hook:

import jsonyx as json
import math

def hook(obj):
    if isinstance(obj, float) and math.isnan(obj):
        return None
    return obj

json.dump(math.nan, hook=hook) # null

Maybe we could do something similar for the standard library?

I really need to finish this issue. It's constantly on my mind but I can't find the time to finish it. Last time I attempted to do it, I couldn't get Cpython to compile at all

Navigation Menu

Repository navigation

Proper or custom JSON serialization of non-finite float values #98306

Feature or enhancement

Previous discussion

Pitch

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Proper or custom JSON serialization of non-finite float values #98306

Description

Feature or enhancement

Previous discussion

Pitch

Linked PRs

Activity

mdickinson commented on Oct 16, 2022

Dzeri96 commented on Oct 16, 2022

Dzeri96 commented on Oct 26, 2022

mdickinson commented on Oct 26, 2022

Dzeri96 commented on Oct 28, 2022

mdickinson commented on Feb 10, 2024

Dzeri96 commented on Feb 11, 2024

mdickinson commented on Mar 3, 2024

Dzeri96 commented on Mar 3, 2024

mdickinson commented on May 12, 2024

nineteendo commented on May 30, 2025

wshanks commented on May 1, 2026

Footnotes

nineteendo commented on May 1, 2026

Dzeri96 commented on May 2, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions