Skip to content

typing.NamedTuple instances are not picklable. #69851

Closed

Description

@ashwch
mannequin
BPO 25665
Nosy @gvanrossum, @rhettinger, @serhiy-storchaka, @ashwch
Files
  • issue25665.patch
  • test_typing_pickle.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = 'https://github.com/gvanrossum'
    closed_at = <Date 2015-11-19.16:21:49.096>
    created_at = <Date 2015-11-19.08:06:26.539>
    labels = ['library']
    title = 'typing.NamedTuple instances are not picklable.'
    updated_at = <Date 2015-11-20.16:34:02.151>
    user = 'https://github.com/ashwch'

    bugs.python.org fields:

    activity = <Date 2015-11-20.16:34:02.151>
    actor = 'python-dev'
    assignee = 'gvanrossum'
    closed = True
    closed_date = <Date 2015-11-19.16:21:49.096>
    closer = 'gvanrossum'
    components = ['Library (Lib)']
    creation = <Date 2015-11-19.08:06:26.539>
    creator = 'ashwch'
    dependencies = []
    files = ['41073', '41098']
    hgrepos = []
    issue_num = 25665
    keywords = ['patch']
    message_count = 7.0
    messages = ['254883', '254889', '254908', '254909', '254988', '254993', '254997']
    nosy_count = 5.0
    nosy_names = ['gvanrossum', 'rhettinger', 'python-dev', 'serhiy.storchaka', 'ashwch']
    pr_nums = []
    priority = 'normal'
    resolution = 'fixed'
    stage = None
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue25665'
    versions = ['Python 3.5', 'Python 3.6']

    Activity

    1. ashwch commented on Nov 19, 2015

      ashwchmannequin
      MannequinAuthor

      Currently namedtuple(https://hg.python.org/cpython/file/3.5/Lib/collections/__init__.py#l418) sets the __module__ attribute by looking up __name__ in calling frame's globals. As in the case of typing.NamedTuple it is always going to be 'typing' pickle will raise an error.

      Instead of this typing.NamedTuple should override the __module__ attribute itself because it has info about the actual caller frame.

      Something like this should work fine:

      def NamedTuple(typename, fields):
      
          fields = [(n, t) for n, t in fields]
          cls = collections.namedtuple(typename, [n for n, t in fields])
          cls._field_types = dict(fields)
          try:
              cls.__module__ = sys._getframe(1).f_globals.get('__name__', '__main__')
          except (AttributeError, ValueError):
              pass
          return cls
      

      Related: http://stackoverflow.com/q/33796490/846892

    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      on Nov 19, 2015
    3. ashwch commented on Nov 19, 2015

      ashwchmannequin
      MannequinAuthor

      Attached patch.

    4. python-dev commented on Nov 19, 2015

      python-devmannequin
      Mannequin

      New changeset 33df0056c148 by Guido van Rossum in branch '3.5':
      Issue bpo-25665: Make NamedTuple picklable.
      https://hg.python.org/cpython/rev/33df0056c148

      New changeset 8a32d44b8359 by Guido van Rossum in branch 'default':
      Issue bpo-25665: Make NamedTuple picklable. (Merge 3.5->3.6)
      https://hg.python.org/cpython/rev/8a32d44b8359

    5. gvanrossum commented on Nov 19, 2015

      @gvanrossum
      Member

      Fixed it! Thanks for the report *and* the patch. I wrote a different test though.

    6. self-assigned this
      on Nov 19, 2015
    7. serhiy-storchaka commented on Nov 20, 2015

      @serhiy-storchaka
      Member

      The test tests pickling only with default protocol. It would be better to test pickling with all supported protocols.

      Why special TestCase methods to check for and report failures are not used in test_typing?

    8. gvanrossum commented on Nov 20, 2015

      @gvanrossum
      Member

      Serhiy, feel free to commit that patch. I was just being lazy. Also, I was using assert instead of self.assertEquals etc. out of laziness (and because at Dropbox people have got the py.test religion and prefer to use assert statements -- but I'm not keen to do this for the stdlib).

    9. python-dev commented on Nov 20, 2015

      python-devmannequin
      Mannequin

      New changeset 4f30b0d47c24 by Serhiy Storchaka in branch '3.5':
      Issue bpo-25665: Test pickling with all protocols in test_typing.
      https://hg.python.org/cpython/rev/4f30b0d47c24

      New changeset 9e65015582a5 by Serhiy Storchaka in branch 'default':
      Issue bpo-25665: Test pickling with all protocols in test_typing.
      https://hg.python.org/cpython/rev/9e65015582a5

    10. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    Labels

    stdlibStandard Library Python modules in the Lib/ directory

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions