Skip to content

Optimize pymalloc for non PGO build #81724

Closed

Description

@methane
BPO 37543
Nosy @tim-one, @methane
PRs
  • bpo-37543: optimize pymalloc #14674
  • bpo-37732 : Fix unintialized pointer warning in Objects/obmalloc.c #15309
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-07-17.12:24:34.174>
    created_at = <Date 2019-07-10.10:59:54.080>
    labels = ['interpreter-core', '3.9', 'performance']
    title = 'Optimize pymalloc for non PGO build'
    updated_at = <Date 2019-08-18.10:11:20.109>
    user = 'https://github.com/methane'

    bugs.python.org fields:

    activity = <Date 2019-08-18.10:11:20.109>
    actor = 'hansrajdas'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-07-17.12:24:34.174>
    closer = 'methane'
    components = ['Interpreter Core']
    creation = <Date 2019-07-10.10:59:54.080>
    creator = 'methane'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 37543
    keywords = ['patch']
    message_count = 2.0
    messages = ['347615', '348057']
    nosy_count = 2.0
    nosy_names = ['tim.peters', 'methane']
    pr_nums = ['14674', '15309']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = 'performance'
    url = 'https://bugs.python.org/issue37543'
    versions = ['Python 3.9']

    Activity

    1. methane commented on Jul 10, 2019

      @methane
      MemberAuthor

      When PGO is not used, compilers don't know which part is hot.

      So gcc failed to inline hot code in pymalloc_alloc and pymalloc_free
      into _PyObject_Malloc and _PyObject_Free. For example, only this code is inlined into _PyObject_Malloc.

          if (nbytes == 0) {
              return 0;
          }
          if (nbytes > SMALL_REQUEST_THRESHOLD) {
              return 0;
          }

      But the hottest part is taking memory block from freelist in the pool.
      To optimize it,

      • make pymalloc_alloc and pymalloc_free inline functions
      • Split code for rare / slow paths out to new functions

      In PR 14674, pymalloc is now as fast as mimalloc in spectral_norm benchmark.

        $ ./python bm_spectral_norm.py --compare-to=./python-master
        python-master: ..................... 199 ms +- 1 ms
        python: ..................... 176 ms +- 1 ms

      Mean +- std dev: [python-master] 199 ms +- 1 ms -> [python] 176 ms +- 1 ms: 1.13x faster (-11%)

    2. methane commented on Jul 17, 2019

      @methane
      MemberAuthor

      New changeset fb26504 by Inada Naoki in branch 'master':
      bpo-37543: optimize pymalloc (bpo-14674)
      fb26504

    3. transferred this issue fromon Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.9 (EOL)end of lifeinterpreter-core(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usage

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions