Repository navigation
Performance regression in 2.5 #45386
Description
| BPO | 1045 |
|---|---|
| Nosy | @tim-one, @birkenfeld, @rhettinger, @amauryfa |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/rhettinger'
closed_at = <Date 2008-01-14.23:27:52.376>
created_at = <Date 2007-08-28.15:06:24.345>
labels = ['interpreter-core', 'type-bug']
title = 'Performance regression in 2.5'
updated_at = <Date 2008-01-14.23:27:52.375>
user = 'https://bugs.python.org/inducer'bugs.python.org fields:
activity = <Date 2008-01-14.23:27:52.375>
actor = 'rhettinger'
assignee = 'rhettinger'
closed = True
closed_date = <Date 2008-01-14.23:27:52.376>
closer = 'rhettinger'
components = ['Interpreter Core']
creation = <Date 2007-08-28.15:06:24.345>
creator = 'inducer'
dependencies = []
files = ['8341']
hgrepos = []
issue_num = 1045
keywords = []
message_count = 6.0
messages = ['55366', '55547', '55610', '57875', '57876', '57878']
nosy_count = 6.0
nosy_names = ['tim.peters', 'georg.brandl', 'rhettinger', 'amaury.forgeotdarc', 'ggenellina', 'inducer']
pr_nums = []
priority = 'normal'
resolution = 'wont fix'
stage = None
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue1045'
versions = ['Python 2.5']
Activity
inducer commentedon Aug 28, 2007
The attached program uncovers a two-fold performance regression in
Python 2.5 with respect to Python 2.4. Below, the "element-wise" case
goes from 2.5 seconds in 2.4 to about 4 in 2.5. Since that's a pretty
serious increase, I thought I'd point it out.
ggenellina commentedon Sep 1, 2007
I've narrowed the problem to the usage of generator expressions.
Generator expressions appear to be MUCH slower on 2.5 than on 2.4.
It appears to be a fixed cost, or startup cost, of the generator
expression; differences get smaller when building large tuples (but
always Python 2.5 runs slower than 2.4)
birkenfeld commentedon Sep 3, 2007
May this be a byproduct of the new generator features in 2.5?
amauryfa commentedon Nov 27, 2007
The slowdown is due to the new function _PyObject_LengthHint: on my
Win2K machine,
the command
python -m timeit "tuple(1 for _ in xrange(3))"
answers:
100000 loops, best of 3: 4.14 usec per loop
By adding a line "return rv;" on the second line of
_PyObject_LengthHint, I get:
100000 loops, best of 3: 2.71 usec per loop
Should we disable this function for some known built-in types?
rhettinger commentedon Nov 27, 2007
I'll take a look at it next week -- want to think through the various
cases. The current setup provides nice speedups when the length_hint
is available. Possibly, it may be worthwhile to skip that check when
the input is a generator. For the most part, I'm more concerned about
the inner-loop time than the constant startup time outside the loop.
rhettinger commentedon Nov 27, 2007
FWIW, the 2.4 to 2.5 timing difference came from renaming __len__ to
__length_hint__. This was at Guido's request so that the value of bool
(iter(obj)) would always be True. The consequence of the change was
that we lost the fast slot lookup for __len__ and instead have to do a
dictionary based attribute lookup.
For the most part, I don't care about the overhead as it is constant.
The inner-loop cost dominates. If you do care, the choices are to add
some ugly, hackish specific type checks to bypass the attribute lookup
in cases like generator objects where we know the type is absent. A
more general, cleaner solution is to add a new slot for a length hint.
Personally, I would rather leave this as is and live with the small
constant lookup time. If you concur, please close this report. If
not, please submit a patch for adding a new slot (model the code after
that in PyObject_Size()).