Repository navigation
Instruction-Offsets vs Byte-Offsets in dis.dis output NOT A BUG #93060
Description
Please ignore this issue. It was caused by my running the 3.7 library dis in the 3.10 environment.
Bug report
This is new to version 3.10
When disassembling code code block with jumps, bytecode for the jump shows the wrong target instruction number, and/or the bytecode with the wrong address flagged with >>.
To reproduce, here is an example, I have interspersed the source code with the disassembled output:
try: x = 1
3 4 SETUP_FINALLY 15 (to 21) --- (to 2 * 15 = 30)
6 SETUP_FINALLY 4 (to 12) --- (to 8 + 2 * 4 = 16)
8 LOAD_CONST 1 (1)
10 STORE_NAME 1 (x)
>> 12 POP_BLOCK --- no >>
14 JUMP_FORWARD 6 (to 22) --- (to 16 + 2 * 6 = 28)
except: x = 2
4 16 POP_TOP --- >> (target of 6)
18 POP_TOP
20 POP_TOP
>> 22 LOAD_CONST 2 (2) --- no >>
24 STORE_NAME 1 (x)
26 POP_EXCEPT
>> 28 POP_BLOCK --- >> (target of 14), not >> (target of 48).
finally: x = 3
5 30 LOAD_CONST 3 (3)
32 STORE_NAME 1 (x)
34 JUMP_FORWARD 3 (to 39) --- (to 36 + 2 * 3 = 42)
36 LOAD_CONST 3 (3) --- >> (target of 4)
38 STORE_NAME 1 (x)
40 CONTINUE_LOOP 0
if x == 3:
7 42 LOAD_NAME 1 (x) --- >> (target of 34)
44 LOAD_CONST 3 (3)
46 COMPARE_OP 2 (==)
48 POP_JUMP_IF_FALSE 28 --- (to 28 * 2 = 56)
x = 4
8 50 LOAD_CONST 4 (4)
52 STORE_NAME 1 (x)
54 JUMP_FORWARD 2 (to 58) --- (to 56 + 2 * 2 = 60)
else:
x = 5
10 56 LOAD_CONST 5 (5)
>> 58 STORE_NAME 1 (x) --- no >>
import sys
14 60 LOAD_CONST 6 (0) --- >> (target of 54)
62 LOAD_CONST 7 (None)
64 IMPORT_NAME 2 (sys)
66 STORE_NAME 2 (sys)
Compare with output from 3.9. The compiled bytecodes are slightly different, but the jump addresses and labelled indictors are correct.
In my opinion, though, the true oparg values (17, etc.), which are shown in 3.10, should be shown instead of the doubled values (34, etc.), which are shown in 3.9. I would consider this a bug in 3.9.
3 4 SETUP_FINALLY 34 (to 40)
6 SETUP_FINALLY 8 (to 16)
8 LOAD_CONST 1 (1)
10 STORE_NAME 1 (x)
12 POP_BLOCK
14 JUMP_FORWARD 16 (to 32)
4 >> 16 POP_TOP
18 POP_TOP
20 POP_TOP
22 LOAD_CONST 2 (2)
24 STORE_NAME 1 (x)
26 POP_EXCEPT
28 JUMP_FORWARD 2 (to 32)
30 RERAISE
>> 32 POP_BLOCK
5 34 LOAD_CONST 3 (3)
36 STORE_NAME 1 (x)
38 JUMP_FORWARD 6 (to 46)
>> 40 LOAD_CONST 3 (3)
42 STORE_NAME 1 (x)
44 RERAISE
7 >> 46 LOAD_NAME 1 (x)
48 LOAD_CONST 3 (3)
50 COMPARE_OP 2 (==)
52 POP_JUMP_IF_FALSE 60
8 54 LOAD_CONST 4 (4)
56 STORE_NAME 1 (x)
58 JUMP_FORWARD 4 (to 64)
10 >> 60 LOAD_CONST 5 (5)
62 STORE_NAME 1 (x)
14 >> 64 LOAD_CONST 6 (0)
66 LOAD_CONST 7 (None)
68 IMPORT_NAME 2 (sys)
70 STORE_NAME 2 (sys)
Your environment
- CPython versions tested on: Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32
- Operating system and architecture: Windows
Activity
sweeneyde commentedon May 21, 2022
Python 3.9 is now only accepting security-related fixes.
mrolle45 commentedon May 21, 2022
Fine.
I wonder if Python releases come with a list of known bugs, and whether they will be fixed in a future release.
Do older versions of Python accept fixes to documentation?
sweeneyde commentedon May 22, 2022
No, not typically for security-only branches.
This isn't a bug so much as a deliberate change in behavior. In Python 3.9, the opcode arguments of jumps were byte offsets. In Python 3.10+, opcode arguments of jumps are stored as instruction offsets. In both cases, the stored number is displayed as the opcode argument. Besides, as noted in the dis module docs:
In the changelog for 3.10, it was noted that
sweeneyde commentedon May 22, 2022
I believe it has been suggested that the even-numbered byte-offset labels down the left-hand side could be replaced by instruction-offsets (half their current value) (and
(to xx)labels likewise halved), which might help clear things up.[-]dis.dis() shows wrong target instruction and/or shows wrong bytecode as a labelled instruction.[/-][+]Instruction-Offsets vs Byte-Offsets in dis.dis output[/+]sweeneyde commentedon May 22, 2022
There was an exchange about this on the original PR for the change.
cc @markshannon @gvanrossum
gvanrossum commentedon May 22, 2022
Um, looking at the 3.10 example it does look wrong — how can the “to” address be odd?
brandtbucher commentedon May 22, 2022
I'm not able to reproduce the 3.10.0 output. It appears that some of its
disoutput (like the odd "to" addresses) may have been modified in the original issue report when the annotations were added? It appears that the annotations don't account for the change in oparg values to use instruction offsets instead of bytes in 3.10.If not, can you give us the source and (unannotated)
disoutput that you believe is incorrect?mrolle45 commentedon May 22, 2022
Sorry, I didn't think to also provide the python file and dis output separately. Now that you asked for it, I realize that it would help you folks in debugging it.
example.py.txt
dis.txt
Interesting discussion over whether to display instruction offsets or byte offsets. Personally I don't care, as long as you are consistent and accurate. My suggestion of showing an instruction offset in the oparg and then showing the byte offset in the (to ...) was, I now see, a bad idea. You can show (1) the bytecode address, (2) the oparg, and (3) the target, as either all instruction offsets or byte offsets.
I wonder why 3.10 doesn't just do what 3.9 does. Does it have something to do with changing the oparg to byte offsets? That sounds like a good idea for reducing the occurrences of opargs >= 256. Just be sure you cover all the cases correctly.
The answer is that dis() was adding an offset to a byte address.
sweeneyde commentedon May 22, 2022
What is your procedure for generating dis.txt? Is it possible that you are using an earlier version of Python to disassemble bytecode from Python 3.10?
Python 3.8+ does not have a
CONTINUE_LOOPopcode (119 is nowRERAISE), so it should not appear in any output from 3.10.mrolle45 commentedon May 22, 2022
Nice detective work, @sweeneyde. You've saved your team a good bit of work. Thank you!
Whoops. In my python 3.10, I seem to be pointing to the lib in 3.7.
Here's how I generated the dis file:
Running with my usual Python (3.7):
This time, it produced a correct (for 3.7) output with the right opargs and targets and addresses.
Apparently, the dis module compiles the source file then analyzes the code object, and the compile() function is a 3.10 builtin.
I'll get my 3.10 environment straightened out, and maybe the output will be correct.
mrolle45 commentedon May 22, 2022
Hooray! It's not a bug, after all. It was my error :(.
I had copied py10.exe from my 3.10 root to my 3.7 root, which was on my %path%. That's why I got the 3.10 python interpreter but the 3.7 library.
I removed this interloper, and added the 3.10 root to %path%, after the 3.7 root. I normally use 3.7 for everything.
Now my session is:
The output is
[-]Instruction-Offsets vs Byte-Offsets in dis.dis output[/-][+]Instruction-Offsets vs Byte-Offsets in dis.dis output NOT A BUG[/+]