Repository navigation
unicodedata checksum-tests only test 1/17th of Unicode's codepoints #81939
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = <Date 2019-09-12.09:25:28.561>
created_at = <Date 2019-08-05.01:06:02.417>
labels = ['type-feature', 'tests', '3.9']
title = "unicodedata checksum-tests only test 1/17th of Unicode's codepoints"
updated_at = <Date 2019-09-12.09:25:28.560>
user = 'https://github.com/gnprice'bugs.python.org fields:
activity = <Date 2019-09-12.09:25:28.560>
actor = 'benjamin.peterson'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-12.09:25:28.561>
closer = 'benjamin.peterson'
components = ['Tests']
creation = <Date 2019-08-05.01:06:02.417>
creator = 'Greg Price'
dependencies = []
files = []
hgrepos = []
issue_num = 37758
keywords = ['patch']
message_count = 5.0
messages = ['349014', '349016', '349523', '351499', '352069']
nosy_count = 4.0
nosy_names = ['twouters', 'vstinner', 'benjamin.peterson', 'Greg Price']
pr_nums = ['15125', '15126', '15302']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue37758'
versions = ['Python 3.9']
Activity
gnprice commentedon Aug 5, 2019
The unicodedata module has two test cases which run through the database and make a hash of its visible outputs for all codepoints, comparing the hash against a checksum. These are helpful regression tests for making sure the behavior isn't changed by patches that didn't intend to change it.
But Unicode has grown since Python first gained support for it, when Unicode itself was still rather new. These test cases were added in commit 6a20ee7 back in 2000, and they haven't needed to change much since then... but they should be changed to look beyond the Basic Multilingual Plane (
range(0x10000)) and cover all 17 planes of Unicode's final form.Spotted in discussion on #59224 (#15019 (comment) ). I have a patch for this which I'll send shortly.
gnprice commentedon Aug 5, 2019
Sent two small PRs!
The first one, #59330, makes the substantive test change I described above.
The second one, #59331, is a small pure refactor to that test file, just cleaning out some bits that made sense when it was first written (as a script) but are confusing now that it's a
unittesttest module. Took me a couple of minutes to sort those out when I first dug into this file, and I figure it'd be kind to the next person to save them the same effort.benjaminp commentedon Aug 13, 2019
New changeset def97c9 by Benjamin Peterson (Greg Price) in branch 'master':
bpo-37758: Clean out vestigial script-bits from test_unicodedata. (GH-15126)
def97c9
Yhg1s commentedon Sep 9, 2019
New changeset 3cbc23a by T. Wouters (Greg Price) in branch 'master':
bpo-37758: Cut always-constant conditionals on sys.maxunicode. (GH-15302)
3cbc23a
benjaminp commentedon Sep 12, 2019
New changeset 6954be8 by Benjamin Peterson (Greg Price) in branch 'master':
closes bpo-37758: Extend unicodedata checksum tests to cover all of Unicode. (GH-15125)
6954be8