Repository navigation
configure incorrectly ignores pkg-config information for libffi and Tcl/Tk in 3.10 #89513
Description
| BPO | 45350 |
|---|---|
| Nosy | @tiran, @ned-deily, @pablogsal, @thesamesam |
| PRs |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/pablogsal'
closed_at = <Date 2021-10-03.23:47:33.351>
created_at = <Date 2021-10-03.04:13:54.679>
labels = ['build', '3.10', '3.11']
title = 'configure incorrectly ignores pkg-config information for libffi and Tcl/Tk in 3.10'
updated_at = <Date 2021-10-29.16:13:11.880>
user = 'https://github.com/ned-deily'bugs.python.org fields:
activity = <Date 2021-10-29.16:13:11.880>
actor = 'thesamesam'
assignee = 'pablogsal'
closed = True
closed_date = <Date 2021-10-03.23:47:33.351>
closer = 'pablogsal'
components = ['Build']
creation = <Date 2021-10-03.04:13:54.679>
creator = 'ned.deily'
dependencies = []
files = []
hgrepos = []
issue_num = 45350
keywords = ['patch']
message_count = 16.0
messages = ['403076', '403081', '403082', '403088', '403093', '403094', '403095', '403096', '403098', '403099', '403100', '403102', '403116', '403117', '403177', '405018']
nosy_count = 4.0
nosy_names = ['christian.heimes', 'ned.deily', 'pablogsal', 'thesamesam']
pr_nums = ['28707', '28708']
priority = 'high'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue45350'
versions = ['Python 3.10', 'Python 3.11']
Activity
ned-deily commentedon Oct 3, 2021
./configure supports using the system or third-party-supplied pkg-config utility to find or override the default location of header and library files when building a few C extensions in the standard library, namely from OpenSSL for the _ssl module, libffi for ctypes, and, new in 3.10, Tcl/Tk for _tkinter (bpo-42603). However, currently for 3.10.0, pkg-config usage is broken for libffi and Tcl/Tk (but not OpenSSL). When running ./configure, there is an unexpected warning that is easily overlooked:
[...]
checking for --with-libs... no
./configure: line 10545: PKG_PROG_PKG_CONFIG: command not found
checking for --with-system-expat... no
[...]
PKG_PROG_PKG_CONFIG is a macro provided by GNU Autotools that is supposed to be in aclocal.m4. Unfortunately, it appears to have been inadvertently deleted in 2fc857a (PR 25860) probably due to an autoconf version mismatch. The net effect is that the configure variable PKG_CONFIG, the location of the pkg-config utility, is undefined so tests in configure for the location of libffi and of Tcl and Tk do not take into account any pkg-config info and use any default locations (i.e. /usr/include). For most builds, that likely still produces the desired results. But it won't if a builder is trying to override these locations or is building on a platform with different default or is using a third-party package manager and pkg-config to supply libraries. Note, the _ssl module builds are not affected by this problem as the AX_CHECK_OPENSSL macro in aclocal.m4 does not depend on PKG_PROG_PKG_CONFIG to find pkg-config.
It appears that the problem can be worked around by explicitly setting the PKG_CONFIG build variable when invoking configure, something like:
./configure [...] PKG_CONFIG=$(which pkg-config)
But the PR 25860 changes to aclocal.a4 should be carefully reviewed and the pkg-config related deletes restored; there might be other problems, too. This is not the first time we've been caught up by unexpected autoconf changes and, as is clear here, it is too easy for them to go unnoticied. Perhaps we should try to figure out how to reduce those risks.
pablogsal commentedon Oct 3, 2021
Since then, I have updated the autotools version to the old one in #28151
so this should have been fixed by that.
What I am missing?
pablogsal commentedon Oct 3, 2021
Regarding preventing this, I have added a check so the autotools version doesn't get updated incorrectly:
#28152
ned-deily commentedon Oct 3, 2021
The updates to aclocal.m4 are generated by the aclocal tool of the GNU build system. That, and other things including autoconf, gets run by autoreconf. Running autoconf by itself is not sufficient.
https://devguide.python.org/setup/#regenerate-configure
https://www.gnu.org/software/autoconf/manual/autoconf-2.68/html_node/autoreconf-Invocation.html
pablogsal commentedon Oct 3, 2021
For regeneration I was using:
https://github.com/tiran/cpython_autoconf
as instructed by Christian.
pablogsal commentedon Oct 3, 2021
This runs:
"autoreconf -ivf"
with the correct version of autotools required by the openssl changes
pablogsal commentedon Oct 3, 2021
I can confirm that running autoreconf with Christian's version doesn't change the git tree:
❯ docker run -v
pwd:/src tiran/cpython_autoconfRebuilding configure script
autoreconf: Entering directory
.' autoreconf: configure.ac: not using Gettext autoreconf: running: aclocal --force autoreconf: configure.ac: tracing autoreconf: configure.ac: not using Libtool autoreconf: running: /usr/bin/autoconf --force autoreconf: running: /usr/bin/autoheader --force autoreconf: configure.ac: not using Automake autoreconf: Leaving directory.'Done
3.10 on 3.10 [$] pyenv 3.9.1 took 6s
❯ git status
On branch 3.10
Your branch is up to date with 'upstream/3.10'.
nothing to commit, working tree clean
pablogsal commentedon Oct 3, 2021
Ah, the problem is that pkgconfig is *not* installed in that DOckerfile. I can confirm that installing pkg-config and rerunning works.
pablogsal commentedon Oct 3, 2021
Created a PR for this:
pablogsal/cpython_autoconf#1
I think we should move this dockerfile to the CPython repo and update the devguide.
ned-deily commentedon Oct 3, 2021
+1, I wasn't aware of the existence of that.
pablogsal commentedon Oct 3, 2021
Something seems wrong with the openssl detection when the PKG_PROG_PKG_CONFIG is active :(
Christian, could you please take a look? I would like to get this ready for the release
tiran commentedon Oct 3, 2021
I have updated the container at https://quay.io/repository/tiran/cpython_autoconf to include pkg-config.
pablogsal commentedon Oct 3, 2021
New changeset f146ca3 by Pablo Galindo Salgado in branch '3.10':
bpo-45350: Rerun autoreconf with the pkg-config macros (GH-28707)
f146ca3
pablogsal commentedon Oct 3, 2021
New changeset a25dcae by Pablo Galindo Salgado in branch 'main':
bpo-45350: Rerun autoreconf with the pkg-config macros (GH-28708)
a25dcae
pablogsal commentedon Oct 4, 2021
New changeset 2c47b80 by Pablo Galindo (Pablo Galindo Salgado) in branch '3.10':
bpo-45350: Rerun autoreconf with the pkg-config macros (GH-28707)
2c47b80
ned-deily commentedon Oct 26, 2021
Note that we have since discovered that the fix for this problem inadvertently did not make it into the 3.10.0 release. It is in the current 3.10 branch and will be in 3.10.1, the first bugfix release for 3.10.