Repository navigation
[FEATURE] SSL verification + custom proxies #3844
Description
What kind of feature are you missing? Where do you notice a shortcoming of PTB?
There is no way to disable ssl verification for proxies and pass a custom httpx.Proxy, instead of just a string.
Describe the solution you'd like
Some proxies work as a man-in-the-middle and may lead to errors, like: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1006).
Due to this reason, I think it's necessary to allow control over ssl verification in HTTPXRequest. Just add an argument like ssl_verify: bool = True and add verify=ssl_verify to self._client_kwargs.
Another thing I'd love to see, is to change the type of:
proxy_url: Optional[str] = None,
to
proxy_url: Optional[Union[str, httpx.Proxy] = None,
This is due to some proxies requiring authorization through headers, which could be passed as:
httpx.Proxy(
url=proxy_url,
headers={
"x-secret-header": "secret value"
},
)
(moreover it might be used for passing logging headers to the proxy and so on)
Describe alternatives you've considered
No response
Additional context
No response
Activity
[-][FEATURE][/-][+][FEATURE] SSL verification + custom proxies[/+]tsnoam commentedon Aug 16, 2023
I really dislike verify=False. It is better to add a custom CA cert, if and only if the user trusts the entity performing ssl mitm.
I would argue that we shouldn't easily allow disabling ssl verification as it compromises the security guarantees of Telegram. If someone really really want it then they should inherit from HTTPXRequest (or monkey patch it) and mangle whatever they want, but then the responsibility and the implications are on them.
The suggested approach to setting a proxy seems like a solid solution. We should do it.
codereptile commentedon Aug 17, 2023
@tsnoam
1 - In my specific case, there is simply no way to provide a custom CA cert, but I can agree that this might be a one-of-a-kind problem, so it shouldn't be implemented. However, I'd ask you to review the patch I'm currently using:
2 - If I could be of any assistance, I'd be glad to help. I can make a pull request, but since this seems like a minor change, I'd probably spend more time reading guidelines.
Bibo-Joshi commentedon Aug 19, 2023
1: Your snippet looks okay to me. Just be aware that we make no guarantees about the stability of protected methods :)
Since in our case we only have one target to query, I see no need to allow for such functionality. Cc @tsnoam
@codereptile you are ofc very welcome to contribute if you are interested :)
Bibo-Joshi commentedon Sep 25, 2023
v0.25.0 of httpx adds support for https proxies is encode/httpx#2845. Would this feature resolve the verification problem mentioned in the original comment (possibly after exposing this feature appropriately via PTBs interface)? CC @tsnoam
codereptile commentedon Sep 28, 2023
@Bibo-Joshi As far as I'm concerned, yes, you could set up the https proxy correctly, if PTB would expose the https proxy feature.
Seems like both tickets could be solved with a single PR.
tsnoam commentedon Oct 21, 2023
@Bibo-Joshi I think that ptb already supports getting a
httpx.Proxyobject overproxy_urlas there's no enforcement to it beingstr.So if I read the code right, the problem is semantics & documentation. Nothing on the code itself.
So basically, in ideal world what could have been changed is:
proxy_urltoproxyproxytype fromstrtohttpx.Proxy.and that's it.
again, I hope I got things right here...
codereptile commentedon Oct 21, 2023
@tsnoam Yes, you can actually just throw in the proxy and it'll work (and that's how I did), but the type/name change would be nice to avoid IDE/mypy errors