Repository navigation
[Feature Request] Privacy in log files #439
Description
Describe the feature idea
Promote user privacy during diagnostic sessions by stripping username and URLs (and any other personally-identifying-information) from logs.
Additional context
I'm at times overly security-focused, but it's not a bad practice to prevent information from being accidentally revealed.
Activity
G4PLS commentedon Jun 25, 2025
Could you tell me where URLs are being exposed? Username makes sense as they are basically always visible, especially when there is a stacktrace.
But the username removal should be doable, for the URLs i would need more information
[-]Privacy in log files[/-][+][Feature Request] Privacy in log files[/+]codefaux commentedon Jun 25, 2025
The URL I've personally noticed is exposed from the Home Assistant plugin. I'm not sure how many other plugins interact with user infrastructure in this way, so I'm not sure if there are other places.
These errors occur every time my Home Assistant service is down, at minimum. I haven't looked through other logs and I literally just purged the folder to make diagnosing less cluttered, go figure.
Seems like a plugin log, so easiest would be for plugin developers to self-filter their logs but best practice would be for the logger itself to do the cleaning, reducing the onus on developers to remember to prune.
G4PLS commentedon Jun 25, 2025
Maybe we can introduce something in the settings that is a list of "banned" words where you can just add words that you want to have removed.
So while the log is then being written to the CLI/Log file those things get replaced beforehand. This way the end user has the most control of things that should be filtered
G4PLS commentedon Jul 2, 2025
Just wanted to give a small update to you.
As Im currently reworking the settings I have added the UI for the banned words! Currently they dont work yet but the UI is in place.
mfutselaar commentedon Jul 15, 2025
Relying on a "banlist" to block out passwords from logs is a terrible idea that begs to be exploited. You're basically allowing plain-text access to the passwords for anyone that has access to the configuration (through the software itself or by simply reading the configuration file).
This is something plugin developers need to handle, the best you can do is look for query strings such as
?password=123or&password=123, or when it comes to json search for things like{"password":"123"}.codefaux commentedon Jul 16, 2025
I have to say, I agree that putting a nice concise list of things you don't want people to see, all in one file, is not my idea of a cleverly implemented security feature.
I was hoping for more along the lines of, an intelligently written series of regex filters which apply basic security-minded logic to their filter sets.
The software can determine my username, and auto-filter that out. That's cake.
RFC3986 Appendix B provides a regex to process any URI, and nothing past the domain is likely to be sensitive -- but you could have an option to filter "domain" or "all" on URI references.
Retrieving the hostname an d network domain should be relatively straight-forward, and a filter could be written to keep that out of logs.
An IPv4 or IPv6 address is really easy to pick up, this could be an option to skip.
^((25[0-5]|(2[0-4]|1\d|[1-9]|)\d)\.?\b){4}$( ^^^^ IPv6 Regex. Yikes, but it seems functional)
Filesystem paths are easy enough to notice and strip or at least obfuscate.
All of these can be done without user attention.
I don't think the idea of coming up with a list of everything important you want someone not to find is even necessary, as most of the things I don't want someone to find won't be in the logs in the first place.
G4PLS commentedon Jul 16, 2025
We can just expand this list to just allow regex and normal strings
Then what we could do is:
I think the user should specify what filters they want as not everyone cares about safety, we could expand that system by having default filters that can be enabled/disabled (Like the regex for ipv4/6)
@codefaux what do you think?
G4PLS commentedon Jul 16, 2025
I also have to agree with @mfutselaar. Things like passwords etc should not be in logs anyways, this is something that plugin developers have to look out for.
I also dont see a reason why plugin devs would print a password in the log except they forgot to remove that when doing local testing
codefaux commentedon Jul 16, 2025
@G4PLS - Any means at all of passing the string through the filter pool would be acceptable. Steps 1 thru 6 can be summarized as a "foreach-style loop iterating against the list of filters and replacing text where necessary"..the replacement could be as simple as
...or-REDACTED-or whatever, that's not the heart of the debate here.I agree that passwords/etc are unlikely to be in a log, but if I run a
curlcommand manually, or publish an MQTT message, I may be forced to unexpectedly put the password in the plugin's fields somewherem and a generic plugin might not know I'm putting it there. Passwords/etc may wind up in logs. Plugin devs aren't likely to include them, but they seemingly are likely to not think about other things they shouldn't put in.Here's my bottom line though:
Protections should not be gatekept behind knowledge.
There should be a default set of protections -- username, URI, filesystem, IP addresses -- which are default enabled.
There should be a way to expand protections for users who have specific/unusual/unpredictable needs, and the knowledge to suit them.
Yes you won't catch every case, but the idea is to minimize what the user must know/do, and allow them to cover the gap.
G4PLS commentedon Jul 16, 2025
That is completely understandable, I think if I get the energy to do some additions I will go in and expand the feature by making everything into a "FilterRule" that can be enabled/disabled and has a name and the actual rule.
Then I can provide some default rules that get setup once (Like determining the username and creating a rule for the username).
As mentioned in the other issue I have no clue when im able to get to that as im working a lot on the SC rework and that is taking most of my energy at the moment
codefaux commentedon Jul 16, 2025
I hear you. I'm less presenting a list of expectations of you/of the immediate future, and more presenting what I would consider to be the guidelines I would expect of a developed product. It's best to know what you're painting before you get too much on the canvas, I believe some would say.
I offer my help with this or the SC rework project, in testing or whatever capacity is feasible. Energy is definitely at a premium lately, things are getting awfully ....heavy, all around. Scary times.
Checking out for now.
G4PLS commentedon Jul 17, 2025
The points you are making are very valid, I wouldnt implement something if I wouldnt think that it makes sense.
With the SC rework there will maybe be an announcement at some point but currently keeping it for myself as there is still lots to do before actually having a minimum running version.
fix(logging): redact home paths, usernames and URL credentials from a…
Core447 commentedon Aug 11, 2026
Added simple log redaction with d294cce. Won't catch everything but it's a start
Core447 commentedon Aug 11, 2026
Will close this for now because I believe this change is enough. If you think otherwise pls reopen
codefaux commentedon Aug 11, 2026
Looks like a pretty good foundation. Can't say what more might conceptually be required without some time on the ground. Good work.