Well, that's floating-point arithmetic for you.  log(x, y) simply computes 
log(x)/log(y) behind the scenes; since both log computations and the 
floating-point division can introduce errors, the result will frequently 
not be correctly rounded.

I don't really see the benefit of special-casing log(x, 10).  In what 
circumstances does it matter that log(x, 10) != log10(x)?  I could 
understand people being upset that log(10**n, 10) doesn't return n 
exactly, but that's what log10 is there for.

See also the discussion in issue 3724.