I found that if the tag contains encodable characters such as '#' (which is quite commonly used for tagging!), the delete tag request neither succeeds nor throws an exception.