Repository navigation
$http always transform response content '[abc}' to JSON even we already set resp content type as 'application/octet-stream' #10349
Description
As title, we ran into this issue when we try to get blob content from azure, using $http.get(url).then();. Angular would throw exception "Syntax error: Invalid ...". Seems Angular always try to render our blob content as JSON. After reading code in /src/ng/http.js, I think I may know why.
var APPLICATION_JSON = 'application/json';
var CONTENT_TYPE_APPLICATION_JSON = {'Content-Type': APPLICATION_JSON + ';charset=utf-8'};
var JSON_START = /^\s*(\[|\{[^\{])/;
var JSON_END = /[\}\]]\s*$/;
var JSON_PROTECTION_PREFIX = /^\)\]\}',?\n/;
function defaultHttpResponseTransform(data, headers) {
if (isString(data)) {
// strip json vulnerability protection prefix
data = data.replace(JSON_PROTECTION_PREFIX, '');
var contentType = headers('Content-Type');
if ((contentType && contentType.indexOf(APPLICATION_JSON) === 0 && data.trim()) ||
(JSON_START.test(data) && JSON_END.test(data))) {
data = fromJson(data);
}
}
return data;
}
In this piece of code, $http will try to detect response's content. If the contentType is equal to application/json, $http will transform the content to JSON object then return to users. It does make sense.
But if the contentType is not application/json, it would use (JSON_START.test(data) && JSON_END.test(data)) to identify whether the content is a JSON object. This is not accurate and not intuitive either. A string starts/ends with '[ { ] }' doesn't mean it's JSON, you can't even say it's like JSON.
For example, when you are writing a markdown article, maye API document, you may write a piece of content like below:
[link](some links...)
function bar:
function bar() {
if (a > b) {
foo()'
}
}
Obviously, it's not JSON while (JSON_START.test(data) && JSON_END.test(data) will return true. This kind of RegEx is weak.
BTW, if users set response content-type explicitly, like application/octet-stream or application/text-HTML, it means yeah I know I am returning you a byte array or a piece of HTML. But AngularJS still try to transform it to JSON, and booom, throw an exception...
For now I can only set the response type while making ajax calls: $http.get(url, {responseType: byteArray}). Can we remove this RegEx check and give users less bother like Jquery does?
Activity
formulahendry commentedon Dec 6, 2014
I've met with the same issue.
Strongly agree with rebornix, two things need to be improved:
pkozlowski-opensource commentedon Dec 6, 2014
This is a valid issue. There are number of things that we could improve in 1.3 (ex. making sure that starting and ending chars are the same in JSON-like strings, doing JSON-like detection only for certain response headers etc.), but the "proper" fix would consist of doing JSON-deserialization only if
application/jsoncontentTypeheader is present. But this would be a breaking change so we can't do this in 1.3.@caitp @petebacondarwin what do you guys think of fixing it in 1.4? I still do afraid that it would break applications for people interacting with non-well-behaving backends but yeh, at least we would avoid issues like this one.
petebacondarwin commentedon Dec 6, 2014
I hope this can be solved elegantly in 1.4.
caitp commentedon Dec 6, 2014
The content-type header was supposed to be respected in all cases in 1.3 --- if we keep saying it's too big of a breaking change, it's never going to be properly respected.
3 remaining items
caitp commentedon Dec 7, 2014
The only reason it's not a good idea is because you lose access to the rest of the $http API, but $templateRequest is just a thin wrapper around $http which removes the builtin JSON parsing transformer. (Well, it does some animation-specific other things, but these don't matter a whole lot).
While caching may be undesirable, it's easily circumvented
gkalpak commentedon Dec 7, 2014
Looking at the code, I can't really see how it could be circumvented at all. (It seems to be hardcoded into the
httpOptionsobject passed to$http.)So, the only reasons I can think of (and I wasn't even aware of the animation-specific stuff):
$templateRequestis supposed to be handling templates, any future changes (which might be appropriate for handling templates) might break non-template requests).Wouldn't it be a better idea to just remove the default transform ?
caitp commentedon Dec 7, 2014
By deleting it from the cache...
No, we have it for a good reason --- supported browsers don't all support responseType=json, and we wouldn't really know when to ask for responseType=json even if they did.
Anyone could write a similar thin wrapper around http, it's just an example --- realistically, it's not going to change, because it doesn't need to do anything special
gkalpak commentedon Dec 7, 2014
I didn't mean for us to remove the default transform. I meant for the user/developer (and not necessarily for all requests).
fix($http): only parse as JSON when opening/closing brackets match
fix($http): only parse as JSON when opening/closing brackets match
fix($http): only parse as JSON when opening/closing brackets match
petebacondarwin commentedon Dec 8, 2014
Using $templateCache is definitely not a valid workaround. It is not supposed to be used for this and as pointed out even adds wrinkles of its own.
The correct workaround, before this is fixed is to change the transform responses.
If you don't provide one for a request then $http uses the
$http.defaults.transformResponsearray. Out of the box it contains[defaultHttpResponseTransform]which is the offending function that does the "guessing" of JSON.If you provide a transformResponse property on your $http request object then this will be used instead of the default.
So your options are to provide a custom transform per request or replace
$http.defaults.transformResponsewith your custom transform.See https://docs.angularjs.org/api/ng/service/$http#transforming-requests-and-responses
fix($http): only parse as JSON when opening/closing brackets match
fix($http): only parse as JSON when opening/closing brackets match
caitp commentedon Dec 8, 2014
It is a valid workaround --- it does not add any wrinkles --- If you were to use $templateRequest (which deals with removing the transform) in addition to removing the cached value on resolve, the only issues you're left with is not having the exact same API.
The real best solution is just writing a thin wrapper service around $http native to your own app, but the point here is that you can use $templateRequest as a guide for doing this. This ain't rocket science folks, there are literally only two constraints here, and they're trivial to implement.
fix($http): only parse as JSON when opening/closing brackets match
fix($http): only parse as JSON when opening/closing brackets match
fix($http): only parse as JSON when opening/closing brackets match