Is It Too Late To Change JSON?

0 comments suggest edit

In my last post, I wrote about the hijacking of JSON arrays. Near the end of the post, I mentioned a comment whereby someone suggests that what really should happen is that browsers should be more strict about honoring content types and not execute code with the content type of application/json.

I totally agree! But then again, browsers haven’t had a good track record with being strict with such standards and it’s probably too much to expect browsers to suddenly start tightening ship, not to mention potentially breaking the web in the process.

Another potential solution that came to mind was this: Can we simply change JSON? Is it too late to do that or has that boat left the harbor?

boat-left-harbor

Let me run an idea by you. What if everyone got together and decided to version the JSON standard and change it in such a way that when the entire JSON response is an array, the format is no longer executable script. Note that I’m not referring to an array which is a property of a JSON object. I’m referring to the case when the entire JSON response is an array.

One way to do this, and I’m just throwing this out there, is to make it such that the JSON package must always begin and end with a curly brace. JSON objects already fulfill this requirement, so their format would remain unchanged.

But when the response is a JSON array, we would go from here:

[{"Id":1,"Amt":3.14},{"Id":2,"Amt":2.72}]

to here:

{[{"Id":1,"Amt":3.14},{"Id":2,"Amt":2.72}]}

Client code would simply check to see if the JSON response starts with {[ to determine whether it’s an array, or an object. There many alternatives, such as simply wrapping ALL JSON responses in some new characters to keep it simple.

It’d be possible to do this without breaking every site out there by simply giving all the client libraries a head start. We would update the JavaScript libraries which parse JSON to recognize this new syntax, but still support the old syntax. That way, they’d work with servers which haven’t yet upgraded to the new syntax.

As far as I know, most sites that make use of JSON are using it for Ajax scenarios so the site developer is in control of the client and server anyways. For sites that provide JSON as a cross-site service, upgrading the server before the clients are ready could be problematic, but not the end of the world.

So what do you think? Is this worth pursuing? Not that I have any idea on how I would convince or even who I would need to convince. ;)

UPDATE: 6/26 10:39 AM Scott Koon points out this idea is not new (I didn’t think it would be) and points to a great post that gives more detail on the specifics of executable JSON as it relates to the ECMAScript Specification.

Found a typo or error? Suggest an edit! If accepted, your contribution is listed automatically here.

Comments

avatar

24 responses

  1. Avatar for secretGeek
    secretGeek June 25th, 2009

    Thought, i'd like to withhold judgement until i see Doug Crockford's retort (or John Ressig's) it certainly sounds neat to me!
    And Always Remember:
    Breaking the Web is nothing to be afraid: it's What We Do For A Living!

  2. Avatar for Roger Pence
    Roger Pence June 25th, 2009

    Where do you draw the line? JSON also needs its use of dates fixed. Is it too soon to think about a an updated JSON RF?

  3. Avatar for John Millikin
    John Millikin June 25th, 2009

    Not a good idea -- if you do this, then JSON no longer matches JavaScript syntax. It also complicates the parser with several special cases, and makes auto-detecting the encoding more difficult.
    A better solution, which retains backwards compatibility, is to modify server-side libraries so that they will refuse to serialize arrays as the outermost container. This solution is also easier to implement, because only one half of the system has to be modified.

  4. Avatar for Eber Irigoyen
    Eber Irigoyen June 25th, 2009

    I think changing the format complicates matters too much, I think there are much simpler ways to prevent JSON execution, on the other hand, I'm all in for using JSON to replace stupid XML all over the place

  5. Avatar for Scott
    Scott June 25th, 2009

    So, this seems to be fixing the symptom instead of the disease. Crockford has a proposal for a JSONRequest object.
    http://www.json.org/JSONRequest.html
    "JSONRequest does not send or receive cookies or passwords in HTTP headers. This avoids false authorization situations. Knowing the name of a site does not grant the ability to use its browser credentials."
    There was a big discussion back in 2007 about this as well. The general consensus was the same conclusion you reached. JSON responses should start with a {.
    http://robubu.com/?p=25
    I think the key thing to take away from this is: Don't use eval as a JSON parser. Use a JSON parser. jQuery has a plugin that I've used on a couple of sites with great success. YUI has built in support for it and there is an Open Source version of one at json.org.
    http://www.json.org/json2.js

  6. Avatar for Scott
    Scott June 25th, 2009

    I am a little confused with some thing. Reading you last post and this post it seems like the problems only exist when the response is only a json array. Why not just make it a general requirement that a response must always be a json object. It seems like that is what this post is kind of doing by giving a json arroy the same type of syntax as an object. If that the case just return an object every time.
    I am sure there is some sort of negative to this because no one seems to talk about it, but no one seemed to point out that negative that I was able to see.

  7. Avatar for Haacked
    Haacked June 25th, 2009

    @John the fact that it doesn't match JavaScript is not a problem. It's still very close. After all, you should NEVER EVER eval JSON. That would open you up to all kinds of hurt. That's why you'll see that the major JSON libraries all have some sort of ParseJson method to actually parse the format.
    @Scott Koon thanks for the links man! I figured I wasn't the first to come to this conclusion, but I didn't find those posts.
    @Scott #2 An array is technically a JSON object though according to the spec. Returning a JSON array seems to be a common enough operation that to suddenly say you can't return JSON arrays seems pretty harsh. My proposed solution is to simply wrap the array in a standard way that everyone agrees upon.
    ASP.NET ASMX Web Services and WCF Web Services already do this by wrapping objects in an object with a property named "d". Thus your array response would look like:
    {'d':[{obj1:val1, obj2:val2}]}
    That solves the problem too, but it's not a standard. I'd like to see some sort of standard codified into the JSON spec.

  8. Avatar for Andrey Shchekin
    Andrey Shchekin June 25th, 2009

    I would say that the best solution lies on browser side. My understanding is that Array constructor is already blocked, so the browser vendors should just block property definitions on Object.prototype (it is a bad practice to extend it anyway).

  9. Avatar for Haacked
    Haacked June 25th, 2009

    @Andrey the flexibility of JavaScript is one of the features that has earned it many fans. I think constraining the language could be problematic.
    Also, you're relying on waiting for new browser versions to stop supporting this feature. Old browsers continue to remain vulnerable.
    One nice benefit of the approach I outlined is that as soon as you update your JSON service to the new format, old browsers are no longer vulnerable.

  10. Avatar for Scott
    Scott June 26th, 2009

    So, even if the array is wrapped and not executed, the attacker still has access to the text node of the script block that was successfully delivered contained all your secret balance information. At that point they can just either
    1) strip the wrapping {} and access the array via eval
    2) Just text parse out the info they want.
    The big security hole here is the script tag. It violates the same-source rule and executes the script as soon as it's loaded.

  11. Avatar for Bryan
    Bryan June 26th, 2009

    Why is the link to your blog via the weblogs.asp.net feed forcing everybody to login?

  12. Avatar for McKamey
    McKamey June 26th, 2009

    I think the cleanest solution is to simply require POST for JSON requests. This avoids the hijacking scenario from script tags and allows the syntax to remain unchanged. The silly { d:[] } wrapper can go as well and we are back to the original, easy to read, flexible JSON syntax.
    This is just one potential vulnerability yet the proposed solutions all impose a significant change. Browsers will catch up and fix Arrays but in the mean time, requiring sensitive JSON service calls to be POST is not a big deal. Brain-dead simple to check on the server side. Make it the default and then make the developer explicitly choose to enable GET.

  13. Avatar for ryan
    ryan June 26th, 2009

    JSON is perfect the way it is. It is about simplicity and only objects and arrays. It works on any platform and needs no new features added. Any developer that uses JSON can decide for him/herself how to validate it, check for security etc.
    The beautiful thing about JSON is it is not XML bloat. XML was a decent idea as well until everyone wanted more and more bloat added. JSON is perfect in it's most base/simple format and that is what it is. No change needed. JSON is a platform/structure you build on top of. KISS.

  14. Avatar for Douglas Crockford
    Douglas Crockford June 26th, 2009

    It is too late to change JSON. JSON is fixed forever. This gives it stability which is one of its most important features.
    You are attempting to solve the wrong problems. There are two problems here. The first is that array literals can be intercepted. This is fixed in ES5.
    The second problem is that you are depending on ambient authority and the Same Origin Policy to protect your data. That is a mistake. Your server should never put bits on the wire unless it intends that they be delivered. This should not be delegated to the browser. It is the server's responsibility.
    Requiring POST is a good suggestion. Avoiding the use of top level arrays is also a good suggestion. Take responsibility for the security of your data.

  15. Avatar for Haacked
    Haacked June 26th, 2009

    @Scott no, the attacker doesn't. Browsers don't allow for that. If you don't believe me, prove your assertion. Demonstrate an attack that unwraps this JSON array.
    This attack hinges on the fact that the response returned to the script tag is valid JavaScript. Starting a statement with a curly brace is not valid JavaScript, thus it doesn't execute. Instead, you get a script error, so the attacker's JavaScript is unable to continue to execute.

  16. Avatar for Haacked
    Haacked June 26th, 2009

    @Douglas First of all, let me say I watched all your JavaScript videos on YUI Theater and bought your book and am a big fan. It gave me a deep appreciation for JavaScript I didn't have before. Thus this video sums up my feeling of having you comment on this topic. I hope you actually return to read this comment. :)
    I'm happy to hear this issue is fixed in EcmaScript 5, though I worry it may be a while before we see that adopted.
    Regarding your second point, I agree. However, as I pointed out, there's a very subtle variant of the attack made possible by caching implementation.
    For example, the bits will always be delivered at some point for legitimate reasons, or why even have the service? So the unsuspecting user visits the vulnerable site and legitimately requests the JSON array. At that point, the data is potentially cached by buggy browsers and buggy proxy servers. When the browser makes the request initiated by the SCRIPT tag, the server is no longer involved in fulfilling that request.
    Given that scenario, how would securing the data on the server have any affect? The data was delivered long before in response to a legitimate authenticated request.
    It may seem I'm hypothesizing here, but these are based on real world scenarios that the ASP.NET team has experienced. As framework developers, we want to find the right balance of usability and helping developers protect themselves.
    I agree that developers should take responsibility for their data, but as you can see in the comments to this post and the previous post, this particular issue is very subtle and even now, most developers (who are very smart) don't have a grasp of it. After all, it bit the Gmail team in the past. If Google got bit, how can we expect us mere mortals to get it right every time. Because the hackers definitely understand the issue, and it only takes one of them.
    This is why I'm writing about the subject and posing hypothetical solutions in the interest of testing my own knowledge of the issue, as well as a means of making people aware of it. As for the ASP.NET MVC framework, we may end up with requiring POST with an option to switch to GET for those who know what they are doing as there doesn't seem any other safe option.

  17. Avatar for Louis DeJardin
    Louis DeJardin June 26th, 2009

    Sounds very well reasoned. Would there still be room to consider a middle-ground default setting of requiring POST only on arrays?
    If there's a POST requirement on all JsonResponse you have two choices:
    * switch all of your client-side invocation to POST
    * disable the enhancement
    However if it's only a POST requirement on array responses a few more options become available:
    * switch only the invocation of array requests to POST
    * change only the array responses to object
    My concern about the first case would be if people under time constraints, especially with an existing app, would simply flip the switch off to make things work. Sort of the penny in the fusebox, run-as-admin, reaction. However they may have a comparatively acceptable number of array responses to correct.
    Just throwing that out there. I do agree there's no reason to use GET over POST in new projects, and it would be entirely reasonable to have new web configs dialed up to a strict json policy.

  18. Avatar for Brad Wilson
    Brad Wilson June 26th, 2009

    I think the problem with "only require POST for arrays" is that it becomes a "less obvious" point of failure... "some JSON thing was working just fine until we changed it to return an array of objects and now it stopped working!"

  19. Avatar for Haacked
    Haacked June 26th, 2009

    Yeah, I agree with Brad. Sometimes you can get too cute with these things and just end up confusing everybody.
    My hope is that we can seed the net ahead of time with the "fix" being to use POST in your client library, which is not that hard, so that people who encounter the problem are more likely to use that solution rather than turning off the option.
    At least that's the hope. :)

  20. Avatar for JC Grubbs
    JC Grubbs June 26th, 2009

    One quick change to ASP.NET MVC and you can prevent IEnumerable from being passed in to JsonResult. I just throw an exception with a warning that array's should not be the top level container for JSON packages. Seems to work pretty well as far as constraining developers and it doesn't require changes to JSON itself. Personally I think this is the best kind of approach. We don't need to change the specification just make it harder to misuse.

  21. Avatar for Scott
    Scott June 27th, 2009

    I don't really know how requireing post makes sense. I have to look at some api's that I have been currently playing with to see how vunerable all these things are. Since they don't require post for everything and I don't now if it is an array or if it is wrapped in an ogject or what.
    I would think that it would be easier that if you are dealing with truly sensitive information either don't use json or wrap it into an object.
    It actually seems to me just as harsh to say. I am going to require POST to get data as it would to docuent my api and say my respose is an object.
    I don't really see how either is that harsh. But, the POST just doesn't seem to make sense to me. Post,Push,Delete,Get all seem to carry meaning in the restful api's that I have seen and just saying everything is goingt to be a post seems to take away from that.
    Anyway, this is very interesting to me. And I am sure that everyone posting here knows alot more than I do about the subject.

  22. Avatar for Scott
    Scott June 30th, 2009

    I thought you could still access the script nodes text even if the parser failed, it looks like you can't access it at all in the case where you are loading a script via the src attribute. My mistake. So it looks like wrapping the array with brackets and/or ensuring that the sensitive data is only retrieved via a POST request will work.
    However, my prescriptive guidance would still be to not send sensitive data in a JSON message via a GET. In fact, I've always avoided sending any kind of sensitive information via JSON at all.
    Good catch, this stuff often gets left to the wayside in the rush to sling JSON and JavaScript around.

  23. Avatar for jaco
    jaco December 17th, 2009

    JSON should not be changed period! Let me decide what and if I need to do something. I do not want to use something that does not work as expected... Quit trying to change things to save a few...

  24. Avatar for Ryan
    Ryan January 6th, 2010

    @Douglas: Don't you have to trust the browser at some point? Even if the server doesn't release data unless a secret was posted, you still have to send the secret to the page. How can you deliver the secret in a way that is more secure than just sending the confidential data in a JSON object?