Lossless JSON Dates
24 October 2007JSON does not purport to be a lossless serialization of JavaScript objects or any other language’s objects. There are numerous data types that do not have a defined method of serialization and deserialization in JSON, like circular and multiple references, dates, classes, prototype chains, and more. There are ongoing efforts to define conventions for using JSON to define some of these forms of data, we have recently had discussions regarding references on JSON.com, which is perhaps one of the most important. Also, JSPON provides conventions for defining prototype chains, object identification, and arrays with string keys. However, another data form that is highly used, is a date. In the old days, JSON did not define how dates should be serialized, and the only convention I know that was somewhat disseminated was Microsoft’s JSON date format “@UTC-millisends@” (not big on readability). However, now ES4 and Crockford’s JSON library seemed to have agreed on using the ISO 8601 format with UTC timezone. Of course, the serialization is only half of the equation for serialization and deserialization, and out of the box json.js leaves strings alone. But, fortunately it is not difficult to test for ISO UTC dates, and convert them back to Date objects. In JavaScript, we can do this with just a few lines of code. Additionaly, Crockford’s json.js library accepts a filter, and a can easily write the date transformation as a filter:
function dateFilter(k, v) {
return (typeof v == "string"
&& (k=v.match(/([0-9]{4})-([0-9]{2})-([0-9]{2})T([0-9]{2}):([0-9]{2}):([0-9]{2})Z$/))) ? new Date(Date.UTC(k[1],k[2]-1,k[3],k[4],k[5],k[6])) : v;
}
Now you can parse JSON and have dates restored to JavaScript Date object, by simply calling myJSONString.parseJSON(dateFilter).
You can create objects with dates, serialize and deserialize the object, and retain your Date objects. You can certainly apply this code in situations where you aren’t using parseJSON with a filter as well. This same technique can also be applied in other languages. I can provide a Java example if desired.
Disclaimer: This is not perfectly lossless, this method does not preserve milliseconds, it rounds (or floors) to seconds, but the idea is to have automatic means of keeping a Date object. Also, I don’t claim to be the foremost expert on dates nor regular expressions, if there is a superior method for doing any part of the conversion, I would love to hear.
8 Responses to “Lossless JSON Dates”
October 25th, 2007 at 8:59 am
Kris,
I know that in XML, a schema just validates the document, but what if we made the conversion using a JSON Schmea extension? If It define an element as a date, then I can create a true JS Date. This would also enable me to create more complex elements: a restricted date, or a JSON object but with methods.
For security, we usually create JSON as objects but only as DATA - what if we could use the schema to link up existing functions?
Ric
October 25th, 2007 at 7:48 pm
Conversion could certainly be assisted by Schema information, however the intent of this snippet was simply to have an easy way to restore serialized dates to JavaScript Date objects. But you are correct that ideally one could be informed by a Schema about a date type which would be a cleaner solution than regex matching (which can yield false positives).
October 27th, 2007 at 12:54 pm
Date.UTC should be used for date conversion.
http://developer.mozilla.org/en/docs/Core_JavaScript_1.5_Reference:Global_Objects:Date:UTC
new Date(Date.UTC(k[1],k[2]-1,k[3],k[4],k[5],k[6]))
October 27th, 2007 at 5:07 pm
Hiroaki,
Excellent, thank you! I updated the code snippet to use the UTC method you suggested.
November 3rd, 2007 at 7:26 pm
It is good to have a standard date format, and ISO 8601 seems like a reasonable spec, but what about keeping it simple and storing dates in epoch seconds? That is a pretty standard format, and pretty much all languages have libraries to convert epoch seconds into a date object.
Also, in my experience, adding a scanner for things that “look like dates” and converting them to Date objects automagically is a bad idea. You are bound to have occasional strings that “look like dates” causing weird exceptions. My experience with the perl library SOAP::Lite is one such example, when users would enter in a description field that began with http://... that would be serialized into a URI object that would cause an exception in our Java service.
November 5th, 2007 at 1:07 am
Brian, epoch seconds is indeed easier to work with, however the web has shown a past precedent for preferring readability over efficiency, consider the dates used in HTTP. The choice of the format is really not mine, and I don’t really care that much. If you want it changed you should take that up with Douglas Crockford and the ES4 working group.
Scanning for strings that look like dates is certainly prone to false positives, you are correct. I will say that the date ISO-UTC date format is much stricter and less risky than “begins with http://“, however there is still a risk. Use it only if you are willing to take that risk.
January 18th, 2008 at 2:47 pm
I concur with Brian on this one. This only works if you have JSON Schema to tell you that a given string really is a date. Otherwise, the risk of false positives exists and it’s one that we didn’t want to take as a framework.
The Microsoft convention you mention is a little outdated. We moved away from that precisiely because of the risk of false positives.
Since then, we have found a better way that completely excludes false positives thanks to a loophole in the JSON spec.
Instead of making a long discussion of it here in a blog comment, I’ll write a short blog post about it and invite you to the discussion… Stay tuned…
January 18th, 2008 at 3:48 pm
Here’s the blog post:
http://weblogs.asp.net/bleroy/archive/2008/01/18/dates-and-json.aspx