[Developers] Change in JSON output

[Alec Flett]

On Mar 4, 2009, at 10:55 AM, Kendra Kuhl wrote:

> Will,
>
> Any knowledge beforehand will work, but the more notice the better.

I want to offer my personal apologies as I was responsible for this  
particular change, without communicating. This one slipped through the  
cracks mainly because our QA tools rely solely on JSON parsers, and  
these parsers were returning us identical output in our tests.

For the pythonistas out there it was a switch from simplejson to  
jsonlib, for performance.

For most other developers, this release results in a 5-50% performance  
boost for /api/service/mqlread (5% for new/uncached queries, and 50%  
for cached queries)

> Will this be coming to the
> rest of your APIs? Right now the search API still returns the old  
> way. Will
> you be changing that too? And, is there a chance that it will get  
> changed
> back in one of the next release cycles?
>

Here is my personal take:
1) jsonlib uses escaped "/"s for security, because there are bugs in  
parsers, in shipping browsers like Firefox, even in FF 3.1 beta 2. For  
instance if you embedded raw, unescaped json in an html attribute, or  
as a JS variable declaration, you would introduce a security  
vulnerability in your code:
http://t3.dotgnu.info/blog/insecurity/quotes-dont-help.html

2) That said, we are still debating internally if we're going to keep  
this or not, and should have an answer by Friday, 3/6. We're trying to  
prevent mashup developers from shooting themselves in the foot because  
someone introduces a XSS attack into the freebase data, but there's  
obviously a limit to how much we can prevent, and how much the onus is  
on developers. I would be very curious if anyone here has any specific  
opinions:

a) "Let me shoot myself in the foot, I will write secure code"
b) "Thank you for closing this hole, my application was not safe!"
c) ?

3) We'll try to make all the APIs consistent, but one way or another  
you need to use a real JSON (or JS) parser, not try to do custom  
decoding, grepping, or string-matching against our APIs. At the moment  
the only guarantee we will make is that we output legal JSON that has  
a consistent semantic meaning AFTER parsing.

Alec

> Thanks,
>
> Kendra
>
> -----Original Message-----
> From: developers-bounces at freebase.com
> [mailto:developers-bounces at freebase.com] On Behalf Of Will Moffat
> Sent: Wednesday, March 04, 2009 12:25 PM
> To: For discussions about MQL, Freebase API and apps built on Freebase
> Subject: Re: [Developers] Change in JSON output
>
> Dear Kendra,
>
>> What is the best way for the developers to know that there is a
>> release
>> cycle coming up?
>
> That's something we need to formalize, I've filled a tracking issue:
> https://bugs.freebase.com/browse/FREEBASE-467
>
> Please let us know what you'd like to see.
> regards,
> --Will
>
> _______________________________________________
> Developers mailing list
> Developers at freebase.com
> http://lists.freebase.com/mailman/listinfo/developers
>
> _______________________________________________
> Developers mailing list
> Developers at freebase.com
> http://lists.freebase.com/mailman/listinfo/developers