[Developers] mqlread http response header (content-type)

Augusto Callejas acallejas at appliedminds.com
Thu Sep 18 20:35:03 UTC 2008 

the query you want to issue is actually:

===
{
  "master_property" : "/type/object/name",
  "operation" : "delete",
  "source" : {
    "guid" : "#9202a8c04000641f8000000006e31736"
  },
  "target_value" : {
    "value" : null
  },
  "timestamp" : "2008-08-30T18:56:23.0000Z",
  "type" : "/type/link",
  "valid" : null
}
===

the query you are issuing is getting the current ³name² value for that guid,
which doesn¹t have the control character in it.

the response you get will be:

===
{
  "master_property" : "/type/object/name",
  "operation" : "delete",
  "source" : {
    "guid" : "#9202a8c04000641f8000000006e31736"
  },
  "target_value" : {
    "value" : "Friesengeist 2: Regelm\u001aässige Zerstö\u001arungen"
  },
  "timestamp" : "2008-08-30T18:56:23.0000Z",
  "type" : "/type/link",
  "valid" : null
}
===

i know that json/mql allows utf-8 encoded characters.
my problem is with the mqlread HTTP response header Content-type, which is
returning char-set=²utf-8².
i contend that the the content type should be either ³application/json² or
³text/plain² WITHOUT char-set=²utf-8².
by not specifying the char-set, the ³\u001a² characters above would pass
thru the HTTP layer without being decoded,
and thus allow any JSON library to decode them properly.

there seems to be a conflation between encoding on the HTTP layer and
encoding on the JSON layer.

³\u001a² --(HTTP utf-8)--> string with one character 0x1a  --(JSON)-->
invalid character in JSON (should be encoded)

³\u001a²--(HTTP w/o utf-8) --> ³\u001a² --(JSON)--> string with one
character 0x1a 


augusto.

On 9/18/08 1:15 PM, "Tim Kientzle" <tim at metaweb.com> wrote:

> There are two problems with this:
> 
>   * I don't see how you're getting \u001a in your decoded text.  I've
> been experimenting with this URL:
> 
> /api/service/mqlread?query={"query":{"id":"/guid/
> 9202a8c04000641f8000000006e31736","name":null}}
> 
> In analyzing the byte values returned from this, I see valid UTF-8;
> none of the byte values are 0x1a and none of the UTF-8 sequences
> decode to anything that looks like \u001a.
> 
>   * \u001a is not forbidden in JSON.  JSON permits any Unicode
> character in JSON strings except for " and \ which have to be \-quoted.
> 
> What JSON library are you using?
> 
> Do you have a short standalone Java program that demonstrates this
> problem?
> 
> I'm going to do some more experiments to see if I can reproduce the
> problem you're seeing.  Any suggestions would help.
> 
> Tim
> 
> On Sep 18, 2008, at 12:05 PM, Augusto Callejas wrote:
> 
>> i don¹t mind if it was just ³text/plain², but the inclusion of char-
>> set=²utf-8² is a problem.
>> is that necessary?
>> 
>> the mql spec already specifies that it supports the utf-8 encoding.
>> to specify it in the content-type of a mqlread HTTP response seems
>> incorrect.
>> in my example i¹m getting back from mqlread:
>> 
>> ===
>> "value" : "Friesengeist 2: Regelm\u001aässige Zerstö\u001arungen"
>> ===
>> 
>> when i read this from the HTTP layer my http client will decode the
>> encoded unicode characters above, since the char-set of the HTTP
>> response
>> says ³utf-8².  but then when it comes time to interpret JSON from
>> the HTTP layer, the encoded characters are gone, and replaced with
>> the actual
>> values, which aren¹t parseable, since \u001a is a control character.
>> 
>> augusto.
>> 
>> On 9/18/08 11:53 AM, "Christine Weibel" <xtine at metaweb.com> wrote:
>> 
>>> Hmm.  mqlread did used to return a content-type of application/
>> json, but
>>> there was a deliberate change to text/plain a while ago.  it is my
>>> recollection that was done for some browser compatibility issue?
>> perhaps
>>> alecf remembers this. ----- Original Message ----- From: "Augusto
>> Callejas"
>>> <acallejas at appliedminds.com> To: "For discussions about MQL,
>> Freebase API and
>>> apps built on Freebase" <developers at freebase.com> Sent: Thursday,
>> September
>>> 18, 2008 11:39:29 AM (GMT-0800) America/Los_Angeles Subject: Re:
>> [Developers]
>>> mqlread http response header (content-type) according to the
>>> documenation: 
>>> http://www.freebase.com/view/guid/9202a8c04000641f800000000544e
>>> 139 ===== 4.2.2. mqlread Output mqlread returns an HTTP response
>> with a
>>> Content-Type header of application/json (or text/plain if the
>> callback
>>> parameter was specified). The body of the response is a JSON-
>> serialized
>>> envelope object that holds a MQL result object (or objects if
>> multiple queries
>>> were submitted). The format of mqlread response envelopes is
>> specified in
>>> Section 4.2.3 ===== however i did not specify a callback
>> parameter, so how do
>>> i get mqlread to return its response as "application/json" and not
>>> text/plain; charset="utf-8". thanks, augusto. On 9/17/08 9:36 AM,
>> "Augusto
>>> Callejas" <acallejas at appliedminds.com> wrote: > hi- > > when i
>> perform a
>>> mqlread query, i get back HTTP response headers that look > like:
>>>> ===== >
>>> Date: Wed, 17 Sep 2008 00:51:35 GMT > , > Server: Apache > , >
>>> X-Metaweb-Success: 1/1 > , > Content-Length: 35616 > , > Content-
>> Type:
>>> text/plain; charset="utf-8" > , > ... > Connection: Keep-Alive >
>> ===== > > >
>>> shouldn't the "Content-Type" header value be "application/json"? > >
>>> http://www.iana.org/assignments/media-types/application/ > > i'm
>> having a
>>> problem with the 'charset="utf-8"' in the header. > the java http
>> client
>>> library that i'm using ( > http://hc.apache.org/httpclient-3.x/)
>> reads the
>>> response (see example > below), and decodes any unicode characters
>> (ie.
>>> /u001a) into their actual > character value.  however, this is
>> before i get a
>>> chance to decode it in my > json library.  however, the unicode
>> characters
>>> have already been decoded at > the http level, but not at the json
>> level. > >
>>> any thoughts? > > thanks, > augusto. > > > query > ===== > {"q":
>> {"query": >
>>> { >   "master_property" : "/type/object/name", >   "operation" :
>> "delete", >
>>> "source" : { >     "guid" : "#9202a8c04000641f8000000006e31736"
>>>   }, >
>>> "target_value" : { >     "value" : null >   }, >   "timestamp" :
>>> "2008-08-30T18:56:23.0000Z", >   "type" : "/type/link", >
>> "valid" : null >
>>> } > }} > ===== > > > response > ===== > { >   "master_property" :
>>> "/type/object/name", >   "operation" : "delete", >   "source" :
>> { >     "guid"
>>> : "#9202a8c04000641f8000000006e31736" >   }, >   "target_value" :
>> { >
>>> "value" : "Friesengeist 2: Regelm\u001aässige Zerstö\u001arungen"
>>>   }, >
>>> "timestamp" : "2008-08-30T18:56:23.0000Z", >   "type" : "/type/
>> link", >
>>> "valid" : null > } > ===== > > >
>>> _______________________________________________ > Developers
>> mailing list >
>>> Developers at freebase.com >
>>> http://lists.freebase.com/mailman/listinfo/developers
>> ______________________
>>> _________________________ Developers mailing
>>> list Developers at freebase.com
>>> http://lists.freebase.com/mailman/listinfo/develo
>>> pers _______________________________________________ Developers
>> mailing
>>> list Developers at freebase.com
>>> http://lists.freebase.com/mailman/listinfo/develo
>>> pers
>> _______________________________________________
>> Developers mailing list
>> Developers at freebase.com
>> http://lists.freebase.com/mailman/listinfo/developers
> 
> _______________________________________________
> Developers mailing list
> Developers at freebase.com
> http://lists.freebase.com/mailman/listinfo/developers

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freebase.com/pipermail/developers/attachments/20080918/65d30b83/attachment.htm

More information about the Developers mailing list