[Developers] Comparison operation in write?
Arthur van Hoff
AVH at zing.net
Tue Jan 15 23:18:40 UTC 2008
Hi Tim,
One of the things that confuses me about the write semantics is that the
query is partially a read followed by a write if there is a match to a
single object. Clearly you are doing a read in order to find the target
object using the id, name, and structure matching.
It was very surprising to me that these semantics do not carry over
seamlessly into writing multiple objects. Updating the broadcaster for a
set of podcasts seemed a reasonable thing to do in a single request.
So, if I am forced to use multiple requests, then I need at least
transactions, so that I can guarantee the atomicity of the operation
over multiple reads and writes.
Alternatively I can try to do a read, followed by a single write to
multiple objects. Again I would need transactions, and I would need a
fix for the "create":"unless_exists" problem (see the other thread on
writing). Also, I've been told that large writes can timeout. How do I
deal with that?
I am trying to write a robust application, but it is too hard!
By the way, I can clearly see how writing to many objects could be
abused, but I that is a somewhat weak argument against it. Perhaps this
should be addressed using user based permissions.
Have fun,
Arthur van Hoff
-----Original Message-----
From: developers-bounces at freebase.com
[mailto:developers-bounces at freebase.com] On Behalf Of Tim Sturge
Sent: Tuesday, January 15, 2008 2:22 PM
To: For discussions about MQL,Freebase API and apps built on Freebase
Subject: Re: [Developers] Comparison operation in write?
Hi Arthur,
This a difference in write philosophy with SQL DELETE/UPDATE on one side
and MQL and SQL INSERT on the other. SQL only allows you to insert one
row per VALUES clause, but you can run "DELETE FROM foo WHERE 1=1" with
the predictable consequences.
MQL decided to treat all write operations the way SQL does INSERT. That
is, every operation must be individually specified. To get around the
question of how to do multiple operations, MQL makes a write look the
result of a read. So suppose you want to do the update you have below.
The easiest way is to run the query you have as a read (I've added "id":
null so the write can uniquely identify all the stations returned)
"query":[{
"type": "/user/cotton/default_domain/internet_station",
"id": null,
"internet_streams": [{
"uri~=": "*live365*",
"uri":null,
"type": "/user/cotton/default_domain/internet_availability"
}],
"/user/cotton/default_domain/broadcast/broadcaster": null
}]
You get back a list of results, and you do something like:
for result in results:
result["/user/cotton/default_domain/broadcast/broadcaster"] = {
"id":"/topic/en/live365", "connect":"update" }
and then submit the whole thing back as a write. (note that the read
result does not contain the ~= operator, so you don't need to remove
it.) If it's too big you can cursor the read and do the write in
batches.
SQL has acclimatized people to the idea of bulk write, but almost no
other language contains that sort of functionality (try zeroing an array
in JAVA without a loop). Particularly on a public database bulk
operations are a nightmare, as:
[{ "id": null,
"type": [{ "id": null, "connect": "delete" }]
}]
(remove all types from all objects) shows so I'm pretty sure we're
better off without it.
Tim
Arthur van Hoff wrote:
> What do you mean legal MQL? This query works just fine as a read
query:
>
> "query":[{
> "type": "/user/cotton/default_domain/internet_station",
> "internet_streams": [{
> "uri~=": "*live365*",
> "uri":null,
> "type": "/user/cotton/default_domain/internet_availability"
> }],
> "id": null,
> "name": null
> }]
>
> Also, what do you mean [{.....}] vs [{...},{...},{...}]?
>
> Since multiple queries are not transactional I would like to write a
> SINGLE query that finds all internet_stations that contain live365 in
> the URL and update their broadcaster field. That should be possible,
> right?
>
> -----Original Message-----
> From: developers-bounces at freebase.com
> [mailto:developers-bounces at freebase.com] On Behalf Of Kurt Bollacker
> Sent: Sunday, December 23, 2007 8:01 PM
> To: For discussions about MQL,Freebase API and apps built on Freebase
> Subject: Re: [Developers] Comparison operation in write?
>
>
> On Sun, Dec 23, 2007 at 06:30:50PM -0800, Arthur van Hoff wrote:
>
>> Here is an example: set the broadcaster for all stations that have
>> live365 in a stream URL.
>>
>> "query":[{
>> "type": "/user/cotton/default_domain/internet_station",
>> "internet_streams": [{
>> "uri~=": "*live365*",
>> "uri":null,
>> "type": "/user/cotton/default_domain/internet_availability"
>> }],
>> "/user/cotton/default_domain/broadcast/broadcaster":{
>> "id":"/topic/en/live365",
>> "connect":"update"
>> }
>> }]
>>
>
> Before I address what I think you are asking for, let me mention that
> MQL write syntax requires that arrays of writes be explicit. In your
> query, you are using the MQL read syntax of [{.....}] to request an
> array result. You should be using [{...},{...},{...}] to specify an
> array read. Also, this particular example also falls afoul of MQL
> write rules by asking for a literal value ("uri") in a write. Could
> you give an example of the query you need that only requires
> additional comparison operators, but is otherwise legal MQL? That
> might help me understand more clearly.
>
> As a workaround, you can break this into a read then a write query
> without any loss of expressiveness. If that won't work, let's try to
> figure something out that does.
>
> Kurt :-)
>
>
>> The result is:
>>
>> {
>> "status": "200 OK",
>> "code": "/api/status/error",
>> "messages": [
>> {
>> "info": {
>> "mode": "write",
>> "key": "uri~="
>> },
>> "query": [
>> {
>> "/user/cotton/default_domain/broadcast/broadcaster": {
>> "id": "/topic/en/live365",
>> "connect": "update"
>> },
>> "type": "/user/cotton/default_domain/internet_station",
>> "internet_streams": [
>> {
>> "type":
>> "/user/cotton/default_domain/internet_availability",
>> "uri~=": "*live365*",
>> "uri": null,
>> "error_inside": "."
>> }
>> ]
>> }
>> ],
>> "message": "Can't use comparison operators in a write, only in
a
>> read",
>> "code": "/api/status/error/mql/parse",
>> "path": "internet_streams"
>> }
>> ]
>> }
>>
>> -----Original Message-----
>> From: developers-bounces at freebase.com
>> [mailto:developers-bounces at freebase.com] On Behalf Of Kurt Bollacker
>> Sent: Sunday, December 23, 2007 6:22 PM
>> To: For discussions about MQL,Freebase API and apps built on Freebase
>> Subject: Re: [Developers] Comparison operation in write?
>>
>>
>> On Sun, Dec 23, 2007 at 05:46:46PM -0800, Arthur van Hoff wrote:
>>
>>> Hi,
>>>
>>>
>>>
>>> Why can't I use a comparison operation in a write? I need to fix a
>>>
>> bunch
>>
>>> of objects that contain a particular hostname in a URL. I would seem
>>> reasonable to do this in a single operation.
>>>
>> Can you give an example of this?
>>
>> Kurt :-)
>>
>>
>>
>>>
>>>
>>> Have fun,
>>>
>>>
>>>
>>> Arthur van Hoff
>>>
>>>
>>>
>>> ___________________________________________
>>>
>>> Arthur van Hoff, Dell-ZING, CTO Software and Services.
>>>
>>> W: 650-267-2413, F: 650-267-2498, M: 650-283-0842
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Developers mailing list
>>> Developers at freebase.com
>>> http://lists.freebase.com/mailman/listinfo/developers
>>>
>> _______________________________________________
>> Developers mailing list
>> Developers at freebase.com
>> http://lists.freebase.com/mailman/listinfo/developers
>> _______________________________________________
>> Developers mailing list
>> Developers at freebase.com
>> http://lists.freebase.com/mailman/listinfo/developers
>>
> _______________________________________________
> Developers mailing list
> Developers at freebase.com
> http://lists.freebase.com/mailman/listinfo/developers
> _______________________________________________
> Developers mailing list
> Developers at freebase.com
> http://lists.freebase.com/mailman/listinfo/developers
>
_______________________________________________
Developers mailing list
Developers at freebase.com
http://lists.freebase.com/mailman/listinfo/developers
More information about the Developers
mailing list