[Developers] cursors, limits, MJT and chocolates

[Kurt Bollacker]

On Mon, Jul 09, 2007 at 03:56:09PM +0200, Will Moffat wrote:
> Hi,
> 
> My map of freebase users is starting to give timeouts. Before I dive
> in and start hacking I wonder if anybody could save me some time:
> 		
> If I use a cursor can I still get a timeout?

Yes, if the query is complex enough or is returning too many
sub-results.

> Or does the use of a cursor guarantee a (possibly empty) set on each call?

Currently, there's no guarantee of avoiding a timeout.  However, it's
not likely if you break your results into small enough pieces.

> Can I use 'limit' to fix the set size with a cursor?

When using cursors, "limit" at the top level of a MQL query acts like
you think "pagesize" should.  You can reduce the "limit" value to the
point where timeouts no longer occur.

{
 "query":
     [{"id":null,
       "name":null,
       "type":"/music/artist",
       "limit":10}],
 "cursor":true
}

Will iterate through music artists, 10 at a time.  When you get a
cursor, place it back into the "cursor" clause to get the next page.

It timeouts still occur when you set "limit":1, then you probably have
a very complex query or one that is trying to return a large number of
sub-results (e.g. iterating through publishers and giving all of their
books published as a sub-result).  If your query is simple, returns a
small amount of data, and "limit":1 still gives you a timeout, then we
have a bug somewhere and this should be reported.

> Or does that refer to the total result limit?

It's only the total limit when there are no cursors or "limit" appears
somewhere in the MQL query other than the top level.
 
> Extra Belgian chocolates if somebody has an example of using the
> cursor feature with MJT.

This should be possible, but I haven't done it. Nix?

							Kurt :-)