[Developers] cursor returning only multiples of 100 results

Tue Jan 29 19:22:00 UTC 2008

After more investigation I found that it has something to do with the
query:

This query enumerates just fine (1735 results). 

[{

    "summary": null,

    "id": null,

    "type": "/user/cotton/default_domain/broadcast",

    "name": null

}]

This query always gives me multiples of 100 results: 

[{

    "summary": {"value":null},

    "id": null,

    "type": "/user/cotton/default_domain/broadcast",

    "name": null

}]

This looks like a bug to me.

From: developers-bounces at freebase.com
[mailto:developers-bounces at freebase.com] On Behalf Of Arthur van Hoff
Sent: Tuesday, January 29, 2008 10:21 AM
To: For discussions about MQL,Freebase API and apps built on Freebase
Subject: Re: [Developers] cursor returning only multiples of 100 results

I'm seeing this again, now on the live database. Here is a trace of the
first and the second request. The first one returns 100 results. The
second (which uses the correct cursor) is empty. If I repeat this I
sometimes get 100 or 500 or 400 results. Could it be that the cursor is
encoded incorrectly by the python library?

GET
/api/service/mqlread?queries=%7B%22q0%22%3A%7B%22query%22%3A%5B%7B%22nam
e%22%3A+null%2C+%22type%22%3A+%22%5C%2Fuser%5C%2Fcotton%5C%2Fdefault_dom
ain%5C%2Fbroadcast%22%2C+%22id%22%3A+null%2C+%22%5C%2Fcommon%5C%2Ftopic%
5C%2Farticle%22%3A+%7B%22optional%22%3A+%22forbidden%22%2C+%22id%22%3A+n
ull%7D%2C+%22summary%22%3A+%7B%22value%22%3A+null%7D%7D%5D%2C%22escape%2
2%3Afalse%2C%22cursor%22%3Atrue%7D%7D HTTP/1.1

Accept-Encoding: identity

Host: www.freebase.com

Cookie:
metaweb-user="A|u_avh|g_#9202a8c04000641f80000000042b3876|4.WF06h7rVjqT+
rRW/hryP1A";
metaweb-user-info=A|p_%2Fuser%2Favh|u_avh|g_#9202a8c04000641f80000000042
b3876|0.; mwLastWriteTime=1201630282; 30-Jan-2008 18:11:22 GMT

Connection: close

User-Agent: Python-urllib/2.5

HTTP/1.0 200 OK

Date: Tue, 29 Jan 2008 18:11:23 GMT

Server: Apache

X-Metaweb-Success: 1/1

Content-Length: 45557

X-Metaweb-Cost: dt=0.087, nreqs=2, tr=0.024, tu=0.024, ts=0.0, pr=0,
pf=0, dw=0, dr=5185, in=3264, ir=278, iw=0, va=4955, te=0.024,
tg=0.0282909870148, tf=0.0379531383514, tm=0.0634942054749, cr=1, cm=1,
ch=0, gqr=0

Content-Type: text/plain; charset="utf-8"

X-Cache: MISS from cache03.p01.sfo1.metaweb.com

Connection: keep-alive

Last-Modified: Tue, 29 Jan 2008 18:11:22 GMT

Cache-Control: public, max-age=1, s-maxage=5

Expires: Tue, 29 Jan 2008 18:11:24 GMT

X-Metaweb-TID: cache;cache03.p01.sfo1:8101;2008-01-29T18:11:22Z;0007

{

  "status": "200 OK", 

  "q0": {

    "cursor":
"eNpdjcEKwjAQRD_GS4OU7qbNbnYR8T_EQ2wbLagtaRX8e0W0onOc4c2rr2nskwI4p-HS6AJ
yREYHVDq1mnVj0G_l9dTGaZVnt25QRJSSBOw8i6bucJyWYsEGX0MFAFRh9PCTOl-_WMeejPl
TuI_icA6zg3neSaf70L4fiLgyZjukXmFXIFqAklmKDUiMT6KmfYPkWKxHeQBMgDpY", 

    "code": "/api/status/ok", 

    "result": [

        .. 100 results (deleted) ..

    ]

  }, 

  "code": "/api/status/ok"

}

Here is the second request:

GET
/api/service/mqlread?queries=%7B%22q0%22%3A%7B%22query%22%3A%5B%7B%22nam
e%22%3A+null%2C+%22type%22%3A+%22%5C%2Fuser%5C%2Fcotton%5C%2Fdefault_dom
ain%5C%2Fbroadcast%22%2C+%22id%22%3A+null%2C+%22%5C%2Fcommon%5C%2Ftopic%
5C%2Farticle%22%3A+%7B%22optional%22%3A+%22forbidden%22%2C+%22id%22%3A+n
ull%7D%2C+%22summary%22%3A+%7B%22value%22%3A+null%7D%7D%5D%2C%22escape%2
2%3Afalse%2C%22cursor%22%3A%22eNpdjcEKwjAQRD_GS4OU7qbNbnYR8T_EQ2wbLagtaR
X8e0W0onOc4c2rr2nskwI4p-HS6AJyREYHVDq1mnVj0G_l9dTGaZVnt25QRJSSBOw8i6bucJ
yWYsEGX0MFAFRh9PCTOl-_WMeejPlTuI_icA6zg3neSaf70L4fiLgyZjukXmFXIFqAklmKDU
iMT6KmfYPkWKxHeQBMgDpY%22%7D%7D HTTP/1.1

Accept-Encoding: identity

Host: www.freebase.com

Cookie:
metaweb-user="A|u_avh|g_#9202a8c04000641f80000000042b3876|4.WF06h7rVjqT+
rRW/hryP1A";
metaweb-user-info=A|p_%2Fuser%2Favh|u_avh|g_#9202a8c04000641f80000000042
b3876|0.; mwLastWriteTime=1201630282; 30-Jan-2008 18:11:22 GMT

Connection: close

User-Agent: Python-urllib/2.5

HTTP/1.0 200 OK

Date: Tue, 29 Jan 2008 18:11:23 GMT

Server: Apache

X-Metaweb-Success: 1/1

Content-Length: 139

X-Metaweb-Cost: dt=0.097, nreqs=9, tr=0.021, tu=0.012, ts=0.004, pr=0,
pf=0, dw=0, dr=4093, in=10456, ir=5582, iw=0, va=6808, te=0.022,
tg=0.0273525714874, tf=0.0359909534454, tm=0.0882680416107, cr=1, cm=1,
ch=0, lr=3, lm=3, lh=0, gqr=0

Content-Type: text/plain; charset="utf-8"

X-Cache: MISS from cache03.p01.sfo1.metaweb.com

Connection: keep-alive

Last-Modified: Tue, 29 Jan 2008 18:11:23 GMT

Cache-Control: public, max-age=1, s-maxage=5

Expires: Tue, 29 Jan 2008 18:11:24 GMT

X-Metaweb-TID: cache;cache03.p01.sfo1:8101;2008-01-29T18:11:23Z;0001

{

  "status": "200 OK", 

  "q0": {

    "cursor": false, 

    "code": "/api/status/ok", 

    "result": []

  }, 

  "code": "/api/status/ok"

}

From: developers-bounces at freebase.com
[mailto:developers-bounces at freebase.com] On Behalf Of John Giannandrea
Sent: Monday, January 28, 2008 11:46 PM
To: For discussions about MQL,Freebase API and apps built on Freebase
Subject: Re: [Developers] cursor returning only multiples of 100 results

Im unable to reproduce this.

I get 1767 broadcasts from sandbox with limit:100, and 18 requests.  30
were returned in the last set.

Can you please send us the X-Metaweb-TID headers from the HTTP requests
so we can investigate.

-jg

On Jan 28, 2008, at 10:57 PM, Arthur van Hoff wrote:

	I'm having a problem using cursors on the sandbox. I'm trying to
enumerate all results using a cursor, but I always get multiples of 100
results. Here is the query I'm using:

	"query": [{

	    "summary": {"value": null},

	    "/common/topic/article": {

	        "optional": "forbidden",

	        "id": null

	    },

	    "type": "/user/cotton/default_domain/broadcast",

	    "id": null,

	    "name": null

	}]

	I always get exactly 100 or 200 results after 1 or 2 iterations,
which can't be right. I would expect the last result set to contain
fewer than 100 results.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.freebase.com/pipermail/developers/attachments/20080129/9e68c2b8/attachment-0001.htm 