[Developers] Steps Toward a Complete History Export in Freebase

[Kurt Bollacker]

On Mon, Jun 01, 2009 at 05:38:56PM -0700, Tyler Pirtle wrote:
> Kurt Bollacker wrote:
> > That won't show deleted or updated links.  In fact I think it's
> > impossible to get the history without significant post processing of
> > MQL results.  I believe there are outstanding bugs related to this.

> Kurt, if I missed something, let me know.

The problem is in specifying the "valid" property of links.  Live
properties are "valid":true, replaced properties are "valid":false,
and deleted properties are "valid":null.  So it's not possible to ask
for deleted links directly.  Instead, I have to get back all links
from the user and sift through to find the deleted ones.  If the user
made 100000 inserts and only a handlful of deletes, I call this
"significant post processing".

Even worse, if I try to ask for updated or valid links by specifying a
value for "valid", as in the query:


[{
  "type":          "/type/link",
  "creator":       "/user/kurt",
  "source": {
    "*": null
  },
  "target": {
    "*": null
  },
  "timestamp":     null,
  "operation":     null,
  "valid":         false,
  "master_property": null,
  "target_value":  null,
  "sort":          "-timestamp"

}]

I get the lovely response:
 {
  "code":          "/api/status/error",
  "messages": [{
    "code":    "/api/status/error/mql/graph",
    "info": {
      "detail":   "duplicate assignment to "live" flag",
      "subclass": "SEMANTICS"
    },
    "message": "error SEMANTICS: duplicate assignment to "live" flag"
  }],
  "status":        "200 OK",
  "transaction_id": "cache;cache01.p01.sjc1:8101;2009-06-02T05:27:31Z;0005"
}

I call that a bug.  If there's an easier way to invalid links without
having to get all of the valid ones, please let me know.


							Kurt :-)