[Developers] WEX (wpid/fbid lookup) database

[Alexander Marks]

Hey Sam. We're working on a new WEX dump that will be out soon. Note that you can, however, generate the guid->wpid map that you want using the quad dump with something like this python script:

  dump = open("freebase-datadump-quadruples.tsv", "r")
  out = open("guid2wpid.tsv", "w")
  for line in dump:
      src, prop, dst, val = line.split("\t")
      if prop == "/type/object/key" and dst == "/wikipedia/en_id":
          out.write("%s\t%s\n" % (src, val))

which will give you a format like this:

  /guid/9202a8c04000641f8000000000009e89 3746
  /guid/9202a8c04000641f8000000000032ded 25493
  ...

As for Wikipedia article redirects, you will always need the WEX dump for that, although replacing /wikipedia/en_id with /wikipedia/en above might get you part of the data you want. The /wikipedia/en Freebase namespace contains both the name of the Wikipedia article, and the name of all redirects to that article.

Hope that helps,

Al

----- Original Message -----
From: "Sam Halliday" <sam.halliday at gmail.com>
To: developers at freebase.com
Sent: Tuesday, November 4, 2008 2:30:09 AM GMT -08:00 US/Canada Pacific
Subject: [Developers] WEX (wpid/fbid lookup) database

Hi all,

I noticed that freebase released a new database dump for October, but  
no corresponding WEX dump. My interests are not in the WEX part of the  
latter database, but in the piece that links wpids to fbuids and  
gathers all the Wikipedia redirects... I am confused why this is not a  
part of the freebase download in the first place.

I used freebase for a project on the assumption that new data would be  
available on a quarterly basis. Is this not the case for the WEX data?

I'd also like to request that the wpid and redirect tables be included  
in the freebase data dumps in the future, and not exclusively in the  
WEX data. I understand that the WEX can be built from the wikipedia  
data, but the wpid/fbuid lookup cannot... it must be freebase that do  
this.
_______________________________________________
Developers mailing list
Developers at freebase.com
http://lists.freebase.com/mailman/listinfo/developers