[Freebase-discuss] Pain point / Freebase list importer UI
stefano at metaweb.com
Thu Jan 28 02:11:05 UTC 2010
On 1/27/10 4:54 PM, Tom Morris wrote:
> On Wed, Jan 27, 2010 at 7:45 PM, Stefano Mazzocchi<stefano at metaweb.com> wrote:
>> On 1/27/10 3:59 PM, Kirrily Robert wrote:
>>> The list importer seems like a really good candidate for rewriting as
>>> an Acre app. Is anyone interested in having a shot at this? If we
>>> come up with one that works better than the current one, we could
>>> probably replace any links in the client (eg. on the topic page, view
>>> page, etc) with links to the app in question, and remove the client
>>> list importer.
>> I'll be taking a shot at this next week.
> I'm confused. I thought the Spreadsheet Loader was the intended
> replacement for the Import List tool.
> How do these various solutions relate to each other?
Peter and I had a long conversation about this today and while it is
obvious that 'data loading' tools that follow the same guidelines will
always have some overlap, I think it's valuable to try different
solutions to the loading problem.
In particular, I want to focus my attention on exploring more the social
and mutual-assistance aspect of data loading.
I believe that the act of adjusting and massaging data to align well
with Freebase's and the act of reconciling individual topics with
Freebase's are separate tasks and require substantially different skills.
Even more importantly, the reconciliation tasks require much less
knowledge of how Freebase operates and can be easily parallelized (we
know all this from the data games and odeskers)
Freebase Loader, in its current form at least, is focusing on being a
tool for an individual that wants to load a dataset and Peter is
spending a great deal on energy to make that even better than it
Because of all the work I've done with RABJ and data games over the last
few months, I want to try to tackle the list importing problem (reusing
code where it makes sense, of course) but coming at it from the social
I have been building similar apps for internal use and I think I can
whip something up very quickly by reusing most of that code, then show
it raw and unfinished here and iterate from there with suggestions from
everybody that ends up caring (also, I'll be working in Acre, so it will
be easier for others to change things and experiment on their own if
they feel like it).
Ultimately, I believe that data loading interests (both internal and
external) will not be served well by vew very powerful tools, but more
by a consolidating collection of different tools with an easy ability to
be tweaked, each very good at doing specific tasks.
The task at that point will how to consolidate this ecosystem of tools,
but it will be a simpler problem then than it is now.
But it's also important to note the differences: Freebase Loader, by
virtue of keeping all its data state in one place instead of being
scattered in a lot of RABJ questions, will always be more suited for
operations such as internal reconciliation or data transformation or
massaging, things that need to happen before a reconciliation stage can
And if the data to reconcile is small, it doesn't make sense to create a
RABJ queue just for that.
But if the data is big enough, I think it's a huge relief to be able to
consider the loading aspect done and spread the reconciliation load to
others that want to help out (or to yourself in a different mood) and
this is not currently possible with Freebase Loader.
This does not mean that the two efforts won't talk to each other or
won't ultimately lead to more convergence or even be just complete
integration into one tool. I frankly don't have any problems with that.
It's like digging a tunnel by starting from different sides of the
mountain... as long as both sides where the others are at, it's not a
Stefano Mazzocchi Application Catalyst
Metaweb Technologies, Inc. stefano at metaweb.com
More information about the Freebase-discuss