[Freebase-discuss] Pain point / Freebase list importer UI

[Stefano Mazzocchi]

On 1/27/10 4:54 PM, Tom Morris wrote:
> On Wed, Jan 27, 2010 at 7:45 PM, Stefano Mazzocchi<stefano at metaweb.com>  wrote:
>> On 1/27/10 3:59 PM, Kirrily Robert wrote:
>>> The list importer seems like a really good candidate for rewriting as
>>> an Acre app.  Is anyone interested in having a shot at this?  If we
>>> come up with one that works better than the current one, we could
>>> probably replace any links in the client (eg. on the topic page, view
>>> page, etc) with links to the app in question, and remove the client
>>> list importer.
>>
>> I'll be taking a shot at this next week.
>
> I'm confused.  I thought the Spreadsheet Loader was the intended
> replacement for the Import List tool.
>
> How do these various solutions relate to each other?

Peter and I had a long conversation about this today and while it is 
obvious that 'data loading' tools that follow the same guidelines will 
always have some overlap, I think it's valuable to try different 
solutions to the loading problem.

In particular, I want to focus my attention on exploring more the social 
and mutual-assistance aspect of data loading.

I believe that the act of adjusting and massaging data to align well 
with Freebase's and the act of reconciling individual topics with 
Freebase's are separate tasks and require substantially different skills.

Even more importantly, the reconciliation tasks require much less 
knowledge of how Freebase operates and can be easily parallelized (we 
know all this from the data games and odeskers)

Freebase Loader, in its current form at least, is focusing on being a 
tool for an individual that wants to load a dataset and Peter is 
spending a great deal on energy to make that even better than it 
currently is.

Because of all the work I've done with RABJ and data games over the last 
few months, I want to try to tackle the list importing problem (reusing 
code where it makes sense, of course) but coming at it from the social 
angle.

I have been building similar apps for internal use and I think I can 
whip something up very quickly by reusing most of that code, then show 
it raw and unfinished here and iterate from there with suggestions from 
everybody that ends up caring (also, I'll be working in Acre, so it will 
be easier for others to change things and experiment on their own if 
they feel like it).

Ultimately, I believe that data loading interests (both internal and 
external) will not be served well by vew very powerful tools, but more 
by a consolidating collection of different tools with an easy ability to 
be tweaked, each very good at doing specific tasks.

The task at that point will how to consolidate this ecosystem of tools, 
but it will be a simpler problem then than it is now.

But it's also important to note the differences: Freebase Loader, by 
virtue of keeping all its data state in one place instead of being 
scattered in a lot of RABJ questions, will always be more suited for 
operations such as internal reconciliation or data transformation or 
massaging, things that need to happen before a reconciliation stage can 
take place.

And if the data to reconcile is small, it doesn't make sense to create a 
RABJ queue just for that.

But if the data is big enough, I think it's a huge relief to be able to 
consider the loading aspect done and spread the reconciliation load to 
others that want to help out (or to yourself in a different mood) and 
this is not currently possible with Freebase Loader.

This does not mean that the two efforts won't talk to each other or 
won't ultimately lead to more convergence or even be just complete 
integration into one tool. I frankly don't have any problems with that. 
It's like digging a tunnel by starting from different sides of the 
mountain... as long as both sides where the others are at, it's not a 
problem.

-- 
Stefano Mazzocchi                              Application Catalyst
Metaweb Technologies, Inc.                      stefano at metaweb.com
-------------------------------------------------------------------