[Developers] Work queues, leader boards, etc

[Tom Morris]

On Fri, Feb 20, 2009 at 4:01 AM, Iain Sproat <iainsproat at gmail.com> wrote:

> My only hangup with the competitive aspect is that the current apps
> (including leaderboards and typewriter) value quantity over quality.

Clearly since metrics drive behavior, you want to choose the right
metrics.  I agree that pure quantity is a bad metric which is why I
proposed a penalty for both users and apps that contributed results
that were rejected/out voted/etc.  By adjusting the positive and
negative weights, you should be able to shape user behavior.  The
database has got to get through the bootstrap phase and reach critical
mass to be useful though, so quantity *is* important, along with
quality.

> I may have made 78k facts, but a huge chunk have just been transcribed from
> wikipedia, so it's a very large risk that I may just be blindly copying
> wikipedia vandalism.

I doubt there's enough vandalism to worry about, but you could easily
be transcribing commonly accepted incorrect information into Freebase
"facts."  That's definitely a weakness in the current attribution
system in that the fact will only be attributed to you, without any
clue what book, database, or web page you referred to.  From a
practical point of view though, it's tough to do this in a way that's
lightweight enough to be acceptable to users and still provide useful
information.  Since so much comes from Wikipedia though, perhaps a
checkbox or data entry mode that says a user is transcribing from WP?
At least stuff that is parsed out of infoboxes gets attributed
automatically.

The general point of understanding the context that the user is
operating in is an important one though.  I was playing with
Geographer yesterday and really question its premise.  I got presented
with a whole raft of things that were a) mispelled, but in GNIS with
their correct spelling, b) U.S. census places, which clearly have geo
information associated with them somewhere, c) small NH towns which
are in GNIS or d) were Roman names for modern day British cities and
towns.  These problems each have solutions which don't have anything
to do with users dragging pushpins around maps of areas that they
aren't familiar with and the fact that they now have geocoordinates is
just going to mask the real problem.  Gazeteers have been well
understood since print days.  Freebase needs to be making better use
of existing data sources like GNIS.

Tom