[Developers] Perl API (hi Hayden!)

[Hayden Stainsby]

For everyone who missed this, sorry, I sent Kirrily an email before  
realising I'd missed the list and then reposting. So, Kirrily, I hope  
you don't mind that I've reposted your reply to the list with my  
response.

On 19 Aug 2007, at 17:20, Kirrily Robert wrote:

> [ You only replied to me, not to the list.  Did you mean to reply  
> to the list?]
Whoops, see above.

>
> Hayden, great stuff.  Looks like we're on the same page.  A few
> further comments...
>
> On 8/19/07, Hayden Stainsby <hds at caffeineconcepts.com> wrote:
>> my $mh = Metaweb->new( blah );
>> my $films = $mh->fetch_query_results([{
>>         directed_by => 'Martin Scorsese',
>>         more blah blah}] );
>
> A hashref in an arrayref?  Why?  I gather you can pass multiple
> queries, but if you just make them a list of hashrefs, they'll show up
> in @_ just fine and not need dereferencing.

The reason for this is not a Perl issue, it's MQL syntax. You need  
the [ and ] to specify you're expecting more than 1 result. What I  
did here is the same as this (pasted from query editor):
{
   "query":[{
     "directed_by":"Martin Scorsese",
     "type":"/film/film"
   }]
}

If I didn't have the square brackets I'd get an error, saying that  
only one value was expected. As I said before, I'm trying to stay as  
close as possible to the Metaweb API, so I'm not planning on fixing  
these sorts of MQL mix ups for programmers, they'll have to submit a  
valid MQL query (even if it's submitted as a Perl structure).


>
>> foreach my $film (@$films)  {
>
> You're returning an arrayref, it seems.  That's a lot of dereferencing
> all over the place for not much benefit. I'd say the best thing to do
> is use wantarray() to return either a real list (my @films =
> $mh->fetch()) or else an iterator object.  Can we even do iterator
> objects?  I hope so!

There is a good reason for the arrayref, that's the potential size of  
the results returned by Metaweb, especially once I include  
transparent cursor support, a query could return thousands of  
results, while Perl can return this as a straight array, it's pretty  
expensive in terms of memory overhead, the entire array is copied  
before being returned, I would much prefer not to have my module  
incurring that sort of over head when something like:

@films = @{$mh->values};

will achieve roughly the same result if I'm returning an arrayref,  
but without the copy overhead. A single dereference is pretty close  
in execution time to a single copy, but I don't want to get in to  
that. But...

We can do iterators. In fact, let's plan to do iterators, we save on  
memory overhead and we can do pretty things when fetching from an  
iterator, like filling out the entire object with all the values for  
all it's types. I've got an example of this - a very, very simple  
example - in the sandbox on the Metaweb subversion repository (http:// 
caffeineconcepts.com/metaweb).

>
> my $films = $mh->fetch(blah blah);
> while (my $f = $films->next()) { ... }
>
>>         print $film->value('name') . "\n";
>
> You could use something like Class::Accessor (???) to automatically
> create read-only accessor methods with the names of the fields you
> get, so this would be $film->name() which is a little nicer.
> I guess the only problem is if you have a field called "isa" or
> something, but that seems improbable.  I really prefer the
> $obj->prop() style of accessor for Perl, but if you have to have a
> single accessor then I guess $obj->value($prop) isn't too bad.  Or
> maybe get().
>
>>         # but also
>>         my $actors = $film->objects('staring');
>
> Take a look at the Class::DBI docs!
>
> my @actors = $film->starring();
>
> Even if we need to have a single accessor ("get" or "value" or
> something) at least make it the common way to get single values *and*
> lists.  If you ask it for something that has multiple values then
> it'll return a list, easy peasy.  Make the elements of those lists be
> objects the same as $film is, and make them stringify to their own
> names.  So you can do something like:
>
> foreach my $actor (@actors) {
>     print "$actor\n"; # stringification!
> }
>
> or:
>
> foreach my $actor (@actors) {
>     print $actor->get('date_of_birth'), "\n"; # acting like an object
> }
>

This is where we have differing opinions. I certainly see the merit  
in not needing 'object' and 'objects' methods (I was writing the  
example code off the top of my head, so please don't take it too  
literally (-: ). But using $film->starring, instead of $film->object 
('staring') seems to be forcing people to write code that's far too  
specific to one circumstance. I'd like to be able to do this (please  
ignore possibly security holes, this is just an example):

$a_field = <STDIN>;
print $film->value($a_field) . "\n";

Which you can't (without eval) do if you're forced to call the fields  
by method name. I'm not actually opposed to including this, but I'd  
prefer to start with a generic accessor method, and then add  
something like the Class::DBI implementation from there.

I think that your idea of using string-ification to differentiate  
objects for names is a great idea, I'll certainly include that.

> Also make it so that if you want to perform a secondary query using  
> $actor, eg:
>
> $metaweb->fetch({
>     blah blah something
>     actor => $actor,
> });
>
> ... then it'll DTRT.  Class::DBI does this by passing in ids
> appropriately if you use it this way.
>
>> This would give programers the ability to traverse trees of Metaweb
>> data while writing little to no actual MQL. Kirrily (and everyone
>> else), do you think this is the sort of API that you would find
>> useful? Does anyone see any problems with working through what will
>> become a 2 level API?
>
> That's what I'd like to use for most day to day purposes.  I would
> occasionally want to get down into the lower layers, but I'd say 90%
> or more in the high level abstraction.
>
> K.
>
> -- 
> Kirrily Robert
> skud at infotrope.net
> http://infotrope.net

As I said though, I'm working an a base layer to begin with, once  
that's more or less completely usable I'll get to work on some of  
these more clever abstractions. Hopefully fairly soon.

Hope everyone's following this. Unfortunately my time difference to  
most of the other people in this discussion is meaning that we're  
only getting one round of conversation each 24 hour cycle, the joys  
and pains of the internet. Perhaps we can plan a time for everyone to  
get together for some slightly more real-time discussion.

--
Hayden


#!/usr/bin/perl
chop($_=<>);@s=split/ /;foreach$m(@s){if($m=='*'){$z=pop at t;$x=
pop at t;$a=eval"$x$m$z";push at t,$a;}else{push at t,$m;}}print"$a\n";
# http://occasionallyhuman.net/
#9202a8c04000641f80000000054c75be