Scott Penrose

Ext.Direct

Scott is an expert software developer with over 30 years experience, specialising in education, automation and remote data.

A set of perl implementation of the Ext.Direct server side stack.

See Also

Live Demonstrations


The implementations

  • Simple - a simple example implementation. Useful as a starting point to write your own
  • ModPerl - a full implementation used in production
  • ACL - An experimental version with access controls and further security
  • module - An experimental version, moving the code into an abstract module, that could be reused e.g. as a base for ModPerl or Catalyst

Simple

The code in the simple/ directory is a short but fully functional implementation. It uses a single config file, one CGI for generate the Javascript API description and one to do the routing. It has a very simple view of class and method requests.

This has really been written as a reference implementation for Perl.

Features

  • Configuration - basic perl format only

Mod Perl

A far more complete and reliable version of the system, which I will use in production.

Features

  • Configuration - any format using Config::Any - e.g. Perl, JSON, Apache, YAML etc.
  • Post configuration OR Child initialisation hooks in Apache to pre-load all code
    • Use Post config, unless doing a database connection (or similar), then use the child init.
  • Automatic dispatch OR specific URLs - Use a single Location configuration and standard URL (./api or ./router) or use your own
  • Overall (DEFAULT) before/after methods - allow any pre-run and post-run methods to run.
    • Useful for: access control/security; opening/committing database requests
  • API (Action and Method) call before/after - allow any pre-run and post-run methods to execute specific to the request
    • Useful for: specific access control/security; checking other requirements.

Future

  • Introspection - maybe, not too important to me, but easy to implement
  • ACL - Ability to use access controls against a user/group - NOTE: You can already use 'before' method to achieve this now.
  • Parallel processing - non-blocking? fork? thread? POE?
  • Run action specific before/after methods only once
  • Security on data inputs/outputs

Example config

Apache configuration:

	PerlModule Apache::RPC::ExtDirect
	<Location /data>
		SetHandler modperl
		PerlResponseHandler Apache::RPC::ExtDirect
		PerlSetVar ConfigFile /full/path/to/config.pl
	</Location>

Config file: (perl format)

{
        Lookup => {
                Class => 'Demo',
                Methods => {
                        suburb => { params => 1 },
                },
        },
}

Out of the box

Introspection... soft of

I would like to write a module like this:

package Demo::Package;
use Attribute::RPC::ExtDirect;
sub calculate : ExtDirect(3) {
   my ($a, $b, $c) = @_;
   return $a * $b * $c;
}
1;

The idea is to use attributes in Perl to allow self description of the module within itself. The mod_perl apache implementation now supports reading configuration from the module. Just specify the package name (currently limited to support modules with '::' in their identifier) instead of a filename.

Attribute::RPC::ExtDirect could then create the configuration by providing a method (e.g. extdirect_config) which can also be used to dump the data if not loaded through Apache. Therefore the attribute module can be used in any of the implementations :-)

This should then be tested with Moose and other object frameworks.


How to write an Ext.Direct Server Stack

NOTE: This code here is not meant to be complete. Please see the Source Code on Git hub for a complete working version. This code has many missing components and here is for description rather than accuracy.

There are three steps in writing a server stack.

  • Describe the API in Javascript
    • Manually - by writing a Javascript static file
    • Automatically - server side, using either config (in Javascript, or anything else) or Introspection
  • Route incoming requests to the backend code
    • Read the JSON or Form posts from the Client

Config

If you want to keep all the configuration and information about the API on the server side, you really have two choices:

  • Introspection - use a small configuration file to work out which classes, then look at the methods
  • Configuration - just a config file that describes all the interfaces.

I have gone for configuration, as I desire more control over each interface. E.g. having high level restrictions, such as access controls before the class is even loaded. Therefore introspection is not much help to me.

But when exposing more complicated APIs with less control, it could be very useful.

This example shows some configuration using perl data format.

#!Perl
{
        Profile => {
                getBasicInfo => { params => 2 },
                getPhoneInfo => { params => 1 },
                getLocationInfo => { params => 1 },
                updateBasicInfo => { params => 2, formHandler => 1, },
        },
};

API

The API is used by most of the example/reference implementations as the name given to the Javascript file generated.

Example:

#!Perl
my $CONFIG = do "config.pl";

my $actions = {};
foreach my $class (keys %$CONFIG) {
        my @methods;
        foreach my $method (keys %{$CONFIG->{$class}}) {
                push @methods, {
                        name => $method,
                        len => $CONFIG->{$class}{$method}{params},
                };
        }
        $actions->{$class} = \@methods;
}

my $url = $ENV{SCRIPT_NAME};
$url =~ s/api/router/;

print header(-type => 'text/plain');
print "Ext.app.REMOTING_API = ";
print to_json({
        url => $url,
        type => 'remoting',
        actions => $actions,
});

Router

The router has only a few small jobs (minimum implementation)

  • Decode the incoming data
    • NOTE: it could be in JSON or Form data format
    • NOTE: In JSON format you may get a single request as a hash (javascript object) or multiple requests as an array of hashes. You need to check.
  • Security - as much or as little as you see fit.
    • Basic input checking is a good idea here
    • Make sure the action and method requested are allowed (e.g. in your configuration file)
  • Execute each request
    • You can even fork/thread for each request if you want to do them concurrently
  • Encode and return results
    • Including the requested action/method and unique id

Example:

#!Perl

my $CONFIG = do "config.pl";


my $buf;
read STDIN, $buf, $ENV{CONTENT_LENGTH};
my $data from_json($buf);
if (ref($data) ne "ARRAY") {
        $data = [ $data ];
}

my @results;
foreach my $request (@$data) {
        my $action = $request->{action};
        my $method = $request->{method};
        my $data = $request->{data};

        unless ($CONFIG->{$action}{$method}) {
                die "BAD REQUEST - Invalid action/method - $action/$method";
        }
        my $class = "Demo";
        my $obj = $class->new();
        my $result = $obj->$method(ref($data) eq "ARRAY" ? @$data : ());
        
        push @results, {
                type => 'rpc',
                tid => $request->{tid},
                action => $action,
                method => $method,
                result => $result,
        };
}

print "Content-type: text/plain\n\n";
print to_json(\@results);

Example code:

#!Perl
sub calc : ExtDirect(3) {
        my ($x, $y, $z) = @_;
        return ($x * $y * $z);
}

  • Perl
  • Javascript