Benchmarks: python. Django vs. perl-> Catalyst

Ok so i made the benchmarks proper way.

Some stuff to know:
Perl: v5.8.8 built for i486-linux-gnu-thread-multi
Python: 2.4.4c1 (#2, Oct 11 2006, 21:51:02)
Linux: ubuntu 2.6.17-11-generic #2 SMP Thu Feb 1 19:52:28 UTC 2007 i686 GNU/Linux
Apache/2.0.55 (Ubuntu) mod_fastcgi/2.4.2

Django was deployed in apache such way:
FastCGIExternalServer /path/to/fastcgi.fcgi -socket /tmp/mysite.sock

Catalyst was deployed that way:
FastCGIServer /path/to/script/mysite_fastcgi.pl -processes 3

Catalyst was easier to run as a FastCGI - in app/script you have ready to use fastcgi script. With Django you need to find the proper script at their web page. There was also problem problem with template paths - I fixed it by changing to absolute in settings.py. I didn't find on Django webpage the same deployment method as for Catalyst. I tried recipt from "Running Django on a shared-hosting provider with Apache" but got:

runfastcgi(method="threaded", daemonize="false")
TypeError: runfastcgi() got an unexpected keyword argument 'method'

All packages were "out of the box" - from system package manager, cpan or loaded from net (flup).

So how good are these frameworks? How much changed apache compared to own-servers benchmarks I made before?

Tested with: ab -c 25 -n 5000 http://adam/(django|catalyst)/; w
Numbers belowe are averages from 3 measurements/framework. Every measurement was started when 1m load average was below 2.0 (typically 1.75-1.9)

DJANGO

===============================>
Concurrency Level:      25
Failed requests:        21 (in 15k reqs)
Average:                127 requests/sec.
1-min load after:       11.93

CATALYST:

=============================>
Concurrency Level:      25
Failed requests:        0 (in 15k reqs)
Average:                118 requests/sec.
1-min load after:       2.49

Django was little faster however this method of deployment give me quite big number of errors. Catalyst with always 3 fastcgi processes was less CPU consuming and 100% error free.

First benchmarks: Catalyst vs. Django

This comparison is little silly because I don't have time to deploy mod_python / fastcgi / lighttpd or anything like such-good-approach. I just wrote the same code (make SQL join, show template with same fields, same logic) and check it with ab (apache benchmark tool) against framework own servers.

Catalyst

Server Software:
Server Hostname:        localhost
Server Port:            3000

Document Path:          /gallery/
Document Length:        7185 bytes

Concurrency Level:      1
Time taken for tests:   64.568406 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      36675000 bytes
HTML transferred:       35925000 bytes
Requests per second:    77.44 [#/sec] (mean)
Time per request:       12.914 [ms] (mean)
Time per request:       12.914 [ms] (mean, across all concurrent requests)
Transfer rate:          554.68 [Kbytes/sec] received

Django

Server Software:        WSGIServer/0.1
Server Hostname:        localhost
Server Port:            8000

Document Path:          /
Document Length:        4981 bytes

Concurrency Level:      1
Time taken for tests:   61.358104 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      25625000 bytes
HTML transferred:       24905000 bytes
Requests per second:    81.49 [#/sec] (mean)
Time per request:       12.272 [ms] (mean)
Time per request:       12.272 [ms] (mean, across all concurrent requests)
Transfer rate:          407.84 [Kbytes/sec] received

With some modifications (eg: running together) the timings are like 100:96 for Django. A little faster. Will it differ much if we deploy it with Apache-and-the-fastest-way?

Django (python) vs. Catalyst (perl)

Intro

I spend many days with Perl MVC framework - Catalyst. Catalyst for perl is like Rails for ruby but without all this marketing hype & noise (and unfortunatelly with lack of good books, tutorials and doc). I tried to use Rails for some easy-but-existing database. I was surprised that Rails ActiveRecord was not able to quote SQL names to get data from - as I remember - "lines" column in MySQL. Little stupid, hmm?

Anyway Ruby is too perlish. I know perl and know how much cost all this TIMTOWTDI when your project is growing up. Python is like clean, better Perl (however Python libraries are not at this quality level like Perl libs in CPAN - eg. unicode translations). So this is "why python".

Few days ago I started small project with Django. Here are first thoughts and differences:

Django view = Catalyst controller

Django call itself MTV framework (model-view-template). Going this way Catalyst is Model-Controller-(View)-Template - the only difference is: Django models == Rails & Catalyst controllers.

Django global switch

- In Django you need to define every URL action in urls.py and create appropiate "views"

Example of urls.py (aka "global switch"):

urlpatterns = patterns('',
    ('', 'hnl.photos.views.index'),                      # root page
    (r'^admin/', include('django.contrib.admin.urls')),  # admin
    (r'^photo/update', 'hnl.photos.views.update'),       # update
)

- In Catalyst, there is no such "global switch". You just create controllers (perl classes), put methods with "local" attribute inside - and they are visible in your URL namespace. Why you should define this namespace mapping if it is enough to do it only by proper view class-function naming? For me it looks like Catalyst dispatcher is more "pythonic" than Django.

Output validation

I created view to import some data from Yaml data-dump to MySQL database. Everything works however Django cannot display some rows because of problems with validation. Message error is not so helpfull -

ValueError at /
year is out of range

Hmm, but which field (i have 2 dates in every record) in which row? Exception is caught in template (first line)

{% for repo in gallery_list %}
    * {{ repo.galleryname }}
{% endfor %}

but as you see - I do not fetch any date-time values here!

After looking into data I found '0000-00-00' date. Who pasted this record into database without problems? Django db model. Who was not able to show it? Django! I think it's a better idea to validate/filter data at input rather than output. Or in both places.

Objects in perl 5

There are many ways to do it. The old-school approach

blessed hash

sub new {
    my $classname = shift;
    my $rh = {};      # Reference to Hash, rh :)
    bless $rh, $classname;
    return $rh;
}

All variables can be created just by assigning something to blessed hash:

my $foo = my_class->new();
$foo->{bar} = 42;
print $foo->{bar};

Unfortunatelly there is lack of encapsulation - you have access to all methods and properties.

Get control of object properties - use fields

package my_class;
use fields qw(bar);     # declare used object vars
sub new {
    my $classname = shift;
    return fields::new $classname;
}

# usage:
my my_class $foo = my_class->new();
$foo->{baz} = 42;  # trying to use not declared field - error!
print $foo->{bar}; # works

We have control on object properties. Anyway, we still do not have private variables.

Private variables (scalar-bless)

package my_class;
{
   my %bar;  # All vars are declared as hashes
   sub new {
       my $classname = shift;
       my $rs = \do{ my $scalar }; # reference to scalar
       bless $rs, $classname;
       return $rs;
   }

   sub set_bar {
       my $this = shift;
       $bar{$this} = shift;
   }

   sub get_bar {
       my $this = shift;
       return $bar{$this};
   }
}

And now all variables are private!!!

Potential problem: it's not working with threads. To have a copy of such object instance in threads, you need to workaround this closure with get_all_data()/set_all_data() methods - and all object data need to be passed as additional parameters when creating new thread.

my $foo = my_class->new();
$foo->{bar} = 42;      # sorry, no bonus! Use set/get method instead.
print $foo->{bar};     # This is private variable, not accessible outside class.
print $foo->get_bar(); # works!

Private, public, protected - objects like in perl6:

like in manual:

use Perl6::Classes;

class Composer {
    submethod BUILD { print "Giving birth to a new composer\n" }
    method compose { print "Writing some music...\n" }
}

class ClassicalComposer is Composer {
    method compose {
        print "Writing some muzak...\n";
        $_->do_private;
    }
    method do_private is private { print "really private thing\n" }
}

class ModernComposer is Composer {
    submethod BUILD($) { $.length = shift }
    method compose() { print((map { int rand 10 } 1..$.length), "\n") }
    has $.length;
}

my $beethoven = new ClassicalComposer;
my $barber    = new ModernComposer 4;
my $mahler    = ModernComposer->new(400);

$beethoven->compose;   # Writing some muzak...
$barber->compose;      # 7214
#$beethoven->do_private;
compose $mahler;       # 8927586934796837469875

Pretty well, uhh? For me it was like "look, this is almost like in Java!" Just change is to extends.

Perl6::Classes works as a filter for code and do all this translation job. Well, we pay for it - it is 20x slower than hash/scalar blessed objects. If this latency doesn't matter - use it.

1, 2, 3, start

For nice start - some text to read - what about language of the future - see Steve Yegge's The Next Big Language