No idea for new project?

No idea how to solve hard IT problems? Try to do something new. Open your mind!


/Stanislaw Wyspianski, "The creation of the world", stained glass, Franciszkanow church, Krakow, PL/

What I hate in perl5 and Catalyst

Perl - references

Let's remove all simple data-structures like @lists and %hashes. When we think "list" let use list-refs, ref-hashes instead of hashes. Let's keep only scalars and references with implicit dereferencing, like in Python or Ruby. No wantarray context-dependent behaviour! No @{$foo} and \@foo and other nasty code! Perl will be clear and more funny!

current perl:
$foo = [ map { ... } @$bar ]

better perl:
$foo = map { } $bar

As I see folks have problems why we write:

%hash = (foo => 42);
$hash{foo}   # where is hash sign, % ?

@list = qw/foo bar baz/;
$list[1]     # where is list sign, @ ?

and there is no much difference between lists and refs, but differences between various forms of using refs are probably bigger than list<->ref:

$list[1][2]      # wolf
$$list[1][2]     # dog1
$list->[1][2]    # dog2
$list->[1]->[2]  # dog3

Sure, there is only one dog, welcome to TIMTOWDI world!

There are also some troubles with

$ref = \@list;
$ref = [@ref]
# maybe it is @list?
$ref = [ $some->method ]
# or maybe it is %hash?
$ref = { $some->method }

# could be better to make something like
$ref = $some->method->return_as_hashref
$ref = $some->method->return_as_arrayref

Maybe I should stop writing perl code because I think too much about all those things? ;)

Catalyst - index is different than other actions

What is a difference between:

Controller app::foo;

sub index : Private {
    my ($self, $c, $param1) = @_;
}

sub show : Local {
    my ($self, $c, $param1) = @_;
}

?

Index will not catch arguments, will not work with /foo/123, although /foo/show/123 will work that way. To fix index, we need to use LocalRegex

sub index : LocalRegex('^(.*)$') {
    my ( $self, $c ) = @_;
    my ($param1) = @{ $c->req->captures };
}

YAPC::Europe 2007 in Vienna

Larry Wall, Damian Conway, many great perl hackers, interesting events, all in one place in one time.

It was my first YAPC and also first YAPC talk (slides from Catalyst refactoring are with others from this yapc on slideshare - also check perlbuzz.com) and Bartosz Jakubski's YAPC::EU 2007 photos.

However more interesting was to see in real life how this OS world works, met many interesting people from the perl community. There is probably nothing more interesting than long talks about POE, Catalyst and other kinda-like-app-servers or any other stuff during evenings near the Dunau Kanal and in other nice places in Vienna.

Huh, many words to say how great and cool was this YAPC!!!

PS: I will try to remember difference between Dbix and (Dbic or Dbix::Class) at least to avoid bot activation on #catalyst /and mst should also be happy ;)

Ruby, Rails and Netbeans (compared to Catalyst)

In current time I am working on my bike-"startup" site, rowerem.org (in Polish: by-bike org) which promotes using bicycles in Krakow and cities at all. This is a free project, no money just fun and opportunity to learn something new.

For long time I was not sure I should do it in perl/Catalyst, some kind of python framework or Ruby/Rails. From the hosting point of view it should be php but... never mind ;) Because I like python, this simplicity, explicit constructions I was checking python frameworks. Unfortunately Django goes own way of MTV, Turbogears is not so well designed (opinions), Pylons - not so popular to have mature quality of all web-related libs. Inspired by latest billboards ads in the city I chose Rails ;)

What is good about Rails?

- Good documentation. You have few good, printed books: Rails Recipes, Rails Cookbook and of course Agile Web Development with Rails. If you have a problem with something, just google for it or check in book. Howtos on rails/doc are also helpful. The step-in cost is very small compared to eg. Perl/Catalyst even when I code in Perl for 10 years when Ruby is still fresh for me.

- not too deep directory structure. It is simpler compared to Catalyst.

- model (ActiveRecord) is really simple to use. Maybe DBIx::Class is powerful but hard to use for night development (after typical day in work). Or I am just too lazy ;)

- good IDE. Have you ever tried Netbeans 6? There are some issues like hard-to-work-without-mouse, hard to (re)define shortcuts / compared to Kdevelop / but... you have good, free IDE for Ruby and Rails projects!

Now it is very funny to extend Catalyst knowledge based on Rails usage - "there should be something like this in Catalyst", this works also in the opposite direction, because both frameworks are quite similar.

Do you use Catalayst? Or maybe you use Rails? What do you think about them?

Mysql, convert from latin1-unicode to unicode-unicode

If you have your unicode data stored in a default latin1 mysql data fields, there is a simple way to migrate it to a proper unicode/utf8 coding:

1) change text fields (char/varchar/text) format to BLOB to loose coding info
2) change BLOB to varchar/text setting proper coding.

ALTER TABLE `news` CHANGE `body` `body` BLOB NULL DEFAULT NULL;
ALTER TABLE `news` CHANGE `body` `body` TEXT CHARACTER SET utf8 NULL DEFAULT NULL;

Description: if you have 2/3-bytes sequences as a one non-ascii character in latin1 field, it means you have unicode (eg. utf8) text there - but for mysql it is still a latin1 string. It can work for some time if you do all your data operations outside database in a programming language and connect to MySQL as a latin1 client. But probably this is not what you would like to have as a long-term solution, e.g. you would like to use SQL for sort/comparisons, any reporting tools so you need to have data properly encoded.
If you do convert directly unicode-in-latin1 to unicode (by single alter table) it would become a 4-byte-mess, as MySQL will take existing utf8 multi-byte sequences as proper latin1 characters. The workaround is to hide coding info for MySQL by going through temporary BLOB step to keep all data untouched.

Be aware: After few several latin1-unicode conversions, I would check in the first step if you do not have a mix of latin1/utf8 coding and perhaps deal with it by conversion in external scripts.

Perl, unicode, utf-8, mysql

Handling unicode/utf8 in Perl is quite trivial when you understand "two string approaches".

Perl 5.8 by default can handle UTF-8 strings like a sequence of bytes (1-4 bytes per one char). You can "compress" them to unicode strings with

$wide_char_string = Encode::decode_utf8($octets)

Such encoded string has unicode flag, you can check it with:

Encode::is_utf8($checked_string)

If they have latin1/2/russian/etc. chars after unicode packing:

length($octets) > length($wide_char_string)

Remember: unicode flag does not mean you MUST have wide characters in string.
Wide characters (ord > 255) can be in such unicode string, but it can also has a set of unicode-octets, "unpacked" in unicode string.

Use Test::utf8 to check you have wide chars or not.

FAQ / typical problems:

- check all sources in your program are the same, coded as unicode (wide) or unicode-string. Typically wide-char-strings is a better approach then byte-string (see perldoc Encode). All inputs/outputs like files, DBI, network need to be converted to your choosen internal format.

- do not use use utf8 pragma unless you really need it. In this case all strings have unicode flag, but it does not mean they have wide chars!

(octet_string eq unicode_string) == false

You cannot compare such strings without decode/encode, they are natively different!

Perl, utf8, MySQL

Two approaches to get unicode working with MySQL:

1) after connecting to database, do("SET NAMES 'utf8');
All unicode strings will be octets, without unicode flag.

2) use DBI connection with flag: mysql_enable_utf8 (since DBD::mysql >= 4)
All unicode strings will have wide chars and unicode flag.

Only the second approach works correctly with AutoReconnect flag.

Good news: there is no difference for inserts/updates you use octets or wide_char_strings.

More to read

Check also this Martin Fowler - utf8 in perl

Linux - Mac OsX - Redhat - Centos - Ubuntu - FreeBSD - Solaris - Windows commands equivalents

Updated [2016-02]: OsX/Mac/Darwin/Yosemite/El Capitan
Update: Added Solaris (both OpenSolaris and SunOS) and Windows (where applicable) coverage for this set of commands.
Update: Coverage for redhat-fedora-centos Linux line / yum package manager.

Systems covered:
Linux groups:
* Redhat based: Redhat, Fedora, Centos (yum/rpm packages)
* Debian based: Debian, Ubuntu, Kubuntu (dpkg/.deb packages)
Mac OsX: Darwin/Yosemite/El Capitan
OsX is usually very close to BSD systems, but sometimes suprisingly has Linux-style commands/tools.
Windows: around XP/2007 and newer

Accounts

Check what groups do I belong to?

{linux, osx, freebsd, solaris}$ groups
{windows - active directory}> dsquery user -samid %USERNAME%|dsget user -memberof

Disk, filesystem

Disk usage

{linux, osx, bsd}$ du -sh

Count subdirectories in current directory:

{linux}$ du --max-depth=1
{osx, bsd}$ du -d1
{SunOS}$ du

Typical approach to find biggest directories/files on disk:

{linux}$ du --max-depth=1 -kx|sort -n
{osx, bsd}$ du -d1 -kx|sort -n

Find some kinds of files (regex is a mask for full path, no need for begin/end marks)

{linux}$ find . -regextype posix-extended -type f -regex ".*\.(java|class)"
{osx, bsd}$ find -E . -type f -regex ".*\.(java|class)"

Show open files and programs:

{linux, osx}$ lsof
{freebsd}$ fstat

Real time disk usage (is there something which shows results for every disk in linux?):

{linux}$ vmstat 3
{osx, bsd}$ iostat 3

Swap info:

{linux}$ free
{freebsd}$ swapinfo
{osx}$ vm_stat
{osx}$ top -l 1 -s 0 -n 0

Networking

Show open ports and apps connected to them:

{linux}$ netstat -apne --inet
{osx}$ lsof -i
{freebsd}$ sockstat
{SunOS}$ netstat
{windows}$ netstat -b
netstat -b -v   # slower but with tree of dependencies

Kernel issues

Show loaded modules:

{linux}$ lsmod
{osx}$ kextstat
{freebsd}$ kldstat

Load kernel module:

{linux}$ modprobe SomeModule
{freebsd}$ kldload SomeModule

Remove loaded module:

{linux}$ rmmod SomeModule
{freebsd}$ kldunload SomeModule

Program development

Trace the system calls of a program:

{linux}$ strace
{osx}$ dtrace   
{freebsd}$ truss
  (strace is also available in /usr/ports/devel/strace)

libraries - show all paths + libs:

{linux}$ ldconfig -p
{freebsd}$ ldconfig -r

Packages management

Different linux distros make it own way. I'll focus on debian-based distributions like Debian, Ubuntu, Kubuntu etc.

Find which package this file belongs to?

{freebsd}$ pkg_info -W /path/to/checked_file
{debian ubuntu}$ dpkg -S /path/to/checked_file
{redhat centos}$ rpm -qf /path/to/file
{osx, for brew}$ ls -l `which node`|perl -lane '{print $F[-1]}'

OsX - it shows the original brew package/file this command is linked to

Have we got a package like... (in (k)ubuntu you can use more friendly tools like synaptic, apt-get, kPackageKit)

{linux}$ apt-cache search your_name
{freebsd}$ cd /usr/ports; make search key=your_name
                          make search name=pear display=name,path
    you can also try simple locate (only in package names):
{freebsd}$ locate -i your_name | grep "/usr/ports/"
{redhat centos}$ yum search name
yum provides name

Install a binary package

{debian ubuntu}$ apt-get install package_name
{redhat centos}$ yum install name
{freebsd}$ pkg_add -r package_name
{windows}$ msiexec /i package.msi
{osx}$ ??? 

In FreeBSD you have packages made in distribution release time - unfortunately there are no binary upgrades for released version)

Update binary packages

{debian ubuntu}$ apt-get update; apt-get upgrade
{redhat centos}$ yum update

Install a package from sources

{debian ubuntu}$ apt-src
{freebsd}$ cd /usr/ports/path/package; make install clean
{osx}$ use brew or macports

...The more I see the less I know...

YAPC Vienna 2007 - my talk

I will have a Catalyst talk at Yet Another Perl Conference in Vienna, 28th to 30th August 2007 in Web track.

More on conference site,
http://vienna.yapceurope.org/ye2007/talks#18

Yaml, yaml-syck, catalyst...

If you have problem like:

Couldn't instantiate component "app::Model::app", "->config->{schema_class} must be defined for this model

check if you have YAML::Syck module also. Seems like pure YAML module have bugs and is not enough to parse Catalyst yaml configs.

Just to remember, you can make hierarchy of config files:

app.yaml
app_local.yaml

to override "default" values (eg. for production environment)

Ruby & rails - thanks for nice weekend ;)

It is early Monday morning and I need to write it (however no one wants to read it).

My wife spent past 2 days in mountains (Tatry, never mind). I spent it with Rails. Or rather - with ruby. When I dig into Rails I sow all these :foo, :bar so I couldn't move forward without knowing WHAT DOES IT MEAN???

This is a time when books like "Programming Ruby" comes in hand. What I think now?

Ruby is really easy especially if you know perl :) I am not a fun of TIMTOWTDI and Ruby seems to be also... However there are ugly constructions like "reverse if/unless" so Ruby programmers can step to the Black World of Many-F^&*-Ways-To-Do-It... but they probably don't want, do they?

We need some ways in language to make different things looking different and Ruby allow us to do it. Oh, it looks like expressive programmers manifesto ;)

So once again:
Pretty nice language, with this object logic you thought "how it will be simple to have it done ONE WAY, objective". Really worth trying. Simple. Code looks nice.

I thought Ruby is too perlish and probably I should focus on clean Python syntax. However Ruby is better designed than Python. There are no "dirties" like:

$> Python
>>> foo = "bla bla bla"
>>> foo.upper()
'BLA BLA BLA'
>>> foo.len()
Traceback (most recent call last):
 File "", line 1, in ?
AttributeError: 'str' object has no attribute 'len'
>>> len(foo)
11

In Ruby it is simple:
(even Python's dir(object) in Ruby is objective: obj.methods or even better obj.methods.sort)

irb> foo.upcase
=> "BLA BLA BLA"
irb> foo.length
=> 11

You can write it "Pythonic/java" way with parentheses, but they are optional:
irb> foo.length()
=> 11

Ok, so what about :foo, :bar and @others?

var = 123   # string
$var = 123  # global variable
@var = 123  # class variable
@@var = 123 # package variable

Classes:

def class1
   attr_reader :duration
end

# which is a shortcut of:

def class1
   attr_reader(:duration)
end

# which is a shortcut of:

def class1
   def duration
       @duration
   end
end

# which is a shortcut of:

def class1
   def duration
       return(@duration)
   end
end

:duration - means you passing "object symbol" not "value of it" (aiui)

If you come from Java world, now you know we don't need "make setters/getters" refactoring option in our Ruby editor/IDE ;)

We can strip methods parentheses(), we can strip returns, we can focus on code logic - this is why frameworks written in Ruby can be pretty nice, easy to understand and to remember. I just wonder when I will see a book named "Ruby best practices".

Java or Ruby? Perl or C++?

Which of the following positions would you say is most true at your company, assuming (for the moment) that you can only choose one of them:

Above all, we need stability. We have enormous scale and massive business complexity. To create order out of inevitable chaos, we need rigorous modeling for both our code and our data. If we don't get the model and architecture mostly correct in the beginning, it will hurt us later, so we'd better invest a lot of effort in up-front design. We need hardened interfaces -- which means static typing by definition, or users won't be able to see how to use the interfaces. We need to maximize performance, and this requires static types and meticulous data models. Our most important business advantages are the stability, reliability, predictability and performance of our systems and interfaces. Viva SOAP (or CORBA), UML and rigorous ERDs, DTDs or schemas for all XML, and {C++|Java|C#|OCaml|Haskell|Ada}.

Above all, we need flexibility. Our business requirements are constantly changing in unpredictable ways, and rigid data models rarely anticipate these changes adequately. Small teams need to be able to deliver quickly on their own goals, yet simultaneously keep up with rapid changes in the rest of the business. Hence we should use flexible, expressive languages and data models, even if it increases the cost of achieving the performance we need. We can achieve sufficient reliability through a combination of rigorous unit testing and agile development practices. Our most important business advantage is our ability to deliver on new initiatives quickly. Viva XML/RPC and HTTP, mandatory agile programming, loose name/value pair modeling for both XML and relational data, and {Python|Ruby|Lisp|Smalltalk|Erlang}.


- read all Steve Yegge's (previous Amazon.com, now Google developer) weak vs. strong typing (not working? check at wayback machine)

***{**}***
Hmm, interesting - i just realized Steve is the same person I suggested reading in first blog entry, "The Next Big Language".

When is no Perl 6, we can try Ruby

You know perl, all the funny $_ / $& / q{} vars/ops, inline regexps but want to use object-oriented syntax with more typical dot, my_object.my_sub("blabla")?

There is Ruby, now probably more known as a one of dependency needed to fire up Rails MVC framework ;)

Start from here
http://www.ruby-lang.org/en/documentation/ruby-from-other-languages/to-ruby-from-perl/

look for some code examples
http://migo.sixbit.org/papers/Introduction_to_Ruby/slide-index.html

and of course compare solutions in Ruby/Perl from mythical Perl Cookbook, done by PLEAC folks,
http://pleac.sourceforge.net/

For me Ruby is too perlish - after years of perl programming I fill better when I can do something in more clean Python syntax, with TIOOWTDI. However it's nice to see that Ruby is so close to Perl and there are some hacks you can do in Perl, Ruby but not in Python.