Scott Penrose


Scott is an expert software developer with over 30 years experience, specialising in education, automation and remote data.

Perl Module Data::Token

use Data::Token;
print token, "\n";

Generate an unpredictable unique token.


What is a Token

"A session token is a unique identifier (usually in the form of a hash generated by a hash function) that is generated and sent from a server to a client to identify the current interaction session. The client usually stores and sends the token as an HTTP cookie and/or sends it as a parameter in GET or POST queries." -


First and foremost - do not write your own authentication system, especially for web based applications (in any language). Use Apache to authenticate. However there are times you need a token, e.g. to embed in a URL.

I recommend looking at Apache::AuthCookie .

Token Security

The one important feature of tokens generated is that they are hard to guess.We want to increase the entropy (scientific term used in cryptography to mean greater randomness). They also should be unique.

Problem 1 - Uniqueness

This is easily solved. Most systems use a combination of process id + time.

$id = $$ . time;

Data::Token uses guaranteed unique ids through Data::UUID. For most applications this is overkill but it is better to be than the other way around.

Problem 2 - Entropy

The most common methods are to add a secret to the id, which is often generated from a simple random number e.g.

$id = $id . rand(time);

The perl documentation tells us that this is valid for cryptographic purposes:

srand (time ^ $$ ^ unpack "%L*", `ps axww | gzip`);

But is it? I am not sure.

Random number generated using "rand" even if seeded correctly are predictable. Other solutions such as using a typed in shared secret are less secure because they are constant and in clear text somewhere on the host.

Data::Token uses Crypt::Random with a moderate setting.

Problem 3 - Hiding the secret

Common method used.

$id = md5_hex($id);

We don't need to do this with a suitable random number. The UUID is unique and the Random number is the entropy. Combined they provide all the features required of a token. Taking a hash such as MD5 or SHA-1 is not necessary and may not offer more security. Since it is done on other systems and there may be something I am overlooking I have chosen to use the more secure of the two SHA-1.

Huge docs, Small code

The documentation and discussions around this module have been large considering the small amount of code in the module. But that is also the reason for the module. And security it is about security.

Existing Vulnerabilities

One of the reasons I wrote this is that all the discussions on PHP and Perl lists suggest the following code (or simple variations).

$id = md5_hex($$, time, rand(time));

This is however predictable. Non-trivial but definitely reduces the range of possibilities to quite small (in brute force terms). And almost none of the systems protect against attack.

The other reason I created this was that the code that does work (or as documented below is broken) is embedded into each of the modules in an un-reusable way. It should be lifted out into a reusable module.

These modules have predictable tokens. That is not to say they need to be secure. If all you are doing is taking someone else' shopping cart. The problem is that they then get used for more.


$md5->add($$ , time() , rand(time) );


substr(Digest::MD5::md5_hex(Digest::MD5::md5_hex(time(). {}. rand(). $$)), 0, $length);

Apache::Session can also use the apache mod_unique_id which is also predictable.

Apache::AuthCookie and Apache2::AuthCookie

Technically these don't generate the cookies, just check them. However the documentation recommends creating a token like this:

my $date = localtime;
my $ses_key = MD5->hexhash(join(';', $date, $PID, $PAC));


This is an extension of Apache::AuthCookie which does the lookup of login/password in a database and then uses Encrypted Cookies to check if it is valid. This looks fairly safe and requires no storage for the cookie. However the encrypted key must be decrypted using a clear text secret stored in the apache config file. This key could be taken and the operator would not know.


This has two bugs. It even documents the issues about questionable session key.

return time . $$ . int rand $$;

The problem is authentication always passes if you have a session key at all. Because to check the session key it uses (which basically just returns what it is given).

sub authen_ses_key ($$$) {
    my ($self, $r, $session) = @_;
    # Validate the session and convert it into REMOTE_USER
    # This is using the session key as the REMOTE_USER
    return $session;
    # This returns undef so no REMOTE_USER is set sending back to login form
    # Make sure there IS a login form before doing this.
    return undef;

NOTE: It is recommended by the author that this not be used in production (although he also notes it is being used).


Uses encrypted cookies rather than tokens, the same as AuthCookieDBI


It all comes down to php_combined_lcg and how random that is. It is certainly safer than most of the modules mentioned above.

    function session_regenerate_id() {
        global $base_uri;
        $tv = gettimeofday();
        $buf = sprintf("%.15s%ld%ld%0.8f", $remote_addr, $tv['sec'], $tv['usec'], php_combined_lcg() * 10);
        if (ini_get('session.use_cookies')) {
            // at a later stage we use sqsetcookie. At this point just do
            // what session_regenerate_id would do
            setcookie(session_name(), session_id(), NULL, $base_uri);
        return TRUE;

By having a single way of generating the tokens used in sessions PHP has slowly improved the code. Looking back to 2001 it was certainly way simpler and only did very simple random number generation.

Using Data::Token in our modules will produce the same effect. Even if we do find a problem with Data::Token, at least we can fix it in one place.

Alternatives to Tokens

For authentication, using encrypted tokens is an alternative with a new set of issues to deal with.



For apache web based authentication use an Apache module (for any language). If you must write your own, otherwise try Apace::AuthCookie and generate tokens with Data::Token.

Storing these tokens as a cookie on clear text sessions is safe enough. But remember that your data would not be protected (if that is important) and your session could be hijacked while the cookie is still valid.


It is common that we need to send a private URL to a user. This is done in applications now like Google Video. Other file sharing systems generate tokens for sending an email with a URL to a private file. This is a good example of useful tokens.

HTML Forms

Using a token in a form helps to protect many forms of attack. Encrypted data in a form however may be an easier option as it does not require storage.

TODO - find a reference.

Invitation Code

You want to send someone a code to allow you to sign up. You can only use it once, but you want it to be hard to guess - use a token.

Implementation Issues

You want to use a token, what are your issues?

Duplicate Tokens

It is highly unlikely you will ever see a duplicate token. However, as we are taking a hash of the entry with SHA-1 it is possible. Therefore new tokens generated should always be checked first.

Brute Force Attack

Even with the unpredictability added to this system it is still possible to brute force attack. Any implementation should protect against this with standard measures such as:

  • Delay return on response (to work it would need to delay on success or failure)
    • This one is difficult as it slows down the system therefore...
  • Increase delay (success and failure) to a particular IP on repeated failures
  • Eventually block an IP/range.

TODO - protecting cookies and auth against a brute force attack needs a reference.

Generating the Token

Login systems that use a token, or URLs that are generated against an object, or any other reason to generate a token still need security. E.g. if you provide a Login/Password and set a token (your basic cookie based web authentication system) - you should consider using HTTPS and other devices to protect those clear text name/passwords.

Yet more checks

If you are using tokens for authentication and you want to be even safer, you should check anything you can tell about the user. What IP they are coming from (but remember that a proxy often has many), what browser name and version are they using. This helps to reduce the possibility that a token is being attacked.

You can also change the token on each request. This is extreme and has quite a bit of overhead but useful. Alternatives may also be to change it over short periods, like 5 minutes.


Data::Token produces quite long tokens. This is to lean to the side of security. However tokens used for invitations and the like can be much shorter. This is something to consider for your implementation - can you accept long tokens - e.g. 160 characters?

The Code

Although I recommend using Data::Token to do the work for you, for the purposes of review and use by other languages, the code is here:



A bit thanks to the crew of Melbourne Perl Mongers on a huge thread helping me produce a more secure Data::Token.

See Also

  • Perl
  • Security