Showing posts with label CPAN. Show all posts
Showing posts with label CPAN. Show all posts

Tuesday, 30 September 2014

IO::Iron Policies - No Typing Errors to Iron.io Services!

Policies is a way to limit the names of message queues, code packages, caches and items (item keys) to a predefined group of possible strings. This can limit the chances for typos and enforce an enterprise policy. The policies are loaded from a JSON file which is specified either when creating a IO::Iron::Iron*::Client object, or in the config file .iron.json (or equivalent).

Policies in Config file

Add the item policies to the config file. The value of the item is the file name of the policies file.

Example config file:

    {
        "project_id":"51bdf5fb2267d84ced002c99",
        "token":"-Q9OEHZPhdZtd0KHBzzdUJIqV_E",
        "host":"cache-aws-us-east-1.iron.io",
        "policies":"iron_policies.json"
    }

Policies file specified when creating the client

    my $policies_filename = '/etc/ironmq/global_policies.json';
    my $client = IO::Iron::IronCache::Client->new('policies' => $policies_filename);

Examples of Policies File and Explanation of Configuration

The 'default' policies JSON file:

    (
    'definition' => {
        'character_group' => {
        },
        'no_limitation' => 1, # There is an unlimited number of alternatives.
    },
    'queue' => { 'name' => [ '[:alnum:]{1,}' ], },
    'cache' => {
        'name' => [ '[:alnum:]{1,}' ],
        'item_key' => [ '[:alnum:]{1,}' ]
        },
    'worker' => { 'name' => [ '[:alnum:]{1,}' ], },
    );

The above file would set an open policy for IronMQ, IronCache and IronWorker alike. The file is divided into four parts: definition for defining meta options, and queue|cache|worker parts for defining the changing strings (queue|cache|worker names and item keys). The character group alnum covers all ascii alphabetic characters (both lower and upper case) and digits (0-9).

N.B. The option no_limitation controls the open/closed policy. If no_limitation is set (1=set), the policy control is turned off.

An example of policies file

    {
        "__comment1":"Use normal regexp. [:digit:] = number:0-9, [:alpha:] = alphabetic character, [:alnum:] = character or number.",
        "__comment2":"Do not use end/begin limitators '^' and '\$'. They are added automatically.",
        "__comment3":"Note that character groups are closed inside '[::]', not '[[:]]' as normal POSIX groups.",
        'definition' => {
            'character_group' => {
                "[:lim_uchar:]":"ABC",
                "[:low_digit:]":"0123"
            },
        },
        "cache":{
            "name":[
                "cache_01_main",
                "cache_[:alpha:]{1}[:digit:]{2}"
            ],
            "item_key":[
                "item.01_[:digit:]{2}",
                "item.02_[:lim_uchar:]{1,2}"
            ]
        }
    }

This policies file sets policies for cache names and item keys. Both have two templates. Template "cache_01_main" is without wildcards: the template list can also only contain predefined names or keys. Sometimes this could be exactly the wanted behaviour, especially in regard to cache and message queue names.

Items beginning with '__' are considered comments. Comments can not be inserted into lists, such as I.

The definition part contains the list character_group for user-defined groups. The following groups are predefined:

[:alpha:]
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
[:alnum:]
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789
[:digit:]
0123456789
[:lower:]
abcdefghijklmnopqrstuvwxyz
[:upper:]
ABCDEFGHIJKLMNOPQRSTUVWXYZ
[:word:]
ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789_

All lower ASCII (7-bit) characters are allowed in names and in character groups, except for the reserved characters (RFC 3986):

!$&'()*+,;=:/?#[]@

A character group definition is closed inside characters '[::]', not '[[:]]' as normal POSIX groups. Only the equivalents of the POSIX groups mentioned above can be used; e.g. POSIX group [[:graph:]] is not available.

When using the character groups in a name or key, only two markings are allowed: [:group:]{n} and [:group:]{n,n}, where 'n' is an integer. This limitation (not being able to use any regular expression) is due to the double functionality of the policy: a) it acts as a filter when creating and naming new message queues, code packages, caches and cache items; 2) it can be used to list all possible names, for example when quering for cache items.

Sunday, 31 August 2014

IO::Iron::Applications - Command line tools for Iron.io services

IO::Iron::Applications is an auxiliary package for IO::Iron. IO::Iron contains the library for using the Iron.io cloud services in Perl programs. IO::Iron::Applications contains command line programs to operate those services.

IO::Iron::Applications is my addition to the IO::Iron interphase library package which I wrote earlier. The Iron.io WWW interface hud.iron.io Dashboard is great but a bit slow to use when you only need to quickly change some values in IronCache, send a message to IronMQ, erase or empty cache for debugging purposes or other similar activities. With these command line utilities the same functions can be performed fast from a normal shell and not using a web browser.

Policies

The programs take use of the IO::Iron package feature policies so wildcard characters can be used in cache names, item keys, etc.

For example, if iron_cache_policies.json:

    {
        "definition":{
            "character_group":{
                "[:lim_uchar:]":"ABC",
                "[:low_digit:]":"01"
            }
        },
        "cache":{
            "name":[
                "cache_[:lim_uchar:]{1}0[:digit:]{1}"
            ],
            "item_key":[
                "item.02_[:lim_uchar:]{1,2}[:low_digit:]{1}"
            ]
        }
    }

then

    ironcache list items .* --cache cache_A01 --policies iron_cache_policies.json

would print out:

    Cache                         Item                expires
    cache_A01                     item.02_A0                               Key not exists.
    cache_A01                     item.02_A1                               Key not exists.
    cache_A01                     item.02_AA0                              Key not exists.
    cache_A01                     item.02_AA1                              Key not exists.
    cache_A01                     item.02_AB0                              Key not exists.
    cache_A01                     item.02_AB1                              Key not exists.
    cache_A01                     item.02_AC0                              Key not exists.
    cache_A01                     item.02_AC1                              Key not exists.
    cache_A01                     item.02_B0                               Key not exists.
    cache_A01                     item.02_B1                               Key not exists.
    cache_A01                     item.02_BA0                              Key not exists.
    cache_A01                     item.02_BA1                              Key not exists.
    cache_A01                     item.02_BB0                              Key not exists.
    cache_A01                     item.02_BB1                              Key not exists.
    cache_A01                     item.02_BC0                              Key not exists.
    cache_A01                     item.02_BC1                              Key not exists.
    cache_A01                     item.02_C0                               Key not exists.
    cache_A01                     item.02_C1                               Key not exists.
    cache_A01                     item.02_CA0                              Key not exists.
    cache_A01                     item.02_CA1                              Key not exists.
    cache_A01                     item.02_CB0                              Key not exists.
    cache_A01                     item.02_CB1                              Key not exists.
    cache_A01                     item.02_CC0                              Key not exists.
    cache_A01                     item.02_CC1                              Key not exists.

On the command line, all normal regular expression are allowed. E.g.

    item.02_A.{1}0

would return

    Cache                         Item                expires
    cache_A01                     item.02_AA0                              Key not exists.
    cache_A01                     item.02_AB0                              Key not exists.
    cache_A01                     item.02_AC0                              Key not exists.

Following command line programs are available:

ironcache

clear: Clear a cache.
E.g. ironcache clear cache_main
delete: Delete a cache.
E.g. ironcache delete cache_main
delete: Delete item from cache.
E.g. ironcache delete item item.01_AB1
get: Get item/items from cache/caches.
E.g. ironcache get item item.02_A.{2} --cache cache_A01 --config iron_cache.json --policies iron_cache_policies_test_01.json --warn
increment: Increment an item/items in cache/caches.
E.g. ironcache increment item item.02_AC1,item.02_BC1 --cache cache_A01 --value 225
list: List caches or items in a cache/caches.
E.g. ironcache list items .* --cache cache_A01
E.g. ironcache list caches
put: Put or replace item/items to a cache/caches.
E.g. ironcache put item item.02_CC1,item.02_CC2 --cache cache_A01 -- value 123
show: Show the properties of a cache/caches.
E.g. ironcache show cache cache_A01