PREFACE

This document provides general information about Test::Async. Technical details are provided in corresponding modules.

General test framework use information can be found in the documentation of Raku's standard Test suite. Test::Async::Base provides information about differences and additions between the standard framework and Test::Async.

INTRODUCTION

Terminology

Throughout documentation the following terms are to be used:

Test suite

This term can have two meanings:

The particular meaning is determined by a context or some other way.

Test bundle or just bundle

A module or a role implementing a set of test tools or extending/modifying the core functionality. A bundle providing the default set of tools is included into the framework and implemented by Test::Async::Base.

Reporter

A test bundle which provides reporting capabilities. For example, Test::Async::Reporter::TAP implements TAP output.

Test tool

This is a routine provided by a bundle to test a condition. Typical and commonly known test tools are pass, flunk, ok, nok, etc.

ARCHITECTURE

The framework is built around test suite objects driven by events. Suites are organized with parent-child relations with a single topmost suite representing the main test compunit. Child suites are subjects of a job manager control.

A typical workflow consist of the following steps:

Test Suite Creation

On startup the framework constructs a custom Test::Async::Suite class which incorporates all core functionality and extensions provided by bundles. The following code:

use Test::Async;
say test-suite.^mro(:roles).map( *.^shortname ).join(", ")

results in:

Suit, Base_class, Base, TAP_class, TAP, Reporter, Hub, JobMgr, Aggregator, Any, Mu
1..0

Note that :roles named parameter is available since Rakudo compiler release 2020.01.

Next paragraphs are explaining where this output comes from.

Let's start with bundles. One is created with either test-bundle or test-reporter keyword provided by Test::Async::Decl module. For example:

test-bundle MyBundle {
    method my-test($got, $expected, $message) is test-tool {
        ...
    }
}

In fact it is nothing else but a role declaration but with two important side effects:

The second item means that this code:

use MyBundle;
use Test::Async;
plan 1;
my-test pi, 2*pi, "whatever";

would just work. BTW, if one would try to dump parents and role of the suite object, as show above, he would get:

Suit, MyBundle_class, MyBundle, TAP_class, TAP, Reporter, Hub, JobMgr, Aggregator, Any, Mu

Becase the framework skips loading the default bundle if there is one explicitly requested by a user. Same applies for TAP which is the default reporter bundle and which wouldn't be loaded if the user uses an alternative.

When all bundles were loaded and registered, time comes for Test::Async module to actually construct the suite class.

Note that this is why Test::Async must always be used last. No bundle registered post-suite construction would be actually used.

The construction algorithm could roughly be written as:

Putting this into a diagram would give us something like this for the default case:

.         Suite -> Base_class -> TAP_class -> Hub -> Any -> Mu
.                  |             |
. bundle roles:    Base          TAP

See example script: examples/multi-bundle.raku

This approach allows custom bundles easily extend the core functionality or even override certain aspects of it. The latter is as simple as overriding parent methods. For example, Test::Async::Base module uses this technique to implement test-flunks tool. It is doing so by intercepting test events passed in to send-test method of Test::Async::Hub. It is then inverts test's outcome if necessary and does few other adjustments to a new test event profile and passes on the control to the original send-test to complete the task.

Job Management

The asynchronous nature of the framework requires a proper job management subsystem. It is implemented by Test::Async::JobMgr role and Test::Async::Job class representing a single job to be done. The subsystem implements the following concepts:

A job is Code instance accompanied with its associated attributes. Code return value is never provided directly but only via a fulfilled Promise.

The way the manager works is it creates a pool (not a queue) of jobs. The order in which they're executed is defined by the user code invoking them. When a job completes the manager removes it from the pool. Though not directly manager's job, but it provides a possibility to postpone a job. In this case it is placed into a queue from where it could be picked up and invoked any time it is needed. For example, Test::Async::Hub is using this to invoke child suites in a random order: jobs for corresponding suites are postponed and when the main code block of the parent suite finishes it takes the postponed queue, shuffles jobs in it and invokes them in the resulting order.

Events

C<Test::Async> framework handles concurrency using event-driven flow control. Each event is an instance of a class
inheriting from
L<C<Test::Async::Event>|https://github.com/vrurg/raku-Test-Async/blob/v0.0.1/docs/md/Test/Async/Event.md> class. Events
are queued using a L<C<Channel>|https://docs.raku.org/type/Channel> where they're read from by a dedicated thread and
dispatched for handling by suite object methods. So it makes each suit own at least two threads: first is for tests
themselves, the other one is for event handling.

   Thread#1 \
             \
   Thread#2 --> [Event Queue] -> Event Handler Thread
             /
   Thread#3 /

The approach allows to combine the best of two worlds: speed of asynchronous operations and predictability of sequential code. In particular, it proves to be useful for object state changes like, for example, for collecting messages from child suites ran asynchronously. Because the messages are stashed in an Array the procedure is prone to race condition bugs. But when the responsibility of updating the array is in hands of a single thread it greatly simplifies the task.

Another advantage of the events is the ease of extending the framework functionality. Look at Test::Async::Reporter::TAP, for example. It takes the burden of reporting to user on its 'shoulders' unloading it off the core. And it does so simply by listening to Event::Test kind of events. It would be as easy to implement an alternative reporter to get the test results be sent anywhere!

Suite Plan And Lifecycle

Suite has a number of parameters affecting it's execution. Those are:

While executed, the suite passes a few stages:

The parameters can only be set or changed while suite is being initialized and no test tools can be invoked at and after the finished stage.

Worth noting that finishing stage is basically same as in progress except that it indicates that the time of postponed jobs has come.

Test Tools

A test tool is a method with test-tool trait applied. It has two properties:

test-bundle Test::Foo {
    method test-foo(...) is test-tool(:!skippable, :!readify) { ... }
    method test-bar(...) is test-tool { ... }
}

SEE ALSO

Test::Async::CookBook, Test::Async::Base, Test::Async::Hub, Test::Async::Event, Test::Async::Decl, Test::Async::X, Test::Async::Utils,

AUTHOR

Vadim Belman <vrurg@cpan.org>