CPAN Testers Statistics

Latest Summary - 3rd January 2009

CPAN Testers Stats - December Summary - Seasons End

December was quite a hive of activity, what with the new Preferences Administration site launch and the reworked Reports site also going live. There is still another member of the website family to be unveiled, but that might be a little longer arriving. In the meantime work towards CT2.0 is being seen as a higher priority for the moment. The data generation tools now encompass a MySQL database update, which is used to feed the Reports site. The first benefits from the change to the MySQL database have been to enable the new Reports site to go live, however in the longer term the AJAX API to data snippets are expected to be used on search.cpan.org, and pretty much anywhere that people can make use of it.

A big data cleanup was also undertaken last month with several entries getting sanitised. As some of the reports get truncated or parsing fails partially, some of the fields, such as osname and osvers can often be left blank or contain erroneous data. As such cleaning this up helps to provide more accurate numbers for the different platforms. Until we move to the complete HTTP submission system, it is likely that some descrepencies will still slip through, so if you spot any please let me know and I'll investigate where I can.

As mentioned last month, the 2009 QA Hackathon event in Birmingham will be featuring work on the CPAN Testers infrastructure. With several key collaborators planning to be there, we're hoping that a lot of the plans to get CT2.0 up and working consistently for all smokers will come to fruition.

We topped 135 testers submitting reports last month, so thank you very much to everyone involved. The mappings this month included 68 total addresses mapped, of which 23 were for newly identified testers.

One side effect of being able to map so many addresses to actual people over the last year, has meant the totals per month keep getting adjusted. With the most recent mappings several months that previous listed over 130 testers, have seen their counts notably reduced, to the point that now December 2008 is the only month to have listed more than 130 testers in a month. I expect that number to reduce too once there are a few addresses mapped in the future. As a result of all the mappings recently, I've held off my unmapped address mailout for the last few months, but I plan to sort that out this month. In the complete database we have 564 unmapped addresses, although only 68 were used during 2008. With a total of 2749 addresses being used, 564 is a significantly high number. My hope is that moving to the new HTTP submission system and releasing the new member of the website family will also help to reduce that number, especially those who use 'example.com' or 'ESMTP' as their domains. CT2.0 is coming ... soon :)

If you want to get involved in CPAN Testing, please checkout the CPAN Testers Wiki for details of how to set things up. You can also join the CPAN Testers Discussion mailing list if you need any further advice or if you have any suggestions for improving the tools or resources.

The Statistics

Everyone loves statistics, and here are several more for you to have fun with. I originally produced these statistics to spot trends in any testing of CPAN that was occurring, but I also wanted to see more clearly what platforms and versions of perl were being tested. The results were rather enlightening, so I extended the analysis to look at the complete recorded history of CPAN Testers. The first productive results of this exercise was to submit patches to the CPAN::WWW::Testers and CPAN::WWW::Testers::Generator modules, that now include the Perl version and platform in the test report summaries. Hopefully future results will be to inspire others to be involved with CPAN testing.

The History

The CPAN testers was conceived back in May 1998 by Graham Barr and Chris Nandor as a way to provide multi-platform testing for modules. Today over 1 million tester reports have been submitted by over 1,000 testers, who have given valuable feedback for users and authors alike.

How To Become A Tester

Whether you have a common platform or a very unusual one, you can help by testing modules you install and submitting reports. There are plenty of module authors who could use test reports and helpful feedback on their modules and distributions. If you'd like to get involved, please take a look at the CPAN Testers Wiki, and read up on who you can become a CPAN Testers with CPAN/CPAN::Reporter or CPAN::YACSmoke, how join the cpan-testers-discuss mailing list and what happens when you start submitting your reports.

Help Refine The Stats

In creating all these statistics, it has taken a notable amount of time to match email addresses to real people. There are a significant number who aren't CPAN authors, and many who have only contributed a few reports. As a consequence, there are still many many email addresses I haven't been able to trace. If you spot an email address that you've used to send reports, please let me know, and I'll update ASAP.

The Code & The Database

Want to see the code and current database for the site, then look no further.