Write a tool that can run any other tool as a child, and collect its output and use it as a CPAN TESTERS test report.

ie: cpantester-reporter run cpanm -v Moose

or even cpantester-reporter -e "perl ./Makefile.PL" -e "make" -e "make test"

This could then by augmented and bolted on to almost any tool/suite that can install/test perl things, ie:
Gentoo, where we can permit all users to run test.
Debian, perhaps they could report test results during packaging.

etc.

Separating the test-reporting system from the install toolchain makes this way more useful, just implementing that sucker is the hard task.

There's an unfortunate thing that can happen with some List::* packages and thats some of the list-operating functions ( ie: all, any , first ) have a nice little bug where to can not actually pass it a list and be blissfully unaware of this.

ie:

return any { foobar( $_ ) }

A cursory glance suggests "Sure, that works", when in fact, any will iterate over an empty list seeing if foobar($_) returns true for any of them.... guess thats not very good.

Its a bit hard to solve this problem without LOTS of dark XS magic and even then, you have to patch it into the right list tools.

A Perl Critic policy suite that checks for missing parameters for the various things exported by the various *(ie: List::AllUtils ) package could come in handy.