GCC Automated Testing System

The GCC automated continuous testing system is, at present, a single machine which waits for a checkin to the GCC Subversion repository; checks out a URL of that tree; checks that it builds (for the native case, checks that it bootstraps); and runs the testsuite, checking whether any test which used to pass now fails. If any of the checks fails, it determines from the ChangeLog entries which checkins were made and sends mail to their authors.

The tester tests an i686-pc-linux-gnu native.

The following information is available:

• current status of the tester also available as a continuously updated JavaScript version;
• logfile containing the results of each test;
• log output of the last build for native;
• .log and .sum files from the last time the tree successfully built for native;
• The list of regressions (tests that used to pass and now fail) from the last time the tree successfully built for native;
• list of tests that are known to pass, for native.

The mail from the tester reporting failures is also sent to the gcc-regression mailing list. The tester also uses the test_summary script to send mail of its successful runs to the gcc-testresults mailing list.

If idle, the tester automatically tries to update its dejagnu and other tools daily, if the GCC tree is stable:

• current status;
• logfile;
• log output of the last build;

Occasionally people ask "Why did I get sent mail? I didn't break anything!" These are the reasons why you might get sent mail by the tester:

• Your patch actually did break something. Yes, I know your patch couldn't possibly have caused the behaviour shown, but stranger things have happened.
• Your patch was tested at the same time as another patch which caused the problem. To provide results faster, the tester doesn't check each patch separately. One feature that would be nice, but isn't implemented yet, is to have the tester look more closely at the failure and try to eliminate patches that couldn't possibly have caused it (for instance, a change to the mips backend can't cause a regression test failure on x86 native).
• The tester sent you mail to warn you that it couldn't fully test your patch, because the tree didn't build. This is not supposed to happen very often; when I was designing the tester, I didn't expect the tree to be unbuildable for weeks or months at a time. I believe the best way to reduce this mail is to try to ensure that the tree doesn't get broken, not by changing the tester. I'd rather not switch this off, because that would only make the underlying problem worse.