MediaWiki code coverage howto

You can see some example results online at: http://files.nickj.org/MediaWiki/coverage/Parser.php.html and http://files.nickj.org/MediaWiki/coverage/Sanitizer.php.html

If you would like to reproduce these results, here's what you need to do:

Install XDebug

First install the xdebug PHP module (this provides the raw code coverage data).

How to install xdebug:

cd ~/tmp
mkdir xdebug
cd xdebug/
wget http://xdebug.org/link.php?url=xdebug200b6
tar -xzf xdebug-2.0.0beta6.tgz
cd xdebug-2.0.0beta6
phpize
./configure --enable-xdebug
make
cd ~
mkdir lib
cp ~/tmp/xdebug/xdebug-2.0.0beta6/modules/xdebug.so ~/lib
nano /etc/php5/apache2/php.ini
--------------------
# add this:
; Xdebug
zend_extension=/root/lib/xdebug.so
# and comment out the "Zend optimizer" stuff (if you have it installed), otherwise will get an error like this in the apache logs: 
# "PHP Fatal error:  [Zend Optimizer] Zend Optimizer 3.0.1 is incompatible with Xdebug 2.0.0beta6 in Unknown on line 0"
--------------------
apache2ctl stop
apache2ctl start

Then view a phpinfo() page, and check that the PHP version string includes something about Xdebug.

Install Spike's PHP coverage tool

Then install Spike's PHP coverage tool, which takes the raw data from Xdebug, and formats it into a nice report.

cd ~/tmp
mkdir phpcoverage
cd phpcoverage/
wget http://developer.spikesource.com/frs/download.php/63/spikephpcoverage-0.6.6.tar.gz
tar -xzf spikephpcoverage-0.6.6.tar.gz
cd spikephpcoverage-0.6.6

Small Changes to Spike's PHP coverage tool

Then you'll need to make a small name change in PHP coverage to prevent a clash with MediaWiki over the "Parser" class name. Files and line numbers changed were as follows:

- parser/PHPParser.php:24:    class PHPParser extends Parser {
+ parser/PHPParser.php:24:    class PHPParser extends SpikeParser {
- parser/Parser.php:32:    abstract class Parser {
+ parser/Parser.php:32:    abstract class SpikeParser {

Then you'll need to fix one small bug in Spike's PHP coverage tool, which causes it to choke on a regex in Parser.php ~line 400, which contains "?>", as it thinks this indicates the end of the PHP file, when it's not.

Line 31 of parser/PHPParser.php in phpcoverage, and change
       private $phpFinisher = '?>';
To something which never happens, like this:
       private $phpFinisher = "LKLKLKHLKH%%";

Update parserTests.php

Then apply this diff to parserTests.php. You'll need to change COVERAGE_REPORT_DIR , PHPCOVERAGE_HOME and $includePaths to appropriate paths :

diff -bBu3 parserTests.php.old parserTests.php
--- parserTests.php.old 2006-07-05 14:55:39.000000000 +1000
+++ parserTests.php     2006-07-05 17:44:59.000000000 +1000
@@ -23,6 +23,28 @@
  */

 /** */
+
+define ("CODE_COVERAGE", true);
+if (CODE_COVERAGE) {
+       error_reporting (E_ALL);
+       ini_set( 'memory_limit', '-1' ); // no memory limit
+
+       // Start the code coverage.
+       define("COVERAGE_REPORT_DIR", "/var/www/hosts/mediawiki/report");
+       exec("rm -rf " . COVERAGE_REPORT_DIR); // oh for a recursive unlink in PHP...
+
+       define("PHPCOVERAGE_HOME", "/root/tmp/phpcoverage/spikephpcoverage-0.6.6/src");
+       require_once PHPCOVERAGE_HOME . "/CoverageRecorder.php";
+       require_once PHPCOVERAGE_HOME . "/reporter/HtmlCoverageReporter.php";
+       $reporter = new HtmlCoverageReporter("ParserTests Parser.php Code Coverage Report", "", COVERAGE_REPORT_DIR);
+       $includePaths = array("/var/www/hosts/mediawiki/phase3/includes/Parser.php",
+                              "/var/www/hosts/mediawiki/phase3/includes/Sanitizer.php");
+       $excludePaths = array();
+       $cov = new CoverageRecorder($includePaths, $excludePaths, $reporter);
+       $cov->startInstrumentation();
+}
+
 require('parserTests.inc');

 if( isset( $options['help'] ) ) {
@@ -60,5 +82,12 @@
 }
 $ok = $tester->runTestsFromFile( $file );

+if (CODE_COVERAGE) {
+       // Finish up the code coverage.
+       $cov->stopInstrumentation();
+       $cov->generateReport();
+       $reporter->printTextSummary();
+}
+
 exit ($ok ? 0 : -1);
 ?>

Run parserTests

Then run parserTests:

root@bling:/var/www/hosts/mediawiki/wiki/maintenance# php ./parserTests.php --quiet --quick --color=no
Running test Table security: embedded pipes (http://mail.wikipedia.org/pipermail/wikitech-l/2006-April/034637.html)... FAILED!
Running test Link containing double-single-quotes '' (bug 4598)... FAILED!
Running test Template with thumb image (wiht link in description)... FAILED!
Running test message transform: <noinclude> in transcluded template (bug 4926)... FAILED!
Running test message transform: <onlyinclude> in transcluded template (bug 4926)... FAILED!
Running test BUG 1887, part 2: A <math> with a thumbnail- math enabled... FAILED!
Running test Language converter: output gets cut off unexpectedly (bug 5757)... FAILED!
Running test HTML bullet list, unclosed tags (bug 5497)... FAILED!
Running test HTML ordered list, unclosed tags (bug 5497)... FAILED!
Running test HTML nested bullet list, open tags (bug 5497)... FAILED!
Running test HTML nested ordered list, open tags (bug 5497)... FAILED!
Running test Parsing optional HTML elements (Bug 6171)... FAILED!
Running test Inline HTML vs wiki block nesting... FAILED!
Running test Mixing markup for italics and bold... FAILED!

Passed 393 of 407 tests (96.56%) FAILED!

##############################################
 Code Coverage Summary: ParserTests Parser.php Code Coverage Report
   Total Files: 2
   Total Lines: 5911
   Total Covered Lines of Code: 2035
   Total Missed Lines of Code: 428
   Total Lines of Code: 2463
   Code Coverage: 82.62%
##############################################
root@bling:/var/www/hosts/mediawiki/wiki/maintenance#

Then you can view your reports, which will be in HTML format in the COVERAGE_REPORT_DIR directory you specified earlier. Enjoy!