4. <report>/report

<report>/report — offline coverage report generator.

4.1. Description

<report>/report is EMMA's offline coverage report generator. It reads in an arbitrary number of data files containing class metadata and runtime coverage data and generates one or several coverage reports of desired types. Several aspects of coverage reporting (detail level, column order, column sorting, coverage metrics failure markup, etc) can be customized for a given report type.

What is reported on. Each invocation of <report>/report requires a set of input metadata and runtime coverage data files. EMMA coverage stats are derived exclusively from the classes that appear in the combined class metadata as represented by this input. To put it differently, a coverage report can reflect as much as the state of the entire product codebase or as little as one Java package or API being worked on by a given developer at the moment.

Report depth. To understand EMMA's approach to generating coverage reports, the following paradigm should be kept in mind:

  • a given coverage report covers all entities in the instrumentation set, referred to as all classes in the reports
  • all classes entity contains Java packages
  • [for classes compiled with full debug info] Java packages contain Java source files
    • [for classes compiled with full debug info] Java source files contain Java classes (in general, more than one)
    • [for classes compiled without full debug info] Java packages contain Java classes

  • Java classes contain methods (which, in turn, could be broken down into basic blocks)
(The reason EMMA makes a distinction between classes with and without full debug info is that without the SourceFile attribute in all input classes it is in general impossible to make the association between classes and their source files and that in turn impacts how metrics like line coverage are rolled up. The above hierarchy is easier to understand if you realize that without the full debug info the source file hierarchy level is absent.)

Correspondingly, <report>/report calculates and presents coverage metrics in a way that allows for drilling down into data in a top-down fashion, starting with all classes and going all the way to the level of individual methods and source lines (in the HTML report). Coverage metrics are rolled up at the levels of individual methods, classes, source files, packages, and for the entire instrumentation set (all classes). The concept of "report depth" represents how deep you are in this hierarchy.

Different report types produced by <report>/report differ in how they reflect this data hierarchy:

  • The plain text report is a low-overhead report type for quick coverage summary viewing and processing by tools like grep and Perl. It starts with an all classes summary and progressively adds further drill-down sections. Because a columnar plain text format is limited in how well it can present hierarchical data, it is recommended that for report depths beyond all and package you use the HTML report instead.
  • The HTML report can provide the most detail and is intended for human viewing. It starts with an all classes summary page and for larger report depths links it to package summary pages and then further to individual source file and class summary pages. Source/class summary pages can further embed source files and show method coverage rollups as well as highlight individual source line coverage states.
  • The XML report exits for integration purposes and leverages the tree structure of an XML document to most truthfully represent the above-mentioned data hierarchy.
Because generating certain report types for very large projects can be time-consuming, reducing the default report depth is a good way to limit the amount of detail that is generated, a useful feature for individual development work.

Valid values for a report depth are all, package, source, class, and method. In general, a certain report depth value implies the level of detail that includes the summary for all items at that level as well as coverage breakdown summaries for their children. The amount of information rendered for a given depth value is always inclusive of lesser depth values, so increasing the report depth always increases the amount of details that is rendered. As a special case, when full debug info is available, class is equivalent to source.

Report units. EMMA coverage metrics could be unweighted or weighted, that is derived from basic block coverage where each block counts either with an equal weight or with a weight proportional to the number of Java bytecode instructions in it. The default <report>/report behavior is to use weighted metrics. This includes all metrics that are sensitive to basic block content: line and block coverage. Weighted basic block coverage is a recommended metric for all situations, because it can simulate line coverage when no debug information has been compiled into application classes. If desired, the traditional (unweighted) metrics could be selected using the units option.

Coverage metrics. A very useful feature of HTML and plain text reports created by <report>/report is the ability to highlight entities that fail a given coverage metric. The plain text report does it by appending a "!" to a failing coverage metric and the HTML report highlights those in red. Combined with ability to sort report columns, this feature allows an individual developer to zoom in to the packages and classes that demand the most attention with respect to coverage.

Sourcepath and source linking. Although EMMA coverage calculations are based on basic block coverage profiling, <report>/report can also map block coverage to Java source file lines. If the HTML report generator is set to method depth and is configured with a valid source path and the instrumented classes were compiled with enough debug information, the generator will embed source files in the source file/class summary report pages and highlight covered/not covered lines accordingly.

Sourcepath and coverage stats

Referencing the original Java source files is optional during coverage report generation and does not affect how EMMA coverage stats are computed (these stats are based entirely on the class metadata and the debug info available in the .class data at the instrumentation time). However, to avoid report generation errors it is your responsibility to ensure that the versions of Java sources used for reporting are the same as the ones used during instrumentation.