Difference between revisions of "Sonar + maven configuration + Jenkins"
Line 18: Line 18:
  
 
=='''Maven plugins'''==
 
=='''Maven plugins'''==
 +
 +
 +
===Configuration===
 +
 +
  
  
Line 25: Line 30:
  
  
−
By default, the Surefire Plugin will automatically include all test classes with the following wildcard patterns:
+
By default, the Surefire Plugin will automatically '''include''' all test classes with the following wildcard '''patterns''':
 
* <code>**/Test*.java</code> - includes all of its subdirectories and all Java filenames that start with "Test".
 
* <code>**/Test*.java</code> - includes all of its subdirectories and all Java filenames that start with "Test".
 
* <code>**/*Test.java</code> - includes all of its subdirectories and all Java filenames that end with "Test".
 
* <code>**/*Test.java</code> - includes all of its subdirectories and all Java filenames that end with "Test".
Line 33: Line 38:
  
 
See [https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html Maven Surefire plugin documentation]
 
See [https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html Maven Surefire plugin documentation]
 +
 +
 +
(i) It is safer to '''exclude''' the integration tests '''patterns''' as well, depending on your own development setup:
 +
* <code>**/*TestAPI.java</code>
 +
* <code>**/*IntegrationTest.java</code>
  
  
Line 38: Line 48:
 
* Results are saved in XML and TXT formats in <code>$module/target/surefire-reports/*</code>
 
* Results are saved in XML and TXT formats in <code>$module/target/surefire-reports/*</code>
 
* Results are saved in BINARY (jacoco format) in <code>$module/target/jacoco.exec</code>
 
* Results are saved in BINARY (jacoco format) in <code>$module/target/jacoco.exec</code>
 +
  
 
<syntaxhighlight lang="xml">
 
<syntaxhighlight lang="xml">
 +
 +
    <properties>
 +
        <!-- Set the reporting settings -->
 +
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
 +
    </properties>
 +
 +
 
     <build>
 
     <build>
 
         <plugins>
 
         <plugins>
Line 85: Line 103:
  
  
−
<syntaxhighlight lang="XML">
+
<syntaxhighlight lang="xml">
 +
 
 +
    <properties>
 +
        <!-- Set the reporting settings -->
 +
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
 +
    </properties>
 +
 
 +
 
 
     <build>
 
     <build>
 
         <plugins>
 
         <plugins>
Line 114: Line 139:
 
     </build>
 
     </build>
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
 +
 +
 +
 +
===Jacoco===
 +
 +
Jacoco is a plugin that will:
 +
# Parse SUREFIRE and FAILSAFE results
 +
# Extract and generate code coverage reports (XML format)
 +
# Convert the XML into binaries (*.exec) for SonarQube
 +
 +
 +
=> Jacoco will generate the following reports:
 +
* 1 unit test report ''jacoco.exec'' per project
 +
* 1 centralized integration test report ''jacoco-it.exec'' for all projects
 +
 +
 +
 +
<syntaxhighlight lang="xml">
 +
 +
    <properties>
 +
        <!-- Project configuration -->
 +
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
 +
        <!-- Sonar -->
 +
        <!-- Tell sonar where to look for the binaries coverage files. Property inherited by submodules -->
 +
        <sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
 +
    </properties>
 +
 +
 +
    <build>
 +
        <plugins>
 +
            ...
 +
 +
            <!-- JACOCO test coverage plugin.
 +
                Use it to compile SUREFIRE (unit tests) and FAILSAFE (integration tests) reports for SonarQube
 +
                (i) attach that plugin to Maven TEST phase
 +
                Reports are generated in "${project.build.directory}/site/jacoco/*" by default
 +
                Good documentations:
 +
                    https://wiki.onap.org/display/DW/Implementing+Code+Coverage
 +
                    https://www.devcon5.ch/en/blog/2015/05/29/multi-module-integration-test-coverage-sonar-jacoco/
 +
                    https://www.eclemma.org/jacoco/trunk/doc/maven.html
 +
                -->
 +
            <plugin>
 +
                <groupId>org.jacoco</groupId>
 +
                <artifactId>jacoco-maven-plugin</artifactId>
 +
                <version>0.8.3</version>
 +
                <configuration>
 +
                    <append>true</append>
 +
                </configuration>
 +
                <executions>
 +
                    <!-- ## UNIT TESTS ## -->
 +
                    <!-- Configure JaCoCo runtime agent. It is passed as VM argument when Maven SUREFIRE plugin is executed. -->
 +
                    <execution>
 +
                        <id>pre-unit-tests</id>
 +
                        <goals>
 +
                            <goal>prepare-agent</goal>
 +
                        </goals>
 +
                    </execution>
 +
                    <!-- Create reports -->
 +
                    <execution>
 +
                        <id>report-unit-tests</id>
 +
                        <goals>
 +
                            <goal>report</goal>
 +
                        </goals>
 +
                    </execution>
 +
                    <!-- ## INTEGRATION TESTS ## -->
 +
                    <!-- Configure JaCoCo runtime agent. It is passed as VM argument when Maven FAILSAFE plugin is executed. -->
 +
                    <execution>
 +
                        <id>pre-integration-tests</id>
 +
                        <goals>
 +
                            <goal>prepare-agent-integration</goal>
 +
                        </goals>
 +
                        <configuration>
 +
                            <!-- Only 1 destination file to aggregate ALL integration tests reports -->
 +
                            <!-- the "session.executionRootDirectory" = parent folder that is being build by Jenkins -->
 +
                            <destFile>${session.executionRootDirectory}/target/jacoco-it.exec</destFile>
 +
                            <append>true</append>
 +
                        </configuration>
 +
                    </execution>
 +
                    <!-- Create reports -->
 +
                    <execution>
 +
                        <id>report-integration-tests</id>
 +
                        <goals>
 +
                            <goal>report-integration</goal>
 +
                        </goals>
 +
                    </execution>
 +
                </executions>
 +
            </plugin>
 +
 +
        </plugins>
 +
    </build>
 +
</syntaxhighlight>
 +
  
  
Line 125: Line 244:
  
  
−
<syntaxhighlight lang="XML">
+
<syntaxhighlight lang="xml">
 +
 
 +
    <properties>
 +
        <!-- Sonar -->
 +
        <!-- Tell sonar where to look for the binaries coverage files. Property inherited by submodules -->
 +
        <sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
 +
    </properties>
 +
 
 +
 
 
     <build>
 
     <build>
 
         <plugins>
 
         <plugins>
Line 235: Line 362:
 
                 Good documentations:
 
                 Good documentations:
 
                     https://wiki.onap.org/display/DW/Implementing+Code+Coverage
 
                     https://wiki.onap.org/display/DW/Implementing+Code+Coverage
 +
                    https://www.devcon5.ch/en/blog/2015/05/29/multi-module-integration-test-coverage-sonar-jacoco/
 
                     https://www.eclemma.org/jacoco/trunk/doc/maven.html
 
                     https://www.eclemma.org/jacoco/trunk/doc/maven.html
 
                 -->
 
                 -->
Line 268: Line 396:
 
                         </goals>
 
                         </goals>
 
                         <configuration>
 
                         <configuration>
−
                             <destFile>${project.basedir}/target/jacoco-it.exec</destFile>
+
                            <!-- Only 1 destination file to aggregate ALL integration tests reports -->
 +
                            <!-- the "session.executionRootDirectory" = parent folder that is being build by Jenkins -->
 +
                             <destFile>${session.executionRootDirectory}/target/jacoco-it.exec</destFile>
 
                             <append>true</append>
 
                             <append>true</append>
 
                         </configuration>
 
                         </configuration>
Line 278: Line 408:
 
                             <goal>report-integration</goal>
 
                             <goal>report-integration</goal>
 
                         </goals>
 
                         </goals>
−
                    </execution>
 
−
                    <!-- ## MERGE UNIT TESTS reports ## -->
 
−
                    <!-- This will merge all unit tests reports into 1 so it can be considered as another "integration" tests report -->
 
−
                    <execution>
 
−
                        <id>merge</id>
 
−
                        <goals>
 
−
                            <goal>merge</goal>
 
−
                        </goals>
 
−
                        <configuration>
 
−
                            <fileSets>
 
−
                                <fileSet implementation="org.apache.maven.shared.model.fileset.FileSet">
 
−
                                    <directory>${project.basedir}</directory>
 
−
                                    <includes>
 
−
                                        <include>**/*.exec</include>
 
−
                                    </includes>
 
−
                                </fileSet>
 
−
                            </fileSets>
 
−
                        </configuration>
 
 
                     </execution>
 
                     </execution>
 
                 </executions>
 
                 </executions>

Revision as of 10:49, 10 April 2019


This article explains how to configure your maven projects to produce reports + how to configure Jenkins with SonarQube, bringing both Unit and Integration tests coverage.


Requirements

  • Maven must run on Java 8 or +
  • You must use Sonarqube 7.x or +


Maven parent POM

You just have to define the configuration once, in the parent POM.


Maven plugins

Configuration

Surefire UNIT Tests

Surefire is for UNIT tests = tests that should only use: plain java / Mockito / PowerMock / etc.


By default, the Surefire Plugin will automatically include all test classes with the following wildcard patterns:

  • **/Test*.java - includes all of its subdirectories and all Java filenames that start with "Test".
  • **/*Test.java - includes all of its subdirectories and all Java filenames that end with "Test".
  • **/*Tests.java - includes all of its subdirectories and all Java filenames that end with "Tests".
  • **/*TestCase.java - includes all of its subdirectories and all Java filenames that end with "TestCase".

If the test classes do not follow any of these naming conventions, then configure Surefire Plugin and specify the tests you want to include.

See Maven Surefire plugin documentation


(i) It is safer to exclude the integration tests patterns as well, depending on your own development setup:

  • **/*TestAPI.java
  • **/*IntegrationTest.java


Results are saved in each Maven module:

  • Results are saved in XML and TXT formats in $module/target/surefire-reports/*
  • Results are saved in BINARY (jacoco format) in $module/target/jacoco.exec


    <properties>
        <!-- Set the reporting settings -->
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    </properties>


    <build>
        <plugins>
            <!-- To run UNIT tests and generate execution reports. These reports are required for SonarQube -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.1</version>
                <configuration>
                    <excludes>
                        <exclude>**/*TestAPI</exclude>
                        <exclude>**/*IntegrationTest</exclude>
                    </excludes>
                </configuration>
            </plugin>

            ...
        </plugins>
    </build>



Failsafe INTEGRATION tests

Failsafe will process the Integration Tests = tests that should use Spring test / black box tests / etc.


By default, the Failsafe Plugin will automatically include all test classes with the following wildcard patterns:

  • **/IT*.java - includes all of its subdirectories and all Java filenames that start with "IT".
  • **/*IT.java - includes all of its subdirectories and all Java filenames that end with "IT".
  • **/*ITCase.java - includes all of its subdirectories and all Java filenames that end with "ITCase".

If the test classes do not follow any of these naming conventions, then configure Failsafe Plugin and specify the tests you want to include.


Results are saved in each Maven module:

  • Results are saved in XML and TXT formats in $module/target/failsafe-reports/*
  • Results are saved in BINARY (jacoco format) in $module/target/jacoco-it.exec


Notes:

  • No tests will be run if mvn clean install -DskipIntegrationTests
  • The exclusion | inclusion list is something that depends on your own situation


    <properties>
        <!-- Set the reporting settings -->
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    </properties>


    <build>
        <plugins>

            <!-- To run INTEGRATION tests and generate execution reports. These reports are required for SonarQube -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.22.1</version>
                <configuration>
                    <includes>
                        <include>**/*TestAPI</include>
                        <include>**/*IntegrationTest</include>
                    </includes>
                </configuration>
                <executions>
                    <execution>
                        <id>default-integration-test</id>
                        <goals>
                            <goal>integration-test</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            ...
        </plugins>
    </build>



Jacoco

Jacoco is a plugin that will:

  1. Parse SUREFIRE and FAILSAFE results
  2. Extract and generate code coverage reports (XML format)
  3. Convert the XML into binaries (*.exec) for SonarQube


=> Jacoco will generate the following reports:

  • 1 unit test report jacoco.exec per project
  • 1 centralized integration test report jacoco-it.exec for all projects


    <properties>
        <!-- Project configuration -->
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <!-- Sonar -->
        <!-- Tell sonar where to look for the binaries coverage files. Property inherited by submodules -->
        <sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
    </properties>


    <build>
        <plugins>
            ... 

            <!-- JACOCO test coverage plugin.
                 Use it to compile SUREFIRE (unit tests) and FAILSAFE (integration tests) reports for SonarQube
                 (i) attach that plugin to Maven TEST phase
                 Reports are generated in "${project.build.directory}/site/jacoco/*" by default
                 Good documentations:
                    https://wiki.onap.org/display/DW/Implementing+Code+Coverage
                    https://www.devcon5.ch/en/blog/2015/05/29/multi-module-integration-test-coverage-sonar-jacoco/
                    https://www.eclemma.org/jacoco/trunk/doc/maven.html
                 -->
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>0.8.3</version>
                <configuration>
                    <append>true</append>
                </configuration>
                <executions>
                    <!-- ## UNIT TESTS ## -->
                    <!-- Configure JaCoCo runtime agent. It is passed as VM argument when Maven SUREFIRE plugin is executed. -->
                    <execution>
                        <id>pre-unit-tests</id>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                    </execution>
                    <!-- Create reports -->
                    <execution>
                        <id>report-unit-tests</id>
                        <goals>
                            <goal>report</goal>
                        </goals>
                    </execution>
                    <!-- ## INTEGRATION TESTS ## -->
                    <!-- Configure JaCoCo runtime agent. It is passed as VM argument when Maven FAILSAFE plugin is executed. -->
                    <execution>
                        <id>pre-integration-tests</id>
                        <goals>
                            <goal>prepare-agent-integration</goal>
                        </goals>
                        <configuration>
                            <!-- Only 1 destination file to aggregate ALL integration tests reports -->
                            <!-- the "session.executionRootDirectory" = parent folder that is being build by Jenkins -->
                            <destFile>${session.executionRootDirectory}/target/jacoco-it.exec</destFile>
                            <append>true</append>
                        </configuration>
                    </execution>
                    <!-- Create reports -->
                    <execution>
                        <id>report-integration-tests</id>
                        <goals>
                            <goal>report-integration</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

        </plugins>
    </build>



SONAR maven plugin

SonarQube released a maven plugin to work with SONAR. This plugin already includes most of the configuration and default behavior is correct.

  • Sonar will read Jacoco "exec" binaries files
  • Sonar will process the Surefire & Failsafe reports


    <properties>
        <!-- Sonar -->
        <!-- Tell sonar where to look for the binaries coverage files. Property inherited by submodules -->
        <sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
    </properties>


    <build>
        <plugins>
            ..

            <!-- SonarQube engine -->
            <plugin>
                <groupId>org.sonarsource.scanner.maven</groupId>
                <artifactId>sonar-maven-plugin</artifactId>
                <version>3.6.0.1398</version>
            </plugin>
        </plugins>
    </build>



Source code POM example

This is how the parent POM should look like to use SonarQube with correct coverage: 

    <properties>
        <!-- Project configuration -->
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>

        <!-- Sonar -->
        <!-- Tell sonar where to look for the binaries coverage files. Property inherited by submodules -->
        <sonar.jacoco.reportPaths>${project.basedir}/../target/jacoco.exec</sonar.jacoco.reportPaths>
    </properties>


    <build>
        <plugins>
            <!-- Build settings -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.3</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>

            <!-- Always generate the source, it is better for debugging -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-source-plugin</artifactId>
                <version>3.0.1</version>
                <executions>
                    <execution>
                        <id>attach-sources</id>
                        <goals>
                            <goal>jar-no-fork</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>attach-test-sources</id>
                        <goals>
                            <goal>test-jar-no-fork</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <!-- To run UNIT tests and generate execution reports. These reports are required for SonarQube -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.1</version>
                <configuration>
                    <excludes>
                        <exclude>**/*TestAPI</exclude>
                        <exclude>**/*IntegrationTest</exclude>
                    </excludes>
                </configuration>
            </plugin>

            <!-- To run INTEGRATION tests and generate execution reports. These reports are required for SonarQube -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
                <version>2.22.1</version>
                <configuration>
                    <includes>
                        <include>**/*TestAPI</include>
                        <include>**/*IntegrationTest</include>
                    </includes>
                </configuration>
                <executions>
                    <execution>
                        <id>default-integration-test</id>
                        <goals>
                            <goal>integration-test</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <!-- JACOCO test coverage plugin.
                 Use it to compile SUREFIRE (unit tests) and FAILSAFE (integration tests) reports for SonarQube
                 (i) attach that plugin to Maven TEST phase
                 Reports are generated in "${project.build.directory}/site/jacoco/*" by default
                 Good documentations:
                    https://wiki.onap.org/display/DW/Implementing+Code+Coverage
                    https://www.devcon5.ch/en/blog/2015/05/29/multi-module-integration-test-coverage-sonar-jacoco/
                    https://www.eclemma.org/jacoco/trunk/doc/maven.html
                 -->
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>0.8.3</version>
                <configuration>
                    <append>true</append>
                </configuration>
                <executions>
                    <!-- ## UNIT TESTS ## -->
                    <!-- Configure JaCoCo runtime agent. It is passed as VM argument when Maven SUREFIRE plugin is executed. -->
                    <execution>
                        <id>pre-unit-tests</id>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                    </execution>
                    <!-- Create reports -->
                    <execution>
                        <id>report-unit-tests</id>
                        <goals>
                            <goal>report</goal>
                        </goals>
                    </execution>
                    <!-- ## INTEGRATION TESTS ## -->
                    <!-- Configure JaCoCo runtime agent. It is passed as VM argument when Maven FAILSAFE plugin is executed. -->
                    <execution>
                        <id>pre-integration-tests</id>
                        <goals>
                            <goal>prepare-agent-integration</goal>
                        </goals>
                        <configuration>
                            <!-- Only 1 destination file to aggregate ALL integration tests reports -->
                            <!-- the "session.executionRootDirectory" = parent folder that is being build by Jenkins -->
                            <destFile>${session.executionRootDirectory}/target/jacoco-it.exec</destFile>
                            <append>true</append>
                        </configuration>
                    </execution>
                    <!-- Create reports -->
                    <execution>
                        <id>report-integration-tests</id>
                        <goals>
                            <goal>report-integration</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

            <!-- SonarQube engine -->
            <plugin>
                <groupId>org.sonarsource.scanner.maven</groupId>
                <artifactId>sonar-maven-plugin</artifactId>
                <version>3.6.0.1398</version>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        ...
  
        <!-- jUnit -->
        <dependency>
             <groupId>junit</groupId>
             <artifactId>junit</artifactId>
             <version>${junit.version}</version>
             <scope>test</scope>
        </dependency>      
    </dependencies>




Jenkins configuration

Now that you can generate these reports, especially the integration results, you need to configure Jenkins to use them!

Jenkins SonarQube configuration


To set up the Sonar Jenkins plugin to process already generated JaCoCo data files, the following properties must be specified in the "Additional properties" text box, under the Sonar installation being configured:

  • -Dsonar.dynamicAnalysis=reuseReports ==> Tells SonarQube to reuse existing reports for unit tests execution and coverage reports
  • -Dsonar.jacoco.itReportPath=${ABSOLUTE_PATH_TO_JENKINS}/workspace/${JENKINS_JOB}/target/reports/jacoco-it.exec => integration tests report to process



Java

Since jUnit 4.11 you can set the order of your tests.


References

This article is based on my daily work in VEHCO + European Parliament.

To update to SonarQube I used the following articles and code examples:

Jenkins configuration:

StackOverflow

Retrieved from "http://www.daxiongmao.eu/wiki/index.php?title=Sonar_%2B_maven_configuration_%2B_Jenkins&oldid=2924"