Chapter 5. Integration and release management – Agile ALM: Lightweight tools and Agile strategies

Chapter 5. Integration and release management

 

This chapter covers

  • Integration and release management basics
  • Maven as a comprehensive integration and release management tool
  • Different ways to store artifacts and dependencies in a repository
  • A real-world use case of releasing with Maven

 

Integration and release management allow you to produce software artifacts and release those configuration items that perform a function for end users. I refer to the process of building the software and providing a final product to the end user as technical releasing.

The first chapters of this book discussed the strategy of using a component repository and the value that a VCS like Subversion adds to the software development process. We’ve discussed the general requirements for ALM as well as the functional and technical aspects of releasing. Now we’ll go one step further: We’ll use Maven to release the complete software—to build and store it in a component repository. We’ll discuss the relevant basics of Maven, how Maven implements component repositories, and how you can host these component repositories. Finally, we’ll work through a real-world use case to illustrate how a complex project can be released with Maven.

In chapter 3, we talked about releasing software and bridging the functional releasing to the technical releasing. Applying a task-based development approach, we’ve improved the development process to focus on work items and make the development progress visible and traceable. That’s what we did in chapter 4. In this chapter, we’ll look at more details of technical releasing. We’ll now talk about integrating and releasing the software technically.

In this chapter, you’ll learn how Maven supports the common tasks of release management in an out-of-the-box way. Aside from building, packaging, and deploying, you’ll also want to support automated unit and integration testing with Maven, which we’ll discuss later in this chapter. We’ll discuss some accompanying tools such as Artifactory which helps to host component repositories and can help streamline your integration and release management tool infrastructure. We’ll also review lightweight alternatives to a full-fledged component repository by discussing hosting binaries in Subversion. I refer to the latter approach to be the “poor man’s solution” because although using Subversion as a component repository has advantages, using a dedicated product for hosting a component repository is often preferred.

The rest of the book will further detail the release and test management functions. We’ll use Maven where appropriate and show other detailed use cases and views on integration and release management. Chapter 6 also contains best practices for using Maven and discusses concepts for setting up productive environments. We’ll review how developers can consume, integrate, and test artifacts efficiently. Chapter 7 contains recipes and tools for continuous integration. We’ll discuss tools such as Maven, Jenkins, and Artifactory, and talk about staging and audits. Chapter 8 will cover test management in more detail and how it can be bridged to the development of software, also partly based on Maven. Chapter 8 will focus on the QA/tester view of Agile ALM and integrate testing activities with the creation of software.

Now let’s start our discussion on the integration and release management function.

5.1. The integration and release management function

The integration and release management function is the backbone of your Agile ALM. How your integration and release ecosystem will look depends a lot on your development process and your individual requirements, but in this chapter, I’ll explain commonly used practices and implement them with tools.

Developers write the software, checking their sources and build scripts into a VCS (such as Subversion). Developers implement requirements in their workspaces. They check in and update sources to the VCS, as shown in figure 5.1.

Figure 5.1. Integration and release management system: Sources and build scripts are shared in a VCS; a CI server builds, tests, and deploys versions; and a component repository stores ongoing versions as well as releases of the software (as binaries). Several roles and responsibilities exist; a release manager takes care of creating and staging releases.

Produced and consumed artifacts are stored in the component repository for further usage. You can configure the component repository to act as a cache or proxy for third-party products and as a shared pool where developers continuously deploy versions of their built code.

A component repository stores binaries and acts as a cache for artifacts.[1] Binaries can be separated into snapshot versions and released versions. Snapshots express artifacts that are currently under development. Released binaries are final, baselined versions of artifacts that fulfill defined quality requirements. A release is a special type of version—either the version after reaching a deadline (in a timeboxed environment, for example) or the version that implements all requirements (tasks) and fulfills the requirements of test management. The release might represent a minor milestone (for instance, the completion of a sprint) or a deployable release candidate that includes all of the requirements expected for delivery to an end user. Typically, the sources that correspond to released binaries and that were produced by the team are labeled in the VCS.

1 In this context, a component is a concise collection of software units that has a specific meaning for the team.

The continuous integration (CI) server (such as Hudson/Jenkins) is set up to automatically trigger a build when the code is committed, using Maven to do the build, packaging, unit tests, and deployment. Ideally, the developers should use the same Maven build scripts for their private builds as are used for the official builds on the central CI server. Build scripts manage build dependencies and change as the project grows or when sources use other or new parts of the software. These build scripts (such as the Maven POM; more on that a bit later) should be version controlled like any other artifact essential for the release. Once this is set up, the CI server builds, packages, unit tests, and integrates components, and it may deploy the release to an integration test machine. This approach and these tools provide a streamlined approach to continuously building, releasing, and integrating the code.

The CI server pulls sources and build scripts from the VCS and builds the software on a dedicated build machine, continuously. The main task of the CI server is to produce artifacts. The generated artifacts, as well as the build process itself are both often called build. Often the CI server integrates and deploys the software on the integration test machine (the developers don’t do that themselves!). For producing artifacts, the CI server has to consume artifacts such as artifacts that the produced artifacts depend on to fulfill build-time dependencies. Those consumed artifacts are artifacts that were produced previously, such as artifacts created by a feature team during a release or components developed by a component team. Most of the consumed artifacts are third-party libraries, though, such as testing frameworks or other Java libraries. All these consumed artifacts are fetched from the component repository, which acts as a pool of shared assets.

As we discussed in chapter 3, developers and their workspaces are the first rung on the staging ladder. Developers consume artifacts in well-defined versions, which means that developers integrate necessary binaries into their workspaces to develop the software that was specified and prioritized by the customer.

For testing and releasing software, the built software is staged. This means different logical repositories can be used inside a physical component repository. Typically, for each test environment, a dedicated logical component repository is in place. In many projects, the CI server continuously deploys snapshot versions to an integration test environment. The software is then deployed to other environments, such as the system test environment, without rebuilding the software, but rather by staging and reconfiguring the baselined software.

By using a tag or label in a VCS, the person responsible for the release management function creates releases that are baselined and may create branches. This job may be performed by an independent release management function (for example, a release manager or a developer, depending on how you slice your roles) or a developer tasked with creating the official release. With the help of branches, fixes can be applied on the release (a branch) without affecting the ongoing development (on the head and trunk).

The software is promoted to higher testing environments by a release manager. The release management function is also performed by the expert who deploys the release artifacts to the component repository or promotes existing artifacts. Depending on the process, that person may stage the artifacts (all artifacts belonging to one release) to another (logical) repository inside the component repository. The overall system should be based on clear responsibilities that are implemented by access and authentication security systems and tools; not all developers should be able to deploy (or remove) artifacts to a release component repository or to stage software.

Often, tagging and branching, as well as automatic testing and parts of the staging process, are triggered by the CI server. If the CI server stages software automatically or supports the staging in parts, this is often called staged builds or build pipelines. The development process and its roles, as well as their activities, are highly integrated. Additionally, the produced artifacts are integrated continuously.

Integration on different levels, meaning integration of software and integration of people and status, directly leads to having synchronization points. The development process consists of implicit synchronization points. This means that aspects like a staged build, automatic testing, or the use of component repositories synchronize people with each other and with the status of the project. By using inherent synchronization points, defects in the process and in the software are discovered early and often and are made visible to the team.

QA (testers or the whole team) are supported by scripts. Many tests can be automated, so test results can be made visible to the team continuously, in an objective and efficient way.[2] QA audits and static analyses are done by scripts. Developers should run those scripts in the developers’ workspaces, and they should also be run on the CI server, depending on your personalized staging strategy and your defined quality gates.

2 Tests that can’t be automated are exploratory tests, for instance. More on tests in chapter 8.

Many teams find it difficult to reach consensus when trying to choose a build management tool. Teams that choose Maven usually do so because of Maven’s focus on convention over configuration, allowing Maven to do the heavy lifting. What this means is that Maven allows you to assume reasonable defaults and reasonable default behavior. Maven also provides a fully featured project description incorporating automatic binding for project dependencies. Many developers also like Maven’s seamless integration with IDEs and the ease of learning another team’s build process (which is based upon Maven’s standardized build convention). Some developers find Maven to be complex, which isn’t surprising because it’s both a lifecycle management and an integration tool.

In the next section, we’ll first discuss some of Maven’s core features, particularly in the context of testing. This basic information will prepare you for the following sections, but it isn’t a full Maven guide. Please refer to Maven’s online documentation (http://maven.apache.org) or to dedicated books for more information about Maven in general.[3]

3 See the free Maven books by Sonatype (www.sonatype.com) or Brett Porter and Maria Odea Ching, Apache Maven 2 Effective Implementation (Packt Publishing, 2009).

5.2. Maven feature set

Maven is free and extensible and it’s distributed as a small core module. All features are implemented as plug-ins and are loaded on demand. These plug-ins are also stored in repositories. You (or other developers) can easily write plug-ins in Java or other scripting languages. Maven scripts can run Ant scripts and vice versa.

To get Maven running with its default settings (including its default repositories), you must start the standard Maven distribution. I’ll show you several examples of running Maven from the command line.

 

Tip

It can be helpful to transition to Maven in steps. You don’t need to immediately use m2eclipse to use Maven from within Eclipse. As a first step, you may use only the console interface (although the IDE integration is much more convenient) because IDE integration adds complexity, and command-line commands can help you understand initially what is happening.

 

Maven provides a simple way to set up projects that follow common best practices, including a default directory structure that makes it easier to understand how a project is structured. A consistent and unified directory structure simplifies software development and provides a standard based on industry best practices. This focus on convention over configuration increases cross-project knowledge and makes it easier to get up to speed on another team’s project. Many Maven features, configurations, and settings are provided implicitly. You can change Maven’s default behavior at many points and can configure almost everything, but generally it’s better to stay with the commonly accepted conventions.

When it runs the first time, Maven creates a local repository for your artifacts and updates it during subsequent runs continuously. Maven requires a settings.xml file that’s shipped with default values as part of Maven’s standard distribution; for the moment, suppose the location of settings.xml is $M2_REPO. All developers have a settings.xml in their local filesystem. Its location is either “~/.m2/settings.xml” or “$M2_HOME/conf/settings.xml.” It contains personal data, such as user credentials, that allows you to access repositories or to route to a private SSH key. This personal data shouldn’t be bundled with a specific project in the project object models (POMs).

You can run Maven from the command line or in your favorite IDE. Or you can have the CI server trigger it automatically. Manually configuring classpaths and dependencies in the IDE can be a difficult and error-prone task. With Maven IDE integration in place, the classpath is resolved automatically, so you don’t need to manually configure project settings in your IDE. We’ll discuss that more in the next chapter.

On centralized build and integration servers, you’ll want to run build scripts (such as Ant or Maven scripts) to build your software. Whether you’re building from within the IDE or on a centralized server, you should use the same build scripts. Eclipse or other IDEs aren’t build tools, and normally you won’t run Eclipse on your integration server to build your software. Maven supports this approach of context-free builds, which is what I call congruent build management (chapter 6 explains this in more detail).

Now let’s take a close look at the Maven project object model (POM).

5.2.1. POMs and dependencies

Each project (in the sense of a module or component) needs one Maven model to be described. This Maven model is called the POM (project object model) and it’s written in XML (you can write the meta information that’s expressed in the POM with Groovy as well).

A POM is defined by its GAV coordinates: groupId, artifactId, and version, as shown in the following listing. Besides its coordinates, the POM includes sections for defining dependencies (also referenced by its coordinates) and for configuring its complete build processing (including testing). More on this later.

Listing 5.1. Basic POM

Normally, the groupId is unique to a company or a project and classifies the origins of its artifacts. Dots are often used in the groupId to form hierarchies, but that’s not required (for example, JUnit uses its name as the groupId). It’s also not mandatory to map the groupId to your package structure, although this is a common best practice. When stored in a directory (or in the repository), the groups act as a separator so that you can distinguish them from the Java packaging structures.

The artifactId is the unique name of the module or project. Consider a project that produces a lot of artifacts. They all share the same groupId and differ in their artifactId. The combination of groupId and artifactId creates a unique key for identifying the artifact. This is also a valid way to address the module in a repository. In our case, the project is located in $M2_REPO/com/huettermann/myartifact.

The combination of groupId and artifactId defines the module, but it doesn’t contain information about its versions. The version element defines that. As soon as you update your own sources to work against a new target version of your software or you use a dependency in another version, your POM should reflect these changes. By adding the version information to the groupId and artifactId, you get the full path to your artifact (for a specific version) in your repository. In our case, the full path would be $M2_REPO/com/huettermann/myartifact/1.0.

The packaging adds the artifact type (the architect would say, deployment unit) to the address structure of groupId:artifactId:version. The model-based approach supports standard output types like JAR, WAR, EAR, or OSGi artifacts. One Maven model produces exactly one deployment unit by default; you can add additional ones. Maven knows how to build these types. Unlike Ant, you don’t have to write the logic for each step of the build again; instead, you declaratively describe what your project is about.

The dependencies section specifies the modules our module is dependent upon . Maven provides dependency management for all dependencies including transitive dependencies (dependencies of your dependencies). You can also use scopes to specify resulting classpaths (during compiling or testing) and limit the transitivity of dependencies. Maven looks for these dependencies in the repositories defined in your POM. Dependency management is a core feature of Maven. For further information about dependency management, see the relevant part of the official Maven documentation.[4]

4http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html

The build section configures the build (via Maven’s build lifecycle) and the reporting section configures the reporting (via Maven’s site lifecycle). The POM also contains information about the repositories to which you need to connect (to resolve your dependencies) and to which you want to deploy your own artifacts . Dependencies can also be managed in Ant with Ivy.

 

Dependency management with Ant and Ivy

Another popular build tool is Ant, which has many features. With Ant, you must code your build scripts imperatively, which means that you have to specify each step and assume nothing. This also means you need to write new scripts for each new build project. You must tell Ant how you want to achieve something.

In contrast to Ant, Maven runs a declarative approach, based on conventions and best practices. You tell Maven what you want to achieve, and Maven knows how to achieve it.

If you have based your projects on Ant and are interested in advanced dependency management, you can use the Maven–Ant bridge, working with dependencies from inside Ant scripts or Ivy. Ivy is a dependency manager that took many concepts from Maven. Using Ivy, you don’t have to leave the Ant ecosystem to gain advanced dependency management (see http://ant.apache.org/ivy/).

You may be driven by requirements or as part of a migration path to use Ant, Ivy, and Maven in parallel or in conjunction.

 

5.2.2. Inheritance and aggregation

You can use Maven inheritance and aggregation to organize your projects.

Inheritance means your project can define a parent project from which it derives settings. An example of inheritance is that the project POM derives from Maven’s Super POM implicitly. The Super POM ships with Maven, and it consists of basic settings that dictate, for example, which artifact repositories to use. If you don’t define optional settings in your POM, the default values are taken from the Super POM.

Aggregation is used when you join different projects into a container POM. This is often used in complex multimodule settings where an application consists of, for example, a Java web archive (WAR), a Java enterprise archive (EAR), or Eclipse OSGi bundles aggregated to a feature or a target platform. Another example is where you want to build and test a project with the parent module referencing and managing all of the other modules specified in the POMs.

The following listing shows a POM that specifies an Eclipse bundle.

Listing 5.2. POM referencing a parent POM

The POM is pretty lean. Besides its coordinates, it only defines itself as an Eclipse plug-in and references its parent POM , inheriting the parent’s configurations (note that for managing OSGi bundles, you need Maven 3). The scripts are referenced using a relative path addressing the parent . The POM inherits the configurations of its parent. The parent is shown in the following listing.

Listing 5.3. Parent POM with central settings and multiple modules

The parent POM defines the packaging type pom , and it has two tasks. The first is the parent POM acts as a container by referencing a couple of modules . The second task is to configure the central settings used by the child POMs, for example, to configure the distributionManagement section describing the target to which the artifacts are deployed . In this example, the parent POM includes the OSGi bundle discussed earlier and adds a test bundle to the aggregation. Therefore, if you run the parent POM, all its child POMs will be built successively.

5.2.3. Lifecycles, phases, and goals

Maven requires that you describe what your project is doing. To this end, Maven offers lifecycles and individual phases that you can configure to tell Maven what to do in these phases.

Maven has three lifecycles: clean (removing previously built artifacts, like a “tidy up” lifecycle), build (which contains the main logic), and site (for reporting). The build lifecycle contains the following main phases:

  • Validate— Validate that the project is correct and all necessary information is available
  • Compile— Compile the source code of the project
  • Test— Test the compiled source code using a unit testing framework
  • Package— Take the compiled code and package it in its distributable format, such as a JAR
  • Integration test— Do integration tests or process and deploy the package into an environment where integration tests can be run
  • Verify— Run any checks to verify that the package is valid and that it meets quality criteria
  • Install— Install the package into the local repository, for use as a dependency in other projects locally
  • Deploy— Copy the final package to the remote repository for sharing with other developers and projects (done in an integration or release environment)

Based on these standard phases, Maven would perform the following tasks in a real-world use case:

  • Download dependencies
  • Compile the code
  • Run the unit tests
  • Package the source code into a WAR archive
  • Download the Java container for the integration tests
  • Deploy the artifacts in the installed Java container
  • Start the Selenium server to run user interface tests
  • Start a browser
  • Run the integration tests
  • Clean up the integration test environment
  • Install the artifact in the local repository

Some parts of this process are already covered by entries in the POM (for example, defining WAR as the packaging type). Others are available out of the box (such as testing), and some can be described declaratively with dedicated plug-ins that you can attach to run in Maven’s phases.

 

Note

Calling one phase (such as running mvn install on the console) does execute the phase, as well as all phases in the Maven lifecycle placed before the called phase.

 

Plug-ins allow you to configure their behavior. The Surefire plug-in (responsible for testing) provides the property maven.test.skip, which you can use to configure the plug-in to do nothing. You do this by passing the Java parameter to your Maven call:

mvn install -Dmaven.test.skip

All phases consist of goals. Goals offer a more technical view, whereas phases provide a high-level functional entry point. Internally, Maven organizes entry points as packages.

Each package maps one or more goals to one or more phases by default—some have more than one, and some have none. Phases often have default goals that are executed automatically based on the project’s packaging type. You can also directly trigger a goal by adding it to its phase, separated by a colon. For the install goal, it would be mvn install:install.

5.2.4. Maven and testing

Maven is ideally suited for running all your tests as part of your normal build setup. You don’t need a customized environment for different types of tests. Unit tests run by default, and integration tests are specified as a phase in the Maven build (between the packaging and the install phases). In this way, you can rely upon the previously built package, which can output a WAR file. To run your integration tests, you need to add and configure the dedicated plug-ins.

The Surefire plug-in is responsible for tests. It can be configured inside the POM’s build element. By default, the plug-in looks for test cases in the src/test/java folder. It runs in the Maven test phase and by default looks for classes with the patterns **/Test*.java, **/*Test.java, and **/*TestCase.java. It’s possible to run both unit tests and integration tests with the Surefire plug-in. In most cases, this will meet your requirements right out of the box.

For more complex integration tests, consider using Maven’s Failsafe plug-in. It’s a fork of the Surefire plug-in and ensures that the post-integration phase runs even if there’s an error in the integration tests. In addition to not breaking the build on test failure, the Failsafe plug-in also enables more convenient configuration. For example, it allows you to skip the integration tests but run the unit tests. To create integration tests (instead of unit tests), you can save the tests with names following any of these patterns, which include the letters “IT”: **/IT*.java, **/*IT.java, and **/*ITCase.java.

Both plug-ins can run JUnit and TestNG tests. All you need to do is include the dependency of the framework in your project POM.

Maven’s plug-ins block further execution until they’ve completed their tasks. This can be helpful if you start a server for integration tests and want to wait until everything is started, deployed, and ready for testing.

The following listing shows an extract of a typical POM.

Listing 5.4. Maven POM

There are a number of things to note in this POM file. First, each model is described through project coordinates containing groupId, artifactId, and version . The project is built in a WAR file . The model also describes the project dependencies by referencing their project coordinates . JUnit version 3.8.1 is a dependency in this case . Another dependency is TestNG 5.10 . You can define the scope of the dependencies and further parameters. Finally, you can configure Maven’s build lifecycle , which also includes testing, compiling, and packaging, by adding and configuring plug-ins.

5.2.5. Maven ecosystem

Tycho (http://tycho.sonatype.org/) extends Maven by providing a toolchain and embedded runtime (Equinox) for building OSGi bundles, Eclipse plug-ins, and rich client platform (RCP) apps. OSGi bundles have features similar to Maven, such as providing dependency management. Most OSGi apps were built with Eclipse’s plug-in development environment (PDE), which is based on Ant scripts that often lead to complex build scripts. With Maven Tycho, OSGi is a first-class citizen in the Java enterprise build and release management ecosystem. Tycho makes it possible to transparently build OSGi apps with Maven—Maven takes care of the building behind the scenes. OSGi and POMs are synchronized and compatible.

Polyglot Maven (http://polyglot.sonatype.org/) adds yet more flexibility to the Maven core by allowing domain-specific languages (DSLs) access to the core functionality.

Maven Shell (http://shell.sonatype.org/) is a command-line interface for Maven enabling faster builds. With Maven Shell, project information and Maven plug-ins are loaded into a single, always-ready Java Virtual Machine (JVM) instance that can execute a Maven build.

Maven and its comprehensive lifecycle management facility aggregates project information as part of the build, and this data can be published on a Maven-generated website. Because of its first-class dependency management support and its integration with various VCSs, it’s able to support automatic release management. All of these features make Maven a compelling choice for many technology professionals.

Another important feature is Maven component repositories.

5.3. Maven component repositories

Maven repositories are essential for organizing build artifacts of varying types and their dependencies on each other. Two types of repositories are remote and local.

A remote repository is any centralized repository that’s accessed by a variety of protocols such as file://, scpexe://, and http://. These repositories might be truly remote, set up by a third party. They provide their artifacts for downloading. For example, the official Maven repository is available under http://repo1.maven.org. Artifacts are provided in Maven’s central repository, and your Maven installation downloads them on an as-needed base. This type of repository is named a public repository, and it’s illustrated in figure 5.2.

Figure 5.2. Maven repository topology: developer repositories, a central proxy repository, and a remote public repository. The public repository is an external system, normally on the internet. All other repositories are internal systems.

Other remote repositories may be internal repositories located inside your network, or what some IT professionals call a DMZ (demilitarized zone). They’re set up on a file or HTTP server within your project or company, and they’re used to share private artifacts between development teams and for releases. This sort of repository is often called a proxy repository. A proxy repository acts as an intermediary for requests from desktop machines seeking or providing binaries.

A key benefit of proxy repositories is that you don’t need to download artifacts multiple times from remote sources; rather, you download quality-assured artifacts once and share them across projects and teams via the repository proxy. This way, the developer machines’ access to the internet can be restricted, and developers can pull unified artifacts from the proxy repository.

Proxy repositories are essential for managing your dependencies. Dependencies are external libraries that are required to compile and run your code and related artifacts. They can be external libraries like Checkstyle and Log4J or internal dependencies like your shared libraries that are deployed to the proxy repository for further reuse.

Local repositories are placed on the developers’ machines. They’re individual caches of artifacts that have been downloaded from the proxy repository. In addition, they contain build artifacts that haven’t yet been released from workspaces and that aren’t yet visible to others.

An artifact is stored in a Java-like package structure derived from its groupId and artifactId data, which also provides version information, as shown in figure 5.3.

Figure 5.3. Extract of a local developer’s repository

Under the repository’s root folder, inside the folders named for the groupId and artifactId, you’ll see different version folders containing several JAR files, a POM file, and additional meta information. In figure 5.3, the repository root is located under repository and the folder structure contains checkstyle as the groupId and the artifactId, followed by the individual folders for versions, in this case 4.1 and 4.4.

I’ve selected the folder named 4.4, so in the file explorer, the lower part of the window displays its contents. JAR is the deployment unit created by this module (and its build), as described in the POM. You can add additional JARs to accomplish tasks such as distributing the sources. The files with the sha1 extensions are files generated with a cryptographic hash function in order to validate their integrity.

Remote and local repositories are structured equally so that scripts can be easily run on either side. They can also be synchronized for offline use, but the layout of the repositories is completely transparent to the Maven user.

An important lifecycle feature is the option to not only consume dependencies, but also to publish the project’s artifacts. The published artifacts can then be consumed as dependencies by other developers or projects. Maven provides a standard lifecycle phase called deploy, which needs to be configured to specify a repository that publishes artifacts, making them available to the other developers. Once deployed, these artifacts can be added to a project’s POM as dependencies.

There are various ways to set up a repository infrastructure:

  • Use a shared filesystem and deploy artifacts to it via network mounts, FTP, or SSH-based secure copy. For serving the repository, a web server like Apache HTTP could run on the same machine, exposing the repository for dependency resolution. Alternatively, the web server, if equipped with WebDAV features, could be used to serve the publishing process as well.
  • Use a repository manager product such as Apache Archiva, Sonatype Nexus, or JFrog Artifactory. These products must be run in a J2EE/JEE web application container (quick-start versions are also available where the distribution is already bundled with a container), accessible by artifact publishers and consumers. They offer many additional benefits such as security models, search features, permission models, workflows, and so on. Later in this chapter, we’ll look at Artifactory to get a basic understanding of the features provided by a repository manager.

Open source projects have the option to deploy to the central Maven repository, and many do so. The central Maven repository is http://repo1.maven.org/maven2 (often mirrored). Therefore, this repository is the source where you can browse and include your dependencies. The process for deploying artifacts to the central repository is described in detail on the Maven website.[5]

5 See Maven’s “Guide to uploading artifacts to the central repository,” http://maven.apache.org/guides/mini/guide-central-repository-upload.html.

Although the central repository is the preferred mechanism for publishing public artifacts, the process requires some administrative effort. Most notably, the recommended way of publishing requires a staging repository that’s accessible by secure copy (scp), which means that the team will still have to establish a self-hosted proxy repository in your internal system.

To accelerate development flow and rapidly work on code in an Agile development effort, relying solely on repositories isn’t enough. Imagine you have different development teams developing components for your overall application, and the components have interdependencies. In continuous release mode, where you’re continuously providing the most recent development state, the output is unreliable. A lot of bugs and unstable developments are released too soon and are included in the development work your colleagues do. But what you want is a productive work environment, with private workspaces that consist of reliable, quality-assured versions of artifacts. You need a way to declare frozen versions of dependencies to other components, so Maven enables you to specify dependencies between components. With Maven, one team can continue working on new features, and another can develop against a stable API and version. In your individual project model, you can decide to provide either a versioned artifact of your work or a snapshot.

 

Snapshots and Sites

If you have a CI server, such as Hudson or Bamboo, you should configure a snapshotRepository, which the CI server uses to store the nightly build results. If you’re also building and deploying the project’s website with Maven, you need to define the site element as well.

 

A snapshot is a special version that indicates a current development state. Unlike regular versions, Maven will check for a new snapshot version in a remote repository for every build, once a day, or however often you choose to define as the frequency. For normal versions (releases), Maven downloads—once only—the artifacts from a remote repository to your local repository. This artifact bootstrapping (where you download the artifact only once to serve all further requests to it out of the local repository) applies to proxy repositories, too, in terms of their dependencies upon the artifacts in remote repositories. Versioned artifacts and snapshot artifacts are stored in both local and remote repositories. For that, different access paths or folder structures are used.

 

Repositories and Mirrors

Repositories can be defined in POMs. You can set up a parent POM that contains the enterprise settings on which all your individual projects will be based. Alternatively, or in combination, you can define repositories in your private Maven settings file. Using a mirror, you can replace a public repository (such as “central,” where Maven connects to when it automatically tries to resolve missing dependencies) with your own proxy repository.

 

Developers can choose to use either snapshots or versioned artifacts for development, depending on whether they’re actively developing or preparing a release of the code. Having a choice between versioned artifacts and snapshots dramatically improves the quality of the software process, purges the workspaces (and avoids workspace clutter), and keeps the component repository clear of duplicate libraries. Developers don’t need to check out sources and their references in a manual, error-prone way, nor do they need to build third-party code again from scratch. They only need to include the binaries in their workspaces. This is what they should do to develop their own components. (Sources for debugging can be attached, though.)

5.3.1. Managing sources and binaries in conjunction

Source code management, including branching strategies, and management of binaries are used in conjunction with each other. For example, an application and its sources are developed on the VCS head or trunk (see figure 5.4). A continuous build consumes and produces binaries, and it deploys the current snapshots to the component repository. During the process of building a release (for instance, during a code freeze), a VCS tag is created (for instance, 1.0.0), and its corresponding release binaries are deployed to the component repository.

Figure 5.4. Branching strategies and the deployment of binaries (snapshot and release) are used in conjunction with each other. The frozen versions that were tagged from the head are 1.0.0 and 1.1.0. Version 1.0.0 had two bugs, so a branch was created, where the two bugs were fixed; the bug fixes were merged into the head as well. At specific moments (the diamonds in the figure), the versions are frozen. The head is frozen to version 1.1.0 and the branch is frozen to version 1.0.1. Both new versions contain the two new bug fixes.

If you release software continuously, in a timeboxed way, you may not need any branches at all, because you can always work on the most recent head version of the software. But if you want to change a formerly released build without including further development that was already done on the head (such as new features), you’ll need to branch.

A branch is an isolated code stream, parallel to the head stream and other branches that are in place already. Changes on the branch are backported to the head: they’re committed to the head ideally directly after the commit on the branch. Many projects find it helpful to create branches as part of the releasing process in the code freeze interval, or even earlier in the frozen zone interval.

It is a good strategy to create VCS branches on demand only. This means that you don’t branch prophylactically but only when needed, at the last possible moment—when a new version of the software is required.[6] A new version is then based on a version that was previously frozen, tagged, and shipped. It’s important that you focus only on changes on the branch that are critical and of high value for the customer. Don’t fix bugs of lower priority and don’t develop new features on a branch, because that counteracts the functional releasing process that was discussed in chapter 3.

6 Using Subversion, you can create branches at any time based on a given tag or a given Subversion revision number. Physically, branches and tags in Subversion are copies of the folder structures in the repository.

The ranges of version numbers are different on branches and heads. Release binaries have unique identifiers and should be deployed to the component repository only once. You should set up the same CI process on the branch that you’ve already set up for the head. This is important in order to find defects there early and often and to start preparing and training the final release of the branched version early. Like the setup on the head, commits on the branch trigger new builds, trigger tests, and create new binaries that are deployed to the component repository. The CI process on the branch also includes the creation of snapshot versions as well as release binaries.

A branch should be closed. Closing a branch retires it: No further work is done on the branch (no new commits to the branch), and all necessary changes are merged to the head. Closing a branch may be implemented by convention (when the release is finished) and can be supported technically, such as by preventing developers from committing to the branch by using Subversion hooks.

You’ll normally want to close a branch soon, because once a branch is created, it’s another development line for the team. CI requires you to integrate software and synchronize the team, and having too many branches that must be merged back to the mainline is additional overhead and often slows down the workflow on the head, counteracting the basic idea of CI.

CI is also a communication vehicle that relies on the current, synchronized, most recent version of the software.[7] Although branches also act as communication points, the development flow and its pace is determined on the head, and branches fragment communication. For the same reason, mechanisms such as developer branches, where developers have their own isolated code lines in the repository, should be avoided.

7 This is the main reason why people should commit early and often. Investing much time in developing software in the workspace without committing changes to the repository and having the changes integrated with the work of others is a lost opportunity.

5.3.2. Artifactory’s enterprise component repository

Artifactory is a component repository manager. It allows you to have full control over your project’s binary artifacts, acting as both a secure warehouse for locally hosted artifacts and a smart caching proxy for remotely hosted artifacts. Artifactory provides a few unique management features and some cross-technology integrations that make it compelling to use in environments ranging from a small startup project to large CI setups.

These are the core features of Artifactory:

  • Stores artifacts (your own artifacts should be created by a CI server)
  • Acts as a proxy for artifacts
  • Avoids hitting public remote repositories
  • Deploys, manages, and shares local artifacts
  • Offers full control over artifacts’ resolution and delivery
  • Provides a fine-grained security system, including repository administration, and distinguishes between read, write, and delete permissions for working with artifacts

 

Artifactory’s origins

In 2006, Yoav Landman started Artifactory’s development with the purpose of providing a better alternative to the existing repository managers at that time (such as Maven-proxy and Proximity) that lacked basic features such as security, filtering, indexed searches, and a user-friendly UI. This was a time when large enterprises were using Maven and managed artifacts. They had no real solution to enterprise needs for artifact management.

The Artifactory project was hosted on SourceForge, where it’s still hosted at the time of this writing. In 2008, JFrog (www.jfrog.org) was founded to support the product and its future development.

 

Artifactory provides support for managing remote repositories. By configuring a public repository (the major ones are already preconfigured), Artifactory creates its own cache out of the box; you don’t need to add repositories and link them to public repositories. For example, using the Maven repo1 central public repository, Artifactory sets up a repo1-cache directly, as shown in figure 5.5.

Figure 5.5. Artifacts in the Artifactory repository browser

Artifactory also supports virtual repositories. Virtual repositories wrap local and remote repositories under a single identifier, acting like a facade, so you can create combinations of repositories and expose them to clients as a single virtual repository. This way, you can decouple the logical repository view for developers from the technical implementation view that a repository admin must have.

Virtual repositories can also wrap around other virtual repositories. Whenever a repository (local, remote, or virtual) is added to, changed, or removed from such a virtual repository, the change is automatically propagated to all virtual repositories.

Artifactory ships with a conveniently embedded virtual repository called repo.

This means that all you need to get your local workspace connected to Artifactory is a settings.xml file, such as the one shown in the following listing.

Listing 5.5. A settings.xml file with server credentials and repositories

To enable non-anonymous access, we set up the server elements to contain repository credentials . These include an ID mapping to the id element of a repository fragment. Next, we mirror all unresolved repository requests ; it can also be useful to hide maven2 central. The ID must be added to the server section with valid credentials . We use a default profile . Next, we shadow the maven2 central repository , overwriting the same entry in the Super POM, and then point to the virtual repo repository , which automatically maps to repo1-cache. This listing skips over the plug-in repository , which contains similar entries for managing plug-ins. After providing an ID for the profile , we set the profile to active . The profile will be applied without any further preconditions.

You also need a distributionManagement section in your POM, as shown in the following listing. This can be placed in a parent POM.

Listing 5.6. POM distribution management

This example shows the advanced features for mirroring repositories. Here we use mirroring with a wildcard, a star (*) in the mirrorOf element of the XML file, so all unresolved repositories will access Artifactory’s virtual repository, named repo). The example also shows advanced permission settings. In the previous listing, Artifactory is configured to prevent anonymous access, so we have to use valid usernames, as you can see in the different username elements of the XML file. The default is to allow anonymous access, so you have to disable that via Artifactory’s web interface.

On top of Artifactory’s open source version, there’s a Power Pack set of commercial add-ons targeted at enterprise users. With a little more effort, many of the commercial enterprise features can be utilized in the open source version by using Artifactory’s REST API. JFrog also offers a fully hosted Artifactory service—Artifactory Online (www.artifactoryonline.com)—targeted at small to medium-sized organizations. This cloud-based repository manager provides its subscribers with the latest version of Artifactory and the Power Pack.

Artifactory is a generic binaries repository. Although it initially started as a Maven repository manager, Artifactory isn’t intended only for Maven users; Grails or Ivy users can use it as well. Artifacts that are deployed to Artifactory with Ivy can be retrieved with Maven, and vice versa. Therefore, the tool offers features that are targeted at other build tools:

  • Generic uploading from the UI to any path
  • Indexing of any artifact type
  • Searching inside the content of Ivy modules (parallel to searching inside Maven POMs)
  • Integration with the Gradle (www.gradle.org) and Ivy (http://ant.apache.org/ivy/) build tools
  • Encrypted password support

For Maven, Artifactory offers many features:

  • Uploading of artifacts with optional POM editing and auto-guess properties
  • Special searches
  • Auto-generated settings.xml files (derived from repositories configured in Artifactory)
  • Centrally controlled snapshot policy and convenient POM views
  • Cleanup of POMs from troublesome remote repository references
  • Finding and deleting of selected artifact versions under a folder
  • Copying and moving of artifacts (single and bulk)

All artifacts in Artifactory are merely pointers to binaries, and binaries can be stored either on the filesystem or in a configurable relational database. Artifactory comes with a built-in Derby database that can be used to store data for production-level repositories (up to hundreds of gigabytes). Artifactory’s storage layer supports pluggable storage implementations; this is made possible by the underlying Jackrabbit Java Content Repository (JCR), so you can configure Artifactory to run with almost any JDBC database or even store data completely on the filesystem. The storage used by Artifactory is checksum-based, which means it doesn’t matter how many times a file exists in Artifac-tory; it will still be stored only once. This makes operations such as moving or copying cheap in terms of speed and space. Copying artifacts is unavoidable if, for example, the same binary needs to be exposed to different groups of people who are allowed access to different repositories, and you can’t (or it’s way too complicated to) do this by sharing one repository and applying fine-grained security rules.

Artifact management and metadata is atomic in Artifactory—moving multiple artifacts, deploying artifacts with their metadata, deleting groups of artifacts, and so on, can all be done in an all-or-nothing fashion.

In Artifactory, all files and folders are candidates for receiving an unlimited amount of metadata that can contain any appropriate information that further describes the artifact. Any XML metadata can be attached using the UI or the REST API and can be managed, created, or deleted. This metadata is also indexed, so it’s fully searchable from the UI or REST. The commercial version of Artifactory takes this one step further by offering user-defined, strongly typed custom properties on top of XML metadata that provide a specialized UI according to the property type. For example, if the user has defined a single-value, closed list property for the performance level of an artifact, they will get a single-select dropdown when annotating an artifact with this property.

The user interface is intuitive and easy to understand. Items are displayed clearly on the screen, and all items are either self-descriptive or have decent online help. The UI is mostly Ajax-based and is implemented using Apache Wicket with many custom components and the Dojo JavaScript library. The UI can be branded (from the UI) with a custom logo and footer.

Artifactory puts searches in the center of the user activity and offers the following types:

  • Quick search
  • Class or archive content (for example, properties file in a JAR) search, with the ability to see the class that was found
  • XPath search inside Maven POMs, Ivy modules, or any deployed XML content
  • groupId, artifactId, version, and classifier search (for Maven artifacts)
  • Properties search (in the commercial version; more about properties later in this section)

All searches are exposed through Artifactory’s REST API.

Artifactory takes searches one step further by making them the prime vehicle for artifact management. All search results are navigable to the repository tree, where you can annotate, move, remove, view, or download the artifact. Figure 5.6 shows Artifactory’s search facility.

Figure 5.6. Artifactory’s search facility: POM/XML search

The commercial version of Artifactory is even more powerful, offering smart searches. You can save your search results, search again, and add or subtract the results from the saved ones (as many times as you wish), tweak the saved results manually, and operate with them as one unit of work. For example, you can search for all artifacts of a certain group and version in a dev repository, save the results, search again for all the sources with the same group and version, subtract the sources from the original result, and finally promote (move) the results to another repository accessible to the QA team.

Artifactory provides a simple, yet powerful, security model that allows the assignment of the following four permissions:

  • Read
  • Write
  • Delete (also implies redeploy or overwrite)
  • Annotate (with metadata)

Permissions are applied to a target consisting of a group of selected repositories and a set of include or exclude path patterns on these repositories. Permissions are then assigned to either individual users or user groups. This simple model works well in practice and is easy to understand and to control, incurring minimal security management overhead. Permissions of artifacts are viewable by an in-place effective permission page. Artifactory also offers out-of-the-box support for LDAP and Active Directory authentication. Support for highly optimized LDAP groups authorization is part of the commercial add-ons. (Standard LDAP support is freely available.)

Have you experienced situations where colleagues accidentally override artifacts and POMs with older versions or outdated sources by locally running only an mvn deploy? Artifactory’s Watches notifies you by mail whenever a create or delete operation is run on a certain Artifactory repository, folder, or artifact.

Maven manages a certain amount of metadata by adding it to artifacts. The coordinates (groupId, artifactId, version) provide important value, but Maven has no feature to add more context metadata to artifacts that would be helpful right when they’re deployed. Properties, or artifact tagging, is a neat feature that can enrich your build infrastructure. Generally, properties are arbitrary meta-information. For example, a property is a Maven or POM property that will become an Artifactory property—a Maven build property exposed to Artifactory. You can use these properties to locate and identify the deployed artifacts that originated from the same build so you can promote them later on.

Properties can be attached to artifacts in many ways: via the UI or REST API using PUT requests, or piggybacked on artifacts that are PUT to Artifactory using matrix parameters. To add parameters, you need to configure your POM’s distributionManagement section; using a good design approach, this configuration should be in one location—in one parent POM. It could look like this:

<distributionManagement> 
  <repository> 
    <id>qa-releases</id> 
    <url>http://srv/artifactory/repo;build=${number}</url> 
  </repository> 
</distributionManagement>

This pair of arguments is added to the deployment repo definition. The values are usually taken from regular Maven properties and can be updated by any POM. Technically, they’re a set of key-value pairs separated by a semicolon—a standard HTTP communication protocol. Using Maven’s native properties approach, this matrix property can be configured and set in a top-level POM or injected by a command-line interface.

Artifactory allows you to search for properties and copy or move the result set to allow sophisticated staging and promoting. Figure 5.7 shows an example of how you can search for artifacts tagged with the properties build.name and build.number and their values MM and 18. As we’ll discuss in chapter 7, the Jenkins/Artifactory integration automatically injects these properties with every build. This way you gain from traceable builds.

Figure 5.7. Property search: searching repositories for properties. Examples of properties are build.name and build.number.

Once you’ve tagged artifacts in your repositories with any set of properties, setting them manually or automatically, you can search for these annotated artifacts. Once they’re found, you can save and refine the search results (add or subtract results from the former result) for later reuse. Artifactory acts as a shopping cart of artifacts, making bulk artifact management a lot easier. Once you’re done, you can move or copy the results.

One use case for searching and performing bulk operations is to navigate to all saved searches, mark the search results you want to work with, and then copy or move all artifacts belonging to the search result to another repository. Figure 5.8 shows the artifacts found by a save search being copied.

Figure 5.8. Working on search results, and copying an artifact set to a different repository. This allows smart artifact staging and promoting.

5.3.3. Using Subversion to serve a simple Maven repository

This section contributed by René Gielen

When you’re starting without Maven, additional infrastructure and software is needed to set up a Maven topology, such as a proxy repository on a dedicated machine. This might not be possible in the organization’s environment for either technical or political reasons. Open source project teams, on the other hand, might fear some of the administrative overhead involved in publishing to the central repository, and resources such as self-hosted repository managers might be beyond the budget. But any such project team should feel highly encouraged to consider central repository publishing, at least for the early stages of a project.

Often, project teams don’t regard central repository publishing as a suitable option, and they lack the ability to establish or access additional infrastructure. The question is, could there be a solution for establishing a repository without the need for extra resources?

Any method of publishing is workable as long as it results in a normalized repository structure tree accessible by the HTTP or HTTPS protocol. Maven’s deploy phase publishing mechanism utilizes a transport abstraction layer called Wagon that provides a service provider interface (SPI) to be implemented by various transport plug-ins, such as Wagon File, Wagon FTP, Wagon WebDAV, or Wagon SSH.

One of the (possibly) less known providers is Wagon SVN, which makes it possible to publish artifacts to the Subversion VCS. Subversion allows you to browse the latest revision of the repository tree via its web server integration using simple HTTP GET requests. Project teams using Subversion over HTTP or HTTPS for version control can utilize Subversion to gain a fully functional Maven repository without needing any additional resources other than those they’re already using.

The first step is to choose and create a suitable folder structure in the Subversion repository. My recommendation is to create a Maven folder along with the usual trunk, branches, and tag folders. Another common practice is to differentiate between release and snapshot artifacts, where you create two additional subfolders, one named snapshots and the other one releases or staging. The latter name implies that this folder might later serve as a staging repository for publishing to a more general repository, maybe even central. The repository contents will be published in these two folders, depending on whether the artifact you’re deploying is a release or a snapshot version, as shown in figure 5.9.

Figure 5.9. Sample Subversion folder layout

Next, the project’s pom.xml has to be extended to configure the target repository locations to which you’re going to deploy and that the Wagon provider will use. This is shown in the following listing.

Listing 5.7. POM configuring Wagon

In the distributionManagement section, we configure the release repository location within the repository element and the snapshot repository location within the snapshotRepository element . Both elements require a unique ID, which we’ll use later to configure the matching private credentials. The name element should contain a handy, human-readable description of the location. The url element defines the transport endpoint for the Wagon mechanism. The configured values for both endpoints , representing the HTTP address of the repository folders created earlier, are prefixed by the svn: pseudo protocol scheme. The standard Wagon mechanism won’t know how to deal with that, so we add the wagon-svn plug-in as a build extension dependency, which, after being resolved, will register itself for the handling of the svn: prefix. By setting the uniqueVersion configuration option to false, we ensure that snapshot version numbers for the artifact won’t be extended by a timestamp on each deployment.

The next step is to configure the credentials needed to authenticate against Subversion for publishing. These credentials are subject to nondisclosure and should reflect the authorization of the person issuing the Maven deployment, so they won’t be configured in the project-wide pom.xml file, but rather in the individual team member’s local Maven configuration file, usually found at $HOME/.m2/settings.xml. An example is shown in the following listing.

Listing 5.8. Configuring credentials in settings.xml

Please note that the server ID values have to match those configured in the distributionManagement section of the project’s pom.xml file. From now on, each authorized team member can publish artifacts to the Subversion-based Maven repository as with any other repository management solution:

> mvn deploy

Whether the artifact will be published to the snapshot or the staging repository depends on whether the project version found in the pom.xml file at the time of deployment is a snapshot or a release version. An example folder structure also containing snapshot versions is shown in figure 5.10.

Figure 5.10. This expanded repository tree contains various deployed artifacts.

To consume the deployed artifacts as dependencies in another project, this project’s pom.xml file must be modified to recognize the additional custom Maven repository to be used for dependency resolution (see the following listing).

Listing 5.9. Dependency resolution pointing to the repository

How you set up the project repository to resolve both the snapshot and the release artifact versions is shown in the following listing.

Listing 5.10. Repository settings with both staging and snapshot

This section described using Subversion as a Maven repository. The next section shows how to create releases with Maven.

5.4. Releasing with Maven

This section contributed by Matthias Weßendorf

The release of software packages is an important phase of the Agile ALM. You should put the source code into a VCS (for example, Subversion) and extract it to be compiled and packaged. In most cases, your customer will require a complete release package (also referred to as a distribution) that also contains additional information. In this section, you’ll learn how to create a software release and distribute it using Subversion, Maven, and some useful Maven plug-ins. You’ll also learn how to leverage additional tools, such as Python scripts, to automate almost all steps of the release process.

Before you start creating your release, you must be clear about what is expected to ship with the software. Typically a release needs a lot more than some compiled Java classes. Common artifacts in a distribution include the following:

  • Executables— The software itself, such as binaries, property files, database scripts, installers, and so on.
  • Documentation— Practical user guides, such as installation guides or how-to documents, and API descriptions, created with tools like Javadoc or TagDocs, need to be made available as part of a release.
  • Examples— Showcase applications, a reference implementation, API usage examples should be included. Most open source releases contain examples, which may be simple or rather complex, showing many features.
  • Distribution of the source code— This may not only be important for open source projects. Releases for in-house projects or business partners may also distribute the source code, which makes debugging easier.
  • Anything else of value in your individual situation— For project-specific requirements, you may need to ship additional artifacts.

The location of the release deliverables is also important. For instance, with Maven, JAR files (including sources and JUnit tests) can be deployed to an in-house repository, which is an HTTP or file server (for example, Artifactory or SVN) that follows a defined directory layout. The complete release package is often placed somewhere else. It’s common to place these files on your company website or an open source project website. Another option is to deliver the distribution on a network filesystem (such as NFS), which you use to exchange files with your business partners or other in-house teams.

Once it’s clear where the distribution will be placed and what it contains, you’re faced with what is probably the most important question: How do you deploy the release? Several tools and options are available to get a release out the door. These can be historic make files or Bash or Perl scripts that run the software build and assemble the final release package. On older Java projects, you may still see the use of home-grown Ant tasks to generate the release and its distribution package. Today, the de facto standard for Java-based projects is Maven. This not only manages the build system of a software project, but also helps to manage the project through its entire lifespan. Maven support includes aspects such as the following:

  • Creating documentation (like the product website or Javadoc documents)
  • Managing source code (the location of the SVN repository)
  • Branching and tagging in SVN
  • Creating the distribution of the release

 

Best Practices in Maven Plug-Ins

Commonly established best practices are coded into many of the available plug-ins. This means the Agile development process starts with the choice of a powerful tool belt.

 

Maven is a framework that executes plug-ins that accomplish a variety of important tasks. The plug-ins can help you perform tasks that are essential for building, packaging, and deploying your software. An example would be to use Maven to post the results of your nightly builds to Twitter (see http://code.google.com/p/maven-twitter-plugin/). Normally, you won’t want to publish results to the public via Twitter, but Maven supports you in making the status visible and ensuring the build gets fixed back to green as soon as possible. In this section, we’ll look at several major plug-ins (and their required configurations) that are ideal for producing a software release.

Before you start to generate a release, you need to understand that there are several ways to do it. You could start in the SVN trunk folder, which is probably not a good idea, because the main codeline is usually developed on the trunk. It’s a common practice to create a separate release branch using a standard naming convention such as “release-version-x.y.z.”

In this example, we use a common SVN layout:

-myProject/
 -branches/
  -branchA
  -branchB
 -tags/
 -trunk/ 
  -src 
  -pom.xml
...

The first task is to include the Maven Release plug-in in your POM as part of the project’s Maven build lifecycle:

<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-release-plugin</artifactId>
<version>2.0</version>

5.4.1. Creating the branch and preparing the release

Creating the entire release from a release branch is a good pattern. Doing so ensures that the main development can continue on the trunk without having any undesired side effects on your release. Imagine if some of the changes weren’t ready in time for a scheduled release. In this case, you’d want to revert the changes that aren’t complete, and this is much easier if you’re using a release branch that contains only the tested and completed features approved for the release.

You can create a branch with the tools provided by SVN, like svn copy. Windows has convenient graphical tools (such as TortoiseSVN, see chapter 2) that support creating branches and tags. Even with Maven there are several ways to create a branch, such as with the Maven SCM plug-in or the Maven Release plug-in. You can use the Maven Release plug-in to help automate the release process, including creating a release branch automatically.

Before you can use the Release plug-in for branch creation, you must configure your source control management (which is named SCM in Maven) settings inside the project’s pom.xml file. See the following listing.

Listing 5.11. Configuring SCM in project’s model (POM)

 

Maven Site Generation

The SCM configuration in the preceding listing is used when the Maven Site plug-in creates the project’s website. The information about the used source code repository is made available on the generated source-repository.html webpage (see MyFaces, at http://myfaces.apache.org/source-repository.html, or CXF, at http://cxf.apache.org/source-repository.html).

 

The connection element represents the URL that’s used by everybody to check the source code . Usually, the anonymous checkout is done through an unsecured HTTP connection. The developerConnection section contains the URL that’s used by the project’s developers to commit their changes back to the source repository . In almost all cases, this is done through a secured HTTP (HTTPS) connection. The only exception would be when the repository is already hidden by a corporate firewall. The last parameter (url) is a “nice to have” setting. Usually, it contains a web interface to the source code stored in the repository . Most Subversion servers have something like ViewVC (www.viewvc.org/) or WebSVN (http://websvn.tigris.org/) installed.

 

Using GIT as your repository

If you plan to use GIT as your repository, you can use Gitweb (http://git.wiki.kernel.org/index.php/Gitweb) or even host your entire project directly on GitHub (http://github.com/). The benefit of a web interface installation is that you can easily view the annotated source code with additional information: You get a colored overview of the difference between two revisions and some more advanced information about who added or changed the file in which revision.

To get similar information from your source code system, you would need to use the command-line tool, like git blame. In SVN you would use the command svn blame. You could use TortoiseSVN, or you could benefit from a toolchain that integrates a VCS browser, such as FishEye with SVN.

 

Once you configure the required settings, it’s time to create the first branch:

mvn release:branch -DbranchName=myFirstRelease

Shortly after executing the plug-in, it asks for the next version number (which is then recorded in the POM) on your trunk. For instance, if your project has the version number 1.0.0-SNAPSHOT (check your pom.xml for its <version/> XML element), you’d see the following prompt:

What is the new working copy version for "my-tool"? (com.book:my-tool) 
     1.0.1-SNAPSHOT:

 

Maven Version Schema

The Maven Release plug-in requires the version scheme to end with -SNAPSHOT, such as 1.2.4-alpha-4-SNAPSHOT.

 

Hitting the Enter key accepts the suggested value, or you can specify a different version number. After that, the plug-in updates the version number in pom.xml and commits the change to the SVN trunk. The plug-in also creates the new branching folder in the source repository. The pattern for the myFirstRelease branch would be something like this: https://server/svn/path/to/project/branches/myFirstRelease.

 

Maven Release plug-in: tools and processes

Tools follow the development process, which means that the process decides about tools; we discussed this in chapter 2. But some people state that the Maven Release plug-in doesn’t support the individual release process that exists in their project.

You shouldn’t customize your process to map to what Maven suggests with the Release plug-in. If the plug-in works for you and your process, that’s fine. If you have the option to personalize your process and align it with commonly accepted best practices, you can also use the Maven Release plug-in. But you can also implement either part or all of the releasing process yourself with other tools than Maven. For instance, it’s easy to write a small Ant script that uses a replace function and that scans your POM files and increments version numbers (see http://ant.apache.org/manual/Tasks/replace.html).

 

The Release plug-in checks whether you have modified files on your project. If this is the case, you’ll get a build failure:

[ERROR] BUILD FAILURE
[INFO] --------------------------------------------------------------------
[INFO] can't prepare the release because you have local modifications :
[pom.xml:modified]

To continue, you need to evaluate the change and commit it to the SVN trunk or revert it if it wasn’t intended. Then you need to check the newly created branch:

svn checkout
  https://server/svn/path/to/project/branches/myFirstRelease/
  directoryToContainTheBranch

If you look at the scm section of the pom.xml file in the version of the code you just checked, you’ll notice that the previous branch goal also updated this section. It contains the right URLs to point to the new SVN branch.

Before you can continue with the release preparation, your project’s pom.xml file needs some extra configuration (see the following listing).

Listing 5.12. Maven Release plug-in in the project’s POM

Inside the build section of your pom.xml file, you must configure the Maven Release plug-in . The important part here is the tagBase element, which tells the plug-in the base location for all of your Subversion tags . By default, the plug-in assumes that you created the release out of the trunk folder. Therefore, it can automatically follow the common pattern to create all tags as subdirectories of the default SVN tag location (the tags/folder). Because we want to run the release preparation from a branch, we specify that configuration in the pom.xml file.

 

Generating Out of the Trunk

Generating releases out of the trunk folder is common, but this approach has some risks, because you’re modifying the trunk. You should take this approach only if you have some experience with the tools. It’s safer to have a separate release branch folder for this step. The good news is that the extra setup isn’t too difficult.

 

Now you can finally start the preparation of your release:

mvn release:prepare

What does preparation mean? If your branch has some Maven snapshot dependencies, the plug-in asks if you want to continue with the release procedure. Generally it’s a bad idea to branch and release a software package that has a dependency on a snapshot, because snapshots are unreleased software or software under development, and they’re usually generated every day with a nightly build. If you depend on snapshots, your released project can become unstable. In the worst case, it can mean that some depreciated APIs will be removed from your dependency overnight, making your released software no longer usable. You want to avoid this scenario and should never release software that has a snapshot dependency; instead you want to reference stable versions.

Once the Release plug-in identifies a snapshot dependency in your project, it gives you the following prompt:

There are still some remaining snapshot dependencies.: Do you want to _
     resolve them now? (yes/no) no:

The default value for the prompt is no, as you should avoid snapshot dependencies. If you continue with the release by entering yes even if there’s one or more snapshot dependencies in your project, the plug-in will force you to upgrade to the released version of the dependency.

 

Tip

To avoid any snapshot dependencies in your project, use the Maven Enforcer plug-in, which we’ll discuss later in this chapter.

 

For instance, if your project depends on a 1.0.2-SNAPSHOT dependency, the plug-in wants to use the 1.0.2 released version. If your project depends on a change that was made in 1.0.2-SNAPSHOT and there’s no such release yet, you should delay releasing the latest changes and instead use an earlier release version of the dependency.

 

“But I must release with snapshots!”

Sometimes there are no other options and you must release with a snapshot dependency; perhaps a critical bug was fixed on the dependent project.

One approach to releasing is to build the dependency on your local desktop and add the dependency to your local repository. Then you can manually change the version to something that would never be picked by the original project team (in order to prevent the situation where an officially released version of that third-party library has the version number you’ve used). For example, using a version string like 1.0.2-modified-by-MyCompany-for-iteration-1 makes it clear that you (or your team) provided the modified (or hacked) dependency. This special dependency is also a candidate for your distribution, as it’s not available elsewhere. Note that this approach is only a valid option if the license of the open source dependency fits into your own schema.

Another option for dealing with snapshots is the versions:lock-snapshots plug-in. It inspects all POMs and replaces all snapshot versions with the current timestamp version of that -SNAPSHOT (for instance, -20100320.172301-4). You can release with this modified code base. You could also switch back to regular snapshots via the versions:unlock-snapshots plug-in. Maven’s Versions plug-in also provides some other interesting goals for working with versions (see http://mojo.codehaus.org/versions-maven-plugin/).

 

To continue with the preparation, let’s assume there’s a release, but you haven’t picked up the released version yet. You could manually update the version and commit the change to SVN or let the Release plug-in do it for you. During the execution of the release:prepare goal, the plug-in asks you for the release’s version number:

What is the release version for "my-tool"? (com.book:my-tool) 1.0.0: : _
     1.0.0-alpha

What is SCM release tag or label for "my-tool"? (com.book:my-tool) _
     my-tool-1.0.0-alpha: :

What is the new development version for "my-tool"? (com.book:my-tool) _
     1.0.1-alpha-SNAPSHOT: :

For the version number, you could accept the suggested 1.0.0 value, or you could specify a different version ID. It’s always good to have some alpha, beta, or release-candidate releases before shipping a final release, so let’s imagine there’s a demand for an alpha release. At the prompt, type in 1.0.0-alpha and accept the suggested defaults for the Subversion tag and the next version number increment. After you have specified the version number, the plug-in triggers the regular project build and executes some SVN tasks. Under the hood, it creates the tag (the URL would be http://server/svn/path/to/project/tags/my-tool-1.0.0-alpha) and updates and commits the new version number (1.0.1-alpha-SNAPSHOT) to your previously created branch.

 

SVN Tag

A tag inside the Subversion repository is, by convention, a folder that’s never updated. Tag folders are a single source of truth because they identify the exact version of the source code that was used to create a baselined release. Later, if you have to patch an existing release—for instance, due to a critical security bug—you usually create a branch by copying the corresponding tag folder to make sure that you’re patching the correct baseline of the code. By committing changes to the tag (which is possible by default), the tag automatically becomes a branch (by convention), which can be confusing. If you want to be sure that this doesn’t happen, restrict the access to tag folders by using SVN hooks.

 

If problems occur during the release:prepare goal, execution could lead to a build failure. In such cases, you need to solve the failures. For instance, if your SVN server is down, you need your admin to get it back up. Once the problems are fixed, you can continue from where the Maven Release plug-in exited the job by entering this command:

mvn release:prepare -Dresume

Another option would be to run mvn release:clean release:prepare after first deleting all temporary files created by the plug-in, and then rerun the entire release preparation again. If the Release plug-in has already changed some of your files, like the version in the pom.xml files, you can roll back these changes (before re-running prepare):

mvn release:rollback

The Release plug-in has built-in safety features, so you won’t spend extra hours reassembling the release again and again. This is part of the Agile strategy behind Maven and similar tools.

5.4.2. Creating the release

Before you can perform the last steps on your release—the creation of the release—configure your pom.xml file’s distributionManagement section. Here’s an example of what this might look like:

<distributionManagement>
 <repository>
  <id>local-repository</id>
  <name>My staging component repo</name>
  <url>file:///m2_repo</url>
 </repository>
</distributionManagement>

Inside the distributionManagement section, you specify the Maven repository to which the release should be deployed. Usually the Maven repository is a remote (HTTP) server, via SSH. For now, we’ll use a folder on the local filesystem, because it works just as well. Later in the chapter, we’ll discuss uploading to a remote Maven repository and the required configuration.

To finish the release procedure, you need to invoke the release:perform goal:

mvn release:perform

The plug-in creates a target/checkout directory and checks the previously created Subversion tagged version of your project. Next, it executes the normal build process. With the release:perform goal, you not only generate the normal JAR file (or whatever deployment unit your project delivers); the plug-in also generates the matching javadoc-jar and source-jar files:

my-tool-1.0.0-alpha-javadoc.jar
my-tool-1.0.0-alpha-sources.jar
my-tool-1.0.0-alpha.jar

During the execution of the release:perform goal, the Maven Release plug-in uses the Deploy plug-in and handles the upload to the Maven repository, which has been configured in the distributionManagement section of your project’s pom.xml file. If no error occurs, the Release plug-in cleans up its temporary files by internally calling the following:

mvn release:clean

This erases the backup POM files (pom.xml.releaseBackup) and the release.properties file, which store information about the ongoing release process. Congratulations! Your release is complete.

You saw many useful Maven features implemented through its plug-ins. But there’s still some room for improving this release process. By configuring a remote repository (the component repository), the invocation of mvn release:perform will deploy the artifacts to the server. But you don’t always want to directly deploy the newly generated Maven artifacts to a production Maven repository. In a case like that, use release:stage goal:

mvn release:stage
-DstagingRepository=remote-repo::default::scpexe://URL_TO_DIR

The release:stage goal doesn’t require a repository URL to be present in the pom.xml file, but the same settings.xml configuration is still needed. Note that the stagingRepository parameter starts with the ID of the specified server.

5.4.3. Testing the release

A software release requires more than merely compiling sources from a Subversion tag. The QA team must review and test the generated artifacts (such as documentation and binary JAR files). Every project has many JUnit tests, and the entire development starts with a test case. The final testing is done by the QA team that eventually approves the release.

The QA team gets the released artifacts from a staging Maven repository. The best solution would be that the QA and the customer get exactly those files that the QA team has tested and verified. Sometimes, you need to make the exact JAR files that you used for testing available. Imagine that your project is implementing some (Java) standard (such as part of the Java Community Process, JCP) and that executing a TCK (technology compatibility kit) is part of the required test plan.

The staging repository could be the previously specified file:///m2_repo directory, which could be mounted as a network device. The QA test plan is executed against the previously created release artifacts.

Once the QA team gives its approval, you should deploy the Maven artifacts to a production or release Maven repository. Here are a few options for automating the deployment:

  • Rebuild the bits from the Subversion tag. Once you build the code, you’ll need to use the Maven Deploy plug-in to deploy the artifacts. This plug-in reruns the important parts of the previous release procedure, without having QA test the generated bits. Testing is important, because mistakes can happen while rebuilding the release from the TAG folder.
  • Copy the artifacts to the final repository. Use secure copy scp (or copy cp) to manually copy the artifacts. This ensures the exact JAR files are made available through the Maven repository, but doing so will destroy the Maven metadata.
  • Use the Maven Stage plug-in with the stage:copy goal: mvn stage:copy
    -Dsource="file:///m2_local_staging_repo/" \
    -Dtarget="scp://maven_repository_server/path/to/repo" \
    -Dversion=1.2.3
    This plug-in is a smarter version of the copy process, because it honors the Maven metadata. The plug-in creates a zip file of everything in the local staging repository and uses scp to upload it to the specified remote server. Then, the zip is extracted and removed. This means that the artifacts are now correctly deployed to your Maven repository. The Stage plug-in is useful, particularly when you care about keeping Maven metadata files intact. But the plug-in itself has some small issues. You have to specify a meaningless version parameter, and it’s not possible to download from a remote server to your local folder. Additionally, your current user account needs to exist on the server, and you’re required to have write access rights for the remote directory. Finally, the password entered for the remote account is displayed in plain text. These are significant limitations, but the overall benefits outweigh the extra effort.
  • Copy the artifact via a repository manager feature set. Artifactory provides a context menu for all artifacts in your repositories (see figure 5.11). You can mark artifacts and copy or move them to another repository (such as a special staging repository). Artifactory provides a dry-run feature, so you can first try to execute the command without any commit to keep a consistent state, even if something goes wrong. If your check is successful, you can then execute the command. You can also use advanced staging strategies by applying matrix parameters and smart searches. For more details on this, see chapter 7.
    Figure 5.11. Copy artifacts to a staging repository withArtifactory

5.4.4. Useful Maven plug-ins for releasing

Previously, we talked about the undesired effect of introducing a snapshot dependency into your project. In most commercial software projects, the code is released only after it has successfully completed a formal release process. The Maven Release plug-in can help catch snapshot dependencies, but there are better options. Catching unreleased dependencies is something that needs to happen as soon as possible, not a few days before the release deadline.

One way to ensure that you never introduce snapshot dependencies is to use the Maven Enforcer plug-in. This plug-in evaluates a given set of rules to make sure they’re honored on every build of your project. It not only helps to prevent dependencies on unreleased software (and force that no snapshots are included as dependencies), but you can also require a specific version of the JDK, as shown in the following listing.

Listing 5.13. Maven Enforcer plug-in ensuring no snapshots are used

In order to use the plug-in, you need to configure it inside the build section of your project. The enforce goal runs the configured rule set against every module of your build. This is important for the requireReleaseDeps rule . For the JDK rule , you could use the enforce-once goal, as this doesn’t change for the submodules of your multimodule project. Violating the rules will cause a build error, so no code is compiled.

It’s recommended that you specify message for each rule to provide information on why the build failed. If you don’t configure the custom message, the Enforcer plug-in will use a default one. For instance, executing our project with Java 5 will display the following warning:

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireJavaVersion _
     failed with message:
Project needs to be compiled with Java 6

The warning message for dependencies that have been disallowed looks similar:

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.RequireReleaseDeps _
     failed with message:
You need approval before you use SNAPSHOTS
Found Banned Dependency: org.project.foo.:jar:1.0-SNAPSHOT

The Enforcer plug-in helps to ascertain which JAR files have caused the problem.

More rules can be used with the Enforcer plug-in. For instance, you could enforce the existence and values of properties, or you can restrict the execution of the build to Linux-based systems (see http://maven.apache.org/enforcer/enforcer-rules/). It’s important to note that the plug-in isn’t limited to its built-in rules: It offers a rich API for creating custom rules (see the documentation at http://maven.apache.org/enforcer/enforcer-api/writing-a-custom-rule.html).

5.4.5. Using cryptography with Maven

Another helpful plug-in is Maven GPG. Security is essential. If your company makes the generated release artifacts available to the wider public, it’s important to sign the release with a cryptographic key. Doing so assures potential users that the package is tamperproof and comes from a legitimate source. The same is true for open source projects. Potential users (or customers) downloading the files from your website or a Maven repository assume that your release manager created those files and that they won’t cause problems. To deliver on the trust you have built, you must sign all generated artifacts of the release by using GPG (GNU Privacy Guard).

The GnuPG project (http://gnupg.org) is a popular implementation of the Open-PGP standard, which is defined in RFC 4880 (OpenPGP Message Format). Merely signing the artifacts isn’t enough. You need to make your public keys for the release managers available. Open source projects usually offer some information on how to verify the distributed files[8]. If something is wrong with the JAR file, the verification will give you an error message:

8 See Apache-MyFaces at http://myfaces.apache.org.

$ gpg --verify my-tool-1.2.3.jar.asc my-tool-1.2.3.jar
gpg: Signature made Fri 12 Feb 2010 12:33:38 PM CET using DSA key ID 1CE17EDC
gpg: BAD signature from "Your name <you@company.de>"

In order to sign the release artifacts, you must install GPG and create your own key pair. GnuPG has instructions on how to install the software on different operating systems. On Ubuntu, the software is usually already on your machine. A good quick introduction is available here: http://wiki.wsmoak.net/cgi-bin/wiki.pl?ReleaseSigning.

Once you have your key pair, you need to sign all the files that you’re uploading to the public:

gpg --armor --output my-tool-1.2.3.jar.asc --detach-sig my-tool-1.2.3.jar

This can be time-consuming on projects with many artifacts. Thankfully, in the Maven ecosystem, the Maven GPG plug-in does the job for you, as shown in the following listing.

Listing 5.14. Maven GPG plug-in signing files on every build
<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-gpg-plugin</artifactId>
  <version>1.0</version>
  <executions>
   <execution>
    <id>sign-artifacts</id>
    <phase>verify</phase>
    <goals>
     <goal>sign</goal>
    </goals>
   </execution>
  </executions>
 </plugin>

This configuration prompts for the private key and then creates a cryptographic signature on every build. Usually, you won’t want to sign the files all the time. It’s only needed when you build the release. A common practice is to put this plug-in into a special release profile. We’ll discuss the definition of profiles in the next section.

5.4.6. Maven assembly

Another helpful plug-in is Maven Assembly. Your project’s JAR files (including -java-doc.jar and -sources.jar files) are usually deployed to a Maven repository. But when your project generates many different artifacts besides the JAR files, such as documentation or WAR files, you’ll generally also want a binary distribution that ships all the artifacts. (Note that the distribution has to contain the JAR files as well, because obviously the Maven repository may not be accessible to a potential user.)

The Assembly plug-in assists you with the creation of distributions. A simple configuration is shown in the following listing.

Listing 5.15. Maven Assembly plug-in
<project ...>
<plugin> 
  <artifactId>maven-assembly-plugin</artifactId> 
  <version>2.2-beta-5</version>
  <configuration> 
   <descriptorRefs> 
    <descriptorRef>jar-with-dependencies</descriptorRef> 
    <descriptorRef>bin</descriptorRef> 
    <descriptorRef>src</descriptorRef> 
    <descriptorRef>project</descriptorRef> 
   </descriptorRefs> 
  </configuration> 
</plugin>

You can invoke the Assembly plug-in with the following command:

mvn assembly:assembly

This starts the regular build and creates a few standard distributions:

  • Dependency JAR— Creates a JAR file that contains all classes of your module and those that it depends on. For instance, if your project has a dependency against Apache Log4j, the plug-in extracts its classes into your my-tool-1.2.3-jar-with-dependencies.jar file. That means that you can ship your application and all its dependencies in a single (huge) JAR file.
  • bin— Contains all JAR files of the module.
  • src— Contains the content of the src folder, including the module’s pom.xml file.
  • Project— Contains the source code of the embedding module (not all modules).

The formats for the distributions are different. Although the Dependency JAR distribution is created as a JAR file, the other distributions (BIN, SRC, and Project) come in different formats, such as these:

  • .zip
  • .tar.gz
  • .tar.bz2
  • .jar* (only used for the “Dependency JAR”)

The four default distribution options are convenient for small projects that have only one module. In most cases, though, they aren’t enough because a typical Maven project consists of multiple modules:

/API
/Implementation
/web-app-spring
/web-app-simple
...

This means that your distribution needs to ship multiple JAR files and all example applications along with the source code for all modules.

To deliver the source code, you can use the built-in project descriptor. For the binary part of the distribution, you need to create a custom description file that you place under the src/assembly of the root directory (see the following listing).

Listing 5.16. Assembly description file

To use the preceding XML file, you must register it with the Assembly plug-in. Because the complete assembly process takes quite some time, you shouldn’t execute it on the regular build. Instead, you can define a special profile (see the following listing).

Listing 5.17. Configuring Assembly as a profile

The new profile with the ID generate-assembly is invoked when the assembly-Project is set to true . The complete command for this profile is as follows:

mvn -DassemblyProject=true package assembly:assembly

Executing the command runs the build process and then creates all the archive files. The configuration is quite simple. The descriptorRefs section reuses the built-in project descriptor , and within the descriptors element , point the Assembly plug-in to your custom assembly file. The result of the archive looks like this:

ARCHIVE-FILE:
/book-api-version.jar
/book-impl-version.jar
/book-web-app-spring-version.war
/book-web-app-simple-version.war

Even with these results, you aren’t done yet. A complete distribution needs more than the binary deliverables. Most projects also make the generated Javadoc files available as well.

To make this happen, you must use the Maven Dependency plug-in, because you need to copy the desired artifacts. The Dependency plug-in provides the capability to manipulate artifacts—it can copy or unpack artifacts from local or remote repositories to a specified location. Inside the generate-assembly profile, you need to configure the Maven Dependency plug-in, as shown in the following listing.

Listing 5.18. Maven Dependency plug-in to include Javadoc

Inside the plug-in, you can define multiple execution elements. You could define a global copy-everything execution ID, but don’t because doing so would mix different artifacts, such as sources-jar and javadoc-jar files in the same folder of the package. To keep things more organized, make sure every artifact classifier has an execution element, like copy-javadoc or copy-sources. In the preceding configuration, the specified copy goal copies all nested artifactItems to the javadoc folder inside the Target directory. If you want to make the -sources-jar files available, you should place them, through their own execution element, into a sources folder in the Target directory.

Next. you need to tell the assembly process to pick up the javadoc artifacts. To do so, add the following fileSet element to your custom assembly descriptor, the bin.xml file:

<fileSets> 
 <fileSet> 
  <directory>target/javadoc</directory> 
  <outputDirectory>javadoc</outputDirectory> 
 </fileSet> 
</fileSets>

Last but not least, it’s commonly required that you make your dependencies available. If your customer gets your artifact from a repository, you don’t need to worry about this, because Maven resolves their dependencies and downloads them along with your files. But for those without a Maven project, you need to ship the dependencies as part of the distribution:

<dependencySets> 
 <dependencySet> 
  <outputDirectory>lib</outputDirectory> 
  <scope>compile</scope> 
 </dependencySet> 
</dependencySets>

You must add the preceding configuration to your custom assembly descriptor. It adds all compile-time dependencies of your root pom.xml file to the final distribution, inside its lib folder. You can use different Maven scopes to bind the operation to the defined phase, such as compile in the preceding example. Dependencies of a different Maven scope require a separate dependencySet element.

The preceding distribution process defines a profile inside of the root pom.xml file to generate the different archives. Consequently, the pom.xml grows hard to read. To keep pom.xml files small, create a separate assembly module as part of the project. As before with the generate-assembly profile, you shouldn’t execute this module as part of the regular build, which means you won’t list this module inside the modules section:

<modules>
 <module>book-api<module>
 <module>book-impl<module>
 ...
</modules>

To generate the distribution, you must navigate into the assembly module and run the mvn package assembly:assembly command from there.

In section 5.4.1, you saw that executing mvn release:prepare increases the version numbers for all (sub)module pom.xml files. Because you’re excluding the assembly module from the modules section, you need to ensure the version of the assembly module gets updated as well. To archive this, introduce a prepare-release profile, as shown in the following listing.

Listing 5.19. Include the assembly as part of a release to update its version
<profiles>
 <profile>
  <id>prepare-release</id>
  <activation>
   <property>
     <name>prepareRelease</name>
    <value>true</value>
   </property>
  </activation>
  <modules>
    <module>my-assembly</module>
  </modules>
 </profile>

The profile is invoked when the prepareRelease parameter is set to true. In addition to the release:prepare goal, specify the parameter:

mvn release:prepare -DprepareRelease=true

This statement applies the command to all (default) modules and to those that are listed in the prepare-release profile.

5.4.7. Tooling beyond Maven and outlook

The plug-ins we’ve discussed help to generate a Maven release with a complete distribution. The Release (or Stage) plug-in also deploys the Maven artifacts to a given Maven repository, but the final distribution should also be made available on your website. Because your release process already signs all generated JAR files, the final distribution should be signed too.

To make this procedure simple and easy for all team members to use, you could create a Python script that automates the distribution signing and uploading. The script should also combine the various mvn commands, such as mvn -Dassembly-Project=true package assembly:assembly or mvn release:stage... The Apache MyFaces project uses a Python script for this: http://svn.apache.org/repos/asf/myfaces/trinidad/trunk/scripts/trinidad-build-release.py. You can download and customize it as needed. The script is executed as follows:

./script.py passpharse /folder/to/the/source

Most plug-ins are configured in the root pom.xml file, because they’re visible for the submodules as well. But because a software company has several products or projects, and all have similar requirements, like a release or the configuration of the source code management, it’s recommended that you create a master POM that contains the most common plug-ins and dependencies. On each project, your root pom.xml will point to your master POM file as its parent:

<project ...>
 <modelVersion>4.0.0</modelVersion>
 <parent>
  <groupId>com.company.devision</groupId>
  <artifactId>devision</artifactId>
  <version>2</version>
 </parent>
 <groupId>com.company.devision.current-project</groupId>
 ...

This is a common pattern in a big organization, such as the Apache Software Foundation (ASF). The ASF releases a master POM (see https://svn.apache.org/repos/asf/maven/pom/trunk/asf/pom.xml) that contains fundamental dependencies and profiles, and each Maven-based Apache project inherits from this POM. Following this pattern makes sense. Over time the master POM evolves and includes more convenient settings that the subprojects inherit.

The Apache Maven project doesn’t only maintain the master POM—it also outlines the release guidelines for every project within Apache (http://maven.apache.org/developers/release/apache-release.html). In companies with different development teams, defining a similar guide makes perfect sense. It’s also important to maintain both the release guide and the master POM, because a build environment (including the release process) isn’t static. Things change over time.

As a final note, it’s recommended that you document the entire process for every release on your company’s wiki. This helps to document any potential problems and workarounds, resulting in a repeatable process whether you release your code monthly, nightly, or hourly, via a CI server.

5.5. Summary

This chapter introduced integration and release management, as well as Maven and its major release tools and plug-ins. We looked at where Maven stores its artifacts and which tooling options you have for hosting a component repository. You learned that it’s also possible to host Maven artifacts in Subversion, although using a dedicated repository manager like Artifactory adds much more value and results in a better solution. Additionally, you saw how Maven can be used for management integration and release management.

This chapter also serves as preparation for the advanced use cases that we’ll discuss later in this book. In the next chapter, we’ll talk about productive working environments, where Maven plays yet another important role.