Python & Java: a unified build process (3/4)

In our first blog post dedicated to Python and Java, we saw how Maven can orchestrate a unified build process for these two languages.

In the last one, we saw how to synchronize versions between Java projects and Python projects.

Now it's time to automate our release and deployment workflow:

• create a tag for each release (on GitHub, see here to setup your Git environment),
• deploy Java artifacts on a distant repository. We are using a Nexus instance configured with two hosted repositories (snapshots and releases, their names being quite straightforward) and a public group repository grouping the hosted ones plus a proxy on Maven Central
• deploy Python artifacts on a private PyPI (could be a fork of pypi.python.org or Django running a djangopypi application)

To do so, we'll use the maven-release-plugin. If you want to follow this article with the sources of the sample project, feel free to clone it from GitHub.

Most of the Maven configuration takes place in the root pom.xml. First add the source management details:

<project>
  <scm>
    <connection>scm:git:git://github.com/shiningpanda/tutorial-python-java.git</connection>
    <developerconnection>scm:git:git@github.com:shiningpanda/tutorial-python-java.git</developerconnection>
    <url>https://github.com/shiningpanda/tutorial-python-java</url>
  </scm>
</project>

Then declare the repositories for snapshot and release artifacts (matching the Nexus ones):

<project>
  <distributionmanagement>
    <repository>
      <id>releases</id>
      <name>Releases</name>
      <url>http://dev.shiningpanda.com/nexus/content/repositories/releases</url>
    </repository>
    <snapshotrepository>
      <id>snapshots</id>
      <name>Snapshots</name>
      <url>http://dev.shiningpanda.com/nexus/content/repositories/snapshots</url>
    </snapshotrepository>
  </distributionmanagement>
</project>

Also add the maven-release-plugin configuration, note the autoVersionSubmodules tag that tells maven that all projects have the same version:

<project>
  <build>
    <pluginmanagement>
      <plugins>
        <plugin>
          <groupid>org.apache.maven.plugins</groupid>
          <artifactid>maven-release-plugin</artifactid>
          <version>2.2.2</version>
          <configuration>
            <goals>clean deploy</goals>
            <autoversionsubmodules>true</autoversionsubmodules>
            <tagnameformat>tutorial-python-java-@{project.version}</tagnameformat>
          </configuration>
        </plugin>
      </plugins>
    </pluginmanagement>
  </build>
</project>

Copy this settings.xml in $HOME/.m2 folder and edit it to modify the Nexus url and the credentials for the nexus, <i>releases</i> and snapshots repositories.

Now it's time to handle the Python side. We saw that the deploy goal is mapped with the python setup.py register sdist upload command. We first need to declare the private PyPI in the $HOME/.pypirc file (replace the credentials and the private PyPI url):

[distutils]
index-servers =
    pypi
    private

[pypi]
username = xxx
password = xxx

[private]
repository = http://dev.shiningpanda.com/pypi
username = xxx
password = xxx

Then we need to tell that the pysample project must be deployed on the private PyPI. To do so, modify or create a setup.cfg file along the setup.py and append the following lines:

[register]
repository = private

[upload]
repository = private

We're now ready to release and deploy. To release, issue the following command and answer the questions:

$ mvn release:prepare
[...]
What is the release version? 0.3 
What is SCM release tag or label? tutorial-python-java-0.3
What is the new development version? 0.4-SNAPSHOT
[...]
[INFO] BUILD SUCCESS

Your project is now released, time to go threw our final step, the deployment:

$ mvn release:perform
[...]
[INFO] Registering pysample to http://dev.shiningpanda.com/pypi
[INFO] Server response (200): OK
[...]
[INFO] Submitting dist/pysample-0.3.tar.gz to http://dev.shiningpanda.com/pypi
[INFO] Server response (200): OK
[...]
[INFO] Uploading: http://dev.shiningpanda.com/nexus/.../jsample/0.3/jsample-0.3.jar
[...]
[INFO] BUILD SUCCESS

In addition to a Hosted Continuous Integration Service, ShiningPanda CI also offers build and release management expertise, so if you have questions or if you are stuck with your internal build process do not hesitate to contact our service team!