Deploy your own custom Eclipse

This is the follow up of my previous post about building a custom Eclipse distribution. In this post I’ll show how to deploy the p2 site and the zipped products on Sourceforge. Concerning the p2 site, I’ll use the same technique, with some modifications, for building a composite update site and deploy it with rsync that I showed on another post.

In particular, we’ll accomplish¬†several tasks:

  • creating and deploying the update site with only the features (without the products)
  • creating and deploying the update site including product definition and the zipped provisioned products
  • creating a self-contained update site (including all the dependencies)
  • providing an ant script for installing your custom Eclipse from the net

The code of the example can be found at:¬† In particular, I’ll start from where I left in the previous post.

The source code assumes a specific remote directory on Sourceforge, that is part of one of my Sourceforge projects, and it is writable only with my username and password. If you want to test this example, you can simply modify the property remote.dir in the parent pom specifying a local path in your computer (or by passing a value to the maven command with the syntax -Dremote.dir=<localpath>). Indeed, rsync can also synchronize two local directories.

Recall that when you perform a synchronization, specifying the wrong local directory might lead to a complete deletion of that directory. Moreover, source and destinations URLs in rsync have a different semantics depending on whether they terminate with a slash or not, so make sure you understand them if you need to customize this ant file or to pass special URLs.  

Creating and Deploying the p2 composite site

This part reuses most of what I showed in the previous posts:

In this blog post we want to be able to add a new p2 site to the composite update site (and deploy it) for two different projects:

  • This is the update site with only our features and bundles
  •¬†This is the update site with our features and bundles and the Eclipse product definition.

To reuse the ant files for managing the p2 composite update site and syncing it with rsync, and the Maven executions that use such ant files, we put the ant files in the parent project customeclipse.example.tycho, and we configure the Maven executions in the pluginManagement section of the parent pom.

We also put in the parent pom all the properties we’ll use for the p2 composite site and for rsync (again, please have a look at the previous posts for their meaning)

The pluginManagement section contains the configuration for managing the composite update site.

ATTENTION: in the following snipped, for the sake of readability, I split the <appArgLine> into several lines, but in your pom.xml it must be exactly in one (long) line.

The pluginManagement section also contains the configuration for updating and committing the composite update site to Sourceforge.

Now, we can simply activate such plugins in the build sections of our site projects described above.

In particular, we activate such plugins only inside profiles; for example, in the project we have:

In we have similar sections, but the profiles are called differently, release-ide-composite and deploy-ide-composite, respectively.

So, if you want to update the p2 composite site with a new version containing only the features/bundles and deploy it on Sourceforge you need to run maven as follows

If you want to do the same, including the custom product definitions you need to run maven as follows¬†(the additional build-ide profile is required because the is included as a Maven module only when that profile is activated; this way, products are created only when that profile is activated – just because provisioning a product requires some time and we don’t want to do that on normal builds)

NOTE: The remote directory on Sourceforge hosting  the composite update site will always be the same. This means that the local composite update site created and updated by both deploy-composite and deploy-ide-composite will be synchronized with the same remote folder.

In the, we added a p2.inf file with touchpoint instructions to add as update site in our Eclipse products the update site hosted on Sourceforge:

Deploying the zipped products

To copy the zipped products on Sourceforge we will still use rsync; actually, we won’t use any synchronization features: we only want to copy the zip files. I¬†could have used the Ant Scp or Sftp tasks, but I experienced many problems with such tasks, so let’s use rsync also for that.

The ant file for rsync is slightly different with respect to the one shown in the previous post, since it has been refactored to pass the rsync macro more parameters. We still have the targets for update/commit synchronization; we added another target that will be used to simply copy something (i.e., the zipped products) to the remote directory, without any real synchronization. You may want to have a look at rsync documentation to fully understand the command line arguments.

In the, in the deploy-ide-composite profile, we configure another execution for the maven ant plugin (recall that in this profile the rsync synchronization configured in the parent’s pom pluginManagement section is¬†also executed); this further execution will copy the zipped products to a remote folder on Sourceforge (as detailed in the previous post, you first need to create such folder using the Sourceforge web interface):

Note that when calling the rsync-copy-dir-contents of the rsync.ant file, we pass the properties as nested elements, in order to override their values (such properties’ value are already defined in the parent’s pom, and for this run we need to pass different values).

Now, if we run

many things will be executed:

  • rsync will synchronize our local composite update site with the remote composite update site
  • a new p2 site will be created, and added to our local composite update site
  • rsync will synchronize our local changes with the remote composite update site
  • Eclipse products will be created and zipped
  • the zipped products will be copied to Sourceforge

A self-contained p2 repository

Recall from the previous post that since in customeclipse.example.ide.feature we added Eclipse features (such as the platform and jdt) as dependencies (and not as included features), then the p2 update site we’ll create will not contain such features: it will contain only our own features and bundles. And that was actually intentional.

However, this means that the users of our features and of our custom Eclipse will still need to add the standard Eclipse update site before installing our features or updating the installed custom Eclipse.

If you want your p2 repository to be self-contained, i.e., to include also the external dependencies, you can do so by setting includeAllDependencies to true in the configuration of the tycho-p2-repository-plugin.

It makes sense to do that in the, so that all the dependencies for our custom Eclipse product will end up in the p2 repository:

However, doing so every time we add a new p2 update site to the composite update site would make our composite update site grow really fast in size. A single p2 repository for this example, including all dependencies is about 110Mb. A composite update site with just two p2 repositories would be 220Mb, and so on.

I think a good rule of thumb is

  • include all dependencies the first time we release our product’s update site (setting the property includeAllDependencies to true, and then setting it to false right after the first release)
  • for further releases do not include dependencies
  • include the dependencies again when we change the target platform of our product (indeed, Tycho will take the dependencies from our target platform)

Provide a command line installer

Now that our p2 composite repository is on the Internet, our users can simply download the zip file according to their OS, unzip it and enjoy it. But we could also provide another way for installing our custom Eclipse: an ant file so that the user will have to

The ant file will use the p2 director command line application to install our Eclipse product directly from the remote update site (the ant file is self-contained since if the director application is not already installed, it will install it as the first task).

Here’s the install.ant file (note that we ask the director to install our custom Eclipse product, customeclipse.example.ide and, explicitly, the main feature customeclipse.example.feature; this reflects what we specified in the product configuration, in particular, the fact that customeclipse.example.feature must be a ROOT feature, so that it can be updatable – see all the details in the previous post)

Note that this will always install the latest version present in the remote composite update site.

For instance, consider that you created zipped products for version 1.0.0, then you deployed a small upgrade only for your features, version 1.0.1, i.e., without releasing new zipped products. The ant script will install the custom Eclipse including version 1.0.1 of your features.

Some experiments

You may want to try and download the zipped product for your OS from this URL:

After I deployed the self-contained p2 repository and the zipped products (activating the profiles release-ide-composite and deploy-ide-composite, with the property includeAllDependencies set to true, using the project, I deployed another p2 repository into the composite site only for the customeclipse.example.feature (activating the profiles release-composite and deploy-composite, i.e., using the project

Unzip the downloaded product, and check for updates (recall that the product is configured with the update site hosted on Sourceforge, through the p2.inf file described before). You will find that there’s an update for the Example Feature:

customeclipse before upgrading customeclipse available updates

After the upgrade and restart you should see the new version of the feature installed:

customeclipse after upgrading

Now, try to install the product using the ant file shown above, that can be downloaded from

You’ll have to wait a few minutes (and don’t worry about cookie warnings); run this version of the custom Eclipse, and you’ll find no available updates: check the installation details and you’ll see you already have the latest version of the Example¬†Feature.

That’s all! Hope you find this post useful and… Happy Easter ūüôā

Be Sociable, Share!

Build your own custom Eclipse

In this tutorial I’ll show how to build a custom Eclipse distribution with Maven/Tycho. We will create an Eclipse distribution including our own features/plugins and standard Eclipse features, trying to keep the size of the final distribution small.

The code of the example can be found at:

First of all, we want to mimic the Eclipse SDK product and Eclipse SDK feature; have a look at your Eclipse Installation details

eclipse SDK installation details

You see that “Eclipse SDK” is the product (org.eclipse.sdk.ide), and “Eclipse Project SDK” is the feature (

Moreover, we want to deal with a scenario such that

Our custom feature can be installed in an existing Eclipse installation, thus we can release it independently from our custom Eclipse distribution. Our custom Eclipse distribution must be updatable, e.g., when we release a new version of our custom feature. 

The project representing our parent pom will be

  • customeclipse.example.tycho

The target platform is defined in

  • customeclipse.example.targetplatform

For this example we only need the org.eclipse.sdk feature and the native launcher feature

We created a plugin project and a feature project including such plugin (the plugin is nothing fancy, just an “Hello World Command” created with the Eclipse Plug-in project wizard):

  • customeclipse.example.plugin
  • customeclipse.example.feature

We also create another project for the p2 repository (Tycho packaging type: eclipse-repository) that distributes our plugin and feature (including the category.xml file)


All these projects are then configured with Maven/Tycho pom.xml files.

Then we create another feature that will represent our custom Eclipse distribution

  • customeclipse.example.ide.feature

This feature will then specify the features that will be part of our custom Eclipse distribution, i.e., our own feature (customeclipse.example.feature) and all the features taken from the Eclipse update sites that we want to include in our custom distribution.

Finally, we create another site project (Tycho packaging type: eclipse-repository) which is basically the same as, but it also includes the product definition for our custom Eclipse product:


NOTE: I’m using two different p2 repository projects because I want to be able to release my feature without releasing the product (see the scenario at the beginning of the post). This will also allow us to experiment with different ways of specifying the features for our custom Eclipse distribution.

Product Configuration

This is our product configuration file customeclipse.example.ide.product in the project and its representation in the Product Configuration Editor:

custom eclipse product configuration1

Note that we use org.eclipse.sdk.ide and org.eclipse.ui.ide.workbench for launching product extension identifier and application (we don’t have a custom application ourselves).

ATTENTION: Please pay attention to “uid” and “id” in the .product file, which correspond to “ID” and “Product” in the Product definition editor (quite confusing, isn’t it? ūüėČ

This product configuration includes our customeclipse.example.ide.feature; we also inserted in the end the standard start level configuration, and other properties, like the standard workspace location.

The pom in this project will also activate the product materialization and archiving (we also specify the file name of the zip with our own pattern):

We chose NOT to include as a module in our parent pom.xml: we include it only when we enable the profile build-ide: installing and provisioning a product takes some time, so you may not want to do that on every build invocation.  In that profile we add the module, this is the relevant part in our parent pom

In this profile, we also specify the environments for which we’ll build our custom Eclipse distribution. When this profile is not active, the target-platform-configuration will use only the current environment.

In the rest of the tutorial we’ll examine different ways of defining customeclipse.example.ide.feature. In my opinion, only the last one is the right one; but that depends on what you want to achieve. However, we’ll see the result and drawbacks of all the solutions.

You may want to try the options we detail in the following by cloning the example from and by modifying the corresponding files.

Include org.eclipse.sdk

The first solution is to simply include the whole org.eclipse.sdk feature in our customeclipse.example.ide.feature:

You can run the maven build specifying the profile build-ide

To get the materialized products (and the corresponding zipped versions).

NOTE: if you enable the tycho-source-feature-plugin in the parent pom to generate also source features, you’ll get this error during the build:

That’s because it tries to include in customeclipse.example.ide.feature.source the source feature of org.eclipse.sdk, which does not exist (org.eclipse.sdk already includes sources of its included features). You need to tell the tycho plugin to skip the source of org.eclipse.sdk:

The build should succeed.

Let’s copy the installed product directory (choose the one for your OS platform) to another folder; we perform the copy because a subsequent build will wipe out the target directory and we want to do some experiments. Let’s run the product and we see that our custom IDE shows our custom feature menu “Sample Menu” and the corresponding tool bar button:

If we check the installation details we see the layout mimicking the ones of Eclipse SDK (which is included in our product)

custom eclipse sdk installation details

Now let’s run the build¬†again with above maven command.

If you have a look at the target directory you see that besides the products, in you also have a p2 repository,

custom ide site target

we will use the p2 repository to try and update the custom ide that we created in the first maven build (the one we copied to a different directory and that we ran in the previous step).¬†So let’s add this built repository (in my case is /home/bettini/work/eclipse/tycho/custom-eclipse/ in the custom ide’s “Install New Software” dialog.

You see our Example Feature, and if you uncheck Group items by category you also see the Custom Eclipse Project SDK feature (corresponding to customeclipse.example.ide.feature) and Custom Eclipse SDK (corresponding to our product definition uid customeclipse.example.ide).

custom ide install new software 1 custom ide install new software 2

But wait… only the product is updatable! Why? (You see that’s the only one with the icon for updatable elements; if you try “Check for updates” that’s the only one that’s updatable)

Why can’t I update my “Example Feature” by itself?

If you try to select “Example Feature” in the “Install” dialog to force the update, and press Next…

custom ide install new software force 1

you’ll get an error, and the proposed solution, i.e., also update the product itself:

custom ide install new software force 2

And if you have a look at the original error…

custom ide install new software force 3

…you get an idea of the problem beneath: since we INCLUDED our “customeclipse.example.feature” in our product’s feature “customeclipse.example.ide.feature” the installed product will have a strict version requirement on “customeclipse.example.feature”: it will want exactly the version the original product was built with; long story short: you can’t update that feature, you can only update the whole product.

Before going on, also note in the target directory you have a zip of the p2 repository that has been created: it’s about 200 MB! ¬†That’s because the created p2 repository contains ALL ¬†the features and bundles INCLUDED in your product (which in our case, it basically means, all features INCLUDED in “customeclipse.example.ide.feature”).

Require org.eclipse.sdk

Let’s try and modify “customeclipse.example.ide.feature” so that it does NOT include the features, but DEPENDS on them (we can also set a version range for required features).

Let’s build the product.

First of all, note that the p2 repository zip in the target folder of is quite small!  Indeed, the repository contains ONLY our features, not all the requirements (in case, you can also force Tycho to include all the requirements), since, as stated above, the required feature will not be part of the repository.

Now let’s do the experiment once again:

  1. copy the built product for your OS into another directory
  2. run the product custom ide
  3. run another maven build
  4. add the new created p2 repository in the custom ide “Install new software” dialog

Well… the Example Feature does not appear as updatable, but this time, if we select it and press Next, we are simply notified that it is already installed, and that it will be updated

custom ide install new software force 4

So we can manually update it, but not automatically (“Check for updates” will still propose to update the whole product).

To make a feature updatable in our product we must make it a “Root level feature” (see also

At the time of writing the Eclipse product definition editor does not support this feature, so we must edit the .product definition manually and add the line for specifying that customeclipse.example.feature must be a root level feature:

Let’s build again, note that this time the p2 director invocation explicitly installs customeclipse.example.feature

Let’s do the experiment again; but before trying to update let’s see that the installed software layout is now different: our Example Feature is now a root level feature (it’s also part of our Custom SDK IDE since it’s still required by customeclipse.example.ide.feature but that does not harm, and you may also want to remove that as a requirement in customeclipse.example.ide.feature).

custom eclipse sdk installation details 2

Hey! This time our “Example Feature” is marked as updatable

custom ide install new software 3

and also Check for updates proposes “Example Feature” as updatable independently from our product!

custom ide install new software 4

What happens if we make also customeclipse.example.ide.feature” a root feature? You may want to try that, and the layout of the installed software will list 3 root elements:¬†our product “Custom Eclipse SDK”, our ide.feature “Custom Eclipse Project SDK” (which is meant to require all the software from other providers, like in this example, the org.eclipse.sdk feature itself) and our “Example Feature”.

This means that also “Custom Eclipse Project SDK” can be updated independently; this might be useful if we plan to release a new version of the ide.feature including (well, depending on) other software not included in Eclipse SDK itself (e.g., Mylyn, Xtext, or something else). At the moment, I wouldn’t see this as a priority so I haven’t set customeclipse.example.ide.feature as a root level feature in the product configuration.

Minimal Distribution

The problem of basing our distribution on org.eclipse.sdk is that the final product will include many features and bundles that you might not want in your custom distribution; e.g., CVS features, not to mention all the sources of the platform and PDE and lots of documentation. Of course, if that’s what we want, then OK. But if we want only the Java Development Tools in our custom distribution (besides our features of course)?

We can tweak the requirements in customeclipse.example.ide.feature and keep them minimal (note that the platform feature is really needed):

Build the product now.

Note also that the installed software has been reduced a lot:

custom eclipse minimal 4

The size of the zipped products dropped down to about 90Mb, instead of about 200Mb as they were before when we were using the whole org.eclipse.sdk feature.

However, by running this product you may notice that we lost some branding

  1. There’s no Welcome Page
  2. Eclipse starts with “Resource” Perspective, instead of “Java” Perspective
  3. Help => About (Note only “About” no more “About Eclipse SDK”) shows:

custom eclipse minimal 2

To recover the typical branding of Eclipse SDK, we have to know that such branding is implemented in the bundle org.eclipse.sdk (the bundle, NOT the homonymous feature).

So, all we have to do is to put that bundle in our feature’s dependencies

Rebuild, and try the product: we have all the branding back! ūüôā

I hope you find this blog post useful ūüôā

The sources of this example can be found here:

Be Sociable, Share!

Publish an Eclipse p2 repository on Sourceforge with rsync

This can be seen as a follow-up post of my previous post on building Eclipse p2 composite repositories. In this blog post I’ll show an automatic way for publishing an Eclipse p2 (composite) repository (a.k.a. update site) on Sourceforge, using¬†rsync for synchronization. You may find online many posts about publishing update sites on Github pages and recently on bintray. (as a reminder, rsync is a one-way synchronization tool, and we assume that the master replica is the one on sourceforge; rysnc, being a synchronization tool, will only transfer the changed files during synchronization).

I prefer sourceforge for some reasons:

  • you have full and complete access to the files upload system either with a shell or, most importantly for the technique I’ll describe here, with rsync. From what I understand, instead, bintray will manage the binary artifacts for you;
  • in order to create and update a p2 composite site you must have access to the current file system layout of the p2 update site, which I seem to understand is not possible with bintray;
  • you have download statistics and your artifacts will automatically mirrored in sourceforge’s mirrors.

By the way: you can store your git repository anywhere you want, and publish the binaries on sourceforge. (see this page and this other page).

I’ll reuse the same example of the previous post,¬†the repository found here¬†, where you find all the mechanisms for creating and updating a p2 composite repository.

The steps of the technique I’ll describe here can be summarized as follows: when it comes to release a new child in the p2 composite update site (possibly already published on Sourceforge), the following steps are performed during the Maven/Tycho build

  1. Use rsync to get an update local version of the published p2 composite repository somewhere in your file system (this includes the case when you never released a version, so you’ll get a local empty directory)
  2. Build the p2 repository with Tycho
  3. Add the above created p2 repository as a new child in the local p2 composite repository (this includes the case where you create a new composite repository, since that’s your first release)
  4. Use rsync to commit the changes back to the remote p2 composite repository

Since we use rsync, we have many opportunities:

  • we’re allowed to manually modify (i.e., from outside the build infrastructure) the p2 composite repository, for instance by removing a child repository containing a wrong release, and commit the changes back;
  • we can release from any machine, notably from Jenkins or Hudson, since we always make sure to have a synchronized local version of the released p2 composite repository.

Prepare the directory on Sourceforge

This assumes that you have an account on Sourceforge, that you have registered a project. You need to create the directory that will host your p2 composite repository in the “Files” section.

For this example I created a new project eclipseexamples,, and I plan to store the p2 composite in the sourceforge file system on this path: p2composite.example/updates.

So I’ll create the directory structure accordingly (using the “Add Folder” button:

sourceforge create folder structure 1 sourceforge create folder structure 2 sourceforge create folder structure 3

Ant script for rsync

I’m using an ant script since it’s easy to call that from Maven, and also manually from the command line. This assumes that you have already rsync installed on your machine (or in the CI server from where you plan to perform releases).

This ant file is meant to be completely reusable.

Here’s the ant file

We have a macro for invoking rsync with the desired options (have a look at rsync documentation for understanding their meaning, but it should be straightforward to get an idea).

In particular, the transfer will be done with ssh, so you must have an ssh key pair, and you must have put the public key on your account on sourceforge. Either you created the key pair without a passphrase (e.g., for releasing from a CI server of your own), or you must make sure you have already unlocked the key pair on your local machine (e.g., with an ssh-agent, or with a keyring, depending on your OS).

The arguments source and¬†dest will depend on whether we’re doing an update or a commit (see the two ant targets). If you define the property¬†dryrun as¬†-n then you can simulate the synchronization (both for update and commit); this is important at the beginning to make sure that you synchronize what you really mean¬†to synchronize. Recall that when you perform an update, specifying the wrong local directory might lead to a complete deletion of that directory (the same holds for commit and the remote directory).¬†Moreover, source and destinations URLs in rsync have a different semantics depending on whether they terminate with a slash or not, so make sure you understand them if you need to customize this ant file or to pass special URLs.

The properties rsync.remote.dir and rsync.local.dir will be passed from the Tycho build (or from the command line if you call the ant script directly). Once again, please use the dryrun property until you’re sure that you’re synchronizing the right paths (both local and remote).

Releasing during the Tycho build

Now we just need to call this ant’s targets appropriately from the Tycho build; I’ll do that in the pom.xml of the project that builds and updates the composite p2 repository.

Since I don’t want to push a new release on the remote site on each build, I’ll configure the plugins inside a profile (it’s up to you to decide when to release): here’s the new part:

Now the URL to access a remote path on sourceforge with ssh has the following shape


So in my case I specified (again, the final / is crucial for what we want to synchronize with rsync, see the note above):


The local URL specifies where the local p2 composite site is stored (see the previous post), in this example it defaults to


Again, the final / is crucial.

We configured the maven-antrun-plugin with two executions:

  1. before updating the p2 composite update site (phase prepare-package) we make sure we have a synchronized local version of the repository
  2. after updating the p2 composite update site (phase verify) we commit the changes to the remote repository
  3. That’s all ūüôā

Let’s try it

Of course, if you want to try it, you need a project on sourceforge and a directory on that project’s Files section (and you’ll have to change the URLs accordingly in the pom file).

To perform a release we need to call the build enabling the profile release-composite, and specify at least verify as goal:

Let’s say we still haven’t released anything.

Since the remote directory is empty, in our local file system¬†we’ll simply have the directory created. In the end of the build, the composite site is created and the remote directory will be synchronized with our local contents:

Let’s have a look at the remote directory, it will contain the create p2 composite site

sourceforge uploaded artifacts 1

sourceforge uploaded artifacts 2

Let’s perform another release; Our local copy is up-to-date so we won’t receive anything during the update phase, but then we’ll commit another release

Let’s have a look at sourceforge and see the new release

sourceforge uploaded artifacts 3

Let’s remove our local copy and try to perform another release, this time the update phase will make sure our local composite repository is synchronized with the remote site (we’ll get the whole composite site we had already released), so that when we add another composite child we’ll update our local composite repository; then we’ll commit the changes to the server (again, by uploading only the modified files, i.e., the compositeArtifacts.xml and compositeContent.xml and the new directory with the new child repository:

Again, the remote site is correctly updated

sourceforge uploaded artifacts 4

Providing the URL of your p2 repository

Now that you have your p2 repository on sourceforge, you only need to give your users the URL to use for installing your features in Eclipse.

You have two forms for the URL

  • This will use the mirror infrastructure of sourceforge:<project>/files/<path>
  • This will bypass mirrors:<project>/<path>

If you use the mirror form, when installing in Eclipse (or provisioning a target platform) you’ll see warnings on the console of the shape

But it’s safe to ignore them.

For our example the URL can be one of the following:

  • With mirrors:
  • Main site:

You may want to try them both in Eclipse.

Please keep in mind that you may hit some unavailability errors now and then, if sourceforge sites are down for maintenance or unreachable for any reason… but that’s not much different when you hit a bad Eclipse mirror, or the main Eclipse download site is down… I guess no hosting site is perfect anyway ūüėČ

I hope you find this blog post useful, Happy releasing! ūüôā


Be Sociable, Share!

Creating p2 composite repositories during the build

I like to build p2 composite repositories for all my Eclipse projects, to keep all the versions available for consumption.

Quoting from

The goal of composite repositories is to make this task easier by allowing you to have a parent repository which refers to multiple children. Users are then able to reference the parent repository and the children’s content will transparently be available to them.

The nice thing of composite repositories is that they can be nested at any level. Thus, I like to have nested composite repositories according to the major.minor, major.minor.service.qualifier.

Thus the layout of the p2 composite repository should be similar to the following screenshot


Note that the name of the directories that contain a standard p2 repository have the same name of the contained feature.

The key points of a p2 composite repository are the two files compositeArtifacts.xml and compositeContent.xml. Their structure is simple, e.g.,

Note that a child location is intended relative to the path of these files; you can also specify absolute paths, not to mention http urls to other remote p2 sites.

The structure is not that complex, so you can also create it by hand; but keeping it up to date might not be that trivial. With that respect, p2 provides some ant tasks for managing composite repositories (creating, adding an entry, removing an entry), and that’s my favorite way to deal with composite repositories. I’ll detail what I usually do in this blog post, in particular, how to create (or update) a p2 composite repository with a new entry during the build.

The ant file is completely reusable and customizable by passing properties; you can reuse it as it is, after you setup your pom.xml as detailed below.

In this blog post I’ll show how to do that with Maven/Tycho, but the same procedure can be done in a Buckminster build (as I’ll hint at the end).

I’ll use a simple example,, consisting of a plug-in project, a feature project, a project for the site, and a releng project (a Maven/Tycho parent project). The plug-in and feature project are not interesting in this context: the most interesting one is the site project (a Tycho¬†eclipse-repository¬†packaging type).

Of course, in order to run such ant tasks, you must run them using the org.eclipse.ant.core.antRunner application. Buckminster, as an Eclipse product, already contains that application. With Tycho, you can use the tycho-eclipserun-plugin, to run an Eclipse application from Maven.

We use this technique for releasing a new version of our EMF-Parsley Eclipse project. We do that directly from our Hudson HIPP instance; the idea is that the location of the final main composite site is the one that will be served through HTTP from the We have a dedicated Hudson job that will release a new version and put it in the composite repository.

The ant file

The internal details of this ant files are not necessary to reuse it, so you can skip the first part of¬†this section (you only need to know the main properties to pass). Of course, if you read it and you have suggestions for improve it, I’d be very grateful ūüôā

The ant file consists of some targets and macro definitions.

The main macro definition is the one invoking the p2 ant task:

Note that we’ll also create a p2.index file. I prefer not to compress the¬†compositeArtifacts.xml and compositeContent.xml files for easier inspection or manual modification, but you can compress them setting the “compressed” to “true” property above.

This macro will be called twice in the main task

First of all, this task will copy the p2 repository created during the build in the correct place inside the nested p2 composite repository.

Then, it¬†will create or update the composite site¬†for the nested repository major.minor, and then it will¬†create or update the composite site for the main site (the one storing all the versions). The good thing about these ant tasks is that if you add a child location that already exists they won’t complain (though you can set a property to make them fail in such situations); this is crucial for updating the main repository, since most of the time you will not release a new major.minor.

This target calls (i.e., depends on) another target to compute the properties to pass to the macrodef, according to the information passed from the pom.xml

Default properties (that can be modified by passing a value from the pom.xml file):

  • the absolute path of the parent folder for¬†the composite p2 site (default is “p2.repositories” in your home directory)
  • updates.dir: the relative path of the composite p2 site (default is “updates”); this is relative to¬†

Thus, by default, the main p2 composite update site will end in ${user.home}/p2.repositories/updates. As hinted in the beginning, this can be any absolute local file system path; in EMF-Parsley Eclipse, since we release from Hudson, it will be the path served by the Eclipse we server So we specify the two above properties accordingly.

These are the properties that must be passed from the pom.xml file

  • site.label: the main label that will appear in the composite site (and that will be recorded in the “Eclipse available sites”). The final label will be “${site.label} All Versions” for the main site and¬†“${site.label} <major.minor>” for the nested composite sites.
  • the location of the p2 repository created during the build (usually of the shape <>/target/repository)
  • unqualifiedVersion: the version without qualifier (e.g., 1.1.0)
  • buildQualifier: the replaced qualifier in the built version

Note that except for the first property, the other ones have exactly the same name as the ones in Tycho (and are set by Tycho directly during the build, so we’ll reuse them).

The ant file will use an additional target (not shown here, but you’ll find it in the sources of the example) to extract the major.minor part of the passed version.

Calling the ant task from pom.xml

Now, we only need to execute the above ant task from the pom.xml file of the eclipse-repository project,

ATTENTION: in the following snipped, for the sake of readability, I split the <appArgLine> into several lines, but in your pom.xml it must be exactly in one (long) line.

As I said, you should pass site.label as you see fit (for the other properties you can use the default).

You may want to put this plugin specification inside a Maven profile, that you activate only when you are actually doing a release (see, e.g., what we do in this pom.xml, taken from our EMF-Parsley Eclipse project).

Try the example

Let’s simulate some releases:

To see what you get, just clone the repository found here, cd to p2composite.example.tycho and run

After Maven finished downloading all the dependencies you should see something like

And here’s the directory layout of your ${user.home}/p2.repositories

p2composite2Run the command again, and you’ll get another child in the nested composite repository 1.0 (the qualifier has been replaced automatically with the new timestamp):

p2composite3Let’s increase the service number, i.e., 1.0.1, (using the tycho-versions-plugin) and rebuild:

and the new child will still be in 1.0 folder:

p2composite4Let’s increase the minor number, i.e., 1.1.0 and rebuild

and you’ll get another major.minor child repository

p2composite5Let’s increase the major number, i.e., 2.0.0

and you’ll get another major.minorp2composite6and so on ūüôā

With Buckminster

As I hinted before, with Buckminster you can directly call the p2 ant tasks, since they are included in the Buckminster headless product. You will only need to add custom actions in the .cspec (or in the .cspex if you’re inside a plugin or feature project) that call the ant task passing the right properties. An example can be found here. This refers to a slightly different ant file from the one shown in this blog post, but the idea is still the same.

Possible Improvements

You may want to add another nesting level, e.g., major -> major.minor etc… This should be straightforward: you just need to call the macrodef another time, and compute the main update site directory differently.

Hope this helps.



Be Sociable, Share!

Analyzing Xtend code with Sonarqube

I recently started to play with Sonarqube to reduce ‚Äútechnical debt‚ÄĚ and hopefully improve code quality (see my previous post). I‚Äôd like to report on my experiences about using Sonarqube to analyze¬†Xtend¬†code.

Xtend compiles into Java source code, so it looks like it is trivial to analyze it with Sonarqube; of course, Sonarqube will analyze the generated Java code, but it’s rather easy to refer to the original Xtend code,¬†since Xtend generates clean Java code¬†ūüôā

However, we Sonarqube 4.4 it looks like it’s harder than I thought due to some facts:

My starting point was another issue: test results did not show in the Sonarqube 4.4 web interface, and that was because test detection has changed in version 4 (

I created an example to reproduce the problem and propose a solution:

In the parent project we specify the actual project with sources to be analyzed, and the project containing tests (in this example I also use jacoco for code coverage, but that’s not crucial for this example):

And we enable all the Maven plug-ins for

The plugin and the plugin.tests projects intentionally contain Xtend and Java files with some Findbugs issues, e.g.,

Now, assuming you have Sonarqube 4.4 running on your machine, you can run the typical maven commands to analyze your code (make sure you set the MaxPermSize in the MAVEN_OPTS otherwise the Xtend compiler will run out of memory):

If you go to Sonarqube web interface you see

sonarqube xtend 1So you see that Sonarqube correctly detected Findbugs issues in all the Java files, but for the Java code generated by Xtend, it only detected the issues in the plugin.tests project, not on the plugin project (as explained here, Sonarqube does “not take into consideration this suppress warnings annotation in test files”).

To deal with this problem, I created an ant file which basically removes all the¬†@SuppressWarnings(“all”) annotations in all the generated Java files in the xtend-gen folder:

and I created a Maven profile in the parent pom that, when activated, invokes the ant target, in the process-sources phase (recall that this phase is executed after generate-sources phase, when the Xtend files are compiled into Java code)

Now, let’s invoke the two maven commands, but this time, the first one activates the above profile

OK, let’s go to the “Issues Drilldown” in the Sonarqube web interface and this time the issues are detected also in the plugin project:

sonarqube xtend 2You may want to select “Since previous analysis” in the combo box, to make sure that this analysis detected these new issues:

sonarqube xtend 3

Hope this helps! ūüôā

The source code can be found here:

Be Sociable, Share!

Dealing with Technical Debt with Sonarqube: a case study with Xsemantics

I recently started to play with Sonarqube to reduce “technical debt” and hopefully improve code quality. I’d like to report on my experiences about using Sonarqube to analyze Xsemantics, a DSL for writing rule systems (e.g., type systems) for Xtext languages.

I was already using the Jenkins Continuous Integration server, and while building I was already using Findbugs and Jacoco, thus, I was already analyzing such software, but Sonarqube brings new analysis rules for Java programs and it also integrates results from Findbugs and Jacoco, aggregating all the code quality results in a web site.

In spite of the Jenkins builds Sonarqube detected some issues when I started

xsemantics sonarqube 1

First of all, I had to exclude the src-gen and emf-gen directories (the former is where Xtext generates all its artifacts, and the latter is where Xcore generates the EMF model files); since these are generated files and I did not want to make them part of the analysis. I’ve done such exclusion with a property in the main pom.xml (for readability I split it into lines):

Note that for the moment I’m also excluding tests from the analysis… it is considered best practice to analyse tests as well (and I have many of them), but I wanted to concentrate on the code first. I also excluded other Java files for which issues are reported, like the Xtext Guice modules, due to the wildcards in the method signatures… I have to live with them anyway ūüôā

After that the number of issues reduced a little bit, but there were still some issues to fix; most of them were easy, basically due to Java conventions I hadn’t use (e.g., name of fields and methods or even names of type parameters).

One of the major ones was due to the wrong implementation of the clone method (“super.clone() should be called when overriding Object.clone()” (

Another thing that I had never considered was dependency cycles among Java packages and files. Sonarqube reports them. Luckily there were only few of them in Xsemantics, and the hardest part was to read the Dependency Structure Matrix, but in the end I managed to remove them (there must be nothing in the upper triangle to have no cycle):

xsemantics sonarqube 2

To solve the cycles I had to change something in the runtime API ( but it was basically a matter of moving Java classes into different packages.

Then came the last major issue: Duplicated Code!!! All by itself this issue was estimated with 13 days of technical debt! And most of the duplicated code was in the model inferrer (a concept from Xbase). Moreover, such inferrer is written in Xtend, a cleaner Java, and the Xtend compiler then generates Java code. Thus, Sonarqube analyses the generated Java code, and the detected duplicate code blocks are on the Java code. This means that it takes some time to understand the corresponding original Xtend code. That’s not impossible since Xtend generates clean Java code, but it surely adds some work ūüôā

Before starting to remove duplicated code (around 80 blocks in the generated Java code) the Xtend inferrer was around 1090 lines long (many parts are related to string templates for code generation) corresponding to around 2500 lines of generated Java code! After the refactoring the Xtend inferrer was around 1045 lines long, and the generated Java code reduced to around 2000 lines.

That explains also the reduction of lines of code and complexity:

xsemantics sonarqube 3

But now technical debt is 0 ūüôā

xsemantics sonarqube 4

And it’s nice to look at this dashboard ūüôā

xsemantics sonarqube 5

By the way, I also had to disable some issues I did not agree on (tabulation characters) and avoid reported issues on method name conventions on a specific file (because methods that start with the underline characters _ have a specific meaning in Xtext/Xtend). Instead of disabling them on the Sonarqube web interface, I preferred to disable them using properties in the pom file so that it works across different Sonarqube installations (e.g., I also have a local Sonarqube instance on my machine to do some quick experiments). Such multi properties are not officially supported in the Sonar invocation (e.g., through the sonar runner or via Maven), but I found a workaround: (but, be careful, it is considered a hack as reported in the mailing list:

That’s all! I strongly suggest to give Sonarqube a try! ūüôā

Be Sociable, Share!

Installing Linux Kubuntu on a Dell Precision M3800

Dell-m3800I recently had to install Linux¬†Kubuntu 13.10 Saucy Salamander (at the time of writing I’ve already upgraded it to 14.04 Trusty Tahr) on a Dell Precision M3800 (a really cool and powerful laptop, see the details here).

The installation went really smooth, and I’m enjoying a very fast and stable Linux OS on this laptop.

In this blog post I’ll detail only a few tips and further tweaks after the installation.

As for the initial setup (Hard disk resize, Backup and UEFI Boot issues) I followed this really nice detailed guide,, and I strongly suggest to do the same, especially if you have the same laptop.

Tweaks after installation

Here some tweaks after the installation.

Adjust Screen Resolution

This laptop comes with the “crazy” resolution of 3200×1800! Unfortunately, this is barely usable at least in my experience: everything is so small that I can’t read almost anything… adjusting the DPI as suggested here really did not help: the fonts, window border become readable and usable, but the system looks ugly… (by the way, the same problem holds in Windows 8, at least for my everyday program, i.e., Eclipse: most fonts and icons are not readable)… until these resolution problems are fixed in Kubuntu (and in some applications as Eclipse), I reverted the resolution to something smaller (and still the resolution is high :), that is 1920×1080.


Enable Hibernate

First check that hibernate actually works by running (remember that your swap partition is at least as large as your available RAM):

After you computer turns off, try and switch it back on. If your open applications re-open you can re-enable hibernate: run below command to edit the config file:

Copy and paste below lines into the file and save it.

Enable Scheduled Trim

First of all, make sure you enable the anotime option for your SSD partition in /etc/fstab to avoid further writings to your SSD disk.

As reported here,, scheduled trim seems to be the preferred way to keep your SSD performant.

Run the following command to create and edit the file in cron.daily

And copy and paste this:

Then make the file executable:

Power optimizations

To keep power consumption low, install the following tools

then TLP:

Also run powertop when you’re on battery to check for further optimizations.

Install Bumblebee, as detailed here:

The problem with Fn keys

At first, I thought that Function keys were not working at all… then I discovered that on new laptops like this one F-keys are default to their media mode¬†(!).¬†You can change the default behavior of the F keys in the BIOS, but I prefer the¬†F-Lock icon on the Esc button: this will take them back to their standard behavior.

Be Sociable, Share!

Switching to Xcore in your Xtext language

This is a followup of my previous post, Switching from an inferred Ecore model to an imported one in your Xtext grammar. The rationale for switching to manually maintained metamodel can be found in the previous post. In this post, instead of using an Ecore file, we will use Xcore,

Xcore is an extended concrete syntax for Ecore that, in combination with Xbase, transforms it into a fully fledged programming language with high quality tools reminiscent of the Java Development Tools. You can use it not only to specify the structure of your model, but also the behavior of your operations and derived features as well as the conversion logic of your data types. It eliminates the dividing line between modeling and programming, combining the advantages of each.

I took inspiration from¬†Jan K√∂hnlein’s blog post; after switching to a manually maintained Ecore in¬†Xsemantics, I felt the need to further switch to Xcore, since I had started to write many operation implementations in the metamodel, and while you can do that in Ecore, using Xcore is much easier ūüôā Thus in my case I was starting from an existing language, not to mention the use of Xbase (not covered in Jan’s post). Things were not easy, but once the procedure works, it is easily reproducible, and I’ll detail this for a smaller example.

So first of all, let’s create an Xtext project, org.xtext.example.hellocustomxcore, (you can find the sources of this example online at; the grammar of the DSL is not important: this is just an example. We will first start developing the DSL using the automatic Ecore model inference and later we will switch to Xcore.

(the language is basically the same of the previous post).

The grammar of this example is as follows: