Tag Archives: xsemantics

Oomph setup for Xtext projects

In this blog post I’ll describe my experience in preparing an Oomph setup for a non-trivial Xtext project, Xsemantics.

This setup was kind of challenging because of the following features of my project, but I guess most of them can be found in any Xtext project:

  • generated sources are not stored in the Git repository (these include Xtend generated Java files and Java files generated during the MWE2 workflow)
  • the MWE2 workflow(s) must be run during the workspace setup (I have several DSLs in this project)
  • one of the DSL “inherits” from another DSL, so when running the MWE2 of the inheriting DSL the parent DSL must have already been built (i.e., Java classes must be compiled)

I hope this post can be useful for other Xtext developers.

This will not be a tutorial: it will be a collection of hints and procedures for preparing the final setup which can be found here: https://github.com/LorenzoBettini/xsemantics/blob/master/devtools/it.xsemantics.workspace/Xsemantics.setup.

By the way, Xsemantics setup is part of the official Oomph catalog, so you can try it yourself (it’s in the “Github projects” node).

This blog post assumes that you’re already familiar with Oomph and its authoring system.

The initial setup file can be created with the Oomph wizard, so I won’t talk about that.

Source folders in the repository

I found that it is better if all the source folders, including the source folders containing generated code, to be in the git repository. By “source folder” I mean a folder in an Eclipse project which is in the build path as a source folder. Thus, src-gen and xtend-gen should be in the git repository, but NOT their contents (at least, that’s what I want). Remember that git does not store empty folders, so you need to put a .gitignore in such folders stating to ignore everything but itself:

This way, when the containing projects will be imported in Eclipse you won’t risk the Java compiler to stop immediately because of a missing source folder.

Note that this does not seem to always be required: there are projects that can be built anyway, but I found it easier to always include them all.

If you put the .gitignore in more than one *-gen folder you’ll get a warning from Eclipse since it tries to copy those files to the bin folder and it would end up with duplicates. You can avoid this warning by setting the preference “Java Compiler” => “Building” => “Output folder” => “Filtered resources” as shown in the screenshot (I also avoid copying other files into the bin folder):

oomph-xsemantics1

Use platform URI in MWE2

You should change the grammarURI in your .mwe2 files: they should be platform URIs as opposed to classpath URIs. Otherwise, the MWE2 workflows will fail to find the Xtext grammars when run during the Oomph setup. An example is shown in the following screenshot

oomph-xsemantics2

Creating a “root” feature for Targlets task

This is not strictly related to Xtext. For the targlets task, in order to specify my own features and bundles, I prefer to specify one single feature which acts as a root for all my Eclipse projects that must be imported in the workspace and that participate to the targ(l)et platform via their requirements. Remember that Oomph will resolve dependencies transitively also for your projects.

To this aim, I define a feature project, e.g., it.xsemantics.workspace (which by the way also contains the Oomph setup file).

In this feature project I specify feature and bundle dependencies to all my other projects (using a feature project just makes the dependency specification easier) in the shape of included plug-ins and included features. Typically the included features are the installable features that you deploy to an update site, and the included plug-ins are the test projects (which are not part of installable features):

oomph-xsemantics3 oomph-xsemantics4

You only need to make sure that transitively these inclusions span all your project’s features and bundles.

However, this won’t help for projects that are neither plug-in projects nor feature projects, like, e.g., all releng projects. Of course you could use the “Project Import” task, but I prefer to create a new “Component Extension” file:oomph-xsemantics5

Here you can specify additional dependencies, in particular, using the type “Project” to refer to Eclipse projects which are not plug-in projects (nor feature projects):

oomph-xsemantics6

Now, when you define your “Targlets” you can refer to this root feature project, representing all your source projects. Then you can specify additional features for your target platform as usual:

oomph-xsemantics7

Use variables for Xtext versions

Since I want to have separate Eclipses and workspaces for developing Xsemantics against the current version of Xtext 2.8.4 and the development version 2.9.0 (taken from the nightly update sites), I find it very important to refer to Xtext update sites using Oomph variables (in my case xtext.site and mwe2.size):

oomph-xsemantics8

The values of such variables are defined in two separate Git branches specifications (you see I have variables also for API baseline settings, but I won’t talk about them since they’re not related to the aim of this post):

oomph-xsemantics9

I’ll use those variables also for the “P2 director” tasks; this will ensure that the Xtext plug-ins I have in Eclipse will be the same as the ones in the target platform:

oomph-xsemantics10

Running MWE2

This was the most challenging part: although Oomph provides a “Launch” task, running mwe2 workflows during the workspace setup has always been a problem (at least, that’s what I find in most places on the web).

First of all, you need to run the mwe2 launch AFTER the “Targlets” task and after a “Project Build” task

oomph-xsemantics11 oomph-xsemantics12

For the “Launch” task, you need to use the name of the .launch file, without .launch.

And here’s another small problem: of course the “Project build” task will leave the workspace full of error markers after the execution since the generated Java files are still not there; so the launch of the mwe2 workflow will make the famous popup dialog appear, asking whether you want to cancel the launch because of errors in the workspace… this is very annoying.

To avoid this, you can put a “Preference” task to always disable that dialog (you may want to renable that check later manually, after the workspace is provisioned):

oomph-xsemantics13Now the launch will start automatically without popup dialogs 🙂

By the way, don’t get fooled by the property name “cancel_launch…”; this actually corresponds to this preference “Continue launch…”:

oomph-xsemantics14Dealing with DSL dependencies

One of the Xsemantics DSL example “FJ cached” extends another DSL example “FJ”, thus, before running the MWE2 for “FJ cached” we must make sure that “FJ” has already been built, i.e., its MWE2 workflow has been executed and its Java sources have been compiled.

So we must insert another “Project Build” task at the right position:

oomph-xsemantics15

That’s all!

Now the whole setup procedure will run smoothly and at the end all the projects will be imported and will show no sign of error (not even a warning 😉

Other features

This setup also features API baseline setting, and Mylyn Github query.

You may want to try it yourself; as stated above, Xsemantics is part of the official Oomph catalog. The whole procedure might take a few minutes to conclude. During the procedure, as always, you might be asked a few passwords, depending on the choices you made before starting the setup.

Conclusions

Oomph is great great great! 🙂 Ed Merks and Eike Stepper really made a wonderful project 🙂

I now started to port all my Xtext projects to Oomph. By the way, if your Xtext project is simpler (i.e., no DSL dependencies) you may want to have a look at another example, Java–, which is also part of the official Oomph catalog.

Happy Oomphing! 😉

 

Be Sociable, Share!

Dealing with Technical Debt with Sonarqube: a case study with Xsemantics

I recently started to play with Sonarqube to reduce “technical debt” and hopefully improve code quality. I’d like to report on my experiences about using Sonarqube to analyze Xsemantics, a DSL for writing rule systems (e.g., type systems) for Xtext languages.

I was already using the Jenkins Continuous Integration server, and while building I was already using Findbugs and Jacoco, thus, I was already analyzing such software, but Sonarqube brings new analysis rules for Java programs and it also integrates results from Findbugs and Jacoco, aggregating all the code quality results in a web site.

In spite of the Jenkins builds Sonarqube detected some issues when I started

xsemantics sonarqube 1

First of all, I had to exclude the src-gen and emf-gen directories (the former is where Xtext generates all its artifacts, and the latter is where Xcore generates the EMF model files); since these are generated files and I did not want to make them part of the analysis. I’ve done such exclusion with a property in the main pom.xml (for readability I split it into lines):

Note that for the moment I’m also excluding tests from the analysis… it is considered best practice to analyse tests as well (and I have many of them), but I wanted to concentrate on the code first. I also excluded other Java files for which issues are reported, like the Xtext Guice modules, due to the wildcards in the method signatures… I have to live with them anyway 🙂

After that the number of issues reduced a little bit, but there were still some issues to fix; most of them were easy, basically due to Java conventions I hadn’t use (e.g., name of fields and methods or even names of type parameters).

One of the major ones was due to the wrong implementation of the clone method (“super.clone() should be called when overriding Object.clone()” (https://github.com/LorenzoBettini/xsemantics/issues/34).

Another thing that I had never considered was dependency cycles among Java packages and files. Sonarqube reports them. Luckily there were only few of them in Xsemantics, and the hardest part was to read the Dependency Structure Matrix, but in the end I managed to remove them (there must be nothing in the upper triangle to have no cycle):

xsemantics sonarqube 2

To solve the cycles I had to change something in the runtime API (http://xsemantics.sourceforge.net/snapshots-for-xsemantics-1-6-for-xtext-2-7/) but it was basically a matter of moving Java classes into different packages.

Then came the last major issue: Duplicated Code!!! All by itself this issue was estimated with 13 days of technical debt! And most of the duplicated code was in the model inferrer (a concept from Xbase). Moreover, such inferrer is written in Xtend, a cleaner Java, and the Xtend compiler then generates Java code. Thus, Sonarqube analyses the generated Java code, and the detected duplicate code blocks are on the Java code. This means that it takes some time to understand the corresponding original Xtend code. That’s not impossible since Xtend generates clean Java code, but it surely adds some work 🙂

Before starting to remove duplicated code (around 80 blocks in the generated Java code) the Xtend inferrer was around 1090 lines long (many parts are related to string templates for code generation) corresponding to around 2500 lines of generated Java code! After the refactoring the Xtend inferrer was around 1045 lines long, and the generated Java code reduced to around 2000 lines.

That explains also the reduction of lines of code and complexity:

xsemantics sonarqube 3

But now technical debt is 0 🙂

xsemantics sonarqube 4

And it’s nice to look at this dashboard 🙂

xsemantics sonarqube 5

By the way, I also had to disable some issues I did not agree on (tabulation characters) and avoid reported issues on method name conventions on a specific file (because methods that start with the underline characters _ have a specific meaning in Xtext/Xtend). Instead of disabling them on the Sonarqube web interface, I preferred to disable them using properties in the pom file so that it works across different Sonarqube installations (e.g., I also have a local Sonarqube instance on my machine to do some quick experiments). Such multi properties are not officially supported in the Sonar invocation (e.g., through the sonar runner or via Maven), but I found a workaround: http://stackoverflow.com/questions/21825469/configure-sonar-sonar-issue-ignore-multicriteria-through-maven (but, be careful, it is considered a hack as reported in the mailing list: http://sonarqube.15.x6.nabble.com/sonar-issue-ignore-multicriteria-td5021722.html):

That’s all! I strongly suggest to give Sonarqube a try! 🙂

Be Sociable, Share!