Finally, I found that
javax.activationwas being loaded by two modules differently (See http://frankkieviet.blogspot.no/2009/03/javalanglinkageerror-loader-constraint.html and http://stackoverflow.com/questions/4270950/compile-time-vs-run-time-dependency-java).
I couldn't exclude
javax.activationfrom any of my modules (tried to explicitly ignore it from my pom.xml files (Maven), but that didn't work).
In the end I discovered that felix automatically loads some packages on startup. This is specified in the
javax.activation;version=1.1.1(The version I wanted) there fixed it.
Now Felix loads the package, and my waring modules fetch it from there instead of trying to load it themselves.