Category: Code generation
Can I break up my set of proxy types and put them into different libraries?
The answer is yes, but you have to really know what you're doing! Breaking up a set of proxy types is easily one of the most sophisticated features in the code generator and frankly, you're advised to stay away from this feature if possible. Not because it doesn't work, but because it is pretty complicated and harder to understand than any other code generator feature.
There are different reasons that this question comes up and sometimes, the reason is a simple misunderstanding about integration requirements. Let's clear up some common misunderstandings first:
- Just because your Java API ships in five jar files does not mean that your C++ API has to ship in five DLLs. You can and should combine all proxy types in one session.
- Just because you would prefer five small DLLs to one large DLL does not mean that you have a good reason to break things up.
There are legitimate reasons for wishing to break up the generated types into different modules. Usually, the reasons are related to development workflow or OEM integration requirements:
- You publish pre-packaged core integration features in the form of a library, but your users might want to generate additional proxy types.
- Your project is split into several development teams, each of which owns a specific piece of Java functionality. Each team wants to generate its own proxies but in the end, they should all be usable together in one application.
- You have a core piece of functionality and several optional pieces of functionality. You don't want the core users to have to carry all the optional pieces around in their applications.
You should try to avoid breaking up your proxy type set if at all possible. The code generator works best and with the least complication if it can generate all desired proxy types in one session (you can of course always regenerate all proxy types in a later session). If you cannot avoid breaking up the types, you have to read this section very carefully and familiarize yourself with the issues you will be facing.
The core problem
The core problem with breaking up your proxy set is relatively easy to decribe but relatively hard to solve. In Figure 1, the two sets represent the output of two different code generation sessions. Each session was totally unaware of the other session. Each session resulted in a set of proxy types that represent a selfcontained and self-consistent type system. You can build the output of each session into a DLL or shared object and use it from an executable without any problems.
The problems start when you try to combine the two libraries in one application. This is due to the overlap in generated types, the purple types in the intersection of the two sets. These "overlapping" types cause problems in two different ways:
- The same type might not have been generated identically by both sessions.
A type only contains a method or a field if the necessary referenced types are also generated. If session #1 generated a referenced type and session #2 did not, session #1's version of the type will contain the field or method declaration and session #2's version will not.
- You cannot link with both libraries because they contain duplicate symbols.
Let's make it perfectly clear that you have absolutely no problem breaking up your set if you have no overlap and if the types don't have circular dependencies between the sets. This can be the case with a very basic type set (built-in Java types) and a highly specialized, custom API. In such a case, the custom API will have dependencies on the basic set, but not vice versa.
Your first attempt at solving the problem will probably involve disabling the "overlap" types in session #2. This would yield the picture from Figure 2.
This picture looks perfectly acceptable at the type level, but it suffers from some severe problems: remember that the code generator never generates code that references disabled types. In session #2, you disabled some very basic types. As a consequence, none of session #2's proxy types will have any methods or fields that use a String or an Object, and the most basic Java exception type will also be missing. If you look at session #2's generated types in detail, you will find that many elements that you wanted to generate are missing. So how can you escape this dilemma?
The solution involves careful dependency analysis and multiple code generator sessions where later sessions are made aware of the output of earlier sessions.
Every code generation does not just generate source and project files but also a so-called set specification or "setspec" file. The setspec file is an XML file containing a list of all generated proxy types as well as the values of some select code generator settings. You can look at that setspec file, but more importantly, you can pass it as input to another code generator session!
When you pass one or more setspec files as input to a code generator session, the code generator parses them and maintains an internal record of the contained information. It is now aware of the types that have already been generated and, if you used the generated project files to build DLLs, it even knows the names of the libraries in which these types can be found!
WIth this knowledge, the code generator can be smart and treat "overlap" types specially. You keep the problematic types enabled in your session #2. Consequently, without a setspec file, you would end up with type duplicates. With a setspec file, the code generator simply supresses the generation of the problematic types without disabling them. This means that it still treats these types as if they were generating, it just doesn't really generate them. Instead, it adds a reference to the library that contains the "missing," not generated types.
Let's assume that the proxy types for session #1 and session #2 are specified in modelfiles called session1.cmm and session2.cmm. You can generate correctly via:
jcpp session1.cmm jcpp -sets=session1_setspec.xml session2.cmm
You would have to use fully qualified path names and the name of the generated XML file might be different (based on project settings), but otherwise this snippet illustrates exactly how to approach such a problem.
What do you need to watch out for?
- Try to structure the sets hierachically. If at all possible, you don't want a derived set to introduce a dependent type that should really be part of a more basic set.
- It might be a good idea to generate all types in some basic Java packages (java.lang, java.util) as part of the first set.
- On Windows, it will be crucial to choose different names for the declspec macro. If a derived set depends on types in a basic set, it will need to __declspec(dllimport) these types while __declspec(dllexport)ing its own types so they can be used by an executable.
- On Windows, your "using" project (the executable that links with all DLLs containing proxy types) will have to __declspec(dllimport) all types. This typically means that you have to define multiple macros.