Survey of Special Purpose Code Generators

ODB is designed to be called as part of the build process in a fashion similar to C++ compilers. It makes very few requirements of the build system, other than the ability to call external programs. Typically, each invocation of the tool contains one or more target header files which are decorated with ODB pragmas, as exemplified in Listing 1.

#include <string>

#pragma db object
class person {
public:
    person() {}

public:
    #pragma db id
    std::string name_;
    unsigned int age_;
};

Users are expected to generate build system rules for each file that requires mappings, as well as rules to compile the generated code into object files. They must also install the ODB supporting libraries, and configure the build system to locate and link the generated code against them.

ODB defines two internal DSLs, hosted within the C++ programming language. The first is the ODB Pragma Language, as demonstrated in Listing 1. Pragma directives are an extensibility mechanism for the C and C++ languages, and are often used to control implementation specific behaviours of compilers. ODB makes use of it to define ORM related constructs.

According to the manual, the ODB Pragma Language

[…] is used to communicate various properties of persistent classes to the ODB compiler by means of special #pragma directives embedded in the C++ header files. It controls aspects of the object-relational mapping such as names of tables and columns that are used for persistent classes and their members or mapping between C++ types and database types. (Synthesis 2018a)

The second DSL is the ODB Query Language, described as

[…] an object-oriented database query language that can be used to search for objects matching certain criteria. It is modeled after and is integrated into C++ allowing you to write expressive and safe queries that look and feel like ordinary C++." (Synthesis 2018a)

ODB offers variability support at two levels:

• Global: Invocations of the ODB tool can inline all command line parameters or instead supply an external text file with the configuration. These parameters will affect all applicable entities.
• Local: In the source code, each mapped entity can be annotated with pragmas that configure code generation.

When combined, these result in a large configuration surface to control ODB's behaviour. Parameters can be grouped into the following broad categories:

• Customisation of Relational Entities: Supply or override names (database name, schema name, index name, table name and so forth), add a prefix or post-fix to relational names, etc.
• Mapping Customisation: Manually override the default mappings of C++ types to SQL types, or supply a different mapping profile; users can choose a profile that is most suitable for their C++ programming environment — e.g. standard C++, Boost or Qt.
• Customisation of Generated Code: Add user supplied epilogues and prologues, place generated code in a user-defined namespaces, change the extension and/or names of generated files, configure the version of the C++ standard, the export of symbols, definition of macros, omit the generation of some aspects — e.g. do not generate SQL insert statements, queries, etc.
• Database Specific Parameters: A number of parameters are specific to a given RDBMS, such as the client tool versions, warnings, etc.
• Tracing and Debugging: Provide debug information of the code generation process, stop generation if the size of generated code is greater than N lines of code, etc.

ODB's flexible approach to variability does not preclude a minimalist use case due to its judicious use of default values. The only mandatory parameters are local pragmas in source code to identify entities to map and global command line arguments to point to the target file.

ODB can be characterised across the following dimensions.

• Usability: Due to its command line interface mimicking a compiler, ODB has a very shallow learning curve for developers. In addition, by making use of internal DSL hosted within C++, it requires little learning for a typical C++ developer.
• Tooling Integration: By making very few demands of the build system and using C++ source code with few modifications as its input, ODB is able to integrate with any development environment and build system. Users need not change their setup in order to use ODB.
• Code Integration: ODB uses a forward-engineering approach, imposing a strict separation between handcrafted code and generated code. Generated code is not intended to be modified by its users; changes must be exclusively made to the handcrafted source code via the ODB Pragma Language followed by regeneration.
• Variability: ODB supports a high-degree of variability but requires very little configuration in order to produce code. This lowers the barrier of entry to new users.
• Dependencies: Generated code requires ODB specific libraries. Whilst producing smaller and simpler code, this also means having to install the libraries and configure the build system to find them, as well as adding dependencies to the deployment.
• Generated Code: Samples of the generated code produced by ODB were manually inspected and found to be of a standard comparable to the handcrafted code of the ODB libraries. This is very advantageous when debugging and troubleshooting problems. In addition, ODB offers a number of options dedicated to customisation of generated code, easing the integration into existing code bases.
• Error Reporting: Error messages are reported to the command line using the formatting defined by the GCC compiler. This is less convenient for users of other compilers — such as Microsoft Visual C++ — as their development environment may not able to interpret error messages.

Users define one or more structured data types in a text file, written in conformance with the Protocol Buffers Language (Google 2018b). Input files typically have the extension .proto. Listing 2 provides an example message.

syntax = "proto3";

message person {
  string name = 1;
  int32 age = 2;
}

Each invocation of the tool is made against one or more .proto files and must supply command line parameters to determine the set of programming languages to generate. Other than the ability of calling external binaries, the compiler makes very few demands from the build system — thus supporting all modern build systems.

Users are responsible for creating build system rules to transform the .proto files, as well as rules to compile the generated code into object files as required by the target language. However, the tool supports the automated generation of rules for make-like build systems. Finally, generated code depends on handcrafted libraries supplied by the Protocol Buffers project, so these must be installed and made visible to the build system.

As described previously, .proto files must conform to the Protocol Buffers Language (Google 2018b), currently at version 3. The language has a C-like syntax, and provides a set of constructs from the domain of message serialisation such as:

• Message definition;
• Ordering of fields in a message;
• Optional, mandatory and reserved fields;
• Primitive types with well-specified machine-level representation, independent of target platform.

The parsing and validation of .proto files is performed by protoc as part of the generation process.

Outside of the structural variability enabled by the creative construction nature of the Protocol Buffers Language, protoc has very limited support for variability. All of its parameters are global, and fall under the following categories:

• Output: Determines if an output language is enabled, the location for its files, whether to concatenate output files, user-created plugins to add support for additional programming languages, etc.
• Tracing and Debugging: Format of error messages, list available free fields, etc.

The protoc compiler can be characterised across the following dimensions.

• Usability: The Protocol Buffers DSL is very similar to typical programming language constructs and other IDLs (Interface Description Language) such as CORBA (Common Object Request Broker Architecture), which greatly facilitates learning. The Protocol Buffers compiler has a very simple command line interface, allowing users to generate code with minimal knowledge of the infrastructure.
• Tooling Integration: The compiler is designed to fit in the existing build systems and development environments, needing very little support in order to do so. It also behaves in a fashion similar to other development tools such as linkers and compilers, facilitating integration.
• Code Integration: protoc uses a forward-engineering approach, so users are not allowed to modify generated code. The generated code is expected to be integrated with the remaining code for the system via rules in the build system.
• Variability: The constrained variability approach taken by protoc reduces the learning curve, but as a consequence it is not possible to customise generated code to handle specific use cases such as adding epilogues or prologues, changing namespaces, etc.
• Dependencies: Generated code requires Protocol Buffers specific libraries. These help keep generated code small, but demand additional setup from the build system in terms of locating dependencies and additional artefacts to deploy.
• Generated Code: Upon inspection, we found that the quality of the generated code for C++ is not at the same level as the handcrafted code in supporting libraries. Nevertheless, the code is simple enough to enable users to debug it.
• Error Reporting: protoc provides the ability to report errors using either GCC or Microsoft's Visual Studio formats, thus integrating with two of the major development environments for C++. However, other programming languages may have different notations for the reporting of errors, and thus do not benefit from the same level of integration.

Whilst SWIG is able to parse C and C++ code directly, the recommended usage is to create a separate SWIG interface file that explicitly defines the API (Application Programming Interface) to export. This is done so as to avoid exporting types inadvertently and also to stop polluting general source code with SWIG annotations. Interface files typically have a .i or .swg extension and contain C/C++ code interspersed with SWIG interface commands, as exemplified in Listing 3.

%module people
%{
#include <string>
%}

class person {
public:
    person() {}

public:
    std::string name_;
    unsigned int age_;
};

Once defined, the interface files can be processed by the command line tool swig. Users can choose to generate wrappers for one or more languages by supplying command line arguments.

SWIG does not make any demands on the build tool, other than the ability to call external processes, so it integrates with most build systems. It is the responsibility of the user to create appropriate build system rules to generate the wrapper code and to build the shared objects that ultimately will be used in the target language.

The DSL used by SWIG in its interface files is based on the C pre-processor, itself a simple text processing language. The SWIG pre-processor adds its own set of commands, escaped with %. The main objective of the SWIG commands is to allow a fine grained control over the exported API.

The following is a sample of the available commands:

• %include: Includes a file into the interface. The original pre-processor #include is ignored to avoid including files into the API unnecessarily such as library headers and other third party code.
• %import: Includes a file to satisfy dependencies, but does not add its contents to the exported interface.
• %define, %inline, %enddef: Provides a more convenient interface for macro definition at the SWIG level.
• %extend: Extends an existing class interface with additional code.
• %typemap: Provides a way to override the default mapping of types.
• %module: Defines a containing module for the exported code. The notion of "module" is mapped to the adequate construct in the target language such as namespace in C# and package in Java.

The pre-processor commands have evolved over the years to cater for a large range of use cases in interoperability, and thus addresses the majority of requirements.

The SWIG DSL produces transformations on the original C and C++ source code, and thus it is a creative construction DSL focused on structural variability.

The remaining support for variability in the swig tool is very limited, and falls under the following categories:

• Input: Add support for C++ (only C is supported by default), change the behaviour of the C pre-processor, etc.
• Output: Configuration of the languages to generate, directories in which to output the files, etc.
• Tracing and Debugging: Dump information on the API to generate, dump symbol tables, dump type mapping, show code after pre-processing, set the warning level, etc.

SWIG can be characterised across the following dimensions.

• Usability: The swig tool itself requires a shallow learning curve, since it uses a command line interface similar to that of a compiler and has a small the number of configuration options — most of which are common to a compiler. However, the SWIG DSL does not share these properties. SWIG interface files — with its two-stage pre-processing pipeline and two sets of pre-processing commands — can become very large and complex and require developers that are knowledgeable about SWIG.
• Tooling Integration: The swig tool is designed to fit in the existing build systems and development environments by following a workflow similar to a compiler.
• Code Integration: SWIG uses a forward engineering approach, thus generated code is not modifiable. Users are expected to design build system rules to build and link the generated code in the same manner as for other handcrafted code.
• Variability: On one hand, the structural variability promoted by the SWIG DSL makes the tool highly configurable and able to handle a variety of very complex use cases. On the other hand, outside of simple scenarios, SWIG has a very steep learning curve due to this support for variability.
• Dependencies: Generated code does not have any third-party dependencies, which makes it easier to integrate.
• Generated Code: SWIG generates thousands of lines of C++ code even for trivial examples, making it difficult to understand. The authors of SWIG state this clearly in the generated code via the following comment: "This file is not intended to be easily readable and contains a number of coding conventions designed to improve portability and efficiency." Unsurprisingly, the quality of generated code is lower than handcrafted code, but it is well-structured and simple enough to make debugging possible.
• Error Reporting: The swig tool reports errors using the GCC output formatting, which makes integration with environments using GCC straightforward. However, it does not support Microsoft's Visual Studio format.

Users create XML schemas using their XML editing tool of choice. Once defined, the XML schema is supplied to the command line tool xsd-4.¹⁶ Listing 4 provides an example XML schema that can be used as input to XSD.

<?xml version="1.0"?>
<xs:schema xmlns:xs=
 "http://www.w3.org/2001/XMLSchema">
 <xs:complexType name="person">
  <xs:attribute
      name="name"
      type="xs:string"/>
  <xs:attribute
      name="age"
      type="xs:integer"/>
 </xs:complexType>
</xs:schema>

The command line tool generates the C++ classes and the XML mapping code; the user must then integrate the generated code into the build system by creating the required build system rules. However, if the build system is a variant of make, the tool can also be used to code generate the rules.

In addition, the generated code depends on handcrafted libraries, so the onus is on the user to install these and to make them visible to the build system.

XML is a mature, standardised language for describing structured data (W3C 2008). There are a variety of tools for editing and processing XML documents and schemas. Due to this, the DSL is completely decoupled from the XSD tool.

In addition to the structural variability enabled by XML schemas, the XSD tool has a number of parameters to configure the generation of code. These can be classified into the following broad categories:

• Backend: As discussed above, the type of XML processing to generate code for. Some of the options are only applicable to a specific backend.
• Output: Directory in which to place the output, whether to generate one file per type, etc.
• Mapping Customisation: Override the default type mapping between C++ types and XML types.
• Customisation of Generated Code: Which C++ standard to target, what character encoding to use, whether to inline functions, override filenames and extensions, modify include paths with regular expressions, header guards, etc.
• Build System: Options related to the generation of build system targets for generated code.
• Tracing and Debugging: Limit generation to a given number of lines of code, tracing of regular expressions, etc.

Whilst there are a large number of command line parameters, the XSD tool is able to generate code with very little configuration supplied, due to the use of defaults. The only mandatory parameter is the backend.

The XSD tool can be characterised across the following dimensions.

• Usability: The command line interface provided by the tool is similar to other command line tools such as compilers, thus lowering the learning curve for new users. In addition, users can use their XML editing tool of choice to create and validate the input XML schema.
• Tooling Integration: The tool integrates with any modern build system and development environment that supports calling external tools.
• Code Integration: The XSD tool uses a forward engineering approach, meaning that generated code should not be modified and is separated from handcrafted code. Changes are made to the XML schema and code is regenerated.
• Variability: Outside of the creative construction of XML schemas, the XSD tool supports a high-degree of variability which enables users to customise the generated code for their particular use case. However, due to defaulting, the tool requires very little customisation in order to generate code, resulting in a low barrier of entry for new users.
• Dependencies: By requiring the installation of dependencies in order to build the generated code, the XSD tool made the setup process more complex than it would have otherwise been without dependencies.
• Generated Code: The code generated by the XSD tool is of a standard comparable to the handcrafted code in their core libraries. It is well-commented and succinct, largely due to its reliance on external libraries.
• Error Reporting: Errors are reported using the GCC formatting, enabling an easy integration to environments which use this compiler. However, given that the input is in standard XML, users can ensure the document is valid via their XML tool of choice before code generation.