MDE and the Software Development Process

SDMs have had a important role to play in the development of software systems from as early as the 1960's (Elliott 2004) (p. 86). According to Ramsin and Paige, "[a] SDM is a framework for applying software engineering practices with the specific aim of providing the necessary means for developing software-intensive systems." (Ramsin and Paige 2008) For their part, Avison and Fitzgerald see it as "a recommended collection of phases, procedures, rules, techniques, tools, documentation, management, and training used to develop a system." (Avison and Fitzgerald 2003) Based on their analysis of earlier work, including that of OMG (- OMG 2017), as well as Avison and Fitzgerald themselves (Avison and Fitzgerald 2003), Ramsin and Paige decomposed an SDM into three main parts:

• Philosophy: a set of assumptions and beliefs made by the authors of the methodology, as part of its motivation or with the intent of supporting and rationalising it.
• Modeling conventions: a set of modeling conventions designed to work together as a modeling language. Within this document, we choose to interpret modeling language in the sense defined by Modeling Languages and Their Purposes — though perhaps Ramsin and Paige's intention is to use the term in a less formal manner.
• Processes: a set of processes that provide guidance as to the activities to take place, their order, the role played by different actors in those activities and ways in which to monitor and evaluate the results of performing them.

Within this framework, the modeling language contains the vocabulary with which to model aspects of the development of the system, the processes cater for the temporal characteristics of how entities interact — i.e. who performs what activities when — and the philosophy supplies a overarching narrative that harmonises and justifies both. These three components come together as a methodology, providing a unified perspective from which to accomplish prescribed objectives.² Therefore, a SDM can be thought of as an abstraction that orchestrates activities related to the development of software systems; and does so with the objective of making them easier to understand for all involved.

It is difficult to overestimate the importance of SDMs in modern software development. Ramsin and Paige (Ramsin and Paige 2008) go as far as considering them an enabler of the software engineering discipline itself (emphasis ours):

Software development methodologies are therefore considered an integral part of the software engineering discipline, since methodologies provide the means for timely and orderly execution of the various finer-grained techniques and methods of software engineering.

Whilst SDMs exact importance may be argued, what should nonetheless be clear from this exposition is that their organisational properties are simultaneously orthogonal and complementary to the characteristics of MDE described thus far. It is for this reason that we sided earlier on with those that argue that MDE by itself cannot be considered a methodology, but methodologies can be developed with model-driven characteristics for particular purposes. Asadi and Ramsin (Asadi and Ramsin 2008) tackled this subject in an MDA context, but presented an argument which, in our opinion, is directly applicable to MDE (emphasis ours):

[…] MDA is not a methodology, but rather an approach to software development. This fact forces organizations willing to adopt the MDA to either transform their software development methodologies into Model-Driven Development (MDD) methodologies, or use new methodologies that utilize MDA principles and tools towards the realization of MDA standards.

It is for this reason that one finds numerous examples of model-driven methodologies in the literature, with differing characteristics (Matinnejad 2011) (Ambler 2008) (Ambler 2007) (Guta, Schreiner, and Draheim 2009) (Estefan and others 2007). However, rather than delve into the specifics of each of these, the more pertinent question — at least for our purposes — is to determine whether MDE demands a particular type of software development methodology or, conversely, has no sensitivity to it.

The literature reveals a range of views with regards to the relationship between MDE and SDM. Ambler appears to suggest MDE is more amenable to a structured environment, declaring that (emphasis ours) "MDD is an approach to software development where extensive models are created before source code is written. With traditional MDD a serial approach to development is often taken where comprehensive models are created early in the lifecycle." (Ambler 2008) Stahl et al. are instead of the view that MDE requires, by its very nature, an iterative process:

The iterative, dual-track process of MDSD […], in which the infrastructure is developed in parallel to the application(s), must be clearly distinguished from traditional waterfall methods that are based on a 'big design up-front' philosophy. (Völter et al. 2013) (p. 375)

Unfortunately, the disagreement may stem, at least in part, from imprecise definitions rather than due to profound ideological differences.

A third view, and one which we align ourselves with, is that of Brambilia et al., who declare MDE to be process-agnostic, claiming "it neither provides nor enforces any specific development process but it can be integrated in any of them." (Brambilla, Cabot, and Wimmer 2012) (p. 53) To be clear, Brambilia et al. are not suggesting that MDE does not have a significant impact on a project's software development methodology and vice-versa. Instead, their argument is that the principles in the MDE body of knowledge are compatible with all SDM — "traditional development processes" in their parlance — and the onus is therefore on the MDE practitioner to unify them into a whole, for any particular application.

These integration efforts are not insignificant, so the literature has been active in developing specific strategies — especially for iterative methodologies (Matinnejad 2011) (Ambler 2008) (Ambler 2007) (Guta, Schreiner, and Draheim 2009). It is also important to understand that this stance does not have any implications with regards to the merits or demerits of structured versus iterative development methodologies, for the latter should be analysed independently of MDE. However, since models "are considered equal to code" (Völter et al. 2013) and, somewhat more arguably, since "programming is a modeling activity" (Madsen and Møller-Pedersen 2010) (cf. Modeling Languages and Programming Languages), it is not surprising that many of the engineering practices which foster the development of high-quality code are equally desirable when the engineering is driven by modeling.

It is then for these reasons that iterative methodologies should be preferred to structured methodologies, rather than due to any intrinsic property or requirement of MDE. The idea can be neatly summarised with the following dictum: "what is good for code is (generally) also good for models". Nonetheless, regardless of whether the approach is iterative or structured, there are specific factors related to model-driven software engineering that must be catered for, as the next section will explain.

One of the most striking differences between traditional software development and the model-driven approach is the additional work required to develop the modeling infrastructure — that which Stahl et al. call the domain architecture (Völter et al. 2013) (p. 253); i.e. the set of modeling languages and their associated MTs, as well as the platforms upon which they depend on, in order to translate a set of instance models into a software system. The development of the domain architecture poses a challenge which straddles theory and application because there is a circular dependency between exploring problem and solution spaces and creating the vocabulary with which to perform that exploration.

Stahl et al. tackled the issue by proposing a two-track development process, composed of the following threads:³,⁴

• Domain Architecture Development Thread: Responsible for developing all of the modeling infrastructure that makes up the domain architecture. Abstracts and generalises the requirements produced by the application thread into infrastructure that can be reused for a number of similar applications.
• Application Development Thread: Concerned with producing a concrete product to satisfy the requirements of end users. Provides exemplars of needed functionality to the domain architecture development thread to help shape its direction and consumes the tooling it produces to implement the product.

The relationship between these two threads is illustrated in Figure 1. Two-track development is very useful as an idealised model because the approach evokes a clear separation of concerns. Nonetheless, our own personal experience has taught us that matters are not quite as clear cut in practice. On one hand, the basic principles are very easy to convey to experienced engineering teams — even where MDE knowledge is lacking. On the other hand, many dangers lie in wait of a naïve application:

• the before-mentioned circular nature of the problem is extremely challenging — and never more so than during the initial stages of development;
• the development of the domain architecture ultimately demands a multidisciplinary skill-set, entailing both good software engineering skills as well as a mastery of the MDE cannon;
• there is great difficulty in demonstrating to management the need for continued investment in domain architecture development and maintenance as the project reaches maturity, and the lack of investment has severe consequences for the long term;
• there is great difficulty in avoiding a disconnect between domain and architecture development teams, and there are far-reaching repercussions when such a disconnect occurs.

The last factor is of great importance because it may lead to a phenomena we named problem domain decoupling, and which happens as the disconnect grows in size. In our opinion, this problem manifests itself more evidently with the application of AC-MDSD, but we do not believe it is solely limited to this use case. Figure 2 illustrates the issue by looking at four hypothetical scenarios covering the application of MDE to infrastructural code, which we shall now enunciate.

Scenario 1 typically happens at the first brush with MDE, where developers create a solution hard-wired to the core problem domain they are exploring, and is closely related to our own personal experiences (Craveiro 2021). From Scenario 2 to Scenario 4 there is a quest for generalisation, accompanied by a corresponding growth in scope of the infrastructure problem domain. Scenario 2 represents a small decoupling of the infrastructure domain to make it useful to more than one product, though still fairly hard-wired. With Scenario 3 we are now looking at providing infrastructure for a larger grouping of software products and their diverse needs, with a resulting ballooning in infrastructural scope. Finally, with Scenario 4, the infrastructure domain becomes a product on its own right, much larger than any one core problem domain; at this stage we are now considering products supplied by external vendors rather than in-house development.

The quantitative change in the size of the infrastructure problem domain produces qualitative changes that may not be readily apparent to engineers, as they develop a system with a dual-track approach. This phase transition is particularly problematic as one transitions from Scenario 1 through to Scenario 3. Let us perform a comparison between these two scenarios at the SDM level to better understand the problem. Figure 3 illustrates the state of the world for Scenario 1 via a two-track approach; the bold arched arrows represent the understanding of the problem as it materialises onto the domain and application development tracks and the dashed line represents the synchronisation points between the two tracks. Though not obvious, the most noteworthy aspect of this diagram is the natural alignment between the two tracks and the problem domain.

This property is made clearer by performing a similar exercise for Scenario 3, as does Figure 4. Even without a detailed analysis, it should be noticeable that the picture becomes considerably more complex; the previous natural alignment now gives way to a far more intricate set of relationships — many of which bidirectional. The figure depicts the separation between the infrastructure problem domain and the core problem domain, and the effect each of these have on each other as the exploration of both domains takes place. In hindsight, the increase in complexity should not be surprising because, as the scope of the infrastructural domain grows, it becomes a software product in its own right. Thus, there is an attempt to simultaneously engineer two tightly interlocked software products, each already a non-trivial entity to start off with.

At this juncture one may consider the ideal solution to be the use of vendor products as a way to insulate the problem domains. Unfortunately, experimental evidence emphatically says otherwise, revealing that isolation may be necessary but only up to a point, beyond which it starts to become detrimental. We name this problem over-generalisation. In our opinion, what is lacking is the deep synchronisation needed between the two development tracks — an ingredient that Stahl et al. had already identified as being crucial to the success of the entire endeavour. On the other hand, adoption literature shows that this close collaboration can be found in abundance within in-house development, but there we suffer from the opposite problem of under-generalisation. That is, solutions are too specific, catering only for internal use cases.

What is called for is a highly cooperative relationship between infrastructure developers and end-users, in order to foster feature suitability — a relationship which is not directly aligned with traditional customer-supplier roles; but one which must also maintain a clear separation of roles and responsibilities — not the strong point of relationships between internal teams within a single organisation, striving towards a fixed goal. Any proposed approach must therefore aim to establish an adequate level of generalisation by mediating between these actors and their diverse and often conflicting agendas. We named this generalisation sweet-spot barely general enough, following on from Ambler's footsteps (Ambler 2007)⁵, and created Figure 5 to place the dilemma in diagrammatic form.

And now that the need for the generalisation has been made clear, we must change our focus towards the machinery needed to implement it. and, at this juncture, the management of variability takes centre stage (cf. MDE and Variability Modeling).