SPLC '20: Proceedings of the 24th ACM Conference on Systems and Software Product Line: Volume A - Volume A

Full Citation in the ACM Digital Library

SESSION: Adoption and experiences

PAxSPL: a feature retrieval process for SPL reengineering

In this extended abstract, we discuss the Journal First summary of our work published in the Journal of Software: Practice and Experience (SPE) [1].

Promote-pl: a round-trip engineering process model for adopting and evolving product lines

Process models for software product-line engineering focus on proactive adoption scenarios---that is, building product-line platforms from scratch. They comprise the two phases domain engineering (building a product-line platform) and application engineering (building individual variants), each of which defines various development activities. Established more than two decades ago, these process models are still the de-facto standard for steering the engineering of platforms and variants. However, observations from industrial and open-source practice indicate that the separation between domain and application engineering, with their respective activities, does not fully reflect reality. For instance, organizations rarely build platforms from scratch, but start with developing individual variants that are re-engineered into a platform when the need arises. Organizations also appear to evolve platforms by evolving individual variants, and they use contemporary development activities aligned with technical advances. Recognizing this discrepancy, we present an updated process model for engineering software product lines. We employ a method for constructing process theories, building on recent literature as well as our experiences with industrial partners to identify development activities and the orders in which these are performed. Based on these activities, we synthesize and discuss the new process model, called promote-pl. Also, we explain its relation to modern software-engineering practices, such as continuous integration, model-driven engineering, or simulation testing. We hope that our work offers contemporary guidance for product-line engineers developing and evolving platforms, and inspires researchers to build novel methods and tools aligned with current practice.

Product line adoption in industry: an experience report from the railway domain

The software system controlling a train is typically deployed on various hardware architectures and must process various signals across those deployments. The increase of such customization scenarios and the needed adherence of the software to various safety standards in different application domains has led to the adoption of product line engineering within the railway domain. This paper explores the current state-of-practice of software product line development within a team developing industrial embedded software for a train propulsion control system. Evidence is collected using a focus group session with several engineers and through inspection of archival data. We report several benefits and challenges experienced during product line adoption and deployment. Furthermore, we identify and discuss improvement opportunities, focusing mainly on product line evolution and test automation.

Automated extraction of domain knowledge in practice: the case of feature extraction from requirements at danfoss

Software product line supports structured reuse of software artifacts in order to realize the maintenance and evolution of the typically large number of variants, which promotes the industrialization of software development, especially for software-intensive products. However, for a legacy system, it is non-trivial to gain information about commonalities and differences of the variants. Meanwhile, software requirements specifications as the initial artifacts can be used to achieve this information to generate a domain model. Unfortunately, manually analyzing these requirements is time-consuming and inefficient. To address this problem, we explored the usage of feature extraction techniques to automatically extract domain knowledge from requirements to assist domain engineers. In detail, we applied Doc2Vec and a clustering algorithm to process the requirements for achieving the initial feature tree. Moreover, we utilized key words/phrases extraction techniques to provide key information to domain engineers for further analyzing the extraction results. In particular, we developed a GUI to support the extraction process. The empirical evaluation indicates that most of the extracted features and terms are beneficial to improve the process of feature extraction.

Incremental migration to software product line engineering

Current market developments require organizations to produce high-quality tailored software variants while simultaneously coping with increased software complexity. Software Product Line Engineering (SPLE) is a well-known approach to manage this variability and complexity, however no step-wise migration process is available allowing a co-existence of SPLE along with established development processes. In this paper, we introduce an incremental SPLE migration strategy and process starting from using the feature model as synchronized product and variant documentation. They can be applied as a first step of SPLE migration along with the continuous software development cycle. We performed initial steps of the process on industrial low voltage drive embedded firmware spanning around few millions lines of code using a commercial SPLE tool and validated short-term benefits by means of stakeholder feedback.

Insights on software product line extraction processes: ArgoUML to ArgoUML-SPL revisited

Software Product Lines (SPLs) are rarely developed from scratch. Commonly, they emerge from monolithic architectures when there is a need to create tailored variants, or from existing variants created in an ad-hoc way once their separated maintenance and evolution become challenging. Despite the vast literature about re-engineering systems into SPLs and related technical approaches, there is a lack of detailed analysis about the process itself and the effort that is involved. We provide and analyze empirical data of an existing SPL extraction process: the ArgoUML monolithic architecture transition to ArgoUML-SPL. The analysis relies on information mined from the version control history of the source-code repository and the discussion with developers that took part in the process. The contribution of this study is an in-depth characterization of the process compared to previous works that focused only on the structural results of the final SPL. We made publicly available the dataset and the analysis scripts to be used as baseline for extractive SPL adoption research and practice.

SESSION: Feature models

Incremental feature model synthesis for clone-and-own software systems in MATLAB/Simulink

Families of related MATLAB/Simulink systems commonly emerge ad hoc using clone-and-own practices. Extractively migrating systems towards a software product line (SPL) can be a remedy. A feature model (FM) represents all potential configurations of an SPL, ideally, in non-technical domain terms. However, yielding a sensible FM from automated synthesis remains a major challenge due to domain knowledge being a prerequisite for features to be adequate abstractions. In incremental reverse engineering, subsequent generation of FMs may further overwrite changes and design decisions made during previous manual FM refinement.

In this paper, we propose an approach to largely automate the synthesis of a suitable FM from a set of cloned MATLAB/Simulink models as part of reverse engineering an SPL. We fully automate the extraction of an initial, i.e., a technical, FM that closely aligns with realization artifacts and their variability, and further provide operations to manually refine it to incorporate domain knowledge. Most importantly, we provide concepts to capture such operations and to replay them on a structurally different technical FM stemming from a subsequent reverse engineering increment that included further systems of the portfolio. We further provide an implementation and demonstrate the feasibility of our approach using two MATLAB/Simulink data sets from the automotive domain.

Consistency-preserving evolution planning on feature models

A software product line (SPL) enables large-scale reuse in a family of related software systems through configurable features. SPLs represent a long-term investment so that their ongoing evolution becomes paramount and requires careful planning. While existing approaches enable to create an evolution plan for an SPL on feature-model (FM) level, they assume the plan to be rigid and do not support retroactive changes. In this paper, we present a method that enables to create and retroactively adapt an FM evolution plan while preventing undesired impacts on its structural and logical consistency. This method is founded in structural operational semantics and linear temporal logic. We implement our method using rewriting logic, integrate it within an FM tool suite and perform an evaluation using a collection of existing FM evolution scenarios.

The benefits of a feature model in banking

This experience report describes the surprisingly beneficial introduction of feature modeling at KfW, a government promotional bank. On behalf of the government and based on promotional directives, KfW grants retail loans to small and medium enterprises, business founders, self-employed professionals, municipalities and private individuals. The promotional directives, called programs, define mandatory and optional properties of these loans. We have now successfully built a feature model from these properties.

Our feature model will be presented with its outstanding characteristic, which is an additional subtree containing the programs as features. Complete and correct cross-tree constraints will also allow us to analyze and scope the portfolio, reduce complexity, and speed-up time-to-market. This is the advent of product line development at KfW.

In order to standardize our portfolio, we have subsequently developed tools on top of the feature model, namely, a browser-based, multi-user configurator assisting non-technical-affine users in their product design, and a generator producing complete product documentation from the feature model and partial configurations. More applications are currently underway.

This is our story of applying Software Product Line Engineering in banking, a domain where it is unusual or even unknown. We share our ideas, analyses, progress, and findings where the results have been thrilling us for the past two years and will continue to do so.

Extensible and modular abstract syntax for feature modeling based on language constructs

Since the definition of feature models in 1990, a large number of language constructs have emerged. Each language construct usually comes with its own abstract and concrete syntax, its semantics, and even its complete language dialect and tool support. Nowadays, there is a consensus in the Software Product Line community about a need for defining a common variability modeling language. But the fact of the matter is that it is very complex to achieve a good compromise between how expressive the language should be and the effort of developing practical tools for a language with all possible language constructs. In this paper, we propose an extensible model-driven engineering approach for defining the abstract syntax of feature modeling language constructs that could be tailored to different needs and domains. We formalize our approach as a set of modular and reusable metamodels that allows practitioners to decide which subset of language constructs to use through: (1) generating a new variability language; and (2) managing feature models with different level of expressiveness. We provide an instantiation and implementation of our approach.

SESSION: Testing I

AMADEUS: towards the AutoMAteD secUrity teSting

The proper configuration of systems has become a fundamental factor to avoid cybersecurity risks. Thereby, the analysis of cybersecurity vulnerabilities is a mandatory task, but the number of vulnerabilities and system configurations that can be threatened is extremely high. In this paper, we propose a method that uses software product line techniques to analyse the vulnerable configuration of the systems. We propose a solution, entitled AMADEUS, to enable and support the automatic analysis and testing of cybersecurity vulnerabilities of configuration systems based on feature models. AMADEUS is a holistic solution that is able to automate the analysis of the specific infrastructures in the organisations, the existing vulnerabilities, and the possible configurations extracted from the vulnerability repositories. By using this information, AMADEUS generates automatically the feature models, that are used for reasoning capabilities to extract knowledge, such as to determine attack vectors with certain features. AMADEUS has been validated by demonstrating the capacities of feature models to support the threat scenario, in which a wide variety of vulnerabilities extracted from a real repository are involved. Furthermore, we open the door to new applications where software product line engineering and cybersecurity can be empowered.

Test them all, is it worth it?: assessing configuration sampling on the JHipster web development stack

This is an extended abstract of the article: Axel Halin, Alexandre Nuttinck, Mathieu Acher, Xavier Devroey, Gilles Perrouin, and Benoit Baudry. 2018. Test them all, is it worth it? Assessing configuration sampling on the JHipster Web development stack. In Empirical Software Engineering (17 Jul 2018). https://doi.org/10.1007/s10664-018-9635-4.

SESSION: Time, space, and more

Real-time-aware reconfiguration decisions for dynamic software product lines

Dynamic Software Product Lines (DSPL) have recently shown promising potentials as integrated engineering methodology for (self-)adaptive software systems. Based on the software-configuration principles of software product lines, DSPL additionally foster reconfiguration capabilities to continuously adapt software products to ever-changing environmental contexts. However, in most recent works concerned with finding near-optimal reconfiguration decisions, real-time aspects of reconfiguration processes are usually out of scope. In this paper, we present a model-based methodology for specifying and automatically analyzing real-time constraints of reconfiguration decisions in a feature-oriented and compositional way. Those real-time aware DSPL specifications are internally translated into timed automata, a well-founded formalism for real-time behaviors. This representation allows for formally reasoning about consistency and worst-case/best-case execution-time behaviors of sequences of reconfiguration decisions. The technique is implemented in a prototype tool and experimentally evaluated with respect to a set of case studies1.

Locating feature revisions in software systems evolving in space and time

Software companies encounter variability in space as variants of software systems need to be produced for different customers. At the same time, companies need to handle evolution in time because the customized variants need to be revised and kept up-to-date. This leads to a predicament in practice with many system variants significantly diverging from each other. Maintaining these variants consistently is difficult, as they diverge across space, i.e., different feature combinations, and over time, i.e., revisions of features. This work presents an automated feature revision location technique that traces feature revisions to their implementation. To assess the correctness of our technique, we used variants and revisions from three open source highly configurable software systems. In particular, we compared the original artifacts of the variants with the composed artifacts that were located by our technique. The results show that our technique can properly trace feature revisions to their implementation, reaching traces with 100% precision and 98% recall on average for the three analyzed subject systems, taking on average around 50 seconds for locating feature revisions per variant used as input.

A conceptual model for unifying variability in space and time

Software engineering faces the challenge of developing and maintaining systems that are highly variable in space (concurrent variations of the system at a single point in time) and time (sequential variations of the system due to its evolution). Recent research aims to address this need by managing variability in space and time simultaneously. However, such research often relies on nonuniform terminologies and a varying understanding of concepts, as it originates from different communities: software product-line engineering and software configuration management. These issues complicate the communication and comprehension of the concepts involved, impeding the development of techniques to unify variability in space and time. To tackle this problem, we performed an iterative, expert-driven analysis of existing tools to derive the first conceptual model that integrates and unifies terminologies and concepts of both dimensions of variability. In this paper, we present the unification process of concepts for variability in space and time, and the resulting conceptual model itself. We show that the conceptual model achieves high coverage and that its concepts are of appropriate granularity with respect to the tools for managing variability in space, time, or both that we considered. The conceptual model provides a well-defined, uniform terminology that empowers researchers and developers to compare their work, clarifies communication, and prevents redundant developments.

SESSION: SAT solvers and reasoning

A BDD for Linux?: the knowledge compilation challenge for variability

What is the number of valid configurations for Linux? How to generate uniform random samples for Linux? Can we create a binary decision diagram for Linux? It seems that the product-line community tries hard to answer such questions for Linux and other configurable systems. However, attempts are often not published due to the publication bias (i.e., unsuccessful attempts are not published). As a consequence, researchers keep trying by potentially spending redundant effort. The goal of this challenge is to guide research on these computationally complex problems and to foster the exchange between researchers and practitioners.

Uniform and scalable SAT-sampling for configurable systems

Several relevant analyses on configurable software systems remain intractable because they require examining vast and highly-constrained configuration spaces. Those analyses could be addressed through statistical inference, i.e., working with a much more tractable sample that later supports generalizing the results obtained to the entire configuration space. To make this possible, the laws of statistical inference impose an indispensable requirement: each member of the population must be equally likely to be included in the sample, i.e., the sampling process needs to be "uniform". Various SAT-samplers have been developed for generating uniform random samples at a reasonable computational cost. Unfortunately, there is a lack of experimental validation over large configuration models to show whether the samplers indeed produce genuine uniform samples or not. This paper (i) presents a new statistical test to verify to what extent samplers accomplish uniformity and (ii) reports the evaluation of four state-of-the-art samplers: Spur, QuickSampler, Unigen2, and Smarch. According to our experimental results, only Spur satisfies both scalability and uniformity.

Variational satisfiability solving

Incremental satisfiability (SAT) solving is an extension of classic SAT solving that allows users to efficiently solve a set of related SAT problems by identifying and exploiting shared terms. However, using incremental solvers effectively is hard since performance is sensitive to a problem's structure and the order sub-terms are fed to the solver, and the burden to track results is placed on the end user. For analyses that generate sets of related SAT problems, such as those in software product lines, incremental SAT solvers are either not used at all, used but not explicitly stated so in the literature, or used but suffer from the aforementioned usability problems. This paper translates the ordering problem to an encoding problem and automates the use of incremental SAT solving. We introduce variational SAT solving, which differs from incremental SAT solving by accepting all related problems as a single variational input and returning all results as a single variational output. Our central idea is to make explicit the operations of incremental SAT solving, thereby encoding differences between related SAT problems as local points of variation. Our approach automates the interaction with the incremental solver and enables methods to automatically optimize sharing of the input. To evaluate our methods we construct a prototype variational SAT solver and perform an empirical analysis on two real-world datasets that applied incremental solvers to software evolution scenarios. We show, assuming a variational input, that the prototype solver scales better for these problems than naive incremental solving while also removing the need to track individual results.

From features to roles

The detection of interactions is a challenging task present in almost all stages of software development. In feature-oriented system design, this task is mainly investigated for interactions of features within a single system, detected by their emergent behaviors. We propose a formalism to describe interactions in hierarchies of feature-oriented systems (hierarchical interactions) and the actual situations where features interact (active interplays). Based on the observation that such interactions are also crucial in role-based systems, we introduce a compositional modeling framework based on concepts and notions of roles, comprising role-based automata (RBAs). To describe RBAs, we present a modeling language that is close to the input language of the probabilistic model checker Prism. To exemplify the use of RBAs, we implemented a tool that translates RBA models into Prism and thus enables the formal analysis of functional and non-functional properties including system dynamics, contextual changes, and interactions. We carry out two case studies as a proof of concept of such analyses: First, a peer-to-peer protocol case study illustrates how undesired hierarchical interactions can be discovered automatically. Second, a case study on a self-adaptive production cell demonstrates how undesired interactions influence quality-of-service measures such as reliability and throughput.

Detecting inconsistencies in annotated product line models

Model-based product line engineering applies the reuse practices from product line engineering with graphical modeling for the specification of software intensive systems. Variability is usually described in separate variability models, while the implementation of the variable systems is specified in system models that use modeling languages such as SysML. Most of the SysML modeling tools with variability support, implement the annotation-based modeling approach. Annotated product line models tend to be error-prone since the modeler implicitly describes every possible variant in a single system model. To identifying variability-related inconsistencies, in this paper, we firstly define restrictions on the use of SysML for annotative modeling in order to avoid situations where resulting instances of the annotated model may contain ambiguous model constructs. Secondly, inter-feature constraints are extracted from the annotated model, based on relations between elements that are annotated with features. By analyzing the constraints, we can identify if the combined variability- and system model can result in incorrect or ambiguous instances. The evaluation of our prototype implementation shows the potential of our approach by identifying inconsistencies in the product line model of our industrial partner which went undetected through several iterations of the model.

Feature-oriented defect prediction

Software errors are a major nuisance in software development and can lead not only to reputation damages, but also to considerable financial losses for companies. Therefore, numerous techniques for predicting software defects, largely based on machine learning methods, have been developed over the past decades. These techniques usually rely on code and process metrics in order to predict defects at the granularity of typical software assets, such as subsystems, components, and files. In this paper, we present the first systematic investigation of feature-oriented defect prediction: the prediction of defects at the granularity of features---domain-oriented entities abstractly representing (and often cross-cutting) typical software assets. Feature-oriented prediction can be beneficial, since: (i) particular features might be more error-prone than others, (ii) characteristics of features known as defective might be useful to predict other error-prone features, (iii) feature-specific code might be especially prone to faults arising from feature interactions. We present a dataset derived from 12 software projects and introduce two metric sets for feature-oriented defect prediction. We evaluated seven machine learning classifiers with three different attribute sets each, using our two new metric sets as well as an existing metric set from the literature. We observe precision and recall values of around 85% and better robustness when more diverse metrics sets with richer feature information are used.

SESSION: Services and applications

Variability management meets microservices: six challenges of re-engineering microservice-based webshops

A microservice implements a small unit of functionality that it provides through a network using lightweight protocols. So, microservices can be combined to fulfill tasks and implement features of a larger software system---resembling a variability mechanism in the context of a software product line (SPL). Microservices and SPLs have similar goals, namely facilitating reuse and customizing, but they are usually employed in different contexts. Any developer who has access to the network can provide a microservice for any task, while SPLs are usually intended to implement features of a specific domain. Due to their different concepts, using microservices to implement an SPL or adopting SPL practices (e.g., variability management) for microservices is a challenging cross-area research problem. However, both techniques can complement each other, and thus tackling this problem promises benefits for organizations that employ either technique. In this paper, we reason on the importance of advancing in this direction, and sketch six concrete challenges to initiate research, namely (1) feature identification, (2) variability modeling, (3) variable microservice architectures, (4) interchangeability, (5) deep customization, and (6) re-engineering an SPL. We intend these challenges to serve as a starting point for future research in this cross-area research direction---avoiding that the concepts of one area are reinvented in the other.

Variability management: re-engineering microservices with delta-oriented software product lines

Combining microservices and software product line engineering (SPLE) is a challenge in variability management. This paper proposes a solution to that challenge by re-engineering existing webshop systems into a product line application. We first perform feature identification to analyze the features of subject systems. We introduce a mechanism that models the variability and designs a software product line architecture based on existing features. We use a UML diagram with the UML-DOP profile to abstract microservice variability in SPLE. Then, a transformation into a product line application is conducted to generate running applications based on selected features. We utilize a preliminary framework of microservice variability based on delta-oriented programming.

Can microservice-based online-retailers be used as an SPL?: a study of six reference architectures

Microservices are deployable software artifacts that combine a set of business features and expose them to other microservices. Ideally, the reuse and interchanging of microservices should be easy as they are supposed to be independent of each other, both conceptually and technologically. Selecting a service to fulfill a given feature (e.g., managing a cart in a website) recalls the way Software Product Lines (SPL) allow variability. However, in practice, interchanging microservices requires knowing the features that the services propose, how they communicate with other services and their types. In this work, we propose to analyze service dependencies as feature dependencies, at the feature, structural, technological, and versioning level, to assess the interchangeability of services. We analyze six community-selected use-cases and report that services are non-interchangeable systematically.

Policy-driven middleware for multi-tenant saas services configuration

In SaaS applications, which are mainly built following a multi-tenant architecture, the support of service variability among tenants is limited. It is due to the principle of sharing the same instance of the SaaS service between the tenants; thus, the particularities of each tenant's needs are not considered. Having to customize and adapt a SaaS service to tenants' needs and their specific contexts, while maintaining multi-tenancy, has emerged as a persisting challenge. The SaaS service adaptation has not only to be according to the functional requirements of the different tenants, but it has also to cope with the non-functional requirements in their various levels that are subject to change in time and space: i.e. within the same tenant and between tenants, respectively. For example, a tenant can just require minimal security level as it needs solely to be authenticated and has no other security concerns; while another may require a high security level and advanced and supplementary security mechanisms such as access control and encryption. There are also situations when the same tenant changes its desired quality level through time; for instance, when there is an increase of service requests in a critical period, the tenant may want to decrease the response time and to increase the uptime value in order to smoothly perform his requests. Therefore, the application needs to be adapted dynamically in order to meet every tenant's quality requirements as if he is the only one to consume the service.

Software Product Lines Engineering (SPLE) has largely tackled the variability management field in the literature. It enables high reusability in shorter time, at lower costs and with higher quality by creating product families or lines that share commonalities and have variation points. The derivation of the final product is performed at the level of those variation points while considering the inter-and intra- dependencies between the artifacts of a product line.

Our proposed approach is based on the SPLE principles to build SaaS services tailored to tenant-specific Service Level Agreements (SLAs). We defined the Domain Engineering phase to build SLA families. Based on domain analysis, the non-functional requirements of the tenants, their commonalities, variation points and variants are captured and modeled. A Generic SLA is generated which contains the terms and the Service Level Objectives (SLOs) of all the contracting tenants. From this Generic SLA, the proposed middleware generates core and tenant-specific policies. The Configurator, a middleware component, adapts the application according to these defined policies based on annotations in the form of key-value pairs.

As contributions of our article [1]: we proposed the architecture of the proposed middleware and its different components; we defined the Policy metamodel, that models policies retrieved from the Generic SLA as annotations, and described the model-to-model transformations from VariableSLA metamodel to Policy metamodel; and we proposeed a Configurator metamodel that models and describes how the Configurator component of the middleware configures the SaaS service.

Our approach differs from the literature in that it enables building a family of SLAs: a generic SLA and tenant-specific policies; linking, thus, the non-functional variability with the SLA. Moreover, it is not limited to a specific quality attribute but supports any quality attribute expressed in the SLA. It allows configurations at design time and at runtime through annotating the SaaS service components with adequate quality levels.

Requirements-driven configuration of emergency response missions with small aerial vehicles

Unmanned Aerial Vehicles (UAVs) are increasingly used by emergency responders to support search-and-rescue operations, medical supplies delivery, fire surveillance, and many other scenarios. At the same time, researchers are investigating usage scenarios in which UAVs are imbued with a greater level of autonomy to provide automated search, surveillance, and delivery capabilities that far exceed current adoption practices. To address this emergent opportunity, we are developing a configurable, multi-user, multi-UAV system for supporting the use of semi-autonomous UAVs in diverse emergency response missions. We present a requirements-driven approach for creating a software product line (SPL) of highly configurable scenarios based on different missions. We focus on the process for eliciting and modeling a family of related use cases, constructing individual feature models, and activity diagrams for each scenario, and then merging them into an SPL. We show how the SPL will be implemented through leveraging and augmenting existing features in our DroneResponse system. We further present a configuration tool, and demonstrate its ability to generate mission-specific configurations for 20 different use case scenarios.

On the use of product-line variants as experimental subjects for clone-and-own research: a case study

Software is often released in multiple variants to address the needs of different customers or application scenarios. One frequent approach to create new variants is clone-and-own, whose systematic support has gained considerable research interest in the last decade. However, only few techniques have been evaluated in a realistic setting, due to a substantial lack of publicly available clone-and-own projects which could be used as experimental subjects. Instead, many studies use variants generated from software product lines for their evaluation. Unfortunately, the results might be biased, because variants generated from a single code base lack unintentional divergences that would have been introduced by clone-and-own. In this paper, we report about ongoing work towards a more systematic investigation of threats to the external validity of such experimental results. Using n-way model matching as a representative technique for supporting clone-and-own, we assess the performance of state-of-the-art algorithms on variant sets exposing increasing degrees of divergence. We compile our observations into four hypotheses which are meant to serve as a basis for discussion and which need to be investigated in more detail in future research.

SESSION: Testing II

Testing configurable software systems: the failure observation challenge

Configurable software systems can be adapted or configured according to a set of features to increase reuse and productivity. The testing process is essential because configurations that fail may potentially hurt user experience and degrade the reputation of a project. However, testing configurable systems is very challenging due to the number of configurations to run with each test, leading to a combinatorial explosion in the number of configurations and tests. Currently, several testing techniques and tools have been proposed to deal with this challenge, but their potential practical application remains mostly unexplored. To encourage the research area on testing configurable systems, researchers and practitioners should be able to try out their solutions in common datasets. In this paper, we propose a dataset with 22 configurable software systems and an extensive test suite. Moreover, we report failures found in these systems and source code metrics to allow evaluating candidate solutions. We hope to engage the community and stimulate new and existing approaches to the problem of testing configurable systems.

Large-scale T-wise interaction sampling using YASA

Testing highly-configurable software systems (i.e., software product lines) is challenging due to their large configuration space. T-wise sampling is one method of finding a representative subset of configurations for a system, which can then be tested. However, for large-scale systems, such as Linux, existing t-wise sampling algorithms do not scale well. To this end, Pett et al. proposed the sampling challenge for large-scale systems at SPLC 2019. In this paper, we attempt to solve the proposed challenge using our sampling algorithm YASA. We report our experience for all three of the given systems FinancialServices01, Automotive02, and Linux. In addition, we present the results for computing samples for all versions of the system FinancialServices01.

Tackling the equivalent mutant problem in real-time systems: the 12 commandments of model-based mutation testing

Mutation testing can effectively drive test generation to reveal faults in software systems. However, it faces a typical efficiency issue as it can produce many mutants that are equivalent to the original system, making it impossible to generate test cases from them.

We consider this problem when model-based mutation testing is applied to real-time system product lines, represented as timed automata. We define novel, time-specific mutation operators and formulate the equivalent mutant problem in the frame of timed refinement relations.

Further, we study in which cases a mutation yields an equivalent mutant. Our theoretical results provide guidance to system engineers, allowing them to eliminate mutations from which no test case can be produced. Our evaluation, based on a proof-of-concept tool and an industrial case from the automotive domain, confirms the validity of our theory and demonstrates that our approach can eliminate many of the equivalent mutants (88% in our case study).

Learning-based prioritization of test cases in continuous integration of highly-configurable software

Continuous Integration (CI) is a practice widely adopted in the industry to allow frequent integration of code changes. During the CI process, many test cases are executed multiple times a day, subject to time constraints. In this scenario, a learning-based approach, named COLEMAN, has been successfully applied. COLEMAN allows earlier execution of the most promising test cases to reveal faults. This approach considers CI particularities such as time budget and volatility of test cases, related to the fact that test cases can be added/removed along the CI cycles. In the CI of Highly Configuration System (HCS), many product variants must be tested, each one with different configuration options, but having test cases that are common to or reused from other variants. In this context, we found, by analogy, another particularity, the volatility of variants, that is, some variants can be included/discontinued along CI cycles. Considering this context, this work introduces two strategies for the application of COLEMAN in the CI of HCS: the Variant Test Set Strategy (VTS) that relies on the test set specific for each variant, and the Whole Test Set Strategy (WST) that prioritizes the test set composed by the union of the test cases of all variants. Both strategies are evaluated in a real-world HCS, considering three test budgets. The results show that the proposed strategies are applicable regarding the time spent for prioritization. They perform similarly regarding early fault detection, but WTS better mitigates the problem of beginning without knowledge, and is more suitable when a new variant to be tested is added.

SESSION: Architectures and languages

Towards facilities for modeling and synthesis of architectures for resource allocation problem in systems engineering

Exploring architectural design space is often beyond human capacity and makes architectural design a difficult task. Model-based systems engineering must include assistance to the system designer in identifying candidate architectures to subsequently analyze tradeoffs. Unfortunately, existing languages and approaches do not incorporate this concern, generally favoring solution analysis over exploring a set of candidate architectures.

In this paper, we explore the advantages of designing and configuring the variability problem to solve one of the problems of exploring (synthesizing) candidate architectures in systems engineering: the resource allocation problem. More specifically, this work reports on the use of the Clafer modeling language and its gateway to the CSP Choco Solver, on an industrial case study of heterogeneous hardware resource allocation (GPP-GPGPU-FPGA).

Based on experiments on the modeling in Clafer, and the impact of its translation into the constraint programming paradigm (performance studies), discussions highlight some issues concerning facilities for modeling and synthesis of architectures and recommendations are proposed towards the use of this variability approach.

Neverlang and FeatureIDE just married: integrated language product line development environment

Language development is inherently complex. With the support of a suitable language development environment most computer scientists could develop their own domain-specific language (DSL) with relative ease. Yet, when the DSL is the result of a configuration over a language product line (LPL)---a special software product line (SPL) of compilers/interpreters and corresponding IDE services---they fail to provide adequate support. An environment for LPL engineering should facilitate the underlying process involving three distinct roles: a language engineer developing the LPL, a language deployer configuring a language product, and a language user using the language product. Neither IDEs nor SPLE environments can cater all three roles and fully support the LPL engineering process with distributed, incremental development, configuration, and deployment of language variants. In this paper, we present an LPL engineering process for the distributed, incremental development of LPLs and an integrated language product line development environment supporting this process, catering the three roles, and ensuring the consistency among all artifacts of the LPL: language components implementing a language feature, the feature model, language configurations and the resulting language products. To create such an environment, we married the Neverlang language workbench and AiDE its LPL engineering environment with the FeatureIDE SPL engineering environment. While Neverlang supports the development of LPLs and deployment of language products, AiDE generates the feature model for the LPL under development, whereas FeatureIDE handles the feature configuration. We illustrate the applicability of the LPL engineering process and the suitability of our development environment for the three roles by showcasing its application for teaching programming with a growable language. In there, an LPL for Javascript was developed/refactored, 15 increasingly complex language products were configured/updated and finally deployed.

WORKSHOP SESSION: Workshops

Third International Workshop on Variability and Evolution of Software-Intensive Systems (VariVolution 2020)

Modern software systems are subject to continuous evolution in time (i.e., revisions) and space (i.e., variants). Different research communities are concerned with techniques for managing the evolution in these two dimensions, particularly in the areas of software configuration management and software product-line engineering. Unfortunately, due to both research areas acting mostly independent of each other, traditional technologies for managing software evolution do not provide unified support for revisions and variants. To address this problem, the 3rd International Workshop on Variability and Evolution of Software-Intensive Systems (VariVolution) aims to gather researchers and practitioners from different communities to present, and work on, advances in unifying evolution in time and space. For instance, reference architectures and analysis techniques that integrate both dimensions have been presented and extended during previous workshop editions, aiming to provide means for systematically managing evolving software systems. VariVolution offers researchers and practitioners an opportunity to exchange ideas, report real-world case studies, discuss open problems, and initiate new research in a collaborative and open environment.

REVE 2020: 8th International Workshop on Reverse Variability Engineering

Software Product Line (SPL) migration remains a challenging endeavour. From organizational issues to purely technical challenges, there is a wide range of barriers that complicates SPL adoption. This workshop aims to foster research about making the most of the two main inputs for SPL migration: 1) domain knowledge and 2) legacy assets. Domain knowledge, usually implicit and spread across an organization, is key to define the SPL scope and to validate the variability model and its semantics. At the technical level, domain expertise is also needed to create or extract the reusable software components. Legacy assets can be, for instance, similar product variants (e.g., requirements, models, source code, etc.) that were implemented using ad-hoc reuse techniques such as clone-and-own. More generally, the workshop REverse Variability Engineering attracts researchers and practitioners contributing to processes, techniques, tools, or empirical studies related to the automatic, semi-automatic or manual extraction or refinement of SPL assets.

Third International Workshop on Languages for Modelling Variability (MODEVAR@SPLC 2020)

Feature models were invented in 1990 and have been recognised as one of the main contributions to the Software Product Line community. Although there have been several attempts to establish a sort of standard variability modelling language, there is still no consensus on a simple feature modelling language. There can be many motivations to have one but among others, there is one that is very important: information sharing among researchers, tools or developers. Following the spirit of the first two editions, this workshop is an interactive event where all participants shall share knowledge, but also first realizations about how to build up a simple feature modelling language that all the community can agree on.

Third International Workshop on Experiences and Empirical Studies on Software Reuse (WEESR 2020)

The Workshop on Experiences and Empirical Studies on Software Reuse (WEESR) aims to allow researchers and practitioners discuss in-progress research regarding experiences and empirical studies applying reuse techniques in non-academic environments. The third edition of this workshop, the WEESR 2020, was co-located with the Software Product Lines Conference at 2020 (SPLC'20). There, attendants discussed three original papers and a journal-first paper presenting empirical studies, reuse in companies not aware of software product lines, systematic reviews and evaluations of companies interested on adopting product lines. In addition, we have two invited talks of Sten Grüner and Thorsten Berger discussing experiences and challenges found in companies adopting SPLE for their products.

TUTORIAL SESSION: Tutorials

Software reuse and mass personalization

This tutorial explores the impact of the socio-economic trends of customization and mass personalization on software reuse through software product line development.

Variability realization in UML/SysML models

Motivated by experiences from different industrial settings, the tutorial reveals the increasing need for guidance and decision support on how to handle variants and variability in SysML and UML models. While a substantial amount of variability realization approaches has already been discussed on the level of source code, there is little guidance for practitioners on the model level. With this, there is major uncertainty in dealing with concurrent changes and parallel modeling of similar system variants.

Machine learning and configurable systems: a gentle introduction

The goal of this tutorial is to give a gentle introduction to how machine learning can be used to support software product line configuration. This is our second practical tutorial in this trending field. The tutorial is based on a systematic literature review and includes practical tasks (specialization, performance and bug prediction) on real-world systems (Linux, VaryLaTeX, x264). The material is designed for academics and practitioners with basic knowledge in software product lines and machine learning.

Variability modeling and implementation with EASy-producer

This tutorial introduces the EASy-Producer product line environment. EASy-Producer is an open-source toolset for engineering product lines, variability-rich software ecosystems, and dynamic software product lines. In particular, it features a variability modeling and a variability implementation DSL. It has been applied in several industrial case studies and research projects showing its practical applicability both from a stability and a capability point of view. While the tool-set also includes an interactive approach to product line definition and configuration through DSLs, the focus of this tutorial will be on the DSL-based techniques. We cover both the modeling of software product lines using the Integrated Variability Modeling Language (IVML) and its underlying conceptual approach, which differs from traditional feature modeling, but we will also provide an overview of the approach to variability instantiation with EASy-Producer (using the Variability Instantiation Language (VIL)).