Conformiq Test Generator ™ - Frequently Asked Questions

What is Conformiq Test Generator ™?
Conformiq Test Generator ™ is a model-based test automation and test generation tool. The product replaces the manual process of writing test scripts with tests generated automatically from test models defined as UML state diagrams. Conformiq Test Generator ™ consists of a testing tool and ready-made or customized adapter toolkits that enable testing of virtually any type of application.

What does test generation mean?
Conformiq Test Generator ™ generates stimuli based on the test model and listens to responses from the System Under Test (SUT). Conformiq Test Generator ™ then compares the responses against the test model thus validating correctness of the System Under Test. If there are inconsistencies between the test model and the behavior of the System Under Test, these inconsistencies are reported as defects.

What is the main benefit of Conformiq Test Generator ™?
The main benefit of Conformiq Test Generator ™ is that based on mathematical algorithms, large amounts of relevant tests are created from the model. We say that the test model spans a test space, from which these individual tests are then generated.

As an analogy you could think of a test model with states and transitions as a map with cities and roads connecting these cities: the more cities and roads there are, the more different routes there are to choose from. Conformiq Test Generator ™ explores the test model (map) using intelligent exploration algorithms and coverage measurement techniques, and thus auickly generates different tests (routes): this ensures that test coverage is more comprehensive and testing is more efficient than by using traditional testing methods.

What other benefits can be achieved with Conformiq Test Generator ™?

With UML state diagrams you can easily model parallelism an d concurrency prevalent in modern software systems. Conformiq Test Generator ™ gives testers control over these phenomena and allows efficient testing of such complex systems.
The modeling formalism allows testers to intuitively define data abstractions in the problem domain. Hence complex data combinations and boundary values can be expressed easily in the test models.
Thansk to its flexible architecture Conformiq Test Generator ™ can interact with the System Under Test via multiple Points of Control and Observation (PCO). For example, you could control a web application via a HTML/HTTP-based user interface and at the same time observe swstem's internal state as stored in the data base via a data base interface.
A test model is easier to maintain than a set of test scripts.
A test model works as a formal and unambiguous test plan.

How does Conformiq Test Generator ™ fit into a software development process?
Conformiq Test Generator ™ can be employed in processes ranging from tradtional waterfall model to modern agile processes such as XP (eXtreme Programming).

Ideally Conformiq Test Generator ™ is included in the development process as early as possible to allow efficient testing from the very beginning of the development process. Thus the test model development becomes a concurrent task together with the system development.

What can be tested with Conformiq Test Generator ™?
The modeling formalism is domain-independent in the sense that it does not restrict use of Conformiq Test Generator ™ to any particular application domain. System adaptation technology, on the other hand, makes Conformiq Test Generator ™ technology-independent and allows testing systems ranging from low level embedded devices to enterprise web-applications.

What kind of defects can be found using Conformiq Test Generator ™ which are difficult to find manually or with other testing tools?
Testing of concurrent functionality with manual methods is almost impossible and this is where Conformiq Test Generator ™ shows its advantage over traditional testing methods. Defects related to synchronization and concurrent use of resources can be found easily with Conformiq Test Generator ™.

Applications with complex input space in terms of data is another area where Conformiq Test Generator ™ excels. Users can define data intuitively in the application domain and Conformiq Test Generator ™ then generates combination and boundary values based on these definitions.

Obviously Conformiq Test Generator ™ is not limited to exotic hard-to-find defects but users gain a significant productivity boost from its advanced test automation and reporting features in their search for less contrived defects.

Which are the supported operating systems?
Conformiq Test Generator ™ runs on Windows NT, Windows 2000, Windows XP and Linux. The System Under Test, however, can run on any operating system.

Is it possible to manage Conformiq Test Generator ™ with any test management tool?
Yes. Conformiq Test Generator ™ supports integration to Mercury's test process management tool TestDirector®. This integration improves test planning, test management and test execution as well as helps report handling and defect management.

Can Conformiq Test Generator generate tests from my state chart design models?
A design model is a model that describes the desired functionality of a system from the systems perspective. Conformiq Test Generator generates tests from "tester models", which are state charts describing testing activities. A design model cannot be used directly as input to Conformiq Test Generator.
Another issue is that if your state charts has been designed e.g. in Rational XDE or Microsoft Vision, the action in your state chart most typically are not written in the syntax that Conformiq Test Generator uses for actions. In Conformiq Test Generator there is a procedural programming language fro writing state chart actions.

How Conformiq Test Generator is connected to the system under test?
Conformiq Test Generator is connected to the system that you want to test through something that we call system adapter. A system adapter is a piece of software that must be created to use Conformiq Test Generator to test a real system.
A system adapter consist of two parts: a standard library from us, and custom code. This standard library ("test bed adapter library") handles the connection with Conformiq Test Generator. It makes data from Conformiq Test Generator available to the custom code part and provides services such as sending data back to Conformiq Test Generator to the custom code part.
Test Bed Adapter libraries are available for C, C++, Java, C#, and Python.

What is a system adapter?
The system adapter is usually a thin component that works as a point of communication between the testing API (Application Programming Interface) and Conformiq Test Generator ™. It maps the abstract messages sent by Conformiq Test Generator ™ to concrete messages for System Under Test and vice versa. A good design of the underlying testing API will also reflect in a straightforward design and implementatin of the system adapter.

Into what kind of interface(s) should Conformiq Test Generator ™ be connected?
You should select an interface(s) that is essential part of the system and enables efficient testing of the system. The selected interface(s) should be clearly defined and stable to reduce the need for later modifications of system adapter and test model.

How does Conformiq Test Generator ™ explore the model? Conformiq Test Generator ™ uses local and global search and planning techniques. It attempts to cover aggressively as a large amount of the test model as possible, taking into account the coverage criterion selected. This accelerates testing and enables more relevant test cases to be generated.

Can the UML models used in product design be used also when testing with Conformiq Test Generator ™?
At the moment we do not support importing models from other modeling tools. Moreover, it is important to realize that design models are not the same as test models.

Do you have to have the whole system developed before you can start planning tests and using Conformiq Test Generator ™?
There is no need for that. In fact, developing the test model in parallel with the actual system development is highly recommended. This way the System Under Test can be tested continuously during the development project which allows you to utilize full benefits of using Conformiq Test Generator ™.

What kind of skills does it require from the testing personnel to use the model editor?
The model editor in Conformiq Test Generator ™ is a user-friendly tool for defining test specifications using modular UML state diagrams. In order to learn modeling quickly, it is good to have some general programming skills as well as knowledge of UML terms. However, these skills are not mandatory.

We recommend that our customers choose to participate in Conformiq Test Generator ™ basic training and utilize our consultants for creating models in the beginning so that the models can be of highest quality.

How is reporting managed with Conformiq Test Generator ™?
Conformiq Test Generator ™ produces a textual test run log that records the events of the test run. The log can be filtered in many ways to produce desired log output.

In addition, a separate reporting interface enables creating customer specific reports of test run results. This makes test result analysis easier.

Top of the page

Conformiq Test Generator Mainpage

last updated: 21.01.2006

© 2004-2006 Conformiq Software Ltd.
Conformiq Test Generator is a product and trademark of Conformiq, Espoo (Finland)
all other trademarks of this site are the property of their respective owners.