Understanding Tests

A test is a JavaScript document which describes the behaviour of your virtual device during simulation. Tests consist of Device Model, Library Management, Protocol Settings and Security Parameters.

Once we have figured out what our device lifecycle looks like and what kind of device state variables we will have, we can jump to writing a test. A test is essentially a JavaScript document which describes how a single simulation will be executed. It's important to clarify that tests describe only a single device's behaviour. When you run a test, you can create multiple devices out of the same test. Each of these multiple devices could behave differently based on their unique client IDs. Every aspect of the device behaviour which needs to be modelled therefore should be described in the test.

How are the Tests used in the simulation? When a simulation is started, the simulation engine will parse the test and create virtual devices out of it. Each virtual device will have its own memory, state, and connect to the IoT cloud platform until your simulation is running. The following diagram describes the instantiation of the tests.

If you are familiar with C++ or any other object-oriented programming language, think of tests as Classes and virtual devices as Objects. Once a test is defined, you could create as many instances of it as you want.

You could also create multiple tests to simulate an end to end system. Take the example of simulating a smart city environment, you would need to create separate tests to simulate street lights, trash cans, parking slots among other things. When you need a complete simulation, you will run all these tests together. These tests would also be able to talk to each other, as we will see in the later documentation.

Device Model

A device model encapsulates the behaviour of the physical device into different stages, each defined as a JavaScript program. There are 2 stages (Init & Finished) which are always present in a device model. There is no limit to the maximum number of stages a test can have, which allows us to simulate devices with a very high level of complexity. The virtual devices can also be programmed to move to any of the stages at any point, by using the next() function.

Each of these JavaScript programs is passed a JavaScript object called state which could be used to hold device-specific information. A device model is independent of the connection protocol and the security parameters so you could reuse a device model across HTTP and MQTT protocols with ease.

So how are these device models executed? Let's understand this with the help of the picture given below.

In the above picture, time is taken on the x-axis. The first stage on any device model is the Init stage. This stage can be used to describe some initial values, retrieve data from the persistent storage or assign random values to different components. During the Init stage a setup function runs which takes care of setting up the virtual devices. This setup function is triggered even before a connection to the backend is attempted. This function doesn't need to return any value.

After the Init stage is complete the rest of the stages can start. These stages are used to define the core functionality of the device and all of its features. There can be any number of such stages according to the complexity of the device. These stages offer a logical separation between the different states the physical device can be in. Each of these stages is also divided into a Sender and a Receiver unit. The Sender is used to send messages to the backend and the Receiver handles the responses or commands coming from the backend. These stages are invoked for the specified number of iterations according to the run settings of the test. The duration between each iteration is also configurable and can be changed in the run settings.

After the device has gone through all the different stages with the device functionality, it finally goes to the Finished stage. The Finished stage can be used to clean up any server-side resources or to summarize the test results.

The other components of the tests are as follows :

Library

We allow users to import any additional npm packages or custom scripts to enhance the device functionality. These npm packages or scripts are imported at every run of the test and can be used everywhere. Currently, we only allow up to a maximum of 5 such packages and scripts in total.

To know how you can use external libraries with IoTIFY, jump to the page linked below.

External Libraries

Protocol

Protocol specifies the server settings and underlying option for the chosen connectivity. The details are specific to each protocol. It's important to note that many of these protocol parameters are scriptable i.e. you could change them dynamically with the help of state objects or other helper functions. This is particularly interesting for MQTT e.g. where you would like to listen to a device-specific MQTT Topic. You could simply subscribe with the following string:

/device/{{client()}}/command

When the test is executed, the part within the double parenthesis will be evaluated and client() function will return a unique client ID, starting from zero. For example, if you are simulating 100 clients, each one of them will have its own unique client ID. The string would thus evaluate to the following for client 66:

/device/65/command

Protocol is also used to specify the client-specific security settings such as username/password or preshared keys/certificates etc. The fields related to security are also scriptable.

To understand the different protocols and how to utilise them with IoTIFY, please visit the page linked below.

Protocol Settings

Summary

A test consists of a device model, protocol and run settings details. Each test is passed a unique JavaScript object called state which could be used to maintain the device status through its run time. There are several helper functions and scripting options that make fields configurable inside the protocol tab.

Should we describe the entire behavior of a device into the single test? Not necessarily. We will learn in the later chapters about glob key-value stores which could save device states in persistent memory. Using this storage, we could create separate tests which retrieve the last known value of a device state, run a scenario and then save the latest state back in the memory.

Now we have a good overview of what a test is, let's jump to the finer details of these tests.