Basic functional test

In this example, we will ensure that the server responds when a trigger condition is met.

Functional testing an IoT platform required developing test methods to ascertain the validity of the cloud platform response with respect to sensor data. A cloud platform may either respond by alerting human user or by controlling the device itself. Let's have an example.

Let's consider a device model for a fire alarm. In real life, the device will have a microcontroller to automatically trigger a fire alarm when the temperature exceeds a certain threshold. However, for the sake of demo, let's assume that the alarm trigger functionality is offloaded to a cloud platform. So the device would simply send measured temperature value to the could platform and expect the cloud platform to trigger an alarm whenever device temperature exceeds a threshold.

We will simulate a simple client using IoTIFY and launch a simple MQTT server application which is supposed to be system under test. Let's go-

Create a Client Template

The first step in building a basic test is to create a device template, which will validate the server functionality.

Step 1. Build a Device Model

You could skip this step and import the client template used in this example directly. Go to the template menu, select the import template button on the right hand side and provide following URL for Import.

https://raw.githubusercontent.com/iotify/nsim-examples/master/functional-testing/functest1.json

Once the template has been imported, you could skip step 1. Do not forget to change the MQTT topic in Step 2 .

1. Create a blank template and with MQTT protocol as a backend. Name this template functest1

2. Provide the following code into the Init function

{
  state.temperature = 75;  // set temperature value to 75 degree
  state.alarm = false;  // Alarm is Off by default
}

Provide the following code into the message function

{
    const threshold = 80;
    if (state.temperature > threshold)
    {
        assert(state.alarm === true, "Alarm not triggered when temperature exceeded "+threshold)
    }
    // Increase the temperature randomly by max 10 degrees on each iteration
	  state.temperature = volatile({min:60, max: 100, delta:10, key:'myvariable'})
    
    return JSON.stringify(state);
}

What is happening above? In line 9, we declare temperature to be a volatile variable, whose values lies in the range between 60 and 100 and can change at max 10 in each step from the last step. In line 4, we assert that when temperature exceeds 80 degree, the alarm must have been raised. (The alarm is raised as a result of server initiated response)

Provide the following code into Response Handler function which will handle any command received from the server.

{ 
  console.log("Command Received", response);              
  
  let cmd = JSON.parse(response);
  if (cmd.alarm)
  {
      console.log("Alarm is triggered by the server")      
      state.alarm = true;
  }
  else  {
      console.log("Alarm is Switched off")      
      state.alarm = false;
  }
}

Above, we parse the payload received by server and set the alarm state as received.

We expect that the server should immediately send the alarm:true condition as soon as it receives payload with temperature value exceeding the threshold.

Step 2. Specify MQTT Protocol

Now go to the MQTT protocol setting and leave default settings as it is. Only change the MQTT Topic and Subscription ID to the following

Topic: /[unique_topic_id]/temperature/{{client()}}

Subscribe: /[unique_topic_id]/command/{{client()}}

Make sure to check the subscribe checkbox as well. This should look something line as follows

The topic id must be set to a sufficiently random string as we are using a public MQTT broker where anyone could listen and publish. If more than one user tries with the same ID, they will end sending messages to each other. Therefore create a unique topic string and make sure to provide it in the server example as well.

That's it. Our client simulator will now publish the temperature on MQTT Topic /qwerty321/temperature/0 and will receive any command back from the server on topic /qwerty321/command/0

Step 3. Security Settings

Security settings should be left default.

Preview the template to make sure everything is looking good and then Save the template.

Create the Server Application

In a real world, you will have an enterprise business application which would listen on MQTT topic for every device and will send a response based on some complex logic. For this demo we will use a simple Javascript program which will act as our own mini server and will respond to the temperature messages received over MQTT.

Copy the unique MQTT topic key you used in the client simulator and make sure to change the MQTT topic in the server code, before you run it.

The sample code for the server side application and instructions to run this server could be found at

A Simple MQTT Alarm Server Examplensim-examples

Once the MQTT topic has been changed to match the client, you could run the server code. You should see the following logs in the server console.

"Running Simple MQTT test server"
"Subscribed to topic."

Run the Test

Once the server program runs in the above code, lets start our IoTIFY client simulator. Click on simulate button and choose the template on the left hand side drop down screen. Set as following:

Number of clients :1

Repeat Message: 5

Gap between iteration: 10

That's it. The client simulator is now running and will send messages to the server page. Check the logs of the server application. You should see incoming temperature messages from our client.

Some of the test cases may fail if the server doesn't send a response in time for the next iteration to trigger. Those cases will be marked as failed, and counted as red.

The server code runs for approx 60 seconds only. You may see evaluation timed out error. If this happens, simple rerun the server via green button.

Optional Assignments

• Plot the value of temperature sent via metrics.add() API
• Change the server code so that it sends the alarm true, if the temperature value remains high for at least 3 consecutive measurements. Make sure to change the assert condition in the client template to reflect the same behavior.

Comgratulations! Now you have learnt how to perform a basic functional test with IOTIFY. Let's see some more advanced use cases.