Developer of Things: IoT API Development and Testing
By Harsh Upreti

In the last few decades we have had a front row seat to witness the evolution of devices that generate and consume information on the internet. The format, frequency and methodology information flow between devices has also changed. You do not have to be developer to notice that the dream of ‘intelligent agents' in the original vision of Tim Berners Lee for Web 3.0 has been slowly shaping into reality. I noticed this initially during the evolution of Facebook from static pages to a more ‘real-time' application that serves the latest and greatest content as soon as it's added or uploaded. This trend further evolved into even more real-time information on our smart-phones and is now reaching the stage when each and every device is capable of providing real-time updates to each other within milliseconds. Furthermore, distinguishing a device as a client or a server is increasingly becoming hard and paradigms like peer to peer, multicast communication have taking root and are thriving.

As this transformation takes place the lines are increasingly blurred between the capabilities of devices. This also means that the way developers write and testers test APIs has also changed. Earlier for client and server tête-à-tête, HTTP was the go-to protocol along with standards as REST defining strategies to make life easier API users. Recently, with the advent of peer to peer paradigms the model of communication is not just limited to synchronous request-response, it has extended to asynchronous and multicast. Peer to peer protocols primarily are asynchronous in nature and hence do not necessarily work in pairs of request and response. This enables lighter, faster and independent processing by devices. IoT protocols and standards have evolved in this transformative age, hence today we have a range of technologies and protocols for IoT confirming to both client-server and peer to peer paradigms. It's important for API developers and testers aspiring to work with IoT to be aware of these.

Server based IoT protocols
IoT protocols, that work on client-server paradigm, have devices (clients) connected to a single broker (Server). The presence of the server is essential for the optimal functioning of the system. This paradigm is well known to traditional developers and testers of web applications, but unfortunately the resemblance with web patterns ends there. Example of such IoT protocols are: MQTT and XMPP-IoT. These protocols work on the publish-subscribe (Pub-Sub) mechanism instead of the request-response method which is so well known to the developers of web applications. Pub-Sub requires connected devices to subscribe to central topics, push updates to them and receive broadcasts from the topic. This is very similar to the web-hooks concept in which web developers provide a provider with a hook (endpoint) that gets invoked once the provider has something to push or an event to broadcast.

It's interesting to note that protocols like XMPP, that have enabled real-time chat for years, has extensions for Pub-Sub. This extension was underutilized till recently, when the usability of pub-sub for IoT became very apparent.

For developers and testers working with pub-sub IoT protocols, it's important to first understand basics of pub-sub. A good article explaining pub-sub is here.

Peer to Peer IoT protocols
It was not until recently that network technology reached a state where peer to peer communication could be done reliably and at scale. This resulted in the evolution and growth of more peer to peer standards and protocols. Thus peer to peer IoT protocols like CoAP are very recent. Unlike MQTT which was started in 1999, CoAP only had the first draft in 2010. CoAP enables interesting use cases for IoT as it allows devices to communicate with each other independent of a central broker (Server). Also CoAP follows REST like methods, i.e. GET to request data and POST to send new data to another device. These method calls are made on a device URL, which is the identifier for the device. Despite its resemblance to REST over HTTP, it's important to note that this protocol utilizes UDP as the transport mechanism, this is in contrast to HTTP and MQTT that use TCP.

Is it important for an IoT developer to know the difference? Yes, it is. And we will discuss these protocols in a section below.

As CoAP has REST methods, developers and testers can leverage their experience with web development and RESTful APIs to work on CoAP integrations. Also the symmetry of CoAP with REST enables connection of devices on CoAP to connect to REST endpoints. Despite the similarities however, understanding the underlying transport protocol differences are critical to configure, code and test these devices right.

A typical IoT setup with devices, broker and the cloud integrations. Integration between devices could be through a server or be just peer to peer.

HTTP Vs IoT protocols
We've defined the categories and variety of IoT protocols but there are many other fundamental differences between IoT protocols and traditional protocols like HTTP. Developers and testers should briefly read the protocol specification of choice for IoT and then think about the fundamental way in which it differs from the traditional web-service API paradigms. I have captured some differences below to define the general direction. IoT is still an evolving field and these are in no way exhaustive.

• Efficiency: IoT protocols are designed to be frugal on message size, so with IoT implementations it's important to validate that message streams adhere to size limits.
• Reliability: UDP is not necessarily reliable, so use cases and tests need to be accordingly adjusted for IoT implementations on UDP based protocols.
• Scalability: Devices tend to send a fixed message with single sensor readings. But they do this multiple times in a second. Also, the devices number can be in thousands and IoT protocols are heavily optimized for this.
• Security: Security is the biggest risk in IoT, hence each protocol specification has security as a component.
• Discovery: There can be millions of devices in an IoT network, hence these protocols define clear ways of identifying each device. The list of devices is often managed through a registry at the broker.
• Novelty: IoT protocols are a new way of designing communication between lightweight devices, hence they often do not follow the traditional HTTP protocol's request-response paradigm.

Note: UDP Vs TCP

It's important to note for developers and testers who are used to working on HTTP that if they wish to work with IoT, they should understand difference between TCP and UDP. HTTP works on TCP. TCP ensures that each packet is delivered to its destination, if a packet does not reach the destination then TCP, albeit at a cost, has the mechanism to send the missing packet again. Hence the chance of loss of information with TCP is minimal. Thus TCP is extensively used for text transmission, as a missing character from a word may change the intended meaning or even make the text unreadable. This reliability is fantastic, but it comes at the cost of additional resources and time overheads in the protocol, which makes it slightly more resource intensive and not exactly ideal for lightweight and resource-constrained IoT devices.

UDP on the other hand enables faster transmission. This is because UDP does not promise the delivery guarantees made by TCP, hence the transmission management overhead is minimal. This is a very popular protocol for voice and video transmission. Your video and audio calls have been transmitting over UDP, its fast and a few packets lost on video does not make much of a difference, except maybe a barely noticeable flicker on the screen. IoT protocols on UDP save time and network resources, thus ensuring quicker and more frequent delivery of sensor readings. A good example is an infrared IoT device that transmits the visual stream to a broker, it needs a high speed channel to transmit data and it's okay if not 100% of the packets are transmitted. In such cases an UDP based protocol will work very well.

IoT Development and Testing
IoT architectures are multi layered, a few sample integrations using IoT API are:

1. Device to Device, using IoT protocols like CoAP
2. Device to Broker, using protocols like MQTT
3. Broker to Cloud, good old HTTP and REST

As a ‘traditional' API developer or tester you may choose to work at one or all layers and integrations. But it's important for you to understand the key concepts of how each protocol works and the exact job it enables. Even if you work at the HTTP layer, it helps to have a context of how the IoT data has been generated. Also, recognize the importance of data like timestamps and format issues like scientific and mathematical notations, as the sensor data being transmitted could require high precision to the ninth or the tenth decimal place. The IoT pipes will be continuously flowing with information, critical decisions like frequency of data uploads to cloud will also have to made in code and tested appropriately.

IoT devices will send data continuously and brokers process data from multitude of devices, thus scalability of your APIs is critical and should be taken care of. Performance of brokers, cloud infrastructure and security will be the prime focus of testing IoT APIs, which is much beyond just functional testing.