Denial of service attack detection through machine learning for the IoT

Sustained Internet of Things (IoT) deployment and functioning are heavily reliant on the use of effective data communication protocols. In the IoT landscape, the publish/subscribe-based Message Queuing Telemetry Transport (MQTT) protocol is popular. Cyber security threats against the MQTT protocol are anticipated to increase at par with its increasing use by IoT manufacturers. In particular, IoT is vulnerable to protocol-based Application layer Denial of Service (DoS) attacks, which have been known to cause widespread service disruption in legacy systems. In this paper, we propose an Application layer DoS attack detection framework for the MQTT protocol and test the scheme on legitimate and protocol compliant DoS attack scenarios. To protect the MQTT message brokers from such attacks, we propose a machine learning-based detection framework developed for the MQTT protocol. Through experiments, we demonstrate the impact of such attacks on various MQTT brokers and evaluate the effectiveness of the proposed framework to detect these malicious attacks. The results obtained indicate that the attackers can overwhelm the server resources even when legitimate access was denied to MQTT brokers and resources have been restricted. In addition, the MQTT features we have identified showed high attack detection accuracy. The field size and length-based features drastically reduced the false-positive rates and are suitable in detecting IoT based attacks. ARTICLE HISTORY Received 16 April 2020 Accepted 7 May 2020


Introduction
Critical infrastructures (CIs) are increasingly aiming to improve their efficiencies to deliver services to their stakeholders, with the Internet of Things (IoT) promising to provide significant opportunities for improving various CI processes for industries and consumers alike. The role of IoT in building future smart cities is indispensable and is already playing an important role in transforming the energy, transportation and communications sectors (Zanella et al., 2014). The increasing role of IoT devices in such infrastructures, is exposing their vulnerabilities to adversarial cyber threats (Bekara, 2014). It is anticipated that the attack surfaces will rapidly evolve in the near future, exposing CIs to a range of cyber security challenges (Asplund & Nadjm-Tehrani, 2016;Sadeghi et al., 2015). The attack surface for a cyber-physical IoT system of a typical CI spans across its layers of operation; Physical to Application (Sadeghi et al., 2015). Moreover, the variety of communication patterns and formats adopted by the heterogeneous IoT protocols of a CI encumber the design of a robust cyber security solution (Heer et al., 2011).
MQTT is a publish/subscribe-based Application layer protocol suitable for Machine to Machine (M2M) communication pattern, where a central message broker routes the published messages to subscribers based on their topics of interest. On a publish/subscribe platform, the publishers would publish messages to a topic and subscribers would subscribe to their respective topics of interest. The message broker, acting as an intermediary, then forwards the published messages to subscribers based on the subscribed topics. Figure 1 shows the message publish/subscribe process as implemented by the MQTT protocol.
The message broker plays an important role in MQTT as it decouples the sensors and actuators or monitoring IoT devices in both space and time. This is achieved by a process known as filtering. The publish/subscribe messaging pattern employs two common forms of filtering: content-based and topic-based. In content-based filtering, the subscribers receive only those messages that contain or match the attributes defined by the subscribers, whereas for topic-based filtering, subscribers receive only a subset of published messages that match the message topics on logical channels subscribed by them. The MQTT protocol does topic-based filtering to route messages to interested subscribers. The protocol specifies various control packets to facilitate message exchange between its endpoints. Some of the common control packets are enumerated in Table 1.  The messages between publishers and subscribers are communicated using various control packets which contains a fixed two byte protocol header. This ensures a small message overhead suitable for constrained devices operating in unreliable communication networks. These features have made MQTT protocol a favourable choice for M2M communication in IoT. As the MQTT adoption rate increases for IoT data communication, it is very important to assess and design its security.
A range of security challenges exist for IoT, but Denial of Service (DoS) attacks that target the data communication system pose a particularly significant challenge to IoT deployments (Heer et al., 2011). Such attacks can also pose as a major challenge in cyber-physical CIs, which rely on real-time inter-device communications (Ten et al., 2010). A consequent delay in message delivery due to DoS attacks would disrupt smooth operations of cyber-physical systems of CI. Most of the work as found in the IoT literature either presents DoS attack detection techniques (Alanazi et al., 2015;Kasinathan et al., 2013) or merely refers to DoS attacks as one of the challenges to the IoT ecosystem (Borgohain et al., 2015;Roman et al., 2013;Shaker & Zarrabi, 2017). Furthermore, the MQTT OASIS specification (Cohn, 2014) lists DoS attacks as one of the security threats to the MQTT protocol. Hence, a security solution to protect MQTT-based IoT infrastructures against such attacks is essential.
In this paper, we propose a machine learning-based MQTT DoS attack detection framework for the IoT platform, based on a custom made DoS attack model. The key contributions of this paper are: . Identification of network traffic features to represent the MQTT protocol meta-data, . Modelling of DoS attacks for MQTT brokers, . Design of an effective detection framework to detect MQTT DoS attacks, and . Testing of the attack model and the attack detection framework on a physical IoT deployment.
The rest of the paper is organized as follows: in Section 2, we present the related work. Section 3 details the DoS attack model. Section 4 presents attack detection framework. Section 5 presents the experimental results and analysis thereof. The conclusion is presented in Section 6.

Literature review
The most common types of DoS attacks aim to exhaust the network bandwidth, CPU cycles or memory on the target system to make services unavailable for legitimate users (Durcekova et al., 2012). To maximize the impact of DoS attacks against a target system, findings reported in the literature indicate that Application layer attacks are being increasingly perpetrated by adversaries (Brenner, 2010;Mantas et al., 2015). Such attacks aim to consume the target resource by sending carefully crafted legitimate requests towards the victim. The most common DoS attacks against Application layer protocols exploit vulnerabilities in the initial connection-establishment message exchanges, as listed in Table 2.
The various performance evaluation methods proposed in Scalagent (2015), Lee et al. (2013), Thangavel et al. (2014), Luzuriaga et al. (2015), Yokotani and Sasaki (2016), Gündoğan et al. (2018) and Fehrenbach (2017) do not evaluate the broker performance during the DoS attacks. In a work done by Fehrenbach (2017), the author conducted tests to simulate a Distributed-DoS (DDoS) attack on the MQTT broker. The main focus of his contribution was to identify the impact of using Transport Layer Security (TLS) for communication and message QoS on the broker resources. One of the drawbacks of his work was that it focused only on the impact of various message QoS levels and TLS connection on the broker performance. Our work is different from the study presented in Fehrenbach (2017) as the focus of this work is on modelling DoS attacks by varying the parameters in control packets and assessing their impact on various MQTT brokers and deployment scenarios. Perrone et al. (2017) presented a security analysis of MQTT protocol and described the various security requirements for IoT deployments. In another work, Andy et al. Andy et al. (2017) presented some attack scenarios as well as a security analysis of the MQTT protocol. In their work, the authors highlighted the security issues of the MQTT protocol and discussed attack scenarios against brokers with open authentication. Feasibility of such attacks is questionable as most MQTT broker deployments in industrial environment disable open authentication feature as it poses security risk of unauthorized access. In order to understand the security issues in the MQTT protocol, a threat model and the impact of SYN-Flood DoS attack on message brokers was presented in Firdous et al. (2017).
Santiago Hernández Ramos and Lacuesta (2018) proposed a fuzzing approach to test vulnerabilities of an MQTT based application. The proposed approach tested the behaviour of the MQTT based application when fuzzed data was inserted between clients and the broker. The authors used a proxy fuzzing technique along with a non-normative packet variable header data template to asses the behaviour of both broker and clients when presented with unexpected data. Failures were detected in certain versions of the broker software and in client applications. A similar MQTT fuzzer tool known as F-secure  Singh et al. (2017), Adi et al. (2016) and Ranjan et al. (2009). b Shan et al. (2017). c Rafique et al. (2009) andLuo et al. (2008). d Ballani and Francis (2008). e Bencsath and Ronai (2007).
MQTT-FUZZ was developed by Vähä-Sipilä (2015) which uses sniffed raw MQTT control packet payload to launch fuzzed MQTT packets against the broker. An existing work that attempts to detect IoT based attacks proposed MQTT transaction based features Moustafa et al. (2019). However, the authors used features based on the TCP protocol analysis, which do not provide sufficient information on the MQTT protocol parameters. In contrast, our proposed MQTT features are based on MQTT header and payload meta-data, which can effectively detect and differentiate such attacks. In addition, the main drawback of Moustafa et al. (2019) is that the performance of their attack detection scheme was not presented for MQTT attacks. The primary reason behind this was that no real MQTT attack datasets existed to evaluate the detection techniques. In our work, we first present various vulnerabilities in MQTT and also develop several attack scenarios to generate actual DoS attack traffic. We also evaluate the detection capability of the proposed IoT attack detection framework. In the following section, the DoS attack model used to create the DoS attack scenarios on MQTT protocol is presented.

DoS attack model
The main goal of DoS attacks is to overwhelm server resources and to deny access by legitimate clients. According to Little's Law (Little & Graves, 2008), the average number of items in a queuing system can be defined as: where λ is the arrival rate of items into the system and W is the average time spent by an item in the system. DoS attacks aim to fill-up the system queue, thus denying service to legitimate clients. DoS attacks can either increase arrival rate of packets or increase the per-packet processing time by forcing complex computing operations at the victim device. In most industrial MQTT applications, authentication and authorization help prevent unauthorized access, and authenticated clients are authorized to either send or receive messages on selected topics only. Access levels available for MQTT clients are one of two: . Valid Credentials to connect to MQTT broker, or . Valid Authorisation to Publish/Subscribe to topics.
An attacker without valid credentials can only vary the parameters of a CONNECT packet as clients cannot publish or subscribe without successful connection to the broker. However, after a successful connection using valid credentials but without valid authorization to publish and subscribe to topics, the attacker can vary PUBLISH or SUB-SCRIBE control packet parameters. Based on the control packet type and the access level available to an attacker, the flooding attacks can be categorized into: (1) Basic CONNECT Flooding (BF1): The attacker only sends a large volume of CONNECT packets to the target server to overwhelm the server with the processing of authenticating requests.
(2) Delayed CONNECT Flooding (BF2): The CONNECT packet transmission is delayed by the attacker after the establishment of the TCP session. This will result in a high volume of half-open TCP sessions at the broker, as it is waiting for the CONNECT request to complete. It also causes the broker to process these invalid credentials, thus leading to an increase in CPU utilization. (3) WILL Payload CONNECT Flooding (BF3): The CONNECT packet size is increased by the attacker through piggy-backing a WILL Payload on a CONNECT packet. This will lead to the consumption of both the entire bandwidth at the victim server as well as CPU resources, preventing it from processing new connections. (4) Invalid Subscription Flooding (IAUTHS): With valid credentials but no authorization to access various topics, an attacker can flood the broker with invalid subscriptions or publish requests to the subscriber. This will result in consumption of broker CPU resources in verifying individual request.

Attack detection framework
In this section, a DoS attack detection framework is proposed. The framework comprises of a network traffic generator, feature extraction engine and a machine learning-based DoS attack traffic classifier.

MQTT traffic generation
An IoT-MQTT network typically comprises of a set of IoT sensors that observe environmental phenomena and constantly communicate with each other or with monitoring or control devices through a centralized message broker. The proposed traffic generation component of the detection framework includes two physical servers, one hosting the VerneMQ MQTT broker virtual machine and the second serving as the attacker. Thirty Raspberry Pi (RPI) devices and four WEMOS ESP8266 devices were connected to two wireless routers; equally distributed. Twenty RPI devices and four ESP8266s, each interfaced with physical sensors was configured to publish sensor data periodically. The sensors comprised of: a PIR motion sensor, CCS811-Air Quality Sensor, DS130 RTC clock, DS18B20 temperature sensor and MH-MQ Gas sensor. The remaining 14 RPI devices were configured to periodically send MQTT messages to the broker. The broker was configured with 1000 username/password combinations to authenticate the clients, and 1000 MQTT Access Control-list (ACL) to authorize devices to publish/subscribe to various topics. Anonymous login was disabled to allow only authenticated access to publish and subscribe topics. Figure 2 illustrates the network traffic generator testbed deployed. Sensors were sending updates to the broker with a varying periodicity with a sleep interval between 4 and 8 s. The reason to keep the sleep intervals below 10 s was to achieve a realistic message publish rate, in alignment with the standard practice (Sivanathan et al., 2018). Sensors were also configured with LAST WILL message generated varying length messages, where a LAST WILL message is transmitted to update the subscribed clients if the publishing client disconnects abruptly.
DoS attack traffic was generated using a custom-built MQTT attack tool based on the Ecliplse-Paho library (Eclipse, 2018). Since these MQTT attacks were generated from a single attack source, a multi-threaded approach was adopted in the attack tool to maximize the impact on victim server's available resources. Each attack was based on specific MQTT protocol settings. The brute force CONNECT flooding attacks were configured with a random length character comprising a ClientID, username and password, similar to a real client. The Subscribe flooding attack sent 200 subscription requests in each session with six subscriptions per request. The number of subscription requests was randomly selected to maximize the number of SUBSCRIBE packets sent to the broker per connection request. The delayed CONNECT flooding attack (BF2) was launched using 250 threads and the remaining three attacks were launched using three threads to generate maximum number of attack packets targeting the broker. Selection of the number of threads for each attack type was based on the evaluation of number of attack packets received by the broker while incrementing the number of threads and the value that generated maximum number of attack packets was chosen.
Attack traffic was generated from a separate physical server connected to the network based on the various attack scenarios described previously. Network traffic was captured on the victim machine using the TCPDUMP (2019) tool in packet capture (pcap) format separately for normal and individual attacks to ease the labelling process for supervised classification. The TCPDUMP tool was configured to save the captured packets in 30MB chunks so as to reduce the processing load associated with feature extraction. Tshark tool (Wireshark, 2019) was deployed to extract specific packet parameters of the TCP and MQTT protocols for subsequent extraction.

MQTT feature extraction
In this phase, the custom-built feature generator module was deployed to generate flowbased statistical data. The tool extracted each flow identified through the following network traffic features: Source IP, Destination IP, Source Port and Destination Port, for each pcap file. The tool also extracted various aggregate values of the MQTT parameters. These aggregate/statistical flow features were calculated based on the parameters of MQTT sessions, such as count, size and field lengths. Specifically, two groups of statistical MQTT flow features namely, session features and MQTT packet and field length features, were generated. Session features were based on the counts of number of packets, number of control packets and number of QoS packets that belonged to the same flow. Packet size and field length features were based on the captured IP packets and the various MQTT field lengths, illustrated in Table 3. Only packet meta-data were utilized to generate the features instead of deep inspection of the payload. Hence, this feature extraction method can also be utilized on flows with encrypted MQTT payloads. In order to measure the accurate number of subscription requests per flow, the feature generation module counted the individual subscription requests and the number of topics in the request as separate request.

Attack detection module
The attack detection module of the framework is a machine learning (ML) based detection system. Statistical flow features extracted from MQTT network traffic serve as input to the classification system and help differentiate normal from attack flows, as well as inter-attack flow classification. The task of differentiating between the various flooding attacks will enable effective counter measures to be applied to thwart such attacks. The use of legitimate requests in Application layer DoS attacks can pose a significant challenge to the detection framework in differentiating between normal and attack network flows. In this study, three fundamentally different machine learning approaches namely, average one-dependence estimator (AODE), C4.5 decision trees and artificial neural network (ANN) were integrated into the detection framework. The steps followed in the detection framework to classify MQTT traffic are illustrated in Figure 3 and the three classifiers adopted in MQTT attack detection are discussed below: AODE Classifier Webb et al. (2005): The AODE classifier is a variant of the Naïve Bayes classifier that estimates the probability of the class of each output variable Y given a set of input features x 1 ,…x n . It is based on a simple Naïve Bayes classifier which relies on the assumption of independence of attributes. Assuming that all attributes are independent given the class, then Naïve Bayes can be defined as: Where C is the class label and a is the attribute. In this scenario, the computation cost is reduced; however, the performance of Naïve bayes decreases if the dependency between the attributes is high (Koc & Carswell, 2015). To counter this effect, the AODE classification technique uses a weaker independence assumption to achieve a higher accuracy rate compared to Naïve Bayes. AODE classifiers are simple to implement and provide high accuracy in classifying data. Decision Trees (DT): The decision tree-based algorithms build training data sets into the tree structure applying the information entropy principle. Each branch of the tree represents an association between the feature vector and the class label. C4.5 is one of widely used DT method which recursively partitions the training data set by choosing the most effective features to differentiate between the classes. In the first step, C4.5 identifies the best feature that can divide the data instances. In further steps, child nodes are created to divide the instances into subclasses. The attributes selected in each division point in the tree are based on the largest information gain using the best attribute. Entropy is used as a measure of information gain, calculated as follows: Multi-Layer Perceptron (MLP): MLP is a type of feed-forward artificial neural network (ANN) that comprises multiple nodes known as artificial neurons, emulating the biological neurons of brain. The nodes in the MLP are grouped as input layers representing the input features, hidden layer and an output layer. In MLP, the nodes of a given layer use activation functions to control the node's output, as well as to serve as an input for the next node. The nodes in MLP are connected by weights which are tuned by using back-propagation algorithms, that adjust the weights to reduce the error between outputs and expected results where, the error is calculated as follows: Where t i (n) is the expected output and y i (n) produced output value of the instance n and output node i.

DoS attack assessment
The individual DoS attack scenarios were evaluated on MQTT protocol version 3.  Figure 4.

Attack impact analysis
Various attack metrics were measured to assess the impact of DoS attacks against the MQTT brokers. These include: CPU utilization, bandwidth and memory utilization. CPU utilization was measured using mpstat Linux command line tool, which provides a break-up of CPU usage by various system tasks. A custom BASH script was run to measure memory and bandwidth utilization at each 1 s interval. All the experiments and measurements were repeated three times to ensure validity and repeatability of results. The metrics measured to evaluate the DoS attack impact were: . The flooding attack results achieved with maximum attack packet rate indicate that the all the three brokers suffered high CPU utilization during the various attack scenarios. The percentage of time the CPU was idle measured during the attack reduced drastically as shown in Figure 5. The VerneMQ and EMQ brokers had the maximum impact as the CPU idle percentage reached zero for more than one attack scenario. However, the idle percentage for Mosquitto broker was close to 20% and reached 0% for invalid subscription attack. The results also show that invalid subscription flooding attack caused the maximum impact on the CPU utilization as all the brokers had CPU idle percentage below 5%.
The CPU utilization break-up in Table 4 shows that during the various attack scenarios the VerneMQ and EMQ brokers spent more time in I/O Wait and application-related processing. In contrast, the Mosquitto broker spent more time in kernel functions and software interrupts. These results show that various MQTT brokers use different techniques to handle connection requests as mentioned in Karagiannis et al. (2015).
The results also indicate that VerneMQ and EMQ brokers are vulnerable to malformed MQTT packets which contain non-ASCII characters. Both the brokers suffered high memory utilization with the non-ASCII characters included in the MQTT fields. The EMQ broker had the worst performance among the brokers as it had high CPU utilization during all the attack scenarios as shown in Figure 6 and higher memory utilization compared to other brokers. Especially, the broker suffered high memory utilization during the bruteforce attack with WILL payload and can be potentially exploited to cause memory-exhaustion attacks to completely incapacitate the broker.

Attack classification results and analysis
The performance of various classifiers was tested using the Hall et al. (2009) machine learning software. The effect of the selected MQTT features on attack detection was evaluated by comparing the detection framework performance with count-based statistical features and the full feature set including the size and length based features, for both the four-class and seven-class datasets.

Dataset description
The normal and attack packets were captured separately and pre-processed to generate the dataset for testing the detection framework. Three different types of MQTT attacks were generated against the broker to capture the attack dataset namely: MQTT-DoS (based on the attack scenarios described in this work ), MQTT-FUZZER (using a MQTT Fuzzing tool Vähä-Sipilä, 2015) and TCP-DOS (using hping3 SYN-Flood tool Sanfilippo, 2006). The MQTT fuzzing tool was configured to send fuzzed packets sniffed from the deployed IoT network. Based on the class labels, two datasets were generated, fourclass and seven-class dataset. The class labels in the four-class dataset were: Normal, MQTT-DOS, MQTT-FUZZ and TCP-DOS. In contrast, the seven-class dataset contained four sub-classes of MQTT-DoS attacks presented in this work namely:MQTT-DOS-BF1, MQTT-DOS-BF2, MQTT-DOS-BF3 and MQTT-DOS-IAUTHS. The seven-class dataset was used to evaluate the detection framework performance in detecting the four attack types discussed in this work. The total number of flows in the four-class dataset was 1,012,052 samples and 1,042,500 for the seven-class dataset. Re-sampling technique was applied to balance the classes to avoid bias in classifier accuracy. The data were under-sampled to produce random sub-samples of the original dataset with following setting in Weka: biasToUniformClass=1.0, noReplacement=True, sampleSizePercent=40.0. The break-up of classes in original and re-sampled datasets is presented in Table 5.

Detection metrics
The performance of the detection framework was evaluated by conducting several experiments with the two datasets described in the previous section. The individual classifiers performances were measured to assess the effectiveness in detecting anomalous MQTT traffic. The following metrics were adopted to evaluate the performance of detection framework: Detection Rate (DR), Accuracy and False-Positive Rate (FPR). These metrics can be calculated by measuring the True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN) from the number of correctly and incorrectly classified instances. TP is the number of correctly detected anomalous instances in the dataset. TN is the number of correctly detected legitimate instances. FP is the number of normal records classified as anomalous while the FN is the number of anomalous instances classified as legitimate. Accuracy (ACC) is the percentage of instances correctly classified as either anomalous or legitimate and it is calculated by: True positive rate (TPR) measures the percentage of instances correctly classified as anomalous and is calculated by: False-positive rate (FPR) is percentage of instances incorrectly classified as anomalous and is calculated by: Several experiments were conducted to measure the performance of AODE, DT and MLP classifiers used in the detection framework. A 5-fold cross-validation method was enforced which trained and tested the machine learning models on complementary subset of data to prevent bias and over-fitting issues. Weka settings used for A1DE classifier are: frequen-cyLimit=1, subsumptionResolution=False, weight=1.0, weightedAODE=False. The J48 DT algorithm an implementation of C4.5 classifier in Weka was configured with the following setting: confidenceFactor=0.25, minNumObj=2, reducedErrorPruning=False, unpruned=False, useLaplace=False and useMDLcorrection=True. The default settings of Weka MLP classifier used in this work was: batchSize=100, decay=False, hiddenLayers = (attributes + classes) / 2, learningRate=0.3, momentum = 0.2, trainingTime = 500.

Detection results
The evaluation results of the classifiers used in the MQTT attack detection framework are presented in terms of the accuracy (%), error (%), TPR and FPR and time to build the model. The classifiers were evaluated with both count-based (counts) features (statistical MQTT session features presented in Table 3) and with full features set. The performance of the three classifiers on the four-class and seven-class datasets respectively, is presented in Table 6. Figures 7 and 8 show that the AODE classifier achieved the highest classification accuracy in detecting the attack traffic for both four-class and seven-class datasets.  The results also indicate that the detection accuracy of classifiers increased when packet size and field length based features were considered along with the countbased features, when compared to using only count-based features. The AODE classifier had the lowest and MLP classifier had the highest training times among the selected classifiers. These results indicate that the proposed MQTT features provided good separation between normal and attack traffic resulting in high detection rates and low false positives. However, the MLP classifier only achieved a classification accuracy of 84% for the sevenclass dataset, when all the features were used. Hence, the MLP classifier was further evaluated with various optimization parameters to identify the most optimal settings to increase its detection performance. The optimization parameters considered in this study for improving MLP classifier were: activation and solver functions.
In an ANN, an activation function of a neuron maps the input signal to an output signal. Choosing the correct activation function supports the MLP classifier in generating more accurate and complex non-linear mappings between the inputs and outputs, hence improving the classifier accuracy (Karlik & Olgac, 2011). The solver functions refer to algorithms that try to estimate the optimal weights for the hidden and output layers in order to reduce the training errors. These are classified into first and second order methods and vary in computation complexity when they are minimizing or maximizing the loss function. Since Weka does not have options to vary the activation and solver functions, the MLP optimization parameters was tested using Python scikit-learn ML platform (Pedregosa et al., 2011). On this platform, the performance was evaluated with three activation functions: Relu, logistic-sigmoid and tanh. In addition, two solver algorithms: Stochastic Gradient Descent (SGD) and limited-Memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) were compared for their optimization performance in tuning the ANN weights for the seven-class dataset. The SGD algorithm uses learning-rate and momentum to optimize the model by iteratively estimating the training loss using samples from the training dataset. The optimal momentum and learning-rate were identified by iteratively varying the two variables and the settings, selecting the least training loss for evaluating the activation functions. Figures 9 and 10 show the observed training loss of the MLP classifier for various values of momentum and learning rates, respectively. These results show that a momentum of 0.9 and a learning-rate of 0.001 yielded the least training loss. Furthermore, Figure 11 shows that the performance of MLP classifier achieved a higher detection accuracy with relu activation function compared to logistic-sigmoid and tanh activation functions when applied to both SGD and L-BFGS optimization algorithms.

Conclusion
In this work, a DoS attack detection framework for MQTT attack detection in IoT environments was proposed and evaluated. The attack detection testbed was designed to capture normal and attack traffic, and count-based statistical flow features. In addition, MQTT control packet field size/length feature sets were evaluated on two datasets. The effectiveness of the proposed feature set was validated using three fundamentally different machine learning algorithms namely, AODE based on Naive Bayes, C4.5 based on Decision Tress and MLP based on ANN. The performance of the classifiers were tested with countbased flow features and field length features, to measure the detection accuracy of normal and attack classes. The MQTT DoS attack modelling results indicate that the adversaries can cause large scale impact with just basic access to the MQTT broker by launching the invalid subscription flooding attack. However, the invalid authentication attacks were found to cause little impact with a single attack source machine, as these attacks depended on a large volume of attack packets. In addition, using a malformed CONNECT request, a high memory utilization on broker machines was witnessed, which could be exploited during a memory-exhaustion attacks. The DoS detection model showed that the proposed MQTT features yielded high detection capabilities, especially when the control packet field size-length based features were selected. Hence, the packet size and field length distribution features can be effectively used in detecting DoS attacks in IoT networks. As future work, we intend to study a real MQTT dataset to assess its features, and how these can be beneficial in clearly de-marking legitimate and malicious MQTT traffic.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Notes on contributors
Syed Naeem Firdous is currently a PhD student at Edith Cowan University, Perth, Australia. He received his Bachelor of Engineering Anna University, Chennai, India in 2005 and MS from King Fahad University of Petroleum and Minerals, Dharan, Saudi Arabia in 2011. He has both academic and industry experience in networking and network security domain. His research interests include IoT, network security, and network forensics.
Zubair Baig is a Senior Lecturer in Cyber Security at the School of Information Technology, Deakin University. He has authored over 65 journal and conference articles and book chapters. He is currently serving as the editor of the IET Wireless Sensor Systems Journal and the PSU -A Review Journal, Emerald Publishing House. He has served on numerous technical program committees of international conferences and has delivered numerous keynote talks on cyber security. His research interests are in the areas of cyber security, artificial intelligence, smart cities and the Internet of Things.
Dr Ahmed Ibrahim is a Lecturer (Computing and Security) at the Edith Cowan University School of Science. Dr Ibrahim's Ph.D. research focused on detecting covertly hidden content in digital objects and electronic communication. He has previously worked as a Post-Doctoral Research Fellow at the ECU Security Research Institute and has held academic positions at the Maldives National University, Villa College, and National Institute of Technology Australia; and worked in the government of Maldives.
Professor Craig Valli has over 30 years experience in the ICT Industry and consults to industry and government on cyber security and digital forensics matters. Along with being Director of the Edith Cowan University Security Research Institute, he is a Professor of Digital Forensics, a Fellow of the Australian Computer Society and Director of the Australian Computer Society Centre of Expertise in Security at ECU. Craig is a member of the High Tech Crime Investigators Association (Australian Chapter) and the INTERPOL Cyber Crime Experts Group. Craig has over 100 peer reviewed academic publications in cyber security and digital forensics. He serves on several cyber security related conference committees and journal editorial boards. Craig is also the founder and current Chair of the Australian Digital Forensics Conference.