IoT Based Electricity Analysis by Machine Learning
This is an academic project I did in my masters with my colleagues it is basically an application of IoT and Machine Learning.
Personal Report on
Electricity Surveillance based on LoRa
Project | Master of Computer Science for Aerospace
11th April, 2019
Personal report by: Rahul Dhole
Ibtissam Elabbadi, Ahmed Zrigui, Nune Ghazaryan, Aigerim Aibatbek, Khachatur Sedrakyan
Supervised by: Prof. Rahim Kacimi
Traditionally there is only one method to keep eye on consumption that is electric meter. Generally, this electric meter can only show the power it cannot identify which component is plugged-in. Imagine if you can identify the connected electric component. It can be useful in some areas where the electricity access is restricted or limited for example government offices, hostels, rental residences. This project is about classifying such devices and integrating it with the LoRa IoT system.
There are two main goals of this project but I am working on LoRa network integration part:
Identifying the plugged-in electrical component by the rate of change of their electricity consumption
Integrating this sensor with LoRa based IoT system
We started our project on the 26th of September 2018. At the beginning of the project, we planned to work around 100 hours for the whole project. The project duration for first semester was 40 hours in 14 weeks, means on average 3 to 4 hours per week except three weeks of holidays. However in reality, for some weeks we worked more or less than planned by us and academic schedule.
Semester I - Management:
On the first day, when the groups were formed I took initiative to be project manager since I have last year experience of working on software tools and project in M1. And after when I all group members became convinced and satisfied with our respective roles we were seeking to have first appointment with our advisor Mr. Rahim Kacimi, for that I sent him first request on 26 Sep 2018 but he couldn’t reply so I met him unofficially after finishing his lecture on 27 Sep 2018 just to get some initial guidelines to start the project and requested to set an appointment, and on 28 Sep 2018 I again sent one reminder.
First 8 Hours has spent to form a team and to have appointment with advisor, but meanwhile, we were studying on "LoRa networks and LoRaWan protocol Stack." as per the advisors guidelines. On 29th Sep, 2018, until we get the next appointment I have decided to read the basic things which are written on the project abstract provided by supervisors.
The project has divided into four topics and divided among four team members to read one topic each.
1. Lora networks - by Khachatur
2. LoRaWAN protocol Stack - by Ahmed
3. Packet Lora forwarder - By Ibtisam
4. TCS tool - by Rahul Dhole
After reading these topics individually each member will share only important resources for saving the time of other team members. Individually we spent around 5-6 hours each and 2 Hours together to explain each others so in total every team has spent only 8 hours and it could be around 24 hours but this is how we saved 18 hours of each member. I had problem to find information about TCS tools so spent quite less time on it but my other team members has spent more or less time so it was balanced and we can say we did great team work by saving 18 hours. We did combine reading on 1 October 2018. Then after we had five times appointment with supervisor. And more than 10 team meetings to work together.
Management Softwares: To manage all this tasks I have introduced Trello to my team. For organizing the research papers referred we are using Zotero.
As a project manager and I have responsibility of communication with supervisor and team members I was reminding team members for every meeting before one day o meeting, an hour before meeting and tracking there details if someone has problem to arrive on time or not sometimes it has happened that somebody came late or didn’t came so at that time as a manager I have been updating missing member about what was happened in the meeting or in the appointment with supervisor. While deciding the appointment with professor sometimes we are facing problem because of our busy and different schedule with professor so on that point professor gives two three choices to us and by coordinating with others I finalise the date. And also if someone doesn’t responds on messaging I have been sending reminders everyday until they don’t reply.
In this semester, our one group member Khachatur has left the group because he was an Erasmus Exchange students for one semester only. In this semester we got new two team members Nune ghazaryan and Aigerim Aibatbek. And our advisor proposed an electricity surveillance application to be built on this LoRa network. Aigerim and Nune got assigned the module of classification of various data using machine learning techniques. Rest of us we continued our roles.
In this semester we had around more than 10 appointments with professor you can see more about our timeline in Gantt chart provided in presentation. After few meeting we got clear with the directions of this project and we started working. We had divided our work among team members after discussion.
Nune & Aigerim - Reading Classification of Electricity Data
Ahmed - Website and Software tools
Ibtissam- Working on LoRa transmitting code and merging it with modbus code
Rahul - Generating graph by reading the MQTT topic, storing into MySQL and showing on grafana
Initially, we spent a lot of time in working with the wrongly configured gateway, it was an technical issue but we solved this issue soon by contacting with the network administrator Mr. Froncoise Thebolt. He provided us special configuration for this project and allocated a MQTT topic and Device key on new gateway. But this wasn’t the end of risks, later we faced many problem with establishment balanced of connectivity of with gateway. Finally we were able to connect for some time and we took some sample MQTT packets received on Node Red. Now because of end of semester constraint we don’t have much time to conduct our last test, to video record it in actual. But we got the whole LoRa network working and generated graphs as results.
Technical Contribution (Semester I)
All integration and communication of LoRa with Gateway through VS Code to Node Red we did it together in the project. So here I will only talk about my solo technical contribution rest of integration of LoRa, MultiTech Gateway, Putty, Raspberry and NodeRed you can refer our group report of this project in first section of 7th chapter.
Since my master thesis also has study of LoRa I was studying the LoRa pycom for it and found Pymakr plugin which can be used with Visual Studio editor to run the python script containing LoRa node commands to be executed through serial port of laptop.
Currently we are trying to decode the payload coming from LoRa node then after we will be able to transfer the payload message to the web server.
This is not our main goal, our main goal is to send data to the MQTT server. For that we are using neocampus server but I have been also reading about other MQTT server Mosquitto. Currently we are using MQTT box to establish connection with neocampus’s MQTT server
Technical Contribution (Semester I)
My team members part is to read sensor data process and send it on Gateway where the MQTT server publishes this packets.
My part begin after publishing the MQTT topic (topic name is device key).
I have created my own local wifi network (with internet access) by installation of following servers:
Node Red (with MySQL and MQTT extensions)
MySQL and PhpMyAdmin
Fig 2.1: An overview of whole system
1. Node Red Flow:
I have created a Node Red flow receiving the MQTT topics TestTopic/lora/# and then we are parsing in a parser node as per the receiving payload format.
After parsing data I have embedded this parsed data into INSERT query of MySQL and run by the next MySQL node.
So now, whenever modbus will take new reading here it will be updated into database. We can also use remote online database for making it reachable outside local network.
2. Grafana Configuration
Grafana is a simple too allow graphical visualization of database values without writing the code.
After giving authentication of MySQL database it fires a query to read the values from database and shows it in graph as shown below.
3. API Interface
I have successfully managed my team as a team leader and I took wise decisions to reach goal, and maintained a good communication with everyone. And technically I have completed my graph visualisation part with extra API testing for future.
“MultiTech Developer Resources » LoRa Network Server.” [Online]. Available: http://www.multitech.net/developer/software/lora/lora-network-server/.
“Architecture - LoRa Server, open-source LoRaWAN network-server.” [Online]. Available: https://www.loraserver.io/overview/architecture/#lora-gateway.
“TTN Packet Forwarder,” The Things Network, 17-Dec-2018. [Online]. Available: https://www.thethingsnetwork.org/docs/gateways/packet-forwarder/ttn.html.
“Semtech UDP Packet Forwarder,” The Things Network, 17-Dec-2018. [Online]. Available: https://www.thethingsnetwork.org/docs/gateways/packet-forwarder/semtech-udp.html.
“On the Limits of LoRaWAN Channel Access - IEEE Conference Publication.” [Online]. Available: https://ieeexplore.ieee.org/document/7810745.
Free Books Online, Internet of things LoRaWAN Network Architecture 5 5. .
“LoRaWAN,” The Things Network, 17-Dec-2018. [Online]. Available: https://www.thethingsnetwork.org/docs/lorawan/.
“LoRaWAN simply explained | Jensd’s I/O buffer.” [Online]. Available: http://jensd.be/755/network/lorawan-simply-explained.
“Mbed OS Documentation | Tutorials - Building a private LoRa network.” [Online]. Available: https://os.mbed.com/docs/latest/tutorials/LoRa-tutorial.html.
“White Papers | Resources | Semtech LoRa Technology | Semtech.” [Online]. Available: https://www.semtech.com/lora/resources/lora-white-papers.
“Videos | Resources | Semtech LoRa Technology | Semtech.” [Online]. Available: https://www.semtech.com/lora/resources/lora-videos.