A total of 51 logs are sending data simultaneously at 1 second intervals to our MQTT server on AWS. For each log, there are currently 3 data points being sent at the same time. loglooper was updated to receive file, device id, and date through command line arguments, since there’s 51 logs from 2018’s field tests to go through. Created a new python script to go through the directory of the data files, then generate a bash script that will launch forever to start or stop the scripts. Some experimentation was done on timing between calls to forever. On the computer, it works fine with 0.5 second delay between calls. However, on the server, sometimes the EC2 instance becomes stuck. When that happens, no messages are being sent or received on MQTT. A solution to this would be to batch the script in logs of 10 or so. Anyway, all 51 are running now on my computer sending the data to the MQTT server. It’s going to be real interesting to see if it will crash. Yesterday was mainly searching for the log files, and wasn’t enough of an update to post yesterday. The next steps are to display the dots — all 51 — on the map.
Check out this video snippet to see the activity:
Here’s a glimpse at the EC2 instance during testing, let’s see how it responds after some time
There’s now a dot being displayed on the map corresponding to the data received through MQTT on the AWS server. The loglooper of Bowie’s data, which sends the MQTT messages, is running continuously on the AWS server too. There was a minor change in the topic structure. Now the sensor value falls under air, water, or soil. The number of data points in each of those topics will be added to a counter which will eventually be displayed along the bottom of the page. The next step is to find a better data structure for multiple devices. The data structure should be chosen with the future possibility of dynamically loading / unloading based on the map position. After this, will be prototyping and testing sending data from an IoT device *live* in the field.
Derived a python program from convert_data.py to loop through an already processed Bowie log file, called loglooper. When it processes a line, it also sends the data to the MQTT broker. There’s a 1 second delay per line. The server was able to receive the data perfectly. The next step was to try to get this program to run continuously using nodejs forever. Originally encountered some difficulties related to the spinSleepTime and minUptime, which were changed from 1000 ms to 5000 ms. It’s now working continuously with forever. The next step will be to receive the data on the map and draw a dot. As well as transferring the python loglooper to the server to run on AWS.
Today the map was created from previous code experiments with mappa.js that Robot Missions worked on in 2018 and 2019. This uses AWS, MQTT, p5.js, mappa.js, and Mapbox GL. The version of Mapbox GL library was changed in mappa.js from v0.43.0 to v1.12.0. Solved a lag issue with the points not following the map by only redrawing when the map is moved. Started a p5.js simple MQTT template example, and tested with our MQTT broker on AWS. Was stuck for a while on not being able to connect, this was due to WSS instead of WS. The next step is to connect MQTT to the main map.
Today MQTT was integrated into the map, and the basic MQTT topic structure was decided. Basically, devicename/latitude, with the data being that value. The navigation controls were added to the map by modifying mappa.js with the correct Mapbox GL function. Tests were completed to detect the lat & lon from mouse location, the current zoom level, and if that lat & lon is visible. This will be useful for unloading / loading devices. The next step is to make a python program that loops through a Bowie log and sends the data over MQTT to AWS.