Select Page
Tech Log #014: 51 logs simultaneously

Tech Log #014: 51 logs simultaneously

A total of 51 logs are sending data simultaneously at 1 second intervals to our MQTT server on AWS. For each log, there are currently 3 data points being sent at the same time. loglooper was updated to receive file, device id, and date through command line arguments, since there’s 51 logs from 2018’s field tests to go through. Created a new python script to go through the directory of the data files, then generate a bash script that will launch forever to start or stop the scripts. Some experimentation was done on timing between calls to forever. On the computer, it works fine with 0.5 second delay between calls. However, on the server, sometimes the EC2 instance becomes stuck. When that happens, no messages are being sent or received on MQTT. A solution to this would be to batch the script in logs of 10 or so. Anyway, all 51 are running now on my computer sending the data to the MQTT server. It’s going to be real interesting to see if it will crash. Yesterday was mainly searching for the log files, and wasn’t enough of an update to post yesterday. The next steps are to display the dots — all 51 — on the map.
Check out this video snippet to see the activity:
Here’s a glimpse at the EC2 instance during testing, let’s see how it responds after some time
Tech Log #011: Loglooper to MQTT broker

Tech Log #011: Loglooper to MQTT broker

Derived a python program from convert_data.py to loop through an already processed Bowie log file, called loglooper. When it processes a line, it also sends the data to the MQTT broker. There’s a 1 second delay per line. The server was able to receive the data perfectly. The next step was to try to get this program to run continuously using nodejs forever. Originally encountered some difficulties related to the spinSleepTime and minUptime, which were changed from 1000 ms to 5000 ms. It’s now working continuously with forever. The next step will be to receive the data on the map and draw a dot. As well as transferring the python loglooper to the server to run on AWS.