Select Page
Tech Log #006: Tagging images

Tech Log #006: Tagging images

Today we took a look at some of the old imagery data from Bowie from Summer 2018. There still needs to be work done on this, such as going through all the photos and tagging them if they include a subject of interest. There were two robots collecting imagery data that season. To start off, we worked on a dataset from OG Bowie’s groundcam: 09-08-2018_11-09 (2pm field test). This was Sept. 8th, 2018 – we had a Field Test, then had the beach closing party at Westboro Beach.

65 images were tagged out of a total of 2299. The items included bark, cigarette butts, feather, glass, leaf, misc, pebbles, plastic, seaweed, stick, stone, twig, and twigs.

Tech Log #005: AWS IoT: integration with Bowie API

Tech Log #005: AWS IoT: integration with Bowie API


Check out the video (not sure why the link doesn’t go properly, but the video is also embedded above)

Now we need to combine the test scripts we’ve been writing with Bowie API. The Bowie API takes a payload sent over serial from the microcontroller to the Raspberry Pi. It’s lightweight – just two key value pairs and some categories to help organise it. Here is how the messages are sent. Here is a look at the API, but it has since evolved since then.

We figured out that you don’t need to upload and download the entire json dict each time. There’s a way to only see the changes.

Something we got stuck on is formatting the json dictionary to be proper for the updates. This is an example of what the json dictionary could look like. Eventually playing with the braces and quotes it worked out. The value has to be formatted as a string.

Integration time! Time to run the script on the Raspberry Pi. It was cool to boot up this Raspberry Pi again, it was entirely the same as Summer 2019. It will be fun to revisit some of those scripts at some point in time, but not right now. Installed the Python AWS IoT SDK.

Installing dependencies:
pip3 install paho-mqtt
pip3 install schedule
pip3 install pyserial

Dun dun dun … and now a huge bug. We weren’t receiving serial data from the Teensy 3.6 to the Raspberry Pi. To debug it, we checked the function was being called in the Teensy. We made a simple test script that only prints out the data received on the Pi. We checked the port configuration and Serial was enabled. We rebooted it. The other thing to check was to verify the data was coming out of the Teensy. Tried to connect it to an FTDI adapter, but still the computer’s drivers issues were there, and rebooting was not an option to make it work. So it was searching endless forum posts to find any information or things to try…

The bug ended up being the name of the serial port in the RPi code. Not entirely sure why this would have changed! But we had to change the serial port in the code from /dev/ttyAMA0 to /dev/ttyS0.

So now it is working for uploading data to AWS IoT. We will have to work on making it receive data, that will be for a future step. Also, found a big guide on everything AWS IoT.

Tech Log #004: Shadow device json vs vanilla MQTT

Tech Log #004: Shadow device json vs vanilla MQTT

Carrying on from the previous tech log. The example we are using also uses a Lamda function to send a log note. In the future, Lambda functions can be used as “glue” to pretty much anything. Something to explore in the future when there’s more time available are SNS rules. This would let us send an email when a value is at a certain threshold. Here is a cool example of this.

We accomplished a lot of things working:

We played with figuring out what Shadow can do and not do. We looked at the pro’s and con’s of both:

We will be going the Shadow route, because it gives is the capability to use Lambda functions, and the persistence store of the data can be useful in the future. So this tech log #004 and #005 are blurred a little bit because a late night was spent tinkering around with this. With sleep deprivation hitting full swing we hope some of this is coherent.

Some of the time this work session was spent finding the documentation. We were trying to find more information about Shadow updates vs vanilla MQTT. All of the AWS IoT SDKs can be found here. If you’re reading “SDK” for the first time, it means software development kit. They usually consist of libraries, example code, and helper functions – so that you can get going on whatever you need to build with that particular service quickly.

Eventually it will be interesting to try this and see if the certificate conversion with pycert will work properly. The link to the example code in that tutorial doesn’t work, but it can be found here. Something else that we found, there’s a cool demo of robots communicating with each other to accomplish a task in this video:

Tech Log #003: AWS IoT hello world!

Tech Log #003: AWS IoT hello world!

It was time to get going on working on getting connected with AWS IoT. This is going to be fun! We are going to connect Bowie’s brain to the cloud. We have done some things previously, such as connecting Bowie to shiftr.io MQTT. This works great, but in order to fully fulfil our vision of the robots working together in teams, we will need the data to be connected to many different services.

Starting can be intimidating especially if you aren’t familiar with AWS IoT. Here is the tutorial that we followed step by step. There were some errors to fix along the way but it makes sense once you start un-puzzling it. Going through this step by step was immensely helpful to understand. Big thanks to soumilshah1995 on YouTube for making that tutorial.

After many hours debugging and tinkering, we made it work. We are seeing data in the AWS cloud! A step closer to getting Bowie connected. Next tech log will make further progress on this.

Tech Log #002: Frozen computer, partial transfers

Tech Log #002: Frozen computer, partial transfers

This is a slightly weird log entry, because not only did everything we do not end up working, it also ended up going slightly backwards more than forwards. But that’s alright – as astronauts say, “there is no problem so bad you can’t make it worse”.

Some time went in to searching for a 900MHz Xbee that is missing, it was in the orange operator interface that is usually paired with OG Bowie. Been looking everywhere but can’t find.

Brenda helped transfer the routerpi image. This lets us use a Raspberry Pi in a special configuration that she worked on to become an access point. It’s 16gb, so there wasn’t a lot of available space to store it. Ended up storing it on a usb key, but it wasn’t readable on anything other than linux. Went to use a separate RPi to read it, but its sd card must have been damaged, so had to reflash it. There wasn’t enough space on my computer to do that, so then had to find a different computer to do so.

Eventually the RPi was up, and we could read the contents of the usb key. Tried to transfer the file, but there wasn’t enough space available. The next guess was to transfer via sftp to my computer. However, then my computer became low on memory. This was happening simultaneously as I was trying to code something for the green foam board Bowie unit.

My computer froze since all space was used up, so all the progress was gone anyway, and all of the windows I had open and code ended up being lost, because a hard shut down was needed. Rebooting the computer worked smoothly, that was a positive note. Started an upload of the file to google drive, and this transferred successfully.

Time to regroup for tomorrow and come back at it with a clean slate. Oh yeah, and the search for the 900MHz Xbee will continue.

The tech logs will be more productive in the future. This just happened to be an anomaly.