Detecting birds with the Raspberry Pi
Creating a pipeline to detect birds on my balcony
Project details
Project Requirements
- Set up a system that takes pictures to gather data.
- Create a model that seperates pictures of birds from accidental images of other objects.
- Save images to a database and add valuable attributes
- Visualize the data to discern patterns
Project Overview
One of my favorite things to do in my free time is bird watching. Unfortunately, I can’t spend as much time outside as I would like. Having a balcony instead of a garden doesn’t help either. That’s why I installed a feeder on the outside of my window and a Raspberry Pi with camera on the inside. I now get daily pictures of birds that come visit me. But I want to do more than just save and watch photos. In this project, I will build a pipeline that decides if a bird is captured on camera with a convoluted neural network and stores the data in an organized database. Perhaps this can give some insights in the daily lives of urban birds.
I will need data and train a model that will recognize birds instead of cars for example. Otherwise, I have to sift through the images manually. The Raspberry Pi will use a models made by Microsoft’s Lobe1 to filter out all photos without. Results will be stored in a neat manner afterwards. As a final step, the pictures will be subdivided in visits. The process looks like this:
The Data
Over the course of a year, I took pictures of birds that came to the feeder. I used the PI-TIMOLO2 library on the Raspberry Pi 3B in combination with the camera module. Every time movement is detected, a picture is taken and stored. After a year of storing pictures the dataset consists of 994 images of birds and 994 images of “non-birds”. Even though the script can be configured to only take photos of fast moving and large (close) objects, false positives can’t be entirely prevented. Cars and other vehicles are often captured as well. The first model will only be trained to recognize a bird. This will make the subsequent second model to determine the species more accurate. In the next steps, I will use Lobe, Python and MySQL to load and prepare the data, train and deploy the model and save the pictures.
In comparison to big datasets, 1988 pictures isn’t that much to train the first model with. Lobe made the dataset larger with a technique called image data augmentation. Every picture is somewhat modified to expand the amount of data available. For every image, Lobe makes five random variations.
Step 1 - Separating birds from the rest
Now that I have a lot more data, it’s time to build the first model. This is the whole ‘brains of the operation’ that will decide if something is a bird or not. The photos, separated in directories for each class, are loaded into Lobe. The software start training directly after importing the data. While it’s great that the software does most of the work, most autonomy is lost. The inner workings are predefined and not customizable. The only parameter I have influence over, is whether to focus on accuracy or speed when building the model. Since it’s more compatible with the Raspberry Pi, I chose the latter.
The result was a model with an accuracy of 97%, which is excellent! Lobe gives a neat overview of the achieved accuracy and shows which images are correctly/wrongly classified in the training data. I exported the model in the TensorFlow Lite format so it can be used with the lightweight and less bulky version of TensorFlow3. Lobe also provides a neat module for Python to use the model and categorize new images4. When an image is taken, it is passed to a Python script that uses the model to make a prediction. The prediction with the highest confidence is returned along with the accuracy. Based on this information I can make a decision on how to handle each photo in the next phase.
Step 2 – Storing the images
When the Raspberry Pi takes images of identified birds, every separate photo is stored in a MySQL database. For each file the name, date and time and the accuracy of the prediction are stored. I chose to store the names of files (which are unique) rather than the images themselves, and let other programming figure out the logic when retrieving raw data.
Photos are taken in succession if a bird keeps moving around so multiple images can belong to the same visit. Every night a Python script runs to divide images into different visits. These have a time of arrival, time of departure and the length of the visit in seconds. Photos that belong together get the same ID to keep them apart. A visit consists of images that are taken within 20 seconds of each other. The database used looks like this:
The ‘Species’ table and other columns related to it are currently not in use. I plan to make a second model that recognizes the species of bird one day. This is a future project but I already added it to the database design. Photos without birds are not that interesting but not entirely useless. They will not be saved in a database but moved to a different folder and not deleted. This data can be used to someday make a new version of the model, but with more data.
Step 3 – Visualizing the data
The last step is to analyze and visualize the data. A graphical representation makes it easier to spot patterns. Please note that up to this point, the diversity in species is not high. Only the Eurasian Blue Tit, Great Tit and a single European Robin visited the feeder. The data to this date only represents these species. Let’s look at some basic facts about the data and visits thus far:
| Mean | 11,90 seconds |
|---|---|
| Max | 128 seconds |
| Min | 4 seconds |
The average visit last only a couple of seconds apparently. Birds, cautious by nature, don’t take the time to enjoy a meal and quickly move on instead. Since the camera can only take a single photo every four seconds, I adjusted the length of visits to this. An image of a higher resolution takes a bit longer to capture. This is a conscious tradeoff between having a slightly less accurate reading on the length of visits, but also having nice pictures to look at.
I started off by visualizing the data as a time series. A line shows the increase and decrease of the sum of visits per day and grouped by month. (click on a graph to enlarge)
This chart shows that the data is irregular and that some months have no entries at all. This is caused by downtime for maintenance on the Raspberry Pi, figuring out the correct settings when starting the project and forgetting to refill the feeder. I concluded that more data with coverage on all months is needed. In the meantime, I will use averages for the data I do have.
Next, I made bar charts of the average amount of visits per month and hours in a day with Matplotlib5. This gives some more insight in the lives of birds when it comes to foraging.
The left graph shows average visits per month. As expected, birds look for extra food during the winter. In these months, food from natural resources becomes more scarce and a helping hand is required. Less expected is the increase of visits in May. This makes sense because there are more mouths to feed since most younglings hatch during this period. The sudden rise in visits is surprising however. Gathering more data will tell if this is a gradual increase in reality, or indeed an abrupt boost in visits.
The bar chart on the right shows the average frequency of visits for every hour of the day. This gives a more detailed view of foraging patterns. It shows two distinct periods in which birds are active and looking for a meal. The first one start in the morning and peaks just before midday. After some downtime during the afternoon, birds return for a late night snack before having a good night’s sleep. This is a broad assumption based on an average spanning almost two years. Maybe visiting hours differ when looking at individual months. This can be visualized with a heatmap. The darker the color, the more frequent the visits are.
This indeed shows that birds visit at different hours based on the month. My assumption is that they look for food during the day and don't forage at night to sleep6. This heatmap clearly shows, in terms of daylight, shorter days in winter and longer days when it's (almost) summer.
What's next?
These findings conclude the project for now. The Raspberry Pi can now automatically gather new data and add it to the database. The next step is creating more fluid graphs that will update when new images are added. A way of getting daily reports would also be nice. The more the dataset grows, the more detailed the insights will be.
Another step is adding a second model to recognize the species of bird. This will add even more value to the dataset as different kinds of birds might have different patterns when looking for food. This will certainly be a challenge since a lot of data is needed to train this model. Unfortunately, I can’t control the species of bird that come visit me either. When I get to this, it will be added to my portfolio as a different project.