Earlier in the year — September 2018 — the Sheda team got to attend the IoT Impact conference in Sydney. We wanted to create an engaging and practical demo of what Industrial IoT or Industry 4.0 can be used for. After much brainstorming and deliberation we decided to create an Internet of Things demo that would show how you could use Industry 4.0 technology to deliver value in a food manufacturing, in this case using these technologies to sort M&Ms by their colour.

In this post I’m going to walk you through how we used Sheda’s delivery process, a Raspberry Pi with computer vision, an iOS app, Google Cloud technologies to create a scaled down example of an Industry 4.0 application built with Lego pieces because..Lego is awesome that’s why.

http://www.freebiechest.com/IoT and Lego = Awesome. Source:

Before I get into the details what is IoT and Industry 4.0?

Sheda uses Design, Artificial Intelligence and technology to solve complex problems with an emphasis on delivering solutions that make and accelerate impact and bring about social change.

What is IoT or The Internet of Things?

The Internet of Things or short IoT basically refers to all devices and sensors collectively known as “things” connected to the internet. That can be anything from your smartwatch used to track your health, your internet connected car finding the quickest route using realtime traffic, or to the smart bulb and smart speakers in your home. Things usually refer to any device that is not a desktop computer, laptop or smartphone that is sending and/or receiving data to/from the internet, all these things are part of the Internet of Things.

What is Industrial IoT or Industry 4.0?

The examples I gave above are related to consumer based IoT, but IoT doesn’t stop at the consumer level. The World Economic Forum estimates that by 2020 more than 50 billion things, ranging from cranes to coffee machines, will be connected to the internet.

It’s clear that sooner rather than later IoT will enter almost every area of our lives, as more machines in manufacturing and production industries are getting connected to each other over virtual networks there will be a shift to having whole production lines powered by machines communicating with each other in real-time to create value and deliver products and services. This type of IoT is commonly referred to as Industrial IoT (IIoT) or Industry 4.0.

How do you make Industry 4.0 accessible?

From speaking to our customers in manufacturing especially in the food and agriculture space, we found that most people have heard of the buzz word Industry 4.0 but couldn’t envision how it would apply in their industry. We wanted to put together a demo that showcases automation powered by internet connected devices aka Industry 4.0.

Since a few of Sheda’s IoT customers are in the agriculture space, we decided that our demo should have some relevance to agriculture and food processing, like these crazy tomato sorting machines.

Our Brief

The team agreed that whatever we came up with had to be fun, spark the imagination, be engaging, easy to assemble, disassemble, be accessible and most importantly be able to be completed within our constraints of time and budget.

Sheda’s IoT M&M Sorter

In order for our colour sorter to be a truly interactive IoT setup, we created a native iOS app that you could use to control what colours get sorted and used Google Cloud’s RealTime database to develop the backend. To cut down development time, we picked the best rapid prototyping material commercially available: LEGO!

A bit of research in Lego Great Ball Contraptions brought up some inspiration like this fantastic post.

Here are a few other M&M sorters other people built:

Sheda’s IoT M&M sorter was designed to be an M&M contraption with a twist. A camera connected to a Raspberry Pi will use computer vision to detect the colours of M&Ms and trigger a servo motor connected to a lever. If the M&M matches the colour we’ve set the machine to sort, the lever will move and extract the M&Ms from the system.

Being a true IoT device the sorter would receive the current colour from our internet connected cloud database in real time. The companion iOS app would be used to change the colour of the M&M’s to be sorted.

Sheda’s IoT M&M sorter


Ingredients

The sorter consisted of

Once we got our Lego parts, we assembled a prototype to test how the sorter will work. Here are our first trials.

First rolling experiments

Pro tip: Crispy M&Ms have the most spherical shape, so they roll the best.

Putting it together

We got the team together and started brainstorming ideas on how to assemble our sorter.

Team Sheda

We opted for a conveyor belt system to transport the M&Ms to the top of the contraption, at the top they M&Ms will roll pass the camera and selection gate. After a few hours of tinkering, we had our first M&M contraption going.

First conveyor prototype

While this structure worked we had to make lots of tweaks around the feeder to make sure M&Ms don’t get stuck inside the conveyor belt and don’t block each other. The picture below shows the final structure we devised to allow a smooth ride for the M&Ms to go up the conveyor belt and ensure that not too many M&M were picked up at the same time.

M&M on the conveyor belt

Giving it eyes

The Raspberry Pi and camera were mounted onto the sorter and aim at the rail to detect the colour of the M&Ms rolling down the from the conveyor belt.

Raspberry Pi camera to track M&M colours, with Chewbacca keeping watch

Lachi Agnew one of Sheda’s software engineers developed the computer vision colour detection and servo triggering mechanism. Read how he developed a computer vision system to detect M&M colours in the linked post.

Connecting it to the hive mind

M&M Sorter Cloud Architecture Diagram

After we got the sorter able to select M&M of a specified colour it was time to turn it into an actual IoT device. We set up a Firebase database on Google Cloud to store the current colour to be selected.

The iOS Lego Sorter mobile app which we deployed to an iPad is the primary means users choose the colour of the M&M they want, the current colour is sent up to Firebase and the Raspberry Pi periodically queries Firebase to see if the colour has been changed. A more elegant solution would be to use a cloud function to alert the Raspberry Pi when the current colour has changed but we ran out of time to do that.

If you’re interested in learning how to create an iOS app that uses Firebase as a backend check out Lachi’s quick guide on Rapid Prototyping with Firebase and iOS

Walkthrough

Once that was all done, we were good to go, check out the little video we created on how to use our IoT M&M sorter sort in conjunction with the iOS app.

Taking it to (IoT) Impact

The beautiful thing about Lego is that you can take it apart and set it up again really quickly. So we packed up the sorter at Sheda HQ and flew with it from Melbourne to Sydney and we were able to set it up again within minutes. Here is the Sheda IoT M&M sorter at IoT Impact sorting out yellow M&Ms.

Sheda’s IoT M&M sorter at IoT Impact

Feedback

We got a lot of interests and people stopping by our stand, conference attendees enjoyed getting a hands-on Industry 4.0 experience with some sweets to take away.

This was quite a fun project and we’ll be creating more like these.

If you enjoyed this…

and if you are interested in keeping up with what we get up to at Sheda, follow me and Sheda’s publication or get in touch at iot@sheda.ltd.

You can also reach me on LinkedIn.

Would love to help you explore how you can use IoT to increase the impact of your business activities.

By

Eike Zeller

Head of IoT Product Development
@ Sheda
You might like this
No items found.
Back to

Call us

Call +61 3 9028 6936

Drop in and say hi

4 Brunswick Pl,
Fitzroy VIC 3065
get directions

Follow us

Contact

Thank you! Your submission has been received!

Hmm... Something's not right. Try submitting again.