Spatial AI is the ability of an artificial intelligence system to reason not just based on what it is looking at, but also based on distance from the camera. Contest participants from the OpenCV Spatial AI Contest sponsored by Microsoft Azure and Intel proposed their solutions for real world problems using spatial AI and OAK devices. Below are five proposals that stood out to us at Roboflow the most.
Roboflow is a platform used by developers that empowers them to build their own computer vision applications, no matter their skillset or experience. Check out Roboflow Universe for datasets and models to build spatial AI applications of your very own.
ARC – Automatic River Cleaning System
We’re a big fan of Keeping Our Oceans Clean With Computer Vision. The team Ocean Crusaders built the first Automatic River Cleaner (ARC) which is currently undergoing sea trials in Brisbane River, Queensland, Australia. The project helps rivers to stop waste from entering the ocean.
Their proposed solution is to mount a camera above a conveyor belt and build an AI model which can identify the objects coming up the conveyor belt to determine if they are bio-material or man-made. Their use case will also monitor for jams and other malfunctions.
Autismo
We’ve seen amazing projects from students in the past on how Using Computer Vision to Improve Disability Access in Music Education can be impactful on not only the end user, but also the people who are building the model. The team Autismo proposes to develop an OAK-D and Lego based robot to help autistic children’s learning process. They plan to use Pose tracking, Emotion Detection & Recognition, Image Recognition, and Face Detection & Recognition to perform interactive games and physical rehabilitation scenarios. Below is an image of their prototype that uses LEGO Mindstorms Robot Inventor kit and an 800×480 LCD as the face, Raspberry Pi as the host computing unit, and Azure cloud.
Pablito
This team’s vision is to use a small drawing robot assisted by an OAK-D-Lite as a “distant descendent of Pablo Picasso.” They would like to build an artistic system that will be able to automatically draw a portrait of a person.
Pablito’s proposal reminded us of How We Built Paint.wtf, an AI Game with 150,000+ Submissions that Judges Your Art. Paint.wtf is an online game that uses AI to score user-submitted digital drawings.
OpenAI in tandem with CLIP also created generative models that can produce images based on a textual description.
Feel-The-Burn
This team plans on building an autonomous crane for a mini garbage incineration plant. They propose that for countries where large-area of lands for land-filling is not possible, waste incineration is the main approach to process the waste. Because incineration plants require cranes to improve the efficiency of the burning,
They propose to use an OAK device to take images of the garbage and infer its depth and contents. After which, information would be processed in order to decide where the crane should go next to dig. Below is an illustration of their vision for the competition.
The EPFL Loutres
Yellow-legged hornet is slowly propagating across Europe and threatening the western bees. This team plans on using spatial AI to identify hornets flying stationary in front of the hive and trigger the partial closure of its entrance.
They also aim to identify other great threats domesticated bees such as the Varroa Mites, which create abnormal physical growth such as crippled wings during their larva stage, parasites feeding from their abdomen as adults. Using spatial AI would help them to recognize abnormal flight patterns or the presence of the parasite on the bee body directly and give a signal to the beekeepers, allowing for action.
Building Your Own Datasets And Models
We’d love to have your spatial AI projects hosted on Roboflow Universe! You can get started by creating a free Public account to upload your dataset and train a model. Visit our Quick Start docs to learn more.
