Machine Learning Technology Preview Coming Soon

WPILib will be making machine learning available to FRC teams with a new object detection technology preview. Machine learning is a fast growing field and is being used in a growing portion of the science and technology world. While normally presented as a complicated subject, Amazon and Google have provided support to make the machining learning solution:

  • Approachable by providing a simplified workflow and set of tools
  • Not require expensive hardware
  • Have a first solution up and running quickly
  • A path to improve your results as you learn more

Here is an example image of the type of results you can expect:

image-asset.png

Advantages of using machine learning over other methods are that it is easy to recognize objects without retroreflective tape, work under a variety of lighting conditions without tuning, and recognize multiple types of elements at the same time.

There are three broad steps to machine learning:

  1. Creating a dataset is where you gather up images of every type of object you want to detect and draw bounding boxes (label) around the objects of interest. You will be provided with a starter set of images, from a full field setup, with labeled Power Cells.
  2. Training is where you upload your tabled dataset to an Amazon server and teach the network how to recognize all of the labeled objects (classes). This is done using software from Amazon and Google.
  3. Inference is where you take your trained network and either verify its operation on the Amazon cloud or on your Raspberry Pi and Google Coral device. This is where the network can recognize all the objects of interest and help operate the robot actuators.

Here are some things you will need to get prepared for the technology preview:

Software

  • AWS Educate. You will have the ability to train and test your machine learning on the Amazon Web Services cloud. To get setup go to the Virtual Kit of Parts and look for the AWS items (https://www.firstinspires.org/robotics/frc/kit-of-parts#AWS). There are mentor and student onboarding guides.
  • supervise.ly is a platform that makes it easy to put together and label a dataset of images. You can sign up for an account here https://app.supervise.ly/ The account is free. After kickoff we will provide a dataset of images that you can clone to your account. It will include labeled power cells, but you can add your own images or add more labeled “classes” to the provided images.
  • FRC Vision Raspberry Pi Image

Hardware

  • A Raspberry Pi Model 3 or 4
  • A webcam, such as the LifeCam HD-3000
  • Google Coral. We will be using a machine learning infrastructure called TensorFlow from Google. They have recently released a USB device called Coral. This device includes a special processor called a TPU, or Tensor Processing Unit, allowing your robot to get around 15 - 20 frames per second while doing object detection. You can get the Coral from Amazon as well as other vendors.

Check back in the coming days for more details!

Austin Shalit