Introduction

The Computer Vision Controlled IoT Connect Robotic Arm project is a collaboration between Intel Software Innovators, TechBubble Technologies & Saft7robotics. The project allows you to create a fully functioning IoT connected robotic arm that can be controlled by a computer vision neural neural network.

The project combines 3 main projects which are as follows:

• Saft7robotics: Arduino Powered Robotic Arm
• Adam Milton-Barker: InceptionFlow
• TechBubble Technologies: TASS AI

The project consists of three programs:

• Robotic Arm Program (Arduino)
• ESP8266 Communication Program (Arduino)
• InceptionFlow Computer Vision Program (Python)

COMPETITION!!!

Want to win your very own robotic arm kit ? We are looking for the best project that utilizes the Computer Vision Controlled IoT Connected Robotic Arm project. To enter your project, you must:

• Star / Fork this repository.
• Like the TechBubble Technologies & Saft7robotics Facebook pages.
• Use all three of the programs that make up this tutorial.
• Add your full project idea including description / graphics / video to your project README file in your forked repository.
• Post the link to your README on the pinned post on the TechBubble Technologies Facebook page.

We are looking for the most innovative, crazy and out of this world projects, but would also like to see a real world use case.

BONUS

Use the IoT JumpWay for autonomous communication with other IoT devices connected to the IoT JumpWay for extra browny points ;)

The competition is open until March 14th.

ESP8266 Communication Program

The first step is to setup the ESP8266 Communication Program. You will find the source code and tutorial in the ESP8266 directory.

The ESP8266 Communication Program acts as a bridge between the Robotic Arm and the IoT JumpWay. The program listens for commands sent to the device and forwards them through to the arm via serial.

Hardware Requirements

1. ESP8266

Software requirements

1. ESP8266WiFi
2. PubSubClient MQTT
3. ArduinoJson
4. WiFiClientSecure

Cloning The Repo

You will need to clone this repository to a location on your development terminal. Navigate to the directory you would like to download it to and issue the following commands.

$ git clone https://github.com/TechBubbleTechnologies/IoT-JumpWay-Intel-Examples.git

Once you have the repo, you will need to find the files in this folder located in this location

Before You Begin

There are a few tutorials that you should follow before beginning, especially if it is the first time you have used the TechBubble IoT JumpWay Developer Program. If you do not already have one, you will require a TechBubble IoT JumpWay Developer Program developer account, and some basics to be set up before you can start creating your IoT devices. Visit the following IoT JumpWay Developer Program Docs (5-10 minute read/setup) and check out the guides that take you through registration and setting up your Location Space, Zones, Devices and Applications (About 5 minutes read).

Connection Credentials

• Follow the TechBubble Technologies IoT JumpWay Developer Program (BETA) Location Device Doc- to set up your device.

• Add your WiFi credentials to the following code which will allow your ESP8266 to connect to the internet.

	const char* ssid = "YourSSID";
    const char* password = "YourWiFiPassword";

• Retrieve your connection credentials and update the config.json file with your new connection credentials and sensor setting.

    String locationID = "YourLocationID";
    String zoneID = "YourZoneID";
    String deviceID = "YourDeviceID";
    char deviceName[] = "YourDeviceName"; 
    char mqttUsername[]   = "YourDeviceMQTTUsername"; 
    char mqttPassword[]  = "YourDeviceMQTTPassword";

Robotic Arm Core Program

The next step is to setup the Robotic Arm Core Program. You will find the source code and tutorial in the Arduino directory.

Computer Vision Program

The final step is to setup the Computer Vision Program. You will find the source code and tutorial in the Python directory.

The Computer Vision Program is based on Adam Milton-Barker's InceptionFlow and allows you train a neural network to detect faces or objects. In this tutorial, we train the neural network to be able to recognize up, down, left and right arrows, once the program detects an arrow, it will send the relevant command to the robotic arm.

Bugs & Issues

Please feel free to create issues for bugs and general issues you come across whilst using this project. For issues with using the IoT JumpWay please visit the TechBubble GitHub repo.

Contributors

• Adam Milton-Barker, TechBubble Technologies Founder

• Saft7Robotics