OpenCV

Did you know that you can use computer vision in your Skill by including the OpenCV library?

In this tutorial we are going to show how to cross compile and bind a C++ library to be used in your Skills. We will be using OpenCV since it is a very useful library for robots who want to visually understand the world.

Cross compiling OpenCV

Let’s start by downloading and extracting OpenCV:

$ curl -OL https://github.com/opencv/opencv/archive/2.4.13.2.tar.gz
$ tar -xzvf 2.4.13.2.tar.gz
$ cd opencv-2.4.13.2

To make OpenCV work in a Skill on the HEXA we need to compile OpenCV for HEXA’s processor architecture (arm7).

Compiling for another architecture than what your development machine is using is called cross-compilation and MIND SDK ships with this functionality built in.

We will be using mind x [COMMAND]to launch a container in the current folder which in turn will invoke [COMMAND]. In our case, we are going to write a small shell scriptbuild.shwhich we will execute inside the cross-compiling container withmind x ./build.sh

Inside the opencv-2.4.13.2 folder that we just extracted, use your favorite editor and add the following content to build.sh, and run chmod +x build.sh to add executing permission to it.

This shell script contains all the commands that are required to build OpenCV directly grabbed from their official documentation.

#!/usr/bin/env bash
apt-get update 
apt-get install -y \
    libavcodec-dev \
    libavformat-dev \
    libswscale-dev \
    libtbb2 libtbb-dev \
    libjpeg-dev \
    libpng-dev \
    libtiff-dev \
    libjasper-dev \
    libdc1394-22-dev
mkdir -p platforms/linux/build_hardfp
cd platforms/linux/build_hardfp && \
    cmake -DCMAKE_TOOLCHAIN_FILE=../arm-gnueabi.toolchain.cmake ../../.. && \
    make && make install

Once the build is finished, you should be able to find all of the built dynamic libraries inside of theinstallfolder.

Binding OpenCV in a Skill

Let’s start by creating a new Skill project.

$ cd .. 
$ mind init OpenCVSkill
$ cd OpenCVSkill
$ tree .
.
├── manifest.json
├── remote
│   └── index.html
└── robot
    ├── deps
    └── src
        └── opencvskill.go

Note thedepsfolder. This is where you put all of the dynamic libraries that your Skill depends on. That means we need to copy the built OpenCV libraries and header files into this folder.

$ cp -r ../opencv-2.4.13.2/platforms/linux/build_hardfp/install/* robot/deps/

Then robot/deps should contains include and lib folder.

$ tree -L 2 robot/deps/
robot/deps/
├── include
│   ├── opencv
│   └── opencv2
└── lib
    ├── libopencv_calib3d.so -> libopencv_calib3d.so.2.4
    ├── libopencv_calib3d.so.2.4 -> libopencv_calib3d.so.2.4.13
    ├── libopencv_calib3d.so.2.4.13
    ├── libopencv_contrib.so -> libopencv_contrib.so.2.4
    ├── libopencv_contrib.so.2.4 -> libopencv_contrib.so.2.4.13

To generate Golang bindings for C libraries, one would typically use CGO, and when binding for C++ libraries, one would use SWIG. Writing the Golang bindings can be a pretty mundane process, and in our case we are lucky. Some cool people have already gone through the effort of writing Golang bindings for OpenCV using SWIG.

Clone the go-opencv repository into thevendorfolder as such:

$ git clone https://github.com/lazywei/go-opencv.git robot/src/vendor/github.com/lazywei/go-opencv

Editrobot/src/opencvskill.goand add the following import:

import (
  ...
  "github.com/lazywei/go-opencv/opencv"
  ...
)

And now you are ready to start using OpenCV inside of your Skill.

Please have a look at our OpenCV Example Skill for some useful getting started code. The project also contains a Makefile that automates the downloading and building of OpenCV.

If you want to know more in depth how the SDK and cross-compilation container works, you can have a look at this blog post which explains it in detail.