Did you know that you can use computer vision in your Skill by including the OpenCV library?
In this tutorial we are going to show how to cross compile and bind a C++ library to be used in your Skills. We will be using OpenCV since it is a very useful library for robots who want to visually understand the world.
Cross compiling OpenCV
Let’s start by downloading and extracting OpenCV:
$ curl -OL https://github.com/opencv/opencv/archive/184.108.40.206.tar.gz $ tar -xzvf 220.127.116.11.tar.gz $ cd opencv-18.104.22.168
To make OpenCV work in a Skill on the HEXA we need to compile OpenCV for HEXA’s processor architecture (arm7).
Compiling for another architecture than what your development machine is using is called cross-compilation and MIND SDK ships with this functionality built in.
We will be using
mind x [COMMAND]to launch a container in the current folder which in turn will invoke
[COMMAND]. In our case, we are going to write a small shell script
build.shwhich we will execute inside the cross-compiling container with
mind x ./build.sh
opencv-22.214.171.124 folder that we just extracted, use your favorite editor and add the following content to
build.sh, and run
chmod +x build.sh to add executing permission to it.
This shell script contains all the commands that are required to build OpenCV directly grabbed from their official documentation.
#!/usr/bin/env bash apt-get update apt-get install -y \ libavcodec-dev \ libavformat-dev \ libswscale-dev \ libtbb2 libtbb-dev \ libjpeg-dev \ libpng-dev \ libtiff-dev \ libjasper-dev \ libdc1394-22-dev mkdir -p platforms/linux/build_hardfp cd platforms/linux/build_hardfp && \ cmake -DCMAKE_TOOLCHAIN_FILE=../arm-gnueabi.toolchain.cmake ../../.. && \ make && make install
Once the build is finished, you should be able to find all of the built dynamic libraries inside of the
Binding OpenCV in a Skill
Let’s start by creating a new Skill project.
$ cd .. $ mind init OpenCVSkill $ cd OpenCVSkill $ tree . . ├── manifest.json ├── remote │ └── index.html └── robot ├── deps └── src └── opencvskill.go
depsfolder. This is where you put all of the dynamic libraries that your Skill depends on. That means we need to copy the built OpenCV libraries and header files into this folder.
$ cp -r ../opencv-126.96.36.199/platforms/linux/build_hardfp/install/* robot/deps/
robot/deps should contains
$ tree -L 2 robot/deps/ robot/deps/ ├── include │ ├── opencv │ └── opencv2 └── lib ├── libopencv_calib3d.so -> libopencv_calib3d.so.2.4 ├── libopencv_calib3d.so.2.4 -> libopencv_calib3d.so.2.4.13 ├── libopencv_calib3d.so.2.4.13 ├── libopencv_contrib.so -> libopencv_contrib.so.2.4 ├── libopencv_contrib.so.2.4 -> libopencv_contrib.so.2.4.13
To generate Golang bindings for C libraries, one would typically use CGO, and when binding for C++ libraries, one would use SWIG. Writing the Golang bindings can be a pretty mundane process, and in our case we are lucky. Some cool people have already gone through the effort of writing Golang bindings for OpenCV using SWIG.
Clone the go-opencv repository into the
vendorfolder as such:
$ git clone https://github.com/lazywei/go-opencv.git robot/src/vendor/github.com/lazywei/go-opencv
robot/src/opencvskill.goand add the following import:
import ( ... "github.com/lazywei/go-opencv/opencv" ... )
And now you are ready to start using OpenCV inside of your Skill.
Please have a look at our OpenCV Example Skill for some useful getting started code. The project also contains a Makefile that automates the downloading and building of OpenCV.
If you want to know more in depth how the SDK and cross-compilation container works, you can have a look at this blog post which explains it in detail.