how to connect to the flight controller
. Install the packages:
sudo apt-get install python-pip python-opencv python-opencv-apps python-zbar zbar-tools vim-python-jedi vim-python-jedi python-editor eric idle vim-nox-py2
pip install Cython numpy
pip install pyrealsense
Local / Remote editing
With Ubuntu running on Intel Aero, you have a full desktop running locally. When you have a keyboard, mouse and hdmi screen connected on Intel Aero, you can launch regular graphical Ubuntu editors (eric, idle, ...). Or you can launch a terminal on your development station and run a command line editor (vim, nano) over ssh.
example: vim over ssh
vim
example: IDLE
idle
example: eric
eric
Flight control
Hello world in PyMAVLINK
Here's a simple python script using the basic pymavlink wrapper to arm the motors for 3 seconds. Arming the motors is the simplest action we can test to show everything is connected. Note: we're using tcp:127.0.0.1:5760 to connect to the flight controller, as we'll do for all the following examples.
UNPLUG THE PROPELLERS BEFORE RUNNING THIS CODE. WE INSIST.
#!/usr/bin/python
from __future__ import print_function
import pymavlink.mavutil as mavutil
import sys
import time
mav = mavutil.mavlink_connection('tcp:127.0.0.1:5760')
mav.wait_heartbeat()
mav.mav.command_long_send(mav.target_system, mav.target_component,
mavutil.mavlink.MAV_CMD_COMPONENT_ARM_DISARM, 0, 1,
0, 0, 0, 0, 0, 0)
time.sleep(3)
mav.mav.command_long_send(mav.target_system, mav.target_component,
mavutil.mavlink.MAV_CMD_COMPONENT_ARM_DISARM, 0, 0,
0, 0, 0, 0, 0, 0)
The motors should spin for 3 seconds. If they don't, it may be because:
you have not calibrated your flight controller with QGroundControl
the radio remote is not connected
your wall power supply does not provide enough power
Hello world in DroneKit
It’s important to know the basics of MAVLINK, as it the base of all communications with the Flight Controllers. But coding frames with python-mavlink is not developer friendly. DroneKit, developed by 3D Robotics, is one of the friendly python abstractions available under Apache v2 Licence : python.dronekit.io To install on Intel Aero, first connect your drone to a WiFi network (in client mode) with internet access and install Dronekit:
pip install dronekit
UNPLUG THE PROPELLERS BEFORE RUNNING THIS CODE. WE INSIST.
Here's the code, still arming the motors for 5 seconds:
#!/usr/bin/python
from dronekit import connect, VehicleMode, LocationGlobalRelative
import time
vehicle = connect('tcp:127.0.0.1:5760', wait_ready=False)
print "Arming motors:"
vehicle.mode = VehicleMode("GUIDED")
vehicle.armed = True
while not vehicle.armed:
print " Waiting for arming to be finished"
time.sleep(1)
print "Keeping motors armed for 5s"
time.sleep(5)
print "Disarming"
vehicle.armed = False
Hello world in PyMAVProxy
On top of being a developer friendly layer on top of MAVLINK, MAVProxy was designed to bridge the gap between programming-only libraries like DroneKit and graphical-only tools like QGroundControl. Check: ardupilot.github.io/MAVProxy Some people use it on a remote computer to control the drone and have optional user interfaces to complex functions, but you can use on the drone itself for autonomous drone development.
MAVProxy is using the MAVLINK protocol, but is focused on the Ardupilot variant, not the PX4 variant. IF you're interested in MAVProxy, you'll have to flash the Flight Controller with the Ardupilot firmware. To install on Intel Aero, first connect your drone to a WiFi network (in client mode) with internet access and install MAVProxy:
pip install MAVProxy
UNPLUG THE PROPELLERS BEFORE RUNNING THIS CODE. WE INSIST.
Launch the shell
mavproxy.py --master=tcp:127.0.0.1:5760 --quadcopter
And type a few commands to arm/disarm the motors:
arm throttle
disarm
bat
Cameras
Intel RealSense
We're covering Python programming with Intel RealSense SDK (pyrealsense) with model R200, included in the Intel Aero Ready To Fly Drone. If you have a newer camera like the D430, you'll need another SDK.
## setup logging
import logging
logging.basicConfig(level = logging.INFO)
## import the package
import pyrealsense as pyrs
## start the service - also available as context manager
serv = pyrs.Service()
## create a device from device id and streams of interest
cam = serv.Device(device_id = 0, streams = [pyrs.stream.ColorStream(fps = 60)])
## retrieve 60 frames of data
for _ in range(60):
cam.wait_for_frames()
print(cam.color)
## stop camera and service
cam.stop()
serv.stop()
OpenCV and RealSense
You can consider the depth data coming from the RealSense camera as an image, and manipulate it with OpenCV. Here's an example.
Barcodes
Let's use zbar to detect barcodes from the camera:
from sys import argv
import zbar
proc = zbar.Processor()
proc.parse_config('enable')
# set the correct device number for your system
device = '/dev/video13'
if len(argv) > 1:
device = argv[1]
proc.init(device)
proc.process_one()
for symbol in proc.results:
print 'barcode type=', symbol.type, ' data=', '"%s"' % symbol.data
And here's a test sheet with various sizes of barcodes.
Networking
Networked drone in WebSockets
Please refer to the module D2 - Software - Networked Drone of the course for more information about the following codes.
UNPLUG THE PROPELLERS BEFORE RUNNING THIS CODE. WE INSIST.
The Python websocket API on Intel Aero:
pip install websocket-server
A WebSocket client in your browser, to simulate a web call:
Optionally, a Library on your development station to send the request from a script instead of your browser:
pip install websocket-client
And here's the server code running on Intel Aero:
#!/usr/bin/python
from websocket_server import WebsocketServer
import re
from dronekit import connect, VehicleMode, LocationGlobalRelative
import time
vehicle = connect('tcp:127.0.0.1:5760', wait_ready=False)
vehicle.mode = VehicleMode("GUIDED")
print("Flight Controller Connected")
def new_client(client, server):
print("Client connected")
server.send_message_to_all("Client connected")
def message_received(client, server, message):
if len(message) > 200:
message = message[:200]+'..'
print("Arming motors")
vehicle.armed = True
while not vehicle.armed:
time.sleep(1)
time.sleep(5)
print("Disarming")
vehicle.armed = False
server = WebsocketServer(8080, '0.0.0.0')
server.set_fn_new_client(new_client)
server.set_fn_message_received(message_received)
server.run_forever()
And the optional client code, running on a remote computer (change the IP 192.168.0.100 with Aero's IP on YOUR NETWORK):
#!/usr/bin/python
from websocket import create_connection
ws = create_connection("ws://192.168.0.100:8080")
ws.send("Alert, send drone")
result = ws.recv()
print("Received '%s'" % result)
ws.close()
Networked drone in MQTT
Please refer to the module D2 - Software - Networked Drone of the course for more information about the following codes.
To install the Python MQTT API on Intel Aero (should be there already):
pip install paho-mqtt
A MQTT client in your browser, to generate a message posting.
UNPLUG THE PROPELLERS BEFORE RUNNING THIS CODE. WE INSIST.
Here's the client code, running on Intel Aero:
#!/usr/bin/python
import paho.mqtt.client as mqtt
from dronekit import connect, VehicleMode, LocationGlobalRelative
import time
vehicle = connect('tcp:127.0.0.1:5760', wait_ready=False)
vehicle.mode = VehicleMode("GUIDED")
print("Flight Controller Connected")
def on_connect(client, userdata, rc):
print("Client connected ")
client.subscribe("aero-paul")
def on_message(client, userdata, msg):
print("Arming motors ("+msg.topic+"/"+str(msg.payload)+")")
vehicle.armed = True
while not vehicle.armed:
time.sleep(1)
time.sleep(5)
print("Disarming")
vehicle.armed = False
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.connect("test.mosquitto.org", 1883, 60)
client.loop_forever()
Deep Learning with Intel Movidius NCS
After installing the Intel Movidius Neural Compute Stick as described on this page and make sure the make examples result is ok, go to the first Python NCS code sample:
cd ~/workspace/ncsdk/examples/apps/hello_ncs_py
python hello_ncs.py
it should output
Hello NCS! Device opened normally.
Goodbye NCS! Device closed normally.
NCS device working.
For a real life example:
cd ~/workspace/ncsdk/examples/caffe/AlexNet
make run_py
it should output:
...
------- predictions --------
prediction 0 (probability 94.970703125%) is n03272010 electric guitar label index is: 546
prediction 1 (probability 4.76684570312%) is n02676566 acoustic guitar label index is: 402
prediction 2 (probability 0.102138519287%) is n02787622 banjo label index is: 420
prediction 3 (probability 0.0409126281738%) is n04517823 vacuum, vacuum cleaner label index is: 882
prediction 4 (probability 0.0347137451172%) is n04141076 sax, saxophone label index is: 776
Peripherals
Intel Aero board LEDs
There's a multicolor LED on top of the board (if the board is in the enclosure, you can see the light from the white cable hole), and an orange LED under the board. As the LEDs are enclosed in the Ready-To-Fly design, it is not very useful. But if you build your own drone design or enclosing you may want to let the LEDs visible and use them. To install the IO module:
pip install python-periphery
And here is a sample code to test all the LED colors:
#!/usr/bin/python
import time
from periphery import GPIO
print "Top LED Blue"
gpio = GPIO(403, "out")
gpio.write(bool(1))
time.sleep(1)
gpio.write(bool(0))
gpio.close()
print "Top LED Green"
gpio = GPIO(397, "out")
gpio.write(bool(1))
time.sleep(1)
gpio.write(bool(0))
gpio.close()
print "Top LED Red"
gpio = GPIO(437, "out")
gpio.write(bool(1))
time.sleep(1)
gpio.write(bool(0))
gpio.close()
print "Bottom LED Orange"
gpio = GPIO(507, "out")
gpio.write(bool(1))
time.sleep(1)
gpio.write(bool(0))
gpio.close()
