Welcome to the fifth Flowstone workshop where we give a beginner’s guide to programming for robotics using the Flowstone FREE graphical programming language. For the past few workshops we have focused on interfacing to hardware, servo controllers, motor controllers, data acquisition devices (DAQs) and even a robot made out of an xBox controller – XbugBot. This issue we are looking behind the scenes of the hardware to see what is going on inside a robot’s brain and look at Digital Signal Processing, or DSP for short.
Digital Signal Processing is the generic name given to the programming used for real time signals. In robotics this is most of the programming required to make your robot move, interact, balance, see and hear etc. The word DSP can also apply to a specialized type of processing chip called a ‘DSP’ chip. In the past these were usually the only option for real time processing as personal computers were too slow and the timing was inaccurate, and microprocessor chips like PIC chips and the Arduino were also too slow or didn’t have enough memory.
Nowadays this isn’t the case, so it’s a bit of a grey area, just about any type of PC or Microprocessor can perform basic DSP functions!
Twenty-five years ago I started my electronics career as a DSP programmer using some of the first commercial DSP chips from Texas Instruments. Back in those days you had only one programming choice: to program in Assembler! While very efficient, the development life cycle was huge and I remember the printouts of code could be several inches thick for just one program. Then came ANSI C++ which was a big step forward in programming speed, but was far less efficient. The big issue with all of these text based legacy programming languages and real-time signals, isn’t the programming time, but the debugging time! Debugging your real-time code is the largest part of any project, and this is still true today! Program in any text based language from BASIC to C# and you have to follow the same old procedure: write your code in a text editor, spot any syntax errors (spelling errors), compile the code (can take several minutes), upload into the hardware (if using a microprocessor or DSP chip), test it to see if it works—no; go back change something, try again, etc. So, is there a better way? Yes FlowStone!
So why is Flowstone so good for real-time programming for robotics? There are two reasons:
1) Its a graphical programming language
This means that you can write a real computer program that you can turn into an EXE application to distribute or sell, without writing a single line of code using the pre-made library of programming modules. If you need to code something there is the full Ruby programming language built inside for you to make your own modules. A graphical approach to DSP programming makes much more sense as your program looks more like an electronic wiring diagram where you can see the flow of the data from left to right.
2) It runs in real-time All of the time!
This means that it is running as you build your application so you can see the real-time signals as you develop your application. Be it just math, inputs, outputs, audio signals, live video, etc. You only need to compile to an EXE file once when you are 100% happy with your program running in the development environment.
Example 1: Sine wave oscillator + scope
Imagine you want your robot to make a noise relative to it’s proximity to an object. For this simple example let’s use a sine wave tone where the frequency increases as it nears an object. How would you program that in FlowStone?
Step 1) Add a soundcard output module (Direct Sound Out) to your FlowStone project and select Primary Sound Driver (your PC Soundcard).
Step 2) Add an Oscillator Module (multi OSC) and select Sine.
Step 3) Add a proximity sensor: n
This would usually be an analogue voltage from an IR or Ultrasonic proximity sensor, fed into a USB input board like a Phidgets 888, or a DSPRobotics FlowBoard (covered in our previous FlowStone Workshop on DAQ, March-April 2012 Robot). But for simplicity let’s simulate this with a slider setting the range to 500 – 2000.
Step 5) Add a scope module for that authentic robot visual!
Step 6) Wire it all up!
Now move the slider to simulate the proximity and low and behold you not only hear the real-time audio but you see it on the scope, no compiling! Now you get the idea; lets look at some of the more advanced DSP processing algorithms you can use in FlowStone (this is just a small sample there are too many to mention).
Fast Fourier Transform.
Analogue Audio DSP Processes
• FFT – Fast Fourier Transform. This gives you a real-time graph of the frequencies in an audio signal (e.g., used for voice recognition and audio processing)
• Correlation. Looks for matching patterns inside a real-time signal
• Bi-Quad Filter. Allows real-time filtering of things like Hi-Pass, Low Pass, Band pass etc.
• FM Modulator/ De-Modulator. Frequency modulate a real-time signal (Like FM Radio does)
• Pulse Width Modulation. PWM like a servo controller
• Running Average. Creates an average of a real-time signal (helps reduce noise)
Real-time Math DSP
There are literally hundreds of Math functions; here are a couple of interesting ones for robotics:
• PID. Proportional Integral Derivative, used to move robotics in a smooth predictable action (Cruz Control).
• IK. Inverse Kinematics, used for calculate robot arm positions.
Video Processing DSP
Use a simple webcam or IP CCTV camera to add video to your project.
• CamShift Tracker. Used to track object in a video image.
• Harr Face Detect. Facial recognition.
• Color Detect. Used to detect colors in a video stream (used for line following and targeting, etc.)
• Motion Detect. Highlights areas of motion and the motion direction.
As you can see there are some pretty powerful functions and you can program your own in Ruby if you have something new you want to create. So let’s put something a bit more complicated together using some of the above to a make a real DSP example.
Example 2) DSP Sentry Robot
Let’s make a sentry Robot that guards an entrance to a high security facility. This robot should use video and motion detection to detect intruders, ask the intruder to identify themselves and then use voice recognition to determine if it knows the intruder or not. If not take a photo and store it to disk (or fire a laser and vaporize the intruder, anyone volunteers?).
To do this we will use a cheap webcam to capture the video, the PC sound card to ask the question, and record the intruder’s voice. This makes a great educational project as you can use any laptop with a webcam to make this.
Step 1) Motion Detection For this we will need a webcam module and a motion detection module. The webcam module captures frames of video and turns them into bitmaps. These are then processed by the motion detection module to detect differences in the video frames. The motion detect module has various inputs setting that can be used to set the sensitivity etc.
Step 2) Audio Out We can use the Direct Sound Out Module to play the question from a sound file (MP3 or Wav) through the PC soundcard from FlowStone: “Please Identify Yourself”. Remembering to mute the audio ‘In’ at this time so it doesn’t listen to itself.
Step 3) Audio In We can use the Direct Sound Audio In module to get real-time audio into FlowStone for analysis.
Step 4) Audio Detection In order to perform the voice recognition analysis we first need to detect the audio when the person speaks. To do this we use a DSP running average module to detect the average level of the audio. Once this goes over a certain threshold the person is speaking.
Step 5) Voice Recognition In order to recognize the voice we can use a simplified frequency analysis system. This basically learns a sentence of words based on the average frequencies and stores these for comparison. To do this we use the FFT module that takes a real-time audio stream and computes a Fast Fourier Transform (FFT). An FFT is a spectral analysis of the audio showing the frequencies present and their relative amplitude.
To compare the stored graph with the live graph we use a differential (basically subtract one from the other) and a sum (add up all of the values). This gives us a percentage probability that the voice is the same or not. This is then compared with a max and min accepted values to decide if it is near enough or not.
Step 6) Open the Door Lock If the intruder is recognized we need to open the door lock, one simple way to do this is to use one of the many USB relay boards. Phidgets and KMTronic make great relay boards that are compatible with FlowStone. So from FlowStone we simply send an ‘On’ value to the relevant module and the relay will open. In this case we used a Phidgets 008 relay board and FlowStone Module.
Step 7) Take Photo In order to take a photo and store it to disk we need to use the Save Bitmap module. This converts the FlowStone bitmap format into any of the popular graphics formats: bmp, jpg, tiff, png etc. and dumps it to disk. So all we need to do is give it a photo of the intruder. We can do this by sampling and holding a bitmap as soon as the motion detection is triggered, then save it to disk if necessary.
Step 8) Testing So now to test our new application, first we need to ask the robot to learn our voice. To do this I just counted from 1 to 5, pressing the learn button first to enable the learn mode. Now I can walk into the room and the robot says “Please Identify yourself”. I then say one, two, three, four, five, and if I said it the same as before, buzz and the door opens. If not I’m locked out and my photo is stored to disk for reference.
As always you can download the source code from the DSPRobotics support page under ‘FlowStone Examples.’
DSP type programs are a lot easier to understand and follow inside FlowStone and are much quicker to program than in a text only programming language. In fact programming this quite complex DSP Sentry Robot application only took a couple of hours, which is testimony to how efficient FlowStone is as a language. Even if you decide to not to use a PC as your final target device you can use FlowStone to at least develop your DSP algorithms and then hard code them later into a chip. FlowStone makes the ideal tool for education as students can visualize the program as it develops and see the live signals on screen.