Saturday, 27 June 2015

End of Semester Self Reflection

As the semester for this Technology Project comes to a close, I am tasked with completing a final reflection based on important points that allow me to reflect on my performance and knowledge gained by partaking in this course.

The first point is about the new knowledge and skills learned during the project process.  Starting the project at the beginning of the semester I had zero knowledge or background in programming.  By partaking in this project, I was able to gain some basic understanding of writing code, C++ and Python languages.  Learning these codes helped me complete the project as they were crucial to the functioning of the lane departure system.

Another new knowledge that I have learned is that I am now aware of the many modules, micro-controllers, minicomputers, and shields that you can use to build useful projects that may aid the general public and can even solve a problem.

Closely along these lines, I also utilised skills that I had learned previously that helped me immensely in completing this project as well.  While the programming part was new to me, the designing, wiring and hardware building I was familiar with.  Using my experiences from the Automotive Electrical and Electronics course, I was able to use my knowledge of calculating basic circuitry and also use drivers, transistors, and free-wheeling diodes to help ensure that the system can function correctly and safely.  My technical advisor helped me in remembering these teachings as well and I had even further consolidated what I had learned previously by partaking in a practical project such as this.

I also used my knowledge of Multi-sim and designing circuitry for this project which was also learned from the electrical course.  This helped the prototype building run without a hitch or hiccup.  The warning system was made within a good time and the right parts were used to maintain optimum function and safety.

Despite the successes experienced, there were still limitations found in a few areas during this project process.  The main limitation experienced was my limited background on programming.  Being a beginner programmer, my code modifications and writing was fairly basic.  The project Raspberry Pi function reflected that with only a single colour detection.  In addition to this, there was also limited help available, therefore I had to read and study the coding over and over to get some idea of what it all meant and how it affected the code.  This was the main factor that made the project progress slowly.

Another limitation was the limited hours allotted for the course.  Had there been more hours allowed, a more manageable pace would result in the the project end-product having been even more refined with better coding.

Despite the limitations experienced, the project still progressed eventually.  There is still much room for progression and improvement of the system in the future.  Further study will continue on Python coding and image processing to improve the accuracy and feasibility of the lane departure system at night and during times where there are a lot of noise.

Further familiarisation with more electrical products and modules can also be done to help enhance the capabilities of project or product.  These include performance enhancements such as an even better frame-rate, improved responsiveness, better accuracy during night or sun streaked driving conditions, and a better heard and visualised warning kit.

Overall, this project has been a very challenging and rewarding experience.  A new interest in minicomputers and image processing has arisen out of this participation.  Further exploration will be done for this and also the improvement of the lane departure warning system shall continue.






Thursday, 25 June 2015

Yellow Tracked, Warning System Built, Prototype Made, Testing Success! - 25/06/15

Today's progress was immense.  First the BGR values for yellow had been added to the code.  These are as follows:

lower = np.array([10,100,100],dtype="uint8")
upper = np.array([50,250,250], dtype="uint8")

Then after the colour change, the warning system was then built.  The warning system consists of two parts, the warning light box and the buzzer.  Initially an HUD unit was to be used, however, a light box would be more visible throughout the day.  The HUD maybe a consideration for the future builds.  The warning system was first designed in both Multisim and Fritzing.  These schematics are found below.




Follow these schematics, the system was then put together.







After the warning system was put together, the unit was then connected to the Raspberry Pi via the designated pins.


It worked! Driving the vehicle through yellow lines triggered the warning system.  Below is the video of the testing of the unit.


Wednesday, 17 June 2015

Tracking Colour Changed, GPIO Output Control, Frame-Rate Play, Code Modified to Trigger LED - 18/06/15

Today was great for progress!  First point of call was to change the BGR values found in the code:

lower = np.array([76,31,4],dtype="uint8") upper = np.array([210,90,70], dtype="uint8")

This original value was set to blue and because we had an orange slot card, we decided to go for orange values. The lower and upper values provide a threshold spectrum for the colour range you want to work with. For example, from dark orange to light orange. This is the code adjusted for orange:


lower = np.array([10,100,200],dtype="uint8")
upper = np.array([50,150,250], dtype="uint8")

After that, we tested it and it worked as it started to track our orange slot card!  However, it was slightly laggy which cannot be applicable for our project as a quick responding warning system is needed. First the size of the video capture feed was reduced from 640x480 to 480x320. Afterwards, the frame-rate was increased to 90. The response was much more fluid and quicker which now suits our application.

After this, we started to import the GPIO (General Purpose Inputs and Outputs) library and to declare which pins are outputs:

import RPi.GPIO as GPIO # Imports GPIO library
GPIO.setmode(GPIO.BOARD) # Use board pin numbering GPIO.setup(36, GPIO.OUT) # Pin 36 set as out GPIO.setup(40, GPIO.OUT) # Pin 40 set as out

Then a simple LED circuit was connected to pin 36 (GPIO 16) to prove that the LED can be triggered via object tracking. It is worth noting that the output voltage of the Pi is 3.3V. The code lines used are:




The code changes worked! We are now able to smoothly track an orange card, and once detected, the LED will turn on and if out of site, the LED is off!


Lastly we noticed that the LED will stay on if we quit the application as it tracks the object.  We then added a few lines to break the loop once you press "q" to quit:

        # if the `q` key was pressed, break from the loop
        if key == ord("q"):
                GPIO.output(36,GPIO.LOW)
                GPIO.output(40,GPIO.LOW)
                break

Here is the demonstration of the LED actuation when object is detected:


The next step is to now design and build the warning system circuit with necessary chip protection lines and to fit and mount the Pi to an RC car and demonstrate it on an artificial road.






Object Tracking Working! (Using Python, OpenCV 3.0.0, Raspberry Pi 2 B and PiCamera) - 15/06/15

We have decided to re-install OpenCV to the newest version and use Python 3 for programming.  We followed the instructions in this link (Thank you Dipto Pratyaksa for the guide):

http://www.linuxcircle.com/2015/05/18/open-computer-vision-opencv3-with-python-3-on-raspberry-pi-2/

But to guide the readers, an explanation in more understandable terms will be made.

First is to update and upgrade the Raspberry Pi unit by typing these commands one at a time in Terminal or Bash:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo rpi-update

This will update and install the latest settings, software and firmwares.  After that, you will need to download and install some dependencies that are essential for OpenCV function:

$ sudo apt-get install cmake python3-dev python3.2-dev python3-numpy gcc gcc-c++ build-essential cmake-curses-gui

I did have a problem with downloading the gcc-c++ package but it didn't affect operation of the library.

("sudo" is to grant you root or administrator privileges while "apt-get" is to download)

$ sudo apt-get -y install pkg-config libpng12-0 libpng12-dev libpng++-dev libpng3 libpnglite-dev zlib1g-dbg zlib1g zlib1g-dev pngtools libtiff4-dev libtiff4 libtiffxx0c2 libtiff-tools libeigen3-dev

Then there are some more applications that are needed:

$ sudo apt-get -y install libjpeg8 libjpeg8-dev libjpeg8-dbg libjpeg-progs ffmpeg libavcodec-dev libavcodec53 libavformat53 libavformat-dev libgstreamer0.10-0-dbg libgstreamer0.10-0 libgstreamer0.10-dev libxine1-ffmpeg libxine-dev libxine1-bin libunicap2 libunicap2-dev swig libv4l-0 libv4l-dev libpython3.2 libgtk2.0-dev

After the dependencies have been installed, the OpenCV will now be downloaded:

$ git clone --branch 3.0.0-rc1 --depth 1 https://github.com/Itseez/opencv.git

This will take a few minutes then once downloaded, navigate to the file folder then you will have to make a release folder directory in the downloaded file:

$ mkdir release

$ cd release

("mkdir" is to make a directory folder while "cd" is to access it")

The next step is to prepare the configuration files:

$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=$(python3 -c "import sys; print(sys.prefix)") -D PYTHON_EXECUTABLE=$(which python3) ..

Don't forget to include the two periods in the end. Afterwards you can then open and from my understanding, extract the different packages within the library:

$ make -j4

This process is the longest one but compared to the previous OpenCV experience, it only took under 2 hours to get everything unpacked. After this, OpenCV is then ready to be installed:

$ sudo make install

After getting successfully installing OpenCV 3.0.0 on Raspberry Pi, a foray into programming the code was then done.  We used the code from the same author of the OpenCV installation guide above:

Blue Object Detection Code

# import the necessary packages from picamera.array import PiRGBArray from picamera import PiCamera import time import cv2 import numpy as np # initialize the camera and grab a reference to the raw camera capture camera = PiCamera() camera.resolution = (640, 480) camera.framerate = 50 camera.hflip = True rawCapture = PiRGBArray(camera, size=(640, 480)) # allow the camera to warmup time.sleep(0.1) # capture frames from the camera for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True): # grab the raw NumPy array representing the image, then initialize the timestamp # and occupied/unoccupied text image = frame.array blur = cv2.blur(image, (3,3)) #hsv to complicate things, or stick with BGR #hsv = cv2.cvtColor(blur,cv2.COLOR_BGR2HSV) #thresh = cv2.inRange(hsv,np.array((0, 200, 200)), np.array((20, 255, 255))) lower = np.array([76,31,4],dtype="uint8") #upper = np.array([225,88,50], dtype="uint8") upper = np.array([210,90,70], dtype="uint8") thresh = cv2.inRange(blur, lower, upper) thresh2 = thresh.copy() # find contours in the threshold image image, contours,hierarchy = cv2.findContours(thresh,cv2.RETR_LIST,cv2.CHAIN_APPROX_SIMPLE) # finding contour with maximum area and store it as best_cnt max_area = 0 best_cnt = 1 for cnt in contours: area = cv2.contourArea(cnt) if area > max_area: max_area = area best_cnt = cnt # finding centroids of best_cnt and draw a circle there M = cv2.moments(best_cnt) cx,cy = int(M['m10']/M['m00']), int(M['m01']/M['m00']) #if best_cnt>1: cv2.circle(blur,(cx,cy),10,(0,0,255),-1) # show the frame cv2.imshow("Frame", blur) #cv2.imshow('thresh',thresh2) key = cv2.waitKey(1) & 0xFF # clear the stream in preparation for the next frame rawCapture.truncate(0) # if the `q` key was pressed, break from the loop if key == ord("q"): break

This code and author can found at this link:
http://www.linuxcircle.com/2015/05/24/object-detection-based-on-color-with-python-3-opencv-3-and-raspberry-pi-2/

Testing the code was a success (Please remember to turn on your PiCamera at Terminal! (sudo raspi-config))  Below is a demonstration of blue object detection:



Blue is shown being tracked while a red circle was drawn to indicated where it has detected the blue colour.  The next step is to adjust the upper and lower BGR values to get the colour we want and to program it to trigger the GPIO output pin and into an LED to indicate that an object has been detected.





Wednesday, 10 June 2015

Raspberry Pi, USB Webcam, and PiCamera - 11/06/15

For the past two days, me and my partner have been working on the Raspberry Pi and studying how it works.  After getting used to the Linux based OS I started working on Raspberry's IDE which is called Python IDLE.  I then installed the OpenCV library which is a process that took 4.5 hours!  After adjusting some settings, we tested the face detection with the USB webcam using OpenCV algorithms.  The results were as read and expected as the Raspberry did not have enough processing power to execute the face detection in real time.  The frame-rate was slow with refresh rates at every half second.  The video shows the experiment while the pictures below give a more clearer view of the program in action.


It works!
But the refresh/frame rate was too slow

This effectively rules out the webcam as the input device.  However, after further researching, I have found these links that makes use of a Raspberry module that can make the object/lane marking detection possible.


http://www.pyimagesearch.com/2015/03/30/accessing-the-raspberry-pi-camera-with-opencv-and-python/

As seen in the video, the feed is so much smoother and would suit our application as split second reactions are so important.  So I went and purchased this module for $50 which adds the total to $190.  This is still under the budget.  This is PiCamera shown below.

PiCamera
The next phase is to follow the links above to develop the object tracking programme demonstrated a month ago.



Raspberry Pi - 9/06/15

Raspberry Pi 2 Model B
Throughout the week I have been studying Python language and getting a crash course as to how it functions.  In most part it is similar to other programming languages but differs in it's lesser emphasis on syntaxes.  This cuts down the chances of errors and greatly simplifies coding.  This is looking promising.

Before getting the Raspberry Pi, I made sure this time I did my proper background check and research.  I read up that because the Raspberry Pi, performance-wise resembles that of a pentium 2 computer back in the day, it might not be able to handle the requirements set by object tracking programs very well.  However, it did state that object tracking is still possible and it still uses OpenCV.  With this in mind, I bought the latest Raspberry Pi 2 B at $89 including class 10 SD card, case, and power charger.  By this point we are over the $250 budget because of wrong purchases with the Arduino unit.  However, counting just the Raspberry Pi unit with the other components necessary for the project, we are still under budget by more than $100.  This is the Raspberry Pi Unit.

In addition to this, we have also built the buzzer and HUD unit on a bread board shown below followed by a schematic of both circuits.

Warning Unit
The Raspberry Pi has 40 general purpose input output (GPIO) pins that you can use to connect simple output or input devices, sensors and actuators.  The output voltage is set at 3.3V and we have decided with the limited voltage, we will be using 3 LED's instead.  We will also be using pins 16 and 21 as output pins to send signals to the buzzer and HUD LED's respectively.







C++ and Arduino Problems - 5/06/15

We have researched for a week on how to convert and execute the C++ file into the Arduino unit.  In addition we were also trying to figure out how to implement the webcam into the Uno as well. Arduino is based on C language which is slightly different but can be converted.  However, the problem found is that Arduino cannot utilise the webcam to provide object tracking.  This is due to two reasons.

The first is that the Arduino is a slave device, much like the webcam.  They are both devices that rely on inputs and commands from a host device.  This can be rectified using a USB host shield to make the Arduino a host device but the biggest downfall is that the Arduino unit does not have the processing power to provide object tracking functions.  This does undo much of the progress the past few weeks but after searching for alternatives, we have arrived with the Raspberry Pi.

This mini computer has just enough processing power to produce object detection as shown in the video link below.


It is important to note that the camera used is a Raspberry Pi Camera module and not a USB camera.  Despite this, it still does use OpenCV to help produce object tracking codes so the Logitech webcam could still be used.  Another learning that I could take from what I've found this week is that you need to look at the processing power before undertaking the project.  Thankfully, however, the programming and such during the past few weeks were not in vain as I am now fairly familiar with the process and especially since this new direction also relies on OpenCV and an IDE based on a new language called Python.  

More exploration and sourcing of the new parts will be done in the next few days.

Further Tracking Refinements - 29/05/15

After last week's developments, I was able to find a code that has the working object highlighting in the normal feed.  It is shown below.

Refined Object Tracking

Manipulating the hue and saturation values again, I highlighted the red cover.  The next development is to modify the code to create a threshold then to think how to install the application into Arduino and to build the head up display unit and buzzer warning systems.

OpenCV and Object Tracking - 22/05/15

First Tracking
Mentioned in the last entry, I had trouble with error codes and could not run or debug the application. However, the OpenCV Lessons website mentioned last week provided sample codes which were working. I decided to compare the sample codes with my own work.  Double checking carefully I found a few syntax errors and capitalisation mistakes which I amended but was still unable to run the code.

I then checked the settings and found that I had to make sure that all the settings must be catering to my computer's 64bit structure.  I changed all relevant settings regarding this and it finally worked.  What popped up is a track-bar, a normal video feed, a white window, and a lensed feed.  Manipulating the hue and saturation values in the track-bar, I isolated the colour I wanted to track which is shown below as the red flower.


The white screen was actually another feed window that eventually helped me isolate the colour I want to track (in white with black contrasting background).  The red lines on the normal feed were the relative positions that I moved the picture.

The next development is to find a code that can highlight the object being tracked.

Monday, 1 June 2015

Parts Procurement Update #3/Experimenting with OpenCV - 15/05/15

Logitech C170
I have now picked up the webcam and all the parts required.  We have been experimenting with them all but to no avail.  Firstly, the openFrameworks option mentioned last week also had problems with getting the right files and software versions.  However, shortly after, I have found another site that uses C++ Visual Studio to program object tracking.  The website is found below:

http://opencv-srf.blogspot.co.nz/p/opencv-c-tutorials.html

This site provides detailed step by step explanations and instructions on how to correctly install OpenCV into another IDE called Microsoft Visual Studio.  The tutorial is entirely done in C++ language and as a win32 console application.

This is an updated tutorial therefore this time I had the right software and firmware versions.  However, I still had trouble getting the code right and the OpenCV integration into the Visual Studio program.  By today, I have gotten the error codes down from 22 to 1 but cannot seem to get the last code right to run the project.  I have requested for another week in hope of sorting out the coding before we present our findings. 



Parts Procurement Update #2 - 15/05/15

There is still no sign of the webcam this week and I'm starting to lose hope of it arriving as it has been nearly a month now since I have originally ordered the item.

However, I have come across a sale in Dick Smith for the same camera! This time it would only cost $16 for the exact camera and it is new.  I am going to pick it up sometime early Monday.  For the mean time though, since the blog last week, I have found an old universal webcam which I have been trying to use to replicate the SparkFun project mentioned the week prior.

After a week and a half of trial and error, we have found that the SparkFun object tracking project is a dead end because of missing, outdated files and software versions.  Crucial to this project is the use of OpenCV or Open Computer Vision Library.  It is a library of visual algorithms that other softwares use for projects like ours which involve pictures, video, cameras and object tracking.  The OpenCV library must be installed into an integrated development environment (IDE) which in this case is Processing.

Processing is made for building, programming and developing applications for everyday computers.  This IDE has a similar structure and operating environment as Arduino.  In fact, Arduino is based on Processing IDE.

However, the versions suggested by the tutorial are unavailable and the Processing files are unavailable as well.  This consequently does not allow OpenCV to be installed correctly into the Processing IDE and would therefore not work.  Other avenues will now be explored. One such avenue is the OpenCV with the use of openFrameworks.  The link to the project is found below:

http://www.creativeapplications.net/tutorials/arduino-servo-opencv-tutorial-openframeworks/

As for the parts procurement, I have found the breadboard and have gotten the servos and all the parts required for the initial project.  These include:

- Buzzer
- 6x Super-bright LED
- Wires
- Generic Camera (while the actual camera is being ordered)
- Servos

So far the running cost is at $70.  This is still under our initial target of producing the lane departure system for under $250.

While we recognise that the lane departure system that we are building does not require pan and tilt servos, our focus is to get object tracking working on a webcam and to eventually modify and take out the servo programming in the code.  In addition, once the servos have been factored out, the costs involved with it will be removed from the total cost of the project to give an accurate account of how much it costs to produce a lane departure warning system.



Parts Procurement Update #1 - 8/05/15

After nearly 2 weeks of holiday, the camera still has not arrived yet.  I have been dealing with the seller but to no avail.  Meanwhile, other parts such as the Arduino Uno board has been sourced at a well-priced $25 from Surplustronics. 


Arduino Uno
I have been playing with the board and getting to know how it works.   I have been reading the Getting Started with Arduino book by Massimo Bengazzi which is shown below:

Introductory Book







Parts Procurement - 24/04/15

This week I have begun searching and ordering for my parts.  I have sourced out the Logitech c170 camera on TradeMe for $20 and have ordered it.


I'm still currently getting quotes on the Arduino Uno units on TradeMe and other retailers and getting the best price.  The main problem is to make sure that we do not get the cheaper clones which was said to have problems downloading sketches.


I'm also looking for my breadboard which I have seemed to misplace somewhere.  We are going through this guide to replicate the experiment:


https://www.sparkfun.com/tutorials/304