Human Following Robot with implementation of FPGA`s

I am going to enter my last year of Electronic Engineering, so for our final year projects, we submitted proposals. First our project was Human Following Robot using 'GPS & Xbee', by subtraction of frames we would have a lock-on on our user and know his position & then follow him one or many units would be placed on the robot and a single unit would be put in the pocket of the user, but anyhow it was rejected, now our teachers have said to us research on making a Human Following Robot with Image Processing using FPGA`s, I have searched and have found that LOGi FPGA is available for use with either Raspberry Pi and BeagleBone. So my main question is that is this project feasible can it be made, and can Kinect or other cameras be interfaced with the LOGi i have seen only the LOGi cams having model no. OV---, can other cameras be also interfaced with it. Also please suggest some good projects related to robotics. Thanks

Comments

  • Hi @hzk17

    Congrats on making to your final year of EE - that is certainly where the fun starts to happen!  

    Jonathan Piat can give you much better infromation that I as he has a great deal of machine vision experience, but he is out of town until Jan 1.  

    As was pointed out in the E14 thread, you will really need to create a detailed spec of what you want your system to do and what pieces your want the FPGA to do.  For the "human following" algorithm you should look into popular current machine vision algorithms that you might want to use directly or customize for your applcation.  Algorithms such as blob (colored region detection) or pattern detection might work well if you had a human wearing a uniquely identifiable shirt color or pattern on the shirt.  

    The overall system will likely end up being quite complex and where you want to focus you attention on the machine vision using FPGA, I believe it would be quite convenient to offload the projects basic processing tasks to the Pi or the bone wihout needing to make the FPGA project any more complex than you need to. 

    Jonathan has some good FPGA applications taht you can use as reference to begin to get a feel fow how to image process data in the FPGA and then customizing to fit you own application.  
    Here is a recent article Jonathan wrote that has some very good details on image processing a laser line:

    Anyhow, I would start with nice block diagram of the complete system showing which parts will do what (FPGA/MCU) and then decide which platform may suit you best.

    Cheers,

    Mike

    hzk17
  • Hi,
    do you have any idea of the algorithm/method you want to use for this purpose ? Human following would require human detection (usually performed through classification or by using special tags) and target tracking. Running this task on the logi-bone/pi would require to understand how you want to partition the processing between the FPGA and the processor.  The FPGA could handle the pixel classification and the processor can be in charge of the tracking part. Image segmentation/classification such as boosting could be particularly suited for the FPGA.

    Your best take on getting something to work, is to start prototyping on a PC with OpenCV and then decide which part of your algorithm you want to move to the FPGA.

    Regards,

    Jonathan Piat
    hzk17
  • Thank you Sir for your kind reply. I have made a block diagram, FPGA`s are complex little beasts i have grip on Microcontrollers but my friends and i have decided to make this project, Sir we have submitted our proposals we havent been yet called by our Instructors to define the project if it gets accepted i will then start the project in full gear. Sir i havent done extensive research yet, i have information but not in depth i am waiting to get the project accepted.
  • Thank you Sir @mjones & @jpiat
  • Sir @jpiat (Jonathan Piat) i dont have any idea of the algorithm/method. Thats why i am asking questions. Many teachers of our university have knowledge about FPGA`s but not in detail. Sir i have used Matlab in my 5th & 6th semester, I think OpenCV is also like matlab, will do what you have said, and if my proposal gets accepted then i will need some help and guidance from the teachers over here. Thanks.
  •  Having some recent experience in this, namely 2D Kalman image filtering for real time fluoroscopy, I would offer the following advice. Start by tracking retro reflectors on the humans shoes, then track the shoes with/without the retros. Then graduate to the larger problem. If you have Matlab with Simulink USE IT. It will save you a absolutely huge amount of time.
    hzk17
  • One other bit of advice comes to mind. Several years after building my FPGA base real time Kalman Image processor I came back to look at the problem with the GPU also added to the signal processing mix. Everything had flipped. Even though I was using high performance FPGAs, there was far more horse power in the GPU. My suggestion would be to use the FPGA to get your image over to a GPU say via an ethernet port, perform your algorithms there at the GPU, and then send back control vectors back to the robot.
    hzk17
  • You are right in saying that a GPU as more horsepower (and easier ti use development tools) than a FPGA, but they have lower power/performance ratio than what a FPGA can achieve.

    Comparison of FPGA vs GPU are often made, but they make little sense because while the FPGA performance comes at high development cost, the resulting architecture can be very close to the optimal case (in every way, area, power consumption, latency, throughput ...) while a GPU is a general purpose parallel machine and will most of the time be over-sized for a given computation or will limit performance application because of their memory sub-system (not very good at random access) and cache management (FPGA can be used to design application specific cache management).

    FPGA are very good at pixel level operations (frame level operations only depends on the ability to store frames, hence memory), so they can perform all the image oriented application for the human detection and then send whatever is extracted from the image (classification results, marker detections ...) to the CPU that can do the tracking part (Kalman is a good candidate for that task). Sending the frame to a GPUfor computation will result in a high latency, preventing the robot from being agile (fast and reactive).
    hzk17
  • I totally agree. However, knowing the particular advantages of one approach vs another takes time to learn and understand. All well beyond the experience level of a young engineer, On the other hand, I quess that was why the question was asked in the first place.
    hzk17
  • Thanks again sirs @cybin & @jpiat.
Sign In or Register to comment.