Vehware Project

"Single camera detecting person via OpenCV - Senior Project for a dual diploma team that would put together a "motion detection" mostly "person motion detection" from a video stream. The goal would be to alert a truck driver when he is backing up that a person is behind the truck. I checked out OpenCV as the basis for this capability using python. OpenCV comes with a body detector and we have used it in some of our work, so adapting it to a student project should be reasonable. The challenge is to get representative and realistic data so that you can define and then estimate an error metric"

"Single camera detecting person via OpenCV - Senior Project for a dual diploma team that would put together a "motion detection" mostly "person motion detection" from a video stream. The goal would be to alert a truck driver when he is backing up that a person is behind the truck. I checked out OpenCV as the basis for this capability using python. OpenCV comes with a body detector and we have used it in some of our work, so adapting it to a student project should be reasonable. The challenge is to get representative and realistic data so that you can define and then estimate an error metric"

Meeting Notes and General Notes

Week9 ~ 26.10.2015:

@Meeting

-Talked about latest condition of project

-Demonstrated a demo code for object detection based on color

-Supplied a camera to work with

@General Notes

-Demo codes are buggy

-Got advices about Senior project representation

-Focusing on first 30 second in presentation of the project

-Focusing on dlib.net and OpenCV object detection libraries

Week10 ~ 04.11.2015:

@Meeting

-We showed and discussed latest code samples that we have

-One sample works like detecting initial color to keep track and applies threshold on the picture. After that process it shows results on white object on black background

-We see that is not good way to proceed on

-Talked about frame difference comparison and object tracking

-Talked problems are caused by background movement

-Talked about whether do we handle the tracking based on direction of the movement

-Discussed about Optical Flow

-Discussed about graph that represents different ways of object tracking ( Point tracking, Kernel Tracking, Silhouette Tracking) options

-Talked about long range tracker topic ( Since it is general term we couldn't find exact documents related with OpenCV object tracking )

-Discussed based on relative speed from environment

-Agreed on move with Optical Flow algorithm to development, we think it solves most of the problems on moving background

Week11 ~

Week12 ~

Updates:

updated: 10/17/2015

- team members will install python and opencv in their machines

- dlib is need to be studied

- gantt chart is goingto be added in wiki

updated: 10/24/2015

- A camera was prepared by Mr. Bill in order to detect humans easly from a vehicle. The IR filter removed camera has a fisheye lens which has a wider range and closer focus. A sample capture is shown below.

- Bill would like the team to focus mostly on object trancking in the first stage rather than detecting humans.

- Object tracking methods are researched and frame difference algorithm is tested.

- In this method objects are easly recognized from the tresholded image. Biggest disadvantage of this medhod is not able to get effective result while camera is moving.

- After the conversation with Prof. Yin, we are advised to consider two algorithm which are Gavrila and Giebel [2002] and Viola et al. [2003]. We will study both paper and share the results in this wikipage.

updated: 10/25/2015

- Differetly from frame difference method, today HSV filtering method is used to track object. In this method range of selected object is detected in the colorspace of HVS. Tresholded image only shows the pixels whose colors is close to selected object with respect to HVS space.

- Issue with this method is even though HSV filtering can takes care with the scenario of mobile camera, since the pedesterians do not always wear the same color, they cannot be detected and classified by their colors.

updated: 10/27/2015

- The gantt chart is updated.

Project Plan:

1 DEFINITION :

There are several cases that truck driver hits a worker behind the vehicle. Since trucks are long vehicles and trails of them blocks drivers vision, these kind of accidents becomes inevitable. In order to provide a safe back driving for truck driver, It is thought that setting a mechanism at the back of trails which warn them whether there is a person or not. With this project we as Vehware team, would like to decrease count of live losts and physical damages.

1.1 Project Background and Overview

The primary focus of the project detecting humans behind the truck and warn driver. If the situation becomes too danger, the system may interfere and stop the vehicle. A list of prerequisites and key reasons for launch:In order to do that, first goal would be detecting human body. This issue can be solved with several different system developments. First method using intel realsense camera which able to determine distance of a object -human in this case-, wouldn’t needed to implement algorithm for estimating distance between camera and human. At the second method using IR camera to get more clear view. IR cameras are able to detect the temperature of all abject into the vision of the camera. Because it is assumed that in a construction zone, human has the higher temperature, detecting humans must be more straightforward on video stream of IR cameras compared to other cameras. Third option is using kinect. -costly-(explain)

1.2 Business Objectives

Vehvare project will provide team members with experience in professional development process and deliverables. We will apply our knowledge of Python, C++, streaming media, OpenCV, software development and product documentation. We will learn the expectations of the software industry for design, planning and development. In addition, we will learn the expected/appropriate level of timely company communications (ie.. Presentations, status updates, progress to plan, etc ). Our main customers will be companies those use big trucks in their business.

At the first stage of the project team members agreed on step by step approach while planning and development process. Since project’s main focus is warn truck drivers before dangerous situations those could cause bad results. Our team aims to decrease those problems. The prototype that we produce starts with recognizing people in a stream. After we adjust main structure of the object recognition, we will increase our prototype and we planning to increase analysis on backside danger situations. Those improvements increases safety of the backside of trucks.

1.3 Project Objectives

Short term objectives: Applying an algorithm that detect human in a picture or video stream. It doesn have to be optimized for working on the real trucks. Short term goals are mainly focused on developing a prototype for demonstration and getting investment with it.

Medium term objectives: Improving the algoritm for tracking humans as well. One of the medium term ojvectives would be decreasing false negative events. In this case, false negative event is where output of algorithm failed to find a human in a capture, while there actually is. The algorithm would be more sensitive to find out humans in the camera vision.

Long term objectives: After human tracking system is developed and tested on trucks, the project would be extented to check other blind spots of truck. The system can warn the driver if any vehicle is traveling into it’s blindspot area.This part of project provides a safe turning to a truck.

1.4 Project Constraints

In our project there are some limitations those may influence how to manage our project and proceed with our project. We can handle our project constraints in several categories.

Time frames: Our design specifications and simple prototype must be completed by first week December 2015. For fully complete work must be done by end of the May 2016.

Resources: For beginning we need to simple video recorder or any other device that we can provide us continuous video stream that we can examine on the data coming from the video stream. For early prototype we work on personal phones. For development part three software will work on the project for eight month.

Activity performance: For Vehware project, all team members have knowledge on basics on image processing and documentation of the project.

Scope: Scope of the project include almost all dangers that can be caused by people. Since backside of the trucks is almost invisible our early prototype can identify people behind the truck and warn the driver. The later versions of the project will divide backside of the truck into parts ( according to danger probability ) Challenges: Lighting conditions, enough equipment, pose

Cost: Not calculated yet....

1.5 Project Scope

Steps to detect humans in a video stream: According to the article “A survey of techniques for human detection from video by Neeti A. Ogale” humans can be detected based on their feature (shape, color, motion). Two main technique has been presented on article which are “background subtraction” and “direct detection” These are background subtraction and direct detection. With the background subtraction method a threshold value is assigned for a picture. In gray scale the pixel values which are above the threshold belongs to object (car, human, signs etc.) and below the threshold belongs to foreground which is unnecessary for this project. Since background subtraction require more steps than just detecting foreground, direct method is only one process to detect human. http://www-lb.cs.umd.edu/Grad/scholarlypapers/papers/neetiPaper.pdf

1.6 Project Scope Inclusions

Histograms of Oriented Gradients for Human Detection: The histogram of oriented gradients (HOG)is a feature descriptor used in computer vision and image processing for the purpose of object detection. The technique counts occurrences of gradient orientation in localized portions of an image. This method is similar to that of edge orientation histograms, scale-invariant feature transform descriptors, and shape contexts, but differs in that it is computed on a dense grid of uniformly spaced cells and uses overlapping local contrast normalization for improved accuracy. Navneet Dalal and Bill Triggs, researchers for the French National Institute for Research in Computer Science and Automation (INRIA), first described HOG descriptors at the 2005Conference on Computer Vision and Pattern Recognition (CVPR). In this work they focused on pedestrian detection in static images, although since then they expanded their tests to include human detection in videos, as well as to a variety of common animals and vehicles in static imagery.

2 APPROACH

2.1 Project Strategy

In order to succeed on our project we decided to clarify our goals and plans based on time ( short term and long term plans ). To mention about our basic steps while doing Vehware projects are could be listed below;

1-Documentation and Planning

2-Preparing appropriate test materials

3-Working on human detection algorithms, documentation and articles

4-Development

5-Testing

2.2 The Project Schedule

Posible solutions:

After the problem is analyzed well, we consider possible solutions which on the side of hardware and software.

Software solutions:

The paper[1] can be a referance since it is demonstrated on a conferance. If a paper cited 150 times and pass over 10 years on it, the problem must have been solved We should study it well when we access to BU network.

Hardware solutions: which hardware does fit best for possible solutions ? A camera with image processing or kinect like motion trackers ? can raspbery pi 2 or similar handle these software solutions? Input devices: intel realsense Camera: able to determine distance of a object, low range, low price considered to kinect. kinect: :higher range, able to classify movement of a human as default, high price($150) double cam: :cheapest solution. more programming work and CPU power required Infrared cam :reasonable cost $40, easy to detect human, difficult to detect other objects.

Input devices are listed based on Phd student's suggestions.