KR100779739B1 - Vehicle occupant head distance extraction method using stereo-vision - Google Patents

Abstract

A method for measuring and tracking a position of a passenger's head by a time difference by using a stereo camera is provided to detect whether the position of the passenger is within IP, OOP, COOP areas simply, quickly and accurately by recognizing the position of the passenger with a time difference of the position of the passenger's head. A method for measuring and tracking a position of a passenger's head by a time difference by using a stereo camera comprises the following steps of: inputting the right and left images of a passenger photographed by the stereo camera attached to the front of a vehicle(S101); making the inputted right and left images leveled by using a Gaussian kernel and reducing the capacity of the right and left images(S102); extracting a time difference image by using a LOG(Laplacian Of Gaussian) filter and an SAD(Sum of Absolute Differences) from the leveled and reduced right and left images(S103); obtaining an accumulated frame difference from the inputted right and left images and extracting an outline image through a morphology method; joining the time difference image and the outline image and extracting a dead area by using Hough transform(S202,S203); extracting a head time difference by using SAD(Sum of Absolute Differences) from the right and left images and the head area(S105,S106); and determining whether the passenger is within a range of IP(In Position), OOP(Out Of Position), and COOP(Critically Out Of Position) from the extracted head time difference(S107).

Description

Vehicle Occupant Head Distance Extraction Method Using Stereo-Vision}

FIG. 1 is a diagram illustrating a normal area and a dangerous area according to whether an airbag in a passenger seat can be operated by the US Highway Traffic Safety Administration.

2 is a flowchart illustrating a method for detecting and tracking a vehicle occupant head position parallax according to an exemplary embodiment of the present invention.

3 is a diagram illustrating a parallax image information map according to a distance between a stereo camera and a passenger.

4 is a diagram extracting contour lines by using a accumulated frame difference and morphology technique.

FIG. 5 is a diagram illustrating stereo geometry for parallax and distance detection of a passenger's head position. FIG.

6 is a diagram illustrating a left eye image and a right eye image photographed by a stereo camera according to an exemplary embodiment of the present invention.

7 is an experimental result diagram successfully showing head position parallax detection and tracking using the method according to an embodiment of the present invention.

The present invention relates to a method for detecting and tracking a passenger's head position parallax using a stereo camera, and more particularly, to a method for identifying a passenger's position by a parallax value of a passenger's head position based on binocular parallax.

In the event of a car accident, the safety of the occupants becomes very dangerous and can result in serious injury or death. Thus, for the safety of vehicle occupants, each car manufacturer has begun installing air bags in cars. The use of airbags in automobiles has greatly improved occupant safety.

However, if the airbag is operated when the occupant is in an improper position, the occupant may be very seriously injured. According to the National Highway Traffic Safety Administration (NHTSA), more than 200 people were killed by direct airbag operation from 1990 to April 2002. Most of these deaths were children (144 children, 87 adults). The number of serious injuries resulting from airbag operation in an inadequate occupant position is increasing.

Accordingly, the US Highway Traffic Safety Administration (NHTSA) has divided the occupant's position into sections as shown in FIG.

FIG. 1 is a diagram illustrating a normal area and a dangerous area according to whether an airbag in a passenger seat can be operated by the US Highway Traffic Safety Administration.

Referring to FIG. 1, in order to determine the correct position of the occupant for the airbag to operate, an IP (In Position), Out of Position (OOP) and Critically Out of Position (COOP) area between the seat and the dashboard Divided by. That is, IP (In Position) is the normal position of the occupant, and the occupant is close to the chair. OOP (Out of Position) is the abnormal position (hazardous position) of the occupant, the occupant is between the chair and the handle. Finally, the COP (Critically Out of Position) is a very dangerous position for the occupant, where the occupant is on the steering wheel.

As such, in order for the airbag to operate normally according to the position of the occupant, there is an urgent need for a method capable of detecting in real time the position of the occupant in the IP, OOP, and COOP areas in real time.

An object of the present invention is to provide a method for determining the position of the occupant by the parallax value of the head position of the vehicle occupant based on binocular parallax.

A vehicle occupant head position parallax detection and tracking method using a stereo camera of the present invention for achieving the above object comprises the steps of: (a) inputting a left image and a right image of the occupant taken from the stereo camera attached to the front of the vehicle; (b) flattening the input left and right images using a Gaussian kernel and reducing the capacity of the left and right images; (c) extracting the parallax image from the flattened and reduced left and right images by using a Laplacian of Gaussian (LoG) filter and Sum of Absolute Differences (SAD); (d) while the steps (a) and (b) are performed, obtaining a cumulative frame difference from the left image or the right image input in the step (a), and extracting the contour image through a morphology technique; (e) combining the parallax image obtained in step (c) and the contour image obtained in step (d) and extracting a head region using a Hough transform; (f) extracting head parallax from the left and right images input in the step (a) and the head region extracted in the step (e) using SAD (Sum of Absolute Differences); And (g) determining whether the occupant is in an IP (In Position), OOP (Out of Position) or COOP (Critically Out of Position) range from the extracted head parallax.

The method may further include tracking the head position using a Kalman filter from the head region extracted in the step (e).

The method may further include determining a size or a feature of the vehicle occupant from the head region extracted in the step (e).

Further, the distance between the stereo camera and the occupant is determined by the following equation 1 from the head parallax extracted in step (f).

[Equation 1]

Where r is the distance between the occupant's head and the stereo camera, b is the distance between the stereo camera, f is the focal length of the lens, and d = dl-dr and dl and dr are between the center of the image and the occupant in the left and right images, respectively. Indicates distance.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

2 is a flowchart illustrating a method for detecting and tracking a vehicle occupant head position parallax according to an exemplary embodiment of the present invention.

Referring to FIG. 2, first, a left image and a right image are obtained from a stereo camera (S101).

The input left and right images are smoothed using a Gaussian kernel and then downsampled to a size of 1/4 (S102). The Gaussian Kernel is mask information for planarization. The planarization removes noise of the original image and makes the surrounding pixels the same. However, although the information of the original image may be lost through flattening, the head position to be searched may be found with little information.

Next, a Laglacian of Gaussian (LoG) filter is taken to reduce the difference between the left and right images, and the SAD (Sum of Absolute Differences), which is a value obtained by adding the difference between the left and right image brightness values, is applied to the edge of the left and right images. ) Since it is a stereo image, the positions of the top and bottom images of the left and right sides are the same and only the left and right images are represented differently. At this time, the parallax image is extracted by disparity extraction from the left and right images, and a parallax image information map is obtained using the disparity extraction (S103).

Here, the Laplacian of Gaussian (LoG) filter is one of the second-order differential edge operators, and according to the present invention, it is a filter for edge detection, and when there is a color boundary of the image, only the region can be extracted as a derivative. Laplacian filters provide edges of second order differentiation and detect edges in all directions at the same time, if more edge detection is possible than other filters. The sign of the resultant image is changed at the edge part, and this change of sign is called zero crossing, and it is necessary to find the place where the zero crossing of the image occurs after the operation and set the output pixel value accordingly. The disadvantage of the Laplacian filter is that it is sensitive to noise so that more edges can be detected than the actual result of zero crossing processing. Thus, Gaussian is processed before Laplacian to reduce noise.

Median filters or morphology techniques are also used to reduce noise in the image, which is then scaled back to its original size.

3 is a diagram showing a parallax image information map according to the distance between the stereo camera and the occupant, (a) is an original image when the distance is 70 cm, (b) is a parallax image after parallax extraction, and (c) The images (a) and (b) are combined images. Also, (d), (e), and (f) are images when the distance is 50 cm.

Meanwhile, a cumulative frame difference is obtained separately from the right image (motion capture) (S202), and only a desired contour region is extracted through a morphology technique (S203). Here, the morphology technique extracts a desired part from the image by reflecting information of an object of which the geometric shape is known in advance. By using this, other than the object of interest is regarded as noise.

4 is a diagram in which contours are extracted using the accumulated frame difference and morphology technique, (a) is one frame, (b) is a difference image accumulated for five frames, and (c) is a result image to which the morphology technique is applied.

Next, when the previously obtained parallax image and the contour image are combined, the head region may be detected (S104). The Hough transform is used to find the ellipse that best fits the head position and its position. Here, the Hough transform is used to extract a pattern over an entire area of the image, and is mainly used in an image including scattered patterns, patterns with holes, patterns with noise, and the like. In the present invention, an ellipse (face) is extracted. It is used to Considering the vertical axis of the ellipse, each pixel is averaged on the vertical axis.

The head disparity is extracted from each of the stereo left and right images and the detected head region using SAD (Sum of Absolute Differences) (S105).

The head disparity may be used to determine in which area of the vehicle the head is located. In this case, the IP (In Position), OOP (Out of Position) or COOP (Critically Out of Position) areas described above are determined according to whether the head is at the front or the rear (S107).

Such a parallax and distance detection of the occupant's head position can be detected by Equation 1 below with reference to FIG.

Where r is the distance between the tracked occupant's head and the stereo camera, b is the distance between the cameras, and f is the focal length of the lens. Also, as d = dl-dr, dl and dr represent the distance between the center of the image and the object in the left or right image, respectively. For example, assuming that f of the stereo camera is 3.6 mm and b is 9.5 cm, the head parallax d is 24 pixels (1 pixel is 0.02 mm), the distance between the object and the stereo camera is 70 cm.

Meanwhile, the Kalman filter tracks the position of the head by using two vectors representing the head position (S106). The Kalman filter is a well-known algorithm whose basic concept is to track the optimal value through iterative operations, and thus a detailed description thereof will be omitted. To compensate for the disadvantage of using fewer vectors, we apply variances between the predicted and actual values to the state and measurement models.

Further, the size or characteristics of the occupant are determined from the head region detected in the step S104 (S108).

FIG. 6 is a diagram illustrating a left eye image and a right eye image captured by a stereo camera according to an embodiment of the present invention, and FIG. 7 is an experiment illustrating successful detection and tracking of head position parallax using a method according to an embodiment of the present invention. The result is also.

6 and 7, an image obtained from a vehicle was used as an experimental image to verify a method according to an exemplary embodiment of the present invention. Stereo cameras were installed in front of the occupants. Over 500 images are 320 x 240 at 15 fps, which includes three experimenters taking various actions. Experimental results show that the proposed method detects and tracks the occupant's head properly.

According to the method of determining the position of the occupant by the parallax value of the head position of the vehicle occupant based on the binocular parallax of the present invention, it is simple, quick and accurate to determine where the occupant's position is in the IP, OOP, and COOP areas. There is an effect that can be detected in real time.

Claims (4)

(a) inputting a left image and a right image of a passenger photographed from a stereo camera attached to the front side of the vehicle; (b) flattening the input left and right images using a Gaussian kernel and reducing the capacity of the left and right images; (c) extracting the parallax image from the flattened and reduced left and right images by using a Laplacian of Gaussian (LoG) filter and Sum of Absolute Differences (SAD); (d) while the steps (a) and (b) are performed, obtaining a cumulative frame difference from the left image or the right image input in the step (a), and extracting the contour image through a morphology technique; (e) combining the parallax image obtained in step (c) and the contour image obtained in step (d) and extracting a head region using a Hough transform; (f) extracting head parallax from the left and right images input in the step (a) and the head region extracted in the step (e) using SAD (Sum of Absolute Differences); And (g) determining whether the occupant is in an IP (In Position), OOP (Out of Position) or COOP (Critically Out of Position) range from the extracted head parallax. Parallax detection and tracking method for occupant head position. The method of claim 1, A method for detecting and tracking a vehicle occupant's head position using a stereo camera, further comprising tracking a head position using a Kalman filter from the head region extracted in the step (e). The method of claim 1, Determining and tracking the vehicle occupant head position using a stereo camera, characterized in that further comprising the step of determining the size or characteristics of the vehicle occupant from the head region extracted in the step (e). The method of claim 1, Method for detecting and tracking a vehicle occupant head position using a stereo camera, characterized in that the distance between the stereo camera and the occupant is determined by the following equation 1 from the head parallax extracted in the step (f). [Equation 1]

Where r is the distance between the occupant's head and the stereo camera, b is the distance between the stereo camera, f is the focal length of the lens, and d = dl-dr and dl and dr are between the center of the image and the occupant in the left and right images, respectively. Indicates distance.

Images