KR20160056189A - Apparatus and method for detecting pedestrian and alert - Google Patents

Abstract

Disclosed are an apparatus and a method for detecting a pedestrian and alerting the same. A pedestrian detection and alerting apparatus according to an embodiment of the present invention includes: a driver face image acquisition part to acquire a face image of a driver; a vehicle front image acquisition part to acquire a front image of a vehicle; a gaze detection part to detect a gaze of the driver in the face image of the driver acquired by the driver face image acquisition part; a sight image determining part to determine the front image of the vehicle corresponding to a fixed viewing angle by referring to the detected gaze of the driver by the gaze detection part in the front image of the vehicle acquired by the vehicle front image acquisition part as a sight image according to the gaze of the driver; a detection area setting part to set an image except the sight image according to the gaze of the driver determined by the sight image determining part in the front image of the vehicle as an image of a range to detect a pedestrian; a pedestrian detection part to detect the pedestrian in the image of the range to detect the pedestrian set by the detection area setting part; and an alarm output part to output an alarm in case the pedestrian is detected by the pedestrian detection part. The apparatus of the present invention can detect the pedestrian efficiently and rapidly.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a pedestrian detection alarm device,

The present invention relates to an apparatus and method for detecting and alerting pedestrians.

Power vehicles, in particular automobiles, are indispensable objects of modern society that are widely used to transport people or things from one place to another. However, the damage caused by the collision of the vehicle is increasing with the sudden increase of the present vehicle. Especially, when the vehicle collides with the pedestrian, the mortality rate of the pedestrian is increasing.

On the other hand, the research for detecting human in real time using a camera is utilized in various fields such as an automatic security monitoring system, a robot application, a medical application system, and an intelligent vehicle system. Especially, in the field of intelligent vehicle system, studies are being conducted to make vehicles operate safely without accident by utilizing human detection research. For example, 'intelligent automobile safety technology' is a technology in which an automobile recognizes its surroundings by a sensor such as a camera, a radar, or an ultrasonic sensor, and when the electronic control device of the car is determined as a dangerous situation, It says. Typical intelligent automotive safety technologies include frontal collision warning systems and automotive emergency braking systems.

That is, a radar sensor or a camera is attached to the front of the vehicle to detect the presence of an obstacle in front of the vehicle and the type of the obstacle, and a device is actively under development to warn the driver even if the driver does not inadvertently decelerate. For example, there is a method of calculating a collision speed (collision speed) with a forward object or a distance to a forward object using a speed sensor or the like, and performing braking in accordance with the resultant value.

An object of the present invention is to provide an apparatus and method for detecting and alerting a pedestrian by reflecting the driver ' s view and the frontal state of the vehicle.

Another problem to be solved by the present invention is to provide a pedestrian detection alarm apparatus and method for efficiently and quickly detecting a pedestrian.

According to an aspect of the present invention, there is provided a pedestrian detection alarm apparatus including a face image acquisition unit for acquiring a face image of a driver, a forward image acquisition unit for acquiring a forward image of the vehicle, A line of sight detection unit for detecting a line of sight of the driver in the face image of the driver acquired by the face image acquisition unit of the vehicle, a line of sight of the driver detected by the line of sight detection unit in the forward image of the vehicle acquired by the forward image acquisition unit of the vehicle A visual field image determining unit for determining a frontal image of the vehicle corresponding to a predetermined viewing angle as a center of gravity of the vehicle based on the sight line of the driver, A detection area setting unit for setting an image excluding the image of the pedestrian to an image of a range for detecting the pedestrian, It includes groups pedestrian detection area detection unit for detecting a pedestrian from the image of the range of detecting the pedestrian set by the setting section and the alarm output section for outputting an alarm in the case where the pedestrian detected by the pedestrian detecting unit.

According to an aspect of the embodiment, the eye-gaze detection unit can determine the eye area of the driver from the face image of the driver to extract the candidate's gaze and select the final gaze of the driver from the candidate gaze of the extracted driver.

According to another aspect of the embodiment, the pedestrian detection alarm device sets an image of a range in which a pedestrian detected by the detection area setting unit is excluded from a video image in a range excluding a predetermined upper and lower range image And may further include a detection area modification unit.

In this case, if it is determined that the driver is in a singular state in the face image of the driver, the detection region correcting unit sets the forward image of the vehicle as an image of a range for detecting the pedestrian, and the alarm output unit, The intensity of the alarm sound can be greatly adjusted and output.

According to still another aspect of the present invention, the pedestrian detection alarm device calculates a danger level for the detected pedestrian using the distance between the vehicle and the pedestrian detected by the pedestrian detection unit and the traveling direction of the detected pedestrian And a risk level calculating unit.

In this case, the danger level calculating unit may calculate the risk level to be low when the detected direction of the walker is away from the vehicle.

Alternatively, the risk level calculator may calculate the risk level to a high level when the detected direction of the walker approaches the vehicle.

Alternatively, the alarm output unit may adjust the intensity of the alarm sound according to the danger level calculated by the danger level calculation unit and output the alarm sound.

According to another aspect of the embodiment, the alarm output unit may output an alarm sound in a direction in which the pedestrian is detected.

According to another aspect of the present invention, the alarm output unit may adjust the intensity of the alarm sound according to the distance detected by the pedestrian and output the alarm sound.

According to an aspect of the present invention, there is provided a pedestrian detection alarm method including: acquiring a face image of a driver; acquiring a forward image of the vehicle; acquiring a driver's gaze from the obtained face image of the driver; Determining a front image of the vehicle corresponding to a predetermined viewing angle as a visual field image according to a driver's line of sight based on the detected driver's gaze in the front image of the obtained vehicle, A step of detecting a pedestrian in an image of a range where the set pedestrian can be detected, and a step of detecting a pedestrian when the pedestrian is detected, And outputting an alarm.

According to another aspect of the present invention, there is provided a computer readable recording medium storing a program for executing a method of detecting a walking number according to an embodiment of the present invention.

As described above, according to the embodiment of the present invention, the pedestrian detection area can be reduced by reflecting the driver ' s view and the frontal state of the vehicle, thereby minimizing the pedestrian detection time.

According to an embodiment of the present invention, the pedestrian detection area can be reduced by reflecting the driver ' s visual field and the frontal state of the vehicle, so that the pedestrian detection time can be minimized even for an image photographed with high image quality.

According to an embodiment of the present invention, the alarm sound may be output in accordance with the distance between the vehicle and the pedestrian or the direction of the pedestrian.

1 is a block diagram showing a schematic configuration of a pedestrian detection alarm device according to an embodiment of the present invention.
2 is a block diagram showing a schematic configuration of a pedestrian detection alarm device according to another embodiment of the present invention.
3 is a view illustrating a viewing angle of a driver according to an exemplary embodiment of the present invention.
FIG. 4 is a view showing an area set as an image of a range for detecting a pedestrian according to an embodiment of the present invention.
5 is a view showing an area set as an image of a range for detecting a pedestrian according to another embodiment of the present invention.
6 is a view showing an area set as an image in a range for detecting a pedestrian according to another embodiment of the present invention.
7 is a flowchart illustrating a pedestrian detection alarm method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail with reference to the accompanying drawings. However, these drawings are only for illustrating the contents and scope of the technical idea of the present invention, and the technical scope of the present invention is not limited or changed. It will be apparent to those of ordinary skill in the art that various changes and modifications can be made within the scope of the technical idea of the present invention based on these examples.

In addition, terms and words used in the present specification are terms selected in consideration of functions in the embodiments, and the meaning of the terms may be changed according to the intention or custom of the invention. Therefore, the terms used in the following embodiments are defined according to their definitions when they are specifically defined in this specification, and unless otherwise defined, they should be construed in a sense generally recognized by ordinary artisans.

1 is a block diagram showing a schematic configuration of a pedestrian detection alarm device according to an embodiment of the present invention.

Referring to FIG. 1, a pedestrian detection and alarm apparatus 100 according to an embodiment includes a face image acquisition unit 110, a vehicle front image acquisition unit 120, a sight line detection unit 130, a vision image determination unit 140 A detection area setting unit 150, a pedestrian detecting unit 170, and an alarm output unit 190. The configuration of the pedestrian detection alarm device 100 shown in Fig. 1 is an example, and the pedestrian detection alarm device 100 may have only a part of the modules shown in Fig. 1 and / Other modules may be additionally provided. For example, the pedestrian detection and alarm apparatus 100 may further include a communication unit having a communication function for communicating with other devices.

The face image acquisition unit 110 of the driver acquires the face image of the driver. Here, 'driver's face image' refers to the front face of the driver's eyes, nose, and mouth, and particularly the image including the driver's eyes. According to the embodiment of the present invention, there is no particular limitation on the method of acquiring the face image of the driver. The face image of the driver can be acquired by the image input sensor, but the present invention is not limited thereto. For example, the face image acquisition unit 110 of the driver can acquire the face image of the driver by using a camera installed in the front of the driver's seat in the vehicle and photographing the driver. For example, the face image acquisition unit 110 of the driver can acquire the three-dimensional face image of the driver using the three-dimensional camera. For example, the face image acquisition unit 110 may acquire a face image of a driver using an infrared camera. In another example, the face image acquisition unit 110 of the driver can acquire the face image of the driver by receiving the face image of the driver photographed by the camera.

A forward image acquisition unit 120 of the vehicle acquires a forward image of the vehicle. According to the embodiment of the present invention, there is no particular limitation on the method of acquiring the forward image of the vehicle, but the forward image of the vehicle can be obtained by the image input sensor, but the present invention is not limited thereto. For example, the vehicle's front image acquisition unit 120 may acquire a forward image of the vehicle using a camera positioned in front of the vehicle and photographing a forward image of the vehicle. For example, the forward image acquisition unit 120 of the vehicle may acquire the forward image of the vehicle three-dimensionally using the three-dimensional camera. For example, the forward image acquisition unit 120 of the vehicle may acquire a forward image of the vehicle by receiving a forward image of the vehicle photographed by the camera.

The line of sight detection unit 130 detects the line of sight of the driver from the face image of the driver acquired by the face image acquisition unit 110 of the driver. Here, 'driver's gaze' means a straight line connecting the driver's eye direction, the center point of the driver's pupil, and the view point of the extraterrestrial.

Specifically, the line of sight detecting unit 130 detects the driver's face image from the driver's face image acquired by the driver's face image obtaining unit 110, detects the driver's eye image, And the driver's gaze can be detected based on the acquired pupil. For example, the eye can be detected by comparing the similarity with the best matching eye model in comparison with the eye model prepared in advance around the predictable eye region in the face region of the driver. Even if an eye-detected region is detected, it is determined that it is not an eye if the degree of similarity is very low. On the other hand, when the degree of similarity is higher than a predetermined reference value, it is determined that the eye is detected.

Next, the line-of-sight detecting unit 130 can acquire the position of the pupil in the detected eye image, compare the obtained pupil image with the pupil model prepared in advance, and detect the line of sight of the driver. For example, the pupil is darker than other parts of the eye, and the size of the pupil varies due to the iris movement due to the amount of light entering the eye. Therefore, the darkest part in the detected eye image is identified as the pupil position And the driver's gaze can be detected based on the obtained pupil.

On the other hand, according to the embodiment of the present invention, there is no particular limitation on the face detection method, but a conventionally known face detection algorithm or the like can be used for face detection. The face detection method resets frames to various sizes based on the face characteristics, and detects faces of all the images of the resized frames. Generally, a pyramid image is generated in order to detect faces of various sizes. Then, the pyramid image is moved by one pixel and a classifier (for example, a neural network (for example, Neural Network), Adaboost, Support Vector Machine, and the like).

According to an embodiment, the line of sight detecting unit 130 may determine the eye area of the driver in the face image of the driver to extract the candidate line of sight of the driver, and may select the driver's final line of sight from the candidate line of sight of the driver. For example, the driver may change his / her face while the vehicle is driving. Since the size of the pupil is variable due to the movement of the iris according to the amount of light entering the eyes, the driver's gaze . The candidate's gaze of the driver may be a gaze having a characteristic that the pupil of the driver's eye region is judged to be pupil of the driver's face image. From the extracted candidate lines, the candidate line with the highest matching is selected by applying the matching rule with the camera, the LED, the relation with the face position, the size of the pupil immediately before, and the like. That is, the line-of-sight detecting unit 130 can select a candidate line of sight that matches the driver's line of sight among the candidate lines of the extracted driver as the final line of sight of the driver.

The view image determining unit 140 determines a view angle of the vehicle corresponding to a predetermined viewing angle based on the sight line of the driver detected by the sight line detecting unit 130 in the forward image of the vehicle obtained by the vehicle front image obtaining unit 120 The forward image is determined as the visual field image according to the driver's gaze. Here, the 'visual field image according to the driver's gaze' means an image corresponding to the range of the driver's visual acuity. The range in which a person can see with one eye is 60 ° in the upper part, 70 ° in the lower part, and 100 ° in the left and right, respectively, centering on the human eye. However, it is difficult to perceive the detailed state of the object and the color such as the field of view out of the range of about 30 degrees centered on the human's gaze.

On the other hand, the viewing angle of the driver depends on the driving speed of the driver. For example, if the driver's vehicle running speed is 63 km / h, the driver's viewing angle is 38, the driver's vehicle running speed is 76 km / h, the driver's viewing angle is 27, h, the driver's viewing angle is 19 °. The viewing angle of the driver will be described later in detail with reference to FIG.

According to one embodiment, the visual field image determining unit 140 may determine the viewing angle differently according to the vehicle driving speed of the driver. For example, the view image determining unit 140 determines the viewing angle of the driver to be 38 degrees when the driver's vehicle running speed is 63 km / h, or when the driver's vehicle running speed is 76 km / h, the driver's viewing angle is 27 ° or the driver's viewing angle is determined to be 19 ° when the driver's vehicle driving speed is 110 km / h, and the front image of the vehicle corresponding to the viewing angle, respectively, You can decide by video.

According to another embodiment, when the driver is looking at the front light and the driver's gaze is included in the range of the front image of the vehicle, a part of the front image of the vehicle may be a visual image according to the driver's gaze. Specifically, the view image determining unit 140 can determine the forward image of the vehicle corresponding to the predetermined viewing angle as the view image according to the driver's line of sight, with the driver's line of sight as the center.

According to another embodiment, when the driver does not gaze forward due to reasons such as navigation or rear view, and the driver's gaze is not included in the range of the forward image of the vehicle, a part of the forward image of the vehicle is displayed on the driver's gaze It can not be the following visual field image. At this time, the pedestrian can be detected in the entire range of the front image of the vehicle.

The detection area setting unit 150 may detect an image of a forward image of the vehicle obtained by the front image acquisition unit 120 of the vehicle, excluding a view image according to the driver's sight determined by the view image determining unit 140, Set the image to the range to be set. The location where the pedestrian can be located in the image obtained from the fixed camera is definite. In the case of a standard lens of a general camera, the angle of view is 48 ° to 60 °, which includes images in a larger area than the driver's viewing angle.

The pedestrian detecting section 170 can detect the pedestrian only in the set range by setting the detection area setting section 150 to set the image in the range for detecting the pedestrian. If the range in which the pedestrian is to be detected is less than the entire range of the front image of the vehicle, the pedestrian detection speed can be increased. In addition, since the field of view according to the sight line of the driver is considered, it is unnecessary to detect the pedestrian with respect to the area recognized by the driver.

According to an embodiment, when the driver watches the back light and a part of the front view image of the vehicle overlaps with the view image of the driver's gaze, the pedestrian may not be detected for the image of the overlap area. At this time, in the forward image of the vehicle, the image excluding the view image according to the driver's gaze is set as the image of the range in which the pedestrian can be detected.

According to another embodiment, when the driver does not watch the front of the vehicle due to reasons such as navigation or the rear view of the vehicle, if the visual field image according to the driver's gaze and a part of the frontal image of the vehicle do not overlap, Is set to a range of images to be displayed.

The pedestrian detecting unit 170 detects the pedestrian from the image in the range where the pedestrian set by the detection area setting unit 150 can be detected. According to the embodiment of the present invention, there is no particular limitation on the method of detecting the pedestrian, but it is possible to detect the pedestrian by using a conventionally known algorithm for detecting the pedestrian in the image. For example, pedestrian detection techniques include Haar Feature-based Cascade Classifier, Histogram of Oriented Gradient (HOG), and Cascade HOG.

According to one embodiment, the pedestrian detector 170 can detect a pedestrian using a pedestrian detector based on Histogram of Oriented Gradient (HOG). The HOG-based method is a general methodology that can be applied to detect various objects such as passenger cars, trucks as well as pedestrians by extracting the local gradient histogram characteristic of an object image .

For example, the simplest way to detect a pedestrian from an image acquired from a camera is the Background Subtraction method, which separates moving objects from the background of the image. Background image method is a very effective way to extract the silhouette of a moving object.

The pedestrian detection method can be classified into a model-based method and a shape-based method according to a pedestrian detection method. First, the model-based method is a method of recognizing using a motion, constructing a walking model for a human walking posture, and analyzing a walking pattern for use in detection. The shape - based method can be classified into shape feature extraction method and shape feature classification method. The shape feature extraction method includes a Harv wavelet-based method, a Histogram of Oriented Gradient method, and a LRF (Local Receptive Field) method. The shape feature classification method includes SVM (Support Vector Machine) , Neural networks (Neural Networks), and Adaboost algorithms. In order to increase the feature extraction and classification speed, two or more algorithms can be applied together to detect the pedestrian in real time.

The alarm output unit 190 outputs an alarm when the pedestrian detection unit 170 detects a pedestrian. Here, the 'alarm' means various various means for informing the driver of the situation where the pedestrian is detected, thereby inducing the awareness. For example, the alarm may be an alarm sound output through a speaker in the vehicle. As another example, the alarm may be an alarm message output through a speaker inside the vehicle. The means of alerting may vary, but is not limited thereto.

According to one embodiment, the alarm output unit 190 can output an alarm sound in the direction in which the pedestrian is detected. For example, when the pedestrian detecting unit 170 detects a pedestrian in the right direction with respect to the driver, the alarm output unit 190 can output an alarm sound through the right speaker in the vehicle. Alternatively, when the pedestrian detecting unit 170 detects the pedestrian in the left direction with the driver as the center, the alarm output unit 190 can output an alarm sound through the left speaker in the vehicle.

According to another embodiment, the alarm output unit 190 may adjust the intensity of the alarm sound according to the distance detected by the pedestrian and output the alarm sound. For example, as the distance detected by the pedestrian is closer to the vehicle, the alarm output unit 190 can output a large alarm sound. Alternatively, as the distance detected by the pedestrian is far from the vehicle, the alarm output unit 190 can output a small alarm sound.

2 is a block diagram showing a schematic configuration of a pedestrian detection alarm device according to another embodiment of the present invention.

2, the pedestrian detection and alarm apparatus 200 includes a face image acquisition unit 210, a vehicle front image acquisition unit 220, a sight line detection unit 230, a vision image determination unit 240 A detection area setting unit 250, a detection area setting unit 260, a pedestrian detecting unit 270, a danger level calculating unit 280, and an alarm output unit 290.

The pedestrian detection alarm device 200 shown in FIG. 2 further includes a detection area correction unit 260 and a risk level calculation unit 280. The pedestrian detection alarm device 100 ). Hereinafter, a description will be given of the pedestrian detection alarm device 200 of FIG. 2, focusing on the difference from the pedestrian detection alarm device 100 of FIG. Therefore, the matters not specifically described with respect to the pedestrian detection alarm device 200 of FIG. 2 are the same as those described with reference to the pedestrian detection alarm device 100 of FIG. 1, except that the inherent characteristics of the respective components are not applicable The same applies.

The detection area correcting unit 260 sets the image of the range excluding the image of the predetermined upper and lower areas as the image of the range where the pedestrian can be detected from the image of the range in which the pedestrian set by the detection area setting unit 250 can be detected. Some of the upper and lower regions in the forward image of the vehicle are excluded from the image of the range where the pedestrian can be detected because the probability of detecting a pedestrian that can be associated with a vehicle accident is low or not present.

According to one embodiment, when it is determined that the driver is in a singular state in the face image of the driver, the detection area correcting unit 260 can set the forward image of the vehicle as an image of a range in which the pedestrian can be detected. At this time, the alarm output unit 290 can greatly adjust the intensity of the alarm sound when the pedestrian detection unit 270 detects the pedestrian. Here, the 'driver's specific state' means that the driver's state is in a peculiar state that does not correspond to the driver's state of normal driving. For example, when the driver yaws while driving, or when the time during which the eyes are closed is longer than a predetermined time, the driver is judged to be tight and the driver's state can be judged to be a specific state. When it is determined that the state of the driver is in a singular state, the detection region correcting unit 260 sets the entire forward image of the vehicle as an image of a range for detecting the pedestrian. Therefore, the pedestrian detecting unit 270 detects the pedestrian with respect to the entire forward image of the vehicle.

The risk level calculating unit 280 calculates the risk level for the detected pedestrian using the distance between the pedestrian detected by the vehicle and the pedestrian detecting unit 270 and the traveling direction of the detected pedestrian. At this time, the alarm output unit 290 may adjust the intensity of the alarm sound according to the danger level calculated by the danger level calculating unit 280 and output the alarm sound intensity.

The pedestrian detection unit 270 can detect the distance between the detected pedestrian and the vehicle together. For example, the pedestrian detection unit 270 can detect the distance between the detected pedestrian and the vehicle using a radar. The radar detects the reflected wave reflected from the detected pedestrian after outputting the microwave within a predetermined angle to the front side of the vehicle. Here, the reflected wave includes information necessary for calculating an angle with respect to the detected pedestrian, a relative distance, a relative speed, and the like. For example, the pedestrian detecting unit 270 can detect the distance between the detected pedestrian and the vehicle by using the ultrasonic sensor.

According to one embodiment, the risk level calculator 280 can calculate the risk level lower when the detected direction of the pedestrian is moving away from the vehicle.

According to another embodiment, the risk level calculator 280 can calculate the risk level higher when the detected direction of the pedestrian is approaching to the vehicle.

3 is a view illustrating a viewing angle of a driver according to an exemplary embodiment of the present invention.

Referring to FIG. 3, the viewing angle of the driver depends on the driver's driving speed. For example, if the driver's vehicle running speed is 63 km / h, the driver's viewing angle is 38, the driver's vehicle running speed is 76 km / h, the driver's viewing angle is 27, h, the driver's viewing angle is 19 °. The viewing angle of the driver is not limited to this, but can be set to other angles. For example, in the case of an inexperienced novice driver, the driver's viewing angle can be set to be narrower. When the driver's viewing angle is narrowed, the number of images in the range for detecting the pedestrian increases, so that the pedestrian can be detected over a wider range.

FIG. 4 is a view showing an area set as an image of a range for detecting a pedestrian according to an embodiment of the present invention.

Referring to FIG. 4, the driver shows a situation in which the driver watches the forward direction, which is the traveling direction of the vehicle. According to an embodiment, when the driver watches the back light and a part of the front view image of the vehicle overlaps with the view image of the driver's gaze, the pedestrian may not be detected for the image of the overlap area. At this time, in the forward image of the vehicle, the image excluding the view image according to the driver's gaze is set as the image of the range in which the pedestrian can be detected. The pedestrian detection units 170 and 270 detect a pedestrian with respect to an image in a range in which the set pedestrian can be detected.

5 is a view showing an area set as an image of a range for detecting a pedestrian according to another embodiment of the present invention.

Referring to FIG. 5, the driver indicates a situation in which the driver is not looking at the forward direction of the vehicle, but is looking at the other direction. For example, when a driver views navigation or a side mirror while driving a vehicle, the driver is not watching the forward direction of the vehicle. At this time, if the visual field image according to the driver's gaze and a part of the front image of the vehicle do not overlap, the entire forward image of the vehicle is set as an image of a range in which the pedestrian can be detected. The pedestrian detecting units 170 and 270 detect a pedestrian with respect to the entire forward image of the vehicle.

6 is a view showing an area set as an image in a range for detecting a pedestrian according to another embodiment of the present invention.

Referring to FIG. 6, the detection area correction unit 260 of the pedestrian detection alarm device 200 described with reference to FIG. 2 detects a pedestrian detected by the detection area setting unit 250, Is set as an image of a range in which a pedestrian can be detected. Some of the upper and lower regions in the forward image of the vehicle are excluded from the image of the range where the pedestrian can be detected because the probability of detecting a pedestrian that can be associated with a vehicle accident is low or not present.

7 is a flowchart illustrating a pedestrian detection alarm method according to an embodiment of the present invention.

The procedures described below including embodiments of the present invention can be implemented in various forms. The pedestrian detection alarm method shown in Fig. 7 may be a pedestrian detection alarm system using the pedestrian detection alarm system 100 of Fig. 1 or the pedestrian detection alarm system 200 of Fig. 2 or an electronic apparatus having the pedestrian detection alarm system 200 of Fig. Therefore, in order to avoid unnecessary repetition, the pedestrian detection alarm method of FIG. 7 will be briefly described, and the matters described with reference to FIGS. 1 to 6 can be applied equally.

Referring to FIG. 7, first, a face image of a driver is obtained, a forward image of the vehicle is acquired, and a driver's gaze is detected from the acquired face image of the driver (S701). At this time, it is possible to determine the eye area of the driver in the face image of the driver to extract the candidate's gaze, and to select the driver's final gaze from the candidate gaze of the extracted driver to detect the driver's gaze.

Next, in step S702, the front image of the vehicle corresponding to the predetermined viewing angle is determined as a view image according to the driver's gaze, centering on the detected driver's gaze in the front image of the obtained vehicle.

Next, in step S703, an image of the acquired image of the vehicle excluding the view image according to the determined driver's gaze is set as an image of a range for detecting the pedestrian. According to an embodiment, an image excluding an image of a predetermined upper and lower regions from an image of a range in which a set pedestrian can be detected can be modified to an image of a range for detecting a pedestrian. According to another embodiment, when it is determined that the driver is in a specific state in the face image of the driver, the forward image of the vehicle can be set as an image of a range in which the pedestrian can be detected.

Next, the pedestrian is detected from the image of the range in which the set pedestrian can be detected (S704). At this time, the danger level for the detected pedestrian can be calculated using the distance between the vehicle and the detected pedestrian and the traveling direction of the detected pedestrian. According to one embodiment, when the detected direction of the pedestrian is moving away from the vehicle, the risk level can be calculated to be low. According to another embodiment, the risk level can be calculated to be high when the detected direction of the pedestrian approaches the vehicle.

Next, it is determined whether or not a pedestrian is detected (S705). If the pedestrian is not detected as a result of the determination, the face image of the driver and the forward image of the vehicle are acquired. On the other hand, when a pedestrian is detected, an alarm is output (S706). According to an embodiment, when a pedestrian is detected in an image of a range in which a pedestrian can be detected when the driver is determined to be in a singular state in the face image of the driver, the intensity of the alarm sound can be greatly adjusted and output. According to another embodiment, the alarm can be outputted in the direction in which the pedestrian is detected. According to another embodiment, the pedestrian can adjust the intensity of the alarm sound according to the detected distance and output it.

Further, such a pedestrian detection warning method may be stored in a computer-readable recording medium on which a program for execution by a computer is recorded. At this time, the computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on the medium may be those specially designed and constructed for the proposed invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium such as an optical or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, or the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa. In addition, the computer-readable recording medium may be distributed to network-connected computer devices so that computer-readable codes can be stored and executed in a distributed manner.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents thereof should be construed as falling within the scope of the present invention.

100, 200: Pedestrian detection alarm device
110, 210: Face image acquisition unit of the driver
120, 220: a front image acquisition unit
130, and 230:
140, and 240:
150, 250: detection area setting unit
260:
170, 270: Pedestrian detection unit
280: Risk level calculating unit
190, 290: Alarm output section

Claims (20)

A driver's face image acquiring unit for acquiring a face image of a driver;
A front image acquiring unit for acquiring a forward image of the vehicle;
A line of sight detection unit for detecting the line of sight of the driver on the face image of the driver acquired by the face image acquisition unit of the driver;
The front image of the vehicle corresponding to the predetermined viewing angle is determined as the visual field image according to the driver's line of sight based on the driver's gaze detected by the sight line detecting unit in the forward image of the vehicle obtained by the front image acquiring unit of the vehicle A field-of-view image determining unit;
A detection area setting unit that sets an image in a range in which a pedestrian can be detected, excluding a view image according to a driver's gaze determined by the view image determining unit in a forward image of the vehicle;
A pedestrian detecting unit that detects a pedestrian from a video in a range in which a pedestrian set by the detection area setting unit is detected; And
An alarm output unit for outputting an alarm when a pedestrian is detected by the pedestrian detection unit;
And a pedestrian detection alarm device.
The method according to claim 1,
Wherein the visual line detection unit determines a driver's eye area on the face image of the driver to extract a candidate's gaze and selects a driver's final gaze from the candidate gaze of the extracted driver.
The method according to claim 1,
Wherein the pedestrian detection alarm device further comprises a detection area modification unit that sets an image of the range excluding the image of the predetermined upper and lower areas in the image of the range where the pedestrian detected by the detection area setting unit is excluded, Detection alarm.
The method of claim 3,
Wherein the detection region correcting unit sets the forward image of the vehicle as an image of a range in which the pedestrian is to be detected when it is determined that the driver is in a singular state in the face image of the driver,
And the alarm output unit adjusts the intensity of the alarm sound to a large extent when the pedestrian detection unit detects the pedestrian.
The method according to claim 1,
Wherein the pedestrian detection alarm device further comprises a risk level calculation unit for calculating a risk level for the detected pedestrian using the distance between the vehicle and the pedestrian detected by the pedestrian detection unit and the traveling direction of the detected pedestrian, Alarm device.
6. The method of claim 5,
Wherein the danger level calculating unit calculates the risk level to be low when the detected direction of the pedestrian moves away from the vehicle.
6. The method of claim 5,
Wherein the danger level calculator calculates a danger level higher when the detected direction of the pedestrian approaches the vehicle.
6. The method of claim 5,
And the alarm output unit adjusts the intensity of the alarm sound according to the danger level calculated by the danger level calculation unit and outputs the alarm sound.
The method according to claim 1,
And the alarm output unit outputs an alarm sound in a direction in which the pedestrian is detected.
The method according to claim 1,
Wherein the alarm output unit adjusts the intensity of the alarm sound according to the distance detected by the pedestrian and outputs the adjusted alarm sound.
Acquiring a face image of a driver;
Obtaining a forward image of the vehicle;
Detecting a line of sight of the driver in the obtained face image of the driver;
Determining a front image of a vehicle corresponding to a predetermined viewing angle as a visual field image according to a driver's line of sight based on the detected driver's line of sight in the forward image of the obtained vehicle;
Setting an image of the acquired image of the vehicle excluding a view image according to the determined driver's gaze as an image of a range for detecting a pedestrian;
Detecting a pedestrian from an image of a range in which the set pedestrian can be detected; And
Outputting an alarm when a pedestrian is detected;
And the pedestrian detection alarm method.
12. The method of claim 11,
Wherein the eyesight detecting step determines a driver's eye area in the face image of the driver to extract a candidate's gaze and selects a driver's final gaze from the candidate gaze of the extracted driver.
12. The method of claim 11,
Further comprising the step of modifying and setting an image of a range excluding an image of a predetermined upper and lower range from an image of a range in which the pedestrian detected in the pedestrian detection range image setting step is excluded, to an image of a range for detecting the pedestrian.
14. The method of claim 13,
In the detection area correction setting step, if it is determined that the driver is in a singular state in the face image of the driver, the forward image of the vehicle is set as the image of the range for detecting the pedestrian,
Wherein the alarm output step adjusts the intensity of the alarm sound to a large extent when the pedestrian is detected in the pedestrian detection step.
12. The method of claim 11,
Further comprising calculating a danger level for the detected pedestrian using the distance between the vehicle and the pedestrian detected by the pedestrian detection unit and the traveling direction of the detected pedestrian.
16. The method of claim 15,
Wherein the risk level calculating step calculates a risk level lower when the detected direction of the pedestrian moves away from the vehicle.
16. The method of claim 15,
Wherein the danger level calculating step calculates a danger level higher when the detected direction of the pedestrian approaches the vehicle.
16. The method of claim 15,
Wherein the alarm output step adjusts the alarm sound intensity according to the danger level calculated in the danger level calculation step and outputs the alarm sound.
12. The method of claim 11,
Wherein the alarm output step outputs an alarm sound in a direction in which the pedestrian is detected.
12. The method of claim 11,
Wherein the alarm output step adjusts the intensity of the alarm sound according to the distance detected by the pedestrian and outputs the adjusted alarm sound.

Images