KR101748497B1 - Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same - Google Patents

Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same Download PDF

Info

Publication number
KR101748497B1
KR101748497B1 KR1020150158085A KR20150158085A KR101748497B1 KR 101748497 B1 KR101748497 B1 KR 101748497B1 KR 1020150158085 A KR1020150158085 A KR 1020150158085A KR 20150158085 A KR20150158085 A KR 20150158085A KR 101748497 B1 KR101748497 B1 KR 101748497B1
Authority
KR
South Korea
Prior art keywords
vehicle
sound
controller
right lane
area
Prior art date
Application number
KR1020150158085A
Other languages
Korean (ko)
Other versions
KR20170055203A (en
Inventor
이강
박지훈
Original Assignee
한동대학교 산학협력단
재단법인 다차원 스마트 아이티 융합시스템 연구단
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 한동대학교 산학협력단, 재단법인 다차원 스마트 아이티 융합시스템 연구단 filed Critical 한동대학교 산학협력단
Priority to KR1020150158085A priority Critical patent/KR101748497B1/en
Publication of KR20170055203A publication Critical patent/KR20170055203A/en
Application granted granted Critical
Publication of KR101748497B1 publication Critical patent/KR101748497B1/en

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/10—Path keeping
    • B60W30/12—Lane keeping
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/013—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over
    • B60R21/0134—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over responsive to imminent contact with an obstacle, e.g. using radar systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R25/00—Fittings or systems for preventing or indicating unauthorised use or theft of vehicles
    • B60R25/10—Fittings or systems for preventing or indicating unauthorised use or theft of vehicles actuating a signalling device
    • B60R25/1004—Alarm systems characterised by the type of sensor, e.g. current sensing means
    • B60R25/1009—Sonic sensors; Signal treatment therefor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units, or advanced driver assistance systems for ensuring comfort, stability and safety or drive control systems for propelling or retarding the vehicle
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/42—Image sensing, e.g. optical camera
    • B60W2550/10—

Abstract

A vehicle system for sensing a rear left and right lane areas of a vehicle includes a sound detection sensor attached to the outside rear left and right sides of the vehicle; A memory for storing sound information collected from the sound detection sensor; And a controller for detecting whether or not an object is positioned in or entering the rear left and right lane areas of the vehicle while the vehicle is traveling and notifying the driver of the vehicle of the detection result as an alarm, Based on the sound information collected from the sound detection sensor, detects whether the object is located or enters the rear left and right lane areas of the vehicle.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of detecting a rear lane area of a vehicle using sound information,

The present invention relates to a method of detecting a rear left and right lane area of a vehicle in a vehicle system, and more particularly, to a method of detecting a rear left and right lane areas of a vehicle, And detecting whether or not an object is located in or entering the lane area.

In a typical vehicle, a room mirror and a side mirror are installed to secure a driver's view to the side and rear. However, when the driver of the vehicle uses only the room mirror and the side mirror, there is a blind spot in the driver's view of the side and rear of the vehicle. Here, the blind spot of the vehicle means an area located at an angle such that the object is not seen by the driver of the vehicle, and can be generated by the structural features of the vehicle, the room mirror and the side mirror.

The absence of the driver's view of the rear left and right lane areas of the vehicle among the blind spots of the vehicle is an important cause of the accident of the vehicle. In order to detect objects located in the rear left and right lane areas of the vehicle, There is a problem that the driver must turn his or her head directly to change the field of view.

Particularly, there is a limit that the driver of the vehicle can not recognize the object located in the rear left and right lane area of the vehicle when the fog, the snow, the rain, and the visibility are covered or the nighttime or the surroundings are dark.

Accordingly, the present invention proposes a technique of detecting the rear left and right lane areas of a vehicle using sound information obtained in a vehicle system installed in a vehicle.

Embodiments of the present invention provide a method, apparatus, and system for sensing whether an object is located or entering a rear left and right lane area of a vehicle based on sound information collected from a sound detection sensor included in a vehicle system installed in the vehicle to provide.

In this case, the embodiment of the present invention may be configured to be set to any one of a mixed mode using a rear camera and a sound detection sensor included in the vehicle system or a sound mode using a sound detection sensor to determine whether the object is located in the rear left and right lane areas of the vehicle A method, apparatus and system for detecting whether or not to enter.

Further, an embodiment of the present invention detects whether or not an object is located or enters an area of interest corresponding to a rear left and right lane area of the vehicle in an image obtained from a rear camera, so that an object in the rear left and right lane areas of the vehicle A device and a system for detecting whether the device is located or is entering.

In the embodiment of the present invention, a directional noise reduction technique is applied to sound information collected from a sound detection sensor to extract sound information related to an object excluding sound information and noise information of the vehicle from sound information collected from the sound detection sensor , A method and apparatus for sensing whether an object is located or entering a rear left and right lane area of a vehicle by calculating the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object .

According to an embodiment of the present invention, a vehicle system for sensing a rear left and right lane areas of a vehicle includes a sound detection sensor attached to the outside rear left and right sides of the vehicle; A memory for storing sound information collected from the sound detection sensor; And a controller for detecting whether or not an object is positioned in or entering the rear left and right lane areas of the vehicle while the vehicle is traveling and notifying the driver of the vehicle of the detection result as an alarm, Based on the sound information collected from the sound detection sensor, detects whether the object is located or enters the rear left and right lane areas of the vehicle.

The controller may apply a directional noise reduction technique to the sound information collected from the sound sensor to extract sound information related to the object, excluding sound information and noise information of the vehicle, from sound information collected from the sound sensor have.

The controller may determine whether the object is located or enters the rear left and right lane areas of the vehicle based on the energy of sound information associated with the object.

The controller may calculate the velocity of the object and the distance between the vehicle and the object based on sound information associated with the object.

The controller may calculate a velocity of the object and a distance between the vehicle and the object based on time delay information in which sound information related to the object is collected by the sound detection sensor.

The controller measures at least one of time delay information in which sound information related to the object is collected by the sound detection sensor using at least one of Time Domain Cross Correlation (TDCC) or Generalized Cross Correlation Phase Transform (GCC-PHAT) .

The directional noise cancellation technique may include at least one of Beam-Forming (BF) or Blind Source Separate (BSS) techniques.

Wherein the memory stores an image obtained from a rear camera further included in the vehicle system and the controller sets an area of interest corresponding to a rear left and right lane area of the vehicle in an image obtained from the rear camera, Based on the result of searching for whether or not the object is located, whether the object is located or entering the rear left and right lane areas of the vehicle.

The controller may set the ROI based on detected left and right lane information in an image obtained from the rear camera.

The controller may determine at least one of a position, a shape, a size, and a number of the ROI using the shadow information of the object.

The controller extracts features from the region of interest by applying at least one of a HOG (Histogram Of Gradients) technique and a Haar-like feature technique, and performs a SVM (Support Vector Machine) technique, an Adaboost technique, or an Artificial Neural Network ANN) technique to identify whether the object is located in the ROI by identifying the object.

The controller recognizes a change in appearance information or identification information of the object or calculates a motion vector of the object to determine whether the object enters the ROI or whether the object travels at the same speed as the vehicle in the ROI Whether you can navigate.

When the object is located in the region of interest, the controller recognizes a change in the appearance information or the identification information of the object, or calculates a motion vector of the object to search whether the object is out of the region of interest have.

At least one of the position, shape, size, and number of the ROIs includes left and right lane information detected in the image obtained from the rear camera, position information of the vanishing point in the image obtained from the rear camera, The appearance information of the object located in the image, the operation speed of the controller, or the power consumption of the controller.

The controller may notify the driver of the vehicle of the detection result via an alarm through at least one of a display device or an audio device provided in the vehicle.

According to an embodiment of the present invention, a vehicle system for sensing a rear left and right lane areas of a vehicle includes a rear camera; A sound detection sensor attached to outside rear left and right sides of the vehicle; A memory for storing images obtained from the rear camera and sound information collected from the sound sensor; And a mixed mode using the rear camera and the sound detection sensor or a sound mode using the sound detection sensor while the vehicle is traveling so that an object is positioned or enters the rear left and right lane areas of the vehicle And notifies the driver of the vehicle of the detection result as an alarm.

The controller may be set to either the mixed mode or the sound mode based on sound information collected from the sound detection sensor.

Wherein the controller set in the mixed mode sets a ROI corresponding to the rear right and left lane areas of the vehicle in the image obtained from the rear camera to search for whether the object is located or enters the ROI, Based on the sound information collected from the sound detection sensor, whether or not the object is located in the rear left and right lane areas of the vehicle.

The controller set in the sound mode may sense whether the object is located or enters the rear left and right lane areas of the vehicle based on sound information collected from the sound sensor.

Embodiments of the present invention provide a method, apparatus, and system for sensing whether an object is located or entering a rear left and right lane area of a vehicle based on sound information collected from a sound detection sensor included in a vehicle system installed in the vehicle .

In this case, the embodiment of the present invention may be configured to be set to any one of a mixed mode using a rear camera and a sound detection sensor included in the vehicle system or a sound mode using a sound detection sensor to determine whether the object is located in the rear left and right lane areas of the vehicle A device and a system for detecting whether or not the user is entering the system.

Further, an embodiment of the present invention detects whether or not an object is located or enters an area of interest corresponding to a rear left and right lane area of the vehicle in an image obtained from a rear camera, so that an object in the rear left and right lane areas of the vehicle A device and a system for detecting whether the device is located or is entering.

In the embodiment of the present invention, a directional noise reduction technique is applied to sound information collected from a sound detection sensor to extract sound information related to an object excluding sound information and noise information of the vehicle from sound information collected from the sound detection sensor A device and a system for detecting whether an object is located or entering a rear left and right lane area of a vehicle by calculating the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object .

1 is a view for explaining a system for a vehicle that senses a rear left and right lane area of a vehicle according to an embodiment of the present invention.
2 is a diagram illustrating an image obtained from a rear camera to describe a method of sensing a rear left and right lane area of a vehicle using an area of interest according to an embodiment of the present invention.
FIG. 3 is a flowchart illustrating a method of detecting a rear left and right lane area of a vehicle using a sound detection sensor according to an embodiment of the present invention.
4 is a view illustrating a rear left and right lane areas of a vehicle according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a method of detecting a rear left and right lane area of a vehicle according to an embodiment of the present invention. Referring to FIG.
6 is a flowchart illustrating a method of detecting a rear left and right lane of a vehicle according to an embodiment of the present invention.
FIG. 7 is a flowchart specifically illustrating a step of detecting whether an object is positioned or entered in a rear left and right lane area of a vehicle when the controller according to an exemplary embodiment of the present invention is set to a sound mode.
FIG. 8 is a flowchart specifically illustrating a step of detecting whether an object is positioned or entered in a rear left and right lane area of a vehicle when the controller according to an exemplary embodiment of the present invention is set to a mixed mode.
9 is a block diagram illustrating a system for a vehicle according to an embodiment of the present invention.
10 is a view for explaining a method of detecting whether an object is positioned or entering a rear left and right lane area of a vehicle based on sound information related to an object according to an embodiment of the present invention.

Hereinafter, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to or limited by the embodiments. In addition, the same reference numerals shown in the drawings denote the same members.

1 is a view for explaining a system for a vehicle that senses a rear left and right lane area of a vehicle according to an embodiment of the present invention.

Referring to FIG. 1, a vehicle system according to an embodiment of the present invention may be installed inside or outside a vehicle 110. Here, the in-vehicle system may include a front camera 120, a rear camera 130, a sound detection sensor 140, a memory, and a controller. Although not shown in the drawing, the front camera 120 and the rear camera 130 and the rear camera 130 are not limited to the illustrated example, and the memory and the controller may be configured to be combined with either the front camera 120 or the rear camera 130 Or may be provided as a separate device. In addition, the front camera 120 may optionally be included in the on-vehicle system. At this time, the rear camera 130 may be a monocular camera.

The front camera 120 acquires an image of the visual field 121 according to a predetermined angle toward the front of the vehicle 110. [ At this time, the front camera 120 can selectively acquire a high-quality image or a low-quality image according to the state of the controller.

Further, the rear camera 130 acquires an image of the field of view 131 along a predetermined angle toward the rear of the vehicle 110. [ At this time, the rear camera 130 can also selectively acquire a high-quality image or a low-quality image according to the state of the controller.

For example, when the controller 110 is set in the running state as the vehicle 110 travels, the front camera 120 and the rear camera 130 can acquire a high-quality image of a predetermined image quality or more, It is possible to obtain a low-quality image that is less than a preset image quality when the controller is set to the parking state. Therefore, the front camera 120 and the rear camera 130 can minimize the power consumption in the process of acquiring the image by selectively controlling the image quality of the acquired image. As such, images obtained from each of the front camera 120 and the rear camera 130 can be stored in a memory.

At this time, the rear camera 130 may be selectively operated depending on the operation of the sound detection sensor 140, instead of always operating. A detailed description thereof will be given below.

The sound detection sensor 140 is attached to the outside rear left and right sides of the vehicle 110. For example, the sound detection sensor 140 may be attached to the rear of the vehicle 110 at a distance of 0.5 m in a straight line. However, the present invention is not limited thereto, and the attachment position and the number of the sound detection sensor 140 may be adaptively changed.

Therefore, the sound detection sensor 140 attached to the left side can collect information about the sound generated in the rear left area of the vehicle 110, and the sound detection sensor 140 attached to the right side can collect information about the sound output from the vehicle 110 Information about the sound generated in the rear right region can be collected. As such, the sound information collected from the sound detection sensor 140 can be stored in the memory.

The controller is set to the running state as the vehicle 110 travels or set to the parked state as the vehicle 110 is parked so that a predetermined event is generated in the image obtained from each of the front camera 120 and the rear camera 130 . At this time. The controller can detect a preset event in the image obtained from each of the front camera 120 and the rear camera 130 with respect to each frame or with a predetermined time interval.

Here, a general event detection technique may be used for an operation of detecting a preset event in the images obtained from the front camera 120 and the rear camera 130, respectively. The detailed description thereof will be omitted because it goes beyond the technical idea of the present invention.

The driver's view 151 using the room mirror 150 installed on the vehicle 110 and the driver's field of view 161 using the side mirrors 160 are used only for the blind spot 170 for the rear left and right lane areas of the vehicle. . Accordingly, the vehicle system according to the embodiment of the present invention can be used for the vehicle 110 (for example, when the controller 110 is running) while the vehicle 110 is running By detecting whether or not the object is located in the rear left and right lane areas, which are the dead zones 170 of the vehicle 110, by detecting the detection result.

In particular, the in-vehicle system can set the mode of the controller to either a mixed mode using both the rear camera 130 and the sound detection sensor 140 or a sound mode using only the sound detection sensor 140, It is possible to detect whether or not an object is located or entering the rear left and right lane areas,

At this time, the controller can be set to either the mixed mode or the sound mode based on the sound information collected from the sound detection sensor 140 (for example, by the external noise of the vehicle 110, If the quality of the sound information collected from the controller is less than or equal to a preset reference value, the controller is set to the mixed mode).

For example, when the controller is set to the mixed mode, the in-vehicle system sets an area of interest corresponding to the rear left and right lane areas of the vehicle 110 in the image obtained from the rear camera 130 to determine whether the object is located in the area of interest Or whether or not the object is located in the rear left and right lane areas of the vehicle 110 based on sound information collected from the sound detection sensor 140,

Alternatively, when the controller is set to the sound mode, the on-vehicle system determines whether the object is located or enters the rear left and right lane areas of the vehicle 110, based on the sound information collected from the sound detection sensor 140 Can be detected.

Accordingly, the sound detection sensor 140 can be used both when the controller is set to the mixed mode and when the controller is set to the sound mode, and the rear camera 130 is used only when the controller is set to the mixed mode .

A detailed description of the process of detecting whether or not an object is positioned in the rear left and right lane areas of the vehicle 110 using the rear camera 130 will be described with reference to FIG. 2, 140 will be described in detail with reference to FIG. 3, for explaining a process of detecting whether or not an object is located in the rear left and right lane areas of the vehicle 110. FIG.

2 is a diagram illustrating an image obtained from a rear camera to describe a method of sensing a rear left and right lane area of a vehicle using an area of interest according to an embodiment of the present invention.

Referring to FIG. 2, the rear camera included in the vehicle system according to the embodiment of the present invention includes an image of a view according to a predetermined angle toward the rear of the vehicle, under the control of the controller set in the running state as the vehicle travels (210, 220).

Accordingly, the controller senses whether the objects 230 and 240 are located or enter the rear left and right lane areas 211 and 221 of the vehicle in the images 210 and 220 obtained from the rear camera while the vehicle is traveling .

At this time, the controller determines whether or not the object 230, 240 is located in the rear left and right lane areas 211, 221 of the vehicle, It is possible to activate the search function for the area of interest 250 corresponding to the rear left and right lane areas 211,

Here, the area of interest 250 is a fixed area with respect to at least a part of the rear left and right lane areas 211 and 221 of the vehicle, and is based on the left and right lane information detected in the images 210 and 220 obtained from the rear camera .

For example, the left and right side lane information detected in the images 210 and 220 obtained from the rear camera and the position information of the vanishing point in the images 210 and 220 obtained from the rear camera, The dead zones of the vehicle's side mirrors and the room mirrors are calculated and an area including the calculated dead zones can be set to the area of interest 250. [ Also, at least one of the position, shape, size, or number of the region of interest 250 may be determined using the shadow information of the objects 230 and 240.

More specifically, for example, the controller may not set the region of interest 250 at all if the shadow of the object in the image obtained from the rear camera is not searched, and may set the region of interest (e.g., 250 to a minimum size, or dynamically determine the location of the region of interest 250 to accurately search for an object.

The controller may activate a search function for the region of interest 250 to search for whether the objects 230 and 240 are located or enter the region of interest 250. [ Accordingly, the controller is able to control the objects 230 and 240 only in the area of interest 250, which is a fixed area for at least a part of the vehicle's rear left and right lane areas 211 and 221 in the images 210 and 220 obtained from the rear camera It is possible to remarkably reduce the search computation amount of the objects 230 and 240 as compared with the technique of tracking and searching the objects 230 and 240 with respect to the entire rear left and right lane areas 211 and 221. [

Specifically, the controller extracts features from the region of interest 250 by applying at least one of the Histogram Of Gradients (HOG) technique or Haar-like feature technique based on the search function for the activated ROI 250 (230, 240) in the region of interest (250) by identifying at least one of the objects (230, 240) by applying at least one of a Support Vector Machine (SVM) technique, an Adaboost technique or an Artificial Neural Network Can be searched for. Hereinafter, the detailed description of the HOG technique, the SVM technique, the Haar-like feature technique, the Adaboost technique, and the artificial neural network technique will be omitted because they are outside the technical concept of the present invention.

The controller also recognizes changes in the appearance information or identification information of the objects 230 and 240 or calculates the motion vectors of the objects 230 and 240 based on the search function for the activated region of interest 250, It is possible to search whether the objects 230 and 240 enter the region 250 or whether the objects 230 and 240 travel at the same speed as the vehicle in the region of interest 250. [ At this time, the process of calculating the motion vectors of the objects 230 and 240 can be performed by a conventional motion vector calculation technique. The detailed description thereof will be omitted because it goes beyond the technical idea of the present invention. In addition, instead of calculating the motion vectors of the objects 230 and 240, the controller also estimates the relative velocity of the objects 230 and 240 to determine whether the objects 230 and 240 enter the region of interest 250 .

For example, as in (a), objects 230 and 240 are not located in the region of interest 250 in the image 210 for the previous one of the images 210 and 220 obtained from the rear camera, If the object 230, 240 is located in the region of interest 250 in the image 220 for the current frame, as in b), the controller searches for the object 230, 240 in the region of interest 250 .

More specifically, for example, the controller determines whether the appearance information or identification information for the first object 230 extracted from the image 210 for the previous frame is identical to the first object 230 extracted from the image 220 for the current frame, Based on the difference between the image 210 for the previous frame and the pixel block included in the first region of interest 251 in the image 220 for the current frame, The first object 230 enters the first area of interest 251 or the first object 230 in the first area of interest 251 travels at the same speed as the vehicle by calculating the motion vector of the object 230. [ Can be searched.

In another specific example, the controller activates the search function for the second region of interest 252 in the image 220 for the current frame, and then, based on the appearance information or identification information of the second object 240, And the SVM technique to find that the second object 240 is located in the second region of interest 252. At this time, when the HOG technique is applied, the controller sets the parameters used in the HOG technique to be optimized based on the appearance information preliminarily formed for each of the plurality of objects including the objects 230 and 240 230, and 240), the angular distribution characteristics of the edges to the objects 230 and 240, and the interval in which the appearance characteristics of the objects 230 and 240 are intensively distributed), and the calculation speed of the HOG technique Can be improved.

The appearance information of the objects 230 and 240 described above includes appearance information (front appearance information or side appearance information) related to the type and size of the objects 230 and 240 and objects 230 and 240, 240). ≪ / RTI > In addition, the identification information of the objects 230 and 240 may mean the license plate size of the vehicle when the objects 230 and 240 are vehicles. Hereinafter, the objects 230 and 240 will be described as the case of a vehicle. However, the objects 230 and 240 are not limited thereto, but may be all objects that are subject to collision when the vehicle is traveling, such as a person, a motorcycle, a bicycle or a sign.

The area of interest 250 is a specific area on the images 210 and 220 for searching the rear left and right lane areas 211 and 221 of the vehicle in the images 210 and 220 obtained from the rear camera, Lt; / RTI >

For example, at least one of the location, shape, size or number of areas of interest 250 may include left and right lane information detected in images 210 and 220 obtained from a rear camera, images 210, 220 based on the location information of the vanishing point or the appearance information of the objects 230, 240 located in the images 210, 220 obtained from the rear camera.

More specifically, the ROI 250 may include a shape of a rectangular shape horizontally left or right depending on the appearance information of the objects 230 and 240 located in the images 210 and 220 obtained from the rear camera, And the like. Here, the appearance information of the objects 230 and 240 may include information about the shadows generated by the objects 230 and 240.

Also, at least one of the position, shape, size or number of the region of interest 250 may be set based on at least one of the operation speed of the controller or the power consumption of the controller set in the travel mode.

For example, at least one of the location, shape, size, or number of the area of interest 250 may detect whether the objects 230 and 240 are located in the rear left and right lane areas 211 and 221 of the vehicle described above The HOG technique applied to extract the features from the images 210 and 220 obtained from the rear camera, the SVM technique applied to identify the Haar-like feature technique and the objects 230 and 240, the Adaboost technique or the artificial neural network And / or < / RTI >

More specifically, at least one of the location, shape, size, or number of regions of interest 250 may be set such that the rate at which the controller applies the HOG technique is minimized.

At this time, instead of sensing all of the rear left and right lane areas 211 and 221 of the vehicle, at least one of the position, shape, size or number of the area of interest 250 is adaptively set, Only the rear left and right lane areas 211 and 221 within a set distance can be detected. Thus, the controller may adjust the power consumed to activate the search function for the region of interest 250, by adaptively setting at least one of the location, shape, size or number of regions of interest 250.

As such, the controller in accordance with an embodiment of the present invention includes objects 230 and 240 in the region of interest 250 in the images 210 and 220 obtained from the rear camera, with respect to each frame or with a predetermined time interval The vehicle driver can be informed of an alarm by searching for whether the vehicle is located or entering the vehicle. For example, the controller can notify the driver of the vehicle of the search result through an alarm through at least one of a display device or an audio device provided in the vehicle.

In addition, if the object 230, 240 is not located in the region of interest 250 or if it is found that the object 230 or 240 is not located in the region of interest 250, then the controller determines that the objects 230, 240 are not located in the region of interest 250, I can notify the driver. At this time, if the objects 230 and 240 are located in the region of interest 250, the controller may determine whether the objects 230 and 240 have the appearance information or the identification information of the objects 230 and 240 based on the search function for the activated region of interest 250 Or to calculate whether the objects 230, 240 are out of the region of interest 250 by calculating the motion vectors of the objects 230,

This notification operation may not be selectively performed according to the setting previously inputted from the driver of the vehicle (for example, according to the setting, the objects 230 and 240 are placed in the rear left and right lane areas 211 and 221 of the vehicle The operation of informing the driver of the vehicle that the vehicle is not positioned may not be performed).

The controller also detects whether the objects 230 and 240 are located or entered by dividing the area of interest 250 corresponding to the rear left and right lane areas 211 and 221 of the vehicle into a plurality of areas, As an alarm. A detailed description thereof will be described with reference to FIG.

FIG. 3 is a flowchart illustrating a method of detecting a rear left and right lane area of a vehicle using a sound detection sensor according to an embodiment of the present invention.

Referring to FIG. 3, the sound detection sensor included in the vehicle system according to an embodiment of the present invention collects sound information outside the vehicle toward the rear of the vehicle under the control of the controller set in the running state as the vehicle travels (310). For example, the sound detection sensor can collect sound information outside the vehicle toward the rear of the vehicle with a predetermined time interval. At this time, the sound detection sensor can be attached to the rear of the vehicle in a straight line, and the position and number of the sound detection sensor can be adaptively changed.

Accordingly, the controller senses whether or not the object is located or enters the rear left and right lane areas of the vehicle based on sound information (hereinafter, referred to as sound information outside the vehicle) collected from the sound detection sensor while the vehicle is traveling do.

Specifically, the controller applies directional noise reduction technology to the sound information collected from the sound detection sensor, and detects the sound information collected from the sound detection sensor and the noise information of the vehicle (noise related to the noise generated in the surrounding environment where the vehicle travels (E.g., the engine sound of the object) related to the object other than the information (e.g., information). For example, the controller may use directional noise reduction techniques such as BF (Beam-Forming) or BSS (Blind Source Separate) techniques to detect sound information collected from a sound detection sensor The sound information can be extracted.

At this time, the process of extracting the sound information related to the object is described as being performed by the controller, but the present invention is not limited thereto and may be performed by the sound sensor itself.

The controller can then determine (330) whether there is an object that is approaching or located within a predetermined distance of the vehicle (including the rear left and right lane areas of the vehicle) based on sound information associated with the object. For example, the controller can determine whether the object is located or is approaching the rear left and right lane areas of the vehicle based on the energy of the sound information associated with the object.

Further, when the object is a vehicle and the sound information associated with the object is information on the engine sound of the object, the controller may guess the vehicle type of the object based on sound information associated with the object. For example, the controller extracts Mel-Frequency Cepstral Coefficient (MFCC) or Linear Predictive Coefficient (LPC) parameters from the sound information associated with the object, and uses a GMM (Gaussian Mixture Model) technique based on a pre- The vehicle type of the object can be guessed.

As a result of the determination, if there is an object approaching or located within a predetermined distance of the vehicle, the controller can calculate the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object (340). For example, the controller can calculate the speed of the object and the distance between the vehicle and the object based on the time delay information at which sound information related to the object is collected by the sound detection sensor. More specifically, for example, the controller uses Time Domain Cross Correlation (TDCC) or Generalized Cross Correlation Phase Transform (GCC-PHAT) techniques to obtain time delay information about the sound information associated with the object Can be measured. A detailed description thereof will be described with reference to FIG. At this time, the controller can calculate the velocity of the object and the distance between the vehicle and the object with a preset time interval.

Based on the calculation result of step 340, the controller can then detect whether the object is located in or approaches the rear left and right lane areas of the vehicle (for example, whether the object approaches the vehicle at a rate that is threatening to the vehicle) (350). For example, when the time delay information is positive, when the object is located on the left rear side of the vehicle and the time delay information is negative, As shown in FIG. Also, the larger the absolute value of the time delay, the closer the object is to the vehicle.

Here, the controller divides the rear left and right lane areas of the vehicle into a plurality of areas, and can detect which of the plurality of areas is located or entered. A detailed description thereof will be described with reference to FIG.

Thereafter, the controller may notify the driver of the vehicle of the detection result (360). For example, the controller can notify the driver of the vehicle of an alarm result through at least one of a display device or an audio device provided in the vehicle.

Further, the controller can notify the driver of the vehicle that the object is not located in the rear left and right lane areas of the vehicle, or is out of the alarm when the object is not located in the rear left and right lane areas of the vehicle, or detects that the object is departing.

This notification operation may not be performed selectively according to the setting previously inputted from the driver of the vehicle (for example, according to the setting, the operation of informing the driver of the vehicle that the object is not located in the rear left and right lane areas of the vehicle is performed .

4 is a view illustrating a rear left and right lane areas of a vehicle according to an embodiment of the present invention.

Referring to FIG. 4, the controller according to an embodiment of the present invention divides the rear left and right lane areas of the vehicle into a plurality of areas, detects which of the plurality of areas is located or enters the area can do.

Here, the controller may determine, based on at least any one of left and right lane information detected in the image obtained from the rear camera or position information of the vanishing point in the image obtained from the rear camera, The right and left lanes can be divided into a first area 410, a second area 420 and a third area 430.

Accordingly, the controller is configured to detect a first area 410 that is not less than a predetermined distance from the vehicle, which is not recognized by the driver but is photographed by the rear camera, through the side mirror of the vehicle, Through a side mirror or a third area 430 that is not captured by the rear camera and is not recognized by the driver through the side mirror .

For example, in the process of detecting whether an object is located or entering an area of the first area 410, the second area 420, and the third area 430, And the location information of the vanishing point in the image obtained from the rear camera. More specifically, for example, the controller can determine whether the object is greater than or less than a predetermined distance in the region of interest based on the left and right lane information detected in the image obtained from the rear camera.

At this time, the controller activates the search function for the area of interest corresponding to the rear left and right lane areas of the vehicle in the image obtained from the rear camera, so that the object is displayed in the first area 410, the second area 420, It is possible to detect which area of the area 430 is located or entering.

Specifically, the controller extracts features from the region of interest by applying at least one of the HOG technique or the Haar-like feature technique based on the search function for the activated region of interest, and performs the SVM technique, the Adaboost technique or the artificial neural network technique It is possible to detect whether the object is located in the first area 410, the second area 420, or the third area 430 by identifying at least one of the objects.

For example, the controller preliminarily stores and stores the appearance information or the identification information for each of the plurality of objects in the database, and then applies at least one of the HOG technique and the Haar-like feature technique, By comparing the appearance information or the identification information of the identified object with the appearance information or the identification information for each of the plurality of objects stored in the database by applying at least one of the SVM technique, the Adaboost technique, and the artificial neural network technique, 1 region 410 or the second region 420. [0050] FIG.

In addition, the controller recognizes the change of the appearance information or the identification information of the object, or calculates the motion vector of the object to determine whether the object is located in any one of the first area 410, the second area 420 or the third area 430 Or whether the object is traveling at the same speed as the vehicle in any of the first area 410, the second area 420, or the third area 430.

Particularly, since the third area 430 is an area that is not recognized by the driver through the side mirror and is not photographed by the rear camera, the controller can recognize the object's appearance information or identification It is possible to recognize whether the object enters the third area 430 by recognizing a change in information or calculating a motion vector of the object.

For example, if the object is located in the second region 420 and the motion vector of the calculated object is directed to the third region 430 (the motion vector of the object in the region of interest for the previous frame is 3 region 430 - towards the second region 420, the object disappears in the second region 420 on the region of interest of the image for the current frame and enters the third region 430 , It may detect whether or not the object enters the third area 430.

At this time, the controller can notify the driver of the distinguished alarm according to the area in which the object is located or enters the first area 410, the second area 420, and the third area 430. For example, if an object is sensed to enter a third area 430, the controller may detect that the object is in a first area 410 or a second area 420, The driver can be notified of the detection result.

When the controller detects that the object enters the third area 430 and notifies the alarm to the driver of the vehicle after the predetermined alarm time elapses or when the object is detected to be out of the third area 430 , The alarm can be released.

For example, the controller senses that the object enters the third area 430 and notifies the driver of the alarm, and then, using the image obtained from the front camera or the image obtained from the rear camera, The alarm notified to the driver can be canceled. At this time, since the object has entered the third area 430 via the second area 420, the appearance of the object extracted when the object is located in the second area 420 in the image obtained from the rear camera By determining whether there is an object matching the appearance information or the identification information in the image obtained from the front camera or the image obtained from the rear camera based on the identification information of the information or the object, It is possible to detect the deviation.

More specifically, if the object has entered the third region 430 via the second region 420 and then moved to the front of the vehicle, the controller may return the object extracted when the object is located in the second region 420 It is determined that there is an object matched with the temporarily stored image information or the identification information stored in the image obtained from the front camera, and if the object is out of the third area 430 Can be detected.

In this way, the controller divides the rear left and right lane areas of the vehicle into the first area 410, the second area 420 and the third area 430 so that the object is divided into the first area 410, 420 or the third area 430 by detecting whether or not the vehicle is located in the rear left and right lane areas of the vehicle, .

In addition, the controller can selectively detect the object only when the object approaches the rear left and right lane areas of the vehicle at a higher speed than the vehicle. For example, as the vehicle travels at a higher speed than the object, the controller searches the object and notifies the driver of the search result by alarm when the object is located or enters the area of interest in the rear left and right lane areas of the vehicle It may not perform. A detailed description thereof will be described with reference to Fig.

At this time, the controller may detect by using a sound detection sensor whether an object is located or entering an area of a plurality of divided areas.

Specifically, the controller extracts sound information related to the object excluding the sound information and noise information of the vehicle from the sound information collected from the sound detection sensor while the vehicle is traveling, so that the speed of the object and the speed of the vehicle And the distance between the object and the object can be calculated.

Thus, the controller can use the distance between the vehicle and the object calculated based on the sound information associated with the object to detect which area of the plurality of areas of the vehicle's rear left and right lane areas is located or enters have.

For example, if the distance between the vehicle and the object calculated based on the sound information associated with the object corresponds to the first area 410, the controller may sense whether the object is located in the first area 410. [

FIG. 5 is a diagram illustrating a method of detecting a rear left and right lane area of a vehicle according to an embodiment of the present invention. Referring to FIG.

5, in one embodiment of the present invention, the controller may selectively provide the object 520 only when the object 520 approaches the rear left and right lane areas of the vehicle 510 at a higher speed than the vehicle 510 Can be detected.

Specifically, the controller recognizes the change of the appearance information or the identification information of the object extracted from the region of interest of the image obtained from the rear camera, or calculates the motion vector of the object so that the object 520 is rotated at a speed It is possible to selectively search the object 520 only when approaching the area of interest in the rear left and right lane areas of the vehicle 510 and notify the driver of the search result as an alarm.

For example, when the object 520 approaches the area of interest of the rear left and right lane areas of the vehicle 510 at a higher speed than the vehicle 510, as shown in (a) The appearance information of the extracted object 520 may be changed in the order of the front appearance information 521 to the side appearance information 522. [ On the other hand, when the object 520 is positioned in the region of interest of the rear left and right lane areas of the vehicle 510 as the vehicle 510 travels at a higher speed than the object 520 as shown in (b) The appearance information of the object 520 extracted from the region of interest of the image obtained from the camera may be changed from the side appearance information 522 to the front appearance information 521 in this order.

Accordingly, the controller can selectively select the object (s) only when the appearance information for the object 520 extracted from the region of interest of the image obtained from the rear camera is changed from the front appearance information 521 to the side appearance information 522 520 can be searched for objects 520 selectively only when the object 520 approaches the area of interest of the rear left and right lane areas of the vehicle 510 at a faster rate than the vehicle 510. [

Further, instead of selectively sensing the object 520, the controller may selectively notify the driver of the vehicle 510 of the detection result. In such a case, the controller may sense the object 520 and then selectively notify the driver of the detection result in an alarm based on the change in the appearance information for the sensed object 520. For example, the controller may selectively notify the driver of the detection result only when the appearance information for the sensed object 520 changes from the front appearance information 521 to the side appearance information 522. [

At this time, the controller may selectively detect the object 520 only when the object 520 approaches the rear left and right lane areas of the vehicle 510 at a higher speed than the vehicle 510 using a sound detection sensor .

Specifically, the controller extracts sound information related to the object 520 excluding the sound information and noise information of the vehicle from the sound information collected from the sound detection sensor while the vehicle 510 is traveling, The speed of the object 520 and the distance between the vehicle 510 and the object 520 can be calculated based on the information.

The controller can use the speed of the object 520 calculated based on the sound information associated with the object 520 such that the object 520 is moved at a faster rate than the vehicle 510 to the rear left and right lane areas & It is possible to selectively detect the object 520 only when the object 520 is accessed.

For example, the controller may selectively detect an object 520, such as by detecting an object 520, only if the speed of the object 520 calculated based on sound information associated with the object 520 is greater than the speed of the vehicle 510 May selectively sense the object 520 only when it approaches the area of interest of the rear left and right lane areas of the vehicle 510 at a faster rate than the vehicle 510. [

6 is a flowchart illustrating a method of detecting a rear left and right lane of a vehicle according to an embodiment of the present invention.

Referring to FIG. 6, a vehicle system according to an embodiment of the present invention collects sound information using a sound sensor (610). Here, the sound information collected by the sound detection sensor may be sound information outside the vehicle generated at the rear of the vehicle. Although not shown in the drawings, the sound information collected from the sound detection sensor may be stored in the memory. At this time, the sound detection sensor can be attached to the rear of the vehicle in a straight line, and the position and number of the sound detection sensor can be adaptively changed.

Subsequently, the vehicle system sets 620 the controller while driving the vehicle to either a mode using a mixed mode-rear camera and a sound detection sensor, or a mode using a sound mode-sound detection sensor. Here, the vehicle system may set the controller to either the mixed mode or the sound mode based on the sound information collected from the sound detection sensor. For example, when the quality of sound information collected from the sound detection sensor by the external noise of the vehicle is equal to or lower than a preset reference value, the controller can be set to the mixed mode.

The in-vehicle system then detects (630) whether the object is located or entered in the rear left and right lane areas of the vehicle using the controller set to either the mixed mode or the sound mode. A detailed description thereof will be described with reference to FIGS. 7 and 8. FIG.

Thereafter, the in-vehicle system notifies the driver of the vehicle of the detection result as an alarm (640). For example, the vehicle system may notify the driver of the vehicle of the detection result through at least one of a display device or an audio device provided in the vehicle. This notification operation may be adaptively performed according to the setting of the vehicle.

FIG. 7 is a flowchart specifically illustrating a step of detecting whether an object is positioned or entered in a rear left and right lane area of a vehicle when the controller according to an exemplary embodiment of the present invention is set to a sound mode.

Referring to FIG. 7, a vehicle system according to an embodiment of the present invention uses a controller to apply a directional noise reduction technique to sound information collected from a sound sensor, And sound information related to the object excluding the noise information (710). The process of extracting the sound information related to the object is performed using the controller. However, the present invention is not limited thereto and may be performed using a sound detection sensor. For example, the controller can extract directional noise reduction techniques such as BF or BSS techniques to extract sound information related to an object excluding sound information and noise information from the sound information collected from the sound detection sensor.

The in-vehicle system may then use the controller to determine whether there are objects within or near the vehicle's predetermined distance based on sound information associated with the object (720). For example, the controller can determine whether the object is located or is approaching the rear left and right lane areas of the vehicle based on the energy of the sound information associated with the object.

As a result of the determination, if there is an object approaching or located within a preset distance of the vehicle, the vehicle system can calculate the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object using the controller ). For example, the controller can calculate the speed of the object and the distance between the vehicle and the object based on the time delay information at which sound information related to the object is collected by the sound detection sensor. More specifically, for example, the controller can use the TDCC technique or the GCC-PHAT technique to measure time delay information in which sound information associated with an object is acquired (reached) by a sound sensor. At this time, the vehicle system can calculate the velocity of the object and the distance between the vehicle and the object at predetermined time intervals using the controller.

The vehicle system then determines, based on the calculated speed of the object and the distance between the vehicle and the object, whether the object is located or approaches the rear left and right lane areas of the vehicle (e.g., Approaching at a threatening rate) (740). For example, when the time delay information is positive, when the object is located on the left rear side of the vehicle and the time delay information is negative, As the absolute value of the time delay becomes larger, it is possible to detect that the object approaches the vehicle.

Here, the vehicle system may use a controller to divide the rear left and right lane areas of the vehicle into a plurality of areas, and to detect which of the plurality of areas is located or entered.

FIG. 8 is a flowchart specifically illustrating a step of detecting whether an object is positioned or entered in a rear left and right lane area of a vehicle when the controller according to an exemplary embodiment of the present invention is set to a mixed mode.

Referring to FIG. 8, a vehicle system according to an embodiment of the present invention may set 810 a region of interest corresponding to a rear left and right lane area of a vehicle in an image obtained from a rear camera. For example, the in-vehicle system may set the region of interest based on detected left and right lane information in the image obtained from the rear camera. More specifically, for example, the vehicle system may include a side mirror of the vehicle and a side mirror of the vehicle on the basis of the left and right lane information detected in the image obtained from the rear camera and the position information of the vanishing point in the image obtained from the rear camera, The blind spot of the room mirror can be calculated, and the area including the calculated blind spot can be set as the area of interest. Also, the vehicle system can determine at least one of the position, shape, size, or number of the ROI using the shadow information of the object.

At least one of the position, shape, size or number of the ROI, which is a fixed area with respect to at least a part of the rear left and right lane areas of the vehicle, includes left and right lane information detected in the image obtained from the rear camera, The position information of the vanishing point in the acquired image, the appearance information of the object located in the image obtained from the rear camera, the operation speed of the controller, or the power consumption of the controller.

At this time, although not shown in the drawings, the vehicle system can acquire images using the rear camera before step 810. [

The in-vehicle system can then use the controller to search 820 for objects in the area of interest corresponding to the rear left and right lane areas of the vehicle. For example, the vehicle system may be configured to determine whether the object is located in an area of interest corresponding to the rear left and right lane areas of the vehicle in the image obtained from the rear camera with respect to each of the frames or with a predetermined time interval while the vehicle is traveling, You can explore whether or not.

Specifically, the in-vehicle system uses a controller to activate a search function for a region of interest corresponding to the rear left and right lane areas of the vehicle in an image obtained from the rear camera so that an object in the region of interest It can be searched whether it is located or entered.

For example, the vehicle system extracts features from a region of interest by applying at least one of HOG (Histogram Of Gradients) technique or Haar-like feature technique based on the search function for the activated ROI, Vector machine, an Adaboost technique, or an artificial neural network (ANN) technique to determine whether an object is located in a region of interest.

In this case, when the HOG technique is applied to the vehicle system, parameters for use in the HOG technique can be set based on the appearance information previously constructed for each of the plurality of objects including the object.

In addition, the on-vehicle system recognizes a change in appearance information or identification information of the object based on the search function for the activated region of interest, or calculates a motion vector of the object to determine whether the object enters the region of interest, It is possible to search whether or not the object is located in the region of interest by searching whether the vehicle travels at the same speed as the vehicle

When the object is located in the area of interest as a result of searching for the object in the area of interest, the vehicle system recognizes the change in the appearance information or the identification information of the object based on the search function for the activated area of interest Alternatively, the motion vector of the object may be calculated to determine whether the object approaches the predetermined range based on the vehicle in the region of interest or whether the object travels at the same speed as the vehicle within a predetermined range based on the vehicle.

Next, the vehicle system uses a controller to apply a directional noise reduction technique to sound information collected from a sound detection sensor, and obtain sound information related to an object excluding sound information and noise information of the vehicle from sound information collected from the sound detection sensor (830).

The automotive system may then calculate the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object (840). For example, the controller may measure the time delay information at which sound information related to the object is collected (reached) by the sound detection sensor using the TDCC technique or the GCC-PHAT technique, and then, based on the time delay information, And the distance between the object and the object can be calculated.

Accordingly, the in-vehicle system can use a controller to search for a result of searching for an object in a region of interest or entering the region of interest, and the sound information collected from the sound sensor (for example, an object calculated based on sound information collected from a sound sensor (850) whether an object is located or entering the rear left and right lane areas of the vehicle based on the speed of the vehicle and the distance between the vehicle and the object - the time delay information at which the sound information is collected.

Further, the vehicle system is not recognized to the driver through the side mirror of the vehicle in the rear left and right lane areas of the vehicle, but is not recognized by the driver through the side mirror, It is possible to detect whether an object is located or enters an area of a third area that is not recognized by the driver and is not photographed by the rear camera through a second area or a side mirror that is less than a predetermined distance from the vehicle photographed by the camera .

For example, when the object is located in the second area in the ROI, the vehicle system recognizes the change in the appearance information or the identification information of the object, or calculates the motion vector of the object to determine whether the object enters the third area Can be detected. The vehicle system senses that the object is located in the third area and informs the driver of the vehicle of the alarm. After the predetermined alarm time elapses or when the object detects that the object is out of the third area, the alarm is canceled . At this time, the on-vehicle system can detect that the object deviates from the third area using the image obtained from the front camera or the image obtained from the rear camera.

The on-vehicle system also uses the distance between the vehicle and the object calculated based on the sound information associated with the object to detect which of the plurality of areas of the vehicle's rear left and right lane areas is located or enters It is possible.

For example, if the distance between the vehicle and the object calculated based on the sound information associated with the object corresponds to the third area, the controller can detect whether the object is located in the third area.

Since the operation using the sound detection sensor is performed in addition to the operation of detecting the position or the entry of an object among the plurality of divided areas using the rear camera described above, The controller can more accurately detect the position of the object.

9 is a block diagram illustrating a system for a vehicle according to an embodiment of the present invention.

9, a vehicle system according to an embodiment of the present invention includes a rear camera 910, a sound detection sensor 920, a memory 930, and a controller 940. [

The rear camera 910 can acquire an image according to a predetermined angle toward the rear of the vehicle.

The sound detection sensor 920 is attached to the outside rear left and right sides of the vehicle so as to collect sound information outside the vehicle toward the rear of the vehicle. For example, the sound detection sensor 920 may be attached in a straight line to the rear of the vehicle, and the location and number of the sound detection sensor 920 may be adaptively changed.

The memory 930 stores image information obtained from the rear camera 910 and sound information collected from the sound detection sensor 920.

The controller 940 is set to either a mixed mode using the rear camera 910 and the sound detection sensor 920 or a sound mode using the sound detection sensor 920 while the vehicle is traveling, Detects whether an object is located or enters the vehicle, and informs the driver of the vehicle of the detection result as an alarm.

At this time, the controller 940 can be set to either the mixed mode or the sound mode based on the sound information collected from the sound detection sensor 920.

The controller 940 set in the mixed mode can set the area of interest corresponding to the rear left and right lane areas of the vehicle in the image obtained from the rear camera 910. [

For example, the controller 940 set to the mixed mode can set the region of interest based on the detected left and right lane information in the image obtained from the rear camera 910. [ More specifically, for example, the controller 940 set in the blending mode determines whether or not the vehicle 910 is in the blind mode based on the left and right lane information detected in the image obtained from the rear camera 910 and the position information of the vanishing point in the image obtained from the rear camera 910 A side mirror of the vehicle and a blind spot of the room mirror among the rear left and right lane areas of the vehicle can be calculated and an area including the calculated blind spot can be set as the area of interest. In addition, the controller 940 set in the mixed mode can determine at least one of the position, shape, size, or number of the ROI using the shadow information of the object.

At least one of the position, shape, size or number of the ROI, which is a fixed area with respect to at least a part of the rear left and right lane areas of the vehicle, includes left and right lane information detected in an image obtained from the rear camera 910, At least one of the position information of the vanishing point in the image obtained from the rear camera 910, the appearance information of the object located in the image obtained from the rear camera 910, the operation speed of the controller 940, Can be set based on one.

Accordingly, the controller 940 set in the mixed mode searches for an object in the area of interest corresponding to the rear left and right lane areas of the vehicle, thereby determining whether or not the object is located in the rear left and right lane areas of the vehicle Whether it is located or is entering. For example, the controller 940 set in the blending mode may be set to the blinking mode so that, during the traveling of the vehicle, the image corresponding to the rear left and right lane areas of the vehicle in the images obtained from the rear camera 910 with respect to each frame or with a predetermined time interval It is possible to search whether the object is located or enters the area of interest.

Specifically, the controller 940 set in the mixed mode acquires from the rear camera 910 by activating the search function for the region of interest corresponding to the rear left and right lane areas of the vehicle in the image obtained from the rear camera 910 It is possible to search whether an object is positioned or entered in the region of interest in the image.

For example, the controller 940, which is set to the mixed mode, applies at least one of HOG (Histogram Of Gradients) technique or Haar-like feature technique based on the search function for the activated ROI, Extracts the object, identifies the object by applying at least one of a Support Vector Machine (SVM) technique, an Adaboost technique, and an Artificial Neural Network (ANN) technique.

In this case, when the HOG technique is applied, the controller 940 set in the mixed mode can set parameters used in the HOG technique based on the appearance information preliminarily formed for each of a plurality of objects including the object.

In addition, the controller 940 set in the mixed mode recognizes the change of the appearance information or the identification information of the object based on the search function for the activated region of interest, or calculates the motion vector of the object to determine whether the object enters the region of interest Or by searching whether the object is traveling at the same speed as the vehicle in the area of interest, whether the object is located in the area of interest

In addition, when the object is located in the ROI as a result of searching for the object in the ROI, the controller 940 set to the ROI mode sets the ROI of the object based on the search function for the ROI Whether the object approaches the predetermined range on the basis of the vehicle in the region of interest or calculates the motion vector of the object so that the object travels at the same speed as the vehicle within a predetermined range based on the vehicle . ≪ / RTI >

In addition, the controller 940 set to the mixed mode applies directional noise reduction technology to the sound information collected from the sound detection sensor 920, and obtains sound information and noise information of the vehicle from the sound information collected from the sound detection sensor 920 The sound information related to the excluded object can be extracted.

Then, the controller 940 set to the mixed mode can calculate the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object. For example, the controller 940 set to the mixed mode can calculate the speed of the object and the distance between the vehicle and the object with a predetermined time interval while the vehicle is running. More specifically, for example, the controller may use a TDCC technique or a GCC-PHAT technique to measure time delay information at which sound information associated with an object is collected (reached) by a sound detection sensor, and then, based on the time delay information, And the distance between the vehicle and the object.

Accordingly, the controller 940 set in the mixed mode is able to determine whether or not the object is located or enters the area of interest, and the sound information collected from the sound detection sensor 920 (e.g., Whether the object is located or enters the rear left and right lane areas of the vehicle based on the speed of the object calculated on the basis of the sound information and the distance between the vehicle and the object and the time delay information in which the sound information is collected) .

In addition, the controller 940 set in the mixed mode has a first region not less than a predetermined distance from the vehicle photographed by the rear camera 910 without being recognized by the driver through the side mirror of the vehicle in the rear left and right lane areas of the vehicle, A third region that is not recognized by the driver through the mirror but is not recognized by the driver through the second region or the side mirror that is less than a predetermined distance from the vehicle photographed by the rear camera 910, It is possible to detect which of the areas the object is located or enters.

For example, when the object is located in the second area, the controller 940 set in the mixed mode recognizes a change in the appearance information or the identification information of the object or calculates the motion vector of the object, It is possible to detect whether or not to enter the area. In addition, the controller 940 set in the mixed mode senses that the object is located in the third area, notifies the driver of the vehicle of the alarm, and after a predetermined alarm time elapses or when the object is detected to be out of the third area In this case, the alarm can be released. At this time, the controller 940 set to the blending mode can detect the deviation of the object from the third area using the image obtained from the front camera (not shown in the figure) or the image obtained from the rear camera 910 have.

Further, the controller 940 set in the mixed mode uses the distance between the vehicle and the object calculated on the basis of the sound information related to the object so that the object is positioned in any one of a plurality of areas of the rear left and right lane areas of the vehicle Or to enter or exit.

For example, when the distance between the vehicle and the object calculated based on the sound information associated with the object corresponds to the third area, the controller 940 set in the mixed mode can sense whether the object is located in the third area .

The controller 940 set to the sound mode applies directional noise reduction technology to the sound information collected from the sound detection sensor 920 and outputs sound information collected from the sound detection sensor 920 to the object excluding sound information and noise information of the vehicle Can be extracted. The process of extracting the sound information related to the object is performed by the controller 940 set to the sound mode. However, the present invention is not limited thereto, and the sound sensor 920 itself may be performed.

Next, the controller 940 set to the sound mode can determine whether there is an object that is approaching or located within a predetermined distance of the vehicle based on the sound information associated with the object. As a result of the determination, when there is an object approaching or located within a preset distance of the vehicle, the controller 940 set in the sound mode can calculate the speed of the object and the distance between the vehicle and the object based on the sound information associated with the object . For example, the controller 940 set to the sound mode can calculate the velocity of the object and the distance between the vehicle and the object by using the Doppler effect based on the sound information associated with the object. At this time, the controller 940 set to the sound mode can calculate the velocity of the object and the distance between the vehicle and the object with a predetermined time interval.

Then, the controller 940 set to the sound mode can detect whether the object is located in or approaches the rear left and right lane areas of the vehicle (for example, whether the object is approaching at a speed that threatens the vehicle) have.

In this case, the controller 940 set in the sound mode can detect the area in which the object is located or enters, by dividing the rear left and right lane areas of the vehicle into a plurality of areas.

10 is a view for explaining a method of detecting whether an object is positioned or entering a rear left and right lane area of a vehicle based on sound information related to an object according to an embodiment of the present invention.

10, a vehicle system according to an embodiment of the present invention includes a sound sensor 1010 and 1020 attached to the rear of the vehicle, It is possible to detect whether an object is located in or entering the area. Hereinafter, it is described that the vehicle system performs the process of detecting the object through the controller, but the present invention is not limited thereto and can be performed using the sound detection sensors 1010 and 1020 themselves.

For example, each of the first sound detection sensor 1010 and the second sound detection sensor 1020, which are attached to the rear of the vehicle with a predetermined distance L,

Figure 112015109841261-pat00001
And
Figure 112015109841261-pat00002
Can be measured. Accordingly, when the object approaches the speed v, the in-vehicle system transmits the time delay information
Figure 112015109841261-pat00003
And
Figure 112015109841261-pat00004
The angle at which the sound information associated with the object reaches
Figure 112015109841261-pat00005
Can be calculated as shown in Equation (1).

&Quot; (1) "

Figure 112015109841261-pat00006

Accordingly, the vehicular system has a rear

Figure 112015109841261-pat00007
It is possible to detect that an object exists at an angle.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (19)

delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete A vehicle system for sensing a rear left and right lane area of a vehicle,
Rear camera;
A sound detection sensor attached to outside rear left and right sides of the vehicle;
A memory for storing images obtained from the rear camera and sound information collected from the sound sensor; And
Wherein the vehicle is selectively set to either a mixed mode using both the rear camera and the sound detection sensor or a sound mode using only the sound detection sensor while the vehicle is traveling so that the object is located in the rear left and right lane areas of the vehicle And notifies the driver of the vehicle of the detection result as an alarm,
Lt; / RTI >
The controller set in the mixing mode
A result of searching for an object in the ROI that corresponds to the rear right and left lane areas of the vehicle and determining whether the object is located or entering the ROI, Based on the information, whether or not the object is located in the rear left and right lane areas of the vehicle,
The region of interest
A side mirror of the vehicle and a blind spot of the room mirror, wherein the blind spot includes left and right lane information detected in the image obtained from the rear camera and a left side lane information detected in the image obtained from the rear camera Of the vehicle is calculated based on the position information of the vehicle. 17. The method of claim 16,
The controller
And the sound mode is set to either the mixed mode or the sound mode based on sound information collected from the sound detection sensor. delete 17. The method of claim 16,
The controller set in the sound mode
And detects whether or not the object is located in a rear left and right lane area of the vehicle based on sound information collected from the sound detection sensor.
KR1020150158085A 2015-11-11 2015-11-11 Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same KR101748497B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150158085A KR101748497B1 (en) 2015-11-11 2015-11-11 Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150158085A KR101748497B1 (en) 2015-11-11 2015-11-11 Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same

Publications (2)

Publication Number Publication Date
KR20170055203A KR20170055203A (en) 2017-05-19
KR101748497B1 true KR101748497B1 (en) 2017-06-19

Family

ID=59049459

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150158085A KR101748497B1 (en) 2015-11-11 2015-11-11 Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same

Country Status (1)

Country Link
KR (1) KR101748497B1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200072579A (en) 2018-11-29 2020-06-23 현대자동차주식회사 Driving safety control system in use with around noise and the method of it

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007234019A (en) * 2006-02-28 2007-09-13 Alpine Electronics Inc Vehicle image area specifying device and method for it
JP2011034435A (en) * 2009-08-04 2011-02-17 Nissan Motor Co Ltd Vehicle location calculation unit and vehicle location calculation method
JP2013068434A (en) * 2011-09-20 2013-04-18 Toyota Motor Corp Sound source detection device
KR101565006B1 (en) * 2014-05-30 2015-11-13 엘지전자 주식회사 apparatus for providing around view and Vehicle including the same

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007234019A (en) * 2006-02-28 2007-09-13 Alpine Electronics Inc Vehicle image area specifying device and method for it
JP2011034435A (en) * 2009-08-04 2011-02-17 Nissan Motor Co Ltd Vehicle location calculation unit and vehicle location calculation method
JP2013068434A (en) * 2011-09-20 2013-04-18 Toyota Motor Corp Sound source detection device
KR101565006B1 (en) * 2014-05-30 2015-11-13 엘지전자 주식회사 apparatus for providing around view and Vehicle including the same

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20200072579A (en) 2018-11-29 2020-06-23 현대자동차주식회사 Driving safety control system in use with around noise and the method of it

Also Published As

Publication number Publication date
KR20170055203A (en) 2017-05-19

Similar Documents

Publication Publication Date Title
KR101692628B1 (en) Method for detecting right lane area and left lane area of rear of vehicle using region of interest and image monitoring system for vehicle using the same
US10796171B2 (en) Object recognition apparatus, object recognition method, and object recognition program
US9902426B2 (en) Apparatus and method for driver assistance
JP5867273B2 (en) Approaching object detection device, approaching object detection method, and computer program for approaching object detection
JP6838248B2 (en) Information processing device
JP5297078B2 (en) Method for detecting moving object in blind spot of vehicle, and blind spot detection device
JP4692344B2 (en) Image recognition device
KR101395089B1 (en) System and method for detecting obstacle applying to vehicle
EP1671216B1 (en) Moving object detection using low illumination depth capable computer vision
US11236991B2 (en) Method for determining a current distance and/or a current speed of a target object based on a reference point in a camera image, camera system and motor vehicle
KR101103526B1 (en) Collision Avoidance Method Using Stereo Camera
CN108482367B (en) Method, device and system for assisting driving based on intelligent rearview mirror
EP2741234B1 (en) Object localization using vertical symmetry
JP2009237898A (en) Image recognition device
KR20140004413A (en) Apparatus and method for assisting safe driving
Berriel et al. A particle filter-based lane marker tracking approach using a cubic spline model
KR20150145685A (en) Apparatus for recognizing target and method thereof using global positioning system
KR101748497B1 (en) Method for detecting right lane area and left lane area of rear of vehicle using sound data and vehicle system using the same
Kim et al. An intelligent and integrated driver assistance system for increased safety and convenience based on all-around sensing
JP2015191437A (en) Driving support device
US20210178966A1 (en) Door opening aid systems and methods
Dhanasekaran et al. A survey on vehicle detection based on vision
Kumar et al. Vision-based vehicle detection survey
KR101808937B1 (en) Method for Detecting Driving Pattern of Object Using Histogram of Gradients and Apparatus Therefor
Inoue et al. Following vehicle detection using multiple cameras

Legal Events

Date Code Title Description
E701 Decision to grant or registration of patent right
GRNT Written decision to grant