KR20210081887A - Danger detection output device and method using captured image of rear and side of vehicle - Google Patents

Abstract

An embodiment of the present invention discloses a risk detection output device using a captured image of rear and lateral sides of a vehicle, which comprises: a first capturing device installed at a first position outside the vehicle to capture an image corresponding to a side view; a second capturing device installed at a second position outside the vehicle to capture the image corresponding to the side view; a capturing control unit receiving images from the first and second capturing devices and distinguishing and detecting persons, the vehicles, leans, traffic lights, and road signs from the images; an output control unit distinguishing the persons, the vehicle, the lanes, the traffic lights, and the road signs from the capturing control unit to receive the detected image, detecting risk information from the image, and generating output data having the image and risk information; and an output device outputting the output data.

Description

DANGER DETECTION OUTPUT DEVICE AND METHOD USING CAPTURED IMAGE OF REAR AND SIDE OF VEHICLE

The present disclosure relates to a risk detection output device and an output method using a photographed image of a vehicle side and rear.

In general, in order to change a lane of a vehicle, a driver observes a side mirror or a room mirror to check the safety of the side or rear of the lane to be changed.

That is, when changing lanes, the vehicle looks at the side mirror or the room mirror. Even at this time, since the vehicle is continuously moving, if the time to check this becomes longer, the time to not look ahead This length becomes longer, impairing the safe operation of the vehicle.

Therefore, skill is required to quickly observe the situation of the side or rear of the lane through a side mirror or a room mirror.

The present invention is to improve the above problems, and the technical problem to be achieved by the present invention is to photograph an image of the side and rear of a vehicle and provide risk information from the photographed image.

A risk detection output device according to embodiments of the present invention includes: a first imaging device installed at a first position outside a vehicle to capture an image corresponding to a side view; a second imaging device installed at a second location outside the vehicle to capture an image corresponding to a side view; a photographing control unit that receives images from the first and second image pickup devices and distinguishes and detects people, vehicles, lanes, traffic lights, and road signs from the images; an output control unit configured to receive an image detected by distinguishing a person, a vehicle, a lane, a traffic light, and a road sign from the photographing control unit, detect danger information from the image, and generate output data including the image and the danger information; and an output device for outputting the output data, it is possible to use a photographed image of the side and rear of the vehicle.

The output controller may detect a lane change event, an event overtaking a preceding vehicle, a cut-in event, a speed change event, and a lane departure event with respect to an object recognized as a vehicle, and may generate risk information.

The first imaging device and the second imaging device may set a range obtained by a side mirror and a rearview mirror of the vehicle as an image range.

The photographing control unit may generate a photographing control signal based on the risk information generated by the output control unit and transmit the generated photographing control signal to the first and second imaging devices.

A risk detection output method according to embodiments of the present invention may include: capturing an image corresponding to a side view by installing the risk detection output device at a first location outside the vehicle through a first imaging device; photographing an image corresponding to a side view by installing the risk detection output device at a second location outside the vehicle through a second imaging device; receiving, by the risk detection output device, images from the first and second imaging devices, and distinguishing and detecting a person, a vehicle, a lane, a traffic light, and a road sign from the images; The risk detection output device receives an image detected by distinguishing a person, a vehicle, a lane, a traffic light, and a road sign from the photographing control unit, detects risk information from the image, and generates output data including the image and the risk information step; and outputting the output data by the risk detection output device.

In the generating of the output data, a lane change event, an event overtaking a preceding vehicle, a cut-in event, a speed change event, and a lane departure event may be detected for an object recognized as a vehicle and generated as risk information.

The computer program according to the embodiment of the present invention may be stored in a medium to execute any one of the output methods according to the embodiment of the present invention using a computer.

In addition to this, another method for implementing the present invention, another system, and a computer readable recording medium for recording a computer program for executing the method are further provided.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

The risk detection output apparatus and output method using the image taken around the vehicle according to the present invention as described above may take an image of the side and rear of the vehicle and provide risk information from the captured image.

1 is a block diagram of a risk detection output device using a photographed image of a side and rear side of a vehicle according to embodiments of the present invention.
2 is a block diagram illustrating operations of a photographing control unit and an output control unit.
3 and 4 are exemplary views of installation positions of imaging devices according to embodiments of the present invention.
5 is an exemplary view of an installation position of an output device according to embodiments of the present invention.
6 and 7 are flowcharts of a risk detection output method according to embodiments of the present invention.
8 and 9 are diagrams for explaining risk information detected by the risk detection output device.

Hereinafter, the configuration and operation of the present invention will be described in detail with reference to the embodiments of the present invention shown in the accompanying drawings.

Since the present invention can apply various transformations and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. Effects and features of the present invention, and a method of achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various forms.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, and when described with reference to the drawings, the same or corresponding components are given the same reference numerals, and the overlapping description thereof will be omitted. .

In the following embodiments, terms such as first, second, etc. are used for the purpose of distinguishing one component from another, not in a limiting sense.

In the following examples, the singular expression includes the plural expression unless the context clearly dictates otherwise.

In the following embodiments, terms such as include or have means that the features or components described in the specification are present, and the possibility of adding one or more other features or components is not excluded in advance.

In the drawings, the size of the components may be exaggerated or reduced for convenience of description. For example, since the size and thickness of each component shown in the drawings are arbitrarily indicated for convenience of description, the present invention is not necessarily limited to the illustrated bar.

In the following embodiments, the x-axis, the y-axis, and the z-axis are not limited to three axes on a Cartesian coordinate system, and may be interpreted in a broad sense including them. For example, the x-axis, y-axis, and z-axis may be orthogonal to each other, but may refer to different directions that are not orthogonal to each other.

Where certain embodiments are otherwise feasible, a specific process sequence may be performed different from the described sequence. For example, two processes described in succession may be performed substantially simultaneously, or may be performed in an order opposite to the order described.

1 is a block diagram of a risk detection output device 100 using a side view camera according to embodiments of the present invention.

The risk detection output device 100 includes a first imaging device 110 , a second imaging device 111 , a photographing control unit 120 , an output control unit 130 , and a first output for photographing the same photographing range as the side mirrors. device 140 .

The risk detection output device 100 may capture an image of the side or rear of the vehicle through the first imaging device 110 and the second imaging device 111 . The risk detection output device 100 may output captured images from the connected first imaging device 110 and the second imaging device 111 .

The first imaging device 110 and the second imaging device 111 may be disposed to face the side and/or the rear when the vehicle moves. The first imaging device 110 and the second imaging device 111 may be implemented to photograph the image area of the rearview mirror as well as the side mirror. The image area captured by the first imaging device 110 and the second imaging device 111 may include a range acquired by a side mirror and/or a rearview mirror of the vehicle, but is not limited thereto, and may be expanded to various ranges. can The installation positions of the first imaging device 110 and the second imaging device 111 may be inside or outside the vehicle.

The first imaging device 110 and/or the second imaging device 111 may capture images with various angles of view, each including lenses having different angles of view. By adjusting the angles of view of the lenses of the first imaging device 110 or the second imaging device 111, the shooting range of the first imaging device 110 and/or the second imaging device 111 may be changed according to circumstances. have. For example, the first imaging device 110 or the second imaging device 111 is set to photograph the side of the vehicle as in the recognition range of the side mirrors, and when a situation in which a rear view is required occurs, the photographing range according to the situation occurs can be set to take pictures.

Through this, information on road conditions occurring on the side and rear of the vehicle and information on surrounding vehicles may be acquired, and driving-related processing information may be generated based on the acquired information.

The photographing control unit 120 may be implemented to transmit a control signal to the first imaging device 110 and/or the second imaging device 111 and to receive a photographed image according to the control signal. The capturing controller 120 may process captured images.

The output controller 130 may receive the captured image and generate output data to be output through the output device 140 .

The output controller 130 may generate output data using a result of analyzing the captured image. The output data may include a photographed image and/or processing information on the photographed image.

The output control unit 130 may distinguish a vehicle, a person, a road, a lane, a road sign, a traffic light, etc. by using a result of analyzing the captured image, and generate the differentiated information as separate image information.

The output controller 130 may generate output data composed of a plurality of layers. For example, a first layer expressing a captured image, a second layer expressing image information expressed by distinguishing vehicles, people, and roads in the captured image, and a third layer expressing information on road conditions obtained from the captured image It may include at least one of a layer and a fourth layer expressing information on a dangerous situation detected in the captured image.

The output device 140 is installed at a designated location inside the vehicle to output output data. The output device 140 may be one output device and may include a plurality of output devices.

2 is a block diagram illustrating operations of the photographing control unit 120 and the output control unit 130 .

The first image obtaining unit 121 of the capturing controller 120 may receive a first captured image from the first capturing apparatus 110 .

The second image obtaining unit 122 of the capturing controller 120 may receive a second captured image from the second capturing apparatus 110 .

The image processing unit 123 may distinguish and detect a vehicle, a person, a road, a lane, a traffic light, a road sign, etc. from the first captured image and/or the second captured image by using an object detection algorithm. Here, the object detection algorithm refers to an algorithm for detecting the existence of an object existing in an image by determining the presence or absence. The object detection algorithm is detected through a process of extracting a feature and determining a boundary from the distribution of features. Methods for extracting features include Haar-like feature, HOG (Histogram of Oriented Gradient), SIFT (Scale Invariant Feature Transform), LBP (Local Binary Pattern), MCT (Modified Census Transform), etc. After extracting the feature, Objects can be detected by using the same detection algorithm (classifier) such as SVM (Support Vector Machine) or Adaboost to classify whether the distribution of a feature represents the object or not. . The image processing unit 123 may identify the type of the detected object using an object recognition algorithm.

The image processing unit 123 may detect information on the shape, position, size, and color of an object in the captured image, and distinguish the object of the object in consideration of the detected object information.

The data transmitter 124 may transmit the captured image processed by the image processing unit 123 and information on the captured image to the output controller 130 . The data transmission unit 124 transmits an image detected by distinguishing a person, a vehicle, a lane, a traffic light, and a road sign to the output control unit 130 .

The control signal generator 125 may generate a shooting control signal according to a set input and transmit it to the first imaging device 110 and/or the second imaging device 111 . The photographing control signal may include a signal for controlling a photographing range photographed through the imaging device. For example, the control signal generator 125 basically transmits a first photographing control signal for photographing in a first photographing range set at a first time point, and generates a second photographing control signal for photographing in a predetermined second photographing range. It can be transmitted at two times.

When the control signal generator 125 is set to the automatic control mode, the control signal generator 125 may automatically generate a control signal for better recognizing the detected risk information and transmit it to the imaging devices.

The data receiving unit 131 of the output control unit 130 may receive a captured image and a result of analyzing the captured image from the capturing control unit 120 .

The risk detection unit 133 of the output control unit 130 may generate risk information in the captured image. Here, the risk information refers to information related to the risk of an accident in the vicinity of the vehicle, and specifically may include information about an appearance vehicle having a dangerous driving habit. The risk information may include information about a vehicle that changed lanes, information about a vehicle that has overtaken, information about a vehicle that continuously changes between low speed and high speed, information about a vehicle that does not follow a lane, and the like. The risk information may include information about a vehicle that is a danger target, weather that is a danger target, and information about a road condition.

The situation analyzer 133 may analyze the captured image to extract road situation information such as the type of road on which the vehicle travels, traffic signals around the vehicle, vehicle movement, and the possibility of colliding with another vehicle.

The risk detector 133 may detect an object corresponding to the vehicle included in the captured image and detect a lane change event of the vehicle and the corresponding object.

The risk detection unit 133 may detect an object corresponding to a vehicle included in the captured image, and detect an event of overtaking a vehicle and/or an interruption event of the object corresponding to the vehicle.

The risk detection unit 133 may detect an object corresponding to a vehicle that moves at a low speed, moves at a high speed, and then moves again at a low speed included in the captured image, and detects a speed change event of the object corresponding to the vehicle.

The danger detector 133 may detect an object corresponding to the vehicle included in the captured image and detect a lane departure event of the object corresponding to the vehicle.

The risk detection unit 133 sets a vehicle that has generated a lane change event, an event overtaking a preceding vehicle, a cut-in event, a speed change event, and a lane departure event as a dangerous object, and can generate movement information for the dangerous object. have.

When a dangerous object is detected in an image captured by the first imaging device 110 and/or the second imaging device 111 , the danger detection unit 133 may acquire movement information of the dangerous object.

The danger detection unit 133 may set a surrounding vehicle as a danger target based on the sensed data sensed by the sensor unit (not shown).

The output data generator 134 may generate output data representing a captured image and risk information. The output data generation unit 134 includes a first layer expressing a captured image, a second layer expressing image information expressed by distinguishing vehicles, people, and roads from the captured image, and a second layer expressing risk information obtained from the captured image. You can create output data that includes 3 layers.

The output data generator 134 may generate driving instruction information in consideration of movement information of a dangerous object included in the danger information. Here, the driving instruction information may include driving instructions for avoiding contact with the dangerous object in consideration of the movement information of the dangerous object. For example, when the first vehicle driving in the first lane is set as a dangerous object, the driving instruction information is instruction information for changing a lane to a lane different from the dangerous object, that is, instruction information for moving to the third or fourth lane. may include.

The first imaging device 110 and the second imaging device 111 of the risk detection output device 100 may be set to the same angle of view as a vehicle side mirror and/or a rearview mirror to capture an image.

The output control unit 130 is a shooting control signal for controlling the shooting range of the first imaging device 110 and/or the second imaging device 111 in consideration of the direction, location, etc. of the dangerous object set by the risk detection unit 133 . can be transmitted.

In the risk detection output device 100 , as shown in FIG. 1 , two imaging devices may be implemented, but the present invention is not limited thereto, and two or more imaging devices may be provided.

3 and 4 are exemplary views of installation positions of the imaging devices 110 and 111 according to embodiments of the present invention.

As shown in FIG. 3 , the imaging devices 110 and 111 may be installed on the side mirror and/or the rear bumper to photograph the side and the rear to photograph the image. As shown in FIG. 4 , it may be installed on the rear bumper of the vehicle to intensively photograph the rear.

5 is an exemplary view of an installation position of the output device 140 according to embodiments of the present invention.

In an embodiment, the output device 140a may be installed on a driver's side glass to provide necessary images and risk information to the driver.

In another embodiment, the output devices 140b ′ and 140b″ may be installed on both lower ends of the windshield of the vehicle to provide necessary images and risk information to the driver.

In another embodiment, the output device 140c may be installed in an area inside the vehicle to provide necessary images and risk information to the driver.

6 and 7 are flowcharts of a risk detection output method according to embodiments of the present invention.

The risk detection output device 100 captures images around the vehicle through the first imaging device and the second imaging device (S110).

The risk detection output device 100 may analyze the image captured around the vehicle, distinguish the vehicle, the person, and the road included in the image and extract it as an object (S120).

The risk detection output device 100 generates risk information from the captured image (S130). Since the process of generating the risk information is the same as the operation of the risk detection unit 133 of the output control unit 130 , a detailed description thereof will be omitted.

The risk detection output device 100 generates output data including a captured image and risk information (S140). Since the process of generating the output data is the same as the operation of the output data generating unit 134 , a detailed description thereof will be omitted.

The risk detection output device 100 outputs output data through the output device (S150). Since the process of outputting the output data is the same as the operation of the output control unit 135, a detailed description thereof will be omitted.

Through this, the risk detection output device 100 may capture images of the side and rear of the vehicle, generate risk information based on the captured images, and provide the captured images and risk information to the driver.

As shown in FIG. 7 , the risk detection output device 100 may capture an image and detect and recognize one or more objects in the image ( S210 and S220 ).

In S230, the risk detection output device 100 may detect an object corresponding to the vehicle included in the captured image and detect a lane change event of the vehicle and the corresponding object.

The risk detection output device 100 may detect an object corresponding to the vehicle included in the captured image, and detect an event of overtaking a vehicle and/or an interruption event of the object corresponding to the vehicle.

The risk detection output device 100 may detect an object corresponding to a vehicle that moves at a low speed included in the captured image, moves at a high speed, and then moves again at a low speed, and detects a speed change event of the object corresponding to the vehicle.

The risk detection output device 100 may detect an object corresponding to the vehicle included in the captured image and detect a lane departure event of the object corresponding to the vehicle.

The risk detection output device 100 sets a vehicle that generates a lane change event, an event overtaking a preceding vehicle, a cut-in event, a speed change event, and a lane departure event as a dangerous object, and generates movement information for the dangerous object. can

The risk detection output device 100 may acquire movement information of the dangerous object when the dangerous object is detected in the image captured by the first imaging device 110 and/or the second imaging device 111 .

The danger detection output device 100 may set a surrounding vehicle as a danger target based on sensing data sensed by a sensor unit (not shown). Through this, it is possible to prevent an unexpected accident on the road by detecting a dangerous target performing retaliatory driving, drunk driving, drowsy driving, and the like.

In S240, the risk detection output device 100 may extract movement information on the dangerous object included in the risk information. The risk detection output device 100 may generate a photographing control signal according to the movement of the dangerous object to control the imaging apparatuses. The photographing control signal may be a signal for controlling a focal length and an angle of view of an imaging device for tracking a dangerous object.

In S250, the danger detection output device 100 generates driving instruction information for preventing a collision with a dangerous object. When the distance to the dangerous object is within a preset danger radius, the danger detection output device 100 may generate driving instruction information for preventing a collision with the dangerous object in consideration of the moving direction and the moving speed of the dangerous object. If the dangerous object approaches from the right within the danger radius, the driving instruction information may include information for changing a lane to the left. If a dangerous object approaches to overtake the corresponding vehicle within the danger radius, information inducing the dangerous object to overtake by slowing the speed may be provided as driving instruction information.

In S260, the danger detection output device 100 outputs movement information and driving instruction information on the dangerous object through the output device.

8 and 9 are diagrams illustrating risk information detected by the risk detection output device 100 .

The vehicle 1 including the danger detection output device may detect road conditions and danger information of a predetermined radius d1 or more through imaging devices. The vehicle 1 may detect a vehicle that exists beyond a certain radius as danger information through the danger detection output device. The danger detection output device may determine the moving direction of the vehicle RO1 within a lane. That is, the risk detection output device may detect that the left and right movement occurs in the lanes L1 and L2 when the vehicle RO1 is driven.

In this case, the hazard detection output device can output including guidance and driving instruction information for hazard information such as "A dangerous object has been detected. Be careful of vehicles approaching from the right (OD1)."

As illustrated in FIG. 9 , a vehicle changing a lane may be detected by the danger detection output device and detected as danger information. The risk detection output device may recognize the vehicles RO2' and RO" and the vehicle O3. The risk detection output device generates an event of overtaking the preceding vehicle among the recognized vehicles RO2' and RO2. ”) can be set as danger information. The danger detection output device can output the captured video and “Dangerous object has been detected. Allow overtaking of an oncoming vehicle from the right (OD2)”.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible from the above description by those skilled in the art. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (9)

a first imaging device installed at a first position outside the vehicle to capture an image corresponding to a side view;
a second imaging device installed at a second location outside the vehicle to capture an image corresponding to a side view;
a photographing control unit that receives images from the first and second imaging devices and distinguishes and detects at least one of a person, a vehicle, a lane, a traffic light, and a road sign from the images;
an output control unit configured to receive an image detected by distinguishing at least one of a person, a vehicle, a lane, a traffic light, and a road sign from the photographing control unit, detect danger information from the image, and generate output data including the image and the danger information; and
And an output device for outputting the output data, a risk detection output device using a photographed image of the vehicle side and rear. According to claim 1,
The output control unit
For an object recognized as a vehicle, a lane change event, an event overtaking a preceding vehicle, a cut-in event, a speed change event, and a lane departure event are detected and generated as risk information. Risk detection using images taken from the side and rear of the vehicle output device.
According to claim 1,
The first imaging device and the second imaging device are
A hazard detection output device using a photographed image of a side and rear of a vehicle, in which a range obtained by the side mirror and the rearview mirror of the vehicle is set as an image range. According to claim 1,
The shooting control unit
A risk detection output device using a photographed image of a side and rear of a vehicle to generate a photographing control signal based on the risk information generated by the output control unit and transmit the generated photographing control signal to the first and second imaging devices. The risk detection output device is installed at a first position outside the vehicle through the first imaging device to capture an image corresponding to the side view;
photographing an image corresponding to a side view by installing the risk detection output device at a second location outside the vehicle through a second imaging device;
receiving, by the risk detection output device, images from the first and second imaging devices, and distinguishing and detecting at least one of a person, a vehicle, a lane, a traffic light, and a road sign from the images;
The risk detection output device receives an image detected by distinguishing at least one of a person, a vehicle, a lane, a traffic light, and a road sign from the photographing control unit, detects risk information from the image, and outputs data including the image and the risk information creating a; and
The risk detection output device comprising the step of outputting the output data, the risk detection output method using a photographed image of the vehicle side and rear.
6. The method of claim 5,
The step of generating the output data is
For an object recognized as a vehicle, a lane change event, an event overtaking a preceding vehicle, a cut-in event, a speed change event, and a lane departure event are detected and generated as risk information. Risk detection using images taken from the side and rear of the vehicle output method.
6. The method of claim 5,
The first imaging device and the second imaging device are
A risk detection output method using a photographed image for the side and rear of a vehicle, in which a range obtained by the side mirror and the rearview mirror of the vehicle is set as an image range.

6. The method of claim 5,
The risk detection output device generates a capture control signal based on the generated risk information and transmits it to the first and second imaging devices, the risk detection output method using the captured images for the side and rear of the vehicle .
A computer program stored in a computer-readable storage medium for executing the method of any one of claims 5 to 8 using a computer.

Images