KR20150130717A - Event detection system of blackbox for vehicle and method for event detection - Google Patents

Abstract

The present invention relates to an event detection system of a black box for a vehicle in parking and an event detection method. The event detection system of a black box for a vehicle comprises: super wide-angle camera units installed in the front, the rear, the left and the right sides of the vehicle to photograph the front, the rear, the left and the right sides outside the vehicle; a sensing senor unit to sense vibration and an impact caused by an external impact to the vehicle; an around view image generating unit to compose and correct images inputted from the super wide-angle camera units installed in the front, the rear, the left and the right sides of the vehicle to generate images in an around view manner; an image recording unit to record the images generated by the around view image generating unit; an image sensing unit to detect and trace an object moving in the corrected images generated in an around view manner; an event sensing unit to determine the object as an event when the object detected and traced in the image sensing unit corresponds to a predefined movement, or information is sensed by the sensor unit; a control unit to store the images recorded by the image recording unit and event information determined as the event by the event sensing unit in a database unit, and to display event generation information on a display unit when the event information stored in the database unit exists simultaneously with the starting of the vehicle; the database unit to store the images recorded in the image recording unit and the event information determined as the event in the event sensing unit; and the display unit to display the event generation information by a video summary playback or a thumbnail image of the event images when there is the event information stored in the database unit, or to display the event generation information by the video summary playback or the thumbnail image of the event images according to a location of the vehicle in which the event occurs.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an event detection system for a vehicle,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an event detection system and an event detection method for a black box for a vehicle during parking, and more particularly, to a method and apparatus for generating an overview image of an image photographed in forward, backward, left, , Detects a predefined event in the generated surround view image, records the continuous image and stores the event information separately, thereby deleting the event image including the event information even during the automatic deletion of the recorded continuous image The present invention also relates to an event detection system and a method for detecting an event of a parking black box in a vehicle, which enables a driver to recognize an event occurrence information at the same time as a vehicle starts, will be.

In the case of an existing traffic accident, the cause of the accident was mainly determined by the witness's statement or the empirical judgment of the police officer. Since the subjective judgment is involved in the method of identifying the cause of the traffic accident, There was a problem that the perpetrator and the victim turned around. In order to solve such a problem, there is an increasing need for a vehicle black box which can grasp the state of the vehicle at the time of an accident in case of a traffic accident or theft, and accurately analyze the cause of the accident, The introduction of mandatory schemes is underway.

Such a black box for a vehicle is composed of a camera installed inside and outside the vehicle and a memory for storing image information for storing images inside or outside the vehicle. It can be controlled in such a manner as continuous recording that continuously records and impact recording that can only record when a certain impact is received

Since the conventional continuous recording method is a method of storing a continuous image regardless of the occurrence of an event, an event such as a collision accident occurring while driving is present in the vehicle when the event occurs, However, in case of an event such as a contact accident caused by a vehicle of another person in the absence of a driver, such as at the time of parking, unless the driver directly confirms a mark on his / her vehicle, he / And eventually it passed through the vehicle black box device without confirming it even though the corresponding event image was recorded. Particularly, the storage capacity of a memory of a conventional vehicle black box is very limited. When the maximum storage capacity of the memory is reached during normal recording due to the limitation of the storage capacity of the memory, the previously stored image is deleted and stored A problem arises in that the event image is already deleted even if the driver attempts to check the vehicle for scratches or the like and check the related incident-related event image.

On the other hand, Korean Patent Registration No. 10-0966288 entitled " Method and Apparatus for Generating a Peripheral Image ", relating to cameras and video technologies, synthesizes images captured through cameras installed on the front, rear, left, and right sides of a vehicle, A method and an apparatus for generating an image in an surround view manner have been described. However, there is no solution for how to store an image of a high resolution, which is generated by the surround view method and has a larger capacity, in a conventional limited vehicle black box memory Korean Patent Registration No. 10-0883632 " Intelligent video surveillance system and method using a high-resolution camera ", in which a high-resolution image input from a high-resolution camera is analyzed in real time to detect / track a moving object, Event detection is performed, and based on this, But it proposes to perform effective surveillance and PTZ create digital images of specific objects from a system and method for high-resolution video had this superior technology does not apply to a vehicle black box system problems.

Korean Patent Publication No. 10-2007-0110809 (November 20, 2007) Korean Registered Patent No. 10-0966288 (Jun. 18, 2010) Korean Patent Registration No. 10-0883632 (Feb.

According to an aspect of the present invention, there is provided a method for generating an overview image of an image captured in front, rear, left, and right directions of a vehicle during parking, detecting predefined events in the generated surround view image, It is possible to prevent the event image including the event information from being deleted even during the automatic deletion of the recorded continuous image, thereby confirming that the event image due to the existing large-capacity image recording is recorded And an object thereof is to provide an event detection system and a method for detecting an event of a parking black box for a vehicle.

In addition, if the stored event information is present, the driver can confirm the event occurrence information at the same time as the vehicle starts, so that it is possible to inform the driver in real time whether or not the event occurred during parking, There is another object of the present invention to provide a black box event detection system and method for detecting such events.

According to an aspect of the present invention, there is provided an image processing apparatus including an ultra-wide angle camera unit installed at the front, rear, left, and right sides of a vehicle and capable of photographing in front, rear, left, A detection sensor unit for detecting vibrations or impacts caused by an external impact, an image input from an ultra-wide angle camera unit installed on the front, rear, left, and right sides of the vehicle, An image capturing unit for capturing an image generated in the surround view image generating unit, an image sensing unit for detecting and tracking an object moving in the image generated by the corrected surround viewing method, If the tracked object detected by the unit corresponds to a predefined motion, or if information is detected by the sensor unit, And an event recording unit for storing event information determined as an event in the event recording unit and the event information recorded in the database unit at the same time when the vehicle is started, A database unit for storing event information determined as an event in the video recording unit and the event sensing unit, and a thumbnail image or video summary playback of the event video when there is event information stored in the database unit, And a display unit for displaying event occurrence information by thumbnail images or video summary reproduction of event images for each vehicle location in which the event occurred.

According to another aspect of the present invention, there is provided a method of driving a vehicle, comprising the steps of: capturing an omnidirectional vehicle outside the vehicle using an ultra-wide angle camera provided at the front, rear, left, and right sides of the vehicle; The method includes the steps of generating an overview image by synthesizing the photographed images and correcting the distorted image, continuously recording the generated overview image in a database unit, detecting the moving object in the generated overview image, Determining whether an event is detected when the detected object corresponding to the predefined movement or the vibration or impact information is detected by the detection sensor unit; and if the event is detected, storing the event information in the database unit If there is event information stored in the database unit, The event occurrence information is displayed on the display unit at the same time as the start of the event, and after the event occurrence information is displayed, the event occurrence information is deleted or stored.

According to the present invention, an image captured in front, rear, left, and right directions of a vehicle is generated as an ambient view image, and a predefined event is detected in the generated surrounding view image to detect and track the object. Not only can an object be detected, but also attribute information such as an accurate color, size, speed, and ratio of the object can be included in the image even when the object is tracked.

In addition, when the detected object is determined as an event occurrence by the predefined motion, only the event information is stored separately, thereby preventing the event image including the event information from being deleted even during the automatic deletion of the recorded continuous image, The problem that the event image due to the large-capacity image recording can not be confirmed even though the event image is recorded is improved.

In addition, if the stored event information is present, the driver can confirm the event occurrence information at the same time as the vehicle starts, so that it is possible to inform the driver in real time whether or not the event occurred during parking, There is an effect that it is possible to prevent the user from recognizing the occurrence or not to pass through.

FIG. 1 is a block diagram illustrating an overall configuration of an event detection system for a vehicle black box according to an embodiment of the present invention. Referring to FIG.
FIG. 2 is a detailed configuration diagram illustrating an overall configuration of an event detection system of a black box for a vehicle during parking according to an embodiment of the present invention.
3 is a flowchart illustrating a method of detecting an event of a black box for a vehicle during parking according to an embodiment of the present invention.

FIG. 1 is a diagram illustrating an overall configuration of an event detection system of a black box for a car during parking according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an overall configuration of an event detection system of a black box for a car during parking according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating a method of detecting an event of a black box for a vehicle during parking according to an embodiment of the present invention. Referring to FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It is to be understood that the same is by way of illustration and example only and is not to be taken by way of limitation, the scope of the present invention being limited only by the terms of the appended claims.

1 and 2, an event detection system for a vehicle-use black box according to the present invention includes an ultra-wide angle camera unit 10, a detection sensor unit 20, an ambient view generation unit 30, An image sensing unit 50, an event sensing unit 60, a control unit 70, a data base unit 80, and a display unit 90. The smart device 200 may further include a communication unit 100 including a communication module so that the smart device 200 can be connected to the mobile communication network.

First, the super wide-angle camera 10 according to the present invention uses four super-wide-angle cameras 10a, 10b, 10c and 10d. The super wide-angle camera has a stronger property than the wide-angle lens, Which has an angle of view of at least 120 degrees, which is capable of distorting an image with a wide angle of view. In general, an ultra-wide angle lens is also called an ultra wide angle lens. When a subject is photographed with a camera having such an ultra-wide angle lens, the center portion is projected and the image is photographed. .

The super wide-angle cameras 10a, 10b, 10c and 10d attached in the present invention are installed respectively on the front, rear, left and right sides of the vehicle C, and the front, rear, left, Angle camera unit 10 as many as possible. The photographing of the outside of the vehicle C using the four super wide angle cameras 10a, 10b, 10c, and 10d in the present invention can detect and track a more accurate object for detecting an event, And judges whether or not a predefined event has been issued. Since the super wide angle cameras 10a, 10b, 10c and 10d provided on the front, rear, left and right sides of the vehicle C according to the present invention include super wide angle lenses having an angle of view of at least 120 degrees, There is a feature that can be photographed over the entire range.

The vehicle C further includes a sensor unit 20 for detecting vibrations or shocks caused by an external impact in addition to the ultra-wide angle camera unit 10. The detection sensor unit 20 detects a vibration or an impact caused by an external impact, and the event detection unit 50, which will be described later, determines that an event has occurred and stores the corresponding event information in the database unit 80. At this time, the event information to be stored includes the event occurrence time, the occurrence position of the vehicle in which the vibration is detected, the metadata of the object detected in the vibration detection, and the time from when the video or event recorded for a predetermined time before and after the event occurs Image frame information of a predetermined time, and the like.

On the other hand, the surround view image generating unit 30 matches the images input from the super-wide angle cameras 10a, 10b, 10c, and 10d installed in the front, rear, left, and right sides of the vehicle C in real- And the distortion-compensated image is corrected by the characteristics of the ultra-wide-angle lens to generate an image in the surround view mode. Usually, the images input from the respective super-wide angle cameras 10a, 10b, 10c, and 10d are photographed in a background of 120 degrees or more, but in the case of compositing into one image, overlapping areas are deleted according to each matching point set for the first time, Therefore, some frames of the image input from the super-wide angle cameras 10a, 10b, 10c, and 10d are deleted. The method of generating the surrounding view image included in the configuration of the present invention has been already known in the art, for example, various technologies have been proposed, and thus the detailed description thereof will be omitted herein.

At this time, if any one of the ultra-wide angle cameras 10a, 10b, 10c, and 10d installed in the front, rear, left, and right sides installed in the vehicle C is changed due to the impact of the vehicle, 10b, 10c, and 10d. In this case, the images input from the ultra-wide angle cameras 10a, 10b, 10c, and 10d whose imaging angles are changed and the ultra-wide angle cameras 10a, An incorrect image is synthesized at the time of image synthesis in the surround-view image generating unit 30 due to a matching point in synthesizing the images input from the images 10a, 10b, 10c, and 10d, There is a problem that can occur.

Therefore, the present invention is applicable to the super wide-angle cameras 10a, 10b, 10c, and 10d when the surround view image generating unit 30 synthesizes the images input from the super-wide angle cameras 10a, 10b, 10c, and 10d provided at the front, rear, left, 10b, 10c, and 10d include images input at different angles due to different positional changes from those at the time of initial installation, the image angles of the super-wide angle cameras 10a, 10b, 10c, and 10d are rotated at the same angle And an averaged view image adjuster 31 configured to allow the user to input an accurate one image by being input to the averaged view image generator 30. The averaged view image adjuster 31 adjusts the angle of view of each of the front, And the adjacent images among the images input from the super-wide angle cameras 10a, 10b, 10c, and 10d installed on the right side have many overlapping pixels. Therefore, The rotated or moved by the image adjustment section 31, the input image may be that there will not have problems with a composite image.

The image recording unit 40 continuously records the images generated from the surround view image generating unit 30 and stores them in the database unit 80. In a case where an image recorded and stored in the image recording unit 40 needs to delete an continuously stored surrounding view image exceeding the storage capacity of the database unit 80, the control unit 70 displays the image or image frame including the stored event information . At this time, the event information includes image frame information of a predetermined time before and after the time of occurrence of the event, the location of the vehicle, the metadata of the object, and the time at which the image or event occurred for a predetermined time before or after the event occurred. In the event information, the event occurrence time, the location of the vehicle where the event occurs, and the metadata are information stored by the user for searching for the event. Particularly, among the event information stored in the database unit 80, The image frame information of the predetermined time before and after the time when the recorded image or event occurs is stored in the storage unit of the database unit 80, Information corresponding to the frame.

The image sensing unit 50 includes an image sensing unit 50 for sensing and tracking an object moving in the surround view image generated by the surround view image generating unit 30. The image sensing unit 50 analyzes the surround view image generated by the surround view image generating unit 30 in real time to detect and track moving objects of interest such as a person or an automobile in the image. At this time, when a moving object is detected from the surrounding view image, a background model is periodically updated and then the foreground region is detected and noise motions are removed. Accordingly, the accuracy of the event determination can be increased by continuously detecting the detected motions and minimizing the sense of the object by detecting the object.

Based on the tracking result of the object extracted by the image sensing unit 50, the ebb detection unit 60 determines that an event has occurred when it corresponds to a predefined motion, and transmits event information about the object to the database unit 80. [ . At this time, the event information includes the time of the event, the location of the vehicle where the event occurred, the metadata of the object, and the video frame information of the predetermined time before and after the time when the video or event occurred for a predetermined time before and after the time the event occurred. In addition, the event detection unit 60 not only generates an event corresponding to a predefined movement in the surround view image, but also information input from the detection sensor unit 20 that detects vibrations or shocks caused by an external impact, It is possible to sense a vibration, an impact, and an image of the vehicle C as an event at the same time, thereby detecting a more accurate event. The predefined event is a specific movement of an object such as a parked vehicle on the left or right side of the vehicle C when the vehicle C is parked, or a person staying for a few seconds around the vehicle C. According to the present invention, there are various known technologies for detecting and tracking an object in an image and determining that the tracked object is an event, so that detailed description is omitted in this specification.

The control unit 70 stores the video image recorded by the video recording unit 40 and the event information determined as an event in the event sensing unit 60 in the database unit 80. The control unit 70 edits and stores the corresponding image or image frame including the event information determined as an event in the event sensing unit 60 among the continuous images recorded by the image recording unit 40, If the surround view image is continuously deleted and the storage capacity of the storage unit 80 is exceeded, it is necessary to control not to delete the image or image frame including the stored event information, So that an event occurring in the driver's vehicle C can be surely confirmed. In addition, the controller 70 may transmit the event occurrence information to the smart device 200 owned by the driver using the communication unit 100 capable of connecting to the wireless mobile communication network.

The database unit 80 stores event information determined as an event in the video image recorded by the video recording unit 40 and the event detection unit 60. The event information stored at this time includes an event occurrence time, The metadata of the object, and the video frame information of the predetermined time before and after the time when the video or the event recorded for a predetermined time before or after the occurrence of the event occurs.

If there is event information stored in the database unit 80, the display unit 90 displays the event occurrence information by thumbnail image or video summary playback of the event image, or displays a thumbnail image or video summary playback of the event image according to the vehicle location where the event occurred, Displays the occurrence information. At this time, the event occurrence information may be displayed as a thumbnail image of the event image or event occurrence information by video summary playback, or a thumbnail image of the event image by the vehicle position where the event occurs or an event occurrence information by video summary playback, You may.

3 is a flowchart illustrating a method of detecting an event of a black box for a vehicle during parking according to an embodiment of the present invention.

Referring to FIG. 3, the camera is photographed 360 ° in all directions by using super-wide angle cameras 10a, 10b, 10c, and 10d installed on the front, rear, left, and right sides of the dominant vehicle C (S100). The images photographed by the super-wide angle cameras 10a, 10b, 10c, and 10d are synthesized by the surround-view image generator 30 and the distorted images are corrected to generate an overview-view image (S110). The surround view image generated by the surround view image generating unit 30 is recorded as a continuous image by the image recording unit 40 and stored in the database unit 80 in real time (S120). At this time, the event image or the image frame including the event information determined as the event is edited and stored separately from the continuous images stored in the event detection unit 60 to delete the continuously stored stored view image in excess of the storage capacity of the database unit The video or image frame including the stored event information is not deleted.

Also, the image sensing unit 50 detects and tracks the moving object in the acquired surround view image generated and acquired by the surrounding view image generating unit 30 (S130). The event detection unit 60 determines that an event has occurred when the detected object corresponds to a predefined movement or vibration or shock information from the vehicle C is sensed by the sensing unit at step S140, The corresponding event information is stored in the database unit 80 (S150). The event information stored at this time includes the event occurrence time, the vehicle occurrence location, the metadata of the object, and the image frame information of the predetermined time before and after the time when the event or the image recorded for the predetermined time before or after the event occurred. In the event information, the event occurrence time, the location of the vehicle where the event occurs, and the metadata are information stored by the user for searching for the event. Particularly, among the event information stored in the database unit 80, The image frame information of the predetermined time before and after the time when the recorded image or event occurs is stored in the storage unit of the database unit 80, And information corresponding to a video or an image frame in the frame.

When the driver starts the vehicle C, the control unit 70 displays the event occurrence information on the display unit 90 at the same time when the vehicle starts, if the event information stored in the database unit exists (S160). At this time, the event occurrence information displayed on the display unit 90 may be displayed as a thumbnail image of the event image, a video summary playback, a thumbnail image of the event image according to the vehicle position, or a video summary playback. In addition, a thumbnail image / video summary list of the event image or a thumbnail image / video summary list of the event image according to the vehicle location can be displayed on the screen.

In addition, the event generation information displayed by the thumbnail image or the video summary playback of the event image, the thumbnail image of the event image by the vehicle location, or the video summary playback is confirmed by the display unit 90 having the touch screen function and the event information is deleted The storage can be maintained (S170). Also, the event information that has not been deleted is displayed on the display unit 90 at the next start-up, so that the driver can always check events per line.

It will be apparent to those skilled in the art that various modifications and equivalent arrangements may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. It is to be understood, therefore, that this invention is not to be limited to the specific forms disclosed in the foregoing description. Accordingly, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents. Should be understood to include.

10: ultra-wide angle camera unit 20: detection sensor unit
30: surrounding view image generating unit 31:
40: image recording unit 50: image sensing unit
60: an event detection unit 70:
80: Database part 90: Display part
100: communication unit 200: smart device

Claims (8)

An ultra-wide-angle camera unit installed at the front, rear, left, and right sides of the vehicle and capable of taking images in front, rear, left, and right directions outside the vehicle;
A sensing sensor unit for sensing vibrations or shocks caused by external impacts of the vehicle;
An averaged view image generation unit that synthesizes and then corrects the images input from the ultra-wide angle camera unit installed at the front, rear, left, and right sides of the vehicle to generate an image in an surround view manner;
An image recording unit for recording the image generated in the surround view image generating unit;
An image sensing unit for detecting and tracking an object moving in the image generated by the corrected surround view method;
An event detection unit for determining that the object detected by the image sensing unit corresponds to a predefined movement or that an event is detected by the sensor unit;
A control unit for storing the video image recorded by the video recording unit and the event information determined as an event in the event detection unit in the database unit and displaying the event occurrence information on the display unit when the event information is stored in the database unit at the same time as the vehicle starts;
A database unit for storing the video image recorded by the video recording unit and the event information determined as an event in the event detection unit; And
A display unit for displaying event occurrence information in a thumbnail image or video summary playback of the event image when there is event information stored in the database unit or displaying a thumbnail image of the event image or a video summary playback of the event image for each vehicle location in which the event occurs, Wherein the vehicle-mounted black-box event detection system comprises:
The method according to claim 1,
When the surround view image generating unit synthesizes the images input from the super wide angle cameras installed at the front, rear, left, and right sides, when any one of the super wide angle cameras includes different angled input images due to the positional change different from that at the initial installation, Further comprising an averaged-view image adjustment unit that rotates or moves the image of the camera at the same angle as that of the initial installation, and inputs the rotated image to the averaged-view image generation unit.
The method according to claim 1,
Wherein the control unit edits and separately stores an event image or an image frame including event information determined as an event in the event sensing unit among the continuous images recorded by the image recording unit, Wherein the control unit controls to prevent the image or image frame including the stored event information from being deleted when the image is to be deleted.
The method according to claim 1,
The event information includes image frame information of a predetermined time before and after the time of occurrence of the event, the location of the vehicle where the event occurred, the metadata of the object, The event detection system of the car black box at the time of parking.
Photographing a 360-degree omnidirectional vehicle outside the vehicle using an ultra-wide angle camera installed at the front, rear, left, and right sides of the vehicle;
Synthesizing images photographed by the respective ultra-wide angle cameras and correcting the distorted images to generate an overview image;
Continuously recording the generated surround view image and storing the same in a database unit;
Detecting and tracing a moving object in the generated surrounding view image;
Determining that an event has occurred when the detected and tracked object corresponds to a predefined motion or vibration or shock information is sensed by the sensor unit;
Storing event information in a database unit when the event is detected;
Displaying event occurrence information on a display unit at the same time when a vehicle starts, if event information stored in the database unit exists; And
The method of claim 1, further comprising the step of, after confirming the displayed event occurrence information, deleting or storing the corresponding event occurrence information.
The method of claim 5,
An event image or an image frame including event information determined as an event by the event detection unit is edited and stored separately from the recorded continuous images, and if the surround view image stored continuously in excess of the storage capacity of the database unit is to be deleted, Wherein the control unit controls to prevent an image or an image frame including event information from being deleted.
The method of claim 5,
The event information includes image frame information of a predetermined time before and after the time of occurrence of the event, the location of the vehicle where the event occurred, the metadata of the object, A method for detecting an event of a black box in a vehicle.
The method of claim 5,
Wherein the event occurrence information displayed on the display unit is displayed as a thumbnail image of the event image, a video summary playback, a thumbnail image of the event image according to the vehicle location, or a video summary playback.

Images