JP6582184B2 - Display control system, display control method, and program - Google Patents

Description

  The present invention relates to a display control system, a display control method, and a program for performing control to display a thumbnail image related to an event unit video recorded by a drive recorder mounted on a vehicle.

  Conventionally, an on-vehicle video that captures a video around the vehicle with a camera installed in the vehicle, and records the peripheral video and vehicle speed when the vehicle is impacted by an event such as sudden braking, sudden steering, or collision Equal recording devices, so-called drive recorders, are generally popular. By providing such a drive recorder in the vehicle, it is possible to verify the cause of the accident by analyzing the recorded information when an accident occurs. In addition, the driver's awareness of safe driving can be improved, and a video recording the daily driving situation can be used for safety driving guidance and the like.

  Patent Documents 1 and 2 collect operation information such as camera images, vehicle speed, and GPS (Global Positioning System) position information recorded by drive recorders mounted on commercial vehicles such as trucks and taxis. Discloses a driving support device that checks the contents of driving information, analyzes how each crew member operates, and uses it for driving training for crew members.

JP 2010-255743 A JP 2010-257484 A

  In the above-mentioned driving support device, the operation manager or the like suddenly brakes from a lot of driving information (hereinafter referred to as event unit video) recorded for each event (event) such as sudden braking, sudden steering or collision. In order to narrow down and select the event unit video of a specific event such as, for example, it is performed using related data such as the magnitude and waveform of acceleration recorded in each event unit video. That is, this related data is recognized as a tag for narrowing down the event unit video of a specific event, and the event unit video is classified and narrowed down. However, in order to identify the event unit video of the event that is ultimately required, the operation manager etc. must check the contents of the camera images included in each event unit video in sequence to identify the event unit video. I must.

  In addition, in the driving support devices disclosed in Patent Documents 1 and 2, a timing signal indicating that a trigger (for example, a recording condition by a drive recorder is satisfied) among a plurality of camera images included in one event unit video. ) Is displayed, a thumbnail image obtained by reducing the camera image recorded at the time of detection is detected. This thumbnail image is, for example, a trigger detection after the time when an event such as sudden braking or sudden steering occurs. Since it is a camera image recorded at the time, it is difficult to find an event unit video of the desired event from this thumbnail image.

  The present invention has been made in view of the above-described circumstances, and an object thereof is to provide a display control system, a display control method, and a program that can easily find a desired event unit video from a thumbnail image. To do.

  In a first aspect, the present invention is a display control system that performs control to display a thumbnail image related to an event unit video recorded by a drive recorder mounted on a vehicle, and includes a range of the event unit video. There is provided a display control system comprising display control means for displaying a plurality of thumbnail images corresponding to different time points.

  In the display control system according to the first aspect, the contents of the event unit video can be confirmed from a plurality of thumbnail images corresponding to different time points in the range of the event unit video. Thereby, compared with the case where the content of the event unit video is confirmed from one thumbnail image, the desired event unit video can be easily found from the thumbnail image.

The event may be an event specified by a predetermined rule, for example. For example, the event unit may be a time range of a unit in which a certain event and another event can be managed separately. For example, it is good to set it as the predetermined period before and behind the occurrence time of an event like the structure of the 2nd viewpoint. For example, the time range of a certain event unit and another event unit may be set so as not to overlap each other, or may be set so that, for example, a part thereof overlaps.
The event unit video may be, for example, a video in a time range of a unit that allows a certain event and another event to be managed separately. For example, the event unit video may be recorded as a separate video for each event. For example, a certain range in one video may be used as the event unit video. In other words, “event unit video recorded by a drive recorder mounted on a vehicle” does not necessarily require video to be recorded in event units.

When a range in one video is set as an event unit video, for example, the range of a plurality of event unit videos may be set so as to allow overlapping of the respective ranges, or each event unit may be set in a range that does not overlap. You may make it set the range of an image | video. For example, the event unit video may be a range that may be related to the event in the video.
In addition, “a plurality of thumbnail images corresponding to different time points in the event unit video range” is, for example, “a plurality of thumbnail images corresponding to videos other than the event unit video at different time points in the event unit video range”. It may be “a plurality of thumbnail images corresponding to the event unit video at different points in the range of the event unit video”. As a specific example of “corresponding” in “a plurality of thumbnail images corresponding to event-unit videos”, “represent” may be used. In addition, “created from”, “image processed”, “created by image processing”, “created by performing predetermined image processing”, and the like may be used.

  In a second aspect, the present invention provides the display control system according to the first aspect, wherein the event unit video is a video over a predetermined period before and after the occurrence of an event, and the thumbnail image It is characterized by comprising images recorded either before or after the occurrence time, or at both times.

  In the display control system according to the second aspect, the contents of the event unit video can be confirmed from the thumbnail images corresponding to the images recorded before or after the event occurrence time, or at both times. Become. As a result, it is possible to more easily find the event unit video related to the events at the time points before and after the event occurrence time point.

  For example, the event occurrence time may be a time when a predetermined condition is satisfied. For example, the event occurrence time may be a time when a predetermined condition representing the event is satisfied. For example, the range of the event unit video may be determined according to the type of the event, with the time when the predetermined condition is satisfied as the time of occurrence of the event. The event may be, for example, a time point when an event corresponding to the occurrence of an accident has occurred. For example, a location where a sudden deceleration corresponding to a collision occurs may be set as the event occurrence time.

  In the drive recorder, for example, the type that records the video before and after the “trigger indicating that the recording condition has been established”, and the video is always recorded (for example, always from engine on to off), and the trigger is applied. There is a type that stores information for enabling the position of an image to be specified. This “information for enabling identification” is, for example, time information when the trigger is applied. For example, which position of which video is triggered is specified from the video recording start time information and the time information when this trigger is triggered.

In the case of the former type of drive recorder, the “event unit video” is, for example, an image recorded over a predetermined period before and after the occurrence of a trigger indicating that the recording condition by the drive recorder is satisfied. Good.
In the case of the latter type of drive recorder, the “event unit video” is, for example, the occurrence of a trigger in a video that is a recording unit (in the above example, one unit from engine on to off). It may be a video of a portion recorded over a predetermined period before and after the time.

  The predetermined period may be, for example, the entire range of the recorded video (for example, the entire range from engine on to off), but is particularly preferably a limited time range in the video. The limited time range in the video is particularly preferably a time range including the time of occurrence of the event. Note that the limited time range in the video may be, for example, a fixed time range or a variable time. Further, the period before the event occurrence time and the period after the event occurrence time constituting the predetermined period may be the same time period, but may be different time periods. In particular, the predetermined period may be a period sufficient to make it possible to specify the cause of the occurrence of the event. In particular, the predetermined period may be different depending on the type of event.

  According to a third aspect, in the display control system according to the second aspect, the present invention determines that the event has occurred as the thumbnail image at a time that is not the time of occurrence of the event within the predetermined period. An image recorded at a time point determined by a predetermined rule using the information used for performing the processing is provided.

  In the display control system according to the third aspect, information (for example, acceleration applied to the vehicle) used to determine that an event has occurred at a time that is not the time of occurrence of the event within a predetermined period is used. The content of the event unit video can be confirmed from the thumbnail image corresponding to the image recorded at the time point determined by the predetermined rule (for example, when the acceleration exceeds the predetermined value). This makes it easy to find an event unit video related to the event at the time point determined by the predetermined rule.

  In a fourth aspect, the present invention is used in the display control system according to the second or third aspect to determine that the event has occurred within the predetermined period as the thumbnail image. An image recorded at a time point determined by a predetermined rule using information different from the information is provided.

  In the display control system according to the fourth aspect, the information (for example, sudden acceleration in the left-right direction of the vehicle by the sudden handle) is different from the information (for example, acceleration in the lateral direction of the vehicle by the sudden handle) within the predetermined period. From a thumbnail image corresponding to an image recorded at a time point determined by a predetermined rule using the brake acceleration in the front-rear direction of the vehicle (for example, a maximum value point in time within a predetermined period of acceleration in the front-rear direction of the vehicle) The content of the event unit video can be confirmed. This makes it easy to find an event unit video related to the event at the time point determined by the predetermined rule.

  In a fifth aspect, the present invention provides the display control system according to the third or fourth aspect, wherein the predetermined rule is a rule for estimating the occurrence time of a sub-event that has caused the occurrence of the event. It is characterized by being.

In the display control system according to the fifth aspect, it is possible to easily find the event unit video related to the event at the time determined by the rule for estimating the time of occurrence of the sub-event that caused the event.
The predetermined rule is a rule for estimating a feature point related to the occurrence of the event retroactively from the time when it is determined that the event has occurred. For example, a maximum point or an inflection point within a predetermined period of vehicle acceleration, etc. It may be a feature point of acceleration. Also, different rules may be used depending on the type of event.
The predetermined period may be set with sufficient time to be considered that the possibility of occurrence of a sub-event that causes an event is low.

  In a sixth aspect, the present invention provides the display control system according to any one of the second to fifth aspects, wherein the time point before or after the occurrence time of the event, or both, is the event. It is characterized in that it is a different point in time depending on the type.

  In the display control system according to the sixth aspect, from the thumbnail image corresponding to the image recorded at a different time point depending on the type of the event as the time point before or after the event occurrence time or both, the event unit You can check the contents of the video. As a result, it is possible to easily find an event unit video related to an event at a different time depending on the type of event.

  In a seventh aspect, the present invention provides the display control system according to any one of the second to sixth aspects, wherein the thumbnail image is either immediately before or immediately after the occurrence of the event, or both It is characterized by comprising an image recorded in

  In the display control system according to the seventh aspect, the contents of the event unit video can be confirmed from thumbnail images corresponding to images recorded immediately before or after the event occurrence time, or at both times. . This makes it possible to more easily and accurately find the event unit video related to the event immediately before and immediately after the event occurrence time.

  In an eighth aspect, the present invention is the display control system according to any one of the second to seventh aspects, wherein the thumbnail images are recorded at both time points immediately before and immediately after the event occurrence time point. An information acquisition interval that is provided with an image and used to determine that the event has occurred is set to be shorter than the recording interval of the video frame, and is recorded at a time immediately before the event occurrence time. The recorded image is an image of a frame immediately before the time when it is determined that the event has occurred, and an image recorded at a time immediately after the time when the event has occurred is an image immediately after the time when it is determined that the event has occurred. It is a frame image.

  In the display control system according to the eighth aspect, the contents of the event unit video can be confirmed from the thumbnail images corresponding to the images recorded at both the time immediately before and after the event occurrence time. This makes it possible to more easily and accurately find the event unit video related to the events at both the time immediately before and after the event occurrence time.

  In a ninth aspect, the present invention provides the display control system according to any one of the first to eighth aspects, wherein the thumbnail image is either the first or last of the event unit videos, or both. An image recorded at the time is provided.

  In the display control system according to the ninth aspect, the contents of the event unit video can be confirmed from the thumbnail images corresponding to the images recorded at the first or last time in the event unit video, or both. Become. Thereby, the event unit video related to the event at the first and last time of the event unit video can be found more easily and more accurately.

  In a tenth aspect, the present invention is the display control system according to any one of the second to ninth aspects, wherein the thumbnail image includes an image recorded at the time of occurrence of the event. .

  In the display control system according to the tenth aspect, the content of the event unit video can be confirmed from the thumbnail image corresponding to the image recorded at the time of occurrence of the event. As a result, the event unit video related to the event at the time of occurrence of the event can be found more easily and more accurately.

  In an eleventh aspect, the present invention provides the display control system according to any one of the first to tenth aspects, according to the state of the vehicle when the event unit video is recorded as the thumbnail image. And an image of a specific period in the range of the event unit video.

  In the display control system according to the eleventh aspect, the content of the event unit video can be confirmed from the thumbnail image corresponding to the image of the specific period according to the state of the vehicle when the event unit video is recorded. Thereby, it is possible to more easily find the event unit video related to the state of the vehicle when the event unit video is recorded.

“Vehicle state when event unit video is recorded” means, for example, (1) sudden braking, (2) sudden handle, (3) vehicle state such as when the vehicle collides with something is there.
“Image of a specific period selected from the event unit video” means, for example, (1) In the case of sudden braking, an image recorded at the time from when the brake is stepped on until it is released, or acceleration applied to the vehicle This is an image recorded from the time when the event appears until the end of the event. (2) In the case of a sudden handle, it is an image or the like recorded at the time from when the handle is started to when it is turned. Furthermore, (3) When the vehicle collides with something, images recorded at the time from when the acceleration behavior of the vehicle due to the reaction of the bumper, guardrail, etc. was born until the acceleration behavior converged, etc. is there.

  In a twelfth aspect, the present invention provides the display control system according to any one of the first to eleventh aspects, wherein the thumbnail image is an image recorded when acceleration applied to the vehicle exceeds a predetermined value. It is characterized by providing.

  In the display control system according to the twelfth aspect, the content of the event unit video can be confirmed from the thumbnail image corresponding to the image recorded when the acceleration applied to the vehicle exceeds a predetermined value. Thereby, it is possible to more easily find the event unit video related to the time point when the acceleration applied to the vehicle exceeds a predetermined value.

  In a thirteenth aspect, the present invention provides the display control system according to any one of the second to twelfth aspects, wherein the occurrence of the event is specific behavior information based on information obtained by detecting the behavior of the vehicle. It is characterized by that.

  In the display control system according to the thirteenth aspect, the content of the event unit video can be confirmed from the thumbnail image corresponding to the image in which the specific behavior information based on the information detecting the behavior of the vehicle is recorded as an event. This makes it easier to find an event unit video in which specific behavior information is recorded as an event.

  “Vehicle behavior detection information” refers to, for example, the detection result of an acceleration sensor, gyro sensor, or pressure sensor (or altitude sensor) mounted on the vehicle, or the analysis result of recorded video. Information obtained.

  In a fourteenth aspect, the present invention provides the display control system according to any one of the second to thirteenth aspects, wherein the occurrence of the event is specific vehicle information based on information related to a functional state of the vehicle. It is characterized by.

  In the display control system according to the fourteenth aspect, the content of the event unit video can be confirmed from the thumbnail image corresponding to the image in which the specific vehicle information based on the information related to the functional state of the vehicle is recorded as an event. Thereby, it is possible to more easily find an event unit video in which specific vehicle information is recorded as an event.

  “Vehicle functional state information” refers to, for example, operation of any of brakes, turn signals, accelerators, steering wheel rotations, shift lever positions, and side brakes during vehicle operation, and OBD (On Board Diagnosis), information obtained from a CAN (Controller Area Network) which is an in-vehicle LAN.

  In a fifteenth aspect, the present invention provides the display control system according to any one of the first to fourteenth aspects, wherein the plurality of thumbnail images form different rows of thumbnail images of different event unit videos, and The thumbnail images at the same time point are displayed in a list on the same screen by a matrix structure that forms different columns.

  In the display control system according to the fifteenth aspect, a plurality of thumbnail images are displayed in a list in a visually easy-to-understand manner in a state where thumbnail images for different event unit videos and thumbnail images at the same time are arranged. Thereby, it is possible to easily determine the contents of the thumbnail image.

  According to a sixteenth aspect, in the display control system according to any one of the first to fifteenth aspects, the present invention provides the plurality of thumbnail images in which the thumbnail images for each same event unit video are in the same line from left to right. Alternatively, it is characterized in that a list is displayed on the same screen by a matrix structure configured to be arranged in chronological order from right to left.

  In the display control system according to the sixteenth aspect, a plurality of thumbnail images are displayed as a list in a visually easy-to-understand manner as thumbnail images arranged in the same row in the same row for each same event unit video. Thereby, it is possible to easily determine the contents of the thumbnail image.

  It can be regarded as a method invention having steps corresponding to the constituent elements of the display control means according to any one of the first to sixteenth aspects. For example, as an invention of a method corresponding to the first aspect, in a seventeenth aspect, the present invention provides a display for performing control to display a thumbnail image related to an event unit video recorded by a drive recorder mounted on a vehicle. A display control method comprising a step of displaying a plurality of thumbnail images corresponding to different time points in the range of the event unit video.

  In the display control method according to the seventeenth aspect, the contents of the event unit video can be confirmed from a plurality of thumbnail images corresponding to different time points in the range of the event unit video. Easy to find from.

  In an eighteenth aspect, the present invention can provide a program for causing a computer to realize the function as the display control system according to any one of the first to sixteenth aspects.

  According to the display control system of the present invention, it is possible to easily find a desired event unit video from a thumbnail image.

1 is a diagram illustrating a schematic configuration of a driving support system according to a first embodiment. 3 is a diagram conceptually showing a data structure of contents recorded in a memory card 25. FIG. It is a figure explaining the mode of the acceleration change at the time of the event occurrence which concerns on Example 1, and the storage method of an image. 6 is a flowchart illustrating thumbnail image creation and display processing according to the first embodiment. 6 is a screen display diagram illustrating an example of a display mode of thumbnail images according to Embodiment 1. FIG. It is a figure which shows the acceleration change in the case of the sudden brake which concerns on Example 2. FIG. 12 is a flowchart illustrating thumbnail image creation and display processing according to the second embodiment. It is a figure which shows the acceleration change in the case of the sudden handle which concerns on Example 3. FIG. 14 is a flowchart illustrating thumbnail image creation and display processing according to the third embodiment. It is a figure which shows the acceleration change in the case of the vehicle collision which concerns on Example 4. FIG. 14 is a flowchart illustrating thumbnail image creation and display processing according to the fourth embodiment. It is a figure which shows the experiment example of the acceleration change in the case of a vehicle-to-vehicle collision. It is a figure which shows the display example of the thumbnail screen which concerns on the experiment example shown in FIG.

  Hereinafter, a display control system according to an embodiment of the present invention will be described in detail with reference to the drawings. It should be noted that the present invention is not construed as being limited thereto, and various changes, modifications, and improvements can be made based on the knowledge of those skilled in the art without departing from the scope of the present invention.

[Example 1]
<Configuration of driving support system>
FIG. 1 is a diagram showing a schematic configuration of a driving support system according to Embodiment 1 of a display control system of the present invention.

  As shown in FIG. 1, the driving support system reproduces an image stored in the drive recorder 10 mounted on the vehicle, a memory card 25 that stores an image when an event occurs in the drive recorder 10, and the memory card 25. And a driving support device 30.

  The drive recorder 10 includes a first control unit 11, an acceleration sensor (G sensor) 12, a gyro sensor 13, an atmospheric pressure sensor 14, a GPS antenna 15, and a GPS that measures GPS position information of a vehicle from a GPS signal received by the GPS antenna 15. (Global Positioning System) A first operation unit 17 including a receiver 16, a switch, a button, etc., a first memory 18 for temporarily storing images, a slot for inserting a memory card 25, and a memory card 25 A first IF (interface) 19 for writing an image to the vehicle, a CCD camera 20 installed in the vehicle so as to continuously image the front of the vehicle at a predetermined rate, a vehicle signal acquisition unit 21, and a drive recorder The power supply unit 22 for supplying power to each of the ten elements is included.

  Further, the drive recorder 10 may be configured to be able to reproduce an image on a display of the navigation device in cooperation with a navigation device (not shown). Further, the GPS receiver 16 is not necessarily an essential configuration, but may be configured to acquire position information received by an external device such as an AVM (Automatic Vehicle Monitoring) system or a navigation device.

  The first control unit 11 includes a microcomputer including a CPU, a ROM, a RAM, and the like, and controls the entire operation of the drive recorder 10.

  The G sensor 12 detects gravity acceleration (hereinafter simply referred to as acceleration) in two axial directions (X axis and Y axis) orthogonal to each other, and the detected acceleration (also referred to as G data) together with time data is a first control unit. 11 to send. The first control unit 11 can acquire G data (Gx, Gy) output from the G sensor 12 at intervals of, for example, 10 milliseconds, and determine the acceleration value (magnitude) from the G data.

  Note that Gx represents acceleration in the X-axis direction (front-rear direction of the vehicle), and Gy represents acceleration in the Y-axis direction (left-right direction of the vehicle). Further, the G sensor 12 may use a three-axis sensor including the Z-axis direction (the vertical direction of the vehicle), and the three-axis sensor does not use the acceleration Gz data in the Z-axis direction. Also good.

  The gyro sensor 13 detects the angular acceleration (rotation) of the vehicle, mainly detects the change in the traveling direction of the vehicle (that is, the rotation of the yaw (Z axis) that is the vertical axis), and detects the detected angular acceleration data. Is transmitted to the first control unit 11 together with the time data. The first control unit 11 can acquire angular acceleration data output from the gyro sensor 13 at intervals of, for example, 10 milliseconds, and determine the value (magnitude) of angular acceleration from the angular acceleration data.

  The atmospheric pressure sensor 14 detects atmospheric pressure outside the vehicle, calculates the altitude based on the detected atmospheric pressure value, and transmits the atmospheric pressure value and altitude value (atmospheric pressure / altitude data) to the first control unit 11 together with time data. . For example, the first control unit 11 can acquire the atmospheric pressure / altitude data output from the atmospheric pressure sensor 14 at intervals of 10 milliseconds, and determine the value (size) of the atmospheric pressure / altitude from the atmospheric pressure / altitude data.

  In the first control unit 11, the CCD camera 20 captures one image every 100 milliseconds, and sequentially stores the captured image together with time data in the first memory 18 (to a memory card 25 described later). Control to perform primary storage) For example, the first memory 18 is configured to always store image data for 24 seconds (240 images) captured by the CCD camera 20. The image captured by the CCD camera 20 is not limited to one image every 100 milliseconds, but may be one image every 50 milliseconds, for example. Moreover, you may comprise so that the moving image image | photographed with the CCD camera 20 may be memorize | stored.

  The vehicle signal acquisition unit 21 is connected to a vehicle failure diagnosis connector, acquires various signals and information (vehicle data) of the vehicle at intervals of, for example, 100 milliseconds, and performs first control on the acquired vehicle data together with time data. To the unit 11. Vehicle data includes, for example, [1] a signal that detects a brake operation while driving the vehicle, [2] a signal that also detects a blinker operation, [3] a signal that also detects an accelerator operation, and [4] a steering wheel rotation. The detected signal, [5] the same signal that detected the shift lever position, [6] the same signal that detected the side brake operation, and [7] the vehicle fault diagnosis information (OBD) obtained from the vehicle fault diagnosis system (OBD) in the vehicle ( Vehicle speed, engine speed, cooling water temperature, etc.] [8] Various vehicle information obtained via in-vehicle LAN (CAN) (functional control data from the vehicle's ECU (Electronic Control Unit), network outside the vehicle, etc. Communication data from other vehicles).

  As an example, when the vehicle signal acquisition unit 21 acquires a signal for detecting a brake operation, the brake operation data indicating whether or not the vehicle brake is operated, and the brake operation start time data indicating the time when the brake operation is started. And the brake operation end time data indicating the time when the brake operation is ended is transmitted to the first control unit 11. That is, according to the brake operation data, the brake operation start time data, and the brake operation end time data from the vehicle signal acquisition unit 21, the first control unit 11 determines whether the brake operation is performed, the time when the brake operation is started, and The time when the brake operation is finished can be determined.

  As another example, when the vehicle signal acquisition unit 21 acquires a signal that detects steering wheel rotation, steering wheel rotation data that indicates whether or not the steering wheel of the vehicle has been rotated, and steering wheel rotation start that indicates the time at which steering wheel rotation has started. The time data and the handle rotation end time data indicating the time when the handle rotation is ended are transmitted to the first control unit 11. That is, according to the steering wheel rotation data, the steering wheel rotation start time data, and the steering wheel rotation end time data from the vehicle signal acquisition unit 21, the first control unit 11 determines whether the steering wheel rotation is performed, the steering wheel rotation start time, and The time when the steering wheel rotation is completed can be determined.

  Note that some or all of the GPS antenna 15, the GPS receiver 16, the first IF 19, the CCD camera 20, the vehicle signal acquisition unit 21, and the like may be configured separately from the drive recorder 10. For example, when linked with an existing navigation device having a GPS function, the drive recorder 10 does not require a function such as the GPS receiver 16, so that the cost of the drive recorder 10 can be reduced by that amount.

  The driving support device 30 includes a second control unit 31, a display unit 32 configured by a display, an output unit 33 configured by a printer that prints a display screen on a paper surface, and input devices such as a keyboard and a mouse. A second operation unit 34, a second memory 35 for temporarily storing an image, a slot for inserting the memory card 25, and a second IF (interface) 36 for reading an image from the memory card 25, etc. And an image processing unit 37 for performing predetermined image processing on the input image.

  The second control unit 31 includes a CPU, a ROM, a RAM, and the like, and is a personal computer (PC) on which driving support software is installed. The second control unit 31 interprets and executes a program included in the driving support software and drives the driving support device 30. The various components according to the present invention are realized by operating each of the components.

  The driving support device 30 is an aspect of a display control system that performs control to display thumbnail images related to the event unit video recorded by the drive recorder 10. In this embodiment, the event unit video is an image recorded in a predetermined period before and after the event occurrence time. The event occurrence time is, for example, the time when an event corresponding to the occurrence of an accident is occurring, for example, the time when an impact is applied to the vehicle due to sudden deceleration such as sudden braking.

  The memory card 25 is a rewritable nonvolatile semiconductor memory card having a storage capacity of 2 Gbytes. The memory card 25 is inserted into a slot constituting the first IF 19 and is stored in a predetermined image (for example, an event Images for 12 seconds before and 12 seconds after the occurrence) are stored. Thereafter, the memory card 25 is ejected from the slot constituting the first IF 19 and reinserted into the slot constituting the second IF 36, and the stored image is output to the driving support device 30.

  Although a semiconductor memory card is used as the memory card 25, the present invention is not limited thereto, and a semiconductor memory other than the card type, a portable hard disk (HD), other storage media, and the like can be used. Further, the storage capacity of the memory card or the like is arbitrary and can be freely determined according to the application.

  When a predetermined condition is satisfied (that is, when an event to be recorded by the drive recorder 10 occurs), the drive recorder 10 specifically includes (a) a sudden brake, a sudden handle, a collision, etc. When an impact is applied to the vehicle due to the occurrence of an event corresponding to the occurrence of another accident, and the value (size) of the G data output from the G sensor 12 provided therein exceeds a predetermined threshold Others (b) Various data of several seconds to several tens of seconds (for example, 12 seconds before and 12 seconds after) triggered by the detection when the emergency switch of the first operation unit 17 is pressed, Acquired together with the event occurrence time data, and records it in the memory card 25 as one piece of driving information.

  The driving information recorded in the memory card 25 includes G data output from the G sensor 12, angular acceleration data output from the gyro sensor 13, atmospheric pressure / altitude data output from the atmospheric pressure sensor 14, and positioning by the GPS receiver 16. GPS position information, camera image data captured by the CCD camera 20, various signals and information of the vehicle acquired by the vehicle signal acquisition unit 21 (detection data such as brake operation and steering wheel rotation, vehicle speed data, etc.).

  The drive recorder 10 supports a plurality of camera images one by one for several seconds to several tens of seconds before and after the trigger is detected after an event to be recorded occurs (for example, 12 seconds before and 12 seconds after). In addition, various data of driving information is recorded in the memory card 25.

<Data structure of memory card>
Next, the data structure of the contents recorded on the memory card 25 will be described with reference to FIG. FIG. 2 is a diagram conceptually showing the data structure of the contents recorded on the memory card 25.

  As shown in FIG. 2, in the memory card 25, corresponding driving information is divided and recorded as driving information 1, driving information 2, and driving information 3 for each event occurrence. The driving information 1 is obtained from 240 camera images 1 to 240 corresponding to 24 seconds before and after the event occurrence (12 seconds before and 12 seconds after) and image related information including other driving information other than the camera images. Composed. The image related information includes G data at the time of event occurrence, main item data including time data thereof, and related data 1 to 240 corresponding to 240 camera images 1 to 240, respectively. The same applies to the driving information 2 and 3.

  Main item data includes G data, angular acceleration data, barometric pressure / altitude data, and time data. Image-related data includes the date and time of shooting of the camera image, and various signals and information of the vehicle at the time of shooting the camera image. (Detection data such as brake operation and steering wheel rotation, vehicle speed data, etc.) and vehicle GPS position information are included.

<Acceleration change at the time of event occurrence and image storage method>
Next, referring to FIG. 3, when an event such as an impact is applied to the vehicle, the change in acceleration (G data) detected by the G sensor 12 (see FIG. 1) of the drive recorder 10; A method for storing images and the like in the drive recorder 10 will be described.

  3A shows a time-series change of G data (combined value of Gx and Gy) output from the G sensor 12, and FIG. 3B shows an example of an image period stored in the memory card 25. FIG. 3C shows an example of the selection timing of the image that is the source of the created thumbnail image. The creation and display of thumbnail images will be described later.

  The value (size) of the G data output from the G sensor 12 suddenly increases from almost zero from when the vehicle is impacted. As shown in FIG. 3A, when the value (magnitude) of the G data of the G sensor 12 reaches a threshold acceleration Gs (for example, 0.3 G) at time tp (time tp), The first control unit 101 recognizes this as a trigger, determines (determines) that an event has occurred, and the CCD camera 20 captures an image that is always temporarily stored in the first memory 18 in a cyclic manner. And the time tp is stored in the memory card 25 as event occurrence determination time data.

  Specifically, as shown in FIG. 3B, control is performed so that the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp are stored in the memory card 25. For example, TA can be 12 seconds and TB can be 12 seconds. It should be noted that the TA and TB periods can be set freely. For example, while displaying the set time (seconds) on the setting screen (not shown) of the drive recorder 10, the time is set by operating the time setting button of the first operation unit 17 of the drive recorder 10, and the set information By saving, you can set any time.

  When an event such as an impact is applied to the vehicle as described above and the value (size) of the G data detected by the G sensor 12 exceeds a predetermined threshold, the first control unit 11 is provided for each event. Is the event occurrence time data (tp), the detected G data (Gx, Gy) of the G sensor 12 and its time data, the detected angular acceleration data of the gyro sensor 13 and its time data, the atmospheric pressure / altitude data of the atmospheric pressure sensor 14 and its data Time data, vehicle data for TA seconds before the time tp acquired by the vehicle signal acquisition unit 21 and TB seconds for the time tp after the time tp, and time data thereof (for example, brake operation data and its time data, steering wheel rotation data and its Time data, vehicle speed data and time data, etc.) are stored in the memory card 25 together with the image data. Note that only a part of the data described above may be stored in the memory card 25.

  The number of events stored in the memory card 25 can be varied according to the storage capacity of the memory card 25. For example, it is preferable that data for 500 events or more can be stored. In addition, the first control unit 11 creates a folder in the memory card 25 for each event, image data, event occurrence time data (tp), G data (Gx, Gy) of the G sensor 12 and time data, corners thereof. It is preferable to store acceleration data and its time data, atmospheric pressure / altitude data and its time data, vehicle data and its time data.

  Note that, when the above-described event occurs, exceeding the threshold acceleration Gs is set as the event generation trigger. For example, when the amount of change in acceleration per unit time exceeds a certain value (in 0.1 seconds, G It may be determined that an event has occurred when the data value (size) changes by 0.1 G or more.

<Creation and display of thumbnail image (Example 1)>
Next, thumbnail image creation and display processing in the driving support device 30 will be described with reference to FIG. In this example, a thumbnail image creation and display processing operation related to one event will be described.
FIG. 4 is a flowchart illustrating thumbnail image creation and display processing according to the first embodiment. This thumbnail image creation and display processing is loaded from the HDD or the like into the RAM of the second control unit 31 of the driving support device 30. This is realized by the CPU executing in accordance with the control program, and is started based on the operation of starting the driving support software of the second operation unit 34. Further, it is assumed that when the processing of the flowchart shown in FIG. 4 is started, the memory card 25 in which the image is stored in the drive recorder 10 is inserted into the second IF 36 constituting the slot.

  In step S <b> 11, the second control unit 31 acquires event occurrence determination time data (tp) from the data stored in the memory card 25.

  Next, in step S12, the second control unit 31 sets a plurality of different timings from the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp shown in FIG. Select multiple captured images.

  Specifically, the control program includes an image PA for TA seconds before time tp and an image PB for TB seconds after time tp shown in FIG. The image P1 photographed at time ts before TA time (for example, 12 seconds) of the indicated time tp is selected as the “first image”, and immediately before time tp (predetermined time TC seconds, for example, 0.2 seconds before) ) Is selected as a “second image”, and an image P3 captured at time tp2 immediately after time tp (after a predetermined time TC, for example, 0.2 seconds) is selected as “second image”. The control process is programmed such that “3 images” is selected, and the image P4 captured at time te after TB seconds (for example, 12 seconds) of time tp is selected as “fourth image”. The second control unit 31 performs selection control of the first to fourth images from the images PA and PB according to such a control program.

Note that time ts, which is TA seconds before time tp, is the time when image storage is started in one event, and the “first image” is also an image taken when the event occurs. The time te, which is TB seconds after the time tp, is also the time when image storage is ended in one event, and the “fourth image” is an image taken at the end of event detection.
The predetermined time TC can be set to 0.2 seconds, for example, but an optimal value including 0 seconds can be selected according to the application. For example, the time is set by setting the time by operating the time setting button of the second operation unit 34 of the driving support device 30 while displaying the set time (seconds) on the setting screen (not shown) of the driving support device 30. You can set any time by saving the information.

  Next, in step S13, the second control unit 201 stores the “first to fourth images” selected in step S12 with respect to the image processing unit 37 according to the control program stored in the ROM in the second control unit 201. ”(P1 to P4) to perform image processing for creating first to fourth thumbnail images.

  In order to create a thumbnail image from the selected first to fourth images, for example, a thumbnail image is created by performing image processing for reducing the original first to fourth images. In addition to the image processing to be reduced, other predetermined image processing such as performing image processing for enlarging the selected first to fourth images or performing image processing for cutting out a part of the image. A thumbnail image may be created by applying. The selected first to fourth images may be used as thumbnail images without any image processing.

In subsequent step S14, the second control unit 31 stores the first to fourth thumbnail images created in step S13 with respect to the display processing unit 38 according to the control program stored in the ROM in the second control unit 31. Display processing to be displayed on the display unit 32 is performed. Specifically, as described later, thumbnail images are arranged in a matrix structure and displayed as a list on the same screen.
This completes the thumbnail image creation and display process for one event.

<Display example of thumbnail image>
Next, an aspect of the display image displayed on the display unit 32 of the driving support device 30 will be described with reference to FIG. FIG. 5 is a screen display diagram illustrating an example of a display mode of thumbnail images according to the first embodiment.
The thumbnail image display mode illustrated in FIG. 5 illustrates a state in which thumbnail images relating to a plurality of events are displayed on the display unit 32 of the driving support device 30.

  In FIG. 5, 16 thumbnail images A1 to D4 arranged in a matrix structure of 4 rows in the vertical direction and 4 columns in the horizontal direction and displayed as a list on the same screen are event unit images 51 related to one event for each row. , 52, 53, 54. That is, each of the event images 51, 52, 53, and 54 is displayed with four thumbnail images related to one event arranged in one row, and four thumbnail images related to different events are displayed in different rows. Four sets of thumbnail images A1 to A4, B1 to B4, C1 to C4, and D1 to D4 are arranged and displayed.

  Note that, at the lower part of the thumbnail images A1 to D4, the shooting time of the image that is the source of each thumbnail image is displayed. In the illustrated example, the shooting time of the image that is the source of each thumbnail image when the event occurrence time tp in the event unit image 51 is “11: 46: 14.60” is displayed.

  In addition, the 16 thumbnail images shown in FIG. 5 are created from images taken for each column at the same time point (more precisely, at a time point around the event occurrence time tp for each event). The point-in-time unit images 61, 62, 63, and 64 are configured. That is, each of the time unit images 61, 62, 63, and 64 is displayed with four thumbnail images related to the same time point arranged in one column, and four thumbnail images related to different time points are different columns. 4 sets of thumbnail images A1 to D1, A2 to D2, A3 to D3, and A4 to D4 are arranged and displayed.

  In the 16 thumbnail images shown in FIG. 5, the first column from the left is a thumbnail image created from the image P1 (first image) taken at the event occurrence time (time ts), and the second column is the second column. A thumbnail image created from the image P2 (second image) taken immediately before the event occurrence determination time (time tp1), and the third column is an image P3 taken immediately after the event occurrence determination time (time tp2). Thumbnail images created from (third image), and the fourth column is thumbnail images created from image P4 (fourth image) taken at the end of event detection (time te).

  Further, in FIG. 5, four thumbnail images arranged in one row are the same event images in the order of X 1 → X 2 → X 3 → X 4 (X is any one of A, B, C, and D). Each thumbnail image is arranged and displayed in the same row in the order of time series from left to right. The display order may be arranged in chronological order from right to left instead of from left to right in the same row.

  In FIG. 5, four events are displayed. However, the number of displayed events is not limited to this, and other display numbers can be selected and other elements (G data, vehicle data, etc.) can be selected. ) And the like can also be displayed.

As described above, the driving support device 30 according to the first embodiment has the following advantages. That is,
(1) When an event such as an impact is applied to the vehicle, a first image P1 taken TA seconds before the event occurrence time (time tp), a second image P2 taken immediately before time tp, and a time The third image P3 photographed immediately after tp and the fourth image P4 photographed TB seconds after the time tp are selected, and the first to fourth thumbnail images are selected from the selected first to fourth images P1 to P4. A control is performed to create and display on the display unit 32. As a result, when the event occurs, the first to fourth thumbnail images corresponding to the first to fourth images P1 to P4 captured at different time points in the event unit images captured for a predetermined period before and after the event are displayed. By checking, the contents of the event unit image stored for each event occurrence can be easily confirmed, and the cause and situation of the event occurrence can be easily grasped.

  For example, when an event unit image related to a specific event is found from a large number of event unit images stored for each event occurrence, the event unit image is extracted from a thumbnail image at one time point related to the event unit image. Compared to searching for a specific event unit image by checking the contents, the specific event unit image to be obtained can be easily found from the thumbnail image.

(2) Like the thumbnail image display mode illustrated in FIG. 5, the thumbnail images are displayed with four thumbnail images related to one event arranged in one row, and four thumbnails related to different events. Since the images are arranged and displayed in different rows, four thumbnail images related to one event can be discriminated at a glance.
In addition, the four thumbnail images arranged in one row are arranged and displayed in the same row from the left to the right in the time series in each event, so four thumbnail images related to one event are displayed. It is possible to clearly determine the order of time passage of the thumbnail images.

  Furthermore, if you look at the thumbnail image in the first row, you can see the situation in front of the vehicle at the time of the event occurrence. If you look at the thumbnail image in the second row, you can see the situation in front of the vehicle just before the event occurrence judgment time. If you look at the thumbnail image, you can see the situation in front of the vehicle immediately after the event occurrence determination time, and you can see the situation in front of the vehicle at the end of event detection by looking at the thumbnail image in the fourth row. By comparing the images, it is possible to compare and verify the situation in front of the vehicle at the same time when the event occurs.

[Example 2]
The second embodiment is a case of sudden braking as an event example in which an impact is applied to the vehicle.
FIG. 6 is a diagram showing a change in acceleration in the case of sudden braking, and shows a time-series change in G data (combined value of Gx and Gy) output from the G sensor 12.

  As shown in FIG. 6, when the sudden brake is depressed (turned on) at time t1, the G sensor 12 mainly detects the acceleration (Gx) in the longitudinal direction of the vehicle, and the acceleration changes. The acceleration (G data) of the G sensor 12 gradually increases from the time t1 (that is, the acceleration in the direction opposite to the direction in which the vehicle travels increases) and reaches a threshold acceleration Gs (for example, 0.3 G) ( At time tp), the first control unit 11 recognizes this as a trigger, determines (determines) that an event has occurred, and the CCD camera 20 captures an image that is always stored cyclically in the first memory 108. Is stored in the memory card 150, and the time tp is stored in the memory card 150 as event occurrence determination time data.

  Specifically, also in the second embodiment, similarly to the first embodiment, control is performed so that the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp are stored in the memory card 25. To do.

  After the time tp, the acceleration continues to increase, reaches a maximum value (maximum value), then gradually decreases instead of decreasing (that is, the acceleration in the direction opposite to the direction in which the vehicle travels decreases), then decreases rapidly, After the zero point, it further decreases (that is, the acceleration in the direction in which the vehicle advances), becomes a minimum value (minimum value), increases rapidly instead of increasing (that is, the acceleration in the direction in which the vehicle advances), It becomes zero when the vehicle stops and the brake is released (OFF) (time t2).

  In FIG. 6, a period Td from time t1 to time tp is “a period from the beginning of dangerous driving to immediately before the occurrence of a trigger”, and a period Te from time tp to time t2 is a “dangerous driving behavior” period.

<Creation and display of thumbnail images (Example 2)>
FIG. 7 is a flowchart illustrating the thumbnail image creation and display process according to the second embodiment. As in the first embodiment, the control program loaded from the HDD or the like into the RAM of the second control unit 31 of the driving support device 30. And is started based on the operation of starting the driving support software of the second operation unit 34. Further, it is assumed that when the processing of the flowchart shown in FIG. 7 is started, the memory card 25 in which the image is stored in the drive recorder 10 is inserted into the second IF 36 constituting the slot.

  In step S <b> 21, the second control unit 31 acquires event occurrence determination time data (tp) from the data stored in the memory card 25. Subsequently, in step S22, the brake operation detection signal (brake operation data) output from the vehicle signal acquisition unit 21 is acquired from the data stored in the memory card 25.

  Next, in step S23, the second control unit 31 relates to the brake operation from the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp shown in FIG. A plurality of images taken at a plurality of different timings are selected.

  Specifically, in this control program, the brake operation data acquired from the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp shown in FIG. Based on the included brake operation start time data, an image captured at the time t1 when the brake operation is started (that is, the appearance start time of the acceleration event) is selected as the “first image” and immediately before the time tp (predetermined time) An image taken before TC, for example, 0.2 seconds before is selected as a “second image”, and an image taken immediately after time tp (after a predetermined time TC, for example, 0.2 seconds) is selected. Image selected as “third image” and imaged at time t2 when the brake operation is ended based on the brake operation end time data included in the acquired brake operation data (that is, when the acceleration event appears) Control processing such selected as "fourth image" is programmed. The second control unit 31 performs selection control of the first to fourth images from the images PA and PB according to such a control program.

The predetermined time TC can be set to 0.2 seconds, for example, but an optimal value including 0 seconds can be selected according to the application.
In addition to the time t1, just before and after the time tp, and at the time t2, when the acceleration reaches a maximum value (maximum value), immediately before and immediately after that, the acceleration becomes a minimum value (minimum value). You may select the image image | photographed just before and immediately after that time.

  Next, in step S24, the second control unit 31 stores the “first to fourth images” selected in step S23 with respect to the image processing unit 37 in accordance with the control program stored in the ROM in the second control unit 31. The image processing for creating the first to fourth thumbnail images is performed.

In subsequent step S25, the second control unit 31 stores the first to fourth thumbnail images created in step S24 with respect to the display processing unit 38 according to the control program stored in the ROM in the second control unit 31. Display processing to be displayed on the display unit 32 is performed. Specifically, as shown in FIG. 5, thumbnail images are arranged in a matrix structure and displayed as a list on the same screen.
This completes the thumbnail image creation and display process for one event.

As described above, the driving support device 30 of the second embodiment has the following advantages. That is,
In case of sudden braking as an event example such as an impact on the vehicle, the first image taken at time t1 when the brake operation was started, the second image taken immediately before time tp, and taken immediately after time tp The selected third image and the fourth image captured at time t2 when the brake operation is finished are selected, and first to fourth thumbnail images are created from the selected first to fourth images. Control the display. As a result, in the case of sudden braking, the time t1 at which the brake operation is started and the brake operation are ended particularly at different time points in the event unit images taken for a predetermined period before and after the event occurrence time (time tp). By confirming the first to fourth thumbnail images corresponding to the first to fourth images taken at the time including the time t2, the content of the event unit image stored in the case of sudden braking as an event example is obtained. It can be easily confirmed, and it is easy to grasp the cause and situation of sudden braking.

  For example, when an event unit image related to sudden braking is found from a large number of event unit images stored for each event occurrence, the content of the event unit image is determined from a thumbnail image at one point related to the event unit image. Compared to searching for an event unit image related to sudden braking, the event unit image related to the desired sudden braking can be easily found from the thumbnail images.

  Also, if you look vertically at the thumbnail image in the first column from the left of the 16 thumbnail images shown in FIG. 5, you can compare and see the images at the time of sudden braking (when you started the brake operation). Can do. Also, if you look at the thumbnail image in the second row vertically, you can compare and view the images immediately before the time when it was determined that an event occurred due to sudden braking. If you look at the thumbnail image vertically, you can compare the images immediately after that. Further, if the thumbnail images in the fourth row are viewed vertically, the images at the time when the brake is released (when the brake operation is finished) can be compared and viewed.

[Example 3]
The third embodiment is a case in which an abrupt steering wheel is used as an event example in which an impact is applied to the vehicle.
FIG. 8 is a diagram showing a change in acceleration in the case of a sudden handle, and shows a time-series change in G data (combined value of Gx and Gy) output from the G sensor 12.

  As shown in FIG. 8, when the sudden handle starts to be turned at time t3, the G sensor 12 mainly detects the acceleration (Gy) in the left-right direction of the vehicle, and the acceleration changes. The acceleration (G data) from the G sensor 12 gradually increases from the time t3 (that is, the acceleration in the direction opposite to the direction in which the vehicle bends increases), and becomes a threshold acceleration Gs (for example, 0.3 G). At the time (time tp), the first control unit 11 recognizes it as a trigger, determines (determines) that an event has occurred, and the CCD camera 20 picks up an image and always stores it cyclically in the first memory 108. The image is stored in the memory card 25, and the time tp is stored in the memory card 25 as event occurrence determination time data.

  Specifically, also in the third embodiment, as in the first and second embodiments, the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp are stored in the memory card 25. To control.

  After the time tp, the acceleration continues to increase, reaches a maximum value (maximum value), then gradually decreases instead of decreasing (that is, the acceleration in the direction opposite to the vehicle turning direction decreases) and then decreases rapidly. After the zero point, it further decreases (ie, the acceleration in the direction of turning of the vehicle increases), becomes a minimum value (minimum value), and increases rapidly instead of increasing (ie, the acceleration in the direction of turning of the vehicle decreases) When the vehicle has finished turning and the steering wheel has been turned (ie, the steering wheel has been returned) (time t4), it becomes zero.

  In FIG. 8, a period Tf from time t3 to time tp is “a period from the beginning of dangerous driving to immediately before the occurrence of a trigger”, and a period Tg from time tp to time t4 is a “dangerous driving behavior” period.

<Creation and display of thumbnail images (Example 3)>
FIG. 9 is a flowchart illustrating a thumbnail image creation and display process according to the third embodiment. Similar to the first and second embodiments, the thumbnail image is loaded from the HDD or the like into the RAM of the second control unit 31 of the driving support device 30. This is realized by the CPU executing in accordance with the control program, and is started based on the operation of starting the driving support software of the second operation unit 34. Further, it is assumed that when the processing of the flowchart shown in FIG. 9 is started, the memory card 25 in which the image is stored in the drive recorder 10 is inserted into the second IF 36 constituting the slot.

  In step S <b> 31, the second control unit 31 acquires event occurrence determination time data (tp) from the data stored in the memory card 25. Subsequently, in step S32, the steering wheel rotation detection signal (steering wheel rotation data) output from the vehicle signal acquisition unit 21 is acquired from the data stored in the memory card 25.

  Next, in step S33, the second control unit 31 relates to the steering wheel rotation from the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp shown in FIG. A plurality of images taken at a plurality of different timings are selected.

  Specifically, the control program includes the acquired handle rotation data from the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp shown in FIG. Based on the included handle rotation start time data, the image picked up at the time t3 at which the handle rotation is started is selected as the “first image”, and immediately before the time tp (before the predetermined time TC, for example, 0.2 seconds before ) Is selected as the “second image”, and the image captured immediately after the time tp (after the predetermined time TC, for example, 0.2 seconds) is selected as the “third image” and acquired. The control process is programmed such that the image captured at time t4 when the handle rotation is ended is selected as the “fourth image” based on the handle rotation end time data included in the handle rotation data. The second control unit 31 performs selection control of the first to fourth images from the images PA and PB according to such a control program.

The predetermined time TC can be set to 0.2 seconds, for example, but an optimal value including 0 seconds can be selected according to the application.
In addition to the time t3, immediately before and immediately after the time tp, and at the time t4, when the acceleration reaches a maximum value (maximum value), immediately before and immediately after that, the acceleration has reached a minimum value (minimum value). You may select the image image | photographed just before and immediately after that time.

  Next, in step S34, the second controller 31 stores the “first to fourth images” selected in step S33 with respect to the image processor 37 in accordance with the control program stored in the ROM in the second controller 31. The image processing for creating the first to fourth thumbnail images is performed.

In subsequent step S35, the second control unit 31 stores the first to fourth thumbnail images created in step S34 with respect to the display processing unit 38 according to the control program stored in the ROM in the second control unit 31. Display processing to be displayed on the display unit 32 is performed. Specifically, as shown in FIG. 5, thumbnail images are arranged in a matrix structure and displayed as a list on the same screen.
This completes the thumbnail image creation and display process for one event.

As described above, the driving support device 30 according to the third embodiment has the following advantages. That is,
In the case of a sudden handle as an event example such as an impact on the vehicle, the first image taken at time t3 when the steering wheel rotation was started, the second image taken immediately before time tp, and taken immediately after time tp The selected third image and the fourth image picked up at the time t4 when the handle rotation is finished are selected, and the first to fourth thumbnail images are created from the selected first to fourth images. Control the display. Thereby, in the case of a sudden handle, at a different time point in the event unit image taken for a predetermined period before and after the event occurrence time (time tp), in particular, at the time t3 when the handle rotation is started and the handle rotation is ended. By confirming the first to fourth thumbnail images corresponding to the first to fourth images taken at the time including the time t4, the content of the event unit image stored in the case of the sudden handle as the event example is obtained. It can be easily confirmed, and it is easy to grasp the cause and situation of the sudden handle.

  For example, when an event unit image related to a sudden handle is found from a large number of event unit images stored for each event occurrence, the content of the event unit image is determined from a thumbnail image at one point related to the event unit image. Compared to searching for an event unit image related to the sudden handle, the event unit image related to the desired sudden handle can be easily found from the thumbnail image.

  If the thumbnail images in the first row from the left of the 16 thumbnail images shown in FIG. 5 are viewed vertically, the images at the time when the handle operation is started (when the handle rotation is started) can be compared and viewed. Can do. If you look at the thumbnail image in the second row vertically, you can compare and view the images immediately before the event that was determined to have occurred due to the handle operation. If you look at the image vertically, you can compare the images immediately after that. Furthermore, if the thumbnail images in the fourth row are viewed vertically, the images at the time when the handle operation is stopped (when the handle rotation is finished) can be compared and viewed.

Furthermore, as in the third embodiment, the creation of the thumbnail image at the time when the steering wheel rotation is started has an advantage when the steering wheel operation and the acceleration event are not linked as described below.
For example, there are cases where acceleration occurs regardless of the handle operation, such as when a vehicle collides or when traveling on a slope, and cases where acceleration occurs due to a handle operation such as a sudden handle. The first thumbnail image in the third embodiment (first column from the left in FIG. 5) is the one at the time when the handle rotation is started, but the handle operation and the acceleration event are not linked. There is.
According to the third embodiment, the thumbnail image at the time when the handle rotation is started and the thumbnail image having a predetermined relationship with the occurrence of the event can be displayed. And the occurrence of an acceleration event do not match, the event occurs when the handle rotation is started from the thumbnail image at the start of the handle rotation instead of the thumbnail image at the start of the acceleration event appearance. Can be used to check the cause of the event and the situation.

[Example 4]
The fourth embodiment is a case of a vehicle collision as an example of an event such as an impact on the vehicle.
FIG. 10 is a diagram showing a change in acceleration in the case of a vehicle collision, and shows a time-series change in G data (combined value of Gx and Gy) output from the G sensor 12.

  As shown in FIG. 10, when the vehicle collides with something at time t5, the G sensor 12 detects and changes the longitudinal acceleration (Gx) and the lateral acceleration (Gy) of the vehicle, respectively. The acceleration (G data) from the G sensor 12 suddenly increases immediately after time t5 and then suddenly increases / decreases, and then increases rapidly, which becomes a threshold acceleration Gs (for example, 0.3G). (Time tp), the first control unit 11 recognizes this as a trigger, determines that an event has occurred (determination), the CCD camera 20 picks up an image, and always stores it cyclically in the first memory 108. The image is stored in the memory card 25, and the time tp is stored in the memory card 25 as event occurrence determination time data.

  Specifically, also in the fourth embodiment, as in the first, second, and third embodiments, the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp are stored in the memory card 25. Control to do.

  After time tp, the acceleration increases, decreases rapidly after reaching the maximum value (maximum value), increases rapidly after reaching the minimum value (minimum value) after passing through the zero point, and then increases again after passing through the zero point. After the value (maximum value) is reached, it rapidly decreases, and after passing through the zero point again, sudden increase and decrease are repeated, and then the vehicle stops and becomes zero when the acceleration converges (t6).

  In FIG. 10, a period Th from time t5 to time tp is “a period of impact before a large impact”. In this period Th, an impact waveform due to impact mitigation due to the effect of a bumper, a guard rail, or the like of the vehicle appears. A period Ti from time tp to time t6 is a period of “impact behavior”.

<Creation and display of thumbnail images (Example 4)>
FIG. 11 is a flowchart showing a thumbnail image creation and display process according to the fourth embodiment. Similar to the first, second, and third embodiments, the RAM of the second control unit 31 of the driving support device 30 is loaded from the HDD or the like. This is realized by the CPU executing in accordance with the control program, and is started based on the operation of starting the driving support software of the second operation unit 34. Further, it is assumed that when the processing of the flowchart shown in FIG. 11 is started, the memory card 25 in which the image is stored in the drive recorder 10 is inserted into the second IF 36 constituting the slot.

  In step S <b> 41, the second control unit 31 acquires event occurrence determination time data (tp) from the data stored in the memory card 25. Subsequently, in step S42, the vehicle speed (vehicle speed data) output from the vehicle signal acquisition unit 21 is acquired from the data stored in the memory card 25. Further, in step S43, detection G data (Gx, Gy) of the G sensor 12 is acquired from the data stored in the memory card 25.

Next, in step S44, the second control unit 31 relates to the vehicle collision from the image PA for TA seconds before the time tp and the image PB for TB seconds after the time tp shown in FIG. A plurality of images taken at a plurality of different timings are selected.
Specifically, in this control program, at the time t5 when the vehicle collides from the image PA for TA seconds before the time tp shown in FIG. 3B and the image PB for TB seconds after the time tp. Immediately after the magnitude (value) of the G data repeats abrupt increase / decrease and reaches the second maximum value tp3 (that is, when the acceleration behavior of the vehicle due to the reaction of the bumper, guardrail, etc. is born) Select as the “first image”, select the image taken immediately before the time tp (before the predetermined time TC, for example, 0.2 seconds) as the “second image”, and immediately after the time tp (the predetermined time TC) After that, for example, an image taken at 0.2 seconds) is selected as a “third image”, and an image taken at time t6 when the vehicle speed becomes zero (that is, when the acceleration behavior converges) is “ Such as selecting “4th image”. Control processing is programmed. The second control unit 31 performs selection control of the first to fourth images from the images PA and PB according to such a control program.

The predetermined time TC can be set to 0.2 seconds, for example, but an optimal value including 0 seconds can be selected according to the application.
Further, immediately after the time t5, immediately before and after the time tp, and at the time t6, when the acceleration reaches a maximum value (maximum value), immediately before and immediately after that, or at a minimum value (minimum value). At that time, an image captured immediately before and immediately after that may be selected.

  Next, in step S45, the second control unit 31 stores the “first to fourth images” selected in step S44 with respect to the image processing unit 37 in accordance with the control program stored in the ROM in the second control unit 31. The image processing for creating the first to fourth thumbnail images is performed.

In subsequent step S46, the second control unit 31 stores the first to fourth thumbnail images created in step S45 to the display processing unit 38 according to the control program stored in the ROM in the second control unit 31. Display processing to be displayed on the display unit 32 is performed. Specifically, as shown in FIG. 5, thumbnail images are arranged in a matrix structure and displayed as a list on the same screen.
This completes the thumbnail image creation and display process for one event.

As described above, the driving support device 30 according to the fourth embodiment has the following advantages. That is,
In the case of a vehicle collision as an event example such as an impact on the vehicle, a first image captured at the time when the G data immediately after time t5 when the vehicle collided repeatedly increased and decreased (at the time of reaction acceleration behavior), time Select and select the second image taken immediately before tp, the third image taken immediately after time tp, and the fourth image taken at time t6 (when the acceleration converges) when the vehicle speed becomes zero The first to fourth thumbnail images are created from the first to fourth images, and are controlled to be displayed on the display unit 32. Thereby, in the case of a vehicle collision, the G data immediately after the time t5 when the vehicle collides suddenly increases or decreases at different time points in the event unit images taken for a predetermined period before and after the event occurrence time point (time tp). 1st to 4th thumbnails corresponding to the 1st to 4th images taken at the time including the time when the vehicle speed is repeated (time of the acceleration behavior of the reaction) and the time t6 when the vehicle speed becomes zero (time when the acceleration converges) By confirming the image, it is possible to easily confirm the contents of the event unit image stored in the event of a vehicle collision as an event example, and it is easy to grasp the cause and situation of the vehicle collision.

  For example, when an event unit image related to a vehicle collision is found from a large number of event unit images stored for each event occurrence, the content of the event unit image from a thumbnail image at one point related to the event unit image. Compared to searching for an event unit image related to a vehicle collision, it is possible to easily find an event unit image related to a desired vehicle collision from the thumbnail images.

  If the thumbnail images in the first column from the left of the 16 thumbnail images shown in FIG. 5 are viewed vertically, the images at the time of the vehicle collision can be compared and viewed. If the thumbnail image in the second row is viewed vertically, it is possible to compare and view the images immediately before the time when the event is determined to have occurred due to a vehicle collision. If you look at it, you can compare the images immediately after that. Furthermore, if the thumbnail images in the fourth row are viewed vertically, the images at the time when the vehicle speed becomes zero can be compared and viewed.

<Experimental example>
FIG. 12 is a diagram showing an experiment example of acceleration change in the case of a vehicle-to-vehicle collision in relation to the acceleration change in the case of a vehicle collision according to the fourth embodiment shown in FIG. The detected acceleration Gx value in the X-axis direction (front-rear direction of the vehicle), the detected acceleration Gy value in the Y-axis direction (left-right direction of the vehicle), and the detected acceleration Gz value in the Z-axis direction (up-down direction of the vehicle) Each time-series change is shown.

In this experimental example, as shown in FIG. 12, when the vehicle collides with something at time t7, the Gx value, Gy value, and Gz value repeated abrupt increase and decrease immediately after time t7 in a relatively short cycle. Later, when the value of Gx becomes a threshold acceleration Gs (for example, 1.0) (time tp), it is recognized as a trigger and it is determined that an event has occurred.
After time tp, the value of Gx may increase slightly, but eventually converges to near zero (time t8). In addition, after time tp, the values of Gy and Gz are relatively small and eventually converge.

  In FIG. 12, the period Tj from the time t7 to the time tp is the “period of impact before a large impact”, similarly to the period Th of FIG. 10, and this period Tj is due to the effects of the bumper, guardrail, etc. of the vehicle. Impact waveform due to impact reduction appears. Further, the period Ti from the time tp to the time t8 is a period of “impact behavior” as in the period Ti of FIG. 10, and the vehicle moves by reaction after impact in this period Tk.

<Display examples>
FIG. 13 is a diagram showing a display example of the thumbnail screen according to the experiment example shown in FIG.
In this display example, the memory card 25 mounted in the drive recorder 10 and recorded with driving information including camera images and image-related information is taken out and displayed on the display unit 32 of the driving support device 30. .
In FIG. 13, 16 thumbnail images arranged in a matrix structure of 4 rows x 4 columns and displayed in a list on the same screen are event unit images 71, 72, 73, 74, and for each column, point-in-time images 71 created from images taken at the same time point (more precisely, at a time point that is relatively the same for each event with respect to the event occurrence time tp). 72, 73, 74 are displayed.

Next, a method for displaying a list of thumbnail images in a form arranged in a matrix structure as described above will be described below.
First, the user selects files 91, 92, 93, and 94 of a plurality of events (here, four) to be displayed by the second operation unit 204 from the playlist 90 on the upper right of the display image shown in FIG. To do. Next, event unit images corresponding to the selected event files 91, 92, 93, 94 are read. Based on the read files, thumbnail image creation and display processing is performed in accordance with the processing procedure of the flowchart shown in FIG. The thumbnail images are arranged in a matrix structure and displayed as a list on the same screen.

In the display example of FIG. 13, the event unit image 71 displayed for each row includes four time points regarding the event of the file 91 (that is, [1] an event occurrence time point TA seconds before the event occurrence time tp, [2 ] Thumbnail images created from images taken immediately before the time tp (immediately before the event), [3] immediately after the time tp (immediately after the event), and [4] at the end of event detection, which is TB seconds of the time tp. Is displayed.
Similarly, the event unit image 72 displays the thumbnail images at the four points in time for the event of the file 92, and the event unit image 73 displays the thumbnail images at the four points of time for the event in the file 93, and the event The unit image 74 displays the thumbnail images at the above-described four times for the event of the file 94.

In the display example of FIG. 13, the point-in-time image 81 displayed for each column is [1] TA seconds before the event occurrence time tp in the four thumbnail images of the event unit images 71, 72, 73, and 74, respectively. A thumbnail image created from an image taken at the event occurrence time is displayed.
Similarly, the time unit image 82 is created from images taken immediately before (2) the event occurrence time tp in the four thumbnail images of the event unit images 71, 72, 73, and 74 (immediately before the event occurrence). The thumbnail image 83 is displayed, and the time unit image 83 is taken immediately after the event occurrence time tp (immediately after the event occurrence) among the four thumbnail images of the event unit images 71, 72, 73, and 74, respectively. The thumbnail images created from the images are displayed, and the time unit image 84 is [4] TB seconds after the event occurrence time tp in the four thumbnail images of the event unit images 71, 72, 73, and 74, respectively. A thumbnail image created from an image taken at the end of event detection is displayed.

<Modification>
The present invention is not limited to the description of the embodiment of the present invention described above, and various modifications are possible. For example, the following modifications may be made.

(1) In the embodiment described above, the time (time tp) when the value (size) of the G data output from the G sensor 12 reaches the threshold acceleration Gs is recognized as a trigger, and it is determined that an event has occurred. However, the present invention is not limited to this. For example, when the value (size) of the angular acceleration data output from the gyro sensor 13 or the atmospheric pressure / altitude data output from the atmospheric pressure sensor 14 becomes a predetermined value, it is recognized as a trigger. It may be determined that an event has occurred. Further, it may be determined that an event has occurred from the result of analyzing the recorded video.
In addition, when an emergency switch of the first operation unit 17 is pressed, when a signal is input from an external port, and communication from an external device such as an AVM-ECU (Automatic Vehicle Monitoring-Engine Control Unit) (for example, The time when an event occurs may be when a command is received by RS-232C).

(2) In the above-described embodiment, the first to first images taken at four points in time for each event such as sudden braking, sudden steering, and vehicle collision from images of a predetermined period stored for each event. The first to fourth thumbnail images are created by selecting four images, but not limited to this, a plurality of images taken at a plurality of different time points other than the four time points are selected and selected. A thumbnail image corresponding to the image may be created. The plurality of different time points may be various combinations depending on the type of event, but is preferably a combination of time points at which event characteristics appear (a specific period within a predetermined period).
The combination of the time when the feature of the event appears is obtained by using various data such as “data detecting the behavior of the vehicle” and “data relating to the functional state of the vehicle” stored in the memory card 25 together with the image data alone or in combination. Based on these data, a plurality of images taken at an appropriate timing may be selected, and thumbnail images corresponding to the selected images may be created.

  The “data detecting the behavior of the vehicle” may be, for example, acceleration data, angular acceleration data, atmospheric pressure / altitude data, video analysis data, or the like. Examples of the “data relating to the functional state of the vehicle” include data relating to operation of any one of brake, turn signal, accelerator, steering wheel rotation, shift lever position, and side brake during vehicle operation, and a vehicle failure diagnosis system. Data obtained from OBD (On Board Diagnosis) and CAN (Controller Area Network) which is an in-vehicle LAN may be used.

(3) In the above-described embodiment, as the G data, the component acceleration (G data) Gx in the X-axis direction (the vehicle longitudinal direction) and the component acceleration (G data) Gy in the Y-axis direction (the vehicle left-right direction) Although the occurrence of the event is determined using the composite value, the present invention is not limited to this, and the occurrence of the event may be determined using each component acceleration (Gx, Gy) individually. For example, when Gx exceeds a predetermined threshold and then Gy exceeds a predetermined threshold within a predetermined time, a change pattern of the value (magnitude) of each component takes a predetermined pattern and is recognized as a trigger. However, it may be determined that an event has occurred. In addition to Gx and Gy, the occurrence of an event may be determined in consideration of component acceleration Gz in the Z-axis direction (the vertical direction of the vehicle).

(4) Not only the embodiment described above, but also information (for example, vehicle by sudden braking) that is different from information (for example, G data (Gy) in the left-right direction of the vehicle by a sudden handle) used to determine that an event has occurred. Alternatively, a thumbnail image may be created by selecting an image picked up when G data (Gx) in the front-rear direction becomes a predetermined value (for example, a maximum value). Thereby, information (for example, G data (Gx) by sudden braking) different from information (for example, G data (Gy) by sudden steering) used to determine that an event has occurred becomes a predetermined value. Event unit images related to events can be easily found.

(5) Not limited to the above-described embodiment, for example, when the G data value (magnitude) reaches the threshold acceleration Gs and it is determined that an event has occurred (time tp), for example, the G data Sub-events that caused the occurrence of events such as the time when it becomes a local minimum, the time when it becomes an inflection point of time series change of G data, and the time when G data changes characteristically such as sudden increase and decrease in a short time A thumbnail image may be created by selecting an image picked up at the time of estimating the occurrence time. This makes it easy to find event-unit images related to the event at the time of estimating the occurrence time of the sub-event that caused the occurrence of the event, such as the time when characteristic changes such as a sudden increase or decrease in G data occur. be able to.

(6) In the above-described embodiment, the drive recorder 10 is a trigger recording type that records images before and after the “trigger indicating that the recording condition is satisfied”. However, the present invention is not limited to this. It may be a continuous recording type in which information is recorded in order to store the information for enabling the position of the video that has been triggered to be recorded. In this continuous recording type drive recorder, for example, which position of which video is triggered may be specified from video recording start time information and triggered time information.
In the case of the above trigger recording type drive recorder, for example, an image recorded for a predetermined period before and after the occurrence of a trigger indicating that a recording condition by the drive recorder is satisfied may be referred to as an “event unit video”. . Further, in the case of the above-described constant recording type drive recorder, for example, in a video in which one recording unit is from on to off of the engine, a portion “ It is recommended to use “event unit video”.

(7) In the above-described embodiment, the image PA for TA seconds and the image PB for TB seconds before and after the time tp determined as the occurrence of the event are stored in the memory card 25. The image for the entire time from the time to time or the image for the limited time within the total time may be stored in the memory card 25. The limited time in the total time may be a time including the time tp determined as the occurrence of the event.
Further, the total time of TA seconds and TB seconds before and after the time tp described above may be a fixed time for each event occurrence or may be a variable time. The total time of TA seconds and TB seconds may be a period sufficient to make it possible to identify the cause of the occurrence of the event. In particular, the total time of TA seconds and TB seconds may be different depending on the type of event.

(8) In the above-described embodiment, the TA second time and the TB second time before and after the time tp determined to be the event occurrence are the same time (in the example, 12 seconds before and 12 seconds after). For example, the time may be different. In particular, it is preferable to increase the time of TA seconds before time tp and shorten the time of subsequent TB seconds (for example, 15 seconds before and 5 seconds after).

(9) In the above-described embodiment, TA seconds before, immediately before (TC seconds), immediately after (TC time) from the images PA for TA seconds before and after the time tp determined to be an event and the image PB for TB seconds. The first to fourth images taken after TB seconds are selected, and the first to fourth thumbnail images are created from the selected first to fourth images. 1 or 4 images are selected by combining two or three images, or any one to four images of the first to fourth images and one or more images at other times It is also possible to select a combination of images and create a thumbnail image from the selected images.

(10) In the above-described embodiment, the images taken at the time points before and after the time tp determined as the occurrence of the event are selected. However, the present invention is not limited to this, and the image taken at the time tp is selected. You may combine an image and the image of the time before and behind time tp.

(11) In the above-described embodiment, as in the example of the thumbnail image display mode shown in FIG. 5, 16 thumbnail images constitute an event unit image related to one event for each row, and for each column, Although the time unit images at the same time point are configured, the present invention is not limited to this, and the row and column may be interchanged to configure the event unit image for each column, and the time unit image may be configured for each row. In addition, when event unit images related to one event are configured for each row (or column), the event unit images in one row (or column) can be arranged in a time-sequential order. Instead of setting the same time point every time, an image at a time point corresponding to the type of event may be used. Furthermore, the display is not limited to 4 rows x 4 columns in the vertical direction, but is arranged in a matrix structure of other combinations of the number of rows and the number of columns, such as 5 rows x 5 columns and 3 rows x 3 columns, and displayed as a list on the same screen. May be. Furthermore, when the display cannot be completed within the screen, it is preferable to scroll the matrix as a whole instead of scrolling left and right for each event.

(12) In the above-described embodiment, the display control system of the present invention has been described as an example of the driving support system (driving support device 30). However, the display control system can be implemented as a function of various electronic devices. For example, you may incorporate as a function of a navigation apparatus, a drive recorder, a radar detector, and a car audio.

(13) Not limited to the above-described embodiment, the control program stored in advance in the ROM of the second control unit 31 of the driving support device 30 is applied to a portable terminal such as a general-purpose personal computer, a mobile phone, a smartphone, or a portable game machine. Downloaded and made the computer mounted in the portable terminal execute the control process of the second control unit 31 to read the image and various data stored in the memory card 25, and created from the read image and various data A thumbnail image may be displayed.

(14) Each function of the above embodiment is realized by describing the function in a computer-readable program language and causing the computer of the second control unit 31 to execute the program. Instead, for example, the program may be distributed to a plurality of computers and distributedly processed.

DESCRIPTION OF SYMBOLS 10 Drive recorder 11 1st control part 12 Acceleration sensor (G sensor)
DESCRIPTION OF SYMBOLS 13 Gyro sensor 14 Barometric pressure sensor 15 GPS antenna 16 GPS receiver 17 1st operation part 18 1st memory 19 1st IF (interface)
DESCRIPTION OF SYMBOLS 20 CCD camera 21 Vehicle signal acquisition part 22 Power supply part 25 Memory card 30 Driving support device 31 2nd control part 32 Display part 33 Output part 34 2nd operation part 35 2nd memory 36 2nd IF (interface)
37 Image processing unit 38 Display processing unit 51-54, 71-74 Event unit image 61-64, 81-84 Time unit image 90 Playlist 91-94 file

Claims (6)

A display control system that performs control to display a thumbnail image related to video recorded by a drive recorder mounted on a vehicle,
A function of displaying a plurality of thumbnail images corresponding to different time points in the video range ,
A display control system for displaying, as one of the plurality of thumbnail images, an image recorded by the drive recorder when a brake operation of the driver of the vehicle is finished. Wherein the plurality of the thumbnail picture image, wherein together with the image when the brake operation is completed, the display control system according to claim 1, characterized in that to display an image when the braking operation is started. And the plurality of thumbnail images image, wherein together with the image when the brake operation is completed, the display control according to claim 1 or 2, characterized in that to display an image when the acceleration becomes a predetermined state system. And the plurality of thumbnail images image, wherein together with the image when the brake operation is completed, according to any one image from claim 1, characterized in that to display the 3 when the acceleration becomes maximum value Display control system. And the plurality of thumbnail images image, an image of when the brake operation is completed, acceleration according to any one of claims 1 to 4, characterized in that to display the image when it becomes the minimum value Display control system.   The program for making a computer implement | achieve the function of the display control system of any one of Claim 1 to 5.