JP6756992B2 - Image recording device, image recording method and image recording program - Google Patents

Description

The present invention relates to an image recording device, an image recording method, and an image recording program.

When a general drive recorder detects an event such as an impact, the image data for a predetermined time including the detection time of the event is moved to a memory area that is not overwritten (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2000-6854

The conventional drive recorder, in which the moving image for which overwriting is prohibited is limited to a predetermined time including the time when the event is detected, is insufficient from the viewpoint of recording the state of the target vehicle after the event occurs. For example, when a collision accident with a vehicle occurs, the target vehicle is already in contact with the own vehicle at the time of detection, and the lower part of the target vehicle often enters the blind spot of the drive recorder. The image captured at this time does not show the lower part of the target vehicle, and may not include important information for identifying the target vehicle such as a license plate. The information obtained from such images is inadequate from the perspective of identifying the target vehicle.

The present invention has been made to solve such a problem, and an object of the present invention is to provide an image recording device or the like capable of accurately recording the state of a target vehicle related to an event.

The image recording device according to the first aspect of the present invention has an image data acquisition unit that sequentially acquires image data that images the surroundings of the own vehicle, and an event signal acquisition unit that acquires an event signal indicating the occurrence of an event for the own vehicle. When the event signal acquisition unit acquires the event signal, the number plate detection unit that detects the number plate from the image of the image data acquired by the image data acquisition unit and the event signal acquisition unit perform the event. When the signal is acquired, the write control unit writes the image data including the period from the time when the event signal is acquired to the time when the number plate detection unit detects the number plate to the memory so as not to be overwritten. And.

The image recording method according to the second aspect of the present invention includes an image data acquisition step of sequentially acquiring image data obtained by imaging the periphery of the own vehicle, and an event signal acquisition step of acquiring an event signal indicating the occurrence of an event for the own vehicle. When the event signal is acquired in the event signal acquisition step, the event is detected in the number plate detection step for detecting the number plate from the image of the image data acquired in the image data acquisition step and the event in the event signal acquisition step. When the signal is acquired, the write control step of writing the image data including the period from the time when the event signal is acquired to the time when the number plate detection unit detects the number plate to the memory so as not to be overwritten. And the step are executed by the image recording device.

The image recording program according to the third aspect of the present invention has an image data acquisition step of sequentially acquiring image data of the surroundings of the own vehicle and an event signal acquisition step of acquiring an event signal indicating the occurrence of an event for the own vehicle. When the event signal is acquired in the event signal acquisition step, the number plate detection step of detecting the number plate from the image of the image data acquired in the image data acquisition step and the event in the event signal acquisition step. When the signal is acquired, the write control step of writing the image data including the period from the time when the event signal is acquired to the time when the number plate detection unit detects the number plate to the memory so as not to be overwritten. And let the computer do it.

According to the present invention, image data including a period from the time when the event occurs to the time when the license plate is detected in the continuously captured image is left, so that the image data includes the number of the vehicle related to the event. It can be expected to include plate information. That is, it is possible to more accurately record the information of the target vehicle related to the event.

It is the schematic which shows the state that the drive recorder which concerns on Embodiment 1 is installed in the own vehicle. It is a block diagram which shows the structure of the drive recorder which concerns on Embodiment 1. FIG. It is a conceptual diagram explaining the ring buffer set in the memory card which concerns on Embodiment 1. FIG. It is the schematic which shows the state immediately after the event occurrence which concerns on Embodiment 1 and the state which the other vehicle separated. It is explanatory drawing explaining the target file of the overwriting prohibition with respect to the lapse of time which concerns on Embodiment 1. It is a flow chart which shows the control flow of the drive recorder which concerns on Embodiment 1. FIG. It is a block diagram which shows the structure of the drive recorder which concerns on Embodiment 2. It is the schematic which shows the state that the other vehicle which concerns on Embodiment 2 is separated. It is a flow chart which shows the control flow of the drive recorder which concerns on Embodiment 2. It is the schematic which shows the appearance that another drive recorder is installed in the own vehicle.

Hereinafter, the present invention will be described through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Moreover, not all of the configurations described in the embodiments are indispensable as means for solving the problem.

[Embodiment 1]
FIG. 1 is a schematic view showing a state in which a drive recorder 100, which is an example of the image recording device according to the first embodiment, is installed in the own vehicle 900. The drive recorder 100 includes a camera unit 110. The camera unit 110 is installed above the windshield in the traveling direction of the own vehicle 900 so that the surrounding environment in front can be imaged. The field of view has an extension of about 130 ° diagonally, for example, as shown by the alternate long and short dash line.

The own vehicle 900 is provided with distance sensors 170 that detect distances to objects such as other vehicles and pedestrians at a plurality of locations in front of the vehicle. The distance sensor 170 is, for example, a millimeter wave radar or an ultrasonic sensor, and outputs a distance signal as a detection result to the drive recorder 100. The distance sensor 170 may be built in the drive recorder 100. Further, the own vehicle 900 includes an acceleration sensor 160 that detects acceleration such as an impact received by the own vehicle 900. The acceleration sensor 160 outputs an acceleration signal, which is a detection result, to the drive recorder 100. The acceleration sensor 160 may be built in the drive recorder 100.

FIG. 2 is a block diagram showing the configuration of the drive recorder 100. The drive recorder 100 is mainly composed of a camera unit 110 and a main body unit 130.

The camera unit 110 mainly includes a lens 112, an image sensor 114, and an AFE (analog front end) 116. The lens 112 guides the incident subject luminous flux to the image sensor 114. The lens 112 may be composed of a plurality of optical lens groups.

The image sensor 114 is, for example, a CMOS image sensor. The image sensor 114 adjusts the charge accumulation time by the electronic shutter according to the exposure time per frame designated by the system control unit 131, performs photoelectric conversion, and outputs a pixel signal. The image sensor 114 passes the pixel signal to the AFE 116. The AFE 116 adjusts the level of the pixel signal according to the amplification gain instructed by the system control unit 131, A / D converts it into digital data, and transmits it as pixel data to the main unit 130. The camera unit 110 may be provided with a mechanical shutter or an iris diaphragm. When a mechanical shutter or an iris diaphragm is provided, the system control unit 131 can also use these to adjust the amount of light incident on the image sensor 114.

The main unit 130 detects system control unit 131, image input IF 132, work memory 133, system memory 134, image processing unit 135, display output unit 136, memory control unit 137, memory IF 138, input / output IF 139, and NP (number plate). The unit 140 and the bus line 142 are mainly provided. The image input IF 132 has a function as an image data acquisition unit that sequentially acquires image data captured by the camera unit 110, receives pixel data from the camera unit 110 connected to the main unit 130, and delivers it to the bus line 142. ..

The work memory 133 is composed of, for example, a volatile high-speed memory. The work memory 133 receives pixel data from the AFE 116 via the image input IF 132, and collectively stores the image data in one frame. The work memory 133 delivers image data to the image processing unit 135 in frame units. Further, the work memory 133 is appropriately used as a temporary storage area even in the middle stage of image processing by the image processing unit 135.

The image processing unit 135 performs various image processing on the received image data to generate image data in a predetermined format. For example, when generating moving image data in MPEG file format, after performing white balance processing, gamma processing, etc. on the image data of each frame, compression processing within the image data of each frame and between the image data of adjacent frames is performed. Execute. The image processing unit 135 sequentially generates display image data from the generated image data and delivers it to the display output unit 136. Hereinafter, the image data of each frame constituting the moving image data will be referred to as frame image data, and the image represented by the frame image data will be referred to as a frame image.

The display output unit 136 converts the display image data received from the image processing unit 135 into an image signal that can be displayed by the display unit 180 and outputs the data. The display unit 180 may be, for example, a display panel of a car navigation system, or may be a dedicated display panel provided integrally with the drive recorder 100. The display unit 180 can sequentially display the image signals received from the display output unit 136.

The system memory 134 is composed of a non-volatile recording medium such as an SSD. The system memory 134 records and holds constants, variables, set values, control programs, and the like necessary for the operation of the drive recorder 100.

The memory IF 138 is a connection interface for mounting a removable memory card 150. The memory card 150 is a non-volatile memory, and for example, a flash memory is used. The memory control unit 137 executes memory control for writing image data to the memory card 150 mounted on the memory IF 138. That is, the memory control unit 137 functions as a write control unit for writing image data to the memory card 150. Specific memory control will be described later.

The input / output IF 139 is an external device that receives a signal from an external device and delivers it to the system control unit 131, or receives a control signal such as a signal request to the external device from the system control unit 131 and transmits it to the external device. Connection interface. The acceleration signal from the acceleration sensor 160 and the distance signal from the distance sensor 170 described above are input to the system control unit 131 via the input / output IF 139. Therefore, the input / output IF 139 cooperates with the system control unit 131 to function as an acceleration signal acquisition unit when receiving an acceleration signal, and cooperates with the system control unit 131 when receiving a distance signal. It also functions as a distance signal acquisition unit.

The system control unit 131 determines that an event such as a collision has occurred when the acceleration signal input from the acceleration sensor 160 via the input / output IF 139 has a magnitude equal to or larger than a preset threshold value. Therefore, the system control unit 131 also functions as an event signal acquisition unit.

Further, the system control unit 131 detects an object such as another vehicle or a person who has approached a distance less than a predetermined threshold value from the own vehicle 900 based on the distance signal input from the distance sensor 170 via the input / output IF 139. .. Therefore, the system control unit 131 also functions as an object detection unit.

The license plate detection unit 140 receives the frame image data that has been subjected to predetermined preprocessing by the image processing unit 135, and detects whether or not the license plate exists in the frame image. The received frame image data includes the time information in which the frame image was captured. The license plate detection unit 140 stores, for example, information on shapes, numbers, and characters unique to the license plate, their arrangement, and color, and detects the license plate by matching the frame image with the stored information. To do. When a specific detection target area is specified for the entire area of the frame image, it is detected whether the license plate exists in that area. When the license plate is detected, the license plate detection unit 140 outputs the time information in which the frame image is captured to the system control unit 131. At this time, the license plate detection unit 140 may output the information such as numbers, characters, and colors detected by matching to the system control unit 131.

The system control unit 131 is, for example, a CPU, and directly or indirectly controls each element constituting the drive recorder 100. The control by the system control unit 131 is realized by a control program or the like read from the system memory 134.

FIG. 3 is a conceptual diagram illustrating a ring buffer set in the memory card 150. The drive recorder 100 processes the images sequentially captured by the camera unit 110 by the image processing unit 135 to generate a moving image file every minute. Then, the memory control unit 137 sequentially records the moving image file generated by the image processing unit 135 on the memory card 150 via the memory IF 138.

Since the memory card 150 has an upper limit on the storage capacity, there is also an upper limit on the number of moving image files that can be recorded. Since the drive recorder 100 continues to generate the moving image file while the own vehicle 900 continues to travel, the latest moving image file generated cannot be recorded on the memory card 150 after a certain period of time has passed. Therefore, when the memory card 150 reaches the upper limit of the storage capacity, the memory control unit 137 continues the recording process in the ring buffer format in which the latest moving image file is overwritten in the storage area in which the oldest moving image file is stored. ..

FIG. 3A is a conceptual diagram when the entire memory area 151 of the memory card 150 is used as the ring buffer 152. When the memory areas that can store one moving image file are represented as X1, X2, ..., Xn, respectively, the memory control unit 137 records the first moving image file as X1, the next moving image file as X2, and so on. .. Then, when the nth moving image file is recorded in Xn, the n + 1th moving image file is overwritten with X1 in which the first moving image file is recorded. Similarly, the n + 2nd moving image file is overwritten with X2. If the moving image file is recorded in such a ring buffer format, the latest moving image file corresponding to the capacity of the ring buffer 152 can be retained.

FIG. 3B is a conceptual diagram illustrating the concept of write control in a conventional drive recorder. The conventional drive recorder recognizes the occurrence of an event when, for example, is collided with another vehicle and receives a large acceleration signal, and copies the moving image file recording the time point to the overwrite-protected area. For example, as shown in the figure, if the moving image file recorded in the memory area X4 belonging to the ring buffer 152 includes the event occurrence time during the imaging period, the memory Xn is changed to the non-volatile buffer 153 and the moving image is concerned. Copy the image file here as an event recording file. The non-volatile buffer 153 is an area excluded from the storage area recorded in the ring buffer format, in other words, an area in which overwriting is prohibited. When the event occurs a plurality of times, Xn-1, Xn-2, ... Are changed to the non-volatile buffer 153 in order each time. That is, the ring buffer 152 decreases each time the non-volatile buffer 153 is added. The memory area of the non-volatile buffer 153 is used again as the ring buffer 152 when the memory card 150 is formatted or the target moving image file is erased according to the user's instruction. The capacity used as the non-volatile buffer 153 may be set in advance.

The conventional drive recorder is insufficient from the viewpoint of recording the state of the target vehicle after the event occurs. As shown in FIG. 3B, the moving image data including the time when the event occurs, for example, for one minute is only prohibited from being overwritten. When a collision with a vehicle occurs, the target vehicle is already in contact with the own vehicle at the time of detection, and the lower part of the target vehicle often enters the blind spot of the drive recorder. The image captured at this time does not show the lower part of the target vehicle, and may not include important information for identifying the target vehicle such as a license plate. The information obtained from such images is inadequate from the perspective of identifying the target vehicle.

Therefore, in the present embodiment, the moving image data including the period from the time when the event occurs to the time when the license plate is detected in the continuously captured image is copied to the non-volatile buffer 153. FIG. 3C is a conceptual diagram illustrating the concept of write control in the drive recorder 100 according to the present embodiment. The method of adding the non-volatile buffer 153, the method of moving the target moving image data, the recording method of the ring buffer 152, and the like are the same as those in FIG. 3B, but in the present embodiment, the target to be copied to the non-volatile buffer 153. The moving image file is different.

Similar to the example of FIG. 3B, it is assumed that the moving image file recorded in the memory area X4 belonging to the ring buffer 152 includes the time when the event occurs during the imaging period. The camera unit 110 continues imaging even after that, and the generated moving image files are sequentially written to X5, X6 ... The license plate detection unit 140 continuously acquires the frame image data constituting these moving image files, and detects that the frame image includes the license plate (license plate detection). Here, it is assumed that the moving image file recorded in the memory area X6 includes the frame image in which the license plate is detected for the first time after the event occurs. The time indicated by the time information of the frame image in which the license plate is detected for the first time after the event occurs is defined as the license plate detection time point.

In this case, the memory control unit 137 records three moving image files of the moving image file of X4 including the time when the event occurs, the moving image file of X5, and the moving image file of X6 including the time of detecting the number plate. Copy as a file to the non-volatile buffer 153. That is, the memory area from Xn-2 to Xn is changed to the non-volatile buffer 153, and the target moving image file is copied there. If another moving image file already exists in the non-volatile buffer 153, the non-volatile buffer 153 is added by avoiding the memory area in which the moving image file is recorded.

If the period until the license plate is detected in the continuously captured image is also left as prohibition of overwriting, the image data includes the license plate information of the target vehicle. An example thereof will be described. FIG. 4A shows the state immediately after the other vehicle 910 collides with the own vehicle 900 (immediately after the event occurs), and FIG. 4B shows the situation immediately after the other vehicle 910 separates from the own vehicle 900 and the license plate. It shows how 911 was detected.

When a collision as shown in FIG. 4A occurs, the system control unit 131 recognizes that the event has occurred by receiving an acceleration signal equal to or higher than the threshold value generated in connection with the collision. The image taken at this point is the range surrounded by the outer frame of FIG. 4 (a). That is, since the other vehicle 910 is close to the own vehicle 900, the lower part thereof is not shown. With only such an image, the license plate 911 of another vehicle 910 that was close at the time of the accident may not be detected, and the purpose of identifying the other vehicle 910 cannot be achieved.

However, if the own vehicle 900 and the other vehicle 910 are relatively separated by a certain distance or more after the accident and an image including the license plate 911 such as the range surrounded by the outer frame of FIG. 4B can be obtained, other The vehicle 910 can be identified ex post facto and also serves the above purposes. That is, the utility value of the image is greatly expanded. For example, even if the vehicle subsequently escapes to another vehicle 910, it is easy to identify the opponent. The image processing unit 135 embeds information on the numbers, characters, and colors of the license plate 911, and information such as the name of the automobile inspection registration office and the purpose of the vehicle that can be identified from them in the frame image to generate a moving image file. It may be added to the header part of the moving image file.

As described above, if the moving image file including the period from the event occurrence time to the license plate detection time is recorded without being overwritten, the value of the subsequent use of the moving image file is dramatically improved. In the present embodiment, a storage period is further added. FIG. 5 is an explanatory diagram illustrating a target file for which overwriting is prohibited with respect to the passage of time in the present embodiment. The figure shows the passage of time from left to right.

In the figure, it is assumed that an event occurs at time ts and a license plate is detected in the image at time ttf. The period from the time ts to the time tf is the main period. Then, a pre-period with the time tp retroactive from the time ts by a predetermined fixed time T1 as the start time and the time ts as the end time is set before the main period. Further, a post-period is set after the main period, with the time tf as the start time point and the time te as the end time point when a predetermined fixed time T2 has elapsed from the time tf.

Then, the moving image file including these three periods is set as an event recording file and is a file subject to overwriting prohibition. That is, these moving image files are copied to the non-volatile buffer 153. In this way, if the period before and after is included, it can be expected that the range of use of the image will be further expanded. For example, the moving image file of the preliminary period may show the approaching vehicle 910, which contributes to identifying the cause of the accident and confirming that the vehicle shown in the main period is the cause of the accident. .. In addition, the moving image file during the post-mortem period may show a rescuer approaching or another vehicle 910 escaping, which contributes to the identification of the persons concerned and the pursuit of criminal liability.

Next, the control flow of the drive recorder 100 will be described. FIG. 6 is a flow chart showing a control flow of the drive recorder 100. The flow starts when the preparation for starting the running of the own vehicle 900 is completed. Completion of the running start preparation of the own vehicle 900 is, for example, starting the engine or turning on the power of the own vehicle 900. Further, the drive recorder 100 may be operated at all times regardless of the state of the own vehicle. Further, when the flow starts, the image pickup by the camera unit 110 is started, and the moving image files sequentially generated by the image processing unit 135 are sequentially recorded in the memory card 150 in the ring buffer format by the memory control unit 137. Further, the system control unit 131 monitors the acceleration signal received from the acceleration sensor 160. The following processing is processing executed during such recording control during normal driving.

In step S101, the system control unit 131 determines whether or not an acceleration signal equal to or higher than the threshold value has been received from the acceleration sensor 160, that is, whether or not an event has occurred. If it is determined that the acceleration signal equal to or higher than the threshold value has been received, it is assumed that an event has occurred, and the process proceeds to step S102. If it is determined that the signal has not been received, the process proceeds to step S108.

When the system control unit 131 proceeds to step S102, the system control unit 131 determines the history of the distance signal received from the distance sensor 170. As described above, since the distance sensors 170 are provided at a plurality of locations in front of the own vehicle 900, there is a possibility that any of the distance sensors 170 captures the approach of the target vehicle. Therefore, if the system control unit 131 determines that any of the distance sensors 170 is capturing the approach of the object, the system proceeds to step S104, and if not, proceeds to step S103.

When the system control unit 131 proceeds to step S103, whether or not the license plate detection unit 140 has a license plate for the entire area of each frame image of the moving image data generated after the event occurs. To search for. On the other hand, when the process proceeds to step S104, the license plate detection unit 140 detects the area in the direction of the approaching object captured by the distance sensor 170 in each frame image of the moving image data generated after the event occurs before the event occurs. Focus on searching. For example, if the distance sensor 170 that detects an approaching object is installed so as to detect the distance with respect to the field of view on the right side 1/4 of the image, the license plate detection unit 140 is located on the right side 1 of the image. / 4 is defined as the search area. If the target area for license plate detection is limited in this way, the license plate detection process can be performed at high speed, and it can be expected that the license plate detection process is closely related to the event generated by another vehicle equipped with the license plate.

In step S105, the system control unit 131 determines whether or not the license plate can be detected from the frame image. If it is determined that the detection was possible, the process proceeds to step S106, and if it is determined that the detection is not possible, the process proceeds to step S107.

When the system control unit 131 proceeds to step S106, as described in FIG. 5, the system control unit 131 determines the main period and the pre-period or the pre-period accompanying the main period or the post-period, and the memory control unit 137 responds to these. The moving image file corresponding to the period of is copied to the non-volatile buffer 153 which is the overwriting prohibited area. That is, the system control unit 131 copies the moving image file including the prior period and the main period to the non-volatile buffer 153, which is the overwriting prohibited area, as the process of step S106. Alternatively, as the process of step S106, the system control unit 131 copies the moving image file including the main period and the post period in addition to the pre-period to the non-volatile buffer 153 which is the overwriting prohibited area.

When the system control unit 131 proceeds to step S108, it determines whether or not the recording stop signal has been received. The recording stop signal is, for example, a signal generated when the running of the own vehicle 900 ends, and is also a signal generated when the user operates the stop button. When the system control unit 131 determines that the recording stop signal has been received, the system control unit 131 stops the imaging by the camera unit 110, stops the recording of the generated moving image file in the ring buffer format, and ends a series of processes. If it is determined that the recording stop signal has not been received, the process returns to step S101 to continue a series of processes.

If the license plate cannot be detected in step S105, the system control unit 131 copies the moving image file for a predetermined period from the occurrence of the event to the non-volatile buffer 153, which is an overwriting prohibited area, and proceeds to step S106. For example, if the license plate cannot be detected until 3 minutes have passed since the event occurred, the system control unit 131 copies the moving image file up to that point as an event recording file to the non-volatile buffer 153, which is an overwriting prohibited area. .. Specifically, if the license plate cannot be detected in step S105, the process proceeds to step S107 to determine whether a predetermined time has elapsed since the event occurred. If it is determined that the predetermined time has not elapsed, the process returns to step S102, and the license plate is continuously detected from the frame image. When it is determined that the predetermined time has elapsed, the process proceeds to step S106.

If the license plate cannot be detected in step S105, the system control unit 131 copies the moving image file to the non-volatile buffer 153 with overwrite prohibition until the storage capacity of the memory card 150 reaches the upper limit. May be good. Further, when performing such recording processing, the moving image file of the section in which the image has changed may be copied to the non-volatile buffer 153 which is the overwriting prohibited area.

Specifically, when the system control unit 131 cannot detect the license plate in step S105, whether or not there is no change in the image in the latest moving image file while continuing the recording of the generated moving image file. Analyze. Specifically, the frame images before and after are compared, and it is determined whether or not they match. Then, if the frames before and after the entire frame match, it is determined that there is no change in the image in the moving image file. If there is a frame image that does not match even a part, it is judged that the image has changed. If there is only a change in a part of the frame image that is relatively narrow, it may be determined that there is no change as a whole.

When the system control unit 131 determines that there is no change in the image in the latest moving image file, the system control unit 131 causes the memory control unit 137 to delete the image file in the section where there is no change from the memory card 150. On the other hand, when it is determined that the image has changed, the memory control unit 137 records whether the ring buffer 152 of the memory card 150 is exhausted, that is, the moving image file of the ring buffer 152 generated before the event occurs is recorded. Determine if the memory area has been used up. If it is determined that the file has been used up, it is assumed that there is no memory area for recording a new moving image file, and the series of processes is terminated. At this time, the memory control unit 137 may change the entire memory area 151 of the memory card 150 to the non-volatile buffer 153. On the other hand, when it is determined that the memory area is not used up, the process returns to step S102 and continues the search for the license plate for the frame images sequentially generated.

By adding such a process, the state after the event occurs can be continuously recorded until the ring buffer 152 is exhausted, so that the remaining moving image file can be used more effectively. Further, since the image file in which the subject does not move is deleted, the moving image file can be recorded until a later period, and the moving image file having high utility value can be left.

[Embodiment 2]
In the first embodiment of the present invention, an example has been described in which the period from the end point to the elapse of a predetermined time is set as the ex post facto period. In the second embodiment, an example of determining the ex post facto period based on the relative arrangement relationship of the target vehicles in the frame image will be described.

FIG. 7 is a block diagram showing a configuration of a drive recorder 200, which is an example of the image recording device according to the second embodiment. The drive recorder 200 further includes an arrangement detection unit 141 in the main body unit 130 included in the drive recorder 100 according to the first embodiment. Other configurations are the same as the configuration of the drive recorder 100. Therefore, common functions or processes are designated by the same reference numerals, and description thereof will be omitted as appropriate.

The arrangement detection unit 141 receives the frame image data that has been subjected to predetermined preprocessing by the image processing unit 135, and detects whether the other vehicle 910 has a predetermined arrangement relationship in the frame image. For example, as shown in FIG. 8A, the arrangement detection unit 141 detects whether the road surface 920 is recognizable between the other vehicle 910 and the own vehicle 900 in the frame image. In this case, the arrangement detection unit 141 detects, for example, a pixel region where the movement vector between the preceding and following frame images is zero between the license plate 911 detected by the license plate detection unit 140 and the own vehicle 900. .. If such a pixel region can be detected, the road surface 920 can be recognized between the other vehicle 910 and the own vehicle 900. When the road surface 920 can be recognized between the other vehicle 910 and the own vehicle 900, the arrangement detection unit 141 outputs the time information in which the frame image is captured to the system control unit 131.

The arrangement detection unit 141 may also detect whether the side surface portion 912 of the other vehicle 910 can be recognized, for example, as shown in FIG. 8B. In this case, the arrangement detection unit 141 stores, for example, the shape pattern and the color pattern when the vehicle is viewed from different angles for various vehicle types, and the license plate 911 detected by the license plate detection unit 140 is stored. The image of the other vehicle 910 provided is identified by matching and tracked on the frame image. When it is determined that the image of the other vehicle 910 is similar to the side surface portion of the vehicle stored in advance, the time information in which the frame image is captured is output to the system control unit 131. At this time, the arrangement detection unit 141 may also output the information of the vehicle type detected by matching to the system control unit 131.

In the second embodiment, the period from the detection of the license plate 911 by the license plate detection unit 140 to the output of the time information to the system control unit 131 by the arrangement detection unit 141 is set as the post-period, and the pre-period, the main period, and the like. The moving image file including the three post-periods is copied to the non-volatile buffer 153 as an event recording file. In this way, if the period until the target vehicle has a predetermined arrangement relationship with the own vehicle is specified as the post-period, it can be expected that the range of use of the image will be further expanded. That is, since the other vehicle 910 can be captured as an image in a wider range, it can be expected that, for example, the vehicle type of the other vehicle 910 and features such as deformation and scratches are also recorded. This information is very useful, for example, in a situation where the related vehicle cannot be identified only from the license plate 911. The image processing unit 135 may generate a moving image file by embedding information such as the vehicle type and color of another vehicle 910 in the frame image, or may add it to the header portion of the moving image file.

Next, the control flow of the drive recorder 200 will be described. FIG. 9 is a flow chart showing a control flow of the drive recorder 200. In the flow chart of FIG. 9, step S106 in the flow chart of FIG. 6 is deleted, and steps S205, S206, and S207 are added instead. Therefore, the same reference numerals are given to the processes common to the flow chart of FIG. 6, and the description thereof will be omitted as appropriate.

In step S105, the system control unit 131 proceeds to step S205 if it determines that the license plate can be detected from the frame image, and proceeds to step S107 if it determines that the license plate cannot be detected. When the system control unit 131 proceeds to step S205, it determines whether or not the other vehicle 910 has a predetermined arrangement relationship in the frame image. If it is determined that the arrangement relationship is determined in advance, the process proceeds to step S206. If it is determined that the arrangement relationship is not determined in advance, the process proceeds to step S207.

When the system control unit 131 proceeds to step S207, it determines whether a predetermined time has elapsed since the event occurred. If it is determined that the predetermined time has not elapsed, the process returns to step S102, and the license plate is continuously detected from the frame image. When it is determined that the predetermined time has elapsed, the process proceeds to step S206.

When the system control unit 131 proceeds to step S206, it determines the main period and the prior period as described in the first embodiment, and detects that the other vehicles 910 have a predetermined arrangement relationship in the frame image. The post-period is determined so that the end point is the time. In response to this, the memory control unit 137 copies the moving image file corresponding to these three periods to the non-volatile buffer 153 which is the overwriting prohibited area.

If the system control unit 131 does not detect in step S205 that the other vehicle 910 has a predetermined arrangement relationship, the system control unit 131 transfers the moving image file for a predetermined period from the license plate detection to the non-volatile buffer 153 which is the overwriting prohibited area. Copy and proceed to step S206. For example, when the system control unit 131 does not detect that the other vehicle 910 has a predetermined arrangement relationship, the moving image file from the license plate detection until 3 minutes have passed is used as an event recording file in the overwriting prohibited area. Copy to non-volatile buffer 153. Specifically, if the license plate cannot be detected in step S205, the process proceeds to step S207 to determine whether a predetermined time has elapsed since the event occurred. If it is determined that the predetermined time has not elapsed, the process returns to step S102, and the license plate is continuously detected from the frame image. When it is determined that the predetermined time has elapsed, the process proceeds to step S206.

If the system control unit 131 does not detect in step S205 that the other vehicle 910 has a predetermined arrangement relationship in the frame image, the moving image is obtained until the storage capacity of the memory card 150 reaches the upper limit. The file may be overwritten and copied to the non-volatile buffer 153. Further, when performing such recording processing, the moving image file of the section in which the image has changed may be copied to the non-volatile buffer 153 which is the overwriting prohibited area.

Specifically, when the system control unit 131 cannot detect the license plate in step S205, whether or not there is no change in the image in the latest moving image file while continuing the recording of the generated moving image file. Analyze. Specifically, the frame images before and after are compared, and it is determined whether or not they match. Then, if the frames before and after the entire frame match, it is determined that there is no change in the image in the moving image file. If there is a frame image that does not match even a part, it is judged that the image has changed. If there is only a change in a part of the frame image that is relatively narrow, it may be determined that there is no change as a whole.

When the system control unit 131 determines that there is no change in the image in the latest moving image file, the system control unit 131 causes the memory control unit 137 to delete the image file in the section where there is no change from the memory card 150. On the other hand, when it is determined that the image has changed, the memory control unit 137 records whether the ring buffer 152 of the memory card 150 is exhausted, that is, the moving image file of the ring buffer 152 generated before the event occurs is recorded. Determine if the memory area has been used up. If it is determined that the file has been used up, it is assumed that there is no memory area for recording a new moving image file, and the series of processes is terminated. At this time, the memory control unit 137 may change the entire memory area 151 of the memory card 150 to the non-volatile buffer 153. On the other hand, when it is determined that the memory area is not used up, the process returns to step S102 and continues the determination as to whether or not the other vehicle 910 has a predetermined arrangement relationship with respect to the frame images sequentially generated.

By adding such a process, the state after the event occurs can be continuously recorded until the ring buffer 152 is exhausted, so that the remaining moving image file can be used more effectively. Further, since the image file in which the subject does not move is deleted, the moving image file can be recorded until a later period, and the moving image file having high utility value can be left.

Although the two embodiments have been described above, some modifications will be described. FIG. 10 is a schematic view showing how another drive recorder 100'is installed in the own vehicle 900'. The own vehicle 900'is different from the above-mentioned own vehicle 900 in that a plurality of distance sensors 170' are provided behind the own vehicle 900, and the drive recorder 100'is different from the above-mentioned drive recorder 100 in that, in addition to the camera unit 110, It differs in that it is equipped with a camera unit 110'that images the surrounding environment behind. According to the own vehicle 900'and the drive recorder 100' configured in this way, it is possible to deal with an object colliding from behind.

In this case, if the memory IF 138 is configured so that two memory cards 150 can be mounted, one memory card 150 can be used for the front and the other memory card 150 can be used for the rear. Of course, the memory area of one memory card 150 may be divided into front and rear and used.

If the direction of the acceleration detected by the acceleration sensor 160 can also be detected, the system control unit 131 can determine the direction in which the object collides with the own vehicle 900'from the acceleration signal. When the system control unit 131 determines that the object has collided from the front, the system control unit 131 executes the above-mentioned write control accompanying the occurrence of the event on the image data acquired from the camera unit 110. On the other hand, when it is determined that the object has collided from behind, the above-mentioned write control accompanying the occurrence of the event is executed on the image data acquired from the camera unit 110'. That is, the system control unit 131 extracts the direction information regarding the direction in which the event occurs with respect to the own vehicle 900'from the acceleration signal, and based on the direction information, the target to execute the number plate detection is the camera unit. Select whether to use image data from 110 or image data from the camera unit 110'. Then, the selected image data is acquired and the above-mentioned writing control is executed. According to the own vehicle 900'and the drive recorder 100' configured in this way, the information of the target vehicle caused by the event can be recorded more accurately.

In the above-described embodiment, the memory area 151 is divided into the continuous ring buffer 152 and the continuous non-volatile buffer 153, but of course, it does not have to be physically continuous. Further, in the above embodiment, a mode in which a part of the ring buffer 152 is changed to the non-volatile buffer 153 and the target moving image file is copied to the non-volatile buffer 152 has been described, but the write control for prohibiting overwriting is this. Not limited to. For example, by setting an overwrite prohibition flag in the memory area in which the target moving image file is recorded, that area can be treated as the non-volatile buffer 153. In this case, the copying process of the moving image file can be omitted.

Further, in the above embodiment, the example in which the memory area 151 of one memory card 150 is divided into the ring buffer 152 and the non-volatile buffer 153 has been described, but the memory card 150 used as the ring buffer 152 and the non-volatile buffer 153 are used. It is also possible to configure the memory card 150 to be used to be installed. Further, instead of using the removable memory card 150, the memory mounted on the main unit 130 may be used. Further, the memory IF 138 may be configured as a wireless IF, and the above-mentioned write control may be executed for memories that are not physically adjacent to each other.

Further, in the above embodiment, for example, a one-minute moving image file has been described as one unit of writing control, but one unit of writing control is not limited to one moving image file. For example, write control may be performed on a frame-by-frame basis. Further, the target image data is not limited to moving image data, and may be, for example, still image data taken at intervals.

For example, when writing control is performed on a frame-by-frame basis, a new moving image file is generated by cutting out a file of the target period from the already generated moving image file of, for example, 1 minute, and this is used as an event recording file. It may be recorded in a non-volatile buffer. In this case, it is possible to leave a moving image file starting from the time when the event occurs. Of course, even when the advance period is added, the time retroactive to this starting point by a predetermined time can be set as the start time of the advance period. By managing the starting point in this way, regardless of the normal moving image file time, for example, a moving image file consisting of 10 seconds before the event occurs and 10 seconds after the event occurs is surely left as an event recording file. be able to.

Further, in the above embodiment, the acceleration sensor 160 is adopted as the sensor for detecting the occurrence of the event, but other sensors may be used. For example, it may be a strain sensor that detects deformation due to collision of an object, or it may be a temperature sensor that detects an abnormal temperature. Of course, a plurality of sensors may be used in combination. Further, the sensor such as the acceleration sensor 160 may be incorporated in the drive recorder.

Further, in the above embodiment, the acceleration sensor 160 outputs an acceleration signal to the system control unit 131, and the system control unit 131 determines whether or not the acceleration signal corresponds to an event signal indicating the occurrence of an event. Although adopted, the method by which the system control unit 131 acquires the event signal is not limited to this. For example, a threshold value may be set in advance for the output value of the acceleration sensor 160, and only acceleration signals equal to or higher than the threshold value may be output to the system control unit 131. In this case, the time when the system control unit 131 receives the acceleration signal can be set as the time when the system control unit 131 acquires the event signal. Of course, the same configuration may be incorporated into the other sensors described above and adopted.

Further, in the processing flow described with reference to FIGS. 6 and 9, the history of the distance signal from the distance sensor 170 is used, but the method of acquiring the information before the event detection is not limited to this. By using the moving image file before the event detection, it is possible to detect the approach of the object by, for example, monitoring the movement vector of the subject image. Further, the branch flow for intensively searching for a region in a specific direction may be omitted depending on the ability of the license plate detection unit 140 and the like. When the branch flow is omitted, for example, steps S102 and S104 in FIG. 6 are omitted.

Further, in the above embodiment, the configuration in which the distance sensor 170 is installed in the own vehicle 900 and the system control unit 131 acquires the distance signal from the distance sensor 170 has been described. However, for example, the camera unit 110 is used as a compound eye. , The distance from the compound eye image to the object may be calculated. In this case, since the connection with the distance sensor 170 can be omitted, the device configuration becomes simpler.

Further, in the above embodiment, the system control unit 131 has described the configuration of detecting the license plate 911 only from the image data, but for example, the own vehicle 900 and the other vehicle 910 are described from the distance signal acquired from the distance sensor 170. The size of the image of the license plate 911 may be estimated from the calculated distance, and the information may be used for image matching. In this case, the license plate can be detected faster and more reliably.

Further, in the above embodiment, the license plate detection unit 140 may detect only the license plate photographed with a size larger than a predetermined size in the image data. In this case, it is possible to reduce the possibility of erroneously detecting the license plate of another vehicle other than the colliding other vehicle 910.

Further, in the above embodiment, as shown in FIGS. 4 and 8, an example in which vehicles collide head-on with each other has been described. However, for example, when the own vehicle 900 collides with another vehicle 910 from behind, it may also occur. The configuration of the present invention can be applied. In this case, the system control unit 131 detects the license plate provided behind the other vehicle 910.

Further, in the first embodiment, the time when T2 elapses for a certain period is defined as the end of the post-period, and in the second embodiment, the time when the target vehicle satisfies the predetermined arrangement relationship in the frame image is the end of the post-period. However, these conditions may be combined. For example, the post-mortem period may be the longer of T2 after the license plate is detected or until the target vehicle satisfies the predetermined arrangement relationship in the frame image. In this case, it is possible to record information such as the vehicle type of the target vehicle while sufficiently securing the recording time after the event occurs. In any case, the image file recorded so as not to be overwritten is useful in accident verification and the like.

100, 100', 200 drive recorder, 110, 110' camera unit, 112 lens, 114 image sensor, 116 AFE, 130 main unit, 131 system control unit, 132 image input IF, 133 work memory, 134 system memory, 135 image Processing unit, 136 display output unit, 137 memory control unit, 138 memory IF, 139 input / output IF, 140 NP detection unit, 141 placement detection unit, 142 bus line, 150 memory card, 151 memory area, 152 ring buffer, 153 non-volatile Buffer, 160 acceleration sensor, 170, 170'distance sensor, 180 display unit, 900, 900' own vehicle, 910 other vehicles, 911 number plate, 912 side surface, 920 road surface

Claims (7)

An image data acquisition unit that sequentially acquires image data that captures the surroundings of the own vehicle,
An event signal acquisition unit that acquires an event signal indicating the occurrence of an event for the own vehicle, and
A license plate detection unit that detects a license plate from an image of the image data acquired by the image data acquisition unit when the event signal acquisition unit acquires the event signal.
When the event signal acquisition unit acquires the event signal, the image data including the period from the time when the event signal is acquired to the time when the license plate detection unit detects the license plate is not overwritten. An image recording device including a write control unit for writing to a memory. The license plate detection unit detects a license plate taken with a size equal to or larger than a predetermined size from the image of the image data acquired by the image data acquisition unit.
The image recording device according to claim 1. The license plate detection unit detects the license plate from the region in the direction of the approaching object captured before the event signal acquisition unit acquires the event signal indicating the occurrence of the event .
The image recording apparatus according to claim 1 or 2. The write control unit writes the image data including the period from the time when the event signal is acquired to the time when the license plate detection unit detects the license plate to the memory so as not to be overwritten.
The image recording apparatus according to claim 1 or 2. The write control unit writes the image data to the memory in a ring buffer format when the event signal acquisition unit does not acquire the event signal, and when the event signal acquisition unit acquires the event signal, the write control unit writes the image data to the memory. The image data including the period from the time when the event signal is acquired to the time when the number plate detection unit detects the number plate from the time when it goes back by a predetermined period is written to the memory so as not to be overwritten.
The image recording apparatus according to any one of claims 1 to 4. An image data acquisition step that sequentially acquires image data that captures the surroundings of the own vehicle,
An event signal acquisition step for acquiring an event signal indicating the occurrence of an event for the own vehicle, and
When the event signal is acquired in the event signal acquisition step, the license plate detection step of detecting the license plate from the image of the image data acquired in the image data acquisition step, and the license plate detection step.
When the event signal is acquired in the event signal acquisition step, the image data including the period from the time when the event signal is acquired to the time when the number plate is detected in the number plate detection step is not overwritten. An image recording method in which an image recording device executes a write control step of writing to memory. An image data acquisition step that sequentially acquires image data that captures the surroundings of the own vehicle,
An event signal acquisition step for acquiring an event signal indicating the occurrence of an event for the own vehicle, and
When the event signal is acquired in the event signal acquisition step, the license plate detection step of detecting the license plate from the image of the image data acquired in the image data acquisition step, and the license plate detection step.
When the event signal is acquired in the event signal acquisition step, the image data including the period from the time when the event signal is acquired to the time when the number plate is detected in the number plate detection step is not overwritten. An image recording program that causes a computer to perform write control steps that write to memory.

Images