JP2012159955A - Image recording control method, image recognition device, and on-vehicle image recording device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain information which is useful to grasp whether a safe drive is being made by a driver at an intersection etc., on a road.SOLUTION: Based upon contents of image data in time series obtained by photographing a subject on a road etc., in front of or behind a local vehicle in the traveling direction by an on-vehicle camera, whether there is a traffic sign displayed on a road surface of the road and the kind of the traffic sign are recognized, and when the traffic sign is recognized, a travel state such as the speed of the local vehicle is monitored. Based upon the kind of the traffic sign and the travel state of the local vehicle, whether the importance of the state is high or low is automatically discriminated, and the discriminated high/low level of the importance of the state is reflected on recording control over information including the image data. An important image etc., with which it can be confirmed whether a driver does not follow the contents of the traffic sign and makes a violation against a temporary stop etc., can be automatically recorded.

Description

  The present invention relates to an image recording control method and an image recognition device for controlling a recording start trigger of information including time-series image data obtained by photographing with a vehicle-mounted camera or switching of predetermined identification information added to the information. The present invention also relates to an in-vehicle image recording apparatus.

  Conventionally, a drive recorder has been used as a vehicle-mounted device that automatically records image data representing the actual operation status of a vehicle. A drive recorder is generally an in-vehicle camera fixed on the vehicle, and continuously shoots subjects that can be seen from the front or rear window in the direction of travel of the vehicle, that is, road surfaces, signs, leading vehicles, oncoming vehicles, etc. The obtained image data is sequentially recorded on a predetermined recording medium.

  As for this type of drive recorder, there are a constant recording type in which image data is always recorded and a trigger recording type in which an image is recorded only in a special situation such as a traffic accident. In the case of a trigger recording type drive recorder, this is regarded as a recording start trigger when a specific condition is met, as in the case of detecting an abnormally large acceleration, and an image is recorded for a predetermined time before and after the trigger generation timing.

  Conventionally, a digital tachograph has been used as a vehicle-mounted device mounted on a business vehicle or the like. A digital tachograph is a time series that includes information that indicates the actual operation status of a vehicle equipped with the tachograph, specifically, vehicle position information, vehicle speed, brake status, turn signal status, etc. Data is sequentially recorded on a predetermined recording medium. The drive recorder described above can be connected to a digital tachograph, and the function of the drive recorder can be built in the digital tachograph.

  By the way, there is an intersection on the road as a place where an automobile traffic accident is likely to occur. Therefore, Patent Document 1 discloses a technique for recording an image representing a driving situation not only when an impact is detected in a vehicle drive recorder but also when approaching an intersection. In Patent Literature 1, standard images of traffic lights, road markings, and road signs are stored in a memory, and an image processing unit performs image recognition by pattern matching processing between an image to be recognized captured by the imaging unit and the standard image. Then, it is determined whether or not the control unit has approached an accident-prone place such as an intersection. If the control unit has approached, the image recorded in the recording unit is transferred to the first recording / storing unit to perform the recording / storing operation. Do. Therefore, an image representing the driving situation at the intersection before the predetermined time range before and after the impact is detected can be saved, which is useful for analyzing the cause of the traffic accident.

  A similar technique is also disclosed in Patent Document 2. Patent Document 2 relates to an apparatus that stores video captured during driving as a video summary in association with map information. In Patent Document 2, a video signal taken while the vehicle is traveling is always input from a camera, and a standard image stored in advance by a guide sign detection unit is subjected to pattern matching, and a road guide sign is included in this video signal. If so, cut out only the road sign. Also, the video signal of the multi-value image is binarized by the binarization processing unit, and whether the binarized video signal includes characters or symbols such as a destination set in advance by the character symbol recognition unit Is detected. At this time, detection of road guidance signs and recognition of characters and symbols can also be started when the vehicle reaches a predetermined distance to the intersection based on the intersection information from the car navigation system. A video signal output from the camera is always encoded by an encoder, and at least a video for a predetermined time during running is stored in the drive video storage unit.

JP 2003-63459 A JP 2009-246503 A

  A drive recorder for a vehicle was shot within a predetermined time range before and after the time when a traffic accident actually occurred or when a dangerous situation was encountered and a large acceleration was detected. In general, image data is recorded and stored. In the case of a drive recorder that always records images, when a large acceleration is detected, an appropriate mark or flag corresponding to the occurrence of an accident or the like is recorded at the corresponding recording position of the time-series image data. .

  That is, a conventional drive recorder for a vehicle is used for the purpose of leaving image data that becomes evidence when a traffic accident occurs or for leaving image data useful for elucidating the cause of the traffic accident.

  On the other hand, for example, in a transportation company that owns commercial vehicles such as trucks, buses, taxis, etc., each crew member who is in charge of each vehicle must perform management and guidance so as to perform safe driving. Therefore, in many cases, a digital tachograph is mounted on each business vehicle, and various types of information representing the driving situation during operation are recorded as time series data. It is possible to grasp whether or not each crew member is driving safely by importing and analyzing the recorded time-series data into a management computer.

  However, the conventional digital tachograph simply acquires vehicle position information, vehicle speed, brake state, turn signal state, etc. periodically and records time series data including these on a predetermined recording medium sequentially. . Therefore, it is not easy for an administrator to analyze whether or not a safe driving is performed by analyzing a huge amount of time-series data recorded by a digital tachograph. Further, when a drive recorder is mounted in addition to the digital tachograph, image information at the time when a large acceleration is detected can be obtained, and this can be used for management of safe driving.

  However, for example, it is not possible to grasp whether or not the corresponding crew member has performed a driving operation including a temporary stop at an intersection where there is no traffic light and a temporary stop is required. For example, if the technique of Patent Document 1 is used, an image representing the driving situation when approaching an intersection can be recorded. However, when a large impact is not detected, there is a possibility that the image is not stored. Also, if all images are stored where there are specific signs or road markings, all images will be recorded even if the crew is driving without safety problems. As a result, the amount of information of the recorded image is enormous. Therefore, a large amount of image data must be analyzed in order to determine whether or not safe driving is being performed, which requires great effort.

  The present invention has been made in view of the above-described circumstances, and an object thereof is an image recording capable of obtaining information useful for grasping whether or not a crew member is performing safe driving at an intersection on a road or the like. A control method, an image recognition device, and an in-vehicle image recording device are provided.

In order to achieve the above-described object, the image recording control method according to the present invention is characterized by the following (1) to (4).
(1) An image recording control method for controlling a recording start trigger of information including at least time-series image data obtained by photographing with a vehicle-mounted camera or switching of predetermined identification information added to the information,
Recognize the presence and type of signs displayed on the road surface of the road based on the content of time-series image data obtained by shooting the subject in front of or behind the traveling direction of the vehicle with the in-vehicle camera,
When the sign is recognized, the driving state of the host vehicle is monitored, and based on the type of the sign and the driving state of the host vehicle, the level of importance of the situation is automatically identified,
Reflect the level of importance of the identified situation in the recording control of information including the image data.
(2) The image recording control method according to (1) above,
When identifying the level of importance of the situation, at least the presence or absence of a temporary stop of the host vehicle is automatically identified, and the presence or absence of the temporary stop is reflected in the recording control of information including the image data.
(3) The image recording control method according to (2) above,
When recognizing the white line of the stop line displayed on the road surface of the road intersection as the sign, calculate the distance from the vehicle to the stop line,
Based on the distance and the vehicle speed of the host vehicle, grasp the timing when the host vehicle is scheduled to approach the stop line,
When the host vehicle approaches the stop line, monitor the running state of the host vehicle and identify whether there is a temporary stop.
(4) The image recording control method according to (3) above,
When the host vehicle approaches the stop line, it is monitored whether or not the vehicle speed of the host vehicle is below a predetermined value for a predetermined time or more to identify whether there is a temporary stop.

According to the image recording control method of the above configuration (1), the crew member is temporarily at a place where a “pause” sign is displayed on the road surface (there is a sign) such as an intersection without a traffic light. When the user fails to stop, it is automatically recognized as a highly important situation, and a record such as image data can be saved. Therefore, information useful for grasping whether or not a crew member is driving safely at an intersection or the like on a road can be obtained.
According to the image recording control method having the configuration (2), information useful for grasping whether or not there is a violation related to “pause” can be obtained.
According to the image recording control method having the configuration (3), it is easy to specify the timing for identifying whether or not there is a violation related to “pause”.
According to the image recording control method having the configuration (4), it is possible to automatically identify whether there is a violation related to “pause”.

In order to achieve the above-described object, an image recognition apparatus according to the present invention is characterized by the following (5) to (8).
(5) An image recognition apparatus that generates a recording start trigger for information including at least time-series image data obtained by photographing with a vehicle-mounted camera or a control signal for controlling switching of predetermined identification information added to the information Because
A sign recognition unit for recognizing the presence and type of a sign displayed on the road surface based on the content of time-series image data obtained by photographing a subject in front of or behind the traveling direction of the host vehicle with an in-vehicle camera; ,
When the sign is recognized, the traveling state of the host vehicle is monitored, and the level of importance of the situation is automatically identified based on the type of the sign and the traveling state of the host vehicle. And an importance level identifying unit that outputs a control signal indicating a state corresponding to the.
(6) The image recognition device according to (5) above,
The importance identifying unit automatically identifies at least whether or not the own vehicle is temporarily stopped, and outputs a control signal indicating whether or not the identified own vehicle is temporarily stopped.
(7) The image recognition device according to (6) above,
The importance identifying unit calculates a distance from the own vehicle to the stop line when the sign recognition unit recognizes a white line of a stop line displayed on the road surface of a road intersection as the sign. Based on the speed of the vehicle, the timing when the host vehicle is scheduled to approach the stop line is monitored, and when the host vehicle approaches the stop line, the traveling state of the host vehicle is monitored to determine whether there is a temporary stop. Identify.
(8) The image recognition device according to (7) above,
The importance identifying unit monitors whether the vehicle speed of the host vehicle is below a predetermined level for a predetermined time or more when the host vehicle approaches the stop line, and identifies whether there is a temporary stop. .

According to the image recognition device having the configuration (5), it is possible to control a vehicle drive recorder, a digital tachograph, and the like using the control signal. Therefore, it is possible to obtain information such as images that are useful for grasping whether or not a crew member is driving safely at an intersection on a road.
According to the image recognition apparatus having the configuration (6), information useful for grasping whether or not there is a violation related to “pause” can be obtained.
According to the image recognition apparatus having the configuration (7), it is easy to specify the timing for identifying whether or not there is a violation related to “pause”.
According to the image recognition apparatus having the configuration (8), it is possible to automatically identify the presence or absence of a violation relating to “pause”.

In order to achieve the above-described object, the in-vehicle image recording apparatus according to the present invention is characterized by the following (9) to (10).
(9) An in-vehicle image recording apparatus that records information including at least time-series image data obtained by photographing with an in-vehicle camera,
A sign recognition unit for recognizing the presence and type of a sign displayed on the road surface based on the content of time-series image data obtained by photographing a subject in front of or behind the traveling direction of the host vehicle with an in-vehicle camera; ,
An importance identifying unit that monitors the traveling state of the host vehicle when recognizing the sign, and automatically identifies the level of importance of the situation based on the type of the sign and the traveling state of the own vehicle;
A recording control unit that controls a recording start trigger of information including the image data or switching of predetermined identification information added to the information in accordance with the level of importance identified by the importance identifying unit.
(10) The on-vehicle image recording device according to (9),
The importance identifying unit automatically identifies at least whether or not the own vehicle is temporarily stopped,
The recording control unit controls a recording start trigger of information including the image data or switching of predetermined identification information added to the information according to whether or not the host vehicle is temporarily stopped.

According to the in-vehicle image recording apparatus having the configuration (9), information useful for grasping whether or not a crew member is driving safely at an intersection or the like on a road can be obtained. The function of this in-vehicle image recording apparatus can be assumed to be incorporated in, for example, a drive recorder or a digital tachograph.
According to the in-vehicle image recording apparatus having the configuration (10), information useful for grasping whether or not there is a violation related to “pause” can be obtained.

  According to the image recording control method, the image recognition apparatus, and the in-vehicle image recording apparatus of the present invention, it is possible to obtain information useful for grasping whether or not a crew member is performing safe driving at an intersection on a road.

  The present invention has been briefly described above. Further, details of the present invention will be further clarified by reading through the modes for carrying out the invention described below with reference to the accompanying drawings.

It is a flowchart which shows the outline | summary of operation | movement of the image recognition apparatus of embodiment. It is a flowchart which shows the detail of a part of operation | movement shown in FIG. It is a block diagram showing an outline of functional composition of an image recognition device and an in-vehicle image recording device of an embodiment. It is a block diagram which shows the structural example of the hardware of the image recognition unit to which this invention is applied. It is a block diagram which shows the structural example of the hardware of the drive recorder to which this invention is applied. It is a block diagram which shows the structural example of the hardware of the digital tachograph to which this invention is applied. It is a top view which shows the example of the sign displayed on the vehicle and road surface on a road.

  Specific embodiments of the image recording control method, the image recognition apparatus, and the in-vehicle image recording apparatus of the present invention will be described below with reference to the drawings.

  An outline of functional configurations of the image recognition apparatus 10 and the in-vehicle image recording apparatus 20 of the present embodiment is shown in FIG. As shown in FIG. 3, the image recognition device 10 includes a sign recognition unit 12 and an importance level identification unit 13. The in-vehicle image recording device 20 further includes a recording control unit 14 in addition to the sign recognition unit 12 and the importance level identification unit 13.

  The in-vehicle camera 11 is mounted on a vehicle such as an automobile. For example, a predetermined range of scenery in the forward direction seen from a windshield glass in front of the driver's seat in the passenger compartment, that is, a road surface, a part of the body of the own vehicle. The vehicle is fixed to the vehicle body so that a preceding vehicle, an oncoming vehicle, or the like can be photographed as a subject.

  In addition, you may image | photograph landscapes, such as a road surface behind, through a back window glass, with the imaging | photography direction of the vehicle-mounted camera 11 facing back of the advancing direction of a vehicle. The in-vehicle camera 11 can always shoot during operation of the apparatus, can repeatedly shoot at a fixed frame period (for example, 30 frames / second), and sequentially output the taken images. .

  The sign recognition unit 12 processes image data output from the in-vehicle camera 11 to perform processing including pattern recognition, and recognizes signs and the like. In the present embodiment, the sign recognition unit 12 identifies the presence and type of various signs (referred to as signs) displayed on the road surface.

  Specifically, as a representative example, in order to be able to identify whether a place where a temporary stop is necessary, a white line (stop line) in the vicinity of an intersection without traffic lights on a road, a sign of a pedestrian crossing, , Recognize the presence and type of "stop" sign.

  FIG. 7 shows an example of a state where a vehicle displayed on a road and a sign displayed on a road surface are viewed from above. In the example shown in FIG. 7, the stop line 54 and the pedestrian crossing 55 are present on the road surface in front of the vehicle 52 traveling on the road 51, and the subject including the stop line 54 and the pedestrian crossing 55 sign is an in-vehicle camera. A case where the image is shot at 53 is assumed. Therefore, if the stop line 54 or the sign of the pedestrian crossing 55 is recognized based on an image obtained by photographing with the in-vehicle camera 53, it is possible to identify whether or not the place requires a temporary stop.

  In addition to signs related to “pause” on public roads, there are various signs such as safety zones, no entry, bus-only (or priority), bicycle-only, traffic classification designation, speed designation, etc. Existing. For these signs, the sign recognition unit 12 can identify the presence and type of the sign.

  The importance level identification unit 13 monitors the running state of the host vehicle when the sign recognition unit 12 recognizes the sign (sign), and the level of importance of the situation is determined based on the type of the sign and the running state of the host vehicle. Are automatically identified, and a control signal representing a state corresponding to the identified level of importance is output.

  For example, it detects the presence or absence of a “temporary stop violation” that is important for crew safety. In other words, in the vicinity of the stop line at the intersection where a temporary stop is required, it is determined whether the crew member is violating the type of sign and is not driving temporarily, and if an important violation is detected, A control signal is output to record the situation. As for the control signal output by the importance level identifying unit 13, a recording start trigger, the type of the detected sign, or the type of violation is assumed.

  The recording control unit 14 performs control for automatically recording important data including an image captured by the in-vehicle camera 11 and storing the important data on the nonvolatile data recording medium 15. For example, when the importance identifying unit 13 detects an important violation, according to the recording start trigger output from the importance identifying unit 13, a predetermined time before and after the trigger time (for example, from 15 seconds before the trigger to 5 seconds after the trigger) All the image data photographed within the range of) are recorded and stored in the data recording medium 15. In addition, information such as the type of the detected sign or violation, the time and the current position of the vehicle is also stored in the data recording medium 15.

  Note that the recording control unit 14 always records image data captured on a buffer memory (not shown) even when a trigger does not occur, and sequentially overwrites image data that has become older in time. Accordingly, since past image data from a predetermined time (for example, 15 seconds) to the present is always held, old image data obtained at a time before the occurrence of the trigger can also be stored in the data recording medium 15.

  When a medium having a very large storage capacity is used as the data recording medium 15, it is possible to save all captured image data on the data recording medium 15 regardless of whether a trigger is generated. In that case, when a trigger occurs, the position of the image data corresponding to the time, the type of detected sign or the type of violation, Information such as the time and the current position of the vehicle is also stored in the data recording medium 15.

  An outline of the operation of the image recognition apparatus 10 shown in FIG. 3 is shown in FIG. FIG. 2 shows details of a part of the operation shown in FIG. The operation shown in FIGS. 1 and 2 will be described below. When the image recognition apparatus 10 is turned on and the ignition switch of the vehicle is turned on, the processing shown in FIG. 1 is executed.

  In step S <b> 11, the sign recognition unit 12 acquires one frame of image data from the forward image in the traveling direction representing the latest situation captured by the in-vehicle camera 11. The image data of one frame includes a predetermined range of scenery in front of the driving shield glass in front of the driver's seat in the passenger compartment, such as the road surface and a part of the vehicle body (bonnet). And other subjects such as a preceding vehicle, an oncoming vehicle, and a pedestrian.

  In step S12, the sign recognition unit 12 processes the input image data of one frame, and searches for a road surface marking by pattern recognition. For example, the entire image frame is sequentially scanned to extract data of a region having a characteristic pattern shape, and the feature amount of the data of this region and various data registered in the database in the sign recognition unit 12 in advance. The feature quantity of the type of sign is compared, and the presence or absence of a matching sign is specified. If there is a matching sign, the process proceeds to the next step S13. If there is no matching sign, the process returns to step S11.

  In step S13, the type of the sign detected in step S12 is recognized. That is, the type of the label that has the most similar feature amount to the corresponding image pattern is specified from various types of marks registered in advance. For example, in the example shown in FIG. 7, the stop line 54 and the pedestrian crossing 55 are detected as independent signs. As shown in FIG. 7, a plurality of types of signs may be detected simultaneously.

  In step S14, the importance level identifying unit 13 executes processing corresponding to the type of label recognized in step S13. Specifically, the driving situation is monitored to automatically identify whether or not the violating action does not follow the indication instruction or the action having a problem in safe driving, and the type of violation. This process will be described later in detail.

  In step S15, the result of the process in step S14 is referred to and whether or not a violation with high importance is detected is identified. If a violation with high importance is detected, the process proceeds to the next step S16. If a violation with low importance is detected, the process returns to step S11.

  For example, traffic accidents are likely to occur at intersections where there are no traffic lights, so crew members who drive vehicles should stop at this place according to signs such as stop lines and crosswalks displayed on the road surface near the intersection. It may be required to do. When driving in a way that violates this, it is considered that the crew is likely to cause a traffic accident in the future, so the importance identifying unit 13 identifies the violation as having a high importance in safety management. To do.

  In step S16, the importance level identifying unit 13 outputs a recording start trigger and information indicating the type of violation detected in order to save data such as an image captured by the in-vehicle camera 11 in the current situation. Information indicating the recording start trigger and the type of violation output from the importance identifying unit 13 is input to the recording control unit 14 and used for recording control of images and the like.

  FIG. 2 shows the details of the processing related to step S14 in FIG. The process of FIG. 2 will be described below.

  In step S21, the importance level identifying unit 13 identifies whether or not the label recognized in step S13 of FIG. In the case of a sign indicating “pause”, the processes in and after step S22 are executed. In other cases, a process (not shown) corresponding to the type of the corresponding sign is performed.

  In step S22, an actual distance L1 from the current vehicle position to a predetermined position on the road where the vehicle should be temporarily stopped, such as a white line of a stop line, is calculated based on the content of the captured image. That is, in the frame of the captured image, a distance L2 from the position of the front end of the host vehicle to the position of the specific sign recognized is obtained, and the distance L2 is converted into an actual distance L1.

  That is, information such as the height of the position where the in-vehicle camera 11 is attached, the inclination in the photographing direction, the distance from the installation position of the in-vehicle camera 11 to the vehicle tip can be stored in advance as known constant data. The actual distance L1 can be calculated from the distance L2 using these constant data.

  In step S23, information on the current vehicle speed output from the vehicle side is input, and based on the distance L1 and the vehicle speed, a time when the host vehicle travels and reaches a position to be stopped such as a stop line is calculated. .

  In step S24, the time calculated in step S23 is compared with the current time to identify whether or not the front end of the host vehicle has approached the position of the stop line. For example, when approaching within a predetermined distance, the process proceeds to step S25, and when not approaching yet, the process returns to step S22.

  In step S25, the current vehicle speed of the host vehicle is compared with a predetermined threshold th1. As a specific example of the threshold th1, “2 km / h” is assumed. If the condition “vehicle speed <th1” is satisfied, the process proceeds to step S26, and if not, the process proceeds to step S27.

  In step S26, the elapsed time after the condition “vehicle speed <th1” is satisfied in step S25 is monitored, and this elapsed time is compared with a predetermined threshold th2. As a specific example of the threshold th2, “1 second” is assumed. If the condition of “elapsed time> th2” is satisfied, the process proceeds to step S26, and if not, the process returns to step S25.

  In step S27, the current time and the planned arrival time calculated in step S23 are compared to identify whether or not the time has been exceeded. That is, before detecting the temporary stop state of the host vehicle, it is identified in step S27 whether or not the host vehicle has passed the position of the stop line. If exceeded, the process proceeds to step S29, and if not exceeded, the process returns to step S25.

  In step S <b> 28, the importance level identifying unit 13 recognizes that there is no violation of the pause for the currently detected pause indication. In step S29, the importance level identifying unit 13 recognizes that there is a violation of the pause for the currently detected pause indication.

  Realistic configuration examples of the image recognition apparatus 10 and the in-vehicle image recording apparatus 20 shown in FIG. 3 are shown in FIGS. 4, 5, and 6, respectively. That is, in the configuration example shown in FIG. 4, a system corresponding to the in-vehicle image recording device 20 is configured by combining the image recognition unit 100 corresponding to the image recognition device 10 of FIG. 3 and the independent drive recorder 200. . In the configuration example shown in FIG. 5, it is assumed that functions corresponding to the image recognition device 10 and the in-vehicle image recording device 20 in FIG. 3 are built in the drive recorder 200B. Further, in the configuration example illustrated in FIG. 6, it is assumed that functions corresponding to the image recognition device 10 and the in-vehicle image recording device 20 illustrated in FIG. 3 are incorporated in the digital tachograph 300.

  First, the configuration example shown in FIG. 4 will be described. The image recognition unit 100 shown in FIG. 4 includes a video amplifier 101, an NTSC decoder 102, an image processing circuit 103, a microcomputer 104, memories 105 and 106, and a power supply circuit 107 as main components.

  4 includes main decoder 201, data processing unit 202, communication processing unit 203, memories 204 and 205, vehicle signal interface 206, memory card 207, audio output unit 208, and monitor as main components. An output unit 209 is provided.

  The video amplifier 101 receives an NTSC (National Television System Committee) standard video signal output from the in-vehicle camera 11 and outputs a video signal amplified to a required level. The output of the video amplifier 101 is connected to the video input of the drive recorder 200 and the input of the NTSC decoder 102.

  The NTSC decoder 102 processes the NTSC video signal input from the video amplifier 101, extracts a plurality of signal components (Y, Cb, Cr) included therein, performs A / D conversion, and performs image processing. A digital image signal that can be used for the above is generated.

  The image processing circuit 103 is a dedicated large-scale integrated circuit (LSI) having a function for performing predetermined image processing such as pattern recognition. The image processing circuit 103 is necessary for detecting a pattern such as a sign displayed on the road surface in step S12 shown in FIG. 1 or identifying the type of the detected pattern based on the input image data. It has various functions.

  The image processing circuit 103 is connected to a memory 105 composed of ROM and a memory 106 composed of RAM. The memory 105 holds in advance a database in which information representing the characteristics of various patterns used by the image processing circuit 103 for pattern matching and various constant data are registered. The memory 106 is used to temporarily hold data such as an image processed by the image processing circuit 103.

  The microcomputer 104 is equipped with a CAN (Controller Area Network) standard communication function, and performs communication with the drive recorder 200 connected to the communication terminal of the image recognition unit 100. Specifically, when the image processing circuit 103 detects a sign in step S12 of FIG. 1, information indicating the vehicle state such as the vehicle speed is acquired via the drive recorder 200, and the vehicle state is monitored. The presence or absence and type of violation are automatically identified in step S14. When a violation with high importance is detected, the microcomputer 104 notifies the drive recorder 200 of a recording start trigger and information indicating the detected violation type.

  In the drive recorder 200, the main decoder 201 processes the NTSC standard video signal output from the video amplifier 101 in the image recognition unit 100, and a plurality of signal components (Y, Cb, Cr) included in the video signal are processed. Each is extracted and subjected to A / D conversion to generate a digital image signal that can be processed by the data processing unit 202.

  The data processing unit 202 includes a microcomputer, and mainly performs processing for sequentially inputting image data output from the main decoder 201, compressing and recording the image data. A memory (ROM) 204 connected to the data processing unit 202 holds in advance programs executed by the data processing unit 202, various constant data used by the data processing unit 202, and the like. A memory (RAM) 205 is used to hold temporary data such as data being processed. Data such as an image finally recorded by the drive recorder 200 is stored on a memory card (for example, an SD card) 207 equipped with a nonvolatile memory.

  The communication processing unit 203 is configured by a microcomputer having a CAN standard communication function, and communicates with the image recognition unit 100 connected thereto via a communication network (CAN) on the vehicle. Do. Further, bidirectional communication is performed between the communication processing unit 203 and the data processing unit 202, and information is transferred as necessary.

  The vehicle signal interface 206 inputs various signals output from the vehicle side and converts them into signals that can be processed by the data processing unit 202, or converts various signals output from the data processing unit 202 into signals that can be processed by the vehicle side. Process to do. Signals handled by the vehicle signal interface 206 include trigger output, ignition (IGN), manual switch (SW), brake, turn signal L side, turn signal R side, vehicle speed pulse, and the like.

  In the system shown in FIG. 4, if the image recognition unit 100 recognizes a sign on the road surface such as a stop line while the vehicle is running based on the video taken by the in-vehicle camera 11, it is important to monitor the state of the vehicle. The image recognition unit 100 identifies the presence and type of violation. Information representing the state of the vehicle is generated as a result of the data processing unit 202 taking in a signal input from the vehicle side via the vehicle signal interface 206 and performing a predetermined conversion or calculation. Information representing the state of the vehicle generated by the data processing unit 202 is input to the microcomputer 104 of the image recognition unit 100 via the communication processing unit 203 and a communication network (CAN) on the vehicle.

  When detecting an important violation, the image recognition unit 100 outputs information indicating the recording start trigger and the detected type of violation from the microcomputer 104 in step S16 of FIG. These signals are input to the drive recorder 200 via a communication network (CAN) on the vehicle and used for recording control. That is, the data processing unit 202 of the drive recorder 200 stores the compressed data including images taken within a predetermined time range before and after the recording start trigger from the image recognition unit 100 side as a reference. Save on the card 207. Information indicating the type of violation input together with the recording start trigger is also stored on the memory card 207.

  Regardless of the presence or absence of a recording start trigger, the data processing unit 202 always records the compressed data of the latest image on the buffer memory allocated on the memory 205 and also displays the image data that has become obsolete in time. Control overwriting. Therefore, when recording is started in accordance with the recording start trigger, image data captured after a predetermined time (for example, 15 seconds) after that point is held in the buffer memory, so the recording was performed before the trigger occurred. The image can be stored on the memory card 207 including the image.

  For example, when the storage capacity of the memory card 207 is very large, it is possible to save all the image data taken regardless of the occurrence of the trigger on the memory card 207. In that case, information indicating the presence / absence of a recording start trigger and the type of violation detected so that an important part can be easily extracted from a huge amount of image data stored on the memory card 207. The image is recorded on the memory card 207 in association with the shooting time or recording position of the corresponding image.

  Next, the configuration example shown in FIG. 5 will be described. A drive recorder 200B shown in FIG. 5 has a function equivalent to that obtained when the image recognition unit 100 and the drive recorder 200 shown in FIG. 4 are integrated.

  In the drive recorder 200B, there are a main decoder 211, a data processing unit 212, memories 213, 213B and 214, a vehicle signal interface 215, an NTSC encoder 216, a monitor output unit 217, a PC setting interface 218, an image processing circuit 220, a memory. A card 231, a vibrator output unit 232, an audio output unit 233, a position information input unit 234, and a communication processing unit 235 are provided. Further, the image processing circuit 220 includes an image recognition circuit 221 and memories 222 and 223.

  In addition to the in-vehicle camera 11 described above, an in-vehicle camera 11B can be optionally connected to the input of the drive recorder 200B shown in FIG. For example, it is possible to photograph a front or rear landscape in the traveling direction of the vehicle using the in-vehicle camera 11 and simultaneously photograph the state of the vehicle interior using the in-vehicle camera 11B. Alternatively, it is also conceivable to photograph a landscape in the forward direction of the vehicle using the in-vehicle camera 11 and simultaneously photograph a landscape in the backward direction of the vehicle using the in-vehicle camera 11B.

  The main decoder 211 processes NTSC standard video signals output from the in-vehicle cameras 11 and 11B, respectively, extracts a plurality of signal components (Y, Cb, Cr) included in these video signals, and performs A / D. Conversion is performed to generate a digital image signal that can be processed by the data processing unit 212 and the image processing circuit 220.

  The image recognition circuit 221 is a large-scale integrated circuit (LSI) capable of executing image recognition processing suitable for in-vehicle devices at high speed, such as “IMAPCAR 2” manufactured by Renesas Electronics. A memory (ROM) 222 and a memory (SDRAM) 223 are connected to the image recognition circuit 221. The memory 223 is used for holding temporary data such as an image being recognized by the image recognition circuit 221. The memory 222 holds in advance reference data necessary for recognizing patterns of various types of signs (signs) displayed on the road surface.

  The image recognition circuit 221 inputs images taken by the in-vehicle cameras 11 and 11B from the output of the main decoder 211, and executes various image recognition processes at high speed according to an algorithm determined by a program registered in advance. In the present embodiment, the image recognition circuit 221 automatically recognizes the presence and type of various signs (signs) displayed on the road surface of the road on which the vehicle is running from the input image. To do.

  The data processing unit 212 is composed of a microcomputer equipped with a basic program (control system real-time OS) suitable for performing real-time control. The data processing unit 212 basically performs an operation of inputting image data of videos taken by the in-vehicle cameras 11 and 11B from the output of the main decoder 211 and recording them after compression processing.

  In addition, when the image recognition circuit 221 recognizes the sign (sign) displayed on the road surface of the road, the data processing unit 212 identifies the corresponding sign type (corresponding to S13 in FIG. 1), and sets the sign type. The state of the vehicle is monitored by the corresponding process (corresponding to S14 in FIG. 1), and the existence of violation and the type of violation are identified. When the occurrence of an important violation to be recorded is detected (S15), the data processing unit 212 generates a recording start trigger therein, and relates to a video shot within a predetermined time range before and after that point. The compressed image data and information indicating the detected violation type are stored in the memory card 231.

  The memory 213 connected to the data processing unit 212 is configured by a ROM that holds a program prepared in advance. The memory 213B is composed of an EEPROM, and is used for holding constant data such as various threshold values referred to by the data processing unit 212. The memory 214 is used to hold temporary data such as image data being processed by the data processing unit 212.

  The vehicle signal interface 215 inputs various signals output from the vehicle side and converts them into signals that can be processed by the data processing unit 212, or converts various signals output from the data processing unit 212 into signals that can be processed by the vehicle side. Process to do. Signals handled by the vehicle signal interface 215 include trigger output, ignition (IGN), manual switch (SW), brake, turn signal L side, turn signal R side, vehicle speed pulse, and the like.

  The NTSC encoder 216 converts the image after the recognition processing of the image recognition circuit 221 and the image output from the data processing unit 212 into an NTSC standard video signal and outputs the video signal to the monitor output unit 217.

  The PC setting interface 218 is an interface for connecting a host device such as a personal computer and the drive recorder 200B. By connecting the host device to the PC setting interface 218, for example, various threshold values referred to by the data processing unit 212 can be changed, or conditions for determining violations can be changed according to an instruction from the host device.

  The memory card 231 is a recording medium having a built-in nonvolatile memory such as an SD card, and is detachable from the drive recorder 200B. The vibrator output unit 232 is used to connect a vibrator (not shown) that generates mechanical vibrations, for example, to alert a crew member to the occurrence of a violation. The audio output unit 233 is used to output an audio signal for urging the crew member to take note of the occurrence of a violation, for example.

  The position information input unit 234 is used to connect a device that can input information on the current position of the vehicle, such as a GPS (Global Positioning System) receiver. The communication processing unit 235 provides a function for performing communication between another in-vehicle device and the drive recorder 200B via a communication network (CAN) on the vehicle.

  Next, the configuration example shown in FIG. 6 will be described. In this system, the in-vehicle cameras 11 and 11B are connected to the input of a digital tachograph 300 mounted on a vehicle as an in-vehicle device, and the above-described drive recorder 200 is connected to the communication processing unit 311 of the digital tachograph 300.

  As shown in FIG. 6, the digital tachograph 300 includes a main decoder 301, a data processing unit 302, memories 303, 304 and 306, a PC setting interface 307, a memory card 308, a vibrator output unit 309, an audio output unit 310, A communication processing unit 311, a position information input unit 312, an image recognition circuit 315, memories 316 and 317, and an NTSC encoder 318 are provided.

  Functionally, the digital tachograph 300 has a function corresponding to the image recognition unit 100 shown in FIG. 4 in addition to a general digital tachograph function. That is, when the digital tachograph 300 recognizes a sign (sign) displayed on the road surface of the road, it monitors the state of the vehicle to identify the violation and the type of violation, and starts recording when an important violation is detected. Output information indicating the type of trigger or violation. In addition, information indicating the recording start trigger and violation type output from the digital tachograph 300 is input to the drive recorder 200 and reflected in the recording control of the drive recorder 200.

  The main decoder 301 processes NTSC standard video signals output from the in-vehicle cameras 11 and 11B, respectively, extracts a plurality of signal components (Y, Cb, Cr) included in these video signals, and performs A / D. Conversion is performed to generate a digital image signal that can be processed by the data processing unit 302 and the image recognition circuit 315.

  The image recognition circuit 315 is a large-scale integrated circuit (LSI) capable of executing image recognition processing suitable for in-vehicle devices at high speed, such as “IMAPCAR 2” manufactured by Renesas Electronics. A memory (ROM) 316 and a memory (SDRAM) 317 are connected to the image recognition circuit 315. The memory 317 is used for holding temporary data such as an image being recognized by the image recognition circuit 315. The memory 316 holds in advance reference data necessary for recognizing patterns of various types of signs (signs) displayed on the road surface.

  The image recognition circuit 315 inputs images taken by the in-vehicle cameras 11 and 11B from the output of the main decoder 301, and executes various image recognition processes at high speed according to an algorithm determined by a program registered in advance. In the present embodiment, the image recognition circuit 315 automatically recognizes the presence and type of various signs (signs) displayed on the road surface of the road on which the vehicle is running from the input image. To do.

  The data processing unit 302 is composed of a microcomputer equipped with a basic program (control system real-time OS) suitable for performing real-time control. When the image recognition circuit 315 recognizes a sign (sign) displayed on the road surface of the road, the data processing unit 302 identifies the type of the corresponding sign (corresponding to S13 in FIG. 1), and according to the type of sign. Through the process (corresponding to S14 in FIG. 1), the state of the vehicle is monitored to identify the violation and the type of violation. When the occurrence of an important violation to be recorded is detected (S15), the data processing unit 302 generates a recording start trigger therein and stores information indicating the detected violation type in the memory card 308. . When the recording start trigger is generated, the data processing unit 302 transmits information indicating the recording start trigger and the detected violation type to the drive recorder 200 via the communication processing unit 311 and the communication network (CAN) on the vehicle. To do. Of course, the data processing unit 302 also performs processing for realizing a general digital tachograph function.

  The memory 303 connected to the data processing unit 302 is configured by a ROM that holds a program prepared in advance. The memory 306 is composed of an EEPROM, and is used to hold constant data such as various threshold values referred to by the data processing unit 302. The memory 304 is used for holding temporary data.

  The digital tachograph 300 is connected to various signal lines on the vehicle side via a vehicle signal interface 305. The vehicle signal interface 305 inputs various signals output from the vehicle side and converts them into signals that can be processed by the data processing unit 302, or converts various signals output from the data processing unit 302 into signals that can be processed by the vehicle side. Process to do. Signals handled by the vehicle signal interface 305 include trigger output, ignition (IGN), manual switch (SW), brake, turn signal L side, turn signal R side, vehicle speed pulse, and the like.

  The NTSC encoder 318 converts the image after the recognition processing by the image recognition circuit 315 into a video signal of the NTSC standard and outputs it to the monitor output unit. The PC setting interface 307 is an interface for connecting a host device such as a personal computer and the digital tachograph 300. By connecting the host device to the PC setting interface 307, it is possible to change, for example, various threshold values referred to by the data processing unit 302 or change conditions for determining violations according to an instruction from the host device.

  The memory card 308 is a recording medium having a built-in nonvolatile memory, such as a CF card, and is detachable from the digital tachograph 300. The vibrator output unit 309 is used to connect a vibrator (not shown) that generates mechanical vibrations, for example, to alert a crew member to the occurrence of a violation. The audio output unit 310 is used to output an audio signal for alerting the crew member to the occurrence of a violation, for example.

  The position information input unit 312 is used to connect a device that can input information on the current position of the vehicle, such as a GPS (Global Positioning System) receiver. The communication processing unit 311 provides a function for performing communication between other in-vehicle devices and the digital tachograph 300 via a communication network (CAN) on the vehicle.

  As described above, the image recording control method, the image recognition apparatus, and the in-vehicle image recording apparatus according to the present invention are assumed to be applied to various in-vehicle devices such as an image recognition unit, a drive recorder, and a digital tachograph installed in a vehicle. Is done. By adopting the present invention, it is possible to automatically collect valuable data useful for performing safety management related to the operation of the crew. That is, it is possible to acquire data such as an image that can easily grasp whether or not a safe driving is being performed without violating a temporary stop or the like in accordance with instructions of various signs displayed on the road surface. Therefore, it can be used to prevent the occurrence of traffic accidents.

  In addition, since only an image in a time zone including highly important information can be stored, a large-capacity recording medium is not required. In addition, when a large-capacity recording medium is used, data can be recorded in a state where important portions can be easily extracted from a large amount of recorded information.

DESCRIPTION OF SYMBOLS 10 Image recognition apparatus 11 In-vehicle camera 12 Sign recognition part 13 Importance recognition part 14 Recording control part 15 Data recording medium 20 In-vehicle image recording apparatus 51 Road 52 Vehicle 53 In-vehicle camera 54 Stop line 55 Crosswalk 100 Image recognition unit 101 Video amplifier 102 NTSC decoder 103, 220 Image processing circuit 104 Microcomputer 105, 106, 213, 214, 222, 223 Memory 107 Power supply circuit 200, 200B Drive recorder 201, 211, 301 Main decoder 202, 212, 302 Data processing unit 203 Communication processing unit 204, 205 Memory 206, 215, 305 Vehicle signal interface 207, 231, 308 Memory card 208, 233, 310 Audio output unit 209, 217 Monitor output unit 216, 3 8 NTSC Encoder 218,307 PC setting interface 221,315 image recognition circuit 232,309 vibrator output unit 234,312 positional information input unit 235,311 communication processing unit 300 digital tachograph 303,304,306,316,317 memory

Claims (5)

An image recording control method for controlling a recording start trigger of information including at least time-series image data obtained by photographing with a vehicle-mounted camera, or switching of predetermined identification information added to the information,
Recognize the presence and type of signs displayed on the road surface of the road based on the content of time-series image data obtained by shooting the subject in front of or behind the traveling direction of the vehicle with the in-vehicle camera,
When the sign is recognized, the driving state of the host vehicle is monitored, and based on the type of the sign and the driving state of the host vehicle, the level of importance of the situation is automatically identified,
An image recording control method, wherein the degree of importance of the identified situation is reflected in recording control of information including the image data. When identifying the level of importance of the situation, at least whether or not the own vehicle is temporarily stopped is automatically identified, and the presence or absence of the temporary stop is reflected in the recording control of the information including the image data. Item 2. The image recording control method according to Item 1. An image recognition apparatus that generates a recording start trigger for information including at least time-series image data obtained by photographing by a vehicle-mounted camera or a control signal for controlling switching of predetermined identification information added to the information. ,
A sign recognition unit for recognizing the presence and type of a sign displayed on the road surface based on the content of time-series image data obtained by photographing a subject in front of or behind the traveling direction of the host vehicle with an in-vehicle camera; ,
When the sign is recognized, the traveling state of the host vehicle is monitored, and the level of importance of the situation is automatically identified based on the type of the sign and the traveling state of the host vehicle. An image recognition apparatus, comprising: an importance level identifying unit that outputs a control signal representing a state corresponding to. The importance level identification unit automatically identifies at least whether or not the own vehicle is temporarily stopped, and outputs a control signal indicating whether or not the identified own vehicle is temporarily stopped. Image recognition device. An in-vehicle image recording apparatus that records information including at least time-series image data obtained by photographing with an in-vehicle camera,
A sign recognition unit for recognizing the presence and type of a sign displayed on the road surface based on the content of time-series image data obtained by photographing a subject in front of or behind the traveling direction of the host vehicle with an in-vehicle camera; ,
An importance identifying unit that monitors the traveling state of the host vehicle when recognizing the sign, and automatically identifies the level of importance of the situation based on the type of the sign and the traveling state of the own vehicle;
A recording control unit that controls a recording start trigger of information including the image data or switching of predetermined identification information added to the information in accordance with the level of importance identified by the importance identifying unit. A vehicle-mounted image recording device.

Images