JP7290506B2 - Specific character string detector and driving evaluation system - Google Patents

Description

The present invention relates to a specific character string detection device and a driving evaluation system.

Conventionally, in-vehicle equipment such as a digital tachograph installed in a vehicle uses driving information such as vehicle speed information to support safe driving and evaluate the driving operation of the driver who drives the vehicle. The safe driving support device of Patent Document 1 activates the peripheral visual recognition device and images and displays the left and right directions in front of the vehicle only when a road marking such as "stop" is detected by image recognition while the vehicle is running. It is something to do. The driving support device of Patent Document 2 recognizes a traffic sign from image data obtained by imaging the surroundings of a vehicle, and when a designated sign is recognized, notifies the existence of the designated sign. If there are two or more, notification is prohibited. Further, the vehicle road marking detection device of Patent Document 3 reduces the image processing load by narrowing down the image processing area on the captured image based on the detection results when detecting the road markings one character at a time. , which identifies road markings based on consecutively detected characters.

JP-A-2002-219998 JP 2017-102665 A JP 2006-127358 A

In the driving evaluation, for example, whether the driver drove appropriately at intersections and other places where a temporary stop should be made, in other words, whether the vehicle was stopped at an appropriate position and checked for safety was included in the evaluation items, thereby improving the driver's safe driving. Awareness can be expected to be raised. Image recognition is used to detect "stop" signs on the road surface in order to detect places where a vehicle should stop. This image recognition has a graphic information database (template image, dictionary) that stores the graphic information representing the shape of each character of "stop", "ma", and "re" in the on-vehicle device, and images of the road surface (photographed images) ) is matched with the graphic information to determine each character.

If one of the characters "stop", "ma", and "re" is recognized and it is determined that the sign is "stop", the determination accuracy tends to decrease. In order to improve the determination accuracy, if two or more characters are recognized as a "stop" sign as in Patent Document 3, the detection rate (recognition rate) of the "stop" sign will drop. In addition, when trying to recognize all three characters, the recognition rate is further reduced because the road marking includes characters that are blurred or partially missing due to wear of the road surface. This problem is not dealt with in Patent Documents 1 and 2, either.

SUMMARY OF THE INVENTION The present invention has been made in view of the circumstances described above, and an object of the present invention is to improve the recognition rate of road markings such as "Stop".

In order to achieve the above object, a specific character string detection device and a driving evaluation system according to the present invention are characterized by the following (1) to (7).
(1) A specific character string detection device that detects a specific character string including a first character and a second character marked on a road surface,
an image acquisition unit that acquires a captured image including at least the road surface;
a recognition unit that recognizes the first character marked on the road surface in the photographed image by referring to a graphic information database that stores graphic information representing the shape of characters;
an estimation unit that estimates a possible location of the second character based on the position of the first character on the road surface;
The recognition unit
searching for traces of the second character at the estimated location, referring to the graphic information database, and identifying candidate characters that are candidates for the second character;
A specific character string detection device that determines whether or not the specific character string is detected based on the recognition result of the first character and the identification result of the candidate character.
(2) The recognition unit
recognizing the first character when the degree of matching between the graphic information to be recognized included in the captured image and the graphic information stored in the graphic information database is equal to or greater than a first threshold;
If the degree of matching between the graphic information at the estimated location and the graphic information contained in the graphic information database is equal to or greater than a second threshold lower than the first threshold, the candidate for the second character is specified. The specific character string detection device according to (1) above, characterized in that:
(3) The recognition unit
recognizing the first character when the degree of matching between the graphic information to be recognized included in the captured image and the graphic information stored in the graphic information database is equal to or greater than a first threshold;
If a value obtained by integrating the degree of matching between the graphic information at the estimated location and the graphic information contained in the graphic information database within a certain range is equal to or greater than a third threshold, the candidate for the second character is specified. The specific character string detection device according to (1) above, characterized in that:
(4) The recognizing unit determines that the specific character string has been detected when all characters forming the specific character string are recognized or specified as the candidate characters. The specific character string detection device according to any one of (1) to (3) above.
(5) A recording unit that records in chronological order the captured images captured by a camera mounted on a vehicle running on the road surface, which are acquired by the image acquisition unit;
The recognition unit calculates a matching degree between the graphic information to be recognized included in the photographed image and the graphic information stored in the graphic information database for each traveling distance of the vehicle, and generates a matching degree graph. is recorded in the recording unit as
The method according to any one of (1) to (4) above, wherein the estimation unit estimates a possible location of the second character based on the position of the first character in the degree-of-match graph. Specific character string detection device described.
(6) the specific character string consists of a plurality of characters having an arrangement order;
The recognition unit recognizes any one of the plurality of characters as the first character,
The estimating unit estimates, based on the position of the first character, the possible location of the second character, which is any one character other than the recognized first character among the plurality of characters. The specific character string detection device according to any one of (1) to (5) above.
(7) an on-vehicle device that is mounted on a vehicle and acquires operation information of the vehicle;
The specific character string detection device according to any one of (1) to (6) above;
A driving evaluation system comprising a driving evaluation device that evaluates a driver of the vehicle based on the operation information acquired by the vehicle-mounted device,
The driving evaluation system, wherein the driving evaluation device evaluates the driver based on the operation information when the recognition unit determines that the specific character string is detected.

According to the specific character string detection device having the configuration (1) above, even if the specific character string includes a faint or missing character, the location of the character is estimated, and traces of the character are detected. By searching for , it can be determined that the specific character string has been detected as a whole. Therefore, it is possible to improve the recognition rate of a specific character string and reduce the recognition error rate. In addition, the first character "first" and the second character "second" are expressions for distinguishing a plurality of characters contained in the character string, and the arrangement order of the characters contained in the character string It does not indicate the order of recognition.

According to the specific character string detection device having the configuration (2) above, even if the character is blurred and the degree of matching does not reach the first threshold for character recognition, the second threshold lower than the first threshold is used to detect the character. Can identify candidates.

According to the specific character string detection device having the configuration (3) above, even if the characters are blurred and the degree of matching does not reach the first threshold value, the integrated value of the degree of matching can be used to identify the character candidate.

According to the specific character string detection device having the configuration (4) above, even if some of the plurality of characters that make up the specific character string cannot be recognized, traces of the characters that cannot be recognized are used as candidate characters. By specifying, the specific character string can be accurately detected as the character string as a whole.

According to the specific character string detection device having the configuration (5) above, the degree of matching is calculated according to the distance traveled (or the time corresponding to the distance traveled). Or it can be estimated that the second character exists in the back. In addition, it is possible to identify a candidate character by determining that there is a trace of the second character when there is a mountain by observing the degree of change in the matching degree graph within the estimated distance range.

According to the specific character string detection device having the configuration (6) above, the specific character string can be detected even when a plurality of characters having an arrangement order includes blurred or missing characters. For example, when a specific character string consisting of three characters "stop" is arranged on the road surface in the order of "re", "ma", and "stop" from the front, when "ma" is first recognized, Based on the position of "ma", it is possible to estimate the possible location of "tome" which should be in the back or "re" which should be in front. Also, for example, when ``re'' is recognized for the first time, based on the position of ``re'', it is possible to estimate a place where an adjacent ``ma'' or a non-adjacent ``stop'' may exist. Furthermore, based on the position of ``ma'' which is recognized first, ``ma'' is recognized first, and ``ma'' is recognized second, the possible location of ``tome'' can be estimated.

According to the driving evaluation system having the above configuration (7), a specific character string such as "stop" marked on the road surface can be detected with a high recognition rate, so that driving evaluation can be accurately performed at the time of a temporary stop or the like. .

According to the present invention, it is possible to improve the recognition rate of road markings such as "Stop" and reduce the recognition error rate.

The present invention has been briefly described above. Furthermore, the details of the present invention will be further clarified by reading the following detailed description of the invention (hereinafter referred to as "embodiment") with reference to the accompanying drawings. .

FIG. 1 is a diagram showing the configuration of the driving evaluation system of this embodiment. FIG. 2 is a diagram showing an example of a road sign indicating "stop". FIG. 3 is a diagram showing an example of time-series changes in the degree of matching between a recognition target and graphic information stored in the graphic information database. FIG. 4 is a diagram showing an example of a character recognition determination threshold value and a character candidate determination threshold value. FIG. 5 is a diagram showing an example of a character candidate determination window. FIG. 6 is a flowchart (1) showing an operation example of a digital tachograph that performs road marking determination. FIG. 7 is a flow chart (2) showing an operation example of a digital tachograph that performs road marking determination. FIG. 8 is a flowchart (3) showing an operation example of a digital tachograph that performs road marking determination. FIG. 9 is a flowchart (4) showing an operation example of the digital tachograph that performs road marking determination. FIG. 10 is a flow chart (5) showing an operation example of a digital tachograph that performs road marking determination. FIG. 11 is a diagram showing an operation example of the office PC.

Specific embodiments relating to the present invention will be described below with reference to each drawing. In the embodiment described below, an example in which the specific character string detection device of the present invention is configured as a service recording device will be shown.

FIG. 1 is a diagram showing the configuration of a driving evaluation system 5 of this embodiment. The driving evaluation system 5 evaluates the driving of the driver who gets on the vehicle, and includes a vehicle-mounted operation recording device (hereinafter referred to as a digital tachograph) 10 and an office PC 30, which are connected via a network 70. It has a configuration including The on-board device and the office PC do not have to be connected via a network. . Also, the vehicle-mounted device may be a drive recorder or the like.

The office PC 30 is composed of a general-purpose computer installed in the office, and manages the operation status of the vehicle. The network 70 is a packet communication network such as the Internet to which the wireless base station 8 that performs wide-area communication with the digital tachograph 10 and the office PC 30 are connected. do. Communication between the digital tachograph 10 and the radio base station 8 may be performed by a mobile communication network (cellular network) such as LTE (Long Term Evolution)/4G (4th Generation), or by a wireless LAN (Local Area Network). Network).

The digital tachograph 10 is mounted on the vehicle, and records operation data (operation information) such as entry/exit time, travel distance, travel time, travel speed, over speed, over engine speed, sudden start, sudden acceleration, sudden deceleration, etc. . The digital tachograph 10 includes a CPU 11 (image acquisition unit, recognition unit, estimation unit), a nonvolatile memory 26A, a volatile memory 26B, a recording unit 17, a card I/F 18, an audio I/F 19, an RTC (clock IC) 21, and a SW input unit. 22 and a display unit 27 .

The CPU 11 comprehensively controls each part of the digital tachograph 10 . The non-volatile memory 26A stores an operating program executed by the CPU 11, a graphic information database (template image, dictionary) for storing graphic information representing the shape of each character "shi", "ma", and "re". The volatile memory 26B is used to temporarily hold various data.

The recording unit 17 records data such as operation data and images in a predetermined storage area. A memory card 65 carried by the driver is removably connected to the card I/F 18 . The CPU 11 writes the operation information (including data such as operation data and images) and warning events recorded by the recording unit 17 to the memory card 65 connected to the card I/F 18 . Note that the recording unit 17 may record operation data and the like in the volatile memory 26B. A built-in speaker 20 is connected to the audio I/F 19 . The built-in speaker 20 emits sounds such as alarms.

The RTC 21 (timekeeping unit) measures the current time. The SW input unit 22 receives ON/OFF signals for various buttons such as an exit button and an entry button. The display unit 27 is composed of an LCD (liquid crystal display), and displays communication and operation statuses as well as alarms and the like. As will be described later, when the digital tachograph 10 detects a "stop" road sign, the digital tachograph 10 may issue an audible warning based on the operation information, for example, when the brake is not depressed, or may issue an audible warning. In addition to this, a display warning may be issued.

In addition, the digital tachograph 10 includes a speed I/F 12A, an engine rotation I/F 12B, an external input I/F 13, a sensor input I/F 14, an analog input I/F 29, a GPS receiver 15, a camera I/F 16, a communication unit 24 and It has a power supply unit 25 .

A vehicle speed sensor 51 that detects the speed of the vehicle is connected to the speed I/F 12A, and a speed pulse from the vehicle speed sensor 51 is input. The vehicle speed sensor 51 may be provided as an option in the digital tachograph 10 or may be provided as a separate device from the digital tachograph 10 . A rotation pulse from an engine rotation speed sensor (not shown) is input to the engine rotation I/F 12B. An external device (not shown) is connected to the external input I/F 13 .

The sensor input I/F 14 is connected to an acceleration sensor (G sensor) 28 that detects acceleration (G value) (perceives impact), and receives a signal from the G sensor 28 . The analog input I/F 29 receives signals from a temperature sensor (not shown) that detects engine temperature (cooling water temperature), a fuel amount sensor (not shown) that detects fuel amount, and the like. The CPU 11 detects various operating states based on information input via these I/Fs.

The GPS receiver 15 is connected to the GPS antenna 15a, receives signals transmitted from GPS satellites, and acquires position information (GPS information) indicating the current position of the vehicle.

The cameras 23A and 23B are connected to the camera I/F 16, and captured images (moving images and still images) are input to the CPU 11 at all times. The camera 23A is installed in the vehicle and acquires image data (captured image) by imaging the surroundings of the vehicle (for example, including the road surface on which the vehicle travels and in front of the vehicle). The camera 23B is installed in front of a meter or the like inside the vehicle, and acquires image data by photographing the inside of the vehicle including the driver's face. The image captured by the camera 23A includes a preceding vehicle in front of the own vehicle, white lines indicating the boundary of the lane in which the vehicle is traveling, road markings such as traffic regulations ("stop", stop lines, pedestrian crossings, etc.). appear. The cameras 23A and 23B each have at least one image sensor with, for example, 300,000 pixels, 1,000,000 pixels, and 2,000,000 pixels arranged on an imaging surface, and can capture images such as stereo images. The image sensor may consist of a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. Note that the camera I/F 16 may be connected to a camera that captures images of the sides and rear of the vehicle.

The CPU 11 of the digital tachograph 10 recognizes the specific character string "stop" road marking (hereinafter also referred to as "specific road marking") by a method described later based on the captured image captured by the camera 23A. . By detecting the specific road marking by the CPU 11 through image recognition, it is possible to grasp the place to stop based on the actual road marking without using the map information. Therefore, it is possible to accurately evaluate driving even at places where road markings have been newly added and where temporary stops cannot be grasped from the map information. Further, the CPU 11 of the digital tachograph 10 may recognize the orientation of the driver's face and the eye open state by a known method based on the image captured by the camera 23B. The images captured by the cameras 23A and 23B are recorded in chronological order by the recording unit 17, respectively. The recording unit 17 stores the captured image captured by the camera 23A, the image captured by the camera 23B, and the operation information collected by the CPU 11 based on the captured image at the time when the specific road marking is recognized and the current position of the vehicle. are recorded in association with In addition to imaging visible light, the camera may be equipped with an infrared camera so that imaging is possible even at night.

The communication unit 24 performs wide-area communication, and when connected to the wireless base station 8 via a mobile network (mobile communication network), communicates with the office PC 30 via a network 70 such as the Internet connected to the wireless base station 8. communicate. The power supply unit 25 supplies power to each unit of the digital tachograph 10 when an ignition switch is turned on or the like.

On the other hand, the office PC 30 is a PC that runs on a general-purpose operating system. The office PC 30 functions as a driving evaluation device, acquires the contents recorded by the recording unit 17 from the digital tachograph 10, and evaluates the driving of the vehicle driver based on the contents. Office PC 30 has CPU 31 , communication section 32 , display section 33 , storage section 34 , card I/F 35 , operation section 36 , output section 37 , audio I/F 38 and external I/F 48 .

The CPU 31 comprehensively controls each section of the office PC 30 . The communication unit 32 can communicate with the digital tachograph 10 via the network 70 . The communication unit 32 can also be connected to various databases (not shown) connected to the network 70, and can acquire necessary data.

The display unit 33 displays a driving evaluation screen and the like. The storage unit 34 stores a driving evaluation program and the like for performing driving evaluation based on data measured by the digital tachograph 10 .

A memory card 65 is detachably attached to the card I/F 35 . The card I/F 35 inputs operation information measured by the digital tachograph 10 and stored in the memory card 65 . The operation unit 36 has a keyboard, a mouse, etc., and receives operations of the administrator of the office PC 30 . The output unit 37 outputs various data. A microphone 41 and a speaker 42 are connected to the audio I/F 38 . The administrator can also make voice calls using the microphone 41 and the speaker 42 .

An external storage device (not shown) such as an operation data database (DB) and a hazard map database (DB) can be connected to the external I/F 48 . In the operation data DB, as operation data, in addition to the time of entry and exit, speed, mileage, etc., temporary Various event information is recorded, including the presence or absence of a stop and the stop time. The hazard map DB stores map data in which marks representing places where accidents have occurred in the past (accident places) and places where stop violations are likely to occur are superimposed on the map. Note that the hazard map may describe areas where disasters such as natural disasters are expected, evacuation sites, and the like.

The office PC 30 has a function of inputting the operation data stored in the memory card 65 and analyzing the actual operation state of the corresponding vehicle. This analysis function also includes a function to process event data (pause event data) at locations where road markings such as "Stop" are detected and reflect them in the evaluation of each crew member. Details will be described later.

Next, an outline of an operation example when the digital tachograph 10 recognizes an image of a "stop" road sign will be described with reference to FIGS. 2 to 5. FIG. FIG. 2 is a diagram showing an example of a road sign indicating "stop". As shown in FIG. 2, on the road surface S, the character string (specific character string) C of "stop" is displayed in the order of "re", "ma", and "stop" from the near side in the traveling direction D of the vehicle 1. Painted (marked). The distance H (pitch) between two adjacent characters varies depending on prefectures and regions, but as an example, it ranges from 2 m at minimum to 3.5 m at maximum. In this embodiment, when the digital tachograph 10 detects the specific character string C by image recognition, if the recognition of "stop" (first character) in the specific character string C is successful, the character " It is estimated that "ma" exists within a certain distance range (for example, 4 m to 1.5 m in front (or back)). Using this estimated range as a detection range R, the digital tachograph 10 searches for traces of "ma" and identifies candidate characters, as will be described later.

FIG. 3 is a diagram showing an example of time-series changes in the degree of matching between a recognition target and graphic information stored in the graphic information database. In the graph (matching degree graph) shown in FIG. 3, the horizontal axis indicates the traveling distance or time of the vehicle 1, and the vertical axis indicates the recognition object (road marking painted on the road surface) and the figure stored in the graphic information database. Indicates the degree of matching with the information. As shown in FIG. 3, in a section where the distance between the vehicle 1 and the recognition target is long, and in a state where the vehicle 1 has passed the recognition target and is not included in the captured image of the camera 23A, the degree of matching is almost 0. When the recognition target is within the recognizable distance range, a mountain of coincidence appears. For this mountain, character recognition or character candidate determination is performed as described with reference to FIGS.

FIG. 4 is a diagram showing an example of a character recognition determination threshold value and a character candidate determination threshold value. In the graph shown in FIG. 4, the horizontal axis indicates the traveling distance or time of the vehicle 1, and the vertical axis indicates the matching degree between the recognition object and the graphic information stored in the graphic information database. The three graphs shown in FIG. 4 show, in order from the top, the degree of matching between the graphic information indicating each character of "re", "ma", and "stop" and the recognition target. In each graph, the character recognition determination threshold value T1 (first threshold value) and the character candidate determination threshold value T2 (second threshold value) are indicated by dashed lines. is also considered to be low.

In the example shown in FIG. 4, the peak of "stop" in the bottom graph exceeds the character recognition determination threshold value T1, and the character recognition of "stop" is successful. Therefore, the digital tachograph 10 estimates that the character "ma" adjacent to "stop" exists in a range of 4 m to 1.5 m before the peak position P, for example, starting from the peak position P of the mountain of "stop". do. When the horizontal axis is time, it is estimated that "ma" exists in the range that goes back by the time that the vehicle 1 has traveled the specified travel distance (4m to 1.5m). Then, the digital tachograph 10 scans the estimated location (fixed distance range) in the matching degree graphs shown in the top and middle graphs to search for peaks of matching degree. Although the mountain of "ma" does not reach the character recognition determination threshold value T1 (that is, cannot be recognized as "ma"), it exceeds the character candidate determination threshold value T2. , and specifies "ma" as a candidate character. As a method of estimating the position where the character "ma" adjacent to "stop" exists, a starting point other than the peak position P of the peak of the degree of matching of "stop" may be used. For example, a point where the matching degree of "stop" exceeds the character recognition determination threshold value T1, an intermediate distance point between where the matching degree of "stop" exceeds the character recognition determination threshold value T1, or a match of "stop" The starting point may be any one of the peak points while the degree exceeds the character recognition determination threshold value T1.

In the example shown in FIG. 4, after specifying "ma" as a candidate character, the digital tachograph 10 similarly detects the character "re" adjacent to "ma" from the peak position of the mountain of "ma". It is presumed that it exists in a predetermined range retroactively. Then, the digital tachograph 10 scans this predetermined range and specifies "re" whose degree of matching exceeds the threshold value T2 for character candidate determination (with traces) as a candidate character. In the example shown in FIG. 4, of the three characters that make up the specific character string "stop", one character "stop" was recognized, and two characters "ma" and "re" were identified as candidate characters. determines that "stop" has been detected by estimating that it was "stop" as a whole.

Referring to FIG. 4, an example of specifying a character candidate by setting a threshold value (character candidate determination threshold value T2) lower than the character recognition determination threshold value T1 is shown. An example of specifying a character candidate from the integrated value of the degree of matching) will be shown.

FIG. 5 is a diagram showing an example of a character candidate determination window. In the graph shown in FIG. 5, the horizontal axis indicates the traveling distance or time of the vehicle 1, and the vertical axis indicates the matching degree between the recognition object and the graphic information stored in the graphic information database. As shown in FIG. 5, for example, in the coincidence graph for "ma", a window W within a certain distance range is set. Then, the digital tachograph 10 monitors the graph area A within this window W, and when the graph area A exceeds a predetermined threshold (third threshold), it determines that there is a trace of the character candidate (second character). do. For example, as described with reference to FIG. 4, when the character recognition of "stop" is successful, the digital tachograph 10 detects that the character "ma" adjacent to "stop" is at the peak position P of the peak of "stop", for example. is assumed to exist in a range of 4 m to 1.5 m before the peak position P, starting from . Then, the digital tachograph 10 moves the window W within a certain distance range in the estimated range in the matching degree graph of "ma" shown in FIG. ” as a candidate character.

In the description with reference to FIGS. 4 and 5, the location where the character adjacent to the recognized character (first character) may exist was estimated, but by appropriately changing the predetermined range, characters not adjacent to the recognized character can be estimated. may be estimated where For example, when ``stop'' is recognized, the positions of ``re'' that are not adjacent to ``stop'' can be estimated, and character candidates can be specified as described above.

Next, an operation example when the digital tachograph 10 performs road marking determination by image recognition will be described with reference to FIGS. 6 to 10. FIG. The CPU 11 in the digital tachograph 10 executes the processes shown in FIGS. 6-10. The image recognition function of the digital tachograph 10 described below always operates when the vehicle 1 is traveling at a speed of 50 kilometers per hour or less, and the series of flows shown in FIGS. 6 to 10 are repeatedly executed at predetermined intervals. be done. By operating the image recognition function that recognizes "Stop" road signs in a realistic speed range of 50 kilometers per hour or less for vehicles approaching the stop position, while ensuring the effectiveness of this recognition function, An increase in the processing load on the CPU 11 can be prevented. In addition, for example, at speeds of 60 kilometers per hour or more, various recognition functions are operated by dividing them into speed ranges, such as activating the recognition function of the distance between the vehicle and the vehicle in front. The processing power of the CPU 11 can be effectively utilized.

FIG. 6 is a flowchart (1) showing an operation example of the digital tachograph 10 that performs road marking determination. The CPU 11 inputs the image captured by the camera 23A as image data for each frame through the camera I/F, performs image processing on the area on the road surface in each frame, and refers to the graphic information database to determine "stop", "or". Perform pattern recognition on each character of "". In the following explanation, as shown in FIG. 2, the road marking "stop" is arranged so that the positional relationship of each character is in the order of "re", "ma", and "stop" from the front side. It shall be The CPU 11 waits until any one of "stop", "ma", and "re" is recognized for the first time (S1), and when any one is recognized for the first time, it determines whether the recognition result is "re" (S2). In the case of YES, the CPU 11 sets the character recognition flag of "re" to 1 (S3), and proceeds to the process of S21 shown in FIG.

If NO in S2, the CPU 11 determines whether the recognition result is "ma" (S4), and if YES, sets the character recognition flag for "ma" to 1 (S5), and proceeds to the process of S31 shown in FIG. . If NO in S4, the CPU 11 determines that the recognition result is "stop" and sets the character recognition flag of "stop" to 1 (S6). The CPU 11 scans the estimated position (range) of "ma" based on the position of "stop" to search for traces of "ma" (S7). The CPU 11 determines whether or not there is a trace of "ma" in the predetermined range (S8). If YES, the character candidate flag for "ma" is set to 1 (S9). proceed to

The CPU 11 scans the estimated position of "re" with reference to the position (recognition position) where "stop" is recognized or the position (recognition candidate position) where "ma" is recognized or specified as a character candidate. (S10). The CPU 11 selects the assumed position of ``ma'', that is, the position where ``ma'' is estimated to exist based on the recognition position of ``stop'' or the recognition candidate position of ``ma''. It is determined whether there is a trace (S11). In the case of YES, the CPU 11 sets the character candidate flag of "re" to 1, and in the case of NO, the CPU 11 directly proceeds to the processing of S61 shown in FIG.

FIG. 7 is a flow chart (2) showing an operation example of the digital tachograph 10 that performs road marking determination. After "re" is recognized (S3 in FIG. 6), the CPU 11 waits for a certain distance until "ma" is recognized (S21), and determines whether "ma" is recognized (S22). If the answer is YES, the CPU 11 sets the character recognition flag for "ma" to 1 (S23), and if the answer is NO, the process proceeds to S31 shown in FIG.

FIG. 8 is a flowchart (3) showing an operation example of the digital tachograph 10 that performs road marking determination. After recognizing "ma" in the three characters of "stop", or after recognizing "re" and determining whether or not "ma" is recognized, the CPU 11 recognizes "stop" for a certain distance. (S31), and judges whether "stop" is recognized (S32). In the case of NO, the CPU 11 determines whether the character recognition flag of "re" is 1 (S33). If NO, that is, if "ma" is recognized, the CPU 11 scans the estimated position of "stop" based on the recognized position of "ma" to search for traces of "stop" (S34). The CPU 11 determines whether or not there is a trace of "stop" in the predetermined range (S35), and if YES, sets the character candidate flag for "stop" to 1 (S36), and if NO, proceeds to S37. proceed to

In S37, the CPU 11 searches for traces of "re" by scanning the estimated position of "re" based on the recognized position of "ma". Then, the CPU 11 determines whether or not there is a trace of "re" in the predetermined range (S38). , the process proceeds to S61 shown in .

On the other hand, if YES in S32, that is, if "stop" is recognized, the CPU 11 determines whether the recognition flag for "re" is 1 (S40). If NO, that is, if "ma" is recognized, the CPU 11 scans the estimated position of "re" based on the recognized position of "ma" to search for traces of "re" (S41). The CPU 11 determines whether there is a trace of "re" in a predetermined range (S42), and if YES, sets the character candidate flag of "re" to 1 (S43). The process proceeds to S61 shown.

If YES in S40, that is, if the character recognition flag for "re" is 1, the CPU 11 determines whether the character recognition flag for "ma" is 1 (S44). In the case of YES, that is, in the case where each character of "stop", "ma" and "re" is recognized, the CPU 11 proceeds to the processing of S61 shown in FIG. In the case of NO, that is, when ``stop'' and ``re'' are recognized, the CPU 11 scans the estimated position of ``ma'' based on the recognized position of ``stop'' and searches for traces of ``ma''. (S45). The CPU 11 determines whether or not there is a trace of "ma" within a predetermined range (S46). If YES, the character candidate flag for "ma" is set to 1 (S47). The process proceeds to S61 shown.

If YES in S33, that is, if the character recognition flag for "re" is 1, the CPU 11 determines whether the character recognition flag for "ma" is 1 (S48). In the case of NO, the CPU 11 searches for traces of "ma" by scanning the estimated position of "ma" based on the recognized position of "re" (S49). The CPU 11 determines whether or not there is a trace of "ma" within a predetermined range (S50). If YES, the character candidate flag for "ma" is set to 1 (S51). proceed to

In S52, the CPU 11 scans the estimated position of "stop" based on the recognized position of "re" to search for traces of "stop". Then, the CPU 11 determines whether or not there is a trace of "stop" in the predetermined range (S53). If YES, the character candidate flag for "stop" is set to 1 (S54). , the process proceeds to S61 shown in .

FIG. 9 is a flowchart (4) showing an operation example of the digital tachograph 10 that performs road marking determination. As described with reference to FIGS. 6 to 8, the CPU 11 recognizes or determines character candidates for each of the characters ``stop'', ``ma'' and ``re''. Based on the total number, it is determined whether or not a "stop" road sign has been detected. The CPU 11 determines whether the sum of the character recognition flags is 2 or more (S61), and if YES, that is, if at least two characters are recognized, determines that the road sign "stop" has been detected (S64). If NO in S61, the CPU 11 determines whether the sum of the character candidate flags is 1 or more (S62), and if NO, determines that no road sign "Stop" was detected (S63). If YES in S62, that is, if one character is recognized and at least one character candidate is specified, it is determined that the road sign "Stop" has been detected (S64). In this way, out of the three characters ``stop'', ``ma'' and ``re'', not only when two or more characters are successfully recognized, but also when one of the characters is recognized and at least one of the remaining two characters is recognized as a character. If it is specified as a candidate, it can be determined that a "stop" road sign has been detected. For this reason, even if one of the characters is blurred or missing and the degree of matching does not reach the recognition threshold, the locations of the other characters are estimated based on the one character that has been recognized, and the By searching for traces and identifying character candidates, the recognition rate of "stop" road markings can be improved.

FIG. 10 is a flow chart (5) showing an operation example of the digital tachograph 10 that performs road marking determination. The digital tachograph 10 can perform the determination process shown in FIG. 10 instead of the "stop" determination process based on the character recognition flag and the character candidate flag shown in FIG. The CPU 11 determines whether the sum of the character recognition flag and the character candidate flag is 3 (S71). It is determined that the road marking "Stop" has been detected (S73). According to the determination process shown in FIG. 10, any two of the three characters "stop", "ma", and "re" are recognized, and all three characters are recognized, including the case where the remaining one character is identified as a character candidate. If it is recognized or identified as a character candidate, it can be determined that the road sign "stop" has been detected.

The CPU 11 records the result of the determination process shown in FIGS. 9 and 10 as, for example, "pause event data". This event data includes information indicating that a "stop" road sign has been detected, information (operation information) such as vehicle speed, position, and time, and captured image information. That is, this event data includes information indicating whether or not the vehicle has stopped temporarily when a "stop" road sign is detected. In addition to the stop position and stop time of the vehicle, this event data also includes information indicating the presence or absence of left and right safety confirmation actions determined from the face direction of the driver recognized based on the image captured by the camera 23B. may contain. The CPU 11 records this pause event data in the memory card 65 . The temporary stop event data recorded in the memory card 65 can be used for driving evaluation in the office PC 30. FIG.

An operation example of the office PC 30 is shown in FIG. The operation shown in FIG. 11 will be described below.
The administrator who operates the office PC 30 turns on the power of the office PC 30 to make it ready for use, and then activates dedicated application software for data analysis (abbreviated as "application"). (S81). According to the input operation of the administrator, the analysis application running on the office PC 30 reads the operation data and event data recorded in the memory card 65 of each crew member (S82). When the administrator instructs the analysis application to analyze the pause event, the application extracts only the corresponding event data (pause event data) from the data read in S82 (S83). ).

Based on the contents of all of the stop event data extracted in S83, the application determines the number of times the digital tachograph 10 has recognized the "stop" road sign (total number of recognitions) and the number of times the stop has not been made (violation number) is detected (S84). Then, in S84, the application calculates the violation rate (number of violations/total number of recognitions). The application uses the violation rate calculated in S84 to add or deduct points for the evaluation of the corresponding crew member (S85). For example, for a crew member whose violation rate is lower than the average value of all crew members, a score corresponding to (average value - violation rate) is added to the evaluation, and for a crew member whose violation rate is higher than the average value, The score corresponding to (violation rate - average value) is subtracted from the evaluation, and the quality of the actual driving condition is reflected in the evaluation.

After completing the evaluation reflecting the operation data of the day for all crew members, the application creates an evaluation ranking based on the evaluation data of all crew members (S86). For example, the evaluation data of all crew members are arranged in descending order of evaluation scores, and several crew members with high evaluation scores and several crew members with low evaluation scores are extracted and displayed together with their evaluation scores.

After completing the evaluation reflecting the operation data of the day for all the crew members, the application extracts the crew members to be instructed from among all the crew members based on the evaluation data of all the crew members (S87). For example, drivers whose rate of violation of specific items such as stop violations is higher than a threshold or drivers whose overall evaluation score is lower than a threshold are extracted and selected as those who need safe driving guidance and education.

The app aggregates the operation data and event data of all crew members for the same location information, and detects locations that require special attention, such as violations of specific items, such as temporary stop violations, tending to be particularly high. Information on the detected caution position is registered in a hazard map DB (not shown) connected to the external I/F 48 together with information on types such as events and violations (S88). The hazard map DB is configured so that all crew members can freely access and share information. As a result, even crew members who only use general on-board equipment without advanced recognition and judgment functions can grasp areas requiring attention, such as those where stop violations are likely to occur.

As described above, in the present embodiment, even if the specific road marking (specific character string) of "Stop" includes a faint or missing character, the location of the character is estimated, and the character It can be determined that the specific character string has been detected as a whole by searching for traces of . Therefore, it is possible to improve the recognition rate of the "stop" road sign and reduce the recognition error rate.

Further, according to the present embodiment, since a specific road marking can be recognized with a high recognition rate, for example, when a road marking of "stop" is recognized, operation information such as the presence or absence of a temporary stop and the length of time the vehicle is stopped is displayed. You can understand and evaluate your driving appropriately.

It should be noted that the present invention is not limited to the above-described embodiments, and can be modified, improved, etc. as appropriate. In addition, the material, shape, size, numerical value, form, number, location, etc. of each component in the above-described embodiment are arbitrary and not limited as long as the present invention can be achieved. For example, in the above-described embodiment, an example in which the character string "stop" is written vertically on the road surface has been shown, but the arrangement of the characters may be different, such as when the character string "stop" is written horizontally. can be detected by the specific character string detection device of the present invention. Further, in the above-described embodiment, an example in which the specific character string detection device detects a road sign "stop" has been shown, but other character strings can be detected. The specific character string detection device can detect road signs such as "stop", "no right turn", "no parking", and "zone", for example.

Further, in the above-described embodiment, an example in which the specific character string detection device is incorporated in the digital tachograph 10 has been shown, but the specific character string detection device may be incorporated in the office PC 30 or may communicate with the digital tachograph 10. It may be configured as a capable data server. Further, in the above-described embodiment, an example in which the office PC 30 functions as a driving evaluation device was shown, but the driving evaluation device may be incorporated in the digital tachograph 10, or may be installed in a data server capable of communicating with the digital tachograph 10. may be provided.

Also, the graphic information database may be stored in the memory card 65 instead of the non-volatile memory 26A. By storing the graphic information database in the large-capacity memory card 65, the storage capacity in the digital tachograph 10 can be saved.

Here, the features of the specific character string detection device and the driving evaluation system according to the embodiments of the present invention described above are briefly listed in [1] to [7] below.
[1] A specific character string detection device (digital tachograph 10) that detects a specific character string (C) including a first character and a second character marked on a road surface (S),
an image acquisition unit (camera 23A, CPU 11) that acquires a captured image including at least the road surface;
A recognition unit (CPU 11, S1 to S6) that recognizes the first character marked on the road surface in the captured image by referring to a graphic information database that stores graphic information representing the shape of characters;
An estimation unit (CPU 11, S7) that estimates a place where the second character may exist based on the position of the first character on the road surface,
The recognition unit
searching for traces of the second character at the estimated location, referring to the graphic information database, and identifying candidate characters that are candidates for the second character (S8 to S9);
Based on the recognition result of the first character and the identification result of the candidate character, it is determined whether or not the specific character string is detected (S64, S73).
A specific character string detection device (digital tachograph 10) characterized by:
[2] The recognition unit
recognizing the first character when the degree of matching between the graphic information to be recognized included in the captured image and the graphic information stored in the graphic information database is equal to or greater than a first threshold (T1);
If the degree of matching between the graphic information at the estimated location and the graphic information contained in the graphic information database is equal to or greater than a second threshold (T2) lower than the first threshold, the second character is The specific character string detection device according to [1] above, which specifies a candidate.
[3] The recognition unit
recognizing the first character when the degree of matching between the graphic information to be recognized included in the captured image and the graphic information stored in the graphic information database is equal to or greater than a first threshold (T1);
When the value obtained by integrating the degree of matching between the graphic information at the estimated location and the graphic information contained in the graphic information database within a certain range is equal to or greater than a third threshold (T3), the second character is The specific character string detection device according to [1] above, which specifies a candidate.
[4] The recognition unit determines that the specific character string has been detected when all the characters forming the specific character string are recognized or specified as the candidate characters (S64, S73)
The specific character string detection device according to any one of [1] to [3], characterized by:
[5] A recording unit (17) for recording in chronological order the captured images captured by a camera mounted on a vehicle traveling on the road surface, which are acquired by the image acquisition unit;
The recognition unit calculates a matching degree between the graphic information to be recognized included in the photographed image and the graphic information stored in the graphic information database for each traveling distance of the vehicle, and generates a matching degree graph. is recorded in the recording unit as
The method according to any one of [1] to [4] above, wherein the estimation unit estimates a possible location of the second character based on the position of the first character in the degree-of-match graph. Specific character string detection device described.
[6] The specific character string consists of a plurality of characters having an arrangement order,
The recognition unit recognizes any one of the plurality of characters as the first character,
The estimating unit estimates, based on the position of the first character, the possible location of the second character, which is any one character other than the recognized first character among the plurality of characters. The specific character string detection device according to any one of [1] to [5] above.
[7] A vehicle-mounted device (digital tachograph 10) that is mounted on a vehicle and acquires operation information of the vehicle;
The specific character string detection device (digital tachograph 10) according to any one of [1] to [6] above;
A driving evaluation system (5) comprising a driving evaluation device (office PC 30) that evaluates the driver of the vehicle based on the operation information acquired by the vehicle-mounted device,
The driving evaluation system, wherein the driving evaluation device evaluates the driver based on the operation information when the recognition unit determines that the specific character string is detected.

1 vehicle 5 driving evaluation system 8 radio base station 10 digital tachograph (vehicle device)
11 CPU (image acquisition unit, recognition unit, estimation unit)
12A Speed I/F
12B Engine rotation I/F
13 External input I/F
14 sensor interface
15 GPS receiver 15a GPS antenna 16 camera I/F
17 recording unit 18 card I/F
19 Audio I/F
20 built-in speaker 21 RTC (clock IC)
22 SW input section 23A, 23B Camera 24 Communication section 25 Power supply section 26A Nonvolatile memory 26B Volatile memory 27 Display section 28 G sensor 29 Analog input I/F
30 office PC (driving evaluation device)
51 vehicle speed sensor 65 memory card 70 network C specific character string S road surface

Claims (7)

A specific character string detection device that detects a specific character string including a first character and a second character marked on a road surface,
an image acquisition unit that acquires a captured image including at least the road surface;
a recognition unit that recognizes the first character marked on the road surface in the photographed image by referring to a graphic information database that stores graphic information representing the shape of characters;
an estimation unit that estimates a possible location of the second character based on the position of the first character on the road surface;
The recognition unit
searching for traces of the second character at the estimated location, referring to the graphic information database, and identifying candidate characters that are candidates for the second character;
A specific character string detection device that determines whether or not the specific character string is detected based on the recognition result of the first character and the identification result of the candidate character. The recognition unit
recognizing the first character when the degree of matching between the graphic information to be recognized included in the captured image and the graphic information stored in the graphic information database is equal to or greater than a first threshold;
If the degree of matching between the graphic information at the estimated location and the graphic information contained in the graphic information database is equal to or greater than a second threshold lower than the first threshold, the candidate for the second character is specified. The specific character string detection device according to claim 1, characterized in that: The recognition unit
recognizing the first character when the degree of matching between the graphic information to be recognized included in the captured image and the graphic information stored in the graphic information database is equal to or greater than a first threshold;
If a value obtained by integrating the degree of matching between the graphic information at the estimated location and the graphic information contained in the graphic information database within a certain range is equal to or greater than a third threshold, the candidate for the second character is specified. The specific character string detection device according to claim 1, characterized in that: The recognition unit determines that the specific character string has been detected when all characters forming the specific character string are recognized or specified as the candidate characters. Item 4. The specific character string detection device according to any one of Items 1 to 3. A recording unit that records in time series the captured images captured by a camera mounted on a vehicle traveling on the road surface, which are acquired by the image acquisition unit;
The recognition unit calculates a matching degree between the graphic information to be recognized included in the photographed image and the graphic information stored in the graphic information database for each traveling distance of the vehicle, and generates a matching degree graph. is recorded in the recording unit as
5. The method according to any one of claims 1 to 4, wherein the estimation unit estimates a possible location of the second character based on the position of the first character in the degree-of-match graph. Specific character string detector. The specific character string consists of a plurality of characters having an arrangement order,
The recognition unit recognizes any one of the plurality of characters as the first character,
The estimating unit estimates, based on the position of the first character, the possible location of the second character, which is any one character other than the recognized first character among the plurality of characters. The specific character string detection device according to any one of claims 1 to 5. An on-vehicle device that is mounted on a vehicle and acquires operation information of the vehicle;
A specific character string detection device according to any one of claims 1 to 6,
A driving evaluation system comprising a driving evaluation device that evaluates a driver of the vehicle based on the operation information acquired by the vehicle-mounted device,
The driving evaluation system, wherein the driving evaluation device evaluates the driver based on the operation information when the recognition unit determines that the specific character string is detected.

Images