JP2008176357A - Abnormality diagnostic system and diagnostic information management device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sign guidance device for properly outputting the necessary guide information of a traffic sign without requiring complicated input operation by an operator. <P>SOLUTION: This sign guidance device is provided with: a sign extraction means 2 for extracting a traffic sign from video data picked up by an on-vehicle camera 11; a notification means 3 for outputting the guide information of the traffic sign extracted by the sign extraction means 2; and a sign guidance control means 4 for operating the notification means 3 based on the traveling state of the vehicle detected by the traveling state detection means 12 for monitoring the traveling state of the vehicle. The sign guidance control means 4 operates the notification means 3 when it is determined or predicted that the traveling state detected by the traveling state detection means 12 violates the traffic information shown by the traffic sign extracted by the sign extraction means 2. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a sign guide device provided with a notification means for outputting guide information of a traffic sign, for example, a sign extraction means for extracting a traffic sign from video data captured by an in-vehicle camera, and an extraction by the sign extraction means The present invention relates to a sign guide device provided with notifying means for outputting guide information of a traffic sign.

  For the purpose of displaying an image such as a road sign imaged by an in-vehicle camera or outputting a content of the road sign or the like by voice, the patent document 1 discloses an in-vehicle camera. A road sign etc. extracting means for extracting a road sign etc. from the image captured by the above, a recording means for recording the road sign etc. image extracted by the road sign etc. extracting means, and a display means for displaying a car navigation image And an image output means for performing image editing so that the car navigation image and the image such as the road sign recorded in the recording means can be simultaneously displayed on the display means and outputting to the display means. An equal display device has been proposed.

Patent Document 2 discloses a means for obtaining an image of at least one object in the vicinity of the main road in order to interpret an object along the main road for the vehicle occupant, a means for displaying the image, In response to the means for receiving the selection of at least one object in the image, the means for identifying the object selected in the image, and the means for identifying information about the selected object in a predetermined language to the user of the device An apparatus comprising means for providing has been proposed.
Japanese Patent Laid-Open No. 2005-4410 JP 2003-323893 A

  However, all of the techniques described in the above-described patent documents display or output a guidance for a corresponding sign for a sign instructed by an operator via a touch panel or voice. Therefore, since the operator requires a very complicated operation of operating a voice instruction and a touch panel each time, there is a problem that the concentration during driving may be hindered.

  An object of the present invention is to provide a sign guide device that can appropriately output necessary traffic sign guide information without requiring an operator to perform a troublesome input operation.

  In order to achieve the above-mentioned object, the sign guide device according to the present invention is characterized by a sign guide device comprising a notification means for outputting guide information of a traffic sign, and a driving state detection for monitoring the driving state of the vehicle. There is a sign guide control means for operating the notification means based on the running state detected by the means.

  According to the above-described configuration, when it is determined that the sign guide control means is necessary for the traffic sign from the running state of the vehicle detected by the running state detecting means, the notifying means is activated to provide the guide information of the traffic sign. Since the information is output, the sign information necessary for traveling is notified without the driver being aware of it.

  As described above, according to the present invention, it is possible to provide a sign guide device that can appropriately output necessary traffic sign guide information without requiring an operator to perform a time-consuming input operation. became.

  Hereinafter, an embodiment in which a sign guide device according to the present invention is built in a vehicle navigation device will be described.

  As shown in FIG. 1, a vehicle includes a plurality of electronic control units (hereinafter referred to as ECUs) 300 that are network-connected via a CAN (controller area network) bus 200. It is configured to perform the desired control in cooperation. The plurality of ECUs 300 include, for example, a large number of ECUs such as an engine control ECU, a brake control ECU, and a steering control ECU.

  In addition, each ECU 300 includes a traveling state detection means 12 that periodically monitors data indicating the state of a control target block that it controls (engine in the case of an engine control ECU, brake in the case of a brake control ECU, etc.). The traveling state detection means 12 transmits the monitored data to the sign guide device 1 according to the present invention described below via the CAN bus 200.

  The CAN bus 200 is connected to a navigation device 400 that displays the current position of the vehicle by a map or voice, or guides to a destination. The navigation device 400 includes a plurality of menus. A liquid crystal touch panel type display means 410 for detecting an operation on each menu key on which a key or the like is displayed, an operation means 420 comprising various input keys arranged on the outer edge of the display section 420, the sign guide device 1 and the like It is configured with.

  The plurality of ECUs 300 and the navigation device 400 described above include an input interface circuit that receives input signals from other ECUs 300, a ROM for storing control programs, and a working used for data processing. It is configured to include a RAM as an area, a CPU that executes the control program, and other necessary peripheral circuits. The control program executed by the CPU realizes a predetermined function. Has been.

  The sign guide device 1 includes a sign extracting means 2 for extracting a traffic sign from video data captured by the in-vehicle camera 11, a notifying means 3 for outputting guide information of the traffic sign extracted by the sign extracting means 2, The sign guide control means 4 is configured to operate the notifying means 3 based on the running condition detected by the running condition detecting means 12 for monitoring the running condition of the vehicle. This is realized by the control program being calculated and executed by the navigation device 400.

  In addition, the sign guide device 1 automatically extracts a traffic sign by the sign extraction means 2, and performs an operation of the notification means 3 by the sign guidance control means 4, and the extraction of the traffic sign or And / or a mode changeover switch (not shown) for switching between manual mode in which the operation of the notification means 3 is performed by manual operation such as voice instruction or switch pressing.

  The mode switch is provided in the display area of the display unit 410 or the lower outer edge of the display unit 410 of the navigation device 400. In this embodiment, the case where the mode change switch is set to the auto mode will be described, and the manual mode will be described in another embodiment.

  The in-vehicle camera 11 is a fixed camera that is installed in front of the vehicle such as a front bumper or the frontmost part of the roof panel and is used for measuring a distance from the front vehicle or an obstacle, or a vehicle top part such as a center part of the roof panel of the vehicle. A rotary camera that is installed in the vicinity and is used for monitoring the surroundings of the front, rear, left, and right is used.

  The sign extracting means 2 selects video data analyzing means 20 for selecting predetermined frame data from a series of frame data constituting video data captured by the in-vehicle camera 11 and outputting the selected frame data to the shape extracting means 22; It is drawn on the traffic sign extracted by the shape extracting means 21 and the shape extracting means 22 for extracting a traffic sign that matches the contour shape of the traffic sign in the sign shape database 21 from the video data received from the video data analyzing means 20. It comprises traffic information extracting means 24 for extracting traffic information that matches the graphic or character information in the sign information database 23 from the graphic or character information. That is, the label extracting means 2 is hierarchized for speeding up the matching process.

  The video data analyzing means 20 includes a video memory (not shown) that stores each of a series of frame data constituting the video data received from the in-vehicle camera 11 for a certain period of time. Of the series of video data stored in the video memory, the frame data received in the past for a predetermined time from the latest frame data received last is output to the shape extracting means 22.

  The predetermined time is determined based on, for example, the traveling speed of the vehicle. When the vehicle speed is low, the predetermined time is set short, and when the vehicle speed is high, the predetermined time is set long. This prevents the vehicle from passing the traffic sign before the notification means 3 outputs the guide information of the traffic sign.

  The sign shape database 21 is composed of a storage medium such as a hard disk having a capacity larger than that of the ROM, and contour shape data existing in the traffic sign is registered and managed separately for the main sign and the auxiliary sign. .

  The shape extraction unit 22 compares the frame data received from the video data analysis unit 20 with each of the contour shape data registered in the marker shape database 21, and the matching region with the contour shape data indicates the matching region. When it is detected in the video data, the data of the matching area is cut out from the frame data and output to the traffic information extracting means 24 described later.

  Further, the shape extracting means 22 is shown as the main sign as shown in FIG. 2 at the position near the top and bottom of the shape shown as the auxiliary sign as shown in FIG. When a certain shape is detected, these two shapes are associated with each other and output to the traffic information extracting means 24.

  Furthermore, in the shape extraction means 22, in the case of FIGS. 3A and 3B in which a plurality of main signs exist in the vicinity of the auxiliary signs, the auxiliary signs are output in association with any of the main signs. The priority order is set. For example, in the case of FIG. 3, when the shape of the auxiliary sign is a small circle, it is preferentially associated with the lower main sign, and when the shape of the auxiliary sign is square, it is preferentially associated with the upper main sign. In this case, the auxiliary sign A and the main sign C are output in association with each other, and in the case of FIG. 3B, the auxiliary sign is output. D and this mark E are associated and output.

  The sign information database 23 is composed of a storage medium such as a hard disk having a larger capacity than the ROM, and as shown in FIG. 2, graphic information composed of colors and diagrams drawn on traffic signs. And character information composed of letters and numbers are registered and managed together with content information as semantic content corresponding to these pieces of information. In the present embodiment, the graphic information is classified and registered for each contour shape data, and the character information is registered and managed regardless of the contour shape data.

  The traffic information extracting unit 24 extracts a graphic portion corresponding to a color or a diagram from the data of the matching area received from the shape extracting unit 22, and the graphic information registered in the sign information database 23. The graphic information extracting means 241 for detecting graphic information matching the data drawn in the graphic portion from the sign information database 23, and the data of the matching area received from the shape extracting means 22 The character part corresponding to a character or a number is extracted, the character part as graphic data is converted into a character code as text data based on a known technique such as optical character recognition (OCR), and its meaning Character information recognition means 242 for detecting the character information as recognized character information, the graphic information detected by the graphic information extraction means and the sentence It is constituted by a traffic information generation unit 243 for generating traffic information as information that associates the detected character information by the information recognition means.

  And the said traffic information extraction means 24 outputs the produced | generated said traffic information to the alerting | reporting means 3 as guide information of the extracted traffic sign. If there is no information in the sign information database 23 that matches the data in the matching area, the traffic information extracting unit 24 determines that the data in the matching area is not a traffic sign.

  Hereinafter, an example in which the sign extraction means 2 extracts a traffic sign M1 as shown in FIG. 3C will be described.

  First, the shape extracting means 22 extracts two matching areas M11 and M12. Since the coincidence area M11 has a shape as a main sign and the coincidence area M12 has a shape as an auxiliary sign and is in a nearby position, the coincidence areas M11 and M12 are associated with each other and output to the traffic information extracting unit 24.

  In the traffic information extracting means 24, the graphic information extracting means 241 applies the matching area M11 to the sign information database 23, thereby extracting the graphic information M13 shown in FIG. Further, the character information recognition means 242 applies the character portion 112 of the matching area M11 and the character portion of the matching area M12 to the sign information database 23, so that the character portion “8-20” becomes the character information “8 o'clock”. To 20:00 ”and the character part“ excluding Sundays and holidays ”is extracted as the character information“ excluding Sundays and holidays ”.

  Then, the traffic information generating means 243 generates traffic information “parking prohibited from 8:00 to 20:00 excluding Sundays and holidays” from the content information “parking prohibited” and the character information of the graphic information M13. The

  The notification means 3 outputs the traffic sign guide information extracted by the sign extraction means 2. The notification means 3 is operated based on a control signal from the sign guidance control means 4 described in the following paragraphs.

  When the notification means 3 is operated, the guide information is displayed on the display means 410 provided in the navigation device 400. At this time, the notifying unit 3 changes the display of the display unit 410 from the original display to be displayed on the navigation device 400, for example, a navigation screen on which GPS map information or the like ahead of the vehicle is displayed. ), The display mode is switched to the display mode in which the extracted traffic sign and the guide information are displayed. Or the structure which opens the window where the traffic sign extracted on the said navigation screen and the said guide information are displayed may be sufficient.

  The sign guide control unit 4 includes a traveling state storage unit 41 that stores data indicating the traveling state of the vehicle detected and transmitted by the traveling state detection unit 12 of each ECU 300. The traveling state storage means 41 stores each traveling state and its value in association with each other in time series. The sign guide control unit 4 periodically refers to each data stored in the traveling state storage unit 41 and operates for the notification unit 3 when each data satisfies a predetermined condition. A control signal for starting is transmitted.

  Here, the running state detected by the running state detecting means 12 is an operating state or an operating state for any one or a combination of vehicle speed, gear ratio, direction indicator, brake, steering and the like.

  The operating state is a current traveling state, for example, a current speed displayed on a speedometer, a current transmission gear position selected by a shift lever, and a vehicle indicated by a direction indicator. Direction of travel (turns right when the right turn indicator lights, turns left when the left turn indicator lights, and goes straight without turning on the turn indicator), whether the brake pedal is depressed by the driver, and steering position ( There is a right turn state when the steering wheel is rotated to the right from the predetermined position, a left turn state when the steering wheel is rotated to the left from the predetermined position, and a straight traveling state when the steering wheel is located at the center of the predetermined position). .

  The operation state is the change when the driving state changes due to the driver's operation. For example, the speed increases due to the accelerator pedal being depressed, and the accelerator pedal that was depressed is released. Decrease in speed, change in gear position due to operation of shift lever, change in state of direction indicator due to operation of turn signal switch, decrease in speed due to depression of brake pedal, brake pedal being depressed There is a decrease in the speed decrease frequency due to the release of.

  Since the traveling state shown above is an element indicating the actual movement of the vehicle, it is a suitable element for use in determining whether or not to activate the notification means.

  Further, the predetermined condition is that the sign guide control means 4 needs to show the guide information of the traffic sign extracted by the sign extraction means 2 from the operating state or the operation state to the driver of the vehicle. This is the case.

  For example, the sign guidance control means 4 activates the notification means 3 when a driving state is detected in which it is determined that the driver of the vehicle is conscious of watching the traffic sign.

  Specifically, when the vehicle driver decreases the speed of the vehicle by releasing the accelerator pedal or depressing the brake pedal, the sign guidance control means 4 is configured to display traffic signs that are difficult for the driver of the vehicle to see at high speed. In order to recognize clearly, it judges that it shifted to low speed driving | running | working and the said alerting | reporting means 3 is operated.

  Even when the predetermined condition is satisfied, the vehicle is decelerating due to a cause such as waiting for a signal due to a red signal detected as a traffic sign ahead by the sign extraction means 2. Is found, the sign guidance control means 4 does not operate the notification means 3.

  As described in the previous paragraph, the determination that the predetermined means is satisfied but the notification means 3 is not activated is performed as follows. That is, the sign guidance control means 4 accumulates and stores the driving state of the vehicle and the change in the driving state in the situation where the traffic sign is reflected in the image captured by the in-vehicle camera 11, that is, the driving pattern for each traffic sign. The notification means 3 is provided depending on whether or not the traveling state referred to in the traveling state storage means 41 matches the traveling pattern stored in the determination database 42. It is determined whether or not to activate.

  The discrimination database 42 is constructed as follows, for example. That is, a switch such as a momentary switch that can be manually turned on is installed in many vehicles in advance, and the switch is turned on when the driver of each vehicle wants to obtain information about traffic signs in the field of view. A survey of many vehicles is conducted. In the investigation, the traveling state of the vehicle when the switch is turned on is stored.

  Then, an external information collecting device is connected to each vehicle later, and when the switch is turned on, information on the traffic sign that the driver wanted to obtain information and the running state of the vehicle at that time are collected. To construct the discrimination database. Of course, the configuration is such that the running state of the vehicle when the switch is turned on is transmitted to an information center or the like responsible for constructing a discrimination database by using an existing communication line such as satellite communication. May be.

  By referring to the constructed discrimination database 42, for example, vehicle deceleration or the like is performed based on a driver's request to view traffic signs or simply because of waiting for a signal or the like. It is possible to determine the difference between the two.

  In addition, the sign guide control unit 4 is configured to determine or predict that the traveling state detected by the traveling state detection unit 12 is contrary to the traffic information indicated by the traffic sign extracted by the sign extracting unit 2. The notification means 3 is activated.

  More specifically, the sign guide control means 4 compares the detected traveling state (for example, vehicle speed) with the traffic sign (for example, the speed sign) related to the traveling state among the extracted traffic signs. When the traveling state violates the contents shown in the traffic sign, the notification means 3 is activated by determining or predicting that the driving information is contrary to the traffic information. A specific example will be described below.

  As a first example, when the sign extraction means 2 has extracted a traffic sign that prohibits vehicle entry on a road in the right turn direction, it is said that the driver has switched the turn signal switch of the vehicle in the right turn direction. When detected by the state detection means 12, the sign guidance control means 4 activates the notification means 2 to prompt the driver to recognize the presence of a traffic sign that prohibits vehicle entry.

  As a second example, the sign extracting means 2 extracts a traffic sign that prohibits traveling in the specified direction (one-way) on the road in the left turn direction, and the extracted traffic sign is in a direction opposite to the traveling direction of the vehicle. If the driving state detecting means 12 detects that the turn signal switch of the vehicle has been switched to the left turn direction by the driver, the sign guide control means 4 activates the notifying means 2. The driver is alerted by recognizing the presence of a traffic sign that prohibits traveling outside the specified direction (traffic in that direction).

  As a third example, in the first and second examples, even if the driver actually turns the vehicle right or left by operating the steering wheel without operating the turn signal switch of the vehicle, When the operating state of the change of the steering position is detected by the traveling state detecting unit 12, the sign guide control unit 4 operates the notifying unit 2 to prohibit the driver from entering the vehicle or prohibiting the vehicle from traveling in the specified direction. Call attention by recognizing the presence of traffic signs.

  As a fourth example, when the sign extracting means 2 is extracting a traffic sign that is temporarily stopped in front of the vehicle, the traveling state detecting means 12 detects that the driver is not decelerating the vehicle. Then, the sign guide control means 4 operates the notifying means 2 to make the driver recognize the traffic sign of the stop and detect the stop line in front of the vehicle to call attention.

  Here, the detection of the stop line is performed as follows. That is, the contour shape of the stop line is registered and managed in the line shape category provided separately from the main sign and the auxiliary sign in the sign shape database 21, and the stop traffic sign is extracted by the sign extracting means 2. In this case, as shown in the screen example of FIG. 3 (e), the guide information output by the notification means 2 is displayed with the distance to the stop line added and displayed. Done by command.

  That is, the sign guidance control means 4 is predicted to violate the traffic information indicated by the traffic sign extracted by the sign extraction means 2 as in the first, second, and fourth examples. The notification means 3 is activated not only when the traffic information is actually violated, as in the third example.

  The driver should check the traffic signs when there is a change in the driving condition due to deceleration, etc., or when it is in a special driving condition such as parking or stopping, compared to normal driving at a constant speed. Will increase. Therefore, according to the above-described configuration, the sign guidance control means operates the notification means based on the traveling state, so that the driver surely confirms the traffic sign guide information in a situation where the driver often confirms the traffic sign. be able to.

  In addition, many traffic signs issue warnings to the driver, and if such signs are missed, traffic violations may occur and sometimes they are dangerous. Therefore, according to the above configuration, when there is a possibility of traffic violation or when the traffic violation has actually occurred, the sign guidance control means activates the notification means, so that the driver may cause a traffic violation. It is possible to quickly confirm that there has been a traffic violation.

  Hereinafter, the processing operation of the sign extracting means 2 and the notifying means 3 of the sign guiding apparatus 1 according to the present invention will be described based on the flowchart shown in FIG.

  When video data is input from the in-vehicle camera 11 to the sign guide device 1 (SA1), the video data analyzing means 20 selects predetermined frame data from a series of frame data constituting the video data and extracts the shape. It outputs to the means 22 (SA2).

  The shape extracting means 22 that has received the frame data compares the frame data with each of the contour shape data registered in the marker shape database 21. If a matching area is detected (SA3), the matching data is detected. An area is cut out from the frame data (SA4).

  If the shape of the auxiliary sign and the shape of the main sign located in the vicinity of the shape of the auxiliary sign are detected in the cutout process in step SA4 (SA5), the shape extracting unit 22 associates the data of both shapes. And output to the traffic information extracting means 24 (SA6), otherwise (SA5), the cut out data is output to the traffic information extracting means as it is.

  The traffic information extracting unit 24 compares the data of the matching area received from the shape extracting unit 22 with the graphic information and / or character information stored in the sign information database 23, and matches the graphic information and / or character information. Is detected from the sign information database 23 (SA7), the contents information thereof is generated as traffic information (SA8) and output to the notification means 3.

  When receiving the control signal from the sign guidance control means 4 (SA9), the notification means 3 outputs and displays the traffic information as traffic sign guide information (SA10).

  If no coincidence area is detected in step SA3, no coincidence information is detected in step SA7, or if no control signal is received in step SA9, the process returns to step SA1 to detect traffic signs again. To do.

  Hereinafter, the processing operation of the sign guidance control means 4 of the sign guidance apparatus 1 according to the present invention will be described based on the flowchart shown in FIG.

  The sign guide control unit 4 refers to the data from the traveling state detection unit 12 stored in the traveling state storage unit 41 (SB1). And when the driving | running | working state of a vehicle comprises the predetermined conditions (SB2) and when it does not correspond with the driving | running | working pattern of the discrimination | determination database 42 (SB3), it transmits the control signal to the alerting | reporting means 3 above. The notification means 3 is activated (SB4).

  On the other hand, when the driving state of the vehicle does not satisfy the predetermined condition at step SB2 (SB2), or when it matches the driving pattern of the discrimination database 42 at step SB3 (SB3), the notification means 3 is activated. I won't let you.

  Hereinafter, another embodiment will be described. In the above-described embodiment, the screen displayed on the display unit 410 by the notification unit 3 has been described with reference to the configuration in which only one extracted traffic sign is displayed as described in FIG. When there are a plurality of traffic signs extracted from the video data captured by the in-vehicle camera 11, the extracted number of traffic signs may be displayed.

  In this case, the screen on which the extracted traffic signs and the guide information about the traffic signs are displayed is divided into areas and displayed in a predetermined arrangement. For example, in the case of FIG. 5A, the screen is divided into nine areas, the priority is set for the displayed guide information, and the traffic sign with high priority is set closer to the driver. The low traffic sign is displayed on the side far from the driver. That is, the area on the right side of the screen where the driver is sitting and the uppermost area is set as the area A1 having the highest priority, and the area on the left side and the lowermost stage is set as the area A9 having the lowest priority. FIG. 5A shows an example in which four traffic signs are extracted, and areas A5 to A9 are not displayed.

  Then, when any one of the areas A1 to A9 in FIG. 5A is pressed by the driver of the vehicle or the like, the notification means 3 has traffic corresponding to the pressed area as shown in FIG. 5B. It is configured to inform the contents of the sign.

  The above-mentioned priority is set so that the priority of traffic signs existing near the vehicle is high. The distance from the vehicle to the traffic sign is determined by the area of the coincidence area extracted by the sign extraction means 2. That is, it is determined that the distance from the vehicle to the traffic sign corresponding to the matching region is shorter as the area of the matching region is larger.

  For example, when the video data of the field of view ahead of the vehicle imaged by the in-vehicle camera 11 is an image as shown in FIG. 6A, the priority of the traffic sign M21 closest to the vehicle is set to be the highest, and the vehicle As a result of setting the lowest priority of the traffic sign M23 farthest from the traffic sign M21, the traffic sign M21 is in the area A1 in FIG. 5A, the traffic sign M22 is in the area A2 in FIG. 5A, and the traffic sign M23 is They are displayed in the area A3 in FIG.

  In the above-described embodiment, the guide information is displayed on the display unit 410. However, the guide information may be an audio output through a speaker provided in a car audio or the like.

  In the case of audio output, as shown in FIG. 6 (a), if there are a plurality of traffic signs M21 to M23 in front of the vehicle, the guide information output via the speaker is any traffic sign information. Although it cannot be recognized, it can be recognized by increasing the priority of the traffic sign near the vehicle and outputting the voice from the guide information of the traffic sign having a high priority.

  That is, after the guide information of the traffic sign M21 closest to the vehicle is output by voice, when the vehicle passes the traffic sign and the traffic sign M21 disappears from the field of view, as shown in FIG. Since the traffic sign M22 which is the second priority is the first priority, the notification means 3 outputs a voice of the guide information of the traffic sign M22 which is newly the first priority. .

  In addition, although the screen display and audio | voice output were demonstrated above, the structure which notifies by turning on a lamp | ramp when another output format, for example, a specific traffic sign, is extracted may be sufficient.

  In the above-described embodiment, the configuration in which the mode by the mode changeover switch provided in the sign guide device 1 is set to the auto mode has been described, but the configuration in which the mode is set to the manual mode will be described below.

  This will be described in detail below. As shown in FIG. 7, the sign extracting means 2 is an area specifying means for limiting the range for extracting traffic signs from the video data based on the instruction information input via the voice instruction means 13 or the contact instruction means 15. 25.

  The voice instruction means 13 includes a ROM for storing a control program, a RAM as a working area, a CPU for calculating and executing the control program, a peripheral circuit, and the like. It is configured as a voice analysis device that executes the analysis process, that is, the function of the voice recognition means 132 described in detail below.

  The voice instruction means 13 includes voice recognition means 132 that analyzes voice instruction information input via the voice input means 14 based on a corpus database 131. The operation of the traffic sign extraction and notification means 3 is performed by the voice This is performed based on the voice instruction information analyzed by the recognition means 132.

  The voice input means 14 is provided on the outer edge of the display unit of the car navigation device, the steering column, or the like, and is composed of a microphone that captures the voice of the driver of the vehicle.

  The voice instruction information is voice generated by a vehicle driver or the like, and is information composed of voice taken in by the voice input means 14. The volume of the voice that can be the voice instruction information depends on the sensitivity of the microphone constituting the voice input unit 14.

  Specific examples of such voice instruction information are voices generated by a driver of a vehicle, etc., which are configured in a language related to designation of a traffic sign, such as “traffic sign”, “road sign”, “right front” There is audio information such as “a certain sign”, “a sign with a car batten”, and “an inverted triangular road sign that says left side stop”.

  The voice instruction means 13 includes a corpus database 131 for analyzing voice information as in the example of the previous paragraph. The corpus database 131 is composed of a storage medium such as a large-capacity hard disk, and is a database that collects many words that are expressed when a human sees a traffic sign. That is, the corpus database 131 is classified into a traffic sign category, a direction category, a sign content category, and the like, and each word is classified and stored in each category.

  The voice recognition unit 132 segments arbitrary voice instruction information input via the voice input unit 14 for each word, and applies each segmented word to the corpus database 131 to thereby generate the voice instruction information. Is analyzed, and the analysis result is output to the area specifying means 25 as specified information.

  As a first example, the voice recognition unit 132 applies any voice instruction information (for example, “traffic sign”) input via the voice input unit 14 to the corpus database 131. The word “traffic sign” is classified into a traffic sign category, and the voice recognition unit 132 analyzes the meaning of the input voice instruction information as “traffic sign extraction start command”, The analyzed instruction information is output to the sign extraction means 2.

  As a second example, the voice recognition unit 132 applies any voice instruction information (for example, “right side”) input via the voice input unit 14 to the corpus database 131. Since the word “right” is classified into the direction category, the voice recognition unit 132 analyzes the meaning of the input voice instruction information as “direction = right” and designates “right”. Information is output to the area specifying means 25.

  As a third example, the voice recognizing means 132 sets the voice instruction information “left side, stop”, “inverted triangle”, “road sign” as “left side stop”. Are applied to the corpus database 131. As a result, each word is classified into the direction of “left side”, the sign of “stop” and “inverted triangle”, and the sign of “road sign”, and the sign of traffic sign. The meaning of the voice instruction information is analyzed as “direction = left”, “type of traffic sign = pause sign”, and the designation information “pause sign present on the left” is designated as the region designation. The data is output to the means 25.

  As a fourth example, the voice recognition unit 132 applies voice instruction information “label content” to the corpus database 131. The word “sign content” is classified into the category of sign content, and the voice recognition means 132 has the meaning of the inputted voice instruction information as “display instruction of designated traffic sign content”. The analyzed instruction information is output to the notification means 3 via the sign extraction means 2. In this case, if the designated traffic sign does not exist, an error message will be displayed.

  According to the above-described configuration, the driver can determine the instruction information for specifying the traffic sign extraction instruction and the extraction range by voice, so the extraction instruction and the extraction range instruction can be safely executed while driving the vehicle. it can.

  The contact instruction unit 15 includes a ROM for storing a control program, a RAM as a working area, a CPU for executing the control program, a peripheral circuit, and the like, and the control program is input to the touch panel 151. On the basis of the coordinates thus obtained, the extraction range determination device is configured to execute processing for determining a range in which a traffic sign is extracted from video data captured by the in-vehicle camera 11.

  The contact instruction means 15 includes a touch panel 151 for inputting coordinates by contact, and the extraction of traffic signs and the operation of the notification means 3 are performed based on the coordinates input to the touch panel 151. In the present embodiment, the touch panel 151 is also used as the display unit 410 of the navigation device 400.

  If the touch panel 151 has a configuration for inputting coordinates by contact, in addition to the display means 410, the touch panel 151 is also used as a display means provided in each part of the vehicle. For example, there is a vehicle windshield provided with a touch sensor and having a configuration in which coordinates are input by contact with the windshield.

  When the touch panel 151 is also used as the display unit 410, a navigation screen composed of GPS map information or the like in front of the vehicle is normally displayed. Therefore, the navigation device 400 has a display area within the display area of the display unit 410. Alternatively, a screen changeover switch is provided in the lower part of the navigation device 400, and the screen display is switched to an image captured by the in-vehicle camera 11 by operating the screen changeover switch.

  As shown in FIG. 8A, the contact instruction means 15 is touched when the driver of the vehicle touches the touch panel 151 when the screen display is switched to an image captured by the in-vehicle camera 11. A predetermined range S1 surrounded by a one-dot chain line in the figure with the coordinates as the center is set as the instruction information to be output to the region specifying means 25.

  Moreover, the structure provided with the sign content display switch which displays the content of the traffic sign designated by pressing on the said touch panel 151 may be sufficient. That is, after a vehicle driver or the like touches the touch panel 151 to determine the predetermined range S1, the content of the traffic sign existing in the predetermined range S1 is displayed on the touch panel 151 by pressing the sign content display switch. It is done.

  As shown in FIG. 8B, when the windshield 100 having a touch sensor is used as the touch panel 151, the windshield 100 including the passenger seat 102 side while sitting on the driver's seat 101 and driving the vehicle. In order to show the touchable area S2, the area where the vehicle driving safety is not hindered, that is, the area display is drawn on the windshield so as to be inconspicuous. May be. That is, only when the driver touches any coordinate in the touchable area S2, the predetermined range S1 centered on the touched coordinate is output to the area designating unit 25 as the instruction information.

  According to the above-described configuration, the driver can specify by directly touching the display of the traffic sign displayed in the video or the like of the in-vehicle camera 11, so that the extraction range can be specified accurately.

  The area designating unit 25 is configured to transmit traffic information from the video data based on the instruction information input as the voice instruction information from the voice instruction unit 13 or the instruction information input as the coordinate range information from the contact instruction unit 15. Determine the extent to which the label is extracted.

  For example, in the case of instruction information from the voice instruction means 13, the area designating means 25 indicates the entire range of the video data when the instruction information is “entire screen” and when the instruction information is “right”. The range on the right half of the video data is set as a range for extracting traffic signs. Further, when the instruction information is “a stop sign on the left”, a stop traffic sign is extracted from the left half of the video data.

  Further, in the case of the instruction information from the contact instruction means 15, the area specifying means 25 sets the predetermined range S1 as the instruction information as a range for extracting a traffic sign. Therefore, in the case of FIG. 8A, only the traffic sign M31 is extracted, and the traffic sign M32 is not extracted.

  According to the above-described configuration, by limiting the range from which traffic signs are extracted by the voice instruction means 13 and the contact instruction means 15, it is possible to reliably extract only the traffic signs that the driver desires to obtain guide information. Can do.

  In the above-described embodiment, the sign guide control means 4 has been described with respect to the configuration in which the notifying means 3 is operated based on the travel state detected by the travel state detection means 12, but the sign guide control means 4 is shown in FIG. As shown in a), the notification means 3 is operated when instructed by the voice instruction means 13 or the contact instruction means 15, that is, regardless of the running state detected by the running state detection means 12. Also good.

  More specifically, the voice instruction information of the vehicle driver or the like input via the voice input means 14 is analyzed by the voice recognition means 132, and the analyzed voice instruction information is output to the sign guidance control means 4. The sign guidance control means 4 operates the notification means 3 based on the received voice instruction information.

  Similarly, coordinate information input via the contact instruction means 15, that is, coordinate information determined by a vehicle driver or the like touching the touch panel 151 is output to the sign guidance control means 4. The sign guidance control means 4 operates the notification means 3 based on the received coordinate information.

  Examples are given below. As a first example, the switching of the display screen from the navigation screen display on the display means 410 of the navigation device 400 executed when the notification means 3 is operated to the video display by the in-vehicle camera 11 for detecting a traffic sign is as follows: Instead of using a screen switching switch, it is executed as follows.

  That is, the voice instruction means 13 that has received a predetermined utterance of the driver (for example, utterance of “switch to camera image”) displays the instruction information for displaying the video by the in-vehicle camera 11 on the display means 410. Output to the sign guidance control means 4. The sign guidance control unit 4 operates the notifying unit 3 to switch the screen display of the display unit 410 with the reception of the instruction information as a trigger.

  As a second example, the display of guide information that is executed when the notifying means 3 is operated is executed as follows.

  That is, the voice instruction unit 13 that has received a predetermined utterance of the driver (for example, utterance of “traffic sign”) displays instruction information for displaying the guide information of the traffic sign on the display unit 410. Output to means 4. The sign guidance control means 4 causes the display means 410 to display a list of traffic signs extracted by the sign extraction means 2 by operating the notification means 3 with the reception of the instruction information as a trigger.

  As a third example, there is the following configuration. That is, when the guide information is output by the operation of the notification means 3, the voice instruction means 13 that has received a predetermined utterance of the driver (for example, the utterance of "Tell me details") 23, instruction information for displaying on the display screen detailed information that is more easily understood by the driver or the like stored in the display 23 is output to the sign guide means 4. The sign guide unit 4 operates the notifying unit 3 to display the detailed information on the display screen with the reception of the instruction information as a trigger.

  Specifically, triggered by the receipt of the driver's "Tell me more" utterance, in addition to the content information of traffic signs already output (eg "pedestrians only"), Detailed notification of “pedestrian only” is made, such as “it is impossible to enter by car because it is impossible to pass”.

  As a fourth example, display screen switching similar to the first example may be executed as follows.

  That is, when the navigation screen is displayed on the display unit 410, the contact instruction unit 15 causes the display unit 410 to display the video by the in-vehicle camera 11 when the driver touches predetermined coordinates on the display screen. Instruction information for display is output to the sign guide means 4. The sign guide unit 4 operates the notifying unit 3 to switch the display of the display unit 410 with the reception of the instruction information as a trigger.

  In the above-described embodiment, the sign guidance control unit 4 has been described with respect to the configuration in which the notification unit 3 is operated based on the traveling state detected by the traveling state detection unit 12, but as shown in FIG. The sign guidance control means 4 includes a situation judgment means 41 for judging the situation of the vehicle, and the situation judgment means 41 replaces the operation of the notification means 3 based on the traveling state, or In addition to operating the notification means 3 based on the running state, the notification means 3 may be operated when the vehicle is placed in a predetermined situation.

  The situation determination means 41 determines whether or not the vehicle is in the predetermined situation based on one of the violation database 16 and the accident database 17 or a combination thereof.

  The violation database 16 is composed of a storage medium such as a large-capacity hard disk, and as shown in FIG. 10, traffic signs are statistically registered with respect to the type of road, the type of area, etc. It is managed. In addition, the violation database 16 determines that traffic signs tend to be violated by the public who have not been able to remember the meaning of the contents by conducting a pre-investigation. It is set high and memorized. The violation frequency shown in FIG. 10 does not match the actual statistics. In FIG. 10, the violation frequency is displayed in five levels from “A” to “E”, but is not limited to five levels.

  The situation determination means 41 compares the traffic sign extracted by the sign extraction means 2 with the violation database 16 and, when a traffic sign with a high violation frequency is detected, the traffic sign is given priority. Notification (for example, display in an area with a high priority in FIG. 5A) or notification of only traffic signs whose violation frequency is higher than a predetermined frequency.

  In FIG. 10, the exclusion flag is a flag for designating that the traffic sign with the exclusion flag set is not notified even if the situation judging means 41 judges that the vehicle is placed in a predetermined situation. It is set by a driver or a dealer. For example, a sign indicating a speed limit among traffic signs may cause an excessive notice if the notice is given even when traveling on a road with a speed limit of 50 km at 51 km. The sign is not notified.

  The accident database 17 is composed of a storage medium such as a large-capacity hard disk, and the locations where accidents have occurred in the past, the details of the accidents, the areas where accidents occur frequently, and the like are statistically registered and managed.

  The situation determination means 41 compares the traffic sign extracted by the sign extraction means 2 with the accident database 17, and frequently causes accidents in areas where vehicles exist (for example, due to excessive speed). When a traffic sign related to (collision accident) (for example, a traffic sign for a speed limit) is detected, the traffic sign is preferentially notified (for example, in a high priority area in FIG. 5A). Or display only the relevant traffic sign.

  In addition, the situation determination means 41 is configured to use an artificial satellite via data stored in a navigation device 400 mounted on a vehicle or a GPS receiver at the time of comparison with the violation database 16 or the accident database 17. It is also possible to use a configuration that uses data fetched from the above.

  For example, when the situation determination means 41 compares the traffic sign extracted by the sign extraction means 2 with the violation database 16, the situation determination means 41 indicates the type of road or region in which the vehicle is currently traveling in the data in the navigation device 400. There is a configuration to acquire more.

  As another example, when the situation determination unit 41 compares the traffic sign extracted by the sign extraction unit 2 with the accident database 16, the situation determination unit 41 obtains the current address from the data in the navigation device 400. There is a structure to clarify information such as where the accident occurred and the area where the accident occurred frequently.

  In the above-described embodiment, the configuration has been described in which the area specifying unit 25 limits the range in which the traffic sign is extracted from the video data based on the instruction information input through the voice instruction unit 13 or the contact instruction unit 15. As shown in FIG. 11A, the area designating unit 25 extracts a range in which a traffic sign is extracted from video data captured by the in-vehicle camera 11 based on instruction information input via the line-of-sight detection device 18. A limited configuration may be used.

  The line-of-sight detection device 18 may use a device that employs a known technique such as detecting the direction of the line of sight by imaging a human eyeball with an infrared camera or the like and detecting a reflected image of the pupil or corneal surface. it can.

  This will be described in detail below. The line-of-sight detection device 18 has a face imaging camera installed at a position suitable for imaging a driver's face contour and eyeballs, such as a steering column, and periodically displays the driver's face contours and eyeballs. Take an image. The gaze detection device 18 detects the driver's gaze direction and face direction based on the face information transmitted from the face imaging camera, and uses the detected gaze direction and face direction as instruction information. The data is output to the area specifying means 25 of the extracting means 2.

  When the area designating unit 25 receives the instruction information having the substantially same line-of-sight direction and face direction for a predetermined number of times, the area designating unit 25 displays the contents of the traffic sign in the line-of-sight direction and face direction based on the instruction information. It is determined that the user wants to know, and the video data in the direction based on the instruction information among the video data captured by the in-vehicle camera 11 is set as a range for extracting a traffic sign.

  Further, the line-of-sight detection device 18 may be configured to output the detected line-of-sight direction and face direction instruction information to the notification means 3 as indicated by a broken line in FIG.

  For example, the notification means 3 that has received the instruction information from the line-of-sight detection device 18 is outside a predetermined range centered around a direction in which the line-of-sight direction and face direction are straight forward for a predetermined period of time. If it is the information, it is determined that the driver is driving aside, and a warning display and a warning sound are output instead of outputting the traffic sign and its guide information.

  As another example, the notification means 3 that has received the instruction information from the line-of-sight detection device 18 indicates that when the instruction information is information that the detection of the line-of-sight direction has failed continuously for a predetermined time, It is determined that the eyes are closed, that is, the driver is dozing, and a warning display and a warning sound are output instead of outputting the traffic sign and its guide information.

  In addition to the above-described embodiment, the sign guide device 1 includes sign guide condition learning means 5 for accumulating and managing the output history of past traffic sign guide information as shown in FIG. 11B. It may be a configuration.

  The sign guide condition learning means 5 indicates what kind of traffic sign the driver of the vehicle has requested in the past to output the guide information, and in what kind of driving state the guide information of the traffic sign A history database that stores information such as whether output is requested is provided.

  The sign guide control means 4 compares the traffic sign extracted by the sign extraction means 2 with the history database, for example, and when the frequency of the traffic sign guide information output request is higher than a predetermined ratio, The notification means 3 is operated to execute processing such as preferential notification of the traffic sign.

  As another example, the sign guide control unit 4 compares the traveling state detected by the traveling state detection unit 12 with the history database, and outputs an output request for guide information of the traffic sign in the traveling state. If the frequency is greater than a predetermined ratio, the notification means 3 is operated to execute processing such as preferential notification of the traffic sign.

  In the above-described embodiment, the sign guide device 1 has been described as using a sign used in Japan. However, when the sign guide device 1 is used in a country other than Japan, The structure using the traffic sign may be used. In that case, the traffic sign information of the country is stored in each database and used.

  In the above-described embodiment, the structure in which the sign extraction unit 2 includes the sign shape database 21 and the sign information database 23 has been described. However, the sign shape database 21 and / or the sign information database 23 are externally attached. It may be.

  In the above embodiment, the configuration in which the voice instruction unit 13 includes the corpus database 131 has been described. However, the corpus database 131 may be externally attached.

  In the above-described embodiment, the configuration in which the sign guidance control unit 4 includes the discrimination database 42 has been described. However, the discrimination database 42 may be externally attached.

  In the above-described embodiment, the structure in which the sign guide control unit 4 includes the history database has been described. However, the history database may be externally attached.

  In the above-described embodiment, an embodiment in which the sign guide device 1 is incorporated in a vehicle navigation device has been described. However, the sign guide device 1 may be configured as an independent device including a display unit, an operation unit, and the like. Good.

  In the above-described embodiment, the vehicle includes a sign extraction unit that extracts a traffic sign from video data captured by an in-vehicle camera, and a notification unit that outputs guide information of the traffic sign extracted by the sign extraction unit. The sign guide device comprising the sign guide control means for operating the notification means based on the running state detected by the running state detection means for monitoring the vehicle has been described in detail. It applies not only to the traffic signs extracted from the video data but also to those in which the above-mentioned sign guidance control means activates the notification means for the traffic signs acquired from any medium during traveling. be able to.

  For example, information on traffic signs input via a communication road-to-vehicle communication system that performs communication between a transmitter provided on the roadside and a receiver mounted on the vehicle, or by a receiver mounted on the vehicle, Alternatively, the above-described sign guidance control means may activate the notification means for information relating to a traffic sign that has received transmission data from a wireless IC tag embedded in a building or attached to a traffic sign.

  In addition, the above-mentioned embodiment is only an example of this invention, and it cannot be overemphasized that the concrete structure of each block etc. can be changed and designed suitably in the range with the effect of this invention.

Functional block diagram of a sign guide device according to the present invention Illustration of sign information database (A) shows a traffic sign with an “end” auxiliary sign inserted between two main signs, and (b) shows an “excluding bicycle” auxiliary sign inserted between two main signs. (C) shows a traffic sign in which an auxiliary sign is inserted under one main sign, (d) shows a screen on which the traffic sign and guide information are displayed, and (e ) Is an explanatory diagram showing a screen on which guide information with the distance to the stop line added is displayed (A) is a flowchart for explaining the processing operation of the sign extraction means 2 and the notification means 3, and (b) is a flowchart for explaining the processing operation of the sign guidance control means 4. (A) shows a screen on which traffic signs and guide information are displayed in a predetermined arrangement, and (b) is an explanatory diagram showing a window display informing the contents of the traffic signs (A) shows the video data of the visual field in front of the vehicle, (b) is an explanatory diagram showing the video data of the visual field in front of the vehicle after the time series of FIG. 6 (a). Functional block configuration diagram of a sign guide device provided with a region specifying means (A) shows the predetermined range at the time of touching a touch panel, (b) is explanatory drawing which showed the area display which can touch a windshield. (A) shows a functional block configuration of a sign guide device in which the sign guide control means operates the notification means in response to an input from the voice instruction means or the contact instruction means, and (b) shows a sign guide device provided with a situation determination means. Explanatory diagram showing the functional block configuration of Illustration of the violation database (A) shows the functional block configuration of the sign guide device that inputs the instruction information from the line-of-sight detection device, and (b) is an explanatory diagram showing the functional block configuration of the sign guide device provided with the sign guide condition learning means.

Explanation of symbols

1: sign guide device 2: sign extraction means 21: sign shape database 22: shape extraction means 23: sign information database 24: traffic information extraction means 25: area designation means 3: notification means 4: sign guidance control means 41: situation determination Means 5: Sign guidance condition learning means 11: In-vehicle camera 12: Traveling state detection means 13: Voice instruction means 131: Corpus database 132: Voice recognition means 14: Voice input means 15: Contact instruction means 151: Touch panel

Claims (5)

A sign guide device comprising a notification means for outputting guide information of a traffic sign,
A sign guide device comprising sign guide control means for operating the notification means based on a running state detected by a running state detecting means for monitoring a running state of a vehicle.   2. The sign guidance according to claim 1, wherein the sign guidance control means activates the notification means when it is determined or predicted that the running state detected by the running condition detection unit is contrary to traffic information indicated by the traffic sign. apparatus.   The sign guide device according to claim 1 or 2, wherein the traveling state detected by the traveling state detection means is an operating state or an operating state for any one of a vehicle speed, a gear ratio, a direction indicator, a brake, a steering, or a combination thereof. .   A sign extracting means for extracting a traffic sign from the video data captured by the in-vehicle camera, the sign extracting means; a shape extracting means for extracting a traffic sign that matches the contour shape of the traffic sign in the sign shape database; 2. Traffic information extracting means for extracting traffic information matching the graphic or character information in the sign information database from the graphic or character information drawn on the traffic sign extracted by the shape extracting means. To the sign guide device according to any one of 3.   5. The sign according to claim 4, wherein the sign extracting means includes an area specifying means for limiting a range in which a traffic sign is extracted from the video data based on instruction information input via a voice instruction means or a contact instruction means. Guide device.