JP7403173B2 - Electronic equipment and programs - Google Patents

Description

The present invention relates to an in-vehicle electronic device that recognizes road signs and provides notification based on the road signs.

There are various road signs on roads, but drivers may accidentally overlook them.

Therefore, a warning device is known that identifies a road sign based on an image taken by a camera and outputs information about the identified sign (Patent Document 1).

Japanese Patent Application Publication No. 9-185703

However, simply recognizing and identifying a road sign and outputting information about the identified road sign is not possible because there are many road signs, and the frequent notifications can be annoying, or users may become accustomed to frequent notifications. There was a problem in that the warning signs were often overlooked and the warning signs were not properly issued.

In particular, there is a problem in that if road signs such as stop signs, no entry, no passing in designated directions, and railroad crossing signs are overlooked, there is a risk of serious accidents.

In addition, in the future aging society, it will be necessary to give appropriate warnings to the elderly as well.

SUMMARY OF THE INVENTION The present invention has been made to solve this problem, and an object of the present invention is to provide an in-vehicle electronic device or the like that can alert people more appropriately than before.

(1) In order to achieve the above-mentioned object, an in-vehicle electronic device according to the present invention includes an in-vehicle camera capable of photographing road signs installed on the road while the vehicle is running, and a road sign photographed by the in-vehicle camera. An in-vehicle electronic device comprising: control means for determining the type of road sign and performing notification control on the notification means based on the determined type of road sign, the control means having an importance level based on the type of the road sign; It is determined whether or not to suppress the notification based on the type of road sign, taking into account the following.

Therefore, it is possible to solve the problem of not being able to alert people appropriately due to frequent notifications, which can be annoying, or people who have become accustomed to frequent notifications and neglect to notify important signs. It is possible to give a warning.

For example, notifications based on the types of road signs with high importance may be performed without being suppressed, and notifications other than those based on the types of road signs with high importance may be suppressed.

The notification is preferably done by sound or light, for example, by outputting a warning tone for each type of road sign, by outputting a warning text that includes the type of road sign, or by outputting a warning text that includes the type of road sign. It is preferable to light up an LED or the like in a lighting pattern provided for each type, or to display a road sign mark or its explanatory text on a liquid crystal display or the like.

Suppressing the notification may mean, for example, not making the notification, or the notification mode may be a mode that is more difficult to recognize than when the notification is not suppressed. For example, if the notification is made by sound, the volume may be lowered, or if the notification is made by light, the amount of light may be reduced. If the notification is to be made on a display, it is preferable to make the display mode less conspicuous, such as by reducing the display area or using a less conspicuous display color.

(2) A road sign that has a relatively high degree of importance based on the type of road sign, and is a road sign that may cause the vehicle to collide with another object if the road sign is not obeyed, and other than the road sign. The notification by road signs may be subject to the above-mentioned suppression.

In this way, the risk of a collision can be avoided in advance, and the frequent notifications can be annoying, or people can get used to frequent notifications and forget to notify important signs. This solves the problem of not being able to alert people appropriately.

Road signs that may cause the vehicle to collide with other objects if they are not obeyed include, for example, stop, slow down, no entry, railroad crossing, one-way street, no proceeding in the designated direction, and traffic lights. It is preferable to use at least one of them.

(3) It is preferable that the vehicle includes an information acquisition means for acquiring information regarding the vehicle, and the control means suppresses notification based on the type of the road sign by taking into account the information regarding the vehicle acquired by the information acquisition means.

In this way, the information from the notification means is suppressed by taking into account the acquired information about the vehicle, so there is a problem that the notification based on the determined type of road sign is always made, making it impossible to issue an appropriate alert. can be solved.

(4) The information acquisition means acquires the presence or absence of a signal in front of the vehicle as information regarding the vehicle, and the control means determines whether the type of the determined road sign is a road sign indicating the duty of the vehicle to stop and the control means It is preferable to adopt a configuration in which the notification is not suppressed when the information acquired from the acquisition means is that there is no signal ahead of the vehicle.

In this way, notifications regarding road signs (for example, "stop" signs) indicating the obligation to stop at intersections without particularly dangerous traffic lights can be made without being suppressed. Therefore, it is possible to issue a warning before an intersection where there is a risk of a serious accident and there are no traffic lights, but where the driver is required to stop.

In this case, the information acquisition means is configured to, for example, determine whether or not there is a signal in an image taken by the vehicle-mounted camera or a vehicle-mounted camera different from the vehicle-mounted camera to acquire the presence or absence of a signal in front of the vehicle. Alternatively, the position where the signal is located may be stored in the storage means in advance, the current position of the vehicle may be obtained, and the position where the signal stored in the storage means exists may be determined in front of the current position. The configuration may be such that the presence or absence of a signal in front of the vehicle is acquired based on whether or not a signal is present in front of the vehicle, or the presence or absence of a signal in front of the vehicle is determined based on whether or not a signal from a vehicle detection beacon located in front of the signal is received. It may also be configured to acquire the information.

(5) It is preferable that the notification is suppressed by not making the notification, and the control means does not perform the process of determining the type of the road sign for the type of road sign for which the notification is not made.
In this way, the determination process is limited to those related to the type of road sign that provides notification, so the determination time can be shortened.

(6) A current position acquisition means for acquiring the current position of the vehicle is provided, and the control means specifies the position of the notified road sign based on the current position acquired by the current position acquisition means. If the current position acquired by the current position acquisition means approaches the stored position, the type of road sign photographed by the in-vehicle camera cannot be determined. Even if there is a road sign, it is preferable to perform notification based on the type of road sign stored in association with the position.

In this way, when the next time you pass, you can be notified even at night or under conditions where it is difficult to distinguish road signs, such as in fog.

(7) Based on the configuration of (6), information regarding the traveling direction of the vehicle at the position where the notification was made is stored in association with the type of road sign notified, and the information is acquired by the current position acquisition means. Only when the current position of the vehicle approaches the stored position and the stored information regarding the traveling direction of the vehicle at the position where the notification was made matches the information regarding the current traveling direction of the vehicle. Even if the type of road sign photographed by the in-vehicle camera is not determined, it is preferable to provide notification based on the type of road sign stored in association with the position.

In this way, a road sign installed pointing toward a vehicle in the opposite direction to the direction in which the vehicle is currently traveling will not be erroneously notified. This is particularly effective when driving on a road with a small number of lanes or when the accuracy of the current position obtained by the current position acquisition means is low and it is difficult to distinguish between the driving lane and the opposite lane.

Note that it is preferable to include a direction acquisition means that obtains information regarding the traveling direction of the vehicle from, for example, a history of the current position acquired by the current position acquisition means, or from a value of a geomagnetic sensor. The information regarding the traveling direction of the vehicle may be, for example, information about the traveling direction itself, or information about the direction in which a sign is facing.

(8) The information acquisition means acquires a vehicle condition as information regarding the vehicle, and the control means determines whether or not the condition indicated by the notified road sign has been complied with after making a notification based on the type of road sign. It is preferable that the determination is made based on the vehicle condition, and notification is made based on the determination result.

In this way, the notification will be made based on the determination result, so even if the driver overlooks a road sign and does not notice the notification based on the type of road sign, for example, there is a possibility that the driver will notice it due to this notification. . Therefore, appropriate warning can be given.

It is determined based on the vehicle condition whether or not the condition indicated by the notified road sign has been complied with, and the notification based on the determination result includes, for example, compliance with the condition indicated by the notified road sign based on the vehicle condition. In this case, the notification based on the determination result may be suppressed, and if the condition indicated by the notified road sign is not observed based on the vehicle condition, the notification based on the determination result may not be suppressed. The vehicle state includes, for example, the running speed of the vehicle, whether the vehicle is accelerating/decelerating/stopping, and position information (trajectory of movement). If the road sign says "temporarily stop," find the stopping time when the speed is 0 km/h from the historical information of the driving speed (you can also find the time of staying in the same position from the history of the location information), and It is determined whether the vehicle has been stopped for a period of time or not, and if the suspension time is short or the vehicle has not stopped, the notification is not suppressed. Further, when the road sign indicates "slow down", it is possible to determine whether the traveling speed is below a certain speed and control the notification based on the determination result. Furthermore, if a road sign restricts the driving direction (route), such as prohibition of turning left or right, one-way traffic, or no entry, it is determined from the movement trajectory whether or not the road sign was followed, and based on the determination result, It is also possible to control notifications. Furthermore, in the case of a temporary stop, it is necessary to stop the vehicle so that the front end of the vehicle is located in front of the stop line. Therefore, the on-vehicle camera installed in each of the above inventions or another camera is used to image the stop line in front of the road, and check whether the front end of the vehicle has stopped before the stop line (has not jumped out in front). If the vehicle stops in front of the vehicle and remains stopped for a certain period of time, it may be determined that the vehicle has complied with the state indicated by the road sign indicating a temporary stop. The determination of whether or not the conditions indicated by these road signs have been complied with is the same in inventions such as (8) below.

(9) The information acquisition means acquires a vehicle condition as information regarding the vehicle, and the control means determines whether the condition indicated by the notified road sign has been complied with after making a notification based on the type of road sign. is preferably determined based on the vehicle condition, and at least one of the vehicle condition or the video of the vehicle-mounted camera is recorded based on the determination result.

This makes it easier to prove whether or not the sign was obeyed. For example, the driver or the driver's manager can confirm whether a sign has been overlooked by playing back the vehicle condition or video using an information processing device that plays back at least one of the recorded vehicle condition and video. .

In addition, in this way, the insurance company can determine whether or not the driver is eligible for a discount on voluntary insurance based on this record. (9) Similar effects can be obtained by doing as in (10).

(10) It is preferable that the information acquisition means acquires a vehicle state as information regarding the vehicle, and the control means records the vehicle state before and after the notification based on the type of road sign.

(11) Preferably, the control means records images of the vehicle-mounted camera before and after the notification based on the type of road sign.

(12) On the premise of any one of the configurations (9) to (11), the information acquisition means acquires a vehicle state as information regarding the vehicle, and the control means controls the information acquisition means during the recording. If a predetermined vehicle state is acquired from the vehicle, the record may be extended.

In this way, if a predetermined vehicle state is acquired, the record will be extended, making it easier to prove whether or not the sign has been obeyed. For example, it is preferable to record the 10 seconds before and after the sign is recognized and notified, and if an impact or sudden brake is detected as a predetermined vehicle state during that time, the 10 seconds thereafter are recorded. In this way, for example, it is possible to prove that the vehicle recognized a traffic sign and suddenly applied the brakes.

In this way, it is preferable to obtain, as the predetermined vehicle state, a vehicle state that indicates a change from a state in which the vehicle does not comply with the traffic sign to a state in which it complies with the traffic sign.

Conversely, after the notification is made based on the type of road sign, the control means determines based on the vehicle condition whether or not the condition indicated by the notified road sign is being observed, and if the condition is not being observed. , the record may be extended.

According to the present invention, it is possible to provide an in-vehicle electronic device, etc., which can call attention more appropriately than before.

1 is a diagram showing the configuration of a radar detector that is a preferred embodiment of the present invention. FIG. 2 is a block diagram of a radar detector. It is a figure which shows the example of a display of a standby screen, a radar scope, and a GPS warning. It is a figure which shows the example of a display of the warning screen in a radar wave warning function. It is a figure showing an example of image data of a road sign. It is a flowchart which shows the flow of processing for realizing a road sign warning function. It is a figure which shows the example of a display of the warning screen in road sign warning processing. It is a figure which shows another example of a display of the warning screen in road sign warning processing.

1 and 2 show the configuration of a radar detector which is a preferred embodiment of the in-vehicle electronic device of the present invention. This radar detector is usually mounted on the dashboard. As shown in Fig. 1, this radar detector has a solar panel 2 and a switch section 3 arranged on the top surface of a case body 1, and the front side of the case body 1 (the side disposed toward the front of the vehicle (windshield side)). It is equipped with an infrared projector 21 for irradiating infrared light toward the front of the vehicle, and an exterior camera 23 for photographing the front of the vehicle, and a micro-microwave for detecting microwaves in the frequency band emitted by the speed measuring device inside the front side. A wave receiver 4 is placed. On the other hand, a display section 5, an alarm lamp 6, an infrared communication device 7, and a remote control receiver 16 are arranged on the rear side of the case body 1 (the side disposed toward the rear of the vehicle (user side (driver side)). Furthermore, a GPS receiver 8 is arranged on the upper surface side inside the case body 1. Furthermore, an adapter jack 9 is arranged on one side of the case main body 1, and a power switch 10 and a power switch (not shown) are arranged on the other side. A DC jack (not shown) is arranged.The case body 1 also includes a speaker 20 and an acceleration sensor 25.The acceleration sensor 25 is installed so as to detect acceleration in the front-rear direction.

In this embodiment, the display unit 5 is a 2.4-inch small liquid crystal display, and the display surface is on the rear side of the case body 1 (the side disposed toward the rear of the vehicle (user side (driver side)).Display unit The height H of the rear side of the case body 1 in which the case body 1 is mounted is larger than the height H0 of other parts.

As shown in FIG. 2, the infrared communication device 7 sends and receives data to and from a communication device including a built-in infrared communication device, such as a mobile phone 12. Adapter jack 9 is a terminal to which memory card reader 13 is connected. By connecting the memory card reader 13 to the adapter jack 9, data stored in the memory card 14 attached to the memory card reader 13 can be imported into the memory card reader 13, and the contents of the memory of the database 19 and the control unit 18 can be transferred to the memory card. 14. More specifically, the data stored in the memory card 14 includes updated information such as information on new objects (location information including longitude and latitude, type information, etc.) and information on road signs for vehicles that require attention. If there is, the control unit 18 stores (downloads) the update information in the database 19 built in the device, and updates the data in the database 19. Note that the function of the memory card reader 13 may be configured to be built into the main body case 1.

The database 19 is a nonvolatile memory (eg, EEPROM) inside the microcomputer of the control unit 18 or externally attached to the microcomputer. In the database 19, information on certain targets and information on road signs for vehicles that require attention are registered at the time of shipment, and data on targets added later can be updated as described above. Can be done. Furthermore, data updating can also be performed via the infrared communication device 7.

The DC jack is for connecting a cigarette plug cord (not shown), and is connected to a cigarette socket of the vehicle via the cigarette plug cord to receive power supply. The radio receiver 15 receives incoming radio waves of a predetermined frequency. The remote control receiver 16 performs data communication with a remote control (portable device: child device) 17 using infrared rays, and performs various settings for the device. Further, the switch section 3 is also connected to the control section 18 (not shown), so that settings similar to those of the remote control 17 can be made. The remote control 17 includes a playback button, a standby switching button, a setting button, a selection button, a cancel button, a decision button, and up, down, left, and right cross buttons.

Further, the control unit 18 is a microcomputer equipped with a main CPU, sub CPU, ROM, RAM, nonvolatile memory, I/O, etc., and is connected to each of the above-mentioned parts as shown in FIG. 2, and receives input from each of the above-mentioned parts. It executes predetermined processing based on the information, controls each of the above parts, and outputs predetermined warnings, messages, and information. Note that these basic configurations can be basically the same as conventional ones. In this embodiment, processes not specifically described are performed by the main CPU.

The functions of the radar detector of this embodiment are realized by being stored in the EEPROM of the control section 18 as a program executed by the computer included in the control section 18, and executed by the computer included in the control section 18.

The functions realized by the computer by the program of the control unit 18 include a GPS log function, a video display function, a recording function, a video playback function, a standby screen display function, a radar scope display function, a GPS alarm function, a radar wave alarm function, and a wireless It has warning functions, road sign warning functions, etc.

The GPS log function is a function in which the control unit 18 stores the current position detected by the GPS receiver 8 every second in the memory card 14 in association with the detected time and speed (vehicle speed) as a position history. This position history is recorded in, for example, NMEA format.

The video display function is a function that causes the display unit 5 to display video data that the control unit 18 receives from the camera 23 outside the vehicle. The recording function includes video data received by the control unit 18 from the vehicle exterior camera 23, vehicle status data consisting of the acceleration obtained from the acceleration sensor 25, and the current position, current time, and speed (vehicle speed) obtained from the GPS receiver 8. This is a function to record the files in a "constant recording folder" provided in the memory card 14 in association with each other at any time. The recording function is always executed while power is supplied by the sub CPU included in the control unit 18. The video playback function is a function in which the control unit 18 develops and displays video data recorded on the memory card 14 on the display unit 5.

The vehicle exterior photographing camera 23 is a CCD camera capable of photographing in the visible light and infrared bands, and is internally connected to a USB port provided in the microcomputer of the control unit 18. The vehicle exterior camera 23 has a function of A/D converting an image taken by a CCD camera and sending the converted image data to the control unit 18. This function has a configuration similar to that of a so-called USB camera. The control unit 18 determines whether the illuminance necessary for reading the road sign is insufficient based on the brightness of the video data of the image taken by the CCD camera, and if the illuminance is insufficient, turns on the infrared projector 21. and radiates infrared light toward the front of the vehicle.
The standby screen display function is a function that displays the speed, latitude, longitude, and altitude of the own vehicle detected by the GPS receiver 8 on the display unit 5, as shown in FIG. 3(a).

As shown in FIG. 3(b), the radar scope display function displays targets within a predetermined range (for example, within a range of about 1 km) from the current position detected by the GPS receiver 8 at the positions stored in the database 19. This function searches information and displays the relative positional relationship between the vehicle position and the target object position on the display unit 5. In Figure 3(b), "W" on the left side indicates the west direction, "E" on the right side indicates the east direction, and "N" on the top side indicates the north direction. The icon at the intersection with the vertical line extending downward from ``'' indicates the vehicle's position. Further, icons having letters such as "L", "RD", "P", and "N" indicate the type and position of the target object.

When a press of the standby switch button provided on the remote controller 17 is detected during execution of the standby screen display function as shown in FIG. 3(a), the screen is switched to the radar scope display function as shown in FIG. 3(b). Further, if pressing of the standby switch button provided on the remote controller 17 is detected while the radar scope display function is being executed, the function is switched to the video display function. Furthermore, when a press of the standby switch button provided on the remote control 17 is detected while the video display function is being executed, processing for switching to the standby screen display function is performed.

The control unit 18 activates the GPS alarm function, the radar function, etc. according to an event that occurs while the video display function, standby screen display function, or radar scope display function (hereinafter these functions are collectively referred to as standby functions) is executed. Executes processing to realize various functions such as a wave alarm function and a wireless alarm function. Additionally, a program that implements the road sign warning function is executed once every 0.2 seconds by a timer interrupt. The priority of each function is set in the following order from highest to lowest: radar wave warning function, radio warning function, GPS warning function, and road sign warning function.

The GPS alarm function is a process that is executed at predetermined time intervals (1 second intervals) based on an event from a timer included in the control unit 18. The distance between the two is determined from the latitude and longitude of the current position, and when the determined distance reaches a predetermined approach distance (for example, within 500 m), a warning to that effect is displayed on the display unit 5 and an approach warning is issued from the speaker 20. This is the process of outputting the audio.

These targets include drowsy driving accident points, radars, speed limit switching points, enforcement areas, inspection areas, parking monitoring areas, N systems, traffic monitoring systems, intersection monitoring points, red light violation prevention systems, police stations, and accident-prone areas. Area, area where vehicles are frequently targeted, sharp/continuous curves (expressway), branching/merging points (expressway), advance ETC lane guidance (expressway), service area (expressway), parking area (expressway), highway Oasis (expressway), smart interchange (expressway), gas station within PA/SA (expressway), tunnel (expressway), highway radio reception area (expressway), prefectural border announcement, roadside station, viewpoint parking, etc. Type information of these objects, latitude and longitude information indicating their positions, schematic diagram or photograph data to be displayed on the display unit 5, and audio data are stored in the database 19 in association with each other.

The radar wave warning function is activated when the microwave receiver 4 detects a signal corresponding to a microwave in a frequency band emitted from a speed measuring device (such as a mobile radar (hereinafter simply referred to as "radar")). This is an alarm function that displays an alarm screen on the display unit 5 and outputs an alarm sound from the speaker 20. For example, when microwaves in the microwave frequency band emitted by a radar are detected by the microwave receiver 4, a schematic diagram or photograph of the radar stored in the database 19 is displayed on the display unit 5, as shown in FIG. In addition to displaying the warning screen, the sound data stored in the database 19 is read out and a sound saying "Radar. Be careful of speed" is output from the speaker 20. During audio output, the alarm lamp 6 is turned on.

The wireless alarm function is a function that issues an alarm when the wireless receiver 15 receives radio waves emitted by an emergency vehicle or the like so as not to interfere with the vehicle's movement. Radio alarm functions include enforcement radio, car location radio, digital radio, special small radio, police radio, police telephone, police activity radio, towing radio, helicopter radio, fire helicopter radio, fire radio, emergency radio, and highway radio. , scans the frequencies of security radios, etc., and when a radio signal is received at the scanned frequency, a schematic diagram indicating that a radio signal corresponding to the frequency has been received stored in the database 19 for each radio type is displayed as an alarm screen. In addition to displaying the information on the display unit 5, the audio data stored in the database 19 for each wireless type is read out, and the speaker 20 outputs an alarm sound indicating the wireless type. For example, when a police radio signal is received, a voice message such as "This is a police radio signal. Be careful of speed" is output. During audio output, the alarm lamp 6 is turned on.

In addition to these functions, the radar detector of this embodiment has a road sign warning function. The road sign warning function stores information regarding road signs to be compared in the database 19, extracts road signs from the video data captured by the external camera 23, and stores the extracted road signs and the database 19. The information about the road sign to be compared is compared with the information about the road sign to be compared stored in This is achieved by performing notification control using This warning is controlled in a manner that calls the driver's attention. To control the alarm notification, the control unit 18 reads the image data together with the display alarm character string stored in the database 19 to display an alarm screen on the display unit 5, or reads the alarm audio data stored in the database 19. This is done by outputting an alarm sound from the speaker 20 or turning on the alarm lamp 6.

The database 19 includes information on road signs such as (a) temporary stop, (b) no vehicles allowed, (c) railroad crossing, (d) closed to vehicles, (e) road closed, and (f) slow speed as shown in FIG. The type of the sign, the pattern of the sign, information on the warning content corresponding to the sign, and the importance of the sign are stored in association with each other. The pattern of the sign is stored as the image data shown in FIG. 5, and the information on the warning content corresponding to the sign is stored in association with the image data, display warning character string data, and warning audio data shown in FIG. ing. Furthermore, the importance level of the label is data indicating one of two types: "low" and "high." For example, regarding the temporary stop in Figure 5(a), the type of sign is "Pause", the image data in Figure 5(a), the character string "Stop!" as the warning character string data for display, and the warning voice data as the character string "Stop!" Warning audio data such as "Beep beep, there is a stop sign" is stored in association with "high" as the importance of the sign. Regarding the prohibition of vehicle entry in Figure 5(b), the type of sign is "Vehicles prohibited", the image data in Figure 5(b), the character string "No entry!" as the display warning character string data, and the warning audio data. The alarm audio data, ``Beep, beep, there is a sign prohibiting entry'', is stored in association with ``high'' as the importance of the sign. Regarding the presence of a level crossing in Figure 5(c), the type of sign is ``There is a level crossing'', the image data of Figure 5(c), the character string ``There is a level crossing!'' as the warning character string data for display, and ``There is a level crossing!'' as the warning audio data. Warning audio data such as "Beep beep, there is a sign indicating a railroad crossing" is stored in association with "high" as the sign's importance level. Regarding the vehicle road closure in Figure 5(d), the type of sign is "Vehicle Road Closed", the image data in Figure 5(d), the character string "Vehicle Road Closed!" as the display warning character string data, and the warning audio data as the character string "Vehicle Road Closed!" Warning audio data such as "Beep beep, there is a sign prohibiting vehicle access" is stored in association with "high" as the sign's importance level. Regarding the road closure in Figure 5(e), the sign type is "Road Closed", the image data in Figure 5(e), the character string "Road Closed" as the display warning character string data, and the warning audio data as "Road Closed Sign". The warning audio data "There is a warning" is stored in association with "low" as the importance of the sign. Regarding slow driving in Figure 5(f), the type of sign is ``slow driving'', the image data of Figure 5(f), the character string ``slow driving'' is the display warning character string data, and the warning audio data is ``slow driving''. The warning audio data "There is a warning" is stored in association with "low" as the importance of the sign. In this embodiment, (a) stop, ( b) Vehicles prohibited, (c) Railroad crossing present, and (d) Vehicles closed, are set as "high" in importance, and road signs other than these road signs in this embodiment are (e) Road closed, and (f) Slow down. The level is ``low.''

FIG. 6 is a flowchart showing the flow of processing by a program executed by the control unit 18 to realize the road sign warning function. As described above, this process is executed once every 0.2 seconds by a timer interrupt (multiple execution is not performed).

First, in step S110 (hereinafter abbreviated as S), it is checked whether there is a sign corresponding to the image data of any of the signs shown in FIG. Determination is made by pattern matching in the order of 5(a) to (f). If there is a sign in the current video data that corresponds to the image data of the sign stored in the database 19, the sign is stored in the RAM as a recognized sign and the process moves to S120; if there is no corresponding sign, the process goes to S210. Transition.

In S120, recording of video data photographed by the vehicle exterior camera 23 and vehicle status data into a "notification folder" provided in the memory card 14 is started. This recording process is performed by the sub CPU (multitasking) in parallel with recording to the "always recording folder".

In the following S130, it is determined whether the importance of the sign recognized in S110 stored in the RAM is "high" based on the importance of the sign stored in the database 19. If this degree of importance is "high" (S130: Y), the process moves to S140, and if this degree of importance is "low" (S130: N), the process moves to S190.

In S140, it is determined by pattern matching whether there is a pattern corresponding to a traffic light in the video data currently photographed by the outside camera 23. If there is no pattern corresponding to a traffic light in the video data (S140: N), the process moves to S150. On the other hand, if there is a pattern corresponding to a traffic light in the video data (S140: Y), the process moves to S170.

In S150, the current position is set as an alarm start position, and this alarm start position is stored in the nonvolatile memory of the control unit 18 in association with the type of sign recognized in S110 stored in the RAM. In the following S160, a major warning is issued, and the process moves to S240. As shown in FIG. 7(a), the major warning is displayed on the display unit 5 by acquiring the image data of the type of sign recognized in S110 and stored in the RAM from the database 19, and also displaying the image data of the type of the sign on the display unit 5. The display state in which the character string of the display alarm character string data corresponding to the display alarm character string data is displayed and the non-display state in which the image data and the character string are not displayed on the display unit 5 as shown in FIG. 7(b) are switched every 0.5 seconds. This is performed by displaying a blinking display that is repeated alternately, reading out alarm audio data corresponding to the type of sign from the speaker 20 and outputting the audio, and causing the alarm lamp 6 to blink.

On the other hand, in S170, the current position is assumed to be an alarm start position, and this alarm start position is associated with the type of sign recognized in S110 and stored in the RAM, and is stored in the nonvolatile memory of the control unit 18. In the following S180, a medium warning is issued, and the process moves to S240. For the medium warning, as shown in FIG. 7(a), the image data of the type of sign recognized in S110 stored in the RAM is acquired from the database 19 and displayed on the display unit 5, and the image data of the sign is displayed on the display unit 5. This is done by displaying a character string of display warning character string data corresponding to the type of sign, reading out warning voice data corresponding to the type of sign from the speaker 20 and outputting the sound, and lighting the warning lamp 6. In this way, the medium alarm does not flash like the large alarm, and the notification is suppressed compared to the large alarm.

On the other hand, in S190, the current position is assumed to be an alarm start position, and this alarm start position is associated with the type of sign recognized in S110 and stored in the RAM, and is stored in the nonvolatile memory of the control unit 18. In the following S200, a small alarm is issued, and the process moves to S240. As shown in FIG. 8, the small alarm is displayed on the display unit 5 by acquiring the image data of the type of sign recognized in S110 stored in the RAM from the database 19, and displaying the image data corresponding to the type of sign on the display unit 5. Displays the string of display alarm string data. No sound is output from the speaker 20, and the warning lamp 6 is not lit. In this way, the notification of a small alarm is suppressed compared to a medium alarm or a large alarm.

In S210, it is determined whether the current position corresponds to the alarm start position stored in the nonvolatile memory of the control unit 18. If the warning start position is reached (S210: Y), the process moves to S220, and if the warning start position is not reached (S210; N), the processing of this road sign warning function is ended.

In S220, recording of video data photographed by the vehicle exterior camera 23 and vehicle status data into a "notification folder" provided in the memory card 14 is started. This recording process is performed by the sub CPU (multitasking) in parallel with recording to the "always recording folder".

In the subsequent S230, the type of sign stored in the non-volatile memory in association with the alarm start position corresponding to the current position is read out, a warning for that type of sign is issued, and the type of the notified sign is stored in the RAM. , the process moves to S240. This notification is made in the form of a medium warning. In the current situation where it was recognized before but not this time, it is easy to overlook the signs, so the signs that would have been issued as a small warning if it had been recognized will be given a medium warning. This is to encourage them.

In S240, the speed history recorded in the "notification folder" provided in the memory card 14 is acquired as vehicle information.

In S250, it is determined from the acceleration and velocity history acquired in S240 whether or not the state indicated by the type of sign stored in the RAM has been observed. If it is observed (S250: Y), the process moves to S260, and if it is not observed (S250: N), the process moves to S280. In this embodiment, if the type of sign is (a) temporary stop, (b) vehicle entry prohibited, (c) level crossing, (d) vehicle closed, or (e) road closed, the speed is 5km/h during the history. If there is a place where the speed is less than h, it is determined that the vehicle has been obeyed, and if the type of the sign is (f) slow speed, it is determined that the vehicle has been obeyed if there is a place where the speed is less than 20 km/h.

In S260, the vehicle status data and video data recorded in the "notification folder" of the memory card from the start of recording in S120 to the present are copied to the "compliance folder" of the memory card.

In S270, it is determined whether the current value of the acceleration acquired from the acceleration sensor 25 is a value indicating an acceleration corresponding to the occurrence of an impact. If the value indicates an acceleration corresponding to the occurrence of an impact (S270: Y), the process moves to S260 and the vehicle status data and video data up to the present are copied again into the "compliance folder" of the memory card. On the other hand, if the value does not indicate an acceleration corresponding to the occurrence of an impact, the process moves to S290.

In S280, a sign non-compliance warning is issued. In this embodiment, the sign non-compliance warning is issued in the form of a major warning for the type of sign.

In S290, the recording of the video data and vehicle condition data started in S120 into the "notification folder" provided in the memory card 14 is finished, and the processing of this road sign warning function is ended.

As described above, in this embodiment, in S130, it is determined whether to suppress notification based on the type of road sign (S200) or not (S180, S190), taking into consideration the degree of importance based on the type of road sign. Therefore, warnings about low-importance signs are frequently issued, which can be annoying, or people can get used to frequent notifications and overlook important signs, and end up not being able to properly alert them. This problem can be solved, and more appropriate warnings can be given than in the past.

In addition, in this embodiment, in order to suppress the notification, the blinking of the alarm lamp 6 is suppressed by turning it on or off, and the display mode of the image on the display unit 5 is compared to the blinking display of the large alarm, and the blinking of the alarm lamp 6 is suppressed for the medium alarm and small alarm. In the above, the output of the sound from the speaker 20 is suppressed and simply left displayed, and the output of the sound is performed for large alarms and medium alarms, but is suppressed and not output for small alarms, but of course other suppression modes may also be used. For example, the output volume of the sound from the speaker 20 may be set to be the highest for a large alarm, smaller for a medium alarm than for a large alarm, and smaller for a small alarm than for a medium alarm. Similarly, the amount of lighting light of the alarm lamp 6 and the brightness of the display on the display unit 5 may be set to be the highest for a large alarm, smaller for a medium alarm than for a large alarm, and smaller for a small alarm than for a medium alarm. Further, the display area of the image data of the sign on the display unit 5 may be made the largest for a large alarm, smaller for a medium alarm than for a large alarm, and smaller for a small alarm than for a medium alarm.

In this embodiment, road signs that may cause a vehicle to collide with another object if they are not obeyed are (a) stop, (b) no vehicles allowed, (c) railroad crossing, and (d) Although the vehicle road is closed, it may be possible to include (e) road closure and (f) slow speed. Further, another instruction sign (mainly a blue sign) or a warning sign other than a railroad crossing (mainly a yellow sign) may be set as a recognition target, and the importance level may be lowered as other signs. In particular, signs that may be given a high level of importance include stop, slow down, no entry, railroad crossing, one-way street, no proceeding in a designated direction, traffic lights, and the like.

Furthermore, in the present embodiment, the presence or absence of a signal in front of the vehicle is acquired as information regarding the vehicle (S140), and the control unit 18 determines that the type of road sign determined is a road sign indicating the duty of the vehicle to stop (S130: If the acquired information indicates that there is no signal ahead of the vehicle (S140: N), the notification is not suppressed (S160). As a result, notifications regarding road signs indicating a duty to stop (for example, "stop" signs) at particularly dangerous intersections without traffic lights are performed without being suppressed. Therefore, it is possible to issue a warning before an intersection where there is a risk of a serious accident and there are no traffic lights, but where the driver is required to stop.

In this embodiment, the configuration is such that the presence or absence of a traffic signal in front of the vehicle is acquired by determining whether or not there is a traffic light in the image taken by the vehicle exterior camera 23. For example, if there is a traffic light in the database 19 in advance The configuration stores the position, acquires the current position of the vehicle, and acquires the presence or absence of a traffic light in front of the vehicle based on whether a stored traffic light position exists in a position corresponding to the front of the current position. Good too. Alternatively, the configuration may be such that the presence or absence of a traffic light in front of the vehicle is acquired based on whether a signal is received from a vehicle detection beacon located in front of the traffic light.

In this embodiment, in order to suppress notifications, signs other than the signs shown in FIG. There is (S110). Therefore, since the determination process is limited to those related to the type of road sign that provides notification, the determination time can be shortened.

This embodiment includes a GPS receiver 8 that acquires the current position of the vehicle, and the control unit 18 specifies the position where the alarm has been issued based on the current position acquired by the GPS receiver 8 and indicates the location of the road sign that has been notified. It is stored in association with the type (S150, S170, S190), and when the current position acquired by the GPS receiver 8 approaches this stored position (S210: Y), the camera 23 outside the vehicle takes a picture. Even if the type of road sign is not determined (S110: N), notification is performed based on the type of road sign stored in association with the position (S230). Therefore, when the vehicle next passes through a place where a road sign has been recognized and a warning has been issued, the warning can be issued even at night or under conditions where it is difficult to distinguish road signs, such as in fog.

Note that, for example, when driving on a road with a small number of lanes, such as a road with alternate traffic on one side, or because the accuracy of acquiring the current position of the GPS receiver 8 is low, it is difficult to distinguish between the current lane and the opposite lane, and S230 There is a possibility that the road sign of the opposite lane will be notified. Therefore, when storing the current position as the alarm start position in S150, S170, and S190, the current direction of travel is specified based on the position history obtained from the most recent GPS log and is stored as the alarm direction. In S210, the process moves to S220 only when the current position acquired by the GPS receiver 8 approaches this stored position and the current traveling direction matches the stored warning direction. In this case, the processing of this road sign warning function may be terminated. In this way, a road sign installed pointing toward a vehicle in the opposite direction to the direction in which the vehicle is currently traveling will not be erroneously notified. Note that the direction at the time of the alarm may be determined from the value of a geomagnetic sensor connected to the control unit 18. Further, the direction at the time of warning may be, for example, not the direction of travel but the direction in which the sign is facing.

Further, in this embodiment, vehicle state data is acquired as information regarding the vehicle, and after notification based on the type of road sign (S160, S180, S200, S230), the control unit 18 controls the state indicated by the notified road sign. It is determined based on the vehicle condition whether or not the above regulations have been complied with (S250), and notification is made based on the determination result (S280). That is, if the condition indicated by the notified road sign is complied with (S250: Y), the notification based on the determination result is not performed (by non-execution of S280), and the condition indicated by the notified road sign is complied with. If not (S250: N), the notification based on the determination result is not suppressed (sign non-compliance warning in S280). By doing this, even if the driver overlooks a road sign and does not notice the notification based on the type of road sign, there is a possibility that the driver will notice the notification based on the type of road sign. Therefore, appropriate warning can be given.

Further, in the present embodiment, after the notification based on the type of road sign (S160, S180, S200, S230), the control unit 18 determines whether the condition indicated by the notified road sign is complied with based on the vehicle condition. Based on the determination result, vehicle condition data and video data are recorded in the "compliance folder" (S260). This makes it easier to prove whether or not the sign was obeyed. For example, the driver or the driver's manager can confirm whether a sign has been overlooked by playing back the vehicle condition or video using an information processing device that plays back at least one of the recorded vehicle condition and video. . Additionally, based on this record, the insurance company can determine whether or not the driver is eligible for discounts under optional insurance. Moreover, apart from this, vehicle condition data and video data are recorded in a "notification folder" before and after the notification based on the type of road sign.

Furthermore, in the present embodiment, if an impact is detected as a predetermined vehicle state during recording (S270:Y), the control unit 18 instructs the controller 18 to extend the recording (re-copying in S260 due to S270:Y). I have to. This way, if an impact is detected, the record will be extended, making it easier to prove whether or not the sign was obeyed. In other words, in the case of this embodiment, since a record is made when an impact occurs even though the conditions indicated by the sign have been complied with, the vehicle condition in the compliance folder is used as proof that the driver is not at fault. Data or video data can be used. Note that, contrary to the present embodiment, after the notification is made based on the type of road sign, the control unit 18 determines based on the vehicle condition whether or not the condition indicated by the notified road sign is being complied with. If not, the record may be extended.

In this embodiment, the vehicle exterior camera 23 is built-in, but for example, video signals and video data may be acquired from a peripheral camera such as a front monitor camera provided in the vehicle. For example, it may be connected to a wired or wireless in-vehicle network, and images from a surrounding camera may be acquired via the in-vehicle network. In this way, there is no need to install a camera, etc. in a retrofitted device like a radar detector, and it is possible to realize an approach warning at a low cost. It can also be used for both purposes.

Furthermore, although the present embodiment has been described using an example of a radar detector, the present invention can be implemented as a function of various types of in-vehicle electronic equipment. For example, it may be incorporated as a function in a navigation device, drive recorder, or car audio. Further, the numerical values described in this embodiment may be changed to values that produce effects as appropriate through experiments and the like. The control unit 18 is also provided with a function to set the priority of each function and alarm based on instructions from the user from the remote control 17, etc., so that the control unit 18 performs processing according to the set priority. may be configured.

1 Case body 2 Solar panel 4 Microwave receiver 5 Display section 6 Lamp 7 Infrared communication device 8 GPS receiver 9 Adapter jack 10 Power switch 11 Mobile phone 12 Memory card reader 14 Memory card 15 Wireless receiver 16 Remote control receiver 17 Remote control 18 Control unit 19 Database 20 Speaker 21 Infrared projector 23 External camera 25 Acceleration sensor

Claims (3)

a function of recording an image of a camera mounted on the vehicle at a first location while the vehicle is running;
Based on a result of recognizing a road sign photographed by the camera while the vehicle is running and a state related to the drive of the vehicle, it is determined whether the recognized state indicated by the road sign is complied with. Function and
a function of recording an image of the camera at a third location different from the first location when the conditions indicated by the road sign are observed;
Electronic equipment with a function of recording an image of a camera mounted on the vehicle at a first location while the vehicle is running;
a function of recording an image captured by the camera when the road sign is recognized while the vehicle is running at a second location different from the first location;
Based on the result of recognizing the road sign based on an image taken by a camera capable of photographing the road sign, and the state related to the driving of the vehicle, whether or not the recognized state indicated by the road sign is observed. A function to determine the
a function of recording an image of the camera in a third location different from the first location and the second location when the state indicated by the road sign is observed;
Electronic equipment with A program for causing a computer to implement the functions of the electronic device according to claim 1 or 2 .

Images