JP4926182B2 - Signal recognition apparatus, signal recognition method, signal recognition program, and recording medium - Google Patents

Description

  The present invention relates to a signal recognition device, a signal recognition method, a signal recognition program, and a recording medium that recognize a display state of a traffic light. However, the use of the present invention is not limited to the signal recognition device, the signal recognition method, the signal recognition program, and the recording medium described above.

  2. Description of the Related Art Conventionally, there is an in-vehicle display device that captures the front of a vehicle with a camera and converts a portion other than red in the captured image to monochrome. According to this in-vehicle display device, when the lighting color of the traffic light in the image displayed on the display screen is red, the lighting color of the traffic light is red in an easy-to-understand manner by highlighting red. It can be easily determined that there is something (see, for example, Patent Document 1 below).

JP 2006-115376 A

  However, in the above-described prior art, when there is red lighting emitted from other than the traffic light near the traffic light, even if the lighting color of the traffic light turns blue, it is mistaken for red lighting near the traffic light, and the traffic light is lit An example of such a problem is that it is difficult to notice that the color has changed. Also, if you are not looking at the display screen display, you will not know the change in lighting color.For example, while you are looking a little away, even if the lighting color of the traffic light in front changes to blue, you may not notice As mentioned.

  Furthermore, although the lighting color of the front traffic light can be displayed in real time, only the lighting color of the front traffic light is imaged, so it is predicted that the lighting color of the front traffic light will change after a predetermined time elapses, for example. An example is a problem in which it is impossible to notify the passenger of the information and prompt the preparation for starting the vehicle.

  In order to solve the above-described problems and achieve the object, a signal recognition apparatus according to the invention of claim 1 includes signal information acquisition means for acquiring information on a display state of a traffic light, and information on the acquired display state of the traffic light. And determining means for determining the output form of the notification information to be notified to the passenger of the moving body, and notifying means for notifying the notification information according to the output form determined by the determination means, the signal information acquisition means The acquisition of the information of the display state of the traffic light for pedestrians located on the side of the moving body and the acquisition of the information of the display state of the traffic lights for the vehicle located on the side are continuously performed. Determination of the output form of the notification information according to a change in the display state of the traffic signal for the pedestrian, determination of the output form of the notification information according to a change in the display state of the traffic signal for the vehicle located on the side, Continuously The notification means notifies the notification information in the output form according to the change in the display state of the pedestrian traffic light and the notification in the output form according to the change in the display state of the vehicle traffic light located on the side. Information output is performed continuously, and the output form according to the change in the display state of the pedestrian traffic light is different from the output form according to the change in the display state of the vehicle traffic light located on the side The output form is characterized in that the output form is based on at least one of light output, operation output, and sound output.

  According to a sixth aspect of the present invention, there is provided a signal recognition method for acquiring information on a display state of a traffic signal, and a notification for informing a passenger of a moving body based on the acquired information on the display state of the traffic signal. A determination step of determining an output form of information, and a notification step of notifying notification information according to the output form determined by the determination step, wherein the signal information acquisition step is a pedestrian located on the side of the moving body Acquisition of information on the display state of the traffic signal for the vehicle and acquisition of information on the display state of the traffic signal for the vehicle located on the side are continuously performed. Continuously performing the determination of the output form of the notification information according to the determination of the output form of the notification information according to the change in the display state of the vehicle traffic signal located on the side, the notification step, Walking The notification information in the output form according to the change in the display state of the traffic signal and the notification information in the output form according to the change in the display state of the vehicle traffic signal located on the side are continuously performed. The output form according to the change in the display state of the pedestrian traffic light and the output form according to the change in the display state of the vehicle traffic light located on the side are different from each other, and the output form is It is characterized by at least one of light output, operation output, and sound output.

  According to a seventh aspect of the invention, a signal recognition program causes a computer to execute the signal recognition method according to the sixth aspect.

  A recording medium according to an eighth aspect of the invention is characterized in that the signal recognition program according to the seventh aspect is recorded in a computer-readable state.

It is a block diagram which shows the functional structure of the signal recognition apparatus concerning this Embodiment. It is a flowchart which shows the signal recognition process sequence of the signal recognition apparatus concerning this Embodiment. It is explanatory drawing which shows an example of dashboard vicinity of the vehicle by which the navigation apparatus concerning a present Example was installed. It is a block diagram which shows an example of the hardware constitutions of the navigation apparatus concerning a present Example. It is a flowchart which shows the content of the process of the navigation apparatus concerning a present Example. It is explanatory drawing shown about the confirmation order of the traffic signal in an intersection.

  Exemplary embodiments of a signal recognition apparatus, a signal recognition method, a signal recognition program, and a recording medium according to the present invention will be explained below in detail with reference to the accompanying drawings.

(Embodiment)
(Functional configuration of signal recognition device)
First, the functional configuration of the signal recognition apparatus 100 according to the embodiment of the present invention will be described. FIG. 1 is a block diagram showing a functional configuration of the signal recognition apparatus according to the present embodiment.

  In FIG. 1, the signal recognition device 100 includes a signal information acquisition unit 101, a behavior information acquisition unit 102, a prediction unit 103, a determination unit 104, and a notification unit 105.

  The signal information acquisition unit 101 acquires information on the display state of the traffic light. The display state of the traffic light is, for example, information on the color that is lit on the traffic light and information on whether or not it is blinking. Specifically, in the case of a vehicle traffic signal, the lighting color is one information of blue, yellow, or red. In the case of a pedestrian traffic signal, any one of blue lighting, blue flashing, and red lighting is selected. Information. Moreover, the information which cannot acquire the display state of a traffic signal may be sufficient. Specifically, the signal information acquisition unit 101 may determine the display state by, for example, acquiring the image information of the traffic signal, or may be a signal of a traffic signal transmitted from a not-shown management server or a communication unit included in the traffic signal. You may acquire the information regarding a display state.

  The behavior information acquisition unit 102 acquires behavior information of the driver of the moving object. The behavior information is, for example, information on the behavior that the driver performs to move the moving body. Specifically, for example, information on depression of an accelerator pedal or a brake pedal, information on operation of a side brake or a steering wheel, current position information of a vehicle, information on an output value from a G sensor, and the like.

  When the signal information acquisition unit 101 acquires the information on the display state of the traffic signal located on the side of the moving body, the prediction unit 103 calculates the prediction information on the display state of the traffic signal positioned in the traveling direction of the mobile body. The traffic light located on the side of the moving body is, for example, a traffic light other than the traffic light according to the moving body on which the passenger is on board. The prediction of the display state is a prediction of a relatively short time that can be expressed by, for example, “coming soon” until the lighting color of the traffic light in the traveling direction changes to blue.

  In addition, when the signal information acquisition unit 101 cannot acquire the display state of the traffic signal, the prediction unit 103, for example, a moving body that moves in front of a moving body, an opposing moving body, or a route that intersects a traveling route. The prediction information of the display state of the traffic signal located in the traveling direction of the moving body may be calculated from the behavior information.

  Based on the information on the display state of the traffic light acquired by the signal information acquisition unit 101, the determination unit 104 determines the output form of notification information to be notified to the passenger of the moving body. Specifically, for example, when the lighting color of the traffic signal located in the traveling direction of the moving body changes to blue, a signal for performing an operation for notifying the passenger of the moving body is output to the notification unit 105 described later. In addition, the determination unit 104 determines the output form of the notification information to be notified to the passenger of the moving body according to the prediction information of the display state of the traffic signal located in the traveling direction of the moving body calculated by the prediction unit 103. Specifically, for example, a signal for performing a foreseeing operation for the passenger of the moving body is output to the notification unit 105.

  In addition, the determination unit 104 notifies the passenger of the moving body based on the information on the display state of the traffic signal acquired by the signal information acquisition unit 101 and the behavior information of the driver acquired by the behavior information acquisition unit 102. The output form of the notification information is determined. Specifically, for example, when the driver does not show the behavior of moving the moving body even after a predetermined time has elapsed since the lighting color of the traffic light located in the traveling direction of the moving body changes to blue, Outputs a signal for prompting the passenger. In addition, as an output form of the urge action, the volume, the light amount, and the operation amount may be increased every time a predetermined time elapses, or only the action, the action and voice, the action and voice and light, etc. It may be changed.

  Furthermore, when the signal information acquisition unit 101 has not acquired the information on the display state of the traffic signal, the determination unit 104 provides notification information for notifying the passenger of the moving body that the information on the display state of the traffic signal has not been acquired. Determine the output form. Specifically, for example, a signal for performing an operation indicating the intention of notifying that the user is searching is output. This invisible operation is, for example, an operation in which the notification unit 105 is moved to the left and right in small increments or appears to be scrambled.

  The notification information may be stored in a storage unit (not shown), and in this case, the determination unit 104 determines the optimum output form of the notification information from the stored notification information. The number of notification information output forms determined may be one, or a plurality of output forms may be combined. Specifically, for example, light output, sound output, and operation output may be classified, and a combination of these may be notified to the passenger of the moving body.

  The notification unit 105 notifies the notification information determined by the determination unit 104. In addition, the notification unit 105 notifies the notification information determined by the determination unit 104 by at least one of light output, operation output, and sound output. The notification unit 105 may be a vehicle-mounted robot that imitates the shape of a person or animal. Furthermore, the vehicle-mounted robot may express a pseudo-intention or pseudo-feeling as if the vehicle-mounted robot represents the intention or emotion by a combination of light output, operation output, and sound output. The pseudo-intention and the pseudo-emotion are, for example, an intention to notify something or an emotion of joy.

(Signal recognition processing procedure of signal recognition device)
Next, a signal recognition processing procedure of the signal recognition apparatus 100 according to the present embodiment will be described. FIG. 2 is a flowchart showing a signal recognition processing procedure of the signal recognition apparatus according to the present embodiment. In the flowchart of FIG. 2, first, the signal information acquisition unit 101 acquires information on the display state of the traffic light (step S201). In step S201, as the information on the display state of the traffic light, specifically, image information around the moving body is acquired by a camera mounted on the moving body.

  Next, based on the information on the display state of the traffic light acquired in step S201, the prediction unit 103 predicts the display state of the traffic signal in the traveling direction (step S202). In step S202, when the information on the display state acquired in step S201 is only a traffic light located in the traveling direction, it may be omitted. Further, the behavior information acquisition unit 102 acquires the behavior information of the driver (step S203).

  Next, either or both of the display state information acquired in step S201, the traffic light display state information predicted in step S202, and the driver behavior information acquired in step S203. Based on the above, the output form of the notification information notified to the passenger of the moving body is determined (step S204). In addition, when the display state information acquired in step S201 is only the traffic signal located in the traveling direction, the information on the display state of the traffic signal acquired in step S201 and the behavior information of the driver acquired in step S203. Based on the above, the output form of the notification information notified to the passenger of the moving body is determined. And the alerting | reporting part alert | reports alerting | reporting information by the output form determined in step S204 (step S205), and complete | finishes a series of processes.

  In addition, the case where the display state information acquired in step S201 is only the traffic light located in the traveling direction means that the display state information of the traffic light located on the side of the mobile body cannot be acquired, or the mobile body It is possible to acquire behavior information and judge the output form of notification information at the time when the display state information of the traffic light located in the traveling direction is acquired before acquiring the display state information of the traffic light located on the side of There are cases. In these cases, since the display state of the traffic signal located in the traveling direction cannot be predicted by the prediction unit 103, the notification information is based on the display state information acquired in step S201 and the behavior information acquired in step S203. The output form is determined.

  On the other hand, when the information on the display state acquired in step S201 is only the information on the display state of the traffic light located on the side of the moving body, the information on the display state of the traffic light predicted in step S202, and step S203 Based on the behavior information acquired in step 1, the output form of the notification information is determined. Note that the case where the acquired display state information is only the display state information of the traffic light located on the side of the moving body is considered when the display state information of the traffic light located in the traveling direction could not be acquired. It is done.

  In addition, the display state information acquired in step S201 is the display state information of the traffic light located in the traveling direction, and the display state information of the traffic light located on the side of the mobile body, and the mobile body If the information on the display state of the traffic light located on the side of the traffic light is obtained first, and the information on the display status of the traffic light located in the traveling direction is obtained later, the information on the display state of the traffic light located in the traveling direction (step S201). The display form of the notification information is determined based on both the display state information acquired in step S202), the traffic signal display state information predicted in step S202, and the behavior information acquired in step S203.

  In addition, when information on the display state of any traffic light cannot be acquired in step S201, for example, it may be detected that the moving body has traveled and an operation that cannot be visually confirmed, which will be described later, may be performed.

  In the flowchart of FIG. 2, the behavior information of the driver is acquired in step S203, but the present invention is not limited to this. For example, it is not necessary to acquire driver behavior information. In this case, in step S204, the boarding of the moving object is performed based on the information on the display state of the traffic signal acquired in step S201 and / or the information on the display state of the traffic signal in the traveling direction predicted in step S202. The output form of the notification information notified to the person is determined.

  In the flowchart of FIG. 2, the notification information is notified in step S205, but the present invention is not limited to this. Specifically, for example, after a predetermined time elapses, the driver's behavior information may be further acquired, and the notification information may be continuously reported until the driver shows the behavior of the start. Moreover, you may alert | report by changing alerting | reporting information, whenever predetermined time passes.

  The signal recognition apparatus, the signal recognition method, the signal recognition program, and the computer-readable recording medium of the present invention are realized by the signal recognition apparatus 100 shown in FIG. The apparatus is not limited to 100, and a plurality of apparatuses may be used as long as the function unit illustrated in FIG. When each functional unit is connected as a different device, the connection between the devices may be established by performing communication using Bluetooth (registered trademark) or the like, regardless of wired or wireless.

  As described above, according to the signal recognition device 100 of the present embodiment, the notification that is notified to the passenger of the moving body by the determination unit 104 according to the information on the display state of the traffic light acquired by the signal information acquisition unit 101. The information output form can be determined, and the notification information can be notified according to the determined output form. Therefore, it is possible to recognize the change in the lighting color of the traffic light and make the passenger aware of it. Thus, for example, when the passenger is keeping an eye on the traffic light, the passenger can notice even if the lighting color of the traffic light changes to blue, and can perform a smooth and pleasant driving.

  In addition, for example, it may be difficult for the passenger to see the lighting color of the traffic light located in the traveling direction due to sunlight reflection, and even if the passenger confirms the lighting color of the traffic light, together with the passenger Since the signal recognition device 100 confirms the lighting color of the traffic light or predicts and notifies the change, it is possible to give the passenger a sense of security. Further, if the waiting time until the display state of the traffic signal changes (so-called signal waiting) is long, the signal recognition device 100 shares the display state of the traffic signal with the passenger, for example, feelings generally felt from there. By representing the signal recognition device 100 and including it in the notification information, an effect of reducing frustration of the passenger can be obtained.

  Further, according to the signal recognition device 100 of the present embodiment, according to the information on the display state of the traffic light acquired by the signal information acquisition unit 101 and the behavior information of the driver acquired by the behavior information acquisition unit 102. Thus, the determination unit 104 can determine the output form of the notification information notified to the passenger of the moving object. Therefore, when the driver starts the moving body promptly (for example, within a predetermined time), it is not possible to notify the passenger of the moving body that the lighting color of the traffic light has changed. Thus, the passenger can drive comfortably without unnecessary notification.

  Moreover, according to the signal recognition apparatus 100 of the present embodiment, the prediction unit 103 predicts the display state prediction information of the traffic light located in the traveling direction of the moving body from the display state information of the traffic light located on the side of the mobile body. The determination unit 104 can determine the output form of the notification information notified to the passenger of the moving body according to the prediction information of the display state of the traffic signal located in the traveling direction of the moving body. Therefore, before the lighting color of the traffic signal in the traveling direction changes to blue, it is possible to notify the passenger of the prediction that the lighting color of the traffic signal will change to blue. Thus, the passenger can prepare to start the moving body as soon as the lighting color of the traffic light changes to blue, and can drive smoothly.

  Further, according to the signal recognition apparatus 100 of the present embodiment, when the signal information acquisition unit 101 has not acquired the signal display state information, the determination unit 104 has not acquired the signal display state information. It is possible to determine the output form of the notification information that notifies the passenger of the moving body. Therefore, it is possible to notify whether or not the display state of the traffic light is recognized. As a result, the user can immediately notice when the lighting color of the traffic light changes to blue because he / she carefully watches the surroundings.

  Further, according to the signal recognition apparatus 100 of the present embodiment, the signal information acquisition unit 101 can determine the display state of the traffic signal by acquiring the image information of the traffic signal. Therefore, image information can be acquired by a camera provided in a drive recorder or a navigation device. As a result, the user can acquire the information on the display state of the traffic signal without installing a new camera, so there is no need to add extra parts and no parts cost.

  Further, according to the signal recognition device 100 of the present embodiment, the output form of the notification information determined by the determination unit 104 can be notified by at least one output form of light output, operation output, and sound output. it can. Therefore, by combining the light output, the operation output, and the sound output, it is possible to convey a pseudo intention or a pseudo emotion to the passenger. As a result, the nuances of various notification information such as foresight, awareness, and prompting can be communicated in an easy-to-understand manner. In addition, this makes it possible for the passenger to reduce the frustration caused by waiting for a signal and to enjoy driving comfortably and comfortably as if there were a passenger or a pet who can sympathize with another person waiting for a signal.

  Examples of the present invention will be described below. In the present embodiment, an example in which the signal recognition device of the present invention is implemented by a navigation device mounted on a moving body such as a vehicle (including a four-wheeled vehicle and a two-wheeled vehicle) will be described.

(Peripheral device configuration of navigation device)
First, the configuration of peripheral devices of the navigation device according to the present embodiment will be described with reference to FIG. FIG. 3 is an explanatory diagram illustrating an example of the vicinity of the dashboard of the vehicle in which the navigation device according to the present embodiment is installed.

  In FIG. 3, the navigation apparatus 300 is installed on the dashboard of the vehicle. The navigation device 300 includes a main body M and a display unit (display) D. The display unit D displays the current location of the vehicle, map information, the current time, and the like.

  The navigation device 300 is connected to a vehicle-mounted robot 310 installed on the dashboard. The vehicle-mounted robot 310 includes a camera, a lamp, a microphone, a speaker, and the like (not shown), and performs various outputs according to control of a control signal output from the navigation device 300.

  Moreover, the vehicle-mounted robot 310 has a shape imitating a human or an animal, for example, and includes a drive unit such as an arm or a drive unit such as a leg on the left and right sides. According to the control of the control signal output from 300, the operation output of the drive unit is performed. Furthermore, the vehicle-mounted robot 310 may include a drive unit that swings the top and bottom, left and right, and a drive unit that moves the body part to the left and right, according to control of a control signal output from the navigation device 300. It is good also as performing the operation output of a drive part.

  Specifically, for example, the top of the vehicle-mounted robot 310 may function as a camera, and the camera can take images inside and outside the vehicle by rotating horizontally and vertically. Further, the vehicle-mounted robot 310 may be configured to include a lamp, a microphone, and a speaker on the body portion, and may perform light output and sound collection and output using the lamp, the microphone, and the speaker. In addition, a display unit that displays images, characters, and the like may be provided, or a function of speaking words may be provided.

  Moreover, the vehicle-mounted robot 310 may be able to emit the whole body or a part thereof, and may be turned on and blinking in various colors in accordance with control of a control signal output from the navigation device 300. It is also possible to change the blinking pattern.

  With the configuration shown in FIG. 3, in this embodiment, the vehicle-mounted robot 310 performs various outputs according to the control of the control signal output from the navigation device 300, thereby indicating the driver's intention and emotion to the driver. It has become.

  The details will be described with reference to FIG. 5 and FIG. 6, but an operation program in the vehicle-mounted robot 310 for indicating the pseudo-intention and the pseudo-emotion is determined according to the information on the display state of the traffic light and the behavior information of the driver. To do. Then, the vehicle-mounted robot 310 that operates according to the determined operation program can notify the passenger of a change in the lighting color of the traffic light.

  In other words, the vehicle-mounted robot 310 is used as a driver's anxious presence, such as when a child or a pet is on board, and the driver is concerned about safe driving. By waiting for a change with the driver and expressing emotions such as willingness and pleasure to notify something according to the driver's behavior, the driver changes the lighting color when the lighting color of the traffic light changes And can reduce the frustration caused by waiting for traffic lights.

(Hardware configuration of navigation device)
Next, a hardware configuration of the navigation device 300 according to the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram illustrating an example of a hardware configuration of the navigation device according to the present embodiment.

  In FIG. 4, a navigation device 300 is mounted on a moving body such as a vehicle, and includes a CPU 401, ROM 402, RAM 403, magnetic disk drive 404, magnetic disk 405, optical disk drive 406, optical disk 407, and audio. I / F (interface) 408, microphone 409, speaker 410, input device 411, video I / F 412, display 413, communication I / F 414, GPS unit 415, various sensors 416, and camera 417 And a drive unit 418. Each component 401 to 418 is connected by a bus 420.

  First, the CPU 401 governs overall control of the navigation device 300. The ROM 402 stores programs such as a boot program, a route search program, a route guidance program, a voice generation program, a map information display program, an operation program, and a signal lighting color prediction program. The RAM 403 is used as a work area for the CPU 401.

  Here, the route search program searches for an optimal route from the departure point to the destination point using map information recorded on the optical disc 407 described later. Here, the optimum route is the shortest (or fastest) route to the destination point or the route that best matches the conditions specified by the user. The guidance route searched by executing the route search program is output to the audio I / F 408 and the video I / F 412 via the CPU 401.

  The route guidance program is based on the guidance route information searched by executing the route search program, the current location information of the navigation device 300 acquired by the communication I / F 414, and the map information read from the optical disc 407. Real-time route guidance information can be generated. The route guidance information generated by executing the route guidance program is output to the audio I / F 408 and the video I / F 412 via the CPU 401.

  The sound generation program generates tone and sound information corresponding to the pattern. That is, based on the route guidance information generated by executing the route guidance program, the virtual sound source corresponding to the guidance point is set and the voice guidance information is generated and output to the voice I / F 408 via the CPU 401.

  Also, the map information display program determines the display format of the map information displayed on the display 413 by the video I / F 412 and displays the map information on the display 413 according to the determined display format.

  Further, the operation program drives a drive unit 418, which will be described later, in accordance with the information on the display state of the traffic light and the behavior information of the driver. The details will be described with reference to FIG. 5 and FIG. 6, but the operation program causes the drive unit 418 to indicate the pseudo-intention or the emotion selected by the information on the display state of the traffic light or the behavior information of the driver. To select and run the operating program.

  Furthermore, the signal lighting color prediction program calculates the prediction information of the display state of the traffic signal in the traveling direction of the vehicle based on the information on the display state of the traffic signal on the side of the vehicle. Although details will be described with reference to FIGS. 5 and 6, the signal lighting color prediction program predicts traffic lights in the traveling direction of the vehicle after a predetermined time by analyzing an image captured by a camera 417, which will be described later, for example. Display state information.

  The magnetic disk drive 404 controls the reading / writing of the data with respect to the magnetic disk 405 according to control of CPU401. The magnetic disk 405 records data written under the control of the magnetic disk drive 404. As the magnetic disk 405, for example, an HD (hard disk) or an FD (flexible disk) can be used.

  The optical disc drive 406 controls reading / writing of data with respect to the optical disc 407 according to the control of the CPU 401. The optical disc 407 is a detachable recording medium from which data is read according to the control of the optical disc drive 406. As the optical disc 407, a writable recording medium can be used. In addition to the optical disk 407, the removable recording medium may be an MO, a memory card, or the like.

  One example of information recorded on the magnetic disk 405 and the optical disk 407 is map information used for route search / route guidance. The map information includes background data that represents features (features) such as buildings, rivers, and the ground surface, and road shape data that represents the shape of the road, and is displayed in two or three dimensions on the display screen of the display 413. Drawn. When the navigation device 300 is guiding a route, the map information and the current location of the host vehicle acquired by the GPS unit 415 described later are displayed in an overlapping manner.

  In this embodiment, the map information is recorded on the magnetic disk 405 and the optical disk 407. However, the present invention is not limited to this. The map information is not recorded only on the one provided integrally with the hardware of the navigation device 300, and may be provided outside the navigation device 300. In that case, the navigation apparatus 300 acquires map information via a network through communication I / F414, for example. The acquired map information is stored in the RAM 403 or the like.

  The audio I / F 408 is connected to a microphone 409 for audio input and a speaker 410 for audio output. The sound received by the microphone 409 is A / D converted in the sound I / F 408. In addition, sound is output from the speaker 410. Note that the voice input from the microphone 409 can be recorded on the magnetic disk 405 or the optical disk 407 as voice data.

  The microphone 409 and the speaker 410 may be installed, for example, in a drive unit 418 (for example, the in-vehicle robot 310 in FIG. 3) described later, and performs rotation driving according to the control of the operation program described above and Input / output may be performed.

  Examples of the input device 411 include a remote controller including a plurality of keys for inputting characters, numerical values, various instructions, a keyboard, a mouse, a touch panel, and the like.

  The video I / F 412 is connected to the display 413 and the camera 417. Specifically, the video I / F 412 includes, for example, a graphic controller that controls the entire display 413, a buffer memory such as a VRAM (Video RAM) that temporarily records image information that can be displayed immediately, and a graphic controller. Based on the output image data, the display 413 is configured by a control IC that controls display.

  The display 413 displays icons, cursors, menus, windows, or various data such as characters and images. As the display 413, for example, a CRT, a TFT liquid crystal display, a plasma display, or the like can be adopted. The display 413 is installed in a manner like the display unit D in FIG. 3, for example.

  The display 413 and the camera 417 may be installed, for example, in a drive unit 418 (for example, the vehicle-mounted robot 310 in FIG. 3), which will be described later. It is good also to do.

  Specifically, for example, the camera 417 captures an image inside or outside the vehicle. The image may be either a still image or a moving image. For example, the behavior of a passenger inside the vehicle may be imaged by the camera 417, or the display state of a traffic light outside the vehicle may be imaged. For example, the captured video may be output to a recording medium such as the magnetic disk 405 or the optical disk 407 via the video I / F 412, or may be used to acquire information on a display state of a traffic light and driver behavior information described later. Good.

  In addition, the communication I / F 414 is connected to a network via wireless and functions as an interface between the navigation device 300 and the CPU 401. The communication I / F 414 is further connected to a communication network such as the Internet via wireless, and also functions as an interface between the communication network and the CPU 401.

  Communication networks include LANs, WANs, public line networks and mobile phone networks. Specifically, the communication I / F 414 includes, for example, an FM tuner, a VICS (Vehicle Information and Communication System) / beacon receiver, a radio navigation device, and other navigation devices. Get road traffic information such as regulations. VICS is a registered trademark.

  Further, the GPS unit 415 uses a received wave from a GPS satellite and output values from various sensors 416 (for example, an angular velocity sensor, an acceleration sensor, a tire rotation number, etc.) to be described later, and the current position of the vehicle (navigation device 300). Information indicating the current location). The information indicating the current location is information for specifying one point on the map information such as latitude / longitude and altitude. In addition, the GPS unit 415 outputs an odometer, a speed change amount, and an azimuth change amount using output values from the various sensors 416. This makes it possible to analyze dynamics such as sudden braking and sudden steering.

  The various sensors 416 are a vehicle speed sensor, an acceleration sensor, an angular velocity sensor, and the like, and output values thereof are used for calculation of the current point by the GPS unit 415 and measurement of changes in speed and direction. The various sensors 416 also include sensors that detect each operation of the vehicle by the driver. The detection of each operation of the vehicle may be configured to detect, for example, steering wheel operation, turn signal input, side brake operation, accelerator pedal depression, brake pedal depression, or the like.

  The signal information acquisition unit 101, behavior information acquisition unit 102, prediction unit 103, determination unit 104, and notification unit 105 included in the signal recognition device 100 illustrated in FIG. 1 are the ROM 402, RAM 403, and magnetic unit in the navigation device 300 illustrated in FIG. The CPU 401 executes a predetermined program using programs and data recorded on the disc 405, the optical disc 407, etc., and realizes the function by controlling each part in the navigation device 300.

  That is, the navigation device 300 according to the embodiment executes the signal recognition program recorded in the ROM 402 as a recording medium in the navigation device 300, so that the functions of the signal recognition device 100 shown in FIG. The signal recognition processing procedure shown can be executed.

(Contents of navigation device processing)
Next, the contents of processing of the navigation device 300 according to the present embodiment will be described. FIG. 5 is a flowchart illustrating the contents of the processing of the navigation device according to the present embodiment. In the flowchart of FIG. 5, first, the vehicle waits until it stops (step S501: No loop). When the vehicle stops (step S501: Yes), it is confirmed whether there is a traffic light around the vehicle (step S502). ). In step S502, whether or not there is a traffic light is specifically confirmed by checking whether or not the traffic light is reflected in an image around the vehicle imaged by a camera mounted on the vehicle-mounted robot 310, for example.

  Next, it is determined whether or not the lighting color of the traffic signal in the traveling direction is blue (step S503). The traffic signal in the traveling direction is a traffic signal that is located on the road on which the vehicle travels and that the vehicle follows. If it is determined in step S503 that the lighting color of the traffic light is blue (step S503: Yes), the process proceeds to step S510 described later. In step S503, when the lighting color of the traffic signal in the traveling direction is not blue (step S503: No), or when the lighting color of the traffic signal in the traveling direction cannot be confirmed, the side traffic signal is monitored (step S504). . The side traffic light will be described in detail with reference to FIG. 6. For example, when the vehicle is stopped at an intersection, the traffic light is a traffic light other than the traffic light located in the traveling direction of the vehicle.

  And it waits until the side pedestrian traffic light blinks (step S505: loop of No). As will be described in detail in FIG. 6, generally, when a pedestrian traffic light and a vehicle traffic light are provided in the same direction, first, the lighting color of the pedestrian traffic light changes from blue to blue flashing, red Change in order. Then, the lighting color of the vehicle traffic light changes from blue to yellow to red. When the side pedestrian traffic light blinks in step S505 (step S505: Yes), the process waits until the side vehicle traffic light turns yellow (step S506: No loop).

  In step S506, when the lighting color of the side vehicle traffic light turns yellow (step S506: Yes), the vehicle-mounted robot 310 performs a foreseeing operation (step S507). In step S507, the foreseeing operation is an operation that indicates to the vehicle occupant that the lighting color of the traffic light in the traveling direction will soon turn blue. For example, it is an operation including the intention of foreseeing notification and the feeling of expectation for resuming traveling, and the vehicle-mounted robot 310 is preparing to start running. Specifically, for example, a driving part such as an arm provided in the vehicle-mounted robot 310 is bent, and a driving part such as a leg is slightly bent and moved up and down, imitating a posture for preparing a person to start running. . Also, audio information may be output from a speaker or the like included in the vehicle-mounted robot 310. Specifically, the audio information is, for example, information such as “It will soon be blue”.

  Next, the traffic signal in the traveling direction is monitored (step S508). If the lighting color of the traffic signal in the traveling direction cannot be confirmed at the stage of step S503, the process proceeds to step S510 after step S507. And it waits until the lighting color of the traffic light of the advancing direction turns into blue (step S509: No loop). In step S509, when the lighting color of the traffic signal in the traveling direction becomes blue (step S509: Yes), the vehicle-mounted robot 310 performs a notice operation (step S510). The notifying operation is an operation that indicates to the vehicle occupant that the lighting color of the traffic signal in the traveling direction has changed to blue. For example, it is an operation including the intention of notifying and the feeling of expectation for the further resumption of driving, for example, the operation in which the vehicle-mounted robot 310 starts to move. Specifically, a drive unit such as an arm provided in the vehicle-mounted robot 310 is moved up or down and directed straight toward the traffic light, and the drive unit such as a leg is slightly bent and moved slowly up and down. Also, audio information may be output from a speaker or the like included in the vehicle-mounted robot 310. Specifically, the audio information is information such as “It turned blue.”

  Further, the behavior information of the driver is acquired (step S511), and it is determined whether or not the vehicle has started (step S512) based on the behavior information acquired in step S511. Specifically, in step S512, the determination as to whether or not there has been a start operation is performed by, for example, whether or not the driver has released the side brake that was being pulled, whether or not the driver has released the brake pedal, The GPS unit 415 and various sensors 416 indicate whether the driver has depressed the accelerator pedal, whether the driver has moved the steering wheel, whether the vehicle has started running, whether the current position of the vehicle has changed, or the like. Judging from the information detected by

  When there is no start operation in step S512 (step S512: No), the process waits until a predetermined time elapses (step S513: No loop). In step S513, the predetermined time is generally the time required from when it is visually recognized that the lighting color of the traffic signal in the traveling direction is blue until the start operation is performed. Here, the predetermined time may be a predetermined time by calculating an average value or a maximum value excluding a singular value from a history of past behavior information for each driver.

  Further, in step 513, it may be determined whether or not the vehicle can start traveling after the lighting color of the traffic light turns blue from the image around the vehicle captured by the camera. For example, even if the lighting color of the traffic light changes to blue from the image around the vehicle, the traffic light from the traffic signal in the traveling direction is recognized as being congested (for example, the previous vehicle is not traveling) and cannot start traveling. It can be judged that it is in the situation. In this case, since it is assumed that the travel cannot be started even after a predetermined time has elapsed, the restriction of the predetermined time is removed, and when the travel is started, a joyful operation in step S515 described later is performed. Also good.

  If the predetermined time has elapsed in step S513 (step S513: Yes), the vehicle-mounted robot 310 performs a prompting operation (step S514). The urging operation is an operation that prompts the vehicle occupant to start the vehicle. For example, it is an operation in which a child is obsessed with “Let's go fast” to the parent. Specifically, a drive unit such as an arm provided in the vehicle-mounted robot 310 is raised and directed toward the driver, and the drive unit such as a foot is slightly bent and moved back and forth. Further, for example, pseudo intentions such as “Let's go fast” and pseudo emotions such as “Do n’t panic” may be output as voice information.

  On the other hand, if there is a start operation in step S512 (step S512: Yes), a joyful operation is performed by the vehicle-mounted robot 310 (step S515), and the series of processing ends. The joy operation is an operation that expresses the joy that the vehicle has started. Specifically, the drive unit such as the body part of the in-vehicle robot 310 is moved to the right so as to face the driver, the drive unit such as the top of the head is slowly moved up and down, and like the body part. Move the right drive to the left again in the original direction. Also, for example, audio information such as “Let's go” may be output. If the lighting color of the traffic light in the traveling direction is blue in step S503 (step S503: Yes), the process may proceed to step S510 as it is, and the subsequent processing may be performed.

  In the flowchart of FIG. 5, it is assumed that there is a traffic light in step S502, but the present invention is not limited to this. For example, when the traffic light cannot be confirmed, the vehicle-mounted robot 310 may perform an operation for searching. The action of searching is an action indicating the intention of notifying that the traffic light is being tried hard. Specifically, for example, a drive unit such as the top of the vehicle-mounted robot 310 is moved finely to the left and right to perform an operation such as searching for the surrounding area. For example, audio information such as “I can't see the traffic light” may be output.

  In the flowchart of FIG. 5, the side traffic light is monitored in step S504, but the present invention is not limited to this. Specifically, for example, when the side signal cannot be monitored from the host vehicle, such as when the side signal cannot be monitored by a vehicle ahead or an oncoming vehicle, the process proceeds to step S508 and the traffic signal in the traveling direction is set. You may make it monitor.

  Further, in the flowchart of FIG. 5, in step S502 and step S508, the traffic signal in the traveling direction is confirmed and monitored, but the present invention is not limited to this. Specifically, for example, when a traffic light in the traveling direction cannot be confirmed and monitored by a large vehicle such as a truck ahead, the process proceeds from step S501 to step S504, the traffic light on the side is monitored, and further from step S507 to step S507. Proceeding to S510, a notice operation may be performed.

  In the flowchart of FIG. 5, the foreseeing operation is performed in step S507, but the present invention is not limited to this. For example, first, the first foreseeing operation is performed when the sidewalk signal for the sidewalk blinks, and then the second foreseeing is performed when the lighting color of the sidewalk signal for the sidewalk turns red. The third foreseeing operation is performed when the lighting color of the side vehicle traffic light turns yellow, and the fourth lighting is performed when the lighting color of the side vehicle traffic light turns red. A foreseeing operation may be performed. In this case, the operation may be continued until the first foreseeing operation is changed to the fourth foreseeing operation, or the movement may gradually increase or become faster. Further, which of the first foreseeing operation to the fourth foreseeing operation is to be performed may be set by the passenger, and is determined from the past history of the passenger's behavior information. It may be configured.

  In the flowchart of FIG. 5, the prompting operation is performed in step S514, but the present invention is not limited to this. Specifically, for example, every time a predetermined time elapses, the operation, volume, and amount of light may be increased. Further, the color of light and the blinking pattern may be changed. Further, for example, the operation may be initially performed without outputting sound or light, and output of sound or light may be added as a predetermined time elapses.

  In the flowchart of FIG. 5, the prompting operation is performed in step S514, but the present invention is not limited to this. Specifically, for example, when it is determined that the driver's line of sight is detected and the traffic light is viewed, the prompting operation may not be performed. Furthermore, in the flowchart of FIG. 5, the lighting color and lighting state of each traffic light may be output by voice information or the like by the operation of the passenger. The passenger's operation may be input by a touch panel or voice.

(Confirmation order of traffic lights at intersections)
Next, the confirmation order of traffic lights at the intersection will be described with reference to FIG. FIG. 6 is an explanatory diagram showing the confirmation order of traffic lights at an intersection. In FIG. 6, the host vehicle 601, the vehicle 602 ahead, the vehicle traffic signal 610 in the traveling direction, the side pedestrian traffic signal 620, and the side vehicle traffic signal 630 are shown. In the vehicle traffic signal 610 in the traveling direction, the respective lighting colors are lit in the order of blue 611, yellow 612, and red 613.

  Next, the lighting color of the side vehicle traffic light 630 and the side pedestrian traffic light 620 turns blue. After elapse of a predetermined time, first, the lighting of blue 621 of the side pedestrian traffic light 620 blinks and changes to lighting of red 622. Thereafter, the lighting color of the side vehicle traffic signal 630 changes from blue 631 to yellow 632 to red 633. Next, the lighting color of the vehicular traffic signal 610 in the traveling direction changes to blue 611. Thus, at the intersection, when the lighting color of the vehicle traffic light 610 in the traveling direction is yellow 612 and red 613, the traffic light changes in the order of the side pedestrian traffic light 620 and the side vehicle traffic light 630. By confirming, next, it is possible to measure the approximate time until the lighting color of the vehicle traffic signal 610 in the traveling direction becomes blue 611.

  Therefore, as shown in step S504 to step S507 in the flowchart of FIG. 5, by acquiring the display state information of the side pedestrian traffic light 620 and the side vehicle traffic light 630, the vehicle for the traveling direction is obtained. Foreseeing that the lighting color of the traffic light 610 is blue can be made.

  As described above, according to the navigation device 300 of the present embodiment, in accordance with the information on the display state of the traffic light acquired by the signal information acquisition unit 101, the notification information notified to the passenger of the moving body by the determination unit 104. The output form can be determined, and the notification information can be notified in the determined output form. Therefore, it is possible to recognize the change in the lighting color of the traffic light and make the passenger aware of it. Thus, for example, when the passenger is keeping an eye on the traffic light, the passenger can notice even if the lighting color of the traffic light changes to blue, and can perform a smooth and pleasant driving.

  In addition, for example, it may be difficult for the passenger to see the lighting color of the traffic light located in the traveling direction due to sunlight reflection, and even if the passenger confirms the lighting color of the traffic light, together with the passenger Since the lighting color of the traffic light is confirmed or the change is predicted and notified, the passenger can be given a sense of security. In addition, if the waiting time until the display state of the traffic signal changes (so-called signal waiting) is long, the display state of the traffic signal is shared with the passenger, for example, expressing feelings that are generally felt from there to the notification information By including, the effect which reduces the frustration which a passenger has can be acquired.

  Further, according to the navigation device 300 of the present embodiment, according to the information on the display state of the traffic light acquired by the signal information acquisition unit 101 and the behavior information of the driver acquired by the behavior information acquisition unit 102, The output form of the notification information notified to the passenger of the moving body by the determination unit 104 can be determined. Therefore, when the driver starts the moving body promptly (for example, within a predetermined time), it is not possible to notify the passenger of the moving body that the lighting color of the traffic light has changed. Thus, the passenger can drive comfortably without unnecessary notification.

  Further, according to the navigation device 300 of the present embodiment, the prediction unit 103 calculates the prediction information of the display state of the traffic light located in the traveling direction of the mobile body from the information of the display state of the traffic light located on the side of the mobile body. Then, the determination unit 104 can determine the output form of the notification information notified to the passenger of the moving body according to the prediction information of the display state of the traffic light located in the traveling direction of the moving body. Therefore, before the lighting color of the traffic signal in the traveling direction changes to blue, it is possible to notify the passenger of the prediction that the lighting color of the traffic signal will change to blue. Thus, the passenger can prepare to start the moving body as soon as the lighting color of the traffic light changes to blue, and can drive smoothly. For example, even if there is a situation where a traffic light cannot be seen due to the presence of a large vehicle such as a truck in front of the vehicle, it can be prepared to start smoothly following the movement of the large vehicle.

  Further, according to the navigation device 300 of the present embodiment, when the signal information acquisition unit 101 has not acquired the information on the display state of the traffic light, the fact that the information on the display state of the traffic signal has not been acquired by the determination unit 104 has been moved. It is possible to determine the output form of the notification information notified to the passengers of the body. Therefore, it is possible to notify whether or not the display state of the traffic light is recognized. As a result, the user can immediately notice when the lighting color of the traffic light changes to blue because he / she carefully watches the surroundings.

  Moreover, according to the navigation apparatus 300 of a present Example, the signal information acquisition part 101 can judge the display state of a signal apparatus by acquiring the image information of a signal apparatus. Therefore, image information can be acquired by a camera provided in a drive recorder or a navigation device. As a result, the user can acquire the information on the display state of the traffic signal without installing a new camera, so there is no need to add extra parts and no parts cost.

  Moreover, according to the navigation apparatus 300 of a present Example, the output form of the alerting | reporting information judged by the judgment part 104 can be alert | reported by at least any one output form of a light output, an operation output, and a sound output. Therefore, by combining the light output, the operation output, and the sound output, it is possible to convey a pseudo intention or a pseudo emotion to the passenger. As a result, the nuances of various notification information such as notice and prompting can be communicated in an easy-to-understand manner. In addition, this makes it possible for the passenger to reduce the frustration caused by waiting for a signal and to enjoy driving comfortably as if there were another passenger or pet.

  As described above, according to the signal recognition device, the signal recognition method, the signal recognition program, and the recording medium of the present invention, the navigation device 300 displays the traffic signals around the vehicle imaged by the camera 417 mounted on the vehicle. According to the state information, the change of the traffic signal located in the traveling direction can be notified to the vehicle occupant by at least one of light output, operation output, and sound output by the drive unit (vehicle-mounted robot). it can. As a result, the passenger can share emotions with the drive unit, and if the lighting color of the traffic light changes, the passenger can notice the change in the lighting color in an easy-to-understand manner and wait for a signal. This reduces the frustration caused by driving and makes driving fun and comfortable.

  The signal recognition method described in this embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by being read from the recording medium by the computer. The program may be a transmission medium that can be distributed via a network such as the Internet.

DESCRIPTION OF SYMBOLS 100 Signal recognition apparatus 101 Signal information acquisition part 102 Behavior information acquisition part 103 Prediction part 104 Judgment part 105 Notification part

Claims (8)

Signal information acquisition means for acquiring information on the display state of the traffic light;
Determining means for determining an output form of notification information to notify a passenger of a moving body based on the acquired display state information of the traffic signal;
Notification means for notifying the notification information in accordance with the output form determined by the determination means;
With
The signal information acquisition means continuously performs acquisition of display state information of a pedestrian traffic light located on the side of a moving body and acquisition of display state information of a vehicle traffic light located on the side. ,
The determination means determines the output form of the notification information according to a change in the display state of the pedestrian traffic light, and outputs the notification information according to a change in the display state of the vehicle traffic signal located on the side. Judging the form continuously,
The notification means is configured to notify notification information in an output form corresponding to a change in display state of the pedestrian traffic light, and informing information in an output form corresponding to a change in display state of a vehicle traffic signal located on the side. We perform notification and continuously,
What output mode der different for doctors each other and the output mode in accordance with a change in the display state of the vehicle traffic to be positioned on the side and the output mode in accordance with a change in the display state of the pedestrian traffic, the output A signal recognition apparatus characterized in that the form is at least one of light output, operation output, and sound output . Comprising behavior information acquisition means for acquiring the behavior information of the driver of the moving object;
The signal recognition apparatus according to claim 1, wherein the determination unit further determines an output form of the notification information based on the behavior information acquired by the behavior information acquisition unit.   When the signal information acquisition means cannot acquire the information on the display state of the vehicle traffic signal located in the traveling direction of the moving body, the information on the display state of the traffic signal for the pedestrian and the vehicle traffic signal located on the side The signal recognition apparatus according to claim 1, wherein the information on the display state of the signal is acquired.   When the determination means has not acquired the information on the display state of the traffic signal for the pedestrian and the information on the display state of the traffic signal for the vehicle located on the side by the signal information acquisition means, the information on the display state of the traffic signal The signal recognition apparatus according to claim 3, wherein an output form of notification information for notifying a passenger of the moving body that the vehicle has not been acquired is determined.   The signal recognition apparatus according to claim 1, wherein the signal information acquisition unit acquires image information of the traffic light. A signal information obtaining step for obtaining information on a display state of the traffic light;
A determination step of determining an output form of notification information to notify a passenger of a moving body based on the acquired display state information of the traffic signal;
A notification step of notifying the notification information by the output form determined by the determination step;
Including
The signal information acquisition step continuously performs acquisition of information on the display state of a pedestrian traffic light located on the side of the moving body and acquisition of information on the display state of a vehicle traffic signal located on the side. ,
The determination step includes determining the output form of the notification information according to a change in the display state of the pedestrian traffic light and outputting the notification information according to a change in the display state of the vehicle traffic signal located on the side. Judging the form continuously,
The notification step includes notification of notification information in an output form corresponding to a change in display state of the pedestrian traffic light and notification information in an output form according to a change in display state of a vehicle traffic signal located on the side. We perform notification and continuously,
What output mode der different for doctors each other and the output mode in accordance with a change in the display state of the vehicle traffic to be positioned on the side and the output mode in accordance with a change in the display state of the pedestrian traffic, the output A signal recognition method characterized in that the form is at least one of light output, operation output, and sound output .   A signal recognition program for causing a computer to execute the signal recognition method according to claim 6.   A computer-readable recording medium on which the signal recognition program according to claim 7 is recorded.

Images