JP2013182521A - Network system and sensing system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a network system and a sensing system for allowing a pedestrian or other vehicle around a vehicle to easily avoid danger.SOLUTION: The network system comprises the sensing system installed in the vehicle and a terminal. The sensing system includes a communication interface, a sensor for sensing a recognition state of a driver of the vehicle, and a vehicle processor for transmitting warning information to the terminal via the communication interface on the basis of the recognition state of the driver. The terminal includes an output section, and a terminal processor for causing the output section to output visual and/or auditory warning on the basis of the warning information.

Description

  The present invention relates to a technique for avoiding a road traffic danger of a vehicle or a pedestrian, and more particularly to a technique for reducing the danger of a pedestrian or other vehicle around the vehicle.

  Conventionally, various techniques for avoiding a traffic accident between a vehicle and a pedestrian and a traffic accident between the vehicle and the vehicle have been proposed.

  For example, Japanese Unexamined Patent Application Publication No. 2009-199583 (Patent Document 1) discloses a dangerous driving prevention awareness determination system and a dangerous driving prevention awareness determination method. According to Patent Document 1, the dangerous driving prevention consciousness determination system includes a computer. The computer, for example, acquires angular velocity data from an angular velocity sensor attached to the driver's head and an angular velocity sensor attached to the right foot, acquires current position data from the position detection device, and stores data in the driving data recording device. Record. When the vehicle approaches a dangerous place such as a no-signal intersection, it is determined based on the angular velocity data whether a visual confirmation operation or a brake stance is performed as an operation for preventing dangerous driving. For example, the visual confirmation operation for the dangerous part and the determination result of the brake stance are recorded in the operation data recording device, and the alarm device issues an alarm when it is determined that the visual confirmation operation and the brake stance are not performed. .

  Japanese Unexamined Patent Application Publication No. 2011-090702 (Patent Document 2) discloses a gaze direction estimation device, a gaze direction estimation method, and a program for causing a computer to execute the gaze direction estimation method. According to Patent Document 2, in the gaze direction detection device, the relative relationship specifying unit acquires in advance a calibration image captured by a monocular camera while a human is looking at the monocular camera, and a plurality of features in the face region are obtained. A relative three-dimensional positional relationship between points is specified. The eyeball center estimation unit detects the projection positions of a plurality of feature points in the target image area captured by the monocular camera, and based on the specified relative three-dimensional positional relationship, the projection position of the human eyeball center Is estimated. The gaze direction estimation unit estimates the gaze direction based on the extracted iris center position and the estimated projection position of the eyeball center.

JP 2009-199583 A JP 2011-090702 A

  However, in the conventional technology, the effect of avoiding the danger depends on the attention of the vehicle driver. In other words, if the driver of the vehicle cannot respond appropriately, the possibility that danger can be avoided is reduced.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a network system and a detection system in which pedestrians or other vehicles around the vehicle can easily avoid danger.

  According to an aspect of the present invention, a network system including a detection system mounted on a vehicle and a terminal is provided. The detection system includes a communication interface, a sensor for detecting a recognition state of the driver of the vehicle, and a vehicle processor for transmitting warning information to the terminal via the communication interface based on the recognition state of the driver. Including. The terminal includes an output unit and a terminal processor that causes the output unit to output at least one of a visual warning, an audible warning, and a vibration warning based on the warning information.

  Preferably, the detection system further includes a first position measurement sensor for detecting the position and orientation of the vehicle. The vehicle processor transmits information indicating the position and orientation of the vehicle to the terminal via the communication interface. The terminal further includes a second position measurement sensor for detecting the position of the terminal. The terminal processor causes the output unit to output a warning when the terminal is located in the traveling direction of the vehicle based on information indicating the position and orientation of the vehicle and information indicating the position of the terminal.

  Preferably, the sensor detects the driver's line of sight as the driver's cognitive state. The vehicle processor determines whether or not to send a warning to the terminal based on the driver's line of sight.

  Preferably, the terminal can be held by a pedestrian.

  Preferably, the terminal is included in a detection system mounted on another vehicle different from the vehicle.

  Preferably, the terminal is installed directly on the road or indirectly through another structure.

  When another situation of this invention is followed, the detection system mounted in a vehicle is provided. The detection system includes a communication interface, a sensor for detecting a recognition state of the driver of the vehicle, and a vehicle processor for transmitting warning information to the terminal via the communication interface based on the recognition state of the driver. Prepare.

  When another situation of this invention is followed, the detection system mounted in a vehicle is provided. The detection system includes an output unit, a sensor for detecting the recognition state of the driver of the vehicle, and a visual warning, an audible warning, and a vibration warning based on the recognition state of the driver. A computer for outputting at least one of them.

  Preferably, the sensor detects the driver's line of sight as the driver's cognitive state. The vehicle processor causes the output unit to output information indicating the direction of the driver's line of sight visually and / or audibly as a warning based on the driver's line of sight.

  As described above, the present invention makes it easier for pedestrians or other vehicles around the vehicle to avoid danger.

FIG. 3 is an image diagram showing an operation outline of the network system 1 according to the first embodiment. It is a block diagram which shows the structure of the network system 1 containing the mobile telephone 200 which a pedestrian has. It is a graph which shows the result of having analyzed the cognitive state of each driver who is an experiment object from a viewpoint of safety confirmation. It is the result of analyzing the cognitive state of each driver who is the subject of experiment from the viewpoint of speed control. 3 is a flowchart showing processing in the network system 1 according to the first embodiment. It is an image figure which shows the state in which the display 230 of the mobile telephone 200 displays warning information. It is a block diagram which shows the structure of the network system 1 containing the detection system 100X of the other vehicle 99X. It is an image figure which shows the state in which the display 130 of the detection system 100X of another vehicle 99X displays warning information. It is an image figure which shows the operation | movement outline | summary of the detection system 100B which concerns on Embodiment 2. FIG. 2 is a block diagram showing a hardware configuration of a detection system 100B mounted on a vehicle 99. FIG. It is an image figure of the vehicle 99 carrying the detection system 100B. It is an image figure which shows the external display 180. FIG. It is an image figure which shows the external speaker 190. FIG. It is an image figure which shows the vehicle 99 carrying the deformation | transformation type of the external display 180B. 10 is a flowchart showing processing in a detection system 100B according to Embodiment 2. It is a graph which shows transition of the death toll according to the state of a traffic accident party. It is an image figure which shows the assumption accident situation and actual experiment environment. It is an image figure which shows the setting of an experiment environment. It is an image figure which shows the flow of one enforcement. It is a graph which shows the oversight rate of the visual stimulus for every condition. It is a graph which shows the error reaction rate for every conditions. It is a graph which shows the average reaction time for every conditions.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[Embodiment 1]
<Overview of operation>
First, an outline of the operation of the network system 1 according to the present embodiment will be described. FIG. 1 is an image diagram showing an operation outline of the network system 1 according to the present embodiment.

  Referring to FIG. 1A, a vehicle 99 is traveling on the road. At this time, the pedestrian is about to cross the pedestrian crossing at the intersection in front of the vehicle 99. In the present embodiment, detection system 100 of vehicle 99 periodically detects the recognition state of the driver of the vehicle. That is, the detection system 100 periodically determines whether or not the recognition state is abnormal.

  In addition, that there is no abnormality in a cognitive state means having acquired the stimulus required for safe driving | running from the outside moderately. On the other hand, that the cognitive state is abnormal means that the stimulus necessary for safe driving has not been acquired appropriately from the outside.

  Referring to FIG. 1B, the vehicle 99 continues to travel on the road. At this time, a pedestrian is starting to cross the pedestrian crossing at the intersection in front of the vehicle 99. Another vehicle 99X is about to cross the intersection. The detection system 100 of the vehicle 99 detects the recognition state of the driver of the vehicle. The detection system 100 detects that there is an abnormality in the cognitive state. For example, the detection system 100 detects that the driver is looking aside or is asleep.

  Referring to FIG. 1C, detection system 100 transmits warning information indicating that there is an abnormality in the recognition state of the driver of vehicle 99 to the outside. When the mobile phone 200 held by a pedestrian around the vehicle 99 and the detection system 100X of the other vehicle 99X around the vehicle 99 receive warning information from the vehicle 99, it outputs a warning display or a warning sound.

  Specifically, the mobile phone 200 and the detection system 100X may output a warning display or a warning sound when the mobile phone 200 and the detection system 100X are located in front of (or the traveling direction of) the vehicle 99 (or the detection system 100). Alternatively, when the mobile phone 200 and the detection system 100X are located within a predetermined distance (for example, 50 m) from the vehicle 99 or the detection system 100, the mobile phone 200 and the detection system 100X may output a warning display or a warning sound.

  Thus, the network system 1 according to the present embodiment increases the possibility of avoiding traffic hazards by alerting the pedestrian around the vehicle 99 or the driver of another vehicle 99X. Can do. In other words, a pedestrian around the traveling vehicle 99 or a driver of the vehicle 99X can take an action that easily avoids danger.

  Hereinafter, the configuration of the network system 1 for realizing such a function will be described in detail.

<Overall Configuration of Network System 1 Including Pedestrian Mobile Phone 200>
Next, the overall configuration of the network system 1 when the detection system 100 mounted on the vehicle 99 communicates with the mobile phone 200 of a pedestrian will be described. FIG. 2 is a block diagram illustrating a configuration of the network system 1 including the mobile phone 200 that a pedestrian has.

  The detection system 100 is mounted on a vehicle. For example, the detection system 100 may be a part of a car navigation system or may be a system different from the car navigation system.

  The detection system 100 includes a CPU 110, a memory 120, a display 130, a GPS (Global Positioning System) 140, a button 150, a communication interface 160, a vehicle exterior sensor 171, a line-of-sight sensor 172, and a steering sensor 173. .

  The memory 120 is realized by various types of RAM (Random Access Memory), ROM (Read-Only Memory), and / or a hard disk. The memory 120 stores a control program executed by the CPU 110 and other data.

  The display 130 is installed in the vehicle 99. The display 130 outputs characters and images based on commands from the CPU 110. Note that the detection system 100 may include a touch panel instead of the display 130.

  The GPS 140 acquires position information indicating where on the earth it is using radio waves from an artificial satellite. The GPS 140 inputs position information to the CPU 110. The GPS 140 according to the present embodiment also acquires the direction of the vehicle 99 or the detection system 100 using a compass.

  The button 150 is installed in the vehicle 99. The button 150 receives a command from the user and inputs the command from the user to the CPU 110. For example, the button 150 receives an ON / OFF command for the detection system 100 and a command for setting a judgment criterion for the recognition state.

  The communication interface 160 performs data communication with a device external to the detection system 100. For example, the communication interface 160 transmits warning information to the mobile phone 200 and the detection system 100X of the other vehicle 99X based on a command from the CPU 110.

  The vehicle exterior sensor 171 captures the outside of the vehicle 99 and inputs the captured data to the CPU 110. Thus, the CPU 110 can grasp the situation outside the vehicle 99. For example, the CPU 110 calculates the positions of the walking car and the other vehicles 99 located around the vehicle 99 and the distance from itself based on the photographing data.

  The line-of-sight sensor 172 captures the driver's face and inputs captured data to the CPU 110. Thereby, the CPU 110 can grasp the direction of the line of sight of the driver of the vehicle 99.

  The steering sensor 173 detects driver steering information for the vehicle 99 and inputs the steering information to the CPU 110.

  The CPU 110 determines the driver's recognition state based on data from the vehicle outside sensor 171, the line-of-sight sensor 172, and the steering sensor 173. If there is an abnormality in the driver's recognition state, CPU 110 transmits warning information to another terminal via communication interface 160. CPU 110 according to the present embodiment transmits position information indicating the position and orientation of vehicle 99 or detection system 100 together with the warning information to other terminals via communication interface 160.

  Below, the specific example for determining a recognition state is demonstrated. Detection system 100 according to the present embodiment enables continuous gaze measurement by having gaze sensor 172 share observations of facial feature points and eye regions even when there is a large change in face posture.

  In the present embodiment, the memory 120 stores a cognitive state model as information for classifying the driver's cognitive state. Therefore, in this implementation, for example, the driving behavior data for the more than 800 people accumulated for the recognition state detection is analyzed in advance, and the initial data regarding the line of sight / steering obtained in the constructed experimental environment is analyzed, thereby realizing the recognition. A state model was created.

  The driving behavior database is composed of driver safety confirmation behavior data based on changes in face orientation that is an approximation of the line of sight direction, accelerator / brake operation behavior data, vehicle speed data, and vehicle travel locus (steering) data using GPS. In the analysis, based on the knowledge about driving safety of the driving school instructor, the cognitive state of how much attention the driver is careful about and driving safely from the viewpoint of driver safety confirmation and vehicle speed control Was evaluated and scored quantitatively, and the tendency was classified.

  FIG. 3 is a graph showing a result of analyzing the cognitive state of each driver as an experiment target from the viewpoint of safety confirmation. Each point in FIG. 3 is a plot of the recognition state score related to each driver's confirmation, and the horizontal axis and the vertical axis represent the knowledge state score related to the left / right safety confirmation, respectively.

  As shown in Fig. 3, the driver's cognitive status regarding safety confirmation is (1) the cognitive status is always good, (2) the cognitive status may get worse, (3) the cognitive status tends to be biased to the right (left side) (4) Cognitive status tends to be biased to the left side, and (5) Cognitive status is always poor. CPU110 which concerns on this Embodiment judges whether a driver | operator's recognition state corresponds to either of said (2)-(5). Based on the data from the line-of-sight sensor 172, the CPU 110 transmits warning information to the outside via the communication interface 160 when the recognition state corresponds to any of the above (2) to (5).

  FIG. 4 shows the result of analyzing the cognitive state of each driver as an experiment subject from the viewpoint of speed control. Referring to FIG. 4, the driver's cognitive state regarding speed control is (1) the speed cognitive state is always good, (2) the speed cognitive state may be bad, and (3) the speed cognitive state is always bad. It is suggested that there are three types. Based on the data from the line-of-sight sensor 172 and the steering sensor 173, the CPU 110 transmits warning information to the outside via the communication interface 160 when the recognition state corresponds to the above (2) or (3).

  Returning to FIG. 2, the mobile phone 200 includes a CPU 210, a memory 220, a display 230, a GPS 240, a button 250, a communication interface 260, and a speaker 270.

  The memory 220 is realized by various RAMs, ROMs, and / or hard disks. The memory 220 stores a control program executed by the CPU 210 and other data.

  The display 230 outputs characters and images based on signals from the CPU 210. For example, the display 230 displays warning information based on a signal from the CPU 210. This makes it easier for the user of the mobile phone 200 to recognize that a vehicle driven by a driver with a poor recognition state exists in the vicinity. Note that the mobile phone 200 may include a touch panel instead of the display 230.

  The GPS 240 acquires position information indicating where on the earth it is using radio waves from an artificial satellite. The GPS 240 inputs position information to the CPU 210. The GPS 240 according to the present embodiment also acquires the orientation of the mobile phone 200 using a compass.

  The button 250 receives a command from the user and inputs the command from the user to the CPU 210. For example, the button 250 receives a cellular phone ON / OFF command, a phone number / character input command, a reference setting command for determining whether or not to output a warning, and the like.

  Communication interface 260 performs data communication with an external device of mobile phone 200. For example, the communication interface 260 receives warning information and position information from the detection system 100 of the vehicle 99.

  The speaker 270 emits a warning sound based on the signal from the CPU 210. This makes it easier for the user of the mobile phone 200 to recognize that a vehicle driven by a driver with a poor recognition state exists in the vicinity.

  The CPU 210 displays a warning on the display 230 based on the warning information from the detection system 100. CPU 210 causes speaker 270 to output a warning sound based on the warning information from detection system 100.

  CPU 210 may output warning information when mobile phone 200 is located in front of vehicle 99 based on the position and orientation of vehicle 99 and the position and orientation of mobile phone 200. Here, the front of the vehicle 99 is substantially in front of the vehicle 99 and means a range of about ± 30 degrees on the left and right with respect to the front of the vehicle 99. Alternatively, CPU 210 may output warning information when mobile phone 200 is located within a predetermined distance from vehicle 99 based on the position and orientation of vehicle 99 and the position and orientation of mobile phone 200.

<Processing of Network System 1 Including Pedestrian Mobile Phone 200>
Next, processing in the network system 1 when the detection system 100 mounted on the vehicle 99 communicates with the mobile phone 200 of a pedestrian will be described. FIG. 5 is a flowchart showing processing in the network system 1 according to the present embodiment.

  Referring to FIG. 5, the CPU 110 of the detection system 100 periodically detects the driver's cognitive state based on data from the outside sensor 171, the line-of-sight sensor 172, and the steering sensor 173 while the vehicle 99 is traveling. (Step S102). CPU110 judges whether a recognition state is good or bad (step S104). Specifically, CPU 110 determines whether or not the detected recognition state satisfies a predetermined condition.

  CPU110 repeats the process from step S102, when a recognition state is good, ie, a recognition state satisfy | fills a predetermined condition (when it is YES in step S104). When the cognitive state is bad, that is, when the cognitive state does not satisfy the predetermined condition (NO in step S104), CPU 110 uses GPS 140 to determine the position and orientation of vehicle 99 (or detection system 100). Obtain (Step S106)

  CPU 110 transmits, via communication interface 160, warning information indicating that the driver's recognition state is poor and position information indicating the position and orientation of vehicle 99 (or detection system 100) to the outside (step S108). ). CPU110 repeats the process from step S102.

  On the other hand, CPU 210 of mobile phone 200 waits for warning information from detection system 100 of vehicle 99 via communication interface 260 (step S152). When CPU 210 receives the warning information (YES in step S152), CPU 210 acquires the position and orientation of mobile phone 200 using GPS 240 (step S154).

  CPU 210 determines whether or not the positional relationship between mobile phone 200 and vehicle 99 (or detection system 100) is a predetermined relationship (step S156). For example, CPU 210 determines whether mobile phone 200 is located in front of vehicle 99 or determines whether mobile phone 200 is within a predetermined distance from vehicle 99.

  CPU210 repeats the process from step S152, when the positional relationship of the mobile telephone 200 and the vehicle 99 is not a predetermined relationship (when it is NO in step S156). CPU 210 causes display 230 to display a warning message when the positional relationship between mobile phone 200 and vehicle 99 is a predetermined relationship (YES in step S156).

  FIG. 6 is an image diagram showing a state in which the display 230 of the mobile phone 200 displays warning information. Based on the position and orientation of the vehicle 99 and the position and orientation of the mobile phone 200, the CPU 210 determines from which direction the vehicle 99 comes toward itself when the mobile phone 200 is located in front of the vehicle 99. Is displayed on the display 230.

  In step S158, CPU 210 may cause speaker 270 to output a warning message when the positional relationship between mobile phone 200 and vehicle 99 is a predetermined relationship (YES in step S156). CPU210 repeats the process from step S152.

<Overall Configuration of Network System 1 Including Other Vehicle 99X Detection System 100X>
Next, the configuration of the network system 1 when the detection system 100 mounted on the vehicle 99 and the detection system 100X of another vehicle 99X communicate with each other will be described. FIG. 7 is a block diagram illustrating a configuration of the network system 1 including the detection system 100X of another vehicle 99X. Note that the configuration of the detection system 100 of the vehicle 99 and the configuration of the detection system 100X of the other vehicle 99 are the same as those described with reference to FIG. 2, and therefore description thereof will not be repeated here.

<Processing of Network System 1 Including Detection System 100X of Other Vehicle 99X>
Next, processing in the network system 1 when the detection system 100 mounted on the vehicle 99 communicates with the detection system 100X of another vehicle 99X will be described. Note that the processing of the detection system 100 of the vehicle 99 is the same as the processing of the detection system 100 described above, and therefore description thereof will not be repeated here.

  Referring to FIG. 5 again, CPU 110 of detection system 100X of another vehicle 99X waits for warning information from detection system 100 of vehicle 99 via communication interface 160 (step S152). CPU110 will acquire the position and direction of other vehicles 99X or detection system 100X using GPS140, if warning information is received (when it is YES at Step S152) (Step S154).

  CPU 110 determines whether or not the positional relationship between other vehicle 99X (or detection system 100X) and vehicle 99 (or detection system 100) is a predetermined relationship (step S156). For example, the CPU 110 determines whether or not another vehicle 99X is located in front of the vehicle 99, or determines whether or not the other vehicle 99X is within a predetermined distance from the vehicle 99.

  CPU110 repeats the process from step S152, when the positional relationship of the other vehicle 99X and the vehicle 99 is not a predetermined relationship (when it is NO in step S156). CPU 110 causes display 130 to display a warning message when the positional relationship between other vehicle 99X and vehicle 99 is a predetermined relationship (YES in step S156).

  FIG. 8 is an image diagram showing a state in which the display 130 of the detection system 100X of another vehicle 99X displays warning information. When the other vehicle 99X is positioned in front of the vehicle 99 based on the position and orientation of the vehicle 99 and the position and orientation of the other vehicle 99X, the CPU 110 is directed toward the vehicle 99 from which direction. A message indicating whether or not to come is displayed on the display 130.

  In step S158, CPU 110 may cause speaker 170 to output a warning message when the positional relationship between other vehicle 99X and vehicle 99 is a predetermined relationship (YES in step S156). CPU110 repeats the process from step S152.

  In the present embodiment, the mobile phone 200 and the other detection system 100X output warning information. However, the network system 1 may have a terminal installed on a road such as an intersection or a junction. That is, the terminal installed on the road may have the same configuration as that of the mobile phone 200 or other detection system 100X. In this case, the detection system 100 transmits warning information to a terminal on the road ahead of the vehicle 99, and the terminal on the road displays a warning or emits a warning sound.

  In the present embodiment, detection system 100 determines whether or not there is an abnormality in the driver's recognition state. However, the detection system 100 may transmit the driver's recognition state to the terminal (the mobile phone 200 or another detection system 100X) and determine whether the terminal has an abnormality in the recognition state. In this case, it is preferable that the determination criterion can be changed for each terminal. For example, it is preferable that any of (1) to (5) in FIG. 3 and (1) to (3) in FIG.

[Embodiment 2]
Next, a second embodiment of the present invention will be described. The network system 1 according to the first embodiment described above transmits warning information to other terminals when the detection system 100 of the vehicle 99 detects an abnormality in the driver's cognitive state. However, in the detection system 100B according to the present embodiment, the detection system 100B itself outputs a warning display or a warning sound according to the driver's recognition state.

<Overview of operation>
First, the operation | movement outline | summary of the detection system 100B which concerns on this Embodiment is demonstrated. FIG. 9 is an image diagram showing an outline of operation of detection system 100B according to the present embodiment.

  Referring to FIG. 9A, vehicle 99 is traveling on the road. At this time, the pedestrian is about to cross the pedestrian crossing at the intersection in front of the vehicle 99. In the present embodiment, detection system 100B of vehicle 99 periodically detects the recognition state of the driver of the vehicle. That is, the detection system 100B periodically determines whether or not the recognition state is abnormal.

  Referring to FIG. 9B, vehicle 99 is traveling on the road. At this time, a pedestrian is starting to cross the pedestrian crossing at the intersection in front of the vehicle 99. Another vehicle 99X is about to cross the intersection. The detection system 100B of the vehicle 99 detects the recognition state of the driver of the vehicle. The detection system 100B detects that there is an abnormality in the cognitive state. For example, the detection system 100B detects that the driver is looking aside or is asleep.

  Referring to FIG. 9C, detection system 100B according to the present embodiment outputs warning information to the outside in accordance with the recognition state of the driver of vehicle 99. More specifically, the detection system 100B determines whether there is a pedestrian or a vehicle in front of the vehicle. The detection system 100B displays warning information or emits a warning sound in the direction of the walking vehicle or vehicle ahead. In addition, the detection system 100B displays warning information or emits a warning sound toward a walking car or vehicle existing within a predetermined distance from the vehicle 99.

  As described above, the detection system 100B according to the present embodiment increases the possibility of avoiding traffic dangers by alerting the driver of a pedestrian around the vehicle 99 or another vehicle 99X. Can do. In other words, a pedestrian around the traveling vehicle 99 or a driver of the vehicle 99X can take an action that easily avoids danger.

  Hereinafter, the configuration of the detection system 100B for realizing such a function will be described in detail.

<Configuration of Detection System 100B>
Next, the configuration of the detection system 100B mounted on the vehicle 99 will be described. FIG. 10 is a block diagram showing a hardware configuration of a detection system 100B mounted on the vehicle 99. As shown in FIG. FIG. 11 is an image diagram of a vehicle 99 equipped with the detection system 100B. FIG. 12 is an image diagram showing the external display 180. FIG. 13 is an image diagram showing the external speaker 190.

  Referring to FIG. 10, detection system 100B is mounted on a vehicle. For example, the detection system 100B may be a part of a car navigation system or may be a system different from the car navigation system.

  The detection system 100B includes a CPU 110, a memory 120, a display 130, a GPS 140, a button 150, a communication interface 160, a vehicle exterior sensor 171, a line-of-sight sensor 172, a steering sensor 173, an external display 180, and an external speaker. 190 is included. The memory 120, the display 130, the GPS 140, the button 150, the communication interface 160, the vehicle exterior sensor 171, the line-of-sight sensor 172, and the steering sensor 173 are the same as those according to the first embodiment. Therefore, the description will not be repeated here (see FIGS. 2 and 7). The determination of the cognitive state is also the same as those according to the first embodiment, and therefore description thereof will not be repeated here (see FIGS. 3 and 4).

  Referring to FIGS. 10, 11, and 12, external display 180 is mounted on the ceiling of vehicle 99, for example. The external display 180 can give a visual stimulus to a person (for example, a pedestrian or driver located in front of the vehicle 99) located in a specific direction (normal direction). External display 180 according to the present embodiment includes an LED (Light Emitting Diode) array 182 and a lenticular plate stacked on LED array 181.

  Referring to FIGS. 10, 11, and 13, external speaker 190 is mounted on the ceiling of vehicle 99, for example. The external speaker 190 can give an auditory stimulus to a person (for example, a pedestrian or driver located in front of the vehicle 99) located in a specific direction (normal direction). External speaker 190 according to the present embodiment includes an ultrasonic transducer array 191.

  The CPU 110 determines the driver's recognition state based on data from the vehicle outside sensor 171, the line-of-sight sensor 172, and the steering sensor 173. If there is an abnormality in the driver's recognition state, the CPU 110 displays warning information on the external display 180. If the driver's cognitive state is abnormal, CPU 110 causes external speaker 190 to emit a warning sound.

  Alternatively, the CPU 110 detects the direction of the driver's line of sight based on the data from the line-of-sight sensor 172. CPU 110 causes external display 180 to display the direction of the driver's line of sight. CPU 110 causes external speaker 190 to output a sound indicating the direction of the driver's line of sight.

  More specifically, in the present embodiment, CPU 110 detects the positions of pedestrians and other vehicles based on data from vehicle exterior sensor 171. CPU 110 determines whether the distance from vehicle 99 or detection system 100B to the pedestrian and each of the other vehicles is within a predetermined distance. CPU 110 causes external display 180 to display warning information toward the direction of pedestrians and other vehicles located within a predetermined distance from vehicle 99. CPU 110 causes external speaker 190 to emit a warning sound toward pedestrians and other vehicles located within a predetermined distance from vehicle 99.

  Here, a modified example of the external display 180 will be described. FIG. 14 is an image diagram showing a vehicle 99 equipped with a modified type of the external display 180B.

  Referring to FIG. 14, external display 180 may be installed on a part or all of the outer surface of a vehicle body such as a bonnet or a windshield. The warning information to be displayed may be an arrow indicating the direction of the driver's line of sight or other animation, or may be text or an image indicating the degree of goodness and badness of the recognition state.

<Processing of detection system 100B>
Next, processing in the detection system 100B mounted on the vehicle 99 will be described. FIG. 15 is a flowchart showing processing in the detection system 100B according to the present embodiment.

  Referring to FIG. 15, CPU 110 of detection system 100 </ b> B detects the driver's cognitive state based on data from outside sensor 171, gaze sensor 172 and steering sensor 173 periodically while vehicle 99 is traveling. (Step S202). CPU110 judges whether a recognition state is good or bad (step S204). Specifically, CPU 110 determines whether or not the detected recognition state satisfies a predetermined condition.

  CPU110 repeats the process from step S102, when a cognitive state is good, ie, when a cognitive state satisfy | fills a predetermined condition (when it is NO in step S204). CPU110 searches for a pedestrian or another vehicle using outside sensor 171, when a cognitive state is bad, ie, a cognitive state does not satisfy a predetermined condition (when it is YES in Step S204) (Step S206). .

  CPU 110 determines whether there is a pedestrian or another vehicle (in front of vehicle 99) and within a predetermined distance from vehicle 99 (step S208). CPU110 repeats the process from step S202, when a pedestrian or another vehicle does not exist within the predetermined distance from vehicle 99 (in front of vehicle 99) (in the case of NO in step S208).

  CPU 110 calculates the direction in which the warning information should be output when there is a pedestrian or another vehicle (in front of vehicle 99) and within a predetermined distance from vehicle 99 (YES in step S208) ( Step S210). The CPU 110 displays warning information on the external display 180 in the output direction (step S212). CPU 110 causes external speaker 190 to emit a warning sound in the output direction. CPU110 repeats the process from step S202.

[Attention effect]
Below, the alerting effect with respect to the pedestrian around the vehicle 99 and other vehicles by the network system 1 and the detection system 100B is demonstrated.

  Many accidents involving pedestrians and cars are caused by carelessness of both drivers and pedestrians and lack of awareness of danger. Therefore, we have proposed a new system that avoids accidents by sharing information such as each other's presence and cognitive status among traffic participants. With this system, it is considered that traffic participants can recognize each other's dangers and avoid danger. In this study, as part of this system, we examined the visibility evaluation of visual information from automobiles to pedestrians using a high-brightness display mounted on the outside of the vehicle, and examined the alerting effect of visual stimuli. As a result, sufficient visibility was obtained at a viewing distance of 50 m. Regarding the alerting of pedestrians by visual stimuli, although the alerting effect depends on the pedestrian's gaze direction, a certain effect was confirmed.

<1. Introduction>
<1.1. Current situation of pedestrian accidents>
FIG. 16 is a graph showing changes in the number of deaths by state of persons involved in a traffic accident. Referring to FIG. 16, the number of deaths due to traffic accidents has been decreasing in recent years. On the other hand, however, the number of pedestrian deaths has increased relative to the total number of traffic fatalities. Since the fatality rate of pedestrian traffic accidents is more than five times higher than the overall fatality rate of traffic accidents, in order to prevent pedestrian fatal accidents, it is necessary to improve the traffic environment in which pedestrians do not encounter accidents. Most important.

  According to statistical data, nearly 90% of the first parties to pedestrian fatalities are cars. Looking at the causes of accidents when the car is the first party and the pedestrian is the second party, the order of the number of fatal accidents is “Wakimi”, “Safety unconfirmed”, “Thinking and random driving”. In other words, it can be understood that many of the causes of accidents are not due to errors in driving operations or judgments, but due to casual driving or carelessness during driving.

  On the other hand, regarding factors on the pedestrian side, which is the second party, the order is “no factor”, “safety unconfirmed”, and “judgment error”. In the case of pedestrians, accidents are caused by driver factors, and lack of danger awareness is a cause of death. By reducing the driver / pedestrian accident factors in such actual accident cases, the risk of accidents can be avoided. In other words, if the driver and pedestrian can appropriately recognize each other's existence and the possibility of an accident, it is considered that the accident can be prevented in advance.

  Therefore, in addition to the conventional driver assistance type safe driving support system, we have proposed a new traffic accident reduction system by sharing the cognitive state.

<1.2. Reduction of traffic accidents by sharing cognitive status>
The "cognitive state sharing system" we propose is a system that aims to avoid traffic accidents by sharing the driver's state with other traffic participants. Until now, various safety devices have been researched and developed in order to improve the safety of automobiles and prevent accidents. Many of these are aimed at driver assistance by automating driving operations and providing information to the driver himself. While driving automation and driver assistance are very effective approaches to improving road safety, their practicality and applicability are still under development. For example, as an example of a driver assistance system, there is a system that reduces accidents by detecting dangerous behaviors of a driver that may lead to accidents such as snoozing or carelessness and transmitting them to the driver. In such a system, since the driver needs to appropriately use information obtained from the system, the effect depends on the ability and cognitive state of the driver. For this reason, the effect with respect to the driver | operator who cannot perform appropriate action by fatigue, aging, etc. cannot be expected so much.

  In the cognitive state sharing system, even if the driver is unable to drive properly, the risk prediction of traffic participants can be made by sharing information that leads to detected accidents with not only the driver but also surrounding traffic participants. It aims to promote risk avoidance. Thereby, it is considered that cooperative actions among traffic participants can be promoted, and accidents that cannot be prevented by conventional driver assistance type safe driving assistance can be prevented.

  This study was conducted as part of the development of this “cognitive state sharing system” with the aim of developing a method to present surrounding traffic participants with a “cognitive state leading to danger” such as a driver's sleep or carelessness. It was. We are currently investigating audio-visual and wireless vehicle-to-vehicle and pedestrian-to-vehicle communication as methods for presenting the driver's cognitive state. The purpose of this study was to examine the effectiveness of visual presentation of information from automobiles to pedestrians and alerting among them. First, as a preliminary survey, a subjective assessment was performed on the visibility and alerting effect of visual information presented on an outdoor high-brightness LCD as a visual information presentation device. After that, behavioral experiments were conducted to examine the alerting effect of visual information when manipulating the pedestrian's gaze direction and attention state.

<2. Preliminary survey>
In order to confirm the visibility of stimuli used in this experiment and the visibility of outdoor displays, a subjective evaluation was conducted by four evaluators on the visibility of colors and graphics and the alerting effect. This survey was conducted in the outdoor environment of the International Telecommunications Research Institute. The test vehicle was installed facing south-southwest. The weather during the experiment was sunny-cloudy, and the illuminance was 18700-64000 lx, with occasional sunlight directly on the display.

<About the device>
A 15-inch outdoor high-brightness liquid crystal display (Mitsubishi Electric, maximum brightness 1500 cd / m2) was used for stimulus presentation. The display was installed on the roof of the experimental vehicle with the screen facing forward. For the stimulation control, Octave 3.2.4 and Psychtoolbox 3 were used on a Windows (registered trademark) PC.

<2.2. Stimulation>
Stimulus evaluation was performed on colors, color combinations, and graphic visibility. For this reason, white, black, red, blue, green, yellow, cyan, magenta, and orange were used as the colors, and a diagram combining a single color and two colors was created. The figure used arrows to create a combination of two colors, the figure color and the background color. The figure pattern used eyes, arrows, and kanji (left). For the evaluation of the visibility of figures, the figure was black and the background was white.

<2.3 Subjective evaluation>
For single colors, the visibility, color combinations and graphic patterns were evaluated for visibility and alerting effects. The evaluation was carried out for the "visibility" and "attention effect" in five stages from "most visible" to "most visible", "not eye-catching" to "very eye-catching". About the pattern which combined the color, it evaluated as the whole screen instead of a single color.

<2.4. Procedure>
The evaluator observed the display from a position about 3m on the right side and a distance of 50m from the center of the vehicle. Stimulation was presented for 1 s at a random timing of 15-45 s from the start of the assessment. The evaluator glanced at the tripod, which is a gaze mark placed 3m on the right side from the center of the vehicle, and when the stimulus was presented, he immediately focused on the screen and performed a rating on the stimulus.

<2.5 Results>
Regarding the ease of color, only orange was rated as difficult to see, and the remaining colors were all evaluated as easy to see. Next, looking at the evaluation results regarding the ease of color when the two colors are combined, some colors such as cyan and magenta may be determined to be different colors. This is considered to be the effect of sunlight directly on the screen. Together with the visibility assessment results for the two color combinations, Cyan, Magenta and Orange were shown to be inappropriate for display. The evaluation result of the alerting effect for the combination of colors was almost the same as the visibility. Regarding the graphic pattern, although it was a slightly low rating value with an arrow graphic, there was not much difference between the patterns in terms of visibility and alerting effect.

  From these results, it was shown that the outdoor LED display has sufficient performance for information presentation from a viewing distance of 50m. Based on the results of this survey, in the next section, behavioral indicators were used to examine the alerting effect of on-board displays.

<3. Behavioral experiment>
FIG. 17 is an image diagram showing an assumed accident situation and an actual experimental environment. Referring to FIG. 17, in the behavioral experiment, conditions in which a pedestrian's line-of-sight direction and attention state are different were set, and the attention raising effect by visual stimulation under each condition was examined. The experimental environment was set based on the case where a car and pedestrian contact accident occurred, and an outdoor environment was set assuming a situation where a pedestrian is approaching and turning left while the car is moving straight across the road.

<3.1. Participants in the experiment>
Five people (3 men and 2 women) with healthy vision participated in the experiment. All participants had a driver's license.

<3.2. Experimental environment>
The experiment was conducted in the outdoor environment of the International Telecommunications Research Institute. The experiment time was between 9:30 and 16:30. The weather was clear or cloudy. The illuminance of the experimental environment was 7720-101100lx. The vehicle was installed northward and sunlight did not hit the screen directly. The experiment was interrupted when sunlight reflected directly from the vehicle was in the eyes of the participants.

  FIG. 18 is an image diagram showing the setting of the experimental environment. Referring to FIG. 18, the same experimental apparatus as that used in the preliminary investigation was used. During the experiment, participants were instructed to stand 50m away from the vehicle, facing the vehicle and gazing at the gaze point indicated by the tripod. The distance between the vehicle and the pedestrian was determined based on the fact that the time required to avoid the accident was 2-3 seconds (equivalent to 30-50m at 60km / h). A tripod was installed in the middle of the pedestrian and vehicle at the left 3m / 18m position to mark the gaze direction of the participants. The purpose of this experiment was to examine the effect of presentation of visual information at a viewing distance of 50 m, so the position of both the vehicle and the pedestrian was fixed.

<3.3 Issues>
FIG. 19 is an image diagram showing a flow of one execution. With reference to FIG. 19, in order to examine the alerting effect to a pedestrian by an outdoor type high brightness display, the discrimination | determination subject of the stimulus displayed on a display mounted outside a vehicle was set. The task was to determine the left-right direction of the arrow presented for 1 s following a 15-45 second blank screen. Participants were required to respond to key presses by determining their orientation as soon as they noticed that an arrow was presented on the screen without paying attention to the preset gazing point. . The color of the arrow / background was set to yellow for the background and black for the figure, based on the preliminary survey results for color combinations that are expected to have high visibility and alerting effects. The size of the arrow was 0.2 ° at a viewing distance of 50 m.

  Two conditions were set for the gaze direction of the participants in the experiment. In frontal conditions, the participants of the experiment focused on the point of interest close to the experimental vehicle under study. On the other hand, in a side-view condition that assumes a situation where a pedestrian is paying attention to a position away from the approaching vehicle, the observer pays attention to a gazing point away from the vehicle. The degree of eccentricity from the gaze position to the screen was 3.4 ° in the front condition and 20.4 ° in the side-view condition. In addition, a mental arithmetic task was set as a caution load condition assuming a situation where the pedestrian's attention deviates from other than the road environment (mental arithmetic condition). Under mental arithmetic conditions, it was required to continue subtracting 13 from 1000 when the main discrimination task started. The result of mental calculation was reported after the task was completed. The condition that does not impose a caution load task was defined as the control condition. The experiment was conducted for each condition, and each experiment was 18 trials, and the experiment time was about 10 minutes per condition.

<4. Results>
FIG. 20 is a graph showing the oversight rate of visual stimuli for each condition. FIG. 21 is a graph showing a false reaction rate for each condition. FIG. 22 is a graph showing the average reaction time for each condition. The reaction time was from the start of stimulus presentation until the participant responded to a key press. Responses after 2000 ms after the start of stimulus presentation were considered as missed stimuli and were excluded from the analysis of reaction time.

<4.1 Oversight rate>
Referring to FIG. 20, when comparing the oversight rate for each condition, it can be seen that the oversight rate for the mental arithmetic condition is higher than that for the frontal condition and the control condition. The average oversight rate of the front condition is 4.6% on average even under the caution load condition, and almost no oversight is seen. On the other hand, oversight has occurred at a relatively high frequency of 20.0% under control conditions and 24.4% under mental arithmetic conditions. When the analysis of variance of the gaze direction (2) x attention load (2) was performed for the oversight rate under each condition, only the main effect in the gaze direction was significant (F (1,4) = 30.03, p <.01 , ηp2 = 0.88).

<4.2. Reaction time, false reaction>
Referring to FIGS. 21 and 22, when compared with the line-of-sight direction, there is a tendency for the reaction time to be delayed and the number of false reactions to increase in the look-ahead condition compared to the front condition. Moreover, in the front condition, the attention load condition also shows a tendency for the results to decrease in the mental calculation condition compared to the control condition. For the reaction time of each condition, an analysis of variance of gaze direction (2) × attention load (2) was performed, and the main effect of gaze direction was significant (F (1, 4) = 17.03, p <.001 , η p2 = 0.97). Although the interaction between gaze direction and attention load was not significant (F (1, 4) = 5.773, p <.08, η p2 = 0.59),
This suggests the possibility of delay in reaction due to attention load due to mental arithmetic.

  Next, the error rate for each condition was compared. As for the false reaction rate, there was a tendency for false reactions to increase under the awkward condition. However, as a result of analysis of variance of the line-of-sight direction (2) × attention load (2) for the false reaction rate, no significant effect was observed.

<5. Consideration>
As a result of the experiment, it was shown that in the frontal condition, this experimental stimulus can be expected to have a high alerting effect regardless of the presence or absence of attention load. When the distance between the pedestrian and the vehicle is 50m, it is possible to recognize the display content from the start of the presentation of visual information to the reaction in approximately 1 second. It shows that there is room to take appropriate actions. For example, it takes about 3 seconds for a vehicle approaching at a speed of 60 km / h to reach a pedestrian. If the pedestrian can quickly recognize the information presented by this vehicle 50m before the pedestrian, the possibility that the pedestrian can take avoidance action is better than when no information is presented. it can. On the other hand, the performance of the side-view condition is generally lower than the front condition. The delay in the reaction time of about 200-300ms compared to the frontal condition is considered to be caused mainly by viewpoint movement, but if this is a delay of this level, it will be to some extent if the pedestrian is aware of the presentation of visual information. Can be expected to avoid the crisis. However, from the high rate of oversight of visual stimuli in the side-viewing condition, the alerting effect by visual stimuli when the pedestrian's line of sight is away from the vehicle cannot be expected. The high oversight rate observed in this experiment is thought to be due to the relative low brightness of the display compared to the outdoor environment. There are cases that show the effect of reducing accidents by turning on the headlights in the daytime, so there is still room for study on the installation location and specifications of the device in order to increase the warning effect by visual information.

  The effect of attention load on task performance was not confirmed from the results of this experiment. However, the results show a tendency to decrease under mental arithmetic conditions compared to the control conditions, and this experiment needs to be examined in more detail in the future because the number of samples is small and the data varies greatly. In particular, the oversight rate tends to be high under the conditions of aside and mental arithmetic. Since the pedestrian behavior patterns are diverse and the attentional state with respect to the traffic environment is constantly changing, it is necessary to conduct a more detailed study on the relationship between the pedestrian state and the visual alerting effect.

  In a preliminary study conducted prior to this experiment, we also tried conditions such as music appreciation and reading using a mobile terminal as an attention load task imposed on the experiment participants. Although the number of samples was small, the results were almost the same as this report. However, under the mobile phone reading conditions, the experiment participants did not notice the presentation of visual information at all. Since it is expected that mobile terminal operations during walking are frequent as pedestrian behavior patterns, an effective alerting method when little or no visual information enters the visual field as in this case Need to be considered. As mentioned above, we have proposed an information presentation technology that combines audiovisual and wireless, and is currently under development. By using a plurality of information presentation methods in combination, a more effective information presentation and alerting effect can be expected compared to a single method.

<6. Summary>
In this study, we examined the effectiveness of information presentation from automobiles to pedestrians using a display mounted outside the vehicle, and the effectiveness of alerting. As a result, the outdoor type high-intensity display has sufficient visibility even at a viewing distance of 50 m, and a certain effect was also observed in alerting pedestrians by visual stimuli. From this, it was shown that the presentation of visual information from a car to a pedestrian is effective for promoting pedestrian risk prediction and avoidance behavior.

<Other embodiments>
It goes without saying that the present invention can also be applied to a case where the object is achieved by supplying a program to a system or apparatus. Then, a storage medium storing a program represented by software for achieving the present invention is supplied to the system or apparatus, and a program code stored in the storage medium by the computer (or CPU or MPU) of the system or apparatus It is possible to enjoy the effects of the present invention also by reading and executing.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code May perform part or all of the actual processing, and the functions of the above-described embodiments may be realized by the processing.

  Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. The CPU of the board or the function expansion unit may perform part or all of the actual processing, and the functions of the above-described embodiments may be realized by the processing.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 Network System 99, 99X Vehicle 100, 100B, 100X Detection System 110 CPU
120 memory 130 display 140 GPS
150 button 160 communication interface 170 speaker 171 vehicle outside sensor 172 line of sight sensor 173 steering sensor 180, 180B external display 190 external speaker 200 mobile phone 210 CPU
220 memory 230 display 240 GPS
250 button 260 communication interface 270 speaker

Claims (9)

A network system comprising a detection system mounted on a vehicle and a terminal,
The detection system includes:
A communication interface;
A sensor for detecting a cognitive state of the driver of the vehicle;
A vehicle processor for transmitting warning information to the terminal via the communication interface based on the driver's cognitive state;
The terminal
An output section;
A network system, comprising: a terminal processor that causes the output unit to output at least one of a visual warning, an audible warning, and a vibration warning based on the warning information. The detection system includes:
A first position measurement sensor for detecting the position and orientation of the vehicle;
The vehicle processor transmits information indicating the position and orientation of the vehicle to the terminal via the communication interface;
The terminal
A second position measurement sensor for detecting the position of the terminal;
The terminal processor causes the output unit to output the warning when the terminal is located in the traveling direction of the vehicle based on information indicating the position and orientation of the vehicle and information indicating the position of the terminal. The network system according to claim 1. The sensor detects the driver's line of sight as the driver's cognitive state,
The detection system according to claim 1, wherein the vehicle processor determines whether to transmit the warning to the terminal based on the driver's line of sight.   The network system according to claim 1, wherein the terminal can be held by a pedestrian.   The network system according to any one of claims 1 to 3, wherein the terminal is included in a detection system mounted on another vehicle different from the vehicle.   The network system according to any one of claims 1 to 3, wherein the terminal is installed directly on the road or indirectly through another structure. A detection system mounted on a vehicle,
A communication interface;
A sensor for detecting a cognitive state of the driver of the vehicle;
A detection system comprising: a vehicle processor for transmitting warning information to the terminal via the communication interface based on the driver's recognition state. A detection system mounted on a vehicle,
An output section;
A sensor for detecting a cognitive state of the driver of the vehicle;
A detection system comprising: a computer for causing the output unit to output at least one of a visual warning, an audible warning, and a vibration warning based on the driver's cognitive state. The sensor detects the driver's line of sight as the driver's cognitive state,
The said vehicle processor makes the said output part output the information which shows the direction of the said driver | operator's eyes visually and / or auditoryly as the said warning based on the said driver | operator's eyes | visual_axis. Detection system.