JP2019114118A - Warning device, vehicle, warning method, and program - Google Patents

Abstract

To provide a warning device capable of giving a warning at a minimum necessary timing.SOLUTION: A warning device 1 comprises: an imaging device 2; a specific region detection unit 52 that detects a red region emitting red light in a front view 90 captured by the imaging device 2; a line-of-sight position detection unit 53 that detects a line-of-sight position of a driver; a determination unit 54 that performs a first determination to determine whether the line-of-sight position matches the specific region by converting the line-of-sight position and the specific region into a reference coordinate 70 and a second determination to determine whether the driver is looking at the specific region on the basis of the first determination; and a warning output unit 4 for outputting a warning when it is determined that the driver does not look at the specific region in the second determination.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a warning device, a vehicle, a warning method, and a program, and more particularly to a warning device mounted on a vehicle.

  Conventionally, as such a warning device, there has been known a device which urges a driver who has stopped at a red light to start the vehicle by giving a warning when the light turns to blue (for example, Patent Document 1) reference). However, in such a warning device, since the warning is issued whenever the signal changes, the driver may feel bothersome to the warning, and the driver may be paralyzed by the warning, thus serving as a warning. There is a risk of being lost.

JP, 2010-123100, A

  The present invention has been made in view of the problems of the prior art and has a first object to provide a warning device capable of issuing a warning at a minimum necessary timing.

  Another object of the present invention is to provide a car that can be driven safely and comfortably.

  The third object of the present invention is to provide a warning method capable of giving a warning at the minimum necessary timing.

  Furthermore, the present invention has a fourth object to provide a program that causes a computer to generate an alert at a minimum necessary timing.

  According to a first aspect of the present invention, an alerting device capable of emitting an alert at a minimum necessary timing is provided. The warning device includes an imaging unit, a specific area detection unit that detects a specific area that emits at least a specific color of an image acquired by the imaging unit, and a gaze position detection unit that detects a gaze position of a user. , And a determination unit. The determination unit determines whether the user views the specific area based on the visual line position detected by the visual line position detection unit and the specific area detected by the specific area detection unit. Make a judgment. The warning device also includes a warning output unit that outputs a warning when it is determined that the user does not look at the specific area in the user recognition determination.

  According to such a warning device, a warning is output when it is determined that the user does not look at the specific area in the user recognition determination based on the specific area emitting the light of the specific color and the gaze position of the user. Therefore, the timing of the warning is minimized, and it is effectively suppressed that the user feels bothered with the warning or paralyzes the warning.

  Note that the user recognition determination includes a first determination to determine whether the gaze position detected by the gaze position detection unit is included in the specific region detected by the specific region detection unit. Is preferred. In this case, the first determination is further performed by associating the sight line position detected by the sight line position detection unit and the specific area detected by the specific area detection unit with predetermined reference coordinates. preferable. Then, in order to perform association with predetermined reference coordinates, the warning device converts the image position acquired by the imaging unit into the reference coordinates, and converts the gaze position into the reference coordinates. It may further comprise a gaze position conversion unit. In this case, preferably, in the first determination, it is determined whether the gaze position converted to the reference coordinate is included in the specific area converted to the reference coordinate. Such a configuration facilitates the construction of an algorithm for user recognition judgment.

  Further, in this warning device, it is preferable that the reference area defined by the reference coordinates is divided into a plurality of blocks having a certain size. In this case, in the first determination, the visual line position is included in the specific area when the visual line position in the reference area and the specific area in the reference area are the same block among the plurality of blocks. It is judged. With such a configuration, occurrence of an erroneous operation in which a warning is output even when the user is looking at the specific area is effectively suppressed.

  Note that at least one of the plurality of blocks may be formed to have a width different from that of the other blocks based on the magnitude of the change in the relative distance between the user and the specific area. Further, in this case, in the image acquired by the imaging unit, a region where light of the specific color is emitted and the relative distance to the user changes is defined as a specific region, and the specific region is the plurality of the plurality The reference area may be divided so as to be included in a wide block among the blocks. With such a configuration, the occurrence of the erroneous operation is effectively suppressed when the relative distance between the user and the specific area changes.

  Preferably, the user recognition determination includes a second determination to determine whether the user is looking at the specific area based on the first determination. Then, in the second determination, for example, when the number of times the sight line position is determined not to be included in the specific area in the first determination reaches a predetermined threshold within a predetermined time, the user determines that the user It may be determined that the specific area is not seen. In the second determination, as another example, in the first determination, the determination that the sight line position is included in the specific region is not performed a predetermined number of times consecutively within a predetermined time. It is also possible to judge that the user does not look at the specific area. Such a configuration makes it possible to make user recognition judgment more accurately.

  Here, the light of the specific color may be red light. With such a configuration, when the warning device is mounted in a car, it is possible to output a warning for prompting the driver to recognize a red light at a necessary minimum timing.

  According to a second aspect of the present invention, there is provided an automobile capable of driving safely and comfortably. This car is equipped with the above-mentioned warning device. The vehicle may further include a windshield defining a reference area defined by predetermined reference coordinates. When the coordinates of the windshield are used as the reference coordinates, the algorithm construction of the user recognition judgment can be facilitated.

  According to a third aspect of the present invention, there is provided a warning method capable of giving a warning at a minimum required timing. According to this warning method, first, an image is acquired, and a specific area that emits light of a specific color is detected among the acquired images. In addition, the gaze position of the user is detected. Then, based on the detected gaze position and the detected specific area, it is determined whether the user is looking at the specific area. Then, when it is determined that the user does not look at the specific area, a warning is output.

  According to a fourth aspect of the present invention, there is provided a program that causes a computer to generate an alert at a minimum required timing. The program causes a computer to acquire an image, and detects a specific area that emits light of a specific color among the acquired images. Further, the computer is made to detect the gaze position of the user, and based on the detected gaze position and the detected specific area, it is determined whether the user is looking at the specific area. Then, when it is determined that the user does not look at the specific area, a warning is output.

  According to the present invention, a warning is output when it is determined that the user does not look at the specific area in the user recognition determination based on the specific area and the gaze position of the user, so the timing of the warning is minimized. be able to.

FIG. 1 is a block diagram showing a warning device in an embodiment of the present invention. FIG. 2 is a view schematically showing a part of the interior of a car equipped with the warning device shown in FIG. FIG. 3 is a block diagram showing an example of a functional configuration of the warning device shown in FIG. FIG. 4 is a schematic view showing reference coordinates of the warning device shown in FIG. FIG. 5 is a flowchart showing the determination process of the warning device shown in FIG. FIG. 6 is a schematic diagram showing red light detection coordinates of the warning device shown in FIG. 3 and determination processing in a determination unit. FIG. 7A is a schematic view showing gaze position detection coordinates of the warning device shown in FIG. 3 and determination processing in a determination unit. FIG. 7B is a schematic view showing gaze position detection coordinates of the warning device shown in FIG. 3 and determination processing in a determination unit. FIG. 8 is a schematic view showing a modification of the reference coordinates shown in FIG.

  Hereinafter, an embodiment of a warning device according to the present invention will be described in detail with reference to FIGS. 1 to 8. Note that, in FIG. 1 to FIG. 8, the same or corresponding components are given the same reference numerals, and duplicate explanations are omitted. In addition, in FIGS. 1 to 8, the scale and dimensions of each component may be exaggerated and shown, or some components may be omitted.

  FIG. 1 is a block diagram showing a warning device 1 according to an embodiment of the present invention, and FIG. 2 is a view schematically showing a part of the interior of a car on which the warning device 1 is mounted. In the present embodiment, as shown in FIG. 2, the warning device 1 is mounted on a car 80, and when the driver (user) stopping at the red light 92 does not look at the red light 92, the red light is It is used as a signal warning device prompting you to look at 92. The application of the warning device 1 is not limited to the signal warning device, and is not limited to mounting on an automobile.

  By the way, as shown in FIG. 2, when the driver is looking at the tail lamps 94A and 94B (stop lamps) of the forward vehicle 93, since the stop of the forward vehicle 93 is recognized, the red signal is indirectly It can be said that they are aware of it. Therefore, when the driver does not look at the red light 92 but looks at the tail lamps 94A and 94B, it is not appropriate to judge that the driver does not look at the red light. In the present embodiment, in consideration of this point, the tail lamps 94A and 94B of the forward vehicle 93 are treated as those according to the red signal 92, and the red signal 92 and the tail lamps 94A and 94B are collectively determined as a red area (specific area). Configured as.

  As shown in FIGS. 1 and 2, the warning device 1 is a dashboard 81 in the same manner as the imaging device 2 (imaging unit) disposed on, for example, the dashboard 81 of an automobile 80 steered by a driver (not shown). And the warning output unit 4 that generates and outputs (generates) a predetermined warning (sound), and a warning output unit 4 at a predetermined timing. And a main control unit 5 that controls the warning device 1 to output the

  As shown in FIGS. 1 and 2, the imaging device 2 includes a lens unit 2A and an imaging element 2B, and the imaging device 2B can provide a front view 90 of the driver (that is, the front glass 82 of the automobile 80). Image of the opposite side of the driver). More specifically, the imaging device 2 is configured to image the front view 90 when the automobile 80 stops at the red light 92 or the like, and the front view 90 includes the red light 92 of the traffic light 91 and the front. The red tail lamps 94A and 94B (stop lamps) of the vehicle 93 are included.

  The gaze position detection device 3 detects the gaze position of the driver by a known method. In the example of the present embodiment, the gaze position detection device 3 detects the gaze position of the driver by the cornea detection method, and includes, for example, a camera 3A and a near infrared LED 3B as shown in FIG. There is. More specifically, the visual line position detection device 3 images the pupil of the driver's eyes with the camera 3A, and emits near infrared rays to the driver's eyes with the near infrared LED 3B to detect reflection on the cornea. . And it is comprised so that a driver | operator's gaze position may be detected based on the positional relationship of a pupil and corneal reflection. In the present embodiment, for example, the position of the line-of-sight input to the car navigation is detected by the line-of-sight position detection device 3.

  The warning output unit 4 outputs a predetermined warning sound based on a command (drive signal) from the main control unit 5 when the driver does not recognize the red signal 92, and sends a warning sound to the driver or the passenger. Promote recognition of red light 92. The warning is not limited to sound, and may be light, vibration or the like.

  As shown in FIG. 1, the main control unit 5 includes a central processing unit (CPU) 6, a read only memory (ROM) 7, and a random access memory (RAM) 8. The ROM 7 stores a program for causing the warning device 1 to execute warning processing described later, data of reference coordinates described later, and the like. The CPU 6 reads the program and data stored in the ROM 7 and executes the warning process. The RAM 8 is a work area of the CPU 6. With such a configuration, the main control unit 5 functions as a computer for executing a program. The main control unit 5 may be configured, for example, as a part of an electronic control unit (ECU) of the automobile 80, or may be included in an external device such as a car navigation system or a smartphone.

  The above program stored in the ROM 7 or the like causes the main control unit 5 (computer) to execute the following warning process in order to cause the warning output unit 4 to generate a warning sound at a predetermined timing. In other words, the main control unit 5 executes the following warning processing based on the above program. That is, (1) The image of the front view 90 (see FIG. 2) acquired by the imaging device 2B of the imaging device 2 is converted (correlated) to predetermined reference coordinates to generate a converted image, and the conversion In the rear image, a red area (specific area) consisting of red signal 92 and tail lamps 94A, 94B etc. is detected, and (2) the driver's gaze position detected by the gaze position detection device 3 is converted to the reference coordinates. (Correspondence), and (3) a judgment as to whether the driver is looking at (recognizing) the red area based on the gaze position converted to the reference coordinates and the red area converted to the reference coordinates By doing this, the warning output unit 4 generates a warning sound. In the present embodiment, the reference coordinates correspond to the coordinates of the windshield 82 of the automobile 80. Hereinafter, in the present specification, the determination of the above (3), that is, whether the driver looks at (recognizes) a specific area such as a red area may be referred to as “user recognition determination”.

  FIG. 3 is a block diagram showing an example of the functional configuration of such a main control unit 5. As shown in FIG. 3, in order to execute the warning process, the main control unit 5 controls the image conversion unit 51, the specific area detection unit 52, the sight line position conversion unit 53, the judgment unit 54, and the warning output control. And a part 55.

  As shown in FIG. 3, the image conversion unit 51 is connected to the imaging element 2B and the specific area detection unit 52 of the imaging device 2, and acquires data of the forward scene 90 imaged through the imaging element 2B. Further, the image conversion unit 51 reads out data of reference coordinates stored in the ROM 7 or the like, converts data of the front sight 90 into reference coordinates based on a predetermined algorithm, and generates a converted image. Then, the data of the converted image is stored in the RAM 8 or the like.

  For example, in the present embodiment, the imaging device 2 has an angle of view capable of capturing the front view 90 through the entire windshield 82. The image conversion unit 51 converts the data of the front view 90 captured by the imaging device 2 into an image estimated to be viewed by the driver through the windshield 82, that is, an image having reference coordinates. Note that this conversion algorithm is the distance from the imaging device 2 to the windshield 82, the distance from the imaging device 2 to the face of the user (driver) (or the position of the headrest at the driver's seat), or It can be constructed based on the distance from the face to the windshield 82 or the like.

  Here, FIG. 4 is a schematic view showing the reference coordinates. As shown in FIG. 4, the reference coordinates 70 are represented by X coordinates and Y coordinates. The X coordinate is a coordinate corresponding to the vehicle width direction of the automobile 80 as shown in FIG. 2, and the Y coordinate is a coordinate corresponding to a direction perpendicular to the X coordinate on the windshield 82. That is, the reference coordinates 70 are composed of the XY coordinates of the windshield 82, while the windshield 82 defines a reference area 71 defined by the reference coordinates 70. The reference area 71 is divided into a plurality of blocks 72 having substantially the same area. In the example of the present embodiment, the reference area 71 is divided into 32 blocks 72 of 4 rows (rows A to D) and 8 columns (rows P to W). Line C in the reference area 71 roughly corresponds to the height at which the driver's eyes are located on the windshield 82.

  In this specification, for example, when a block in row A and column P (a block located at the upper left end in FIG. 4) is specified, it is described as a block 72AP. That is, when one of the divided blocks 72 is specified, the row and the column of the block are described together in the code "72". Needless to say, the number of such divisions (blocks) can be changed as appropriate.

  Returning to FIG. 3, the specific area detection unit 52 is connected to the image conversion unit 51 and the determination unit 54, reads out the data of the converted image from the image conversion unit 51, and analyzes the data. More specifically, the specific area detection unit 52 analyzes the converted image to detect a red area (specific area) including the red signal 92 and the tail lamps 94A and 94B, and the detected red area corresponds to the above-described plurality of blocks. The specific light detection coordinates are generated by specifying which block is included in 72. Then, data of the generated specific light detection coordinates is stored in the RAM 8 or the like. As mentioned later, FIG. 6 shows such specific light detection coordinates 73.

  As shown in FIG. 3, the gaze position conversion unit 53 is connected to the gaze position detection device 3 and the determination unit 54, and reads gaze position data of the driver from the gaze position detection device 3. Further, the read visual point position data is converted into the reference coordinates 70, and the visual axis position detection coordinate is generated by specifying in which block among the plurality of blocks 72 the visual line position of the driver is included. Then, data of the generated gaze position detection coordinates is stored in the RAM 8 or the like. 7A and 7B show such gaze position detection coordinates 74, which will be mentioned again later.

  For example, in the present embodiment, the gaze position conversion unit 53 converts the gaze position data of the driver for car navigation into gaze position data of the driver on the windshield 82 (that is, gaze position data at reference coordinates).

  As shown in FIG. 3, the determination unit 54 is connected to the specific area detection unit 52, the gaze position conversion unit 53, the warning output control unit 55, etc., and includes the first determination and the second determination. Execute user recognition judgment. That is, the determination unit 54 reads the data of the specific light detection coordinate 73 from the specific area detection unit 52, and reads the data of the gaze position detection coordinate 74 from the gaze position conversion unit 53. Then, by comparing the read-out specific light detection coordinate 73 and the gaze position detection coordinate 74, a first determination is made as to whether or not the red area and the driver's gaze position coincide with each other at the reference coordinate 70. Also, based on the first determination, a second determination is made as to whether the driver is looking at the red region. More specifically, the determination unit 54 internally includes a counter C, and uses the counter C to make the second determination. When it is determined that the driver does not look at the red region, a signal is generated and the signal is output to the warning output control unit 55.

  As shown in FIG. 3, the warning output control unit 55 is connected to the determination unit 54 and the warning output unit 4. When the signal from the determination unit 54 is input, the warning output control unit 55 generates a predetermined drive signal based on the signal and outputs the drive signal to the warning output unit 4.

  Next, an example of a process in which the warning device 1 performs the warning process will be described in detail. In this warning processing process, for example, as shown in FIG. 5, steps S1 to S10 are executed. It is needless to say that the warning processing process described below is an example, and the warning processing process including other steps may be configured.

  First, as shown in FIG. 2, when the traffic light 91 in front becomes red, the driver stops the automobile 80. In the warning processing process in the present embodiment, the time when the vehicle stops is the "start" of FIG. As shown in FIG. 2, a red area (a specific area) consisting of a red light 92 of a traffic light 91 and two tail lamps 94A and 94B (stop lamps) of a front vehicle 93 exists in the front view 90 at this start time. ing. When detecting that the vehicle has stopped, the main control unit 5 outputs a signal to the imaging device 2 and the gaze position detection device 3 to operate the imaging device 2 and the gaze position detection device 3. In the present embodiment, it is assumed that the driver is looking at the tail lamp 94B of the forward vehicle 93 at the start time.

(Step S1)
When the signal from the main control unit 5 is input, the imaging device 2 images the front view 90 by the lens unit 2A and the imaging element 2B. When the image conversion unit 51 acquires data of the front view 90 from the imaging device 2B, the image conversion unit 51 reads data of the reference coordinates 70 shown in FIG. 4 from the ROM 7 and converts the captured front view 90 into reference coordinates 70 (corresponds to ) Generate an image after conversion. Then, the image conversion unit 51 outputs the data of the generated converted image to the specific area detection unit 52.

  When the data of the image after conversion is input from the image conversion unit 51, the specific region detection unit 52 specifies (detects) the red region from the image after conversion. Here, in the front view 90 of FIG. 2, the tail lamp 94A is at a height substantially the same as the position of the driver's eyes and at a position slightly offset in the −X direction (left side) from the center of the windshield 82 in the X direction. Existing. Further, the tail lamp 94B is present at a height substantially the same as the position of the driver's eyes and at a position slightly offset to the right from the center of the windshield 82 in the X direction. Further, the red light 92 is present near the upper edge of the windshield 82 and at substantially the same position as the tail lamp 94A in the X direction. Such tail lamps 94A, 94B and red signal 92, when converted to reference coordinates 70 shown in FIG. 4, will be located at block 72CR, block 72CT and block 72AR, respectively.

  Therefore, when the specific area detection unit 52 detects from the image after conversion the red area (tail lamps 94A and 94B and the red signal 92) present at the position as described above, as shown in FIG. Tail light 94A), block 72CT (tail light 94B), and block 72AR (red light 92) generate specific light detection coordinates 73 specified. The specific area detection unit 52 outputs the data of the specific light detection coordinates 73 to the determination unit 54.

(Step S2)
When the gaze position detection device 3 receives a signal from the main control unit 5, the gaze position of the driver is detected by the camera 3A and the near infrared LED 3B (see FIG. 2). Then, the data of the gaze position is output to the gaze position conversion unit 53.

  When the gaze position conversion unit 53 receives the gaze position data from the gaze position detection device 3, it reads the data of the reference coordinates 70 shown in FIG. 4 from the ROM 7 and converts the driver's gaze position to the reference coordinates 70 (corresponds) . Then, the gaze position conversion unit 53 identifies the block 72 to which the gaze position after conversion belongs, and determines the identified block as the gaze position at the reference coordinates 70.

  That is, when the sight line position conversion unit 53 detects that the driver looks at the tail lamp 94B at the “start” time of FIG. 5, as shown in FIG. 7A, driving the block 72CT to which the tail lamp 94B belongs at the reference coordinate 70 A gaze position of the person is identified, and gaze position detection coordinates 74 are generated. Then, the data of the sight line position detection coordinates 74 is output to the determination unit 54.

  In another embodiment, the order of step S1 and step S2 may be switched.

When the determination unit 54 receives data of the specific light detection coordinates 73 from the specific area detection unit 52 and receives data of the gaze position detection coordinates 74 from the gaze position conversion unit 53, the first determination and the second determination are The user recognition judgment including is performed as follows.

(Step S3 (first determination))
That is, the determination unit 54 determines whether the sight line position (block 72CT) in the sight line position detection coordinates 74 matches the red area (blocks 72AR, 72CR, 72CT) in the specific light detection coordinates 73 (first Judgment).

(Step S4)
As described above, the driver looks at the tail lamp 94B of the forward vehicle 93 at the start time of FIG. That is, the gaze position detection coordinates 74 input by the determination unit 54 are as shown in FIG. 7A, and the block 72CT is specified as the gaze position. As shown in FIG. 6, this block 72CT is one of the red areas 72AR, 72CR, 72CT of the specific light detection coordinate 73. Therefore, the determination unit 54 determines that the red line area includes the gaze position. Then, in this case, the determination unit 54 returns the warning processing process before step S1. In this case, the value of the counter C does not change.

  By the above-described return, steps S1 to S3 (second time) are executed again. Here, after the return and before the second step S2, it is assumed that the driver looks aside at the right end 82A (see FIG. 2) of the windshield 82 without changing the eye height.

(Step S2 (second time))
The gaze position conversion unit 53 converts the gaze position of the driver into the reference coordinates 70 again in the second step S2. As described above, the driver looks at the right end 82A of the windshield 82 aside. Therefore, as shown in FIG. 7B, the gaze position conversion unit 53 identifies the block 72CW corresponding to the gaze position before conversion as the gaze position at the reference coordinate 70, and generates a new gaze position detection coordinate 74. Then, the data of the new gaze position detection coordinates 74 is output to the determination unit 54.

(Step S3 (second time))
After the second step S2, the determination unit 54 again determines whether the line-of-sight position of the driver is included in the red region (first determination).

(Step S5)
Here, as described above, in the second step S2, the gaze position detection coordinates 74 shown in FIG. 7B are generated, and the gaze position in the gaze position detection coordinates 74 is the block 72CW. This block 72CW does not correspond to any of the red areas 72AR, 72CR, 72CT of the specific light detection coordinates 73 shown in FIG. Therefore, the determination unit 54 determines that the red line area does not include the gaze position.

(Step S6)
Then, the determination unit 54 increases the above-described counter C by one by determining that the line-of-sight position is not included in the red region.

(Step S7 (second determination))
Thereafter, the determination unit 54 determines whether the value of the counter C has reached a predetermined threshold N (second determination). For example, when the value of the threshold N is “5”, the value of the counter C after step S6 is still “1” and has not reached the threshold N. In this case, the determination unit 54 returns the warning processing process to the state before step S1, and executes the third step.

(Step S8 (second determination))
The determination unit 54 repeats such processing for a predetermined time. Then, when the value of the counter C reaches a predetermined threshold N (for example, “5”) within the predetermined time, the determination unit 54 determines that the driver does not look at the red regions 72AR, 72CR, 72CT. That is, it is determined that the driver does not look at the red light 92 and does not look at the tail lamps 94A and 94B of the preceding vehicle 93 which can be determined to be equal to looking at the red light 92 (second judgment). Then, in this case, the determination unit 54 generates a signal and outputs the signal to the warning output control unit 55.

  In the present embodiment, when the counter C does not reach the threshold N when the predetermined time (“the first predetermined time”) has elapsed, the time count is reset, but the value of the counter C is maintained. Are configured to Therefore, after the first predetermined time has elapsed, the first step S1 in the second predetermined time is executed. Then, when the counter C reaches the threshold value N within the second predetermined time, the determination unit 54 determines that “the driver does not look at the red signal 92”. However, it is not necessary to configure in this way, and when the value of the counter C does not reach the threshold N when the first predetermined time has elapsed, the value of the counter C may be reset together with the time count. .

(Step 9)
When the warning output control unit 55 receives the signal from the determination unit 54, the warning output control unit 55 generates a predetermined drive signal and outputs the generated drive signal to the warning output unit 4. Then, the warning output unit 4 outputs a warning sound based on the drive signal.

(Step 10)
When the warning output unit 4 outputs the warning sound, the main control unit 5 resets the value of the counter C of the determination unit 54 and the time count. Then, time counting is started again, that is, the second time counting is started, and the first step S1 is performed again.

  The warning processing process consisting of the above steps S1 to S10 is executed until the stop state of the automobile 80 is released. Then, when the stopped state of the automobile 80 is released, the main control unit 5 ends the warning processing process.

  As described above, according to the warning device 1 of the present embodiment, after the determination unit 54 performs the user recognition determination, it is determined that the driver does not look at a red signal in the user recognition determination. The warning is output to. More specifically, after the first determination of whether or not the driver's gaze position is included in the red region, the number of determinations of “not included” reaches a threshold within a predetermined time. , "It is judged that the driver does not look at the red light" (second judgment), and a warning is output when it is judged that "the driver does not look at the red light" in the second judgment. It is supposed to be Therefore, the timing at which the warning is issued is minimized, and as a result, it is effectively suppressed that the driver feels bothered with the warning or paralyzed by the warning. Therefore, according to the automobile 80 equipped with such a warning device 1, the driver can drive safely and comfortably.

  By the way, even when the driver is looking at the red area, the driver's gaze position may move slightly. Therefore, in such a case, it is not appropriate that it is determined that the driver does not look at a red light. In this embodiment, since the gaze detection coordinates and the specific light detection coordinates are generated by the reference area divided into a plurality of blocks, the gaze position is included in the block of the red region when the fluctuation of the gaze position is slight. It remains as it is. Therefore, it is effectively suppressed that a warning is issued although the driver is looking at the red area.

  Further, in the present embodiment, since both the red area and the gaze position are converted to the XY coordinates (reference coordinate 70) of the windshield 82, the red area and the gaze position can be easily compared. Therefore, the algorithm of the user recognition judgment can be easily configured.

  Although the preferred embodiments of the present invention have been described above, it is needless to say that the present invention is not limited to the above-described embodiments and may be implemented in various different forms within the scope of the technical idea thereof. For example, the following other embodiments may be configured.

  In the above embodiment, an example in which the warning device operates when the vehicle stops at a red light has been described, but in other embodiments, the warning device may operate even when the vehicle is traveling. Good. In this case, since the driver's car is traveling, the relative distance between the driver and, for example, the signal changes. Therefore, the position of the signal at the reference coordinates changes with time, and there is a possibility that an erroneous operation may occur in which the judgment unit judges that “the driver does not look” although the driver looks at the signal. Therefore, in order to avoid such a problem, for example, as shown in FIG. 8, a reference coordinate 170 is generated in which the block 172 above the height (line C) of the driver's eyes is larger than the block 72. May be Thus, even when the position of the signal changes with time in the reference coordinates, this change in position is a change in the same block, and thus the occurrence of the above-mentioned malfunction is suppressed. In addition, since the relative distance between the driver and the signal changes with time during traveling, it is conceivable that the position of the red light source (for example, red signal) changes on the captured image. Therefore, in the embodiment in which the warning device is operated when the vehicle travels, among the upper (large) blocks 172, a block including a red light source whose position changes on the photographed image may be set as the specific area. With such a configuration, the malfunction can be suppressed more effectively.

  Moreover, although the above-mentioned embodiment demonstrated the example which made the coordinate (XY coordinate) of the windshield 82 of a motor vehicle the reference coordinate, you may make the camera coordinate of the imaging device 2 into a reference coordinate in other embodiment. In this embodiment, since the camera coordinates themselves become the reference coordinates, it is not necessary to provide the image conversion unit 51. In this embodiment, the gaze position conversion unit converts (associates) the gaze position to the camera coordinates, thereby determining whether the gaze position matches the specific area (first determination). In another embodiment, the gaze position detection device may be configured to detect the gaze position of the driver on the windshield 82. According to such a configuration, it is possible to directly use the detected gaze position as the reference coordinate.

  Moreover, although the above-mentioned embodiment demonstrated the example which warns based on the frequency | count that it was judged that a gaze position is not contained in a specific area, in other embodiment, it judges that a gaze position is contained in a specific area. It may be configured to issue a warning based on the number of times performed. In the user recognition judgment in such an embodiment, the user does not look at the specific area if the judgment (first judgment) that the sight line position is included in the specific area is not continuously performed a predetermined number of times within a predetermined time. It may be determined that (the second determination).

  Moreover, although the above-mentioned embodiment demonstrated the example which warns, when a user (driver) does not recognize red light (red signal), in other embodiment, it is another color other than red light. It may be configured to output a warning when the light is not recognized by the user. And in such an embodiment, a field (for example, a field (blue field) which emits blue light) which emits light of other colors turns into a specific field. In another embodiment, the color to be determined as the specific area may be changed between stopping and traveling. For example, a warning may be issued when a green light (blue area) is not recognized while the vehicle is stopped, and a warning may be issued when a red light (red area) is not recognized while traveling.

  In another embodiment, the main control unit may be configured to include another processor in addition to the CPU. In this case, the other processor functions as part or all of the functional configuration (that is, the image conversion unit, the specific area detection unit, the gaze position conversion unit, the determination unit, the warning output control unit, etc.) It is also good.

  In another embodiment, the program stored in the ROM 7 or the like is stored in the HDD (Hard Disk Drive) of a server for data distribution and distributed to an electronic device such as a car navigation system or a smartphone via a network. May be Further, the storage medium may be sold or distributed in a state where the program is stored in a storage medium such as an optical disc such as a CD, a DVD, and a BD (Blu-Ray Disk), a USB memory, and a memory card. When the program downloaded through the server or storage medium described above is installed in the electronic device having the same configuration as that of this embodiment, the same effect as that of the above embodiment can be obtained.

  The terms "lower", "upper", "upper", "lower" and other terms used in the present specification are used in the context of the illustrated embodiment, It changes with the relative positional relationship of.

1 Warning device 2 Imaging device (imaging unit)
2A lens unit 2B image pickup device 3 gaze position detection device (gaze position detection unit)
3A camera 3B near infrared LED
4 warning output unit 5 main control unit (computer)
6 CPU
7 ROM
8 RAM
51 image conversion unit 52 specific region detection unit 53 gaze position conversion unit 54 determination unit 55 warning output control unit 70, 170 reference coordinates 71 reference region 72, 172 blocks 73 specific light detection coordinates 74 gaze position detection coordinates 80 automobile 81 dashboard 82 Windshield 90 Front view 91 Traffic light 92 Red light 93 Forward vehicle 94A, 94B Tail lamp

Claims (11)

An imaging unit,
A specific area detection unit that detects a specific area that emits light of at least a specific color in the image acquired by the imaging unit;
A gaze position detection unit that detects a gaze position of the user;
A determination unit that makes a user recognition determination as to whether or not the user is looking at the specific region based on the gaze position detected by the gaze position detection unit and the specific region detected by the specific region detection unit When,
And a warning output unit that outputs a warning when it is determined that the user does not look at the specific area in the user recognition determination.   The user recognition determination includes a first determination to determine whether the sight line position detected by the sight line position detection unit is included in the specific area detected by the specific area detection unit. The warning device according to Item 1.   The first determination is performed by associating the sight line position detected by the sight line position detection unit and the specific area detected by the specific area detection unit with predetermined reference coordinates. Warning device. An image conversion unit configured to convert an image acquired by the imaging unit into the reference coordinates;
And a visual point position conversion unit for converting the visual line position into the reference coordinates,
The warning device according to claim 3, wherein, in the first determination, it is determined whether or not the gaze position converted to the reference coordinate is included in the specific area converted to the reference coordinate. The reference area defined by the reference coordinates is divided into a plurality of blocks having a certain size,
In the first determination, when the sight line position in the reference area and the specific area in the reference area are the same block among the plurality of blocks, it is determined that the sight line position is included in the specific area The warning device according to claim 3 or 4.   The warning device according to claim 5, wherein at least one of the plurality of blocks is formed to have a width different from other blocks based on the magnitude of change in relative distance between the user and the specific area. . In the image acquired by the imaging unit, a region that emits light of the specific color and in which the relative distance to the user changes is set as a specific region,
The warning device according to claim 6, wherein the reference area is divided such that the specific area is included in a wide block among the plurality of blocks.   In the user recognition determination, the user looks at the specific area when the number of times the visual line position is determined not to be included in the specific area in the first determination reaches a predetermined threshold within a predetermined time. The warning device according to any one of claims 2 to 7, comprising a second determination to determine that it is not.   A car comprising the warning device according to any one of the preceding claims. Get an image,
Detecting a specific area emitting light of a specific color in the acquired image;
Detect the user's gaze position,
Based on the detected gaze position and the detected specific area, it is determined whether the user is looking at the specific area.
A warning method, which outputs a warning when it is determined that the user does not look at the specific area. On the computer
Let me get an image,
Detecting a specific area emitting light of a specific color in the acquired image;
Make the user's gaze position detected
Based on the detected gaze position and the detected specific area, it is determined whether the user is looking at the specific area or not.
A program for outputting a warning when it is determined that the user does not look at the specific area.

Images