JP7376996B2 - Vehicle dangerous situation determination device, vehicle dangerous situation determination method, and program - Google Patents

Description

The present invention relates to a device for determining a dangerous situation of a vehicle, a method of determining a dangerous situation of a vehicle, and a program.

BACKGROUND ART Conventionally, for example, Patent Document 1 listed below describes detecting a change in the condition of a passenger of a moving body and transmitting external situation information including a photographed image of the outside of the moving body in accordance with the change in condition.

JP2017-21745A

However, the technology described in the above patent document uses an in-vehicle camera to capture the passenger's movements, and if a change in the passenger's condition is detected, a part of the image received from the outside camera is extracted. The transmission is performed by Therefore, individual judgments regarding events that occur on routes such as roads are based on images. Therefore, there is a problem in that complicated processing is required to judge the image.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a new and improved vehicle that is capable of determining dangerous situations outside the vehicle with a simple configuration. An object of the present invention is to provide a dangerous situation determination device, a method for determining a dangerous situation of a vehicle, and a program.

In order to solve the above problems, one aspect of the present invention includes a line-of-sight information acquisition unit that acquires the line-of-sight movement of a vehicle occupant ; A dangerous situation determination unit that determines which of the plurality of preset dangerous situations outside the vehicle corresponds to the situation and based on a plurality of parameters related to changes in the line of sight . A hazardous situation determination device is provided.
The dangerous situation determination unit may determine the dangerous situation based on a speed vector of the vehicle and a speed vector of coordinates of a point where the lines of sight of the left and right eyes of the occupant intersect.

The dangerous situation determination unit may determine the dangerous situation by comparing the movement of the line of sight with a predetermined trajectory.

Further, the device may include a position information acquisition unit that acquires position information of the dangerous situation when the dangerous situation determination unit determines that the dangerous situation exists.

Furthermore, the vehicle may include a transmission processing section that transmits the dangerous situation and the position information to outside the vehicle.

The vehicle may also include a presentation processing section that performs a process of presenting a question regarding the dangerous situation to the occupant when the dangerous situation determining section determines that the dangerous situation exists.

Moreover, the dangerous situation determination unit may determine the dangerous situation by considering answers to the questions.

Further, the presentation processing section may present an answer to the question in the form of options.

Furthermore, the information processing apparatus may include an answer acquisition unit that acquires an answer based on the selection of the options for the question.

Furthermore, the vehicle may include an answer acquisition unit that acquires the answer to the question using the voice of the occupant.

In order to solve the above problems, according to another aspect of the present invention, the computer acquires the movement of the line of sight of the occupant of the vehicle, the movement of the line of sight , and a plurality of preset outside of the vehicle. determining which of the plurality of predetermined dangerous situations outside the vehicle corresponds to a plurality of dangerous situations outside the vehicle based on a plurality of parameters related to changes in line of sight ; A method for determining dangerous situations is provided.

Further, in order to solve the above problems, according to another aspect of the present invention, there is provided a means for acquiring the movement of the line of sight of an occupant of a vehicle, a means for acquiring the movement of the line of sight of a vehicle occupant , and a plurality of dangerous situations outside the vehicle set in advance. In response to this, a program is provided for causing a computer to function as a means for determining which of the plurality of preset dangerous situations outside the vehicle falls under , based on a plurality of parameters related to changes in line of sight. be done.

As described above, according to the present invention, it is possible to provide a dangerous situation determination device for a vehicle, a method for determining a dangerous situation for a vehicle, and a program that can determine a dangerous situation outside the vehicle with a simple configuration. becomes.

1 is a schematic diagram showing a vehicle system 1000 according to an embodiment of the present invention and the configuration of its surroundings. It is a flowchart showing the processing performed in this embodiment. It is a flowchart showing the processing performed in this embodiment. FIG. 3 is a schematic diagram showing a list of information indicating a dangerous situation transmitted to the server 2000 side. It is a schematic diagram which shows the calculation method of coordinate Pe. FIG. 2 is a schematic diagram showing a case where a passenger is looking at a fallen object on the road. FIG. 2 is a schematic diagram illustrating a state in which a person, an animal, or the like crossing a road in the direction of travel is observed. FIG. 7 is a schematic diagram illustrating a case where compatibility between a movement trajectory of coordinates Pe and another model is confirmed. FIG. 2 is a schematic diagram showing a method for distinguishing moving objects from animals, people, and the like.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that, in this specification and the drawings, components having substantially the same functional configurations are designated by the same reference numerals and redundant explanation will be omitted.

In this embodiment, by incorporating human danger recognition and judgment abilities into risk determination, dangerous situations that cannot be detected by a vehicle are detected, and the danger information is shared by transmitting the information through vehicle-to-vehicle communication or the like. FIG. 1 is a schematic diagram showing a vehicle system 1000 according to an embodiment of the present invention and the configuration of its surroundings. Vehicle system 1000 is basically a system configured in a vehicle such as an automobile. As shown in FIG. 1, the vehicle system 1000 includes a line of sight detection device 100, a position detection device 200, an information presentation device 300, an information input device 400, a control device 500, a navigation device 700, a communication device 800, and a database 950. It is configured. Vehicle system 1000 is configured to be able to communicate with external server 2000.

The line of sight detection device 100 is installed, for example, on a dash panel inside a vehicle, and detects the line of sight of an occupant . The position detection device 200 acquires the current position of the vehicle using a global positioning system (GPS) or the like.

The information presentation device 300 includes a speaker, a monitor, a HUD (Head-up Display) device, etc., and presents information to the occupant. The information input device 400 is a device through which a passenger inputs information. The information input device 400 may be one that inputs information by operating a button, a touch panel, or the like, for example. Further, the information input device 400 may be a device that inputs the voice of the occupant, such as a microphone.

The control device 500 recognizes a dangerous situation outside the vehicle based on the passenger's line of sight detected by the line of sight detection device 100, and performs processing for transmitting information regarding the dangerous situation to the server 2000. For this reason, the control device 500 includes a line-of-sight information acquisition section 502, a dangerous situation determination section 504, a position information acquisition section 506, a transmission processing section 508, a presentation processing section 510, and an answer acquisition section 512. The line-of-sight information acquisition unit 502 acquires the movement of the passenger's line of sight from the line-of-sight detection device 100. The dangerous situation determination unit 504 determines the dangerous situation outside the vehicle based on the movement of the line of sight. The position information acquisition unit 506 acquires the position information of the dangerous situation when it is determined that the dangerous situation exists. The transmission processing unit 508 performs a process of transmitting the dangerous situation and position information to the server 2000 outside the vehicle or another vehicle via the communication device 800. The presentation processing unit 510 performs processing for presenting a question regarding a dangerous situation to the occupant via the information presentation device 300. The answer acquisition unit 512 performs a process of acquiring an answer to a question via the information input device 400. Note that each component of the control device 500 can be configured by a circuit (hardware) or a central processing unit such as a CPU and a program (software) for making this function.

Communication device 800 communicates with server 2000 outside the vehicle and sends and receives various information. Navigation device 700 searches for a route from the current location to the destination based on map information. Therefore, the navigation device 700 can obtain the current position of the vehicle using a global positioning system (GPS) or the like. Furthermore, the navigation device 700 stores the route that the vehicle has traveled to the current location.

The server 2000 acquires information regarding dangerous situations from the vehicle system 1000 and registers it in the database. Additionally, the server 2000 performs a process of transmitting the accumulated information to the vehicle system 1000. For this reason, the server 2000 includes a dangerous situation acquisition section 2010, a data registration section 2020, a transmission processing section 2030, and a database 2040. Dangerous situation acquisition unit 2010 acquires information regarding a dangerous situation from vehicle system 1000. The data registration unit 20 2 0 performs a process of registering the information acquired by the dangerous situation acquisition unit 2010 in the database 2040 . Transmission processing section 2030 performs processing for communicating with communication device 800 of vehicle system 1000. Information regarding dangerous situations obtained from the vehicle system 1000 is registered in the database 2040. Note that each component of the server 2000 can be configured by a circuit (hardware) or a central processing unit such as a CPU and a program (software) for making this function.

2 and 3 are flowcharts showing the processing performed in this embodiment. First, in step S10, driving of the vehicle is started with the line of sight detection device 100 turned on. In the next step S12, the occupant's eye closure rate E is calculated. In the next step S14, the coordinate Pe where the passenger's line of sight is directed is calculated. Note that although a driver is illustrated as an occupant, the occupant may be someone other than the driver. In the next step S16, it is determined whether the change in the coordinate Pe matches a preset parameter, and if it matches, the process proceeds to step S18.

As will be described later, a plurality of parameters representing changes in the line of sight are set in advance, and each of the plurality of parameters corresponds to a plurality of dangerous situations. Therefore, if the change in coordinate Pe matches any of the parameters, it can be determined that the line of sight remains and the occupant has recognized any of the dangerous situations. Furthermore, by determining which parameter the change in the coordinate Pe matches, it is possible to determine the type of dangerous situation that the occupant is looking at. On the other hand, in step S16, if the change in coordinate Pe does not match the preset parameters, the process returns to step S14.

In step S18, it is determined whether the eye-closing rate E is within a predetermined threshold E1, and if the eye-closing rate E is within the predetermined threshold E1, the process advances to step S19. On the other hand, if the eye closure rate E exceeds the predetermined threshold E1 in step S18, the process returns to step S12.

If the process proceeds to step S19, the change in the coordinate Pe matches the preset parameter, and the eye closure rate E is within the predetermined threshold E1, so the occupant is awake and outside the vehicle. It can be determined that a dangerous situation has been recognized. Therefore, in step S19, the position coordinate Pv of the own vehicle at the time when the condition of step S16 is satisfied and the coordinate Pe where the line of sight remains are acquired. Note that the position coordinate Pv is position information in a geographic coordinate system (latitude, longitude, altitude, etc.), and the coordinate Pe is position information in a vehicle coordinate system (X, Y, Z), etc.

In the next step S20, the coordinates Pe of the occupant's gaze point are converted into a geographic coordinate system to obtain the occupant's attention point Pve. Note that the coordinate Pe can be converted to the caution point Pve by setting the origin to the own vehicle position Pv, which is a geographical coordinate system, with respect to the coordinate Pe, which uses the vehicle position as the origin. In the next step S21, a dangerous situation is determined based on the match between the change in the coordinate Pe and a preset parameter. This allows the vehicle system 1000 to recognize the location information of the caution point Pve and the details of the dangerous situation. In the next step S22, the positional information of the caution point Pve, the time information at the time when the condition of step S16 was satisfied, the details of the dangerous situation, etc. are registered in the database 950 and transmitted to the server 2000.

In subsequent processing, the details of the dangerous situation are clarified by asking the crew members questions. Note that the subsequent processing may not be performed. First, in step S22, the information presentation device 300 issues a question. Questions are asked by displaying multiple types of questions in the form of choices. Question options include accidents, broken vehicles, etc. (choice 1-1), falling objects, damage to roads and property, etc. (choice 1-2), and accidents caused by moving objects such as people or animals jumping out (choice 1-3). , frozen roads, snow, deep puddles, etc. (options 1-4), and others (options 1-5).

In step S24, the occupant operates the information input device 400 to select from these options one that matches the recognized dangerous situation. The selection result by the occupant is registered in the vehicle database 950 in step S26, and is transmitted to the server 2000 in step S28. Note that if the occupant does not select any option, that is, does not answer the question, the process returns to step S12.

In the next step S30, the passenger is asked whether or not to supplement the information further, and if an answer is obtained that the information will be supplemented, the process proceeds to step S32. On the other hand, if the answer is not to supplement in step S30, the process returns to step S12.

In step S32, the information input device 400 acquires voice data captured by the occupant through utterances. In the next step S34, the acquired audio data is registered in the database 950. Note that the voice data may be registered in the database 950 as is, or the voice data may be converted into text and registered. Alternatively, the audio data may be converted into text, classified into detailed items, and then registered in the database 950. In the next step S36, the information registered in step S34 is transmitted to the server 2000. Furthermore, the processing of audio data may be performed on the vehicle system 1000 side or on the server 2000 side.

In the next step S38, it is determined whether or not there is a vehicle capable of inter-vehicle communication, and if there is a vehicle capable of inter-vehicle communication, the process proceeds to step S40. In step S40, the information registered in steps S22, S26, and S36 is transmitted to a vehicle capable of vehicle-to-vehicle communication.

As described above, according to the process shown in FIG. 2, the type of dangerous situation can be determined based on the coordinate Pe where the line of sight is directed, and it can be registered in the vehicle along with the position information and transmitted to the server 2000 and other vehicles. I can do it. Furthermore, by asking questions to the occupants, dangerous situations can be determined in more detail, and detailed information on dangerous situations can be registered in the vehicle and transmitted to the server 2000 or other vehicles. The vehicle system 1000 that has determined a dangerous situation can also warn the occupant using the information presentation device 300. Further, the vehicle system 1000 that has determined a dangerous situation can also be utilized for vehicle control such as vehicle deceleration control and steering control.

FIG. 4 is a schematic diagram showing a list of information indicating a dangerous situation transmitted to the server 2000 side. The information shown in FIG. 4 is transmitted from a plurality of vehicles to server 2000 and registered in database 2040 of server 2000. In FIG. 4, "occurrence time" is time information at the time when the condition of step S16 is satisfied, and "latitude, longitude" is position information of the caution point Pve. "Automatic" in the "determination means" indicates a case where a dangerous situation is determined in step S21 based on a match between a change in the coordinate Pe and a preset parameter. "Manual" in the "determination means" indicates a case where a dangerous situation is determined based on the answer to the question in step S24 and the supplementary voice data in step S32 in addition to step S21. “Audio” indicates a data file of voices uttered by the occupant. "Text" indicates the number of texts based on the audio data in step S32. Further, "detailed classification" indicates detailed classification of dangerous situations. "Count" indicates the number of vehicle systems 1000 that have transmitted information on each caution point Pve. Note that the vehicle system 1000 or the server 2000 may include a machine learning unit that learns the correspondence between line of sight movements and detailed classifications. By learning the correspondence between eye movement and detailed classification based on a large amount of data, it becomes possible to further improve the accuracy of determining dangerous situations based on eye movement.

Next, the calculation of the coordinate Pe in step S14 of FIG. 2 and the determination of whether the change in the coordinate Pe and the parameter match in step S16 will be explained based on FIGS. 5 to 9. 5 to 9 show the vehicle 10 with the occupant 20 riding in it as seen from above. FIG. 5 is a schematic diagram showing a method of calculating the coordinate Pe. The line of sight detection device 100 detects the line of sight of the left and right eyes, and the point where the line of sight of the left and right eyes intersect is set as a coordinate Pe. Although the origin of the coordinate Pe is the reference point of the vehicle 10, the latitude and longitude of the reference point of the vehicle 10 are obtained from the position detection device 200 so that it can be converted to the latitude and longitude of the caution point Pve.

FIG. 6 shows a case where the occupant 20 is looking at a fallen object on the road. In this case, as the vehicle 10 moves forward, the position of the coordinate Pe toward which the line of sight is directed approaches the vehicle 10 side. Here, Vv is the velocity vector of the vehicle 10, and Ve is the velocity vector of the coordinate Pe. Further, let Vex be the x component of Ve, and let Vey be the y component of Ve. In this case, as a determination condition in step S16, a state in which the occupant 20 is staring at a certain point on the road is defined by the following AND condition of condition (1), condition (2), and condition (3).
Condition (1) |Vv+Vex|≦Vx1 (However, Vx1 is a value close to 0)
Condition (2) |Vey|≦Vy1 (However, Vy1 is a value close to 0)
Condition (3) Conditions (1) and (2) continue for T1 [s] or more (T1 can be any value)
In addition to the above conditions, a condition such as "no height difference" may be added.

FIG. 7 shows a state in which the occupant 20 is watching people, animals, etc. crossing the road in the direction of travel. In this case, the position of the coordinate Pe toward which the line of sight is directed moves from the front left of the vehicle 10 to the front right. In this case, as the determination condition in step S16, the state of looking at a person, animal, etc. crossing the road is defined by the AND condition of the following condition (4), condition (5), and condition (6). do.
Condition (4) |Vv+Vex|≦Vx2 (However, Vx2 is a value close to 0)
Condition (5) |Vey|≧Vy2 (However, Vy2 is a predetermined value)
Condition (6) Conditions (1) and (2) continue for more than T1 [s] (However, T1 can be any value)
In addition to the above conditions, a condition such as "no height difference" may be added.

FIG. 8 shows a case where the compatibility between the movement trajectory of the coordinate Pe and another model is confirmed. In this case, as a determination condition in step S16, the degree of coincidence between the mobile body model Pth set in advance and the coordinates Pe is determined. The mobile model Pth assumes the movement of a person, an animal, etc., and a plurality of types may be set in advance.

FIG. 9 shows a method for distinguishing between moving objects such as animals and people. When checking the surrounding situation, the line of sight changes at a very high speed in order to check various places. The speed is different from the speed at which moving objects such as animals and people move, so the speed is used for discrimination. In this case, the following is defined as the determination condition in step S16. Note that the determination in FIG. 9 can be performed in combination with other determinations.
When Ve≧V1, there is no moving object such as a person or animal.
If Ve<V1, there is a possibility that a moving object such as a person or animal exists.
If Ve<V1 continues for a certain period of time or more, it is determined that there is a moving object such as a person or animal.
Further, when Ve is stationary (or almost stationary), it may be estimated that there is a risk that a person or animal is stationary.

Although preferred embodiments of the present invention have been described above in detail with reference to the accompanying drawings, the present invention is not limited to such examples. It is clear that a person with ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea stated in the claims. It is understood that these also naturally fall within the technical scope of the present invention.

502 Line of sight information acquisition unit 504 Danger situation determination unit 506 Position information acquisition unit 508 Transmission processing unit 510 Presentation processing unit 512 Answer acquisition unit

Claims (12)

a line-of-sight information acquisition unit that acquires the movement of the line-of-sight of a vehicle occupant;
Which of the plurality of preset dangerous situations outside the vehicle is determined based on the movement of the line of sight and a plurality of parameters related to the change in the line of sight corresponding to a plurality of preset dangerous situations outside the vehicle. a dangerous situation determination unit that determines whether the
A dangerous situation determination device for a vehicle, comprising: The dangerous situation determination unit determines the dangerous situation based on a speed vector of the vehicle and a speed vector of coordinates of a point where the lines of sight of the left and right eyes of the occupant intersect. 1. The dangerous situation determination device for a vehicle according to 1. 3. The dangerous situation of a vehicle according to claim 1, wherein the dangerous situation determining unit determines the dangerous situation by comparing the movement of the line of sight with a predetermined trajectory. Status determination device. 4. The apparatus according to claim 1, further comprising a location information acquisition section that acquires location information of the dangerous situation when the dangerous situation determination section determines that the dangerous situation exists. dangerous situation determination device for vehicles. The dangerous situation determination device for a vehicle according to claim 4, further comprising a transmission processing unit that performs a process of transmitting the dangerous situation and the position information to outside the vehicle. 6. The vehicle according to claim 1, further comprising a presentation processing section that performs a process of presenting a question regarding the dangerous situation to the occupant when the dangerous situation determining section determines that the dangerous situation exists. A dangerous situation determination device for a vehicle according to any one of the above. 7. The dangerous situation determining device for a vehicle according to claim 6, wherein the dangerous situation determining unit determines the dangerous situation by considering answers to the questions. The dangerous situation determination device for a vehicle according to claim 6 or 7, wherein the presentation processing section presents an answer to the question in the form of options. The dangerous situation determination device for a vehicle according to claim 8, further comprising an answer acquisition unit that acquires an answer to the question by selecting the option. The dangerous situation determination device for a vehicle according to any one of claims 6 to 8, further comprising an answer acquisition section that acquires an answer to the question using a voice of an occupant. The computer is
Obtaining eye movement of an occupant of the vehicle;
Which of the plurality of preset dangerous situations outside the vehicle is determined based on the movement of the line of sight and a plurality of parameters related to the change in the line of sight corresponding to a plurality of preset dangerous situations outside the vehicle. a step of determining whether the
A method for determining a dangerous situation of a vehicle. means for obtaining the movement of the line of sight of an occupant of a vehicle;
Which of the plurality of preset dangerous situations outside the vehicle is determined based on the movement of the line of sight and a plurality of parameters related to the change in the line of sight corresponding to a plurality of preset dangerous situations outside the vehicle. means to determine whether
A program that allows a computer to function as a computer.

Images