WO2022030191A1

WO2022030191A1 - In-vehicle device and driving assistance method

Info

Publication number: WO2022030191A1
Application number: PCT/JP2021/026182
Authority: WO
Inventors: 克志古澤
Original assignee: フォルシアクラリオン・エレクトロニクス株式会社
Priority date: 2020-08-06
Filing date: 2021-07-12
Publication date: 2022-02-10
Also published as: JP2022029911A

Abstract

Provided is an in-vehicle device that is capable of notifying a driver of the likelihood of suffering road rage prior to the occurrence of road rage behavior.　This in-vehicle device 100 comprises: a state detection unit 133 which determines whether or not a host vehicle 5 is in a specific state that could pose a risk to other vehicles around the host vehicle 5; an image acquisition unit 135 which, in the case when occurrence of the specific state is determined, acquires an image captured of the surroundings of the host vehicle 5 at the time of the occurrence; a distance acquisition unit 136 which acquires distance information indicative of the distances to the other vehicles around the host vehicle 5; a target vehicle detection unit 137 which, in the case when the distance to any of the other vehicles indicated by the distance information is detected to be equal to or less than a set value, detects said vehicle from the image captured at the time of the occurrence; and a notification control unit 138 which, in the case when said vehicle has been detected by the target vehicle detection unit 137, causes a display unit 80 or an audio output unit 90 mounted in the host vehicle 5 to execute a first notification action.

Description

In-vehicle device and driving support method

The present invention relates to an in-vehicle device and a driving support method.

Conventionally, there is known a technology for detecting a driving motion and preventing an accident or trouble with a following vehicle. For example, Patent Document 1 determines the presence / absence and amount of acceleration / deceleration of another vehicle existing around the own vehicle, the steering state, the lighting state, the presence / absence of an obstacle, and the like, and based on the determined state of the other vehicle. Disclose a driving support device that determines the possibility of a fanning action on the own vehicle, and based on the determination result, performs notification control and / or behavior control of the own vehicle to the driver of the own vehicle.

Japanese Unexamined Patent Publication No. 2006-205773

However, when determining the possibility of a fanning action on the own vehicle based on the condition of another vehicle, there is a concern that the judgment will be delayed and the driver will be notified after the fanning action occurs.

The present invention has been made in view of the above circumstances, and provides an in-vehicle device and a driving support method capable of notifying the driver of a vehicle that there is a possibility of being fanned before the fanning occurs. The purpose is.

This specification includes all the contents of the Japanese patent application / Japanese Patent Application No. 2020-133511 filed on August 6, 2020.

In order to solve the above problems, the in-vehicle device of the present invention is an in-vehicle device mounted on a vehicle, and includes a first state information acquisition unit that acquires driving state information indicating the running state of the vehicle, and driving of the vehicle. Based on the information acquired by the second state information acquisition unit that acquires the driver state information indicating the state of the person, the first state information acquisition unit, and the second state information acquisition unit, the vehicle travels. A state determination unit that determines whether or not the vehicle is in a specific state that may pose a danger to other vehicles around the vehicle, and a state determination unit that determines that the vehicle is in the specific state. The distance indicated by the distance information is set, the image acquisition unit that acquires the image taken at the time of occurrence of the surroundings, the distance acquisition unit that acquires the distance information indicating the distance between the vehicle and another vehicle around the vehicle, and the distance information. When another vehicle below the value is detected, the detection unit that detects the other vehicle from the image taken at the time of occurrence by comparing the photographed image of the other vehicle with the image taken at the time of occurrence, and the said When the other vehicle is detected by the detection unit, the notification unit mounted on the vehicle is provided with a notification control unit for executing a first notification operation for alerting the driver. ..

According to the present invention, it is possible to notify the driver of the vehicle that there is a possibility of being fanned before the fanning occurs.

FIG. 1 is a block diagram showing a configuration of a warning system. FIG. 2 is a flowchart showing a procedure for detecting a specific state. FIG. 3 is a flowchart showing the operation at the time of warning.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a block diagram showing a configuration of the driving support system 1. Hereinafter, the vehicle equipped with the driving support system 1 is referred to as the own vehicle 5, and the vehicle other than the own vehicle 5 is referred to as another vehicle.
The driving support system 1 includes an external photography unit 10, an interior photography unit 20, a sonar unit 30, a biological sensor 40, a vehicle speed sensor 50, an acceleration sensor 60, an operation unit 70, a display unit 80, an audio output unit 90, and an in-vehicle device 100. Be prepared.

The vehicle outside shooting unit 10 includes four cameras, a front camera 11, a rear camera 13, a left side camera 15, and a right side camera 17. The front camera 11 photographs the front of the own vehicle 5. The rear camera 13 photographs the rear of the own vehicle 5. The left side camera 15 photographs the left side of the own vehicle 5. The right side camera 17 photographs the right side of the own vehicle 5. The present embodiment describes a case where the vehicle exterior photographing unit 10 includes four cameras, but the number of cameras for photographing the outside of the vehicle is arbitrary.

The front camera 11, the rear camera 13, the left side camera 15, and the right side camera 17 are images from image sensors such as CCD (Charge-Coupled Device) and CMOS (Complementary Metal-Oxide-Semiconductor) and the light receiving state of the image sensors, respectively. It is equipped with a data processing circuit that generates a data processing circuit.

The angles of view of the front camera 11, the rear camera 13, the left side camera 15, and the right side camera 17 are adjusted so that they can shoot a range of 360 ° around the own vehicle 5. The front camera 11, the rear camera 13, the left side camera 15, and the right side camera 17 shoot each shooting range at a predetermined frame rate to generate a shot image. The photographed image of the vehicle outside photographing unit 10 is hereinafter referred to as an outside vehicle photographed image. The images taken outside the vehicle generated by each of the front camera 11, the rear camera 13, the left side camera 15, and the right side camera 17 are output to the in-vehicle device 100. The in-vehicle device 100 temporarily stores the out-of-vehicle image taken from the in-vehicle photography unit 10 in the storage unit 110 included in the in-vehicle device 100.

The vehicle interior photographing unit 20 includes a camera 25 that photographs the vehicle interior of the own vehicle 5. The camera 25 also includes an image sensor such as a CCD or CMOS, and a data processing circuit that generates an image from the light receiving state of the image sensor. The camera 25 is installed on, for example, a dashboard, a rear-view mirror, a ceiling, or the like, and is oriented so that a driver seated in the driver's seat can be photographed, and the angle of view, the installation position, and the like are set. The camera 25 photographs the interior of the vehicle at a predetermined frame rate to generate a captured image. The photographed image taken by the vehicle interior photographing unit 20 is hereinafter referred to as an indoor photographed image. The vehicle interior shooting unit 20 outputs the generated indoor shooting image to the vehicle-mounted device 100. The in-vehicle device 100 temporarily stores the indoor photographed image input from the vehicle interior photographing unit 20 in the storage unit 110.

The sonar unit 30 is mounted at a plurality of locations such as the front, rear, left side, and right side of the own vehicle 5, and uses ultrasonic waves to obstruct other vehicles, pedestrians, structures, etc. existing around the own vehicle 5. Detects the orientation and distance of objects. The sonar unit 30 outputs sonar information indicating the direction and distance of an obstacle to the in-vehicle device 100. The in-vehicle device 100 temporarily stores the sonar information input from the sonar unit 30 in the storage unit 110.

The biosensor 40 is a sensor that detects the biometric information of the driver seated in the driver's seat. The biological information is information representing the state of biological phenomena such as heartbeat, respiration, brain wave, pulse, blood pressure, sweating, and body temperature. In the present embodiment, the case where the biosensor 40 is a heart rate sensor provided on the seat belt of the seat and the biometric information is information representing the driver's heart rate will be described. However, the biosensor 40 is a breathing and brain wave. , Pulse, blood pressure, sweating, body temperature and other biometric information may be detected. The biosensor 40 outputs the detected biometric information of each occupant to the in-vehicle device 100. The in-vehicle device 100 temporarily stores the biological information input from the biological sensor 40 in the storage unit 110.

The vehicle speed sensor 50 is a sensor that detects the vehicle speed of the own vehicle 5. The vehicle speed sensor 50 outputs vehicle speed information indicating the detected vehicle speed of the own vehicle 5 to the in-vehicle device 100.
The acceleration sensor 60 is a sensor that detects the acceleration of the own vehicle 5. The acceleration sensor 60 outputs acceleration information indicating the detected acceleration of the own vehicle 5 to the in-vehicle device 100.
The in-vehicle device 100 temporarily stores the input vehicle speed information and acceleration information in the storage unit 110.

The operation unit 70 functions as a reception unit that accepts the operations of the occupants. The operation unit 70 includes hardware such as switches and buttons, and outputs an operation signal corresponding to the hardware that has received the operation to the in-vehicle device 100.

The display unit 80 includes a touch panel 85. The touch panel 85 includes a display panel such as a liquid crystal panel or an organic EL panel, and a touch sensor for detecting a touch operation on the display panel. A display image is displayed on the display panel under the control of the vehicle-mounted device 100. The touch sensor detects the position of the display panel touched by the occupant of the own vehicle 5, and outputs coordinate information indicating the detected position to the in-vehicle device 100.

The audio output unit 90 includes an audio processing unit 91 and a speaker 93.
The audio processing unit 91 includes a D / A converter, a volume circuit, an amplifier circuit, etc., digitally / analog-converts audio data input from the in-vehicle device 100 by the D / A converter, and adjusts the volume level by the volume circuit. , It is amplified by the amplifier circuit and output as audio from the speaker 93.

The in-vehicle device 100 is a computer device including a storage unit 110 and a processor 130. When the own vehicle 5 detects a "specific state" which is a state in which the own vehicle 5 may pose a danger to other vehicles around the own vehicle 5, the in-vehicle device 100 detects the specific state of the own vehicle. When another vehicle traveling around 5 is photographed and another vehicle whose distance from the own vehicle 5 is less than or equal to a preset set distance is detected, the other vehicle detects a "specific state". Occasionally, it is determined whether or not the vehicle has been traveling around the own vehicle 5. Then, the in-vehicle device 100 executes a notification operation to alert the driver when the other vehicle is traveling around the own vehicle 5 when the "specific state" is detected. Assist the driver in driving operations.

The storage unit 110 includes, for example, a flash memory and a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). Further, the storage unit 110 may be configured to include a volatile memory such as a RAM (Random Access Memory) in addition to the non-volatile memory.
The storage unit 110 stores a control program executed by the processor 130. Further, the storage unit 110 temporarily stores an image taken outside the vehicle, an image taken indoors, biological information, vehicle speed information, and acceleration information.

The processor 130 is composed of a CPU (Central Processing Unit) and a microcomputer such as an MCU (MicroControllerUnit) and an MPU (MicroProcessorUnit) equipped with a CPU. Further, the in-vehicle device 100 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The in-vehicle device 100 includes a first state information acquisition unit 131, a second state information acquisition unit 132, a state detection unit 133, a driving tendency determination unit 134, an image acquisition unit 135, a distance acquisition unit 136, a target vehicle detection unit 137, and notification control. A unit 138 is provided. These functional blocks indicate the functions of the vehicle-mounted device 100 realized by the processor 130 executing the control program stored in the storage unit 110.

The first state information acquisition unit 131 acquires the running state information indicating the running state of the own vehicle 5. The first state information acquisition unit 131 acquires an image taken outside the vehicle, vehicle speed information, and acceleration information from the storage unit 110 as traveling state information. The first state information acquisition unit 131 outputs the acquired out-of-vehicle image, vehicle speed information, and acceleration information to the state detection unit 133.

The second state information acquisition unit 132 acquires driver state information indicating the state of the driver of the own vehicle 5. The second state information acquisition unit 132 acquires indoor photographed images and biological information from the storage unit 110 as driver state information. The second state information acquisition unit 132 outputs the acquired indoor photographed image and biological information to the state detection unit 133.

The state detection unit 133 is input with an image taken outside the vehicle, vehicle speed information, and acceleration information from the first state information acquisition unit 131, and an image taken indoors and biometric information are input from the second state information acquisition unit 132. Based on these input information, the state detection unit 133 detects a specific state in which the own vehicle 5 can adversely affect the running of another vehicle.

The state detection unit 133 corresponds to the state determination unit of the present invention, and determines a specific state based on the running state of the own vehicle 5, a specific emotion of the driver, and a specific operation. That is, when the state detection unit 133 detects the running state of the own vehicle 5, a specific emotion of the driver, or a specific motion, the running of the own vehicle 5 poses a danger to other vehicles around the own vehicle 5. Judge that you are in a specific condition that may affect you. The traveling state of the own vehicle 5 includes interrupted driving, sudden acceleration and deceleration, and low-speed traveling. In addition, the driver's specific movement includes a forward leaning posture, a gesture of hitting a steering wheel, an instrument panel, or the like. In addition, the driver's specific emotions include emotions that adversely affect safe driving, such as impatience, irritation, and tension.

The state detection unit 133 detects the interrupted driving of the own vehicle 5 based on the image taken outside the vehicle. The interrupted operation of the own vehicle 5 means that the own vehicle 5 changes its course to the front on the course of another running vehicle. For example, the state detection unit 133 detects a lane change based on an image taken outside the vehicle of the road surface, and when the lane change is detected, interrupts driving based on the distance to another vehicle traveling around the own vehicle 5. To detect.

Further, the state detection unit 133 detects the sudden acceleration or deceleration of the own vehicle 5 by comparing the acceleration indicated by the acceleration information with the preset threshold value, and presets the vehicle speed indicated by the vehicle speed information. The low-speed running of the own vehicle 5 is detected by comparing with the threshold value.

Further, the state detection unit 133 detects the driver's posture and type from the captured image of the vehicle interior photographing unit 20 to detect a specific state. The posture detected by the state detection unit 133 includes a forward leaning posture, and the type includes an act of hitting a steering wheel, an instrument panel, or the like.

Further, the state detection unit 133 detects the driver's face image from the captured image of the vehicle interior photographing unit 20, and detects the driver's specific emotion based on the detected face image. Specific emotions are emotions that adversely affect driving, such as impatience, irritation, and tension.
For example, the state detection unit 133 detects a specific emotion of the driver based on the master image stored in the storage unit 110. This master image is a face image of the driver taken in advance, and is an image taken for each specific emotion such as impatience, irritation, and tension. The state detection unit 133 compares the face image captured in the captured image of the vehicle interior photographing unit 20 with the master image, and detects a specific emotion of the driver.

Further, the state detection unit 133 may detect a specific emotion of the driver by using the biological information of the biological sensor 40 in addition to the captured image of the vehicle interior photographing unit 20. For example, when the state detection unit 133 detects an angry emotion as a specific emotion from the image taken by the vehicle interior photographing unit 20, and the heart rate indicated by the biological information of the biological sensor 40 is equal to or higher than a preset value. , It may be determined that the driver is feeling angry.

The driving tendency determination unit 134 counts the number of detections for each detection item detected by the state detection unit 133 as a specific state such as interrupted operation, sudden acceleration, sudden deceleration, low-speed running, specific emotion, and specific operation. The storage unit 110 stores the count value of the counter for each detection item. The driving tendency determination unit 134 adds 1 to the count value of the counter of the corresponding detection item each time the state detection unit 133 detects a specific state.

The driving tendency determination unit 134 determines the driving tendency of the driver based on the count value of each detection item stored in the storage unit 110. For example, the driving tendency determination unit 134 compares the count value of each detection item with the threshold value set for each detection item, and detects the detection item whose count value is equal to or greater than the threshold value. The driving tendency determination unit 134 determines the driving tendency of the driver based on the detected detection item.

The image acquisition unit 135 causes the outside vehicle photographing unit 10 to perform photography when the state detection unit 133 detects a specific state. The vehicle outside shooting unit 10 shoots the outside of the vehicle for a preset time to generate an externally shot image. In the externally captured image captured by the vehicle exterior photographing unit 10, another vehicle existing around the own vehicle 5 is photographed when a specific state occurs. The image acquisition unit 135 stores the image taken outside the vehicle input from the image acquisition unit 10 outside the vehicle in the storage unit 110. An externally captured image taken by the outside vehicle photographing unit 10 when the state detecting unit 133 detects a specific state is referred to as an image taken at the time of occurrence.

The distance acquisition unit 136 acquires distance information indicating the distance between the own vehicle 5 and another vehicle existing around the own vehicle 5. The distance acquisition unit 136 acquires sonar information of the sonar unit 30 from the storage unit 110 as distance information. The distance acquisition unit 136 outputs the acquired sonar information to the target vehicle detection unit 137.
Further, the distance acquisition unit 136 may detect the distance to another vehicle based on the captured image acquired by the image acquisition unit 135 and generate the distance information. For example, the camera included in the outside vehicle photographing unit 10 may be configured by a stereo camera, and the distance from the own vehicle 5 to another vehicle may be detected based on the image taken by the stereo camera.

The target vehicle detection unit 137 corresponds to the detection unit of the present invention. The distance information acquired by the distance acquisition unit 136 is input to the target vehicle detection unit 137. The target vehicle detection unit 137 detects another vehicle whose distance from the own vehicle 5 is equal to or less than the set distance based on the distance indicated by the input distance information. Hereinafter, the detected other vehicle is referred to as a target vehicle. When the target vehicle is detected, the target vehicle detection unit 137 outputs information such as the direction of the detected target vehicle to the image acquisition unit 135 and instructs the image acquisition unit 135 to take a picture of the target vehicle. The image acquisition unit 135 causes the vehicle outside shooting unit 10 to perform shooting according to the instruction of the target vehicle detection unit 137, and outputs the shot outside vehicle shooting image to the target vehicle detection unit 137.

When the image acquisition unit 135 inputs an image taken outside the vehicle, the target vehicle detection unit 137 compares the input external image of the target vehicle with the image taken at the time of occurrence, and detects an image. It is determined whether or not the target vehicle is photographed. That is, when a specific state is detected, it is determined whether or not the target vehicle is traveling around the own vehicle 5.
The target vehicle detection unit 137 outputs the first notification to the notification control unit 138 when the target vehicle is photographed in the detection image. The first notification is a notification indicating that a vehicle whose distance to the own vehicle 5 is less than or equal to the set distance and which was traveling around the own vehicle 5 was detected when a specific state was detected. be.
Further, the target vehicle detection unit 137 outputs a second notification to the notification control unit 138 when the target vehicle is not photographed in the image captured at the time of occurrence. The second notification is a notification indicating that a vehicle whose distance from the own vehicle 5 is equal to or less than the set distance has been detected.

The notification control unit 138 controls the operation of the display unit 80 and the voice output unit 90, which are notification units. The notification control unit 138 corresponds to the notification control unit and the display control unit of the present invention. When the first notification is input from the target vehicle detection unit 137, the notification control unit 138 controls the display unit 80 and the voice output unit 90 to execute the first notification operation. As the first notification operation, the notification control unit 138 displays a warning image on the display unit 80 and outputs a predetermined warning sound from the voice output unit 90. In the warning image, for example, another vehicle whose inter-vehicle distance is less than or equal to the set distance is detected, and the detected other vehicle is a vehicle that was traveling around the own vehicle 5 when a specific state was detected. That there is, etc. are displayed.

Further, when the second notification is input from the target vehicle detection unit 137, the notification control unit 138 controls the display unit 80 to execute the second notification operation. In the second notification operation, the level of alerting is set lower than that in the first notification operation. Therefore, when the second notification is input from the target vehicle detection unit 137, the notification control unit 138 causes the display unit 80 to display the warning image without outputting the warning sound from the voice output unit 90. In the warning image, for example, it is displayed that another vehicle whose inter-vehicle distance is less than or equal to the set distance has been detected.

Further, the notification control unit 138 causes the display unit 80 to display the driving tendency determined by the driving tendency determination unit 134. For example, when the driving tendency determination unit 134 determines that the driver's driving tendency tends to have a high frequency of sudden acceleration, the notification control unit 138 provides guidance to notify that the frequency of sudden acceleration tends to be high. It is displayed on the display unit 80. Further, when the driving tendency determination unit 134 determines that the notification control unit 138 tends to take an action expressing an angry feeling while driving, the frequency of taking an action expressing an angry feeling increases. The display unit 80 displays a guide notifying that the price is high.

Next, the operation of the in-vehicle device 100 will be described with reference to the flowcharts shown in FIGS. 2 and 3.
First, the vehicle-mounted device 100 determines whether or not the own vehicle 5 is traveling (step S1). The in-vehicle device 100 determines whether or not the own vehicle 5 is traveling based on the vehicle speed information. When the own vehicle 5 is not running (step S1 / NO), the in-vehicle device 100 waits until the own vehicle 5 starts running.

When the own vehicle 5 is traveling (step S1 / YES), the in-vehicle device 100 acquires an image taken outside the vehicle (step S2). The in-vehicle device 100 detects interrupted driving based on the acquired image taken outside the vehicle (step S3). Further, the in-vehicle device 100 may detect the interrupted driving by referring to the acceleration information of the own vehicle 5. When the in-vehicle device 100 detects an interrupted operation (step S3 / YES), the in-vehicle device 100 determines that a specific state has been detected, causes the vehicle-mounted photographing unit 10 to perform shooting, and acquires a captured image at the time of occurrence (step S4). ..

Next, the in-vehicle device 100 stores the detection time when a specific state is detected and the captured image at the time of occurrence in the storage unit 110 in association with each other (step S5). Further, the vehicle-mounted device 100 increments the count value of the counter that counts the number of detections of the interrupt operation by 1 (step S6).

Further, the vehicle-mounted device 100 acquires the vehicle speed information detected by the vehicle speed sensor 50 when the determination in step S3 is a negative determination (step S3 / NO) or after the processing in step S6 (step S7). The in-vehicle device 100 compares the acquired vehicle speed information with a preset threshold value, and determines whether or not the own vehicle 5 is in a low-speed running state and this low-speed running state continues for a set time or longer. Is determined (step S8). When the in-vehicle device 100 determines that the low-speed running state has continued for a set time or longer (step S8 / YES), the in-vehicle device 100 determines that a specific state has been detected, and causes the outside-vehicle photographing unit 10 to perform imaging when the occurrence occurs. Acquire the captured image (step S9).

Next, the in-vehicle device 100 stores the detection time when a specific state is detected and the captured image at the time of occurrence in the storage unit 110 in association with each other (step S10). Further, the vehicle-mounted device 100 increments the count value of the counter that counts the number of detections of low-speed operation by 1 (step S11).

Further, the in-vehicle device 100 acquires the acceleration information detected by the acceleration sensor 60 when the determination in step S8 is a negative determination (step S8 / NO) or after the processing in step S11 (step S12). The in-vehicle device 100 compares the acquired acceleration information with a preset threshold value to detect sudden acceleration or deceleration of the own vehicle 5 (step S13).

When the in-vehicle device 100 detects a sudden acceleration or a sudden deceleration of the own vehicle 5 (step S13 / YES), the in-vehicle device 100 determines that a specific state has been detected, causes the outside shooting unit 10 to take a picture, and captures an image taken at the time of occurrence. Acquire (step S14).

Next, the in-vehicle device 100 stores the detection time when a specific state is detected and the captured image at the time of occurrence in the storage unit 110 in association with each other (step S15). Further, the in-vehicle device 100 increments the count value of the counter that counts the number of detections of sudden acceleration or deceleration by 1 (step S16).

Further, the in-vehicle device 100 acquires an indoor photographed image and biological information of the vehicle interior photographing unit 20 when the determination in step S13 is a negative determination (step S13 / NO) or after the processing of step S16 (step S17). .. The in-vehicle device 100 detects a specific action or a specific emotion of the driver based on the acquired indoor photographed image and biological information (step S18). When the in-vehicle device 100 detects a leaning posture in front of the driver or a gesture of hitting the steering wheel, instrument panel, or the like, it is determined that a specific motion is detected. Further, the in-vehicle device 100 determines that a specific emotion has been detected when the estimated emotion is an emotion such as impatience, irritation, or tension.

When a specific motion or a specific emotion is detected (step S18 / YES), the in-vehicle device 100 causes the outside vehicle photographing unit 10 to perform shooting and acquires a captured image at the time of occurrence (step S19). The in-vehicle device 100 stores the detection time at which a specific motion or a specific emotion is detected in the storage unit 110 in association with the captured image at the time of occurrence (step S20). Further, the vehicle-mounted device 100 increments the count value of the corresponding counter in the detected specific state by 1 (step S21).

The in-vehicle device 100 determines whether or not there is a counter whose count value is equal to or greater than the threshold value when the determination in step S18 is a negative determination (step S18 / NO) or after executing the process in step S21 (step S18 / NO). Step S22). The in-vehicle device 100 returns to the determination in step S1 when there is no counter whose count value is equal to or greater than the threshold value (step S22 / NO). Further, when there is a counter whose count value is equal to or higher than the threshold value (step S22 / YES), the in-vehicle device 100 determines and determines the driving tendency of the driver based on the specific state associated with this counter. The driving tendency is displayed on the display unit 80 (step S23).

Next, the operation of the in-vehicle device 100 from the detection of the specific state to the notification operation will be described with reference to the flowchart shown in FIG.
First, the vehicle-mounted device 100 determines whether or not a specific state has been detected (step S31). When a specific state is detected (step S31 / YES), the in-vehicle device 100 acquires sonar information of the sonar unit 30 (step S32).

The in-vehicle device 100 detects another vehicle whose distance from the own vehicle 5 is less than or equal to the set distance based on the acquired sonar information (step S33). When the vehicle-mounted device 100 does not detect another vehicle within the set distance (step S33 / NO), the vehicle-mounted device 100 returns to the determination in step S31.

Further, the in-vehicle device 100 determines whether the own vehicle 5 is traveling in order to exclude the case where another vehicle within the set distance is detected (step S33 / YES) and the inter-vehicle distance is shortened due to waiting for a signal or the like. It is determined whether or not (step S34). When the own vehicle 5 is not traveling (step S34 / NO), the in-vehicle device 100 returns to the determination in step S31.

Further, when the own vehicle 5 is traveling (step S34 / YES), the in-vehicle device 100 instructs the outside shooting unit 10 to take a picture of the target vehicle with the other vehicle detected in step S33 as the target vehicle (step). S35), the externally photographed image of the target vehicle is acquired.

Next, the in-vehicle device 100 reads out the captured image at the time of occurrence from the storage unit 110. The in-vehicle device 100 refers to the detection time stored in association with the image taken at the time of occurrence, and reads out the image taken at the time of occurrence within a predetermined time from the detection time to the current time. The in-vehicle device 100 compares the read image taken at the time of occurrence with the image taken externally taken by the target vehicle, and determines whether or not the target vehicle is photographed in the image taken at the time of occurrence (step S36). When the target vehicle is photographed in the image captured at the time of occurrence (step S36 / YES), the vehicle-mounted device 100 executes the first notification operation (step S37). As the first notification operation, the notification control unit 138 displays a warning image on the display unit 80 and outputs a predetermined warning sound from the voice output unit 90.
Further, when the target vehicle is not captured in the captured image at the time of occurrence (step S36 / NO), the vehicle-mounted device 100 executes the second notification operation (step S38). The notification control unit 138 causes the display unit 80 to display a warning image as the second notification operation.

As described above, the in-vehicle device 100 of the present embodiment has a first state information acquisition unit 131, a second state information acquisition unit 132, a state detection unit 133, an image acquisition unit 135, a distance acquisition unit 136, and a target vehicle detection unit 137. And a notification control unit 138.

The first state information acquisition unit 131 acquires the running state information indicating the running state of the own vehicle 5.
The second state information acquisition unit 132 acquires driver state information indicating the state of the driver of the own vehicle 5.
Based on the information acquired by the first state information acquisition unit 131 and the second state information acquisition unit 132, the state detection unit 133 may cause the traveling of the own vehicle 5 to pose a danger to other vehicles around the own vehicle 5. Determine if you are in a particular state.
When the state detection unit 133 determines that the vehicle is in a specific state, the image acquisition unit 135 acquires an image taken at the time of occurrence in which the surroundings of the own vehicle 5 are photographed.
The distance acquisition unit 136 acquires distance information indicating the distance between the own vehicle 5 and other vehicles around the own vehicle 5.
When another vehicle whose distance indicated by the distance information is equal to or less than the set value is detected, the target vehicle detection unit 137 compares the photographed image of the other vehicle with the photographed image at the time of occurrence, and when the other vehicle is generated. Detect from captured images.
When another vehicle is detected by the target vehicle detection unit 137, the notification control unit 138 performs a first notification operation for calling the driver's attention to the display unit 80 and the voice output unit 90 mounted on the own vehicle 5. Let it run.
Therefore, when the own vehicle 5 detects a specific state that may pose a danger to other vehicles around the own vehicle 5, the in-vehicle device 100 is present around the own vehicle 5 at the time of detecting the specific state. The distance between the other vehicle and the own vehicle 5 is detected, and when the detected distance is equal to or less than the set value, the first notification operation for alerting the driver is executed. Therefore, it is possible to notify the driver of the vehicle that there is a possibility of being fanned before the fanning occurs.

Further, the second state information acquisition unit 132 acquires the driver's biological information, and estimates the driver's emotion as the driver's state information based on the acquired biological information.
The state detection unit 133 determines a specific state based on the driver's emotions estimated by the second state information acquisition unit 132.
Therefore, it is possible to improve the estimation accuracy of the driver's emotion and improve the detection accuracy of a specific state.

The second state information acquisition unit 132 acquires a photographed image of the interior of the own vehicle 5, and acquires information indicating the driver's operation based on the acquired photographed image as driver state information.
The state detection unit 133 determines a specific state based on the driver's operation estimated by the second state information acquisition unit 132.
For example, as a driver's motion, the driver's emotion is estimated by detecting the forward leaning posture and the action of hitting the steering wheel, instrument panel, etc., and a specific state is detected based on the estimated driver's emotion. It is possible to improve the detection accuracy of the state of.

When the target vehicle detection unit 137 cannot detect another vehicle from the captured image when the target vehicle is generated, the notification control unit 138 executes a second notification operation having a lower level of alerting than the first notification operation.
Therefore, even if the other vehicle does not exist around the own vehicle 5 when the specific state is detected, the driver can be alerted when the distance to the own vehicle 5 is equal to or less than the set value.

Further, the in-vehicle device 100 counts and counts the number of times the state detection unit 133 detects a specific state based on the traveling state information and the number of times the specific state is detected based on the driver state information. The driving tendency determination unit 134 for determining the driving tendency of the driver based on the number of times is provided.
The notification control unit 138 causes the display unit 80 to display the driving tendency determined by the driving tendency determination unit 134.
Therefore, it is possible to determine the driving tendency of the driver based on the number of detections of a specific state and notify the driver of the determined driving tendency.

The above-described embodiment is merely an example of one aspect of the present invention, and can be arbitrarily modified and applied without departing from the gist of the present invention.
For example, the configuration of the vehicle-mounted device 100 shown in FIG. 1 is a schematic diagram showing the functions of the vehicle-mounted device 100 classified according to the main processing content, and the configuration of the vehicle-mounted device 100 is based on the processing content. It can be divided into more blocks. Further, this functional block may be configured to execute more processing by one block shown in FIG. 1. Further, the processing of each block may be executed by one hardware or may be executed by a plurality of hardware. Further, the processing of each block may be realized by one program or may be realized by a plurality of programs.

Further, the processing units of the flowcharts shown in FIGS. 2 and 3 are divided according to the main processing contents in order to make the processing of the in-vehicle device 100 easy to understand, and depending on the method and name of division of the processing units. The present invention is not limited. Further, the processing of the in-vehicle device 100 can be divided into more processing units depending on the processing content, or one processing unit can be divided so as to include more processing. Further, the processing order of the above flowchart is not limited to the illustrated example.

Further, when the operation support method of the present invention is realized by a computer, it is also possible to configure the program to be executed by the computer in the form of a recording medium or a transmission medium for transmitting the program. As the recording medium, a magnetic or optical recording medium or a semiconductor memory device can be used. Specific examples of the recording medium include a flexible disk, an HDD (Hard Disk Drive), a CD-ROM (Compact Disk Read Only Memory), a DVD, a Blu-ray (registered trademark) Disc, and a magneto-optical disk. Further, as the recording medium, a portable recording medium such as a flash memory or a card-type recording medium, or a fixed recording medium can be mentioned. Further, the recording medium may be a non-volatile storage device such as RAM, ROM, or HDD, which is an internal storage device included in the display device.

1 Driving support system 5 Vehicle 10 External photography unit 11 Front camera 13 Rear camera 15 Left side camera 17 Right side camera 20 Vehicle interior photography unit 25 Camera 30 Sonar unit 40 Biosensor 50 Vehicle speed sensor 60 Acceleration sensor 70 Operation unit 85 Touch panel 90 Voice Output unit 91 Voice processing unit 93 Speaker 100 In-vehicle device 110 Storage unit 130 Processor 131 First state information acquisition unit 132 Second state information acquisition unit 133 State detection unit 134 Driving tendency determination unit 135 Image acquisition unit 136 Distance acquisition unit 137 Target vehicle Detection unit 138 Notification control unit

Claims

It is an in-vehicle device mounted on a vehicle.
The first state information acquisition unit that acquires the running state information indicating the running state of the vehicle, and
A second state information acquisition unit that acquires driver state information indicating the state of the driver of the vehicle, and
Whether or not the running of the vehicle is in a specific state that may pose a danger to other vehicles around the vehicle based on the information acquired by the first state information acquisition unit and the second state information acquisition unit. A state judgment unit that determines whether or not
An image acquisition unit that acquires an image taken at the time of occurrence by photographing the surroundings of the vehicle when the state determination unit determines that the vehicle is in the specific state.
A distance acquisition unit that acquires distance information indicating the distance between the vehicle and other vehicles around the vehicle, and
When another vehicle whose distance indicated by the distance information is equal to or less than the set value is detected, the photographed image of the other vehicle is compared with the photographed image at the time of occurrence, and the other vehicle is photographed at the time of occurrence. The detector to detect from
When the other vehicle is detected by the detection unit, a notification control unit that causes the notification unit mounted on the vehicle to execute a first notification operation for calling attention to the driver.
An in-vehicle device characterized by being equipped with.
The second state information acquisition unit acquires the driver's biometric information, estimates the driver's emotions as the driver's state information based on the acquired biometric information, and determines the driver's emotions.
The vehicle-mounted device according to claim 1, wherein the state determination unit determines the specific state based on the driver's emotions estimated by the second state information acquisition unit.
The second state information acquisition unit acquires a photographed image of the interior of the vehicle, and acquires information indicating the driver's operation based on the acquired photographed image as the driver state information.
The vehicle-mounted device according to claim 1 or 2, wherein the state determination unit determines the specific state based on the driver's operation estimated by the second state information acquisition unit.
When the detection unit cannot detect the other vehicle from the captured image at the time of occurrence, the notification control unit executes a second notification operation having a lower level of alerting than the first notification operation. The vehicle-mounted device according to any one of claims 1 to 3, wherein the in-vehicle device is characterized by the above.
The state determination unit counts the number of times the specific state is detected based on the driving state information and the number of times the specific state is detected based on the driver state information, and is based on the counted number of times. The tendency determination unit that determines the driving tendency of the driver,
The vehicle-mounted device according to any one of claims 1 to 4, further comprising a display control unit for displaying the driving tendency determined by the tendency determination unit on the display unit.
Steps to acquire driving state information indicating the running state of the vehicle,
A step of acquiring driver state information indicating the state of the driver of the vehicle, and
Based on the acquired driving state information and driver state information, a step of determining whether or not the running of the vehicle is in a specific state that may pose a danger to other vehicles around the vehicle, and
When it is determined that the vehicle is in the specific state, a step of acquiring an image taken at the time of occurrence, which is a photograph of the surroundings of the vehicle, and
A step of acquiring distance information indicating the distance between the vehicle and another vehicle around the vehicle, and
When another vehicle whose distance indicated by the distance information is equal to or less than the set value is detected, the photographed image of the other vehicle is compared with the photographed image at the time of occurrence, and the other vehicle is photographed at the time of occurrence. And the steps to detect from
When the other vehicle is detected from the image taken at the time of occurrence, the step of causing the notification unit mounted on the vehicle to execute the first notification operation for calling the driver's attention.
A driving support method characterized by having.