JP7422018B2 - Vehicle control system and vehicle control method - Google Patents

Description

The present invention relates to a vehicle control system and a vehicle control method.

2. Description of the Related Art In-vehicle warning devices are known that notify the approach of an object such as a rear vehicle or a two-wheeled vehicle when the behavior of an occupant in a vehicle is detected to be attempting to open a door (for example, see Patent Document 1). In Patent Document 1, when it is detected that an occupant is about to reach a doorknob, a light source (such as an LED element) and a speaker installed near the doorknob in the vehicle are turned on and emitted sound.

Japanese Patent Application Publication No. 2015-209154

However, with conventional configurations, even if the occupant happens to reach for the doorknob without intending to exit the vehicle, a light source and speaker near the doorknob notify the occupant of the approach of an object, which may be a nuisance for the occupant. . Furthermore, if the occupant is not facing the door, the occupant will not notice the lighting of the light source near the doorknob.
The present invention has been made in view of this background, and an object of the present invention is to facilitate notification appropriate to the condition of the occupant.

A first aspect of achieving the above object includes an approach detection unit that detects the approach of a notification target to a vehicle , and detects the orientation of a first part that can identify the direction of the line of sight or face of an occupant in the vehicle. a direction detection section; a motion detection section that detects a predetermined motion of the occupant in getting off the vehicle based on a motion of a second region of the occupant that is different from the first region; and a first notification section that notifies the occupant of the vehicle by an audible sound. , a second notification unit that notifies the occupant using visible light, and when the proximity detection unit detects the approach of the notification object, based on a combination of detection results of the direction detection unit and the motion detection unit; a notification-related determination unit that determines a notification mode including specifying a notification unit to be used among the first notification unit and the second notification unit, and a notification timing by the notification unit to be used; a notification control unit configured to notify the approach of the notification object at the notification timing by a notification unit to be used among the first notification unit and the second notification unit, based on a determination result of the determination unit; Examples include vehicle control systems.

In the vehicle control system, the notification-related determination unit may cause the operation detection unit to determine, as the predetermined operation, an operation in which the occupant attempts to open a door of the vehicle when the approach of the notification target is detected. is detected, and when the direction detection section detects that the first part of the occupant is facing the door, both the first notification section and the second notification section are set as the notification mode. In addition to determining the usage target, simultaneous notification by the first notification unit and the second notification unit may be determined as the notification timing.

In the vehicle control system, the notification-related determination unit may cause the operation detection unit to cause the occupant to open a door of the vehicle as the predetermined operation when the approach detection unit detects the approach of the notification target. When an attempt to open the door is detected, and the direction detection unit detects that the first part of the occupant is not facing the door, the first notification unit and the first part of the door are detected as the notification mode. Both of the two notification sections may be determined to be used, and the notification timing may be determined to be the timing at which the second notification section notifies after the first notification section has notified.

In the vehicle control system, the notification control unit may control the notification control unit to control the direction of the passenger when the direction detection unit detects that the first member of the occupant is facing the door after the first notification unit has notified. The notification may be made using the 2 notification section.

The vehicle control system includes a door opening/closing detection unit that detects an open/closed state of a door of the vehicle, and a door control unit that controls the door, and the notification related determination unit detects the approach of the notification target object. when the door opening/closing detection section detects that the door is in an open state, and the direction detection section detects that the first part of the occupant is facing the door, the notification mode , it is determined that both the first notification unit and the second notification unit are to be used and the door is controlled to be in a closed state, and the notification control unit is configured to: The approach of the object to be notified may be notified by both the first notification unit and the second notification unit, and the door may be controlled to be closed by the door control unit.

In the above vehicle control system, the notification-related determination unit 16 is configured to determine whether the predetermined movement of the occupant is not detected by the movement detection unit and the direction detection unit detects the presence of the occupant when the approach of the notification target object is detected. If it is detected that the first part of the occupant is facing the door, the second notification part is determined to be used as the notification mode, and the period during which the first part of the passenger is facing the door is determined as the notification timing. may be determined.

The vehicle control system includes a door opening/closing detection unit that detects an open/closed state of a door of the vehicle, and a door control unit that controls the door, and the notification related determination unit detects the approach of the notification target object. when the door opening/closing detection section detects that the door is in an open state, and the direction detection section detects that the first part of the occupant is facing the door, the notification mode , it is determined that both the first notification unit and the second notification unit are to be used and the door is controlled to be in a closed state, and the notification control unit is configured to: The approach of the object to be notified may be notified by both the first notification unit and the second notification unit, and the door may be controlled to be closed by the door control unit.

A second aspect for achieving the above object includes an approach detection unit that detects the approach of a notification target, a first notification unit that notifies the occupant by an audible sound, and a first notification unit that notifies the occupant by using visible light. A vehicle control method executed by a computer to control the operation of a vehicle comprising a second notification section, the method comprising: a second notification section; a direction detection unit that detects the orientation of a first part that can identify the direction of the line of sight or the face; and a movement detection unit that detects a predetermined movement of the occupant regarding getting off the vehicle based on a movement of a second part of the occupant that is different from the first member. A notification that determines a notification mode including specifying a notification unit to be used among the first notification unit and the second notification unit, and a notification timing by the notification unit to be used, based on a combination of detection results. Based on the determination result of the association determination step and the notification association determination step, a notification unit to be used of the first notification unit and the second notification unit notifies the approach of the notification target object at the notification timing. A vehicle control method includes a notification control step of:

According to the present invention, it becomes easier to give appropriate notifications tailored to the occupant's condition.

FIG. 1 is a configuration diagram of a vehicle equipped with a vehicle control system. FIG. 2 is a view of the door on the right side of the vehicle body, viewed from the passenger compartment side. FIG. 3 is a configuration diagram of the vehicle control device. FIG. 4 is a diagram showing an example of a notification determination table. FIG. 5 is a flowchart showing notification processing when the door is closed. FIG. 6 is a diagram for explaining the notification process in FIG. 5. FIG. 7 is a diagram for explaining the notification process in FIG. 5. FIG. 8 is a diagram for explaining the notification process in FIG. 5. FIG. 10 is a diagram for explaining the notification process in FIG. 5. FIG. 5 is a flowchart showing notification processing when the door is open. FIG. 11 is a diagram for explaining the notification process in FIG. 10. FIG. 12 is a diagram for explaining the notification process in FIG. 10. FIG. 13 is a diagram for explaining the notification process in FIG. 10.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a diagram showing the configuration of a vehicle equipped with a vehicle control system according to an embodiment of the present invention. This vehicle control system 100 is composed of a vehicle control device 10 provided in the vehicle 1, and sensors and control target parts provided in the vehicle 1 and electrically connected to the vehicle control device 20, which will be described later. .

[1. Vehicle configuration]
With reference to FIG. 1, the configuration of a vehicle 1 equipped with a vehicle control device 10 of this embodiment will be described. As shown in FIG. 1, the vehicle 1 is a passenger car with a seating capacity of five people, including a driver's seat 7a, a passenger seat 7b, a second row right seat 7c, a second row center seat 7d, and a second row left seat 7e. We are prepared. Each seat 7a to 7e is provided with a seatbelt switch 74a to 74e (not shown) that detects whether or not a seatbelt is worn.

The vehicle 1 includes a right power hinge door 2, a left power hinge door 3, a right power sliding door 4, a left power sliding door 5, and a power tailgate 6. The right power hinge door 2, the left power hinge door 3, the right power sliding door 4, the left power sliding door 5, and the power tailgate 6 correspond to the doors of the present invention. A door touch sensor 101 is provided on the doorknob of the right power hinge door 2, and the user U1 who has the mobile key 200 of the vehicle 1 can unlock the doors 2 to 6 of the vehicle 1 by touching the door touch sensor 101. Can be locked.
FIG. 1 illustrates a case where a user U2 rides in the passenger seat 7b of the vehicle 1, and a user U3 rides in the second row left seat 7f.

The right power hinge door 2 includes a right PHD (Power Hing Door) drive unit 41 that drives the right power hinge door 2 to open and close using an electric actuator (not shown), and the left power hinge door 3 drives the left power hinge door 3 to open and close using an electric actuator. The left PHD drive section 51 is provided to perform the following. The right power sliding door 4 includes a right PSD (Power Slide Door) drive unit 42 that drives the right power sliding door 4 to open and close using an electric actuator, and the left power sliding door 5 drives the left power sliding door 5 to open and close using an electric actuator. The left PSD drive section 52 is equipped with a left PSD drive section 52 that performs. The power tailgate 6 includes a PTG (Power Tail Gate) drive unit 61 that drives the power tailgate 6 to open and close using an electric actuator.

A front camera 30 for photographing the front of the vehicle 1 is provided at the front of the vehicle 1, and a rear camera 60 for photographing the rear of the vehicle 1 is provided at the rear of the vehicle 1. A right side camera 40 for photographing the right side of the vehicle 1 is provided on the right side of the vehicle 1, and a left side camera 50 for photographing the left side of the vehicle 1 is provided on the left side of the vehicle 1. These cameras 30 to 60 can capture images of people around the vehicle 1. Note that the user of the vehicle 1 includes a wide range of people who use the vehicle 1, and is not limited to the owner of the vehicle 1, but includes the owner's family, acquaintances, and the like. In the following description, when it is necessary to distinguish between the user of the vehicle 1 whether the user is inside the vehicle or outside the vehicle, the user inside the vehicle will be referred to as an "in-vehicle user."

A front seat camera 70 capable of photographing in-vehicle users seated in a driver's seat 7a and a passenger seat 7b and a display 31 are provided on the dashboard of the vehicle interior. As the display 31, a wide variety of known display devices such as a liquid crystal display can be used. A rear seat camera 71 is provided on the ceiling of the vehicle interior and is capable of photographing users in the vehicle seated in the second row right seat 7c, the second row center seat 7d, and the second row left seat 7e.
Each of the doors 2 to 5 is provided with door switches 75a to 75d that detect whether the door is opened or closed. The power tailgate 6 is provided with a door switch 75e that detects whether the power tailgate 6 is opened or closed. Further, near the driver's seat 7a, a power switch 72 for instructing on/off of the battery 150 and a shift switch 73a for detecting the shift position of the shift lever 73 are provided. When the battery 150 is on, power is supplied from the battery 150 to the vehicle drive system, and when the battery 150 is off, the power supply from the battery 150 to the vehicle drive system is cut off.

Furthermore, a front right speaker 32a and a front right light 33a are provided in the area outside the driver's seat including the right power hinge door 2, and a front left speaker 32b and a front left light 33b are provided in the area outside the passenger seat including the left power hinge door 3. is provided. Further, a rear right speaker 32c and a rear right light 33c are provided in the outer area of the right rear seat including the right power sliding door 4, and a rear left speaker 32d and a rear left light 33d are provided in the left power sliding door 5. ing. The speakers 32a to 32d are arranged at each of the doors 2 to 5 toward the interior of the vehicle, and function as an audible sound unit that outputs audible sound from each of the doors 2 to 5 toward the interior of the vehicle. Note that the positions and numbers of the speakers 32a to 32d can be changed as appropriate within the range in which audible sounds can be output from the doors 2 to 5 into the vehicle interior.

The front right speaker 32a is located on the right side of the front seat, so if this speaker 32a were used alone to issue audible warnings to the users inside the car, it would cause the users inside the car to pay attention to the right power-hinged door 2 side, or the nearest driver It becomes easier to give a warning to the user in the car seat 7a. Since the front left speaker 32b is located on the left side of the front seat, if this speaker 32b alone gives audible warning to the user in the car, the user in the car will be alerted to the left power hinge door 3 side, and the user in the passenger seat 7b will be alerted to the user in the car. It becomes easier to issue warnings.
Since the rear right speaker 32c is located on the right side of the rear seat, if this speaker 32c alone gives audible warnings to the users in the car, it may cause the users in the car to pay attention to the right side of the rear seat, or the right side of the rear seat (second row right seat). It becomes easier to issue a warning to the in-vehicle user in step 7c). The rear left speaker 32d is located on the left side of the rear seat, so if this speaker 32d alone gives audible warnings to the users in the car, it may cause the users in the car to pay attention to the left side of the rear seat, or the left side of the rear seat (second row left seat). 7e) It becomes easier to give a warning to the in-vehicle user.

Each of the lights 33a to 33d is placed in a position where it can be seen by the occupants who turn their eyes or faces toward each of the doors 2 to 5, and functions as a visible light unit that outputs visible light from each of the doors 2 to 5 toward the interior of the vehicle. .
More specifically, since the front right light 33a is located on the right side of the front seat, when the user in the vehicle turns his or her eyes or faces towards the right power hinge door 2 on the right side of the front seat, the user in the vehicle turns on the front right light 33a. can be noticed. Since the front left light 33b is located on the left side of the front seat, when the in-vehicle user turns his/her line of sight or face towards the left power hinge door 3 side on the left side of the front seat, the in-vehicle user can notice that the front left light 33b is turned on.

Similarly, when the in-vehicle user turns his/her line of sight or head toward the right power sliding door 4 on the right side of the rear seat, the in-vehicle user can notice that the rear right light 33c is lit. M. Furthermore, when the user in the vehicle turns his/her line of sight or head toward the left power sliding door 5 on the left side of the rear seat, the user in the vehicle can notice that the rear left light 33d is turned on.
Note that the position, number, and structure of each of the lights 33a to 32d can be changed as appropriate within the range in which visible light can be output from each door 2 to 5 toward the interior of the vehicle.
In addition, when there is no need to specifically distinguish and write each of the lights 33a to 32d, for convenience of explanation, they will be written as door side lights 33a to 32d.

FIG. 2 is a view of the right side doors (right power hinge door 2, right power sliding door 4) of the vehicle body as viewed from the passenger compartment side. In FIG. 2, reference numeral 104 is a front pillar, and reference numeral 106 is a center pillar.
Door trims 111 and 121, which are interior materials, are attached to the cabin side of each door 2 and 4. The door trims 111 and 121 are resin molded parts, and in order from the top, door trim uppers 112 and 122, door trim lower 113, 123, and armrests 114, 124 are provided.
The door trim upper 112 of the right power hinge door 2 is provided with a door inner handle 115 that is operated when opening and closing the door, a switch section 116 for opening and closing the door glass, and a front right light 33a. The front right light 33a is formed into a sequential lamp that has a plurality of light sources (for example, LEDs) arranged one behind the other, extends in a line in the front-rear direction, and can move the lighting area in the front-rear direction when turned on.
A front right speaker 32a is provided at the front of the door trim lower 113, and a door pocket 118 is provided behind the front right speaker 32a.

The door trim upper 122 of the right power sliding door 4 is provided with a door inner handle 125 that is operated when opening and closing the door, and a switch section 126 for opening and closing the door glass. A rear right light 33c is provided in the body portion above the right power sliding door 4. Similarly to the front right light 33a, the rear right light 33c is also a sequential lamp in which a plurality of light sources (for example, LEDs) are arranged one after the other and extends in a line in the front and back direction, and the lighting area can be moved in the front and back direction when turned on. is formed. Since the rear right light 33c is provided in the body portion, it does not move together with the right power sliding door 4. Note that when the rear seat door is a hinged door, a rear right light 33c may be provided on the door trim upper 121, as shown by the two-dot chain line in FIG.
A front right speaker 32a is provided at the front of the door trim lower 123, and a door pocket (not shown) is provided behind the front right speaker 32a.
Note that the doors on the left side of the vehicle body (left power hinge door 3, left power sliding door 5) and their surrounding structure are symmetrical with the doors on the right side of the vehicle body (right power hinge door 2, right power sliding door 4) and their surrounding structure. It is.

As shown in FIG. 1, the vehicle 1 further includes door lock mechanisms 100a to 100d that lock the doors 2 to 5 and the power tailgate 6, a speed sensor 110 that detects the traveling speed of the vehicle 1, and a mobile key 200. , a communication unit 120 capable of communicating with user U1's mobile terminal 210, etc., and a navigation device 130. The navigation device 130 has a GPS (Global Positioning System) sensor (not shown) and map data, and executes route guidance to a destination, etc. based on the position of the vehicle 1 detected by the GPS sensor and the map data.

[2. Configuration of vehicle control device]
The configuration of the vehicle control device 10 will be described with reference to FIG. 3. The vehicle control device 10 is an ECU (Electronic Control Unit) that is configured of a CPU (Central Processing Unit, which corresponds to a computer according to the present invention), a memory 80, an interface circuit (not shown), etc., and controls the operation of the vehicle 1. be.

Images of the surroundings of the vehicle 1 taken by the front camera 30, right side camera 40, left side camera 50, and rear camera 60 are input to the vehicle control device 10. Furthermore, images of the interior of the vehicle captured by the front seat camera 70 and the rear seat camera 71 are input to the vehicle control device 10 .

Furthermore, the vehicle control device 10 includes detection signals of a power switch 72, a shift switch 73a, seat belt switches 74a to 74e, and door switches 75a to 75d, and door lock sensors (not shown) provided in the door lock mechanisms 100a to 100d. ) lock detection signals for each of the doors 2 to 5 and the power tailgate 6 are input.

Additionally, a touch detection signal from the door touch sensor 101, a speed detection signal from the speed sensor 110, and information on the current position of the vehicle 1 detected by the navigation device 130 are input to the vehicle control device 10.

Control signals output from the vehicle control device 10 control the right PHD drive section 41, the right PSD drive section 42, the left PHD drive section 51, the left PSD drive section 52, the PTG drive section 61, the door lock mechanisms 100a to 100d, and the battery. The operation of 150 is controlled. Furthermore, the screen display on the display 31 and the sounds output from each of the speakers 32a to 32d (voice guidance, chime sound, notification sound such as alarm sound) are controlled by the control signal output from the vehicle control device 10. Ru. The vehicle control device 10 communicates with the mobile key 200 and the mobile terminal 210 owned by the user U1 via the communication unit 120.

By reading and executing the control program 81 of the vehicle control device 10 stored in the memory 80, the CPU operates the key authentication section 11, the vehicle interior situation recognition section 12, the vehicle surrounding situation recognition section 13, the door opening/closing detection section 14, It functions as a notification section 15, a notification related determination section 16, a door control section 17, and a notification control section 18. The control program 81 includes a vehicle control program of the present invention. The memory 80 stores an authentication ID code 82 used for authentication of the mobile key 200, a notification determination table 83 used for notification processing, notification information 84, and the like.

The authentication ID code 82 is a code that identifies the mobile key 200 linked to this vehicle 1. The notification decision table 83 is table data for making decisions regarding notification of notification objects. The notification information 84 includes data that defines the volume and audio content of each speaker 32a to 32d, the amount of light, lighting pattern, color, etc. of the door side lights 33a to 32d for each notification process. The data corresponding to the notification determination table 83 and the notification information 84 may be written in the control program 81, or may be obtained from an external device via a communication network.

As shown in FIG. 1, when the user U1 in possession of the mobile key 200 approaches the vehicle 1, the key authentication unit 11 communicates with the mobile key 200 via the communication unit 120 to authenticate the mobile key 200. An ID code is received from the key 200. Then, when the received ID code and the authentication ID code 82 stored in the memory 80 match, the key authentication unit 11 permits the user U1 to use the vehicle 1.

As shown in FIG. 1, the portable key 200 includes a lock/unlock button 201 for instructing locking and unlocking of the vehicle 1, a right power hinge door button 202 for instructing opening/closing of the right power hinge door 2, and a left power hinge button 202 for instructing opening/closing of the right power hinge door 2. Left power hinge door button 203 for instructing opening/closing of hinge door 3, right power sliding door button 204 for instructing opening/closing of right power sliding door 4, left power sliding door for instructing opening/closing of left power sliding door 5. A button 205 and a power tailgate button 206 for instructing opening/closing of the power tailgate 6 are provided. Hereinafter, the right power hinge door 2, the left power hinge door 3, the right power sliding door 4, the left power sliding door 5, and the power tailgate 6 will also be collectively referred to as electric doors.

The user U1 can operate the lock/unlock button 201 of the portable key 200 to unlock each door 2 to 6 of the vehicle 1 and get into the vehicle 1. Further, the user U1 can instruct the vehicle 1 to open and close the electric door by operating the buttons 202 to 206 of the portable key 200. A user U1 who has boarded the vehicle 1 with the portable key 200 turns on the power switch 72 to start the vehicle 1 and starts using the vehicle 1.

The in-vehicle situation recognition unit 12 recognizes the situation in the vehicle 1 based on images captured by an in-vehicle camera including a front seat camera 70 and a rear seat camera 71. The in-vehicle situation recognition section 12 includes an orientation detection section 12A, a movement detection section 12B, and a locked state detection section 12C.
The orientation detection unit 12A detects the orientation of a predetermined first part of the user in the vehicle from images taken by cameras 70 and 71 in the vehicle. As the first part, a part that can specify the line of sight or the direction of the face of the user in the vehicle is applied, for example, an appropriate part in the eyes, face, or head is applied. The orientation detection unit 12A detects the orientation of the first part of all the in-vehicle users, so that the line of sight or the face of any in-vehicle user is located at the door (for example, the door inner handle) that the in-vehicle user uses when exiting the vehicle. Detects whether or not the person is facing the direction.

More specifically, the direction detection unit 12A detects whether the line of sight or face of the in-vehicle user seated in the driver's seat 7a is directed toward the right power hinge door 2, and detects whether or not the in-vehicle user seated in the passenger seat 7b , detect whether the line of sight or the face is directed toward the left power hinge door 3. Further, the direction detection unit 12A detects whether the line of sight or face of the in-vehicle user seated in each of the second row seats 7c to 7e is directed toward either the door 4 or 5. Note that for the in-vehicle user seated in the second row right seat 7c, only whether the line of sight or face is directed toward the right power sliding door 4 is detected, and for the in-vehicle user seated in the second row left seat 7e, Alternatively, only whether the line of sight or the face is directed toward the left power sliding door 5 may be detected.

The motion detection unit 12B detects the motion of a predetermined second region of the user in the vehicle, which is different from the first region, from the captured images of the cameras 70 and 71 in the vehicle, thereby determining whether the user in the vehicle has performed a predetermined motion related to getting out of the vehicle. Detect. This predetermined action is an action that can be considered as the start of getting off the vehicle, and in this embodiment, it is an action to open the door. In order to detect this movement of trying to open the door, the second part may be a hand, an arm, another part of the upper body, or the entire upper body. The following is an example of a case where the following is adopted. Note that the predetermined motion may be any motion other than the motion of attempting to open the door, and in short, any motion that can be regarded as the start of getting off the vehicle by detecting the motion of any part other than the first part can be widely applied.
The motion detection unit 12B can detect the motions of the second parts of all the in-vehicle users, and can detect whether any of the in-vehicle users has attempted to open the door.

More specifically, the motion detection unit 12B detects whether or not the hand of an in-vehicle user sitting in the driver's seat 7a has moved to the right power hinge door 2, and detects whether or not the in-vehicle user sitting in the passenger seat 7b has moved his or her hand to the right power hinge door 2. It is detected whether the hand has moved to the left power hinge door 3. Furthermore, the motion detection unit 12B detects whether or not the hand of the in-vehicle user seated in each of the second row seats 7c to 7e moves toward either the door 4 or 5.
Since the movement of the hand that moves within the body is wide and is easily photographed by the in-vehicle cameras 70 and 71, the movement of the hand trying to open the door is detected, so the detection accuracy of the movement of trying to open the door can be maintained sufficiently. It becomes easier.

A known image recognition technique may be applied to the detection by the orientation detection section 12A and motion detection section 12B. Note that the present invention is not limited to the case where the image recognition technique is applied, and other techniques capable of detecting the movement of the first part or the second part of the user in the vehicle may be applied.
The locked state detection unit 12C detects whether each of the doors 2 to 5, etc. is in a locked state by acquiring the operating states of the door lock mechanisms 100a to 100e.

The vehicle surrounding situation recognition unit 13 recognizes the surrounding situation of the vehicle 1 based on images captured by an external camera including a front camera 30, a right side camera 40, a left side camera 50, and a rear camera 60. The vehicle surrounding situation recognition unit 13 includes an approach detection unit 13A that detects whether a notification target is approaching based on a photographed image. Objects to be notified include moving objects and stationary objects that may obstruct the user in the vehicle from exiting the vehicle. The moving objects are, for example, four-wheeled vehicles, two-wheeled vehicles, and pedestrians approaching the vehicle 1, and the stationary objects are poles, walls, and objects parked next to the doors 2 to 5 of the vehicle 1. vehicles etc.

A known image recognition technique may also be applied to the detection by the approach detection section 13A. Note that the notification target object is not limited to the case where the image recognition technology is applied, and other known devices such as millimeter wave radar and sonar may be used to detect the notification target.

The door open/close detector 14 detects the open/close states of the doors 2 to 6 based on the operating states of the door switches 75a to 75d. The door control section 17 controls each of the right PHD drive section 41, the right PSD drive section 42, the left PHD drive section 51, the left PSD drive section 52, the PTG drive section 61, and the door lock mechanisms 100a to 100e. Controls operating parts related to doors 2-6.

The notification section 15 includes a first notification section 15A that notifies the in-vehicle user using audible sound, and a second notification section 15B that notifies the in-vehicle user using visible light. The first notification unit 15A emits various audible sounds into the vehicle by using the speakers 32a to 32d. Further, the second notification section 15B emits visible light toward the interior of the vehicle by using the door side lights 33a to 32d. Note that the second notification unit 15B may further utilize the display 31 to emit image light made of visible light.

The notification-related determination unit 16 performs a process of determining matters related to the notification when notifying the approach of the notification target detected by the approach detection unit 13A, and is responsible for determining matters related to the notification, as well as the notification mode determination unit 16A and timing determination. 16B. The notification mode determining unit 16A determines the notification mode when notifying the approach of a notification target. Regarding the notification mode, matters other than the notification timing are determined, and in this embodiment, it indicates which of the first notification unit 15A and the second notification unit 15B is to be used and the magnitude of the notification intensity. Determine notification level and presence or absence of door control. Note that the higher the notification level, the higher the notification intensity (corresponding to the degree of warning).
The timing determination unit 16B performs a process of determining the notification timing for notifying the in-vehicle user of the approach of the notification target. For example, when using the first notification unit 15A and the second notification unit 15B, the timing determination unit 16B 15B is determined.

FIG. 4 shows an example of the notification determination table 83.
As shown in FIG. 4, the notification determination table 83 includes the orientation of the first part (for example, the eyes) detected by the orientation detection unit 12A, and the motion of the second part (for example, the hand) detected by the motion detection unit 12B. , notification mode, and notification timing are described in association with each other. Thereby, the notification mode and notification timing are specified according to the combination of the orientation of the first part and the motion of the second part.
This notification determination table 83 associates the orientation of the first part, the movement of the second part, the notification mode, and the notification timing for each case when the door is in a closed state and when the door is in an open state. It is described as follows. In other words, even if the combination of the orientation of the first part and the motion of the second part is the same, different notification modes and notification timings are determined depending on whether the door is open or closed.

When the door is open, the user inside the car is more likely to jump out of the car than when the door is closed. Therefore, in the notification determination table 83 of FIG. 4, the notification level when the door is closed is set to LEVEL 1 to 3. However, the notification level when the door is open is set to LEVEL 4 to 6.
The first notification unit 15A performs notifications that match the notification level by changing the volume of the audible sound, audio content, etc., or increasing or decreasing the number of speakers that output the audible sound, depending on the notification level. Further, the second notification unit 15B performs notification that matches the notification level by changing the visible light lighting pattern, light amount, light color, etc. according to the notification level. The audible sound or visible light depending on the notification level may be set as appropriate. Further, when the door is in an open state, door control is set at a predetermined notification level. In FIG. 4, when the notification level is LEVEL6, door closing control is described as door control.

The door control section 17 controls each of the right PHD drive section 41, the right PSD drive section 42, the left PHD drive section 51, the left PSD drive section 52, the PTG drive section 61, and the door lock mechanisms 100a to 100e. Controls operating parts related to doors 2-6. The notification control unit 18 performs a notification process of notifying various information to in-vehicle users and the like by controlling the display 31, the speaker 32, the communication unit 1202, and the like.

The notification processing performed by the notification control unit 18 includes "notification processing when the door is closed" which notifies the in-vehicle user of the approach of a notification target when the door near the in-vehicle user is closed; There is a "notification process with the door open" which notifies the user in the vehicle of the approach of the object to be notified when the door of the vehicle is open. These notification processes are automatically executed when the vehicle 1 is in a parked state (for example, the shift position is in the parking position).

[3. Notification processing when door is closed]
FIG. 5 is a flowchart showing notification processing when the door is closed. FIGS. 6 to 9 are diagrams for explaining this notification processing.
First, the notification control unit 18 uses the vehicle surrounding situation recognition unit 13 to determine whether or not a notification target is approaching (step S1A: detection step), and if it is determined that the notification target is not approaching (step S1A). ;NO), the notification process is temporarily terminated. Note that even if the notification process is finished, if the door near the user in the vehicle is closed and the vehicle 1 is parked, the notification process is repeatedly executed at a predetermined cycle.

If it is determined that the object to be notified is approaching (step S1A; YES), the notification control unit 18 determines that the orientation detection unit 12A causes the first part of the user in the vehicle to face the door, and the motion detection unit 12B determines that the second part of the user faces the door. It is determined whether or not the vehicle is in a first situation in which the vehicle is not moving (step S2A: first situation detection step).
In FIG. 6, in a situation where a moving object M1, which is a notification target, is approaching from the rear left of the vehicle 1, in-vehicle users U1 to U3 are seated in the driver's seat 7a, the passenger seat 7b, and the second row left seat 7e, respectively. A case is shown in which a user U2 inside the vehicle turns his face toward a nearby door 3 after getting into the vehicle. In FIG. 6, arrows indicate the directions of the faces of the in-vehicle users U1 to U3. In this case, the determination in step S2A is affirmative (step S2A; YES), and the notification control unit 18 causes the notification related determination unit 16 to determine the notification mode and notification timing based on the notification determination table 83 (see FIG. 4). , as shown in FIG. 5, the LEVEL1 notification process is performed using the sound from the first notification unit 15A (step S3A). Note that, as shown in FIG. 4, default timing is determined as the notification timing. The default timing is set to a period during which the first part of the user inside the vehicle is facing the door.

As a result, as shown in FIG. 6, as a notification process of LEVEL1, the speaker 32b provided on the door 3 side toward which the in-vehicle user U2 faces, and the speaker 32b on the left side of the vehicle, which is the side where the mobile object M1 approaches, are activated. A process is executed in which the speaker 32d outputs a notification sound notifying the approach of the mobile object M1, which is the notification target, and the other speakers 32a and 32c do not output notification sounds. Since the notification sound is output from the speaker 32b, the in-vehicle user U2 who may open the door 3 and exit the vehicle can be effectively notified of the approach of the moving object. Furthermore, since the notification sound is output from the other speaker 33d on the side where the moving object M1 approaches, it is possible to effectively notify the other user U3, who is located on the same left side as the user U2 in the vehicle, of the approach of the moving object.
Furthermore, the notification timing is set during the period in which the first part of the in-vehicle user U2 is facing the door, so when the first part of the in-vehicle user U2 is no longer facing the door 3, the notification process of LEVEL1 is stopped.

FIG. 7 shows a case where an in-vehicle user U2 who is seated near the door 3 turns his face toward the door 3 in a situation where the stationary body T1, which is the object to be notified, is located within the opening/closing range of the door 3. In this case, as a notification process of LEVEL 1, a notification sound notifying the approach of the notification object is output from the speaker 32b provided on the door 3 side toward which the in-vehicle user U2 faces, and a notification sound is output from the other speakers 33a, 33c, and 33d. does not output notification sound. As a result, the presence of the stationary body T1 is notified only to the in-vehicle user U2 whose stationary body T1 influences the opening and closing of the door. Note that the contents of the notification process of LEVEL1 may be changed as appropriate.

Returning to FIG. 5, if the situation is not the first situation (step S2A; NO), the notification control unit 18 determines that the orientation detection unit 12A determines that the first part of the user in the vehicle is not facing the door, and the motion detection unit 12B determines that the first part of the user in the vehicle is not facing the door. It is determined whether the second situation is such that the object moves to the door (step S4A).

Reference numeral A in FIG. 8 shows an in-vehicle user U1 riding in the driver's seat 7a, passenger seat 7b, and second row left seat 7e in a situation where a moving object M1, which is a notification target, is approaching from the left rear of the vehicle 1. - U3 all show the case where the user U2 in the car approaches the door 3 with his or her face facing forward. In this case, the determination in step S4A is a positive result (step S4A; YES), and the notification control unit 18 causes the notification related determination unit 16 to determine the notification mode and notification timing based on the notification determination table 83 (see FIG. 4). , as shown by symbols A and B in FIG. 8, notification processing of LEVEL 2 is performed using sound and light by the first notification unit 15A and the second notification unit 15B (step S5A). Note that, as shown in FIG. 4, the notification timing is determined to be performed by the second notification unit 15B after the notification by the first notification unit 15A.

As a result, as indicated by reference numeral A in FIG. 8, as a notification process of LEVEL 2, the notification target is first transmitted from the speaker 32b provided on the door 3 side and the other speaker 33d on the door 5 side to which the mobile object M1 approaches. A process is executed in which a notification sound notifying the approach of a certain mobile object M1 is output, and the notification sound is not output from the other speakers 32a and 32c. Thereafter, as shown in the symbol B in FIG. 8, as a notification process of LEVEL 2, the approach of the moving body M1 is detected from the front left light 33b provided on the door 3 side and the rear left light 33d on the door 5 side where the moving body M1 approaches. The process of outputting light to notify the user is executed.

Here, the timing at which the second notification section 15B notifies may be the timing at which the direction detection section 12A detects that the first part of the user inside the vehicle is facing the door, or the timing at which the first notification section 15A notifies the user. The timing may be set at a time when a predetermined time period has elapsed.

Therefore, after guiding the in-vehicle user U2 who has brought his hand close to the door 3 to turn his face toward the door 3 by the sound from the speaker 32b, the light from the front left light 33b is also used to notify the in-vehicle user U2 of the approach of the moving object M1, It becomes easier for the in-vehicle user U2 to notice the approach of the mobile object M1 by both the sound and the light. Furthermore, after the user U3 in the vehicle on the side where the mobile object M1 approaches is guided to turn his face toward the door 5 by the sound from the speaker 33d, the rear left light 33d also notifies the user U3 of the approach of the mobile object M1. , it becomes easier to notice the approach of the moving object M1.

8, if the object to be notified is not the moving object M1 but the stationary object T1 shown in FIG. As a notification process, a process is performed in which a speaker 32b provided on the door 3 side is used for notification, and then a front left light 33b provided on the door 3 side is used for notification. Note that when the door side lights 33a to 32d are used to notify the approach of a notification target, it is preferable to control the lighting so as to indicate from which direction the notification target approaches. For example, if the notification target approaches from the front, the lighting area may be moved from the front to the rear, and if the notification target approaches from the rear, the lighting area may be moved from the back to the front.
In addition, in the case of LEVEL 2 notification processing, compared to LEVEL 1 notification processing, in addition to using the door side lights 33a to 32d, at least one of the volume or audio content is changed so that the user in the car can more easily notice it. is preferred.

Returning to FIG. 5, if the situation is not the second situation (step S4A; NO), the notification control unit 18 determines that the orientation detection unit 12A causes the first part of the user in the vehicle to face the door, and the motion detection unit 12B causes the second part of the user to face the door. It is determined whether or not the third situation is movement (step S6A).
In FIG. 9, in a situation where a mobile object M1, which is a notification target, is approaching from the left rear of the vehicle 1, an in-vehicle user U2 in the passenger seat 7b turns his face toward the nearby door 3, and the in-vehicle user U2 A case is shown in which the hand approaches door 3. In this case, the determination in step S6A is a positive result (step S6A; YES), and the notification control unit 18 causes the notification related determination unit 16 to determine the notification mode and notification timing based on the notification determination table 83 (see FIG. 4). As shown in FIG. 9, notification processing of LEVEL 3 is performed using sound and light by the first notification unit 15A and the second notification unit 15B (step S7A). Note that, as shown in FIG. 4, the notification timing is determined to be simultaneous notification by the first notification unit 15A and the second notification unit 15B.

As a result, as shown in FIG. 9, as a notification process of LEVEL 3, a notification is sent from the speaker 32b provided on the door 3 side and the other speaker 33d on the door 5 side to which the moving object M1 approaches, notifying the approach of the moving object M1. In addition to outputting the sound, a process is executed to output a notification sound notifying the approach of the mobile object M1 from the front left light 33b provided on the door 3 side and the rear left light 33d on the door 5 side toward which the mobile object M1 approaches. Ru.
The third situation in which the first part of the in-vehicle user faces the door and the second part moves toward the door by the motion detection unit 12B is assumed to be a situation in which there is an extremely high possibility that the in-vehicle user opens the door and moves out of the vehicle. Ru. In this case, the notification is sent from among the speakers 32a to 32d on the door side that the user inside the car is viewing, and the notification is simultaneously sent from the door side light 33a to 32d that is on the door side that the user inside the car is viewing. This makes it easier for the in-vehicle user to notice the approach of the notification target.

In FIG. 9, if the notification target object is not the moving object M1 but the stationary object T1 shown in FIG. 7, it is only necessary to notify the user U2 in the passenger seat 7b of the stationary object T1, so the , the speaker 32b provided on the door 3 side provides notification, and the front left light 33b provided on the door 3 side simultaneously provides notification.
In the case of the LEVEL 3 notification process, it is preferable to change at least one of the volume, audio content, light intensity, lighting pattern, or color so that the in-vehicle user can more easily notice the notification process compared to the LEVEL 2 notification process. However, regarding the lighting pattern, it is preferable to control the lighting to indicate which direction the notification target is approaching from, so while performing the lighting control, for example, the speed at which the lighting area is moved in the front and back direction may be increased. All you have to do is make the following changes.
The process of determining the notification mode and notification timing in steps S3A, S5A, and S7A above corresponds to the notification related determination step of the present invention, and the notification process in steps S3A, S5A, and S7A corresponds to the notification control step of the present invention. Equivalent to.

[4. Notification processing when door is open]
FIG. 10 is a flowchart showing the notification process when the door is open. Note that steps similar to those in the notification process shown in FIG. 5 are shown with the same numbers, and redundant explanation will be omitted. FIGS. 11 to 13 are diagrams for explaining this notification processing.
The notification control unit 18 determines that the object to be notified is approaching by the vehicle surrounding situation recognition unit 13 (step S1A; YES), and the direction detection unit 12A determines that the first part of the user in the vehicle is facing the door, and the motion detection unit 12B, it is determined whether the first situation is such that the second part has not moved to the door (step S2A; YES).

FIG. 11 shows a case where, when the door 3 is open, a moving object M1, which is a notification target, approaches the vehicle 1 from the rear left, and the user U2 in the vehicle turns his face toward the nearby door 3. . In this case, the determination in step S2A is affirmative (step S2A; YES), and the notification control unit 18 causes the notification related determination unit 16 to determine the notification mode and notification timing based on the notification determination table 83 (see FIG. 4). , LEVEL 4 notification processing is performed using the sound from the first notification unit 15A (step S3B). Note that, as shown in FIG. 4, default timing is determined as the notification timing.

In this case, the notification process is executed in such a way that the in-vehicle user 2 notices it more easily than the notification process such as LEVEL1. In this embodiment, as shown in FIG. 11, as a notification process of LEVEL 4, all the speakers 32a to 32d including the speaker 32b provided on the side of the door 3 toward which the in-vehicle user U2 faces move A process of outputting a notification sound notifying the approach of the body M1 is executed (step S3B). Furthermore, it is preferable that the volume of the notification sound and the audio content be set so that the in-vehicle user 2 can more easily notice the notification sound.
Thereby, when the in-vehicle user turns his face toward the door 3 in the open state, it is possible to make the in-vehicle user more easily notice the object to be notified than when the in-vehicle user turns his face toward the closed door 3.

In addition, in the case of FIG. 11, the case where the approach of the mobile object M1 is notified from all the speakers 32a to 32d is illustrated, but the point is that the in-vehicle user 2 is more likely to notice or the degree of vigilance is increased than in the notification process of LEVEL1. Any notification processing that does this may be sufficient. For example, the notification may be made with a loud notification sound only from the speakers 32a and 32d on the side where the mobile object M1 approaches.
In FIG. 11, if the object to be notified is not the moving object M1 but the stationary object T1 shown in FIG. 7, it is only necessary to notify the in-vehicle user U2 of the passenger seat 7b of the stationary object T1. In this case as well, notification processing that makes it easier for the in-vehicle user 2 to notice or increases the level of vigilance than notification processing such as LEVEL 1 can be widely applied.

Returning to FIG. 10, if the situation is not the first situation (step S2A; NO), the notification control unit 18 determines that the first part of the user in the vehicle is not facing the door according to the orientation detection unit 12A, and that the second part of the user is not facing the door according to the motion detection unit 12B. It is determined whether the second situation is such that the object moves to the door (step S4A).
12, when the door 3 is open, a mobile object M1, which is a notification target, approaches the vehicle 1 from the left rear, and all of the users U1 to U3 in the vehicle turn their faces forward and enter the vehicle. A case is shown in which the user U2's hand approaches the door 3. In this case, the determination in step S4A is a positive result (step S4A; YES), and the notification control unit 18 causes the notification related determination unit 16 to determine the notification mode and notification timing based on the notification determination table 83 (see FIG. 4). , as shown by symbols A and B in FIG. 12, notification processing of LEVEL 5 is performed using sound and light by the first notification unit 15A and the second notification unit 15B (step S5B). Note that, as shown in FIG. 4, the notification timing is determined to be performed by the second notification unit 15B after the notification by the first notification unit 15A.

In this case, the notification process is executed in such a way that the in-vehicle user 2 notices it more easily than the notification process such as LEVEL2. In this embodiment, as shown by reference numeral A in FIG. 12, as a notification process of LEVEL 5, all speakers including the speaker 32b provided on the door 3 side and the other speaker 33d on the door 5 side to which the mobile object M1 approaches A process is executed to output a notification sound from the speakers 32a to 32d to notify of the approach of the mobile object M1, which is the object to be notified. Thereafter, as indicated by reference numeral B in FIG. 12, as a notification process of LEVEL 5, all door sides including the front left light 33b provided on the door 3 side and the rear left light 33d on the door 5 side where the mobile object M1 approaches. A process is executed in which the lights 33a to 33d output light that notifies the approach of the moving body M1.
Note that the timing at which the second notification section 15B notifies may be the timing at which the orientation detection section 12A detects that the first part of the user in the vehicle is facing the door, or the timing at which the first notification section 15A notifies. The timing may be the timing when a predetermined time has elapsed.

Therefore, after the in-vehicle user U2 who approaches the open door 3 is guided to turn his/her face toward the door 3 by the sound from the speaker 32b, the light from the front left light 33b on the side facing the user The approach of the mobile body M1 is notified, and the user in the vehicle can easily be made aware of the approach of the mobile body M1 by both sound and light. Furthermore, after the user U3 in the vehicle on the side where the mobile object M1 approaches is guided to turn his face toward the door 5 by the sound from the speaker 33d, the rear left light 33d also notifies the user U3 of the approach of the mobile object M1. , it becomes easier to notice the approach of the moving body M1.

In the case of FIG. 12, the case where all the speakers 32a to 32d and the door side lights 33a to 33d are used to notify the approach of the mobile object M1 is illustrated, but the point is that the notification process inside the vehicle is more effective than the notification process such as LEVEL2. Any notification processing that makes the user U2 more aware or increases the level of vigilance may be used. For example, the notification may be made with a loud notification sound using only the speakers 32a, 32d and the door lights 33a, 33d on the side where the moving body M1 approaches.
Furthermore, in FIG. 12, if the object to be notified is not the moving object M1 but the stationary object T1 shown in FIG. 7, it is only necessary to notify the in-vehicle user U2 in the passenger seat 7b of the stationary object T1. In this case as well, notification processing that makes it easier for the in-vehicle user 2 to notice or increases the level of vigilance than notification processing such as LEVEL 2 can be widely applied.

Returning to FIG. 10, if it is not the second situation (step S4A; NO), the notification control unit 18 determines that the orientation detection unit 12A causes the first part of the user in the vehicle to face the door, and the motion detection unit 12B causes the second part of the user to face the door. It is determined whether or not the third situation is movement (step S6A).
In FIG. 13, when the door 3 is in an open state, a moving object M1, which is a notification target, approaches the vehicle 1 from the left rear, an in-vehicle user U2 in the passenger seat 7b turns his face toward the nearby door 3, and , a case where the hand of the in-vehicle user U2 approaches the door 3 is shown. In this case, the determination in step S6A is a positive result (step S6A; YES), and the notification control unit 18 causes the notification related determination unit 16 to determine the notification mode and notification timing based on the notification determination table 83 (see FIG. 4). , LEVEL 6 notification processing is performed using sound and light from the first notification unit 15A and the second notification unit 15B (step S7B). Note that, as shown in FIG. 4, the notification timing is determined to be simultaneous notification by the first notification unit 15A and the second notification unit 15B.
Further, the notification control unit 18 causes the notification related determination unit 16 to determine whether door control is enabled based on the notification determination table 83 (see FIG. 4), and the door control unit 17 determines whether the door is open (door 3 in FIG. 13). A door closing process is performed to bring the door into a closed state (step S8B: door closing step).

As a result, as shown in FIG. 13, all the speakers 32a to 32d, including the speaker 32b provided on the door 3 side and the other speaker 33d on the door 5 side to which the moving body M1 approaches, are moved as a notification process of LEVEL6. All the door side lights 33a to 33d, including the front left light 33b provided on the door 3 side and the rear left light 33d on the door 5 side where the moving object M1 approaches, output a notification sound to notify the approach of the body M1. A process for outputting a notification sound notifying the approach of the mobile object M1 is executed.

If the user faces an open door and approaches the door with his or her hand, there is a high possibility that the user inside the car will jump out of the car through the open door. The user is most likely to notice the approach of the notification target. Furthermore, since the open door is closed, it is possible to proactively avoid a situation where the user inside the vehicle gets out of the vehicle while the notification target is approaching. In the door closing process of step S8B, the doors (at least the doors 3 and 5 to which the notification target approaches) may be controlled to be locked after the open door is closed, or When the approach is no longer detected, a locked door may be unlocked, or a closed door may be controlled to be opened.

In addition, in the case of FIG. 13, the case where all the speakers 32a to 32d and the door side lights 33a to 33d are used to notify the approach of the mobile object M1 is illustrated, but the point is that the notification is the one that is most easily noticed by the user in the vehicle. The process is widely applicable. For example, only the speakers 32a, 32d and the door lights 33a, 33d on the side where the mobile object M1 approaches may be used to notify with the notification sound at the maximum volume.
Further, in FIG. 13, if the object to be notified is not the moving object M1 but the stationary object T1 shown in FIG. 7, it is only necessary to notify the in-vehicle user U2 in the passenger seat 7b of the stationary object T1. In this case as well, the notification process that makes the in-vehicle user most likely to notice and be most alert can be widely applied.

The process of determining the notification mode and notification timing in steps S3B, S5B, and S7B above corresponds to the notification related determination step of the present invention, and the notification process in steps SS3B, S5B, and S7B corresponds to the notification control step of the present invention. Equivalent to.

As described above, the vehicle control system 100 includes the orientation detection unit 12A that detects the orientation of the first part of the user in the vehicle when the approach detection unit 13A detects the approach of the notification target object, and the orientation detection unit 12A that detects the orientation of the first part of the user in the vehicle. Based on the combination of the detection results with the motion detection section 12B, which detects a predetermined motion related to getting off the vehicle by the user in the vehicle based on the motions of the two parts, the notification related determination section 16 determines the notification mode and the notification mode when notifying the approach of the notification target object. Determine notification timing. Next, the vehicle control system 100 causes the notification control unit 18 to notify the approach of the notification target at the determined notification timing by using either the first notification unit 15A or the second notification unit 15B specified by the determined notification mode. Notice.
As a result, the notification mode and notification timing can be varied depending on the combination of the orientation of the first part of the user in the vehicle and the predetermined action related to getting off the vehicle, making it easier to provide notifications appropriate to the occupant's condition. .

The notification related determination unit 16 is configured such that when the approach of the notification target is detected, the movement detection unit 12B detects a predetermined movement in which the user in the vehicle attempts to open the door of the vehicle 1, and the orientation detection unit 12A detects a predetermined movement in which the user in the vehicle attempts to open the door of the vehicle 1. When it is detected that the first part of the user is facing the door, both the first notification section 15A and the second notification section 15B are determined to be used as the notification mode, and the first notification is determined as the notification timing. Simultaneous notification by the section 15A and the second notification section 15B is determined.
As a result, when a user inside the car performs an action that is likely to result in immediate exit, such as turning to the door and attempting to open the door, the simultaneous audible and visible light notification makes it easier to recognize the approach of the notification target. .

Further, the notification related determination unit 16 determines that when the approach of the notification target is detected, the movement detection unit 12B detects the movement of the user in the vehicle to open the door of the vehicle 1, and the direction detection unit 12A detects the movement of the in-vehicle user attempting to open the door of the vehicle 1. When it is detected that the first part of the user in the vehicle is not facing the door, both the first notification section 15A and the second notification section 15B are determined to be used as the notification mode, and the notification timing is set as the first notification section 15A and second notification section 15B. After notification is made by the notification unit 15A, the timing for notification by the second notification unit 15B is determined.
As a result, if a user inside the car tries to open the door without facing the door, an audible notification will be displayed to the user inside the car that can be seen after the user turns to the door (including outside the car). Notification using visible light becomes possible. Therefore, it becomes easier to recognize the approach of the notification target. Another advantage is that notifications using visible light, which may not be seen by users in the car, can be avoided.

In addition, the notification control unit 18 causes the second notification unit 15B to notify when the orientation detection unit 12A detects that the first member of the in-vehicle user is facing the door after the first notification unit 15A has notified. do. This allows visible light notifications to be controlled at appropriate times.

Further, the notification related determination unit 16 determines that when the approach of the notification target object is detected, the predetermined movement is not detected by the movement detection unit 12B, and the first part of the user in the vehicle is facing the door by the orientation detection unit 12A. If it is detected that the second notification section 15B is being used as the notification mode, the notification timing is determined as the period during which the first part of the user inside the vehicle is facing the door. As a result, if the user inside the car is facing the door (including the outside of the car) but is not making any movement to open the door of vehicle 1, a visible light notification will be sent as soon as the user inside the car is no longer facing the door. This makes it easier to avoid unnecessary notifications.

Further, the notification-related determination unit 16 determines that when the approach of the notification target object is detected, the door opening/closing detection unit 14 detects that the door is in an open state, and the direction detection unit 12A determines that the first part of the user in the vehicle is located at the door. If it is detected that the user is facing the door, it is determined that both the first notification unit 15A and the second notification unit 15B are to be used as the notification mode, and the door is controlled to be closed. Then, the notification control unit 18 notifies the approach of the notification object using both the first notification unit 15A and the second notification unit 15B based on the determination result of the notification related determination unit 16, and also causes the door control unit 17 to notify the door of the object to be notified. is controlled to the closed state. As a result, in a situation where an occupant can jump out of an open door, appropriate notification and door closing control can be performed in accordance with the occupant's condition.

The present invention is not limited to the configuration of the above-described embodiments, and can be implemented in various forms without departing from the gist thereof.
For example, when performing each of the notification processes in the embodiments described above, a configuration provided in the vehicle 1 that is capable of sending notifications to outside the vehicle may be used to also send notifications to outside the vehicle. Examples of configurations that can notify the outside of the vehicle include lamps such as hazard lamps that can notify the outside of the vehicle using visible light, and sound emitting units such as horns that can notify the outside of the vehicle using audible sounds. For example, when the approach of a notification target is detected and the motion detection unit 12B detects a predetermined motion of the user in the vehicle attempting to open the door of the vehicle 1, visible light and audible sound may be used as appropriate. The notification may be sent outside the vehicle.

Further, in the above embodiment, a case has been described in which the present invention is applied to the vehicle control system 100 and the vehicle control method shown in FIG. 1 etc., but the present invention is not limited thereto. For example, although the case where a single vehicle control device 10 is disposed within the vehicle control system 100 has been illustrated, the vehicle control device 10 may be distributed to a plurality of control devices, or the vehicle control device 10 may be disposed outside the vehicle control device 10. A part of the configuration (for example, cameras 70 and 71 for in-vehicle photography) may be built into the vehicle control device 10.

Further, the configuration of each part shown in FIG. 3 may be realized by hardware, or may be realized by cooperation of hardware and software. Further, the method of dividing the processing units of each flowchart and the processing order are not limited to the illustrated example.
Furthermore, the LEVELs 1 to 6 of each notification process described above are merely examples, and the value of LEVEL of each notification process may be changed as appropriate.

Further, the control program 81 may be stored in an external apparatus or device and acquired via the communication unit 120 or the like. Further, it is also possible to record the control program 81 on a computer-readable recording medium. As the recording medium, a magnetic or optical recording medium or a semiconductor memory device can be used. Specifically, portable or fixed recording media such as flexible disks, various optical disks, magneto-optical disks, flash memories, and card-type recording media can be mentioned. Further, the recording medium may be a nonvolatile storage device such as a RAM, ROM, or HDD, which is an internal storage device included in the image display device.

DESCRIPTION OF SYMBOLS 1...Vehicle, 2...Right power hinge door, 3...Left power hinge door, 4...Right power sliding door, 5...Left power sliding door, 6...Power tailgate, 10...Vehicle control device, 11...Key authentication section, 12... Vehicle interior situation recognition section, 12A... Direction detection section, 12B... Movement detection section, 12C... Locked state detection section, 13... Vehicle surrounding situation recognition section, 13A... Approach detection section, 14... Door opening/closing detection section, 15... Notification section , 15A...first notification section, 15B...second notification section, 16...notification related determination section, 16A...notification mode determination section, 16B...timing determination section, 17...door control section, 18...notification control section, 30...front Camera (external camera), 31... Display, 32a-32d... Speaker, 33a-33d... Door side light, 40... Right side camera (external camera), 50... Left side camera (external camera), 60... Rear camera (camera outside the vehicle), 70... Front seat camera (camera inside the vehicle), 71... Rear seat camera (camera inside the vehicle), 80... Memory, 81... Control program, 83... Notification determination table, 84... Notification information , 100...Vehicle control system, 120...Communication unit, 130...Navigation device, 150...Battery, U1, U2, U3...User (occupant), M1...Moving object (notification target object), T1...Stationary object (notification target object) ).

Claims (7)

an approach detection unit that detects the approach of a notification target to the vehicle ;
an orientation detection unit that detects the orientation of a first part that can identify the direction of the line of sight or face of an occupant in the vehicle;
a motion detection unit that detects a predetermined motion related to the occupant getting off the vehicle based on a motion of a second region different from the first region of the occupant;
a first notification unit that notifies the occupant by an audible sound;
a second notification unit that notifies the occupant using visible light;
When the approach detection unit detects the approach of the notification target, use of the first notification unit and the second notification unit is determined based on a combination of the detection results of the orientation detection unit and the motion detection unit. a notification-related determination unit that determines a notification mode including identification of a target notification unit and a notification timing by the target notification unit;
a notification control unit that notifies the approach of the notification target object at the notification timing by a notification unit to be used of the first notification unit and the second notification unit, based on a determination result of the notification related determination unit; ,
A vehicle control system equipped with The notification-related determination unit may detect, as the predetermined movement, a movement of the occupant attempting to open a door of the vehicle, when the movement detection unit detects the approach of the object to be notified; If the direction detection unit detects that the first part of the occupant is facing the door, determining that both the first notification unit and the second notification unit are to be used as the notification mode, and , determining simultaneous notification by the first notification unit and the second notification unit as the notification timing;
The vehicle control system according to claim 1. The notification-related determination unit is configured to cause the operation detection unit to determine, as the predetermined operation, an operation in which the occupant attempts to open a door of the vehicle when the approach detection unit detects the approach of the notification object. and when the direction detection unit detects that the first part of the occupant is not facing the door, both the first notification unit and the second notification unit are used as the notification mode. determining the target, and determining, as the notification timing, the timing to notify by the second notification unit after the notification by the first notification unit;
The vehicle control system according to claim 1 or 2. The notification control unit notifies the second notification unit when the orientation detection unit detects that the first part of the occupant is facing the door after the first notification unit has notified. ,
The vehicle control system according to claim 3. The notification-related determination unit may be arranged such that when the approach of the notification target object is detected, the predetermined movement of the occupant is not detected by the movement detection unit, and the first part of the occupant is not facing the door by the direction detection unit. If it is detected that the vehicle occupant is facing the door, the second notification unit is determined to be used as a notification mode, and a period during which the first part of the occupant is facing the door is determined as a notification timing.
The vehicle control system according to any one of claims 1 to 4. a door opening/closing detection unit that detects the opening/closing state of the door of the vehicle;
A door control unit that controls the door,
The notification-related determination unit is configured such that when the approach of the notification object is detected, the door opening/closing detection unit detects that the door is in an open state, and the direction detection unit detects that the first part of the occupant is in the If it is detected that the user is facing the door, the notification mode is determined to target both the first notification unit and the second notification unit and to control the door to a closed state;
The notification control unit notifies the approach of the notification object using both the first notification unit and the second notification unit based on the determination result of the notification related determination unit, and causes the door control unit to notify the door of the object. control the closed state,
A vehicle control system according to any one of claims 1 to 5. an approach detection unit that detects the approach of a notification target;
a first notification unit that notifies the occupant by an audible sound;
A vehicle control method executed by a computer to control the operation of a vehicle, comprising: a second notification unit that notifies the occupant using visible light;
an orientation detection unit that detects an orientation of a first part that can identify the direction of the passenger's line of sight or face when the proximity detection unit detects the approach of the notification target; and a first part of the passenger; Based on the combination of detection results with a motion detection section that detects a predetermined motion related to the occupant's exit from the vehicle based on motions of different second sections, a notification section to be used among the first notification section and the second notification section; a notification-related determination step of determining a notification mode including identification of the notification unit and a notification timing by the notification unit to be used;
a notification control step of notifying the approach of the notification target object at the notification timing by a notification unit to be used of the first notification unit and the second notification unit, based on the determination result of the notification related determination step; ,
Vehicle control method including.

Images