JP6201415B2 - Vehicle interior monitoring device - Google Patents

Description

  The present invention relates to a vehicle interior monitoring device that displays a monitoring image of a rear seat of a vehicle cabin on a display that can be viewed by a driver.

  With this type of device, when a child's crying is detected from the rear seat, the image of the rear seat occupant (child) captured by the rear seat camera is displayed on the front seat display. There is known an in-vehicle video display device in which a guardian can monitor the state of an occupant (child) in a rear seat while driving (Patent Document 1).

JP 2003-025911 A

  However, in the conventional device, the rear seat image is presented without considering the driving situation of the vehicle, so even in the scene where the driver should concentrate on the vehicle operation, the driver's attention is uniformly applied to the rear seat. There is a problem of being attracted to the video.

  The problem to be solved by the present invention is to prevent the driver's attention from being drawn in situations where the driver should concentrate on the vehicle operation.

The present invention determines whether or not the current running state is a steady running state with little change in the running environment based on the running information of the vehicle, and the rear seat is limited to a scene where the running state of the vehicle is a steady running state. In the scene where the vehicle is in an unsteady driving state, display of the rear seat image is prohibited, and if the information indicating that the driving support device is operating is acquired, For the criterion used to determine whether or not the state is a steady driving state, the upper threshold of the criterion is changed to a higher value so that the current driving state is easily determined to be a steady driving state. The above-mentioned problem is solved by changing the threshold value to be low .

According to the present invention, presentation of the image of the rear seat is prohibited in a scene where the current driving situation is determined to be a driving situation where the driver should concentrate on vehicle operation, and the rear seat is limited to other scenes. Therefore, the driver can concentrate on the vehicle operation and can confirm the situation of the backseat as necessary in a scene where the driving situation should be concentrated on the vehicle operation.

It is a block block diagram of the compartment monitoring system 1 which concerns on this embodiment. It is a figure which shows the example of installation of the camera of this embodiment. It is a figure which shows the example of installation of the microphone of this embodiment. It is a figure which shows an example of the image | video of a backseat. 4A is a diagram illustrating an example of a display, FIG. 4B is a diagram illustrating an example of a wide area rear seat image, and FIG. 4C is a diagram illustrating an example of a narrow area rear seat image. It is a flowchart which shows the control procedure of the compartment monitoring system which concerns on this embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, a case where the vehicle interior monitoring device according to the present invention is applied to a vehicle interior monitoring system will be described as an example.

  FIG. 1 is a diagram illustrating a block configuration of a passenger compartment monitoring system 1 including a passenger compartment monitoring device 100 according to the present embodiment. The vehicle interior monitoring device 100 and the vehicle interior monitoring system 1 including the vehicle interior monitoring device 100 are mounted on a vehicle. The vehicle interior monitoring device 100 monitors the state of the vehicle interior of the host vehicle and provides necessary information to the driver.

  As shown in FIG. 1, the passenger compartment monitoring system 1 includes a passenger compartment monitoring device 100 and an in-vehicle device 200. The in-vehicle device 200 includes a vehicle controller 50, various sensors 60, a driving support device 70, a navigation device 80, and another vehicle detection device 90. The vehicle interior monitoring device 100 and each vehicle-mounted device 200 are connected by a CAN (Controller Area Network) or other vehicle-mounted LAN in order to exchange information with each other.

  Various sensors 60 of the present embodiment detect a steering angle sensor 61 that detects a steering angle according to a steering operation of the vehicle, a vehicle speed sensor 62 that detects the speed of the vehicle, and acceleration in the front-rear direction with respect to the vehicle traveling direction. Longitudinal acceleration sensor 63, lateral acceleration sensor 64 for detecting acceleration in the lateral direction with respect to the vehicle traveling direction, vehicle attitude sensor 65 for detecting the attitude of the vehicle, key sensor 66 for detecting key lock of the vehicle, and vehicle And a shift sensor 67 for detecting the shift state. These various sensor groups 60 output the detected information to the vehicle interior monitoring apparatus 100 via the vehicle controller 50 or directly.

  The driving support device 70 according to the present embodiment is a device that supports a driver's accelerator operation, brake operation, steering operation, or the like. Although not particularly limited, the driving support device 70 of the present embodiment supports inter-vehicle distance maintenance traveling, constant speed traveling, vehicle approach prevention, and other driving operations. The driving support device 70 outputs information on whether or not the driving support operation is being performed to the vehicle interior monitoring device 100.

  Moreover, as shown in the figure, the vehicle interior monitoring device 100 of the present embodiment includes a control device 10, a detection device 20, and a display 30.

  The detection device 20 includes a camera 21 that captures an image of the rear seat of the vehicle and a microphone 22 that acquires sound generated at the rear seat of the vehicle.

The camera 21 of this embodiment is a camera using an image sensor such as a CCD.
FIG. 2A is a diagram illustrating an installation example of the camera 21. 2A (A) shows the installation position of the camera 21 when the vehicle V is viewed from above, and FIG. 2A (B) shows the installation position of the camera 21 when the vehicle V is viewed from the side. The number and location of the cameras 21 are not particularly limited, and in the present embodiment, one wide-angle camera 21 is installed near the vehicle interior ceiling of the vehicle V as shown in FIG.

  FIG. 3 shows an example of a rear seat image captured by the camera 21. As shown in FIG. 3, the camera 21 captures images of the front driver's seat STFR and the passenger seat STFL, and the rear seat right seat STRR and left seat STRL. This image includes an image of the occupant (child) PS1 sitting in the left seat STRL of the rear seat. Further, as shown in the figure, a plurality of cameras including a camera 21RR that images the right seat of the rear seat and a camera 21RL that images the left seat of the rear seat may be installed.

  FIG. 2B is a diagram illustrating an installation example of the microphone 22. FIG. 2B (A) shows the installation position of the microphone 22 when the vehicle V is viewed from above, and FIG. 2B (B) shows the installation position of the microphone 22 when the vehicle V is viewed from the side. Although the number and arrangement location of the microphones 22 are not particularly limited, in the present embodiment, as shown in FIG. 2B, the sound emitted from the right seat is mainly acquired near the ceiling in front of the rear seat of the vehicle V. There is provided a microphone 22RR and a microphone 22RL provided mainly in the vicinity of the left seat of the rear seat and for mainly acquiring sound emitted from the left seat.

  The display 30 of the present embodiment is installed around a dashboard that can be visually recognized by the driver of the vehicle. The display 30 according to the present embodiment switches information to be presented on the display 30 according to the judgment of the vehicle interior monitoring apparatus 100. Of course, the display 30 can also switch the information presented on the display 30 in accordance with the input to the display changeover switch 31. The display changeover switch 31 accepts an input of a user command related to display control of the display 30.

  The display 30 of the present embodiment switches and displays the rear seat image and other information according to the control command of the vehicle interior monitoring device 100. Information other than the image of the rear seat includes information relating to driving of the vehicle and entertainment information such as television broadcasting. In particular, the display 30 according to the present embodiment controls display switching between the rear seat image and information related to driving of the vehicle. The “information relating to driving of the vehicle” of the present embodiment includes guidance information provided by the navigation device 80, driving support information provided by the driving support device 70, and information provided by the other vehicle detection device 90. In this embodiment, since the display for the navigation device 80 is used as the display 30 for the vehicle interior monitoring device 100, map information for route guidance of the navigation device 80 is usually presented as shown in FIG. Is done.

  The display 30 according to the present embodiment displays a wide-area rear-seat image, which is a wide-range image including the rear-seat, when the rear-seat image is displayed according to the control command of the vehicle interior monitoring device 100, and the wide-area rear-seat image. A narrow-area rear-seat image that is an image of a partial range of the rear-seat that is narrower than the range included in the rear-seat image is switched and displayed. FIG. 4B shows an example of a wide area rear seat image, and FIG. 4C shows an example of a narrow area rear seat image.

  Then, the control apparatus 10 of this embodiment is demonstrated. The control device 10 of the vehicle interior monitoring device 100 according to the present embodiment determines whether or not the vehicle is in a steady running state and stores a program (ROM) that stores a program for controlling the presentation process of the display 30 according to the determination result. A CPU (Central Processing Unit) 11 as an operation circuit that functions as the vehicle interior monitoring apparatus 100 and a RAM that functions as an accessible storage device by executing a program stored in the ROM and a program stored in the ROM (Random Access Memory) 13.

  The control device 10 of the vehicle interior monitoring apparatus 100 according to the present embodiment has at least a rear seat image acquisition function, a presentation function, a traveling state determination function, and a presentation control function. Furthermore, the control apparatus 10 of this embodiment further has a backseat sound collection function, a change detection function, and a driving assistance operation information acquisition function, depending on the embodiment. The control apparatus 10 of this embodiment performs each function by cooperation of the software for implement | achieving the said function, and the hardware mentioned above.

  Hereinafter, each function of the vehicle interior monitoring apparatus 100 according to the present embodiment will be described.

  First, the rear seat image acquisition function of the control device 10 of the present embodiment will be described. The control device 10 acquires a captured image in the passenger compartment of the host vehicle captured by the camera 21 (21RR, 21RL). The acquired captured image is used to determine whether or not the passenger in the rear seat has changed, and is also used as an image to be presented on the display 30 as a rear seat video.

  The rear seat sound collecting function of the control device 10 of this embodiment will be described. The control device 10 acquires sound generated at the rear seat of the vehicle collected by the microphone 22 (22RR, 22RL). The acquired voice is used to determine whether or not the passenger in the rear seat has changed.

  The change detection function of the control apparatus 10 of this embodiment is demonstrated. The control device 10 detects a change in the occupant in the rear seat of the vehicle. The control device 10 can detect a change in the occupant of the rear seat based on the acquired image information of the video or the acquired audio information. The control device 10 determines that an abnormality has occurred in the rear seat when a large change greater than or equal to a predetermined value is detected based on the change in the passenger in the rear seat. The change in the occupant in the rear seat is detected from the video imaged by the camera 21 and / or the audio information collected by the microphone 22.

A method for determining a change in the occupant of the rear seat by the change detection function of the present embodiment will be described.
First, the control device 10 according to the present embodiment performs the rear of the vehicle based on the amount of movement of the rear seat passenger extracted from the image information of the rear seat video acquired by the rear seat video acquisition function described above. Detect changes in seat occupants. The control device 10 according to the present embodiment extracts the movement in each seat in the rear seat from the temporal change in the image of the rear seat (for example, successive difference processing by image processing) based on the captured image of the vehicle interior camera 21. Specifically, the control device 10 extracts feature points that are part of the image of the occupant from the image information of the image of the rear seat, tracks the feature points over time, and changes the amount of change within a predetermined time (images). The travel distance, the accumulated travel distance, etc. are calculated. If the amount of change per time is greater than or equal to a predetermined amount, it is determined that there is a change in the passenger in the rear seat of the vehicle. For example, when a child has little movement, the child does not have to monitor the child carefully because the child is asleep or is in good mood even when waking up. On the other hand, when the child's movement (change amount or accumulated amount) is large, the child feels pain or discomfort, so the driver needs to carefully monitor the child. When a child moves his / her limb or extends his / her hand to something, the movement of feature points extracted from the image of the limbs is observed, so depending on the amount of movement of the feature points (movement speed and movement accumulation amount) The movement of the child (rear seat occupant) can be extracted and its change detected.

  In addition, the change detection function of the present embodiment specifies the position (seat position where the seat is seated) of the occupant who has changed based on the position (coordinates) on the image of the feature point to be tracked when detecting the change. can do. The position information of the occupant who has changed is used for switching control between a wide area rear seat image and a narrow area rear seat image, which will be described later.

  Secondly, the control device 10 of the present embodiment is based on the volume of the sound generated by the occupant of the rear seat extracted from the voice information of the voice of the rear seat acquired by the above-described rear seat voice acquisition function. Detects changes in passengers in the rear seats. Specifically, the control device 10 extracts features of audio information collected by the microphones 22RR and 22RL installed in the left and right rear seats, tracks the features over time, and changes in a predetermined time (Volume, frequency, etc.) is calculated. If the amount of change per time is greater than or equal to a predetermined amount, it is determined that there is a change in the passenger in the rear seat of the vehicle. For example, when a child is quiet, the driver does not need to monitor the child carefully because the child is asleep or is in good mood when waking up. On the other hand, when a child crys or makes a loud voice, the child feels pain and discomfort, so the driver needs to monitor the child carefully. When a child crys or makes an unusual voice, changes in the volume and frequency extracted from the audio information are observed, so changes in the state of the child (rear seat occupant) are detected accordingly. can do. As will be described later, the vehicle interior monitoring apparatus 100 according to the present embodiment displays the rear seat image on the display 30 when the change (movement / sound) of the rear seat occupant is greater than or equal to a predetermined amount.

  Further, the change detection function of the present embodiment can specify the position (seat position where the user is seated) where a moving occupant is located based on the position of the microphone 22 that has detected the change. For example, when the microphone 22RR installed near the rear right seat detects a sound with a sound volume equal to or higher than a predetermined value, it can be determined that the passenger who has changed is located in the rear right seat. When a plurality of microphones 22 are installed, and the plurality of microphones 22 detect sound with a volume higher than a predetermined value, the position of the occupant may be determined based on the position of the microphone 22 that detected the sound with the highest volume. it can. The position information of the occupant who has changed is used for switching control between a wide area rear seat image and a narrow area rear seat image, which will be described later.

  As a method for determining the change in the occupant of the rear seat, either the above-described method based on the video imaged by the camera 21 or the method based on the audio information collected by the microphone 22 may be used. You may combine.

  The control device 10 of the present embodiment further includes driving support operation information acquisition. The control device 10 acquires from the driving support device 70 driving support operation information that supports at least one driving operation of a steering operation, an accelerator operation, or a braking operation of the vehicle. The driving assistance operation information is used in a reference state update process for driving state determination described below.

  Next, the traveling state determination function will be described. Based on the travel information acquired from the vehicle, the control device 10 according to the present embodiment determines whether or not the current travel state of the vehicle is a steady travel state that is defined in advance as a travel state in which a change in the travel environment is small. Determine whether. Specifically, the control device 10 has a state in which the change in the travel environment is small and the change in the state of the vehicle posture is small based on the travel information based on the output values of the various sensors 60 acquired through the vehicle controller 50 of the in-vehicle device 200. Is determined to be in a steady running state.

  The control device 10 of the present embodiment is based on vehicle travel information, (1) the vehicle is traveling straight, (2) the vehicle is traveling at a constant speed, and (3) the vehicle is running at low speed. If any one of the conditions that the vehicle is traveling and (4) the distance between the vehicle and the other vehicle is a predetermined value or more is satisfied, the vehicle is in a steady traveling state. Judge that there is. In other words, conversely, when none of the above conditions (1) to (4) is satisfied, it is determined that the traveling state of the vehicle is an unsteady traveling state.

  Specifically, (1) whether the vehicle is traveling straight ahead is determined by whether the steering angle acquired from the steering angle sensor 61 is less than a predetermined value, or the lateral acceleration acquired from the lateral acceleration sensor 64 is less than a predetermined value. Judgment based on that. (2) Whether or not the vehicle is traveling at a constant speed is determined by the fact that the amount of time-dependent change in vehicle speed acquired from the vehicle speed sensor 62 is less than a predetermined value, and the longitudinal acceleration acquired from the longitudinal acceleration sensor 63 is less than a predetermined value. Judgment based on that. (3) Whether the vehicle is traveling at a low speed is determined based on the fact that the vehicle speed acquired from the vehicle speed sensor 62 is less than a predetermined value. (4) Whether the distance between the vehicle and the other vehicle is less than a predetermined value is based on the existence of the other vehicle output from the other vehicle detection device 90 and the distance to the other vehicle being less than the predetermined value. to decide.

From a different point of view, the control device 10 of the present embodiment, based on the vehicle travel information, (5) the vehicle is traveling non-straight (curve travel), (6) the vehicle is traveling at a non-constant speed (acceleration). Among the conditions that (7) the vehicle is traveling at a low speed (high speed traveling), and (8) the distance between the vehicle and the other vehicle is less than a predetermined value, When any one or more of the conditions are satisfied, it is determined that the traveling state of the vehicle is an unsteady traveling state (not a steady traveling state). Conversely, when none of the above conditions (5) to (8) is satisfied, it is determined that the traveling state of the vehicle is a steady traveling state.

Specifically, (5) whether or not the vehicle is traveling non-straightly (curve traveling) is determined by whether the steering angle acquired from the steering angle sensor 61 is a predetermined value or more, or the lateral angle acquired from the lateral acceleration sensor 64 Judgment is made based on the fact that the acceleration is not less than a predetermined value. (6) Whether or not the vehicle is traveling at a non-constant speed (acceleration / deceleration traveling) is acquired from the longitudinal acceleration sensor 63 that the change over time of the vehicle speed acquired from the vehicle speed sensor 62 is a predetermined value or more. Judgment is made based on the fact that the longitudinal acceleration is equal to or greater than a predetermined value. (7) Whether or not the vehicle is traveling at a low speed (high speed traveling) is determined based on the fact that the vehicle speed acquired from the vehicle speed sensor 62 is equal to or higher than a predetermined value. (8) whether the distance between the vehicle and the other vehicle is less than a predetermined value, based on presence of and distance to other vehicles of the other vehicle that is outputted from the other vehicle detection device 90 is less than the predetermined value to decide.

  In the present embodiment, the first determination method for determining whether the traveling state is the steady traveling state or the non-providing traveling state using the conditions (1) to (4) may be used. Then, a second determination method for determining whether the traveling state is the unsteady traveling state or the traveling state is the steady traveling state using the conditions (5) to (8) may be used. That is, any method may be used.

  Further, when the first determination method using the conditions (1) to (4) and the second determination method using the conditions (5) to (8) are used in combination, the above (1) Even if any one or more of the conditions (4) to (4) are satisfied, if any one or more of the conditions (5) to (8) are satisfied, the current traveling state is unsteady traveling It is determined that the vehicle is in a state (not a steady running state). For example, there may be a scene in which the vehicle is traveling at a low speed while the vehicle is traveling non-straightly (curving). If the vehicle is traveling at a low speed, the above condition (3) is satisfied. According to the first determination method, it is determined that the vehicle is in a steady traveling state, and if the vehicle is traveling non-straightly (curving), Condition (5) is satisfied, and according to the second determination method, it is determined that the vehicle is in an unsteady traveling state. As described above, when a contradictory conclusion is derived between the first determination method and the second determination method, the second determination method for determining the unsteady traveling state is prioritized. Thereby, even if the contradictory conclusion comes out when using the 1st judgment method and the 2nd judgment method together, it can adjust.

  When judging whether the running state is “steady running state” or “unsteady running state (not steady running state)”, “determination item (condition)”, “input information”, and “determination” An example of correspondence information related to “result” is shown in the table below. The determination result of “the steady running state” or “the unsteady running state (not the steady running state)” is sent to the presentation control function described next.

  When the driving support operation information acquisition function acquires information indicating that the driving support device 70 is operating, the control device 10 of the present embodiment determines whether or not the driving state of the vehicle is a steady driving state. Relax criteria. That is, based on the travel information, the upper limit threshold used when determining that the current travel state is the steady travel state is changed to a higher value, and the lower limit threshold is changed to a lower value. Thereby, when the driving support device 70 is operating, it is easy to determine that the current traveling state is the steady traveling state. In the vehicle interior monitoring device 100 of the present embodiment, when the driving support device 70 is operating in the normal mode (not a mode for calling attention), the driving load on the driver is reduced. Based on this, while the driving support process is in operation, the video is presented to the rear seat as much as possible for the attention. According to the vehicle interior monitoring apparatus 100 of the present embodiment, the image of the rear seat can be presented as much as possible to the driver concerned about the rear seat.

Finally, the presentation control function will be described. The control apparatus 10 of this embodiment switches the information shown on the display 30 according to the determination result of the traveling state determination function.
In a scene where the vehicle is in steady running, the control device 10 of the present embodiment detects the rear seat acquired by the rear seat image acquisition function when the change detection function detects a change in the occupant of the rear seat of the vehicle. Are displayed on the display 30. Of course, the control device 10 can also display the rear seat image on the display 30 when a display change command is input to the display changeover switch 31.
On the other hand, in a scene where the vehicle is not traveling, the control device 10 of the present embodiment does not display the rear seat image on the display 30.

  Specifically, the control device 10 of the present embodiment displays an image of the passenger in the rear seat on the display 30 when the current driving state of the vehicle is determined to be “steady driving state” by the driving state determination function. On the other hand, when it is determined that the current running state of the vehicle is not “steady running state”, the display of the rear seat image on the display 30 is prohibited. That is, the control device 10 of the present embodiment displays the rear seat image on the display 30 only when the current traveling state of the vehicle is determined to be “steady traveling state” by the traveling state determination function.

  Further, the control device 10 according to the present embodiment, when the driving support operation information acquisition function described above is in operation and the driving support device 70 is operating, and in the operation, information indicating that driving needs attention is acquired. Presenting the seat image on the display 30 is prohibited. The “information that requires attention to driving” can be appropriately defined according to the driving support processing content of the driving support device 70. For example, when a vehicle speed higher than a predetermined vehicle speed is detected during execution of constant speed driving support, or when driving support for a constant speed / predetermined inter-vehicle distance is executed, the lane is changed before and after the host vehicle. For example, when another vehicle is detected.

  Furthermore, when displaying the image of the passenger in the rear seat, the control device 10 of the present embodiment displays a “wide area rear seat image” that is a wide range image including the rear seat and a range included in the wide area rear seat image. A function of switching the “narrow rear seat video”, which is a video of a partial range of the rear seat smaller than that, to be displayed on the display 30 is provided. The display mode of “narrow-range rear-seat video” is an enlarged display mode (zoom-up mode) for displaying an image in which a narrow range is greatly enlarged. 4B described above is an example of a wide-area rear seat image, and FIG. 4C is an example of a narrow-area rear seat image. When a change in the occupant in the rear seat of the vehicle is detected by the change detection function, the control device 10 is based on the seat position where the occupant having the change detected by the change detection function is located (sits). A “narrow area rear seat video” (see FIG. 4C) of the rear seat where the change is detected is displayed on the display 30.

  Of course, for example, when the display 30 is a touch panel type input / output device, the control device 10 of the present embodiment can also use the display switching function of the display 30. In other words, when the user touches a portion that he wants to enlarge (a place where he / she wants to pay attention), the user changes to the enlarged display mode enlarged around that portion, and the user performs a predetermined operation (touch) again in the enlarged display mode. To return to the wide display mode.

  Subsequently, a control procedure of the vehicle interior monitoring apparatus 100 of the present embodiment will be described based on the flowchart of FIG. The processing content shown in FIG. 5 is continuously performed at a predetermined cycle.

  As shown in FIG. 5, the processing is started due to the engine start (IGN_ON) of the vehicle. First, in step S <b> 1, the control device 10 confirms the presence or absence of a rear seat image display request operation by the driver, which is an output from the display changeover switch 31.

  Next, in step S2, if there is a request operation confirmed in step S1, the process proceeds to step S6, and if there is no request operation, the process proceeds to the next. In step S <b> 3, the control device 10 detects a change in the occupant based on the image information from the vehicle interior camera 21 or the audio information from the vehicle interior microphone 22. In the process using image information, for example, the motion in each seat in the rear seat is extracted from the temporal change in the image of the rear seat in the vehicle interior by successive difference processing by image processing. In the process using the voice information, the movement in each seat in the rear seat is extracted from the temporal change in the volume and frequency of the collected voice.

  In step S4, a branching process is performed based on the amount of change (image movement / sound change) of each seat area in the rear seat, which is the extraction result of step S3. If a large change greater than or equal to a predetermined value is detected in any one area for each seat area, it is determined that there is a change or more in the rear seat, and the process proceeds to step S5. Otherwise, the process returns to step S1. Repeat the process.

  Next, in step S5, since it is determined in step S4 that there is a change or abnormality in the rear seat, the seat position with a change or abnormality is stored and the process proceeds to the next. And in step S6, each sensor output value of the various sensors 60 (61-67) which is a vehicle signal which functions as driving information is acquired.

  Here, after step S6, the process may proceed to step S7, but in the present embodiment, the processes of steps S61 and S62 are performed. In step S61, the control device 10 acquires information as to whether or not the driving support device 70 is operating. In step S62, when each driving support function is operating, the standard (threshold value) used for determining whether or not “the vehicle is in a steady running state” is changed so as to relax. That is, the upper threshold value for determining that “the vehicle is in a steady running state” is changed to a higher value, and the lower threshold value is changed to a lower value. After changing the reference, the process proceeds to step S7. As a result, it is easy to determine that the vehicle is in a steady running state. If each driving support function is not activated in step S61, the process proceeds to step S7.

  In step S7, the control device 10 determines the current traveling state based on the output values of the various sensor groups 60 acquired in step S6. As the determination method of the running state, the method described with Table 1 is used. In step S8, the control device 10 performs a branch process based on the traveling state that is the determination result of step S7. If it is determined in step S7 that the current running state is the steady running state, the process proceeds to step S9, and if not, the process returns to step S6 and the process is repeated. That is, as long as the traveling state is an unsteady traveling state, the rear seat image display process is not performed until the traveling state is reached.

  In step S9, based on the request operation confirmed in step S1 or the changed or abnormal rear seat position saved in step S5, the rear seat video display mode is set to the wide display mode or the enlarged display mode (zoom). Check which is the case. When a specific seat is specified in the request operation, or when the abnormal rear seat is specified as the right seat, for example, the enlarged display mode as shown in FIG. The wide display mode is set as shown in (B).

  In step S10, branch processing is performed based on the display mode confirmed in step S9. In step S10, if the enlarged display mode is confirmed, the process proceeds to the next step. Otherwise, the process proceeds to step S13. In step S11, position information for carrying out enlarged display is acquired. Next, in step S12, the display 30 is enlarged and displayed based on the position information acquired in step S11. On the other hand, if the enlarged display mode is not confirmed in step S10, the rear seat image is displayed in a wide area on the display 30 in step S13.

  In step S14, if a predetermined time (for example, 10 seconds) has elapsed since the display was started, the process proceeds to the next process, and if not, this step is repeated to continue the display. In the meantime, the process of step S17 is executed.

  In step S <b> 17, branch processing is performed based on the alarm output state of the driving support device 70. If a warning regarding each driving support function (for example, an approach warning to a preceding vehicle or a wobbling warning in the lane) is generated from the driving support device 70, the process proceeds to step S15 to stop the display of the rear seat image. Otherwise, the process returns to step S14 and the display is continued.

  If it is determined in step S15 that the predetermined time has elapsed since the display was started in step S14, the display of the rear seat video using the display 30 is stopped and the process proceeds to the next. Finally, if the engine is turned off (IGN_OFF) in step S16, all the processes are terminated. If not, the process returns to step S1 to repeat the process.

  In the control procedure shown in FIG. 5, if there is a request operation confirmed in step 1 in the branching process in step S <b> 2, the process proceeds to step S <b> 6, and the display of the rear seat image is restricted depending on the traveling state of the vehicle. If priority is given to the driver's request operation, the processing may be continued from step S9.

  In this embodiment, it is determined whether or not the current driving state is a steady driving state with little change in the driving environment based on the driving signal, and the rear seat image is displayed only during the normal driving. The image of the rear seat is not displayed in a scene where the driver needs to concentrate during driving or acceleration / deceleration, and the rear seat image can be presented without attracting the driver's attention.

  According to the vehicle interior monitoring apparatus 100 of the present embodiment, the following effects are obtained.

[1] According to the vehicle interior monitoring apparatus 100 of the present embodiment, it is determined whether or not the current running state is a steady running state with little change in the running environment based on the running information, and only during steady running. to display the image of the rear seat, since the time of non-steady-state traveling to prohibit the display of the image of the rear seat, in the scene that the driver, such as during acceleration or deceleration cornering there is a need to concentrate on driving, the driver to concentrate on vehicle operation Can check the status of the backseat as needed.

  [2] According to the vehicle interior monitoring apparatus 100 of the present embodiment, based on the travel information of the vehicle, the vehicle is traveling straight, the vehicle is traveling at a constant speed, and the vehicle is traveling at a low speed. The vehicle running state is determined to be a steady running state when one or more of the following conditions are satisfied and the distance between the vehicle and the other vehicle is less than a predetermined value. It is possible to quantitatively determine whether the running state is a steady running state.

  [3] According to the vehicle interior monitoring apparatus 100 of the present embodiment, the vehicle is traveling non-straight, the vehicle is traveling at a non-constant speed, the vehicle is traveling at a non-low speed, Or, when the distance between the vehicle and another vehicle satisfies one or more of a predetermined value or more, it is determined that the vehicle is in an unsteady running state, so the current running state of the vehicle is in an unsteady running state. It can be determined quantitatively whether or not there is.

  [4] According to the vehicle interior monitoring apparatus 100 of the present embodiment, the vehicle is traveling straight, the vehicle is traveling at a constant speed, the vehicle is traveling at a low speed, and the vehicle Even if it is determined that the running state of the vehicle is a steady running state based on the condition that the distance to the other vehicle is less than a predetermined value, the vehicle is running non-straight, the vehicle is running at a non-constant speed. The vehicle is traveling at a low speed, or the distance between the vehicle and another vehicle is greater than or equal to a predetermined value. Therefore, even if a contradictory conclusion comes out depending on the judgment condition, it can be adjusted. In this case, the priority is given to the determination that the vehicle is in an unsteady driving state, and therefore, in situations where the driver concentrates on driving, even if the conditions for the steady driving state are satisfied, the presentation of the rear seat image is suppressed. can do.

  [5] According to the vehicle interior monitoring apparatus 100 of the present embodiment, the image of the rear seat is displayed when a change in the rear seat occupant is detected, so the state of the rear seat occupant is confirmed at an appropriate timing. be able to.

  [6] According to the vehicle interior monitoring apparatus 100 of the present embodiment, a change in the occupant in the rear seat of the vehicle is detected based on the amount of movement of the occupant in the rear seat extracted from the image information of the rear seat image. Therefore, when a large movement is detected in the rear seat, the rear seat image can be displayed. For example, when a child is moving greatly and something is likely to happen, the driver can view the rear seat image without performing a special display operation.

  [7] According to the vehicle interior monitoring apparatus 100 of the present embodiment, a change in the occupant in the rear seat of the vehicle is detected based on the volume of the sound generated by the occupant in the rear seat extracted from the acquired audio information. Therefore, the rear seat image can be displayed when a loud sound is detected in the rear seat. For example, when the child crys or makes a loud sound, the driver can view the rear seat image without performing a special display operation.

  [8] According to the vehicle interior monitoring apparatus 100 of the present embodiment, when a change in the occupant in the rear seat is detected, a narrow rear seat image (enlarged image) of the rear seat in which the change is detected is displayed. So, when a change or abnormality is detected in the rear seat, the seat where the change or abnormality is detected is displayed in an enlarged manner, so that the driver can see a video that allows them to closely observe the location they want to check without performing any special operations. be able to.

  [9] According to the vehicle interior monitoring device 100 of the present embodiment, when information indicating that the driving support device 70 is operating is acquired, it is determined whether or not the driving state of the vehicle is a steady driving state. Since the standard is relaxed, in the scene where the driving support device 70 is activated and the driving load is reduced, the rear seat responds as much as possible to the driver's request for the rear seat, rather than the scene that bears the normal driving load. Can be presented.

  [10] In the vehicle interior monitoring device 100 of the present embodiment, when the driving support device 70 is in operation and information indicating that driving is required is acquired, the rear seat image is displayed on the display 30. Since this is prohibited, it is possible to prevent the driver's attention from being attracted in a scene where the driving support device 70 determines that the driver needs attention.

  The embodiment described above is described for facilitating the understanding of the present invention, and is not described for limiting the present invention. Therefore, each element disclosed in the above embodiment is intended to include all design changes and equivalents belonging to the technical scope of the present invention.

  That is, in this specification, although the vehicle interior monitoring system 1 provided with the vehicle interior monitoring device 100 and the vehicle-mounted device 200 according to the present invention will be described as an example, the present invention is not limited to this.

  In the present specification, a vehicle interior monitoring device 100 including a control device 10, a detection device 20, and a display 30 will be described as an example of a vehicle interior monitoring device according to the present invention. However, the present invention is not limited thereto. It is not a thing.

  In the present specification, as an example of a vehicle interior monitoring device according to the present invention including a rear seat video acquisition unit, a presentation unit, a traveling state determination unit, and a presentation control unit, a rear seat video acquisition function, The vehicle interior monitoring device 100 including the control device 10 that executes the determination function and the presentation control function and the display 30 will be described as an example, but is not limited thereto.

1. Car compartment monitoring system
DESCRIPTION OF SYMBOLS 100 ... Vehicle interior monitoring apparatus 10 ... Control apparatus 11 ... CPU
12 ... ROM
13 ... RAM
DESCRIPTION OF SYMBOLS 20 ... Detection apparatus 21, 21RR, 21RL ... Camera 22, 22RR, 22RL ... Microphone 30 ... Display 200 ... In-vehicle apparatus 50 ... Vehicle controller 60 ... Various sensors 70 ... Driving assistance apparatus 80 ... Navigation apparatus 90 ... Other vehicle detection apparatus

Claims (9)

A rear seat video acquisition means for acquiring a video of the passenger in the rear seat from a camera that captures the rear seat of the vehicle;
Presenting means for switching and presenting the acquired image of the rear seat and information other than the image of the rear seat to a passenger in the front seat of the vehicle;
Based on the travel information acquired from the vehicle, a travel state determination unit that determines whether or not the current travel state of the vehicle is a steady travel state that is defined in advance as a travel state in which a change in travel environment is small. When,
Presentation control means for switching information to be presented to the presenting means according to the determination result of the running state judging means;
Driving support operation information acquisition means for acquiring operation information of a driving support device that supports at least one driving operation of steering operation, accelerator operation, or braking operation of the vehicle ,
The presentation control means causes the presentation means to display an image of the rear seat when the running state judging means determines that the current running state of the vehicle is the steady running state, and the running state When the determination means determines that the current driving state of the vehicle is not the steady driving state but an unsteady driving state, prohibiting the display of the rear seat image on the presentation means ;
When the driving state determination unit acquires information indicating that the driving support device is operating from the driving support operation information acquisition unit, the driving state determination unit determines whether the driving state of the vehicle is the steady driving state. The determination criterion used for determination is characterized in that the upper threshold value of the determination criterion is changed higher and the lower threshold value is changed lower so that it is easier to determine that the current driving state is the steady driving state. Car interior monitoring device.   The travel state determination means is based on travel information of the vehicle, the vehicle is traveling straight, the vehicle is traveling at a constant speed, the vehicle is traveling at a low speed, 2. The vehicle interior according to claim 1, wherein when the distance between the vehicle and another vehicle satisfies at least one of a predetermined value and more, the vehicle interior is determined to be in the steady running state. Monitoring device. The traveling state determination means is configured to determine that the vehicle is traveling non-straight, that the vehicle is traveling at a non-constant speed, and that the vehicle is traveling at a non-low speed based on traveling information of the vehicle. The vehicle is determined to be in the unsteady running state when satisfying any one or more of being a vehicle or a distance between the vehicle and another vehicle being less than a predetermined value. The vehicle interior monitoring apparatus according to 2. Based on the traveling information of the vehicle, the vehicle is traveling straight, the vehicle is traveling at a constant speed, the vehicle is traveling at a low speed, or the vehicle and another vehicle Even if the distance of is satisfying any one or more of a predetermined value or more,
Based on the travel information of the vehicle, the vehicle is traveling non-straight, the vehicle is traveling non-constant speed, the vehicle is traveling non-low speed, and the vehicle The vehicle interior monitoring apparatus according to claim 2, wherein the vehicle interior monitoring apparatus determines that the vehicle is not in the steady traveling state when any one or more of a distance from another vehicle is less than a predetermined value is satisfied. Further comprising a change detecting means for detecting a change in an occupant in the rear seat of the vehicle,
The presentation control means causes the presentation means to display the video of the rear seat acquired by the rear seat video acquisition means when the change detection means detects a change in the occupant of the rear seat of the vehicle. The vehicle interior monitoring device according to any one of claims 1 to 4, wherein   The change detection means detects a change in the occupant of the rear seat of the vehicle based on the amount of movement of the occupant of the rear seat extracted from the image information of the video acquired by the rear seat video acquisition means. The vehicle interior monitoring device according to claim 5. Further comprising rear seat sound collecting means for acquiring sound generated at the rear seat of the vehicle;
The change detection means detects a change in the occupant of the rear seat of the vehicle based on the volume of the sound generated by the occupant of the rear seat extracted from the audio information acquired by the rear seat sound collection means. The vehicle interior monitoring device according to claim 5 or 6. The presentation control means includes
When displaying the image of the occupant in the rear seat, a wide area rear seat image that is a wide range image including the rear seat, and a partial range of the rear seat that is narrower than the range included in the wide area rear seat image It has a function to switch and display the narrow rear seat image that is the image of
If the passenger changes the seat after the vehicle has been detected by said change detecting means, claim, characterized in that a change to display the narrow-area rear seat image detected the rear seat 5-7 The vehicle interior monitoring device according to any one of the above. When the driving support operation information acquisition unit acquires information indicating that the driving support device is operating and driving needs attention, the presentation control unit displays the video of the rear seat as the presentation unit. The vehicle interior monitoring device according to any one of claims 1 to 8 , wherein the display is prohibited.

Images