JP5549571B2 - Vehicle status display device - Google Patents

Description

  The present invention relates to an in-vehicle situation display device that is suitable for use by a driver who drives a vehicle to check the situation inside the vehicle, particularly the situation in the rear seat of the vehicle.

  In recent years, when a child under 6 years old gets on a car, it is obliged to have the child sit on a child seat attached to the seat of the vehicle. In addition, it is recommended to install the child seat in the rear seat of the vehicle to ensure safety. In this way, when the parent driver drives the vehicle with the child seated on the child seat installed in the back seat, the driver can change the state of the child in the back seat without greatly shifting the line of sight from the front. It was difficult to confirm.

  For this reason, it is desired to develop technology that can confirm the situation of passengers in the vehicle, such as the situation of a child in the rear seat, without causing the driver of the vehicle to greatly change the line of sight from the front. Has been proposed (see, for example, Patent Documents 1 to 5).

  Specifically, in Patent Document 1, by operating a switch provided on the instrument panel, the rear seat situation is photographed by a camera, and the photographed image is arranged at a position where it can be seen by a passenger in the front seat. A technique for displaying on a display is described.

  In Patent Document 2, the situation of the rear seat is photographed by a camera module that can detect the entire interior of the vehicle installed in the center of the ceiling of the vehicle, and the photographed image is displayed on a display unit provided at the upper center of the dashboard or the like. The technology to display is described.

  Patent Document 3 relates to an occupant sitting in a rear seat by photographing a situation in a vehicle interior with a 3D camera installed around an inner rear view mirror of a vehicle and processing the photographed 3D video (three-dimensional video). A technique for extracting information is described.

  The technique described in Patent Document 4 is a technique for displaying information detected by various sensors such as a vibration sensor and a temperature sensor provided on a child seat on a display unit provided on an instrument panel or the like. A rear seat monitor mirror is provided to check the status of the rear seat.

  Patent Document 5 discloses a technique in which a mirror that reflects a front seat occupant or a rear seat occupant is installed on the ceiling of a vehicle, and each occupant reflected in the mirror is photographed by a camera disposed at the rear of the vehicle. Have been described.

JP 2003-025911 A JP 2003-104132 A JP 2008-261749 A JP 2009-137484 A JP 2010-125882 A

  In the techniques proposed so far, for example, the techniques described in Patent Documents 1 to 5, etc., switching used when switching the video displayed on the display unit such as the display to the video displaying the rear seat situation It is necessary to provide a switch. The driver can display the status of the rear seat on the display unit by operating the changeover switch.

  In such a case, the driver needs to turn his / her line of sight toward the changeover switch from the front to operate the changeover switch, which is not preferable when driving the vehicle safely. Furthermore, since it is necessary to separately provide a changeover switch, there is a problem that the number of switches increases and the load on the driver who performs the operation increases. In addition, when a changeover switch is separately provided, a switch body and parts related to the switch are additionally provided, which causes a problem that the manufacturing cost increases.

  In addition, when the security camera that shoots a person who has illegally entered the vehicle for the purpose of vehicle theft or vandalism and the camera that shoots the situation of the backseat are shared, the above-mentioned changeover switch is provided. If there is, there is a problem that an intruder may know the existence of the security camera.

  The present invention has been made to solve the above-described problem, and an object of the present invention is to provide an in-vehicle situation display device capable of controlling the in-vehicle situation displayed without increasing the number of switches used for operation. And

In order to achieve the above object, the present invention provides the following means.
The in-vehicle situation display device according to the present invention includes a storage unit that stores a predetermined gesture corresponding to control for displaying an image of the passenger compartment imaged on an imaging unit that captures an image of the passenger compartment of the vehicle on the display unit, An image signal relating to the passenger compartment output from the imaging unit is processed to identify an operator gesture included in the passenger compartment image, and the identified gesture matches the predetermined gesture stored in the storage unit A control unit that performs control to display the captured image of the passenger compartment on the display unit when the identified gesture matches the predetermined gesture, and a rear seat in the passenger compartment An illuminating unit that mainly illuminates the imaging unit, and the imaging unit is a part that performs the gesture for the operator and is disposed at a position for imaging a non-illuminated part that is not illuminated by the illuminating unit. And wherein the are.

  In this way, by displaying the image of the vehicle interior (situation in the vehicle interior) captured by the imaging unit, in particular, the status of the rear seat, on the display unit, the operator simply looks at the display unit without looking back. With this, it is possible to check the situation in the passenger compartment, particularly the situation in the rear seat.

Furthermore, since the operator can perform a predetermined gesture to display the vehicle interior condition on the display unit, there is no need to separately provide a switch for displaying the vehicle interior condition on the display unit. Further, since the operator does not need to operate the above-described switch when displaying the situation in the vehicle interior on the display unit, it is not necessary to direct the line of sight to the in-vehicle situation display device. In addition, the brightness of the rear seat portion in the image of the passenger compartment increases, and the operator can easily grasp the situation of the rear seat.
The in-vehicle situation display device according to the present invention includes a storage unit that stores a predetermined gesture corresponding to control for displaying an image of the passenger compartment imaged on an imaging unit that captures an image of the passenger compartment of the vehicle on the display unit, An image signal relating to the passenger compartment output from the imaging unit is processed to identify an operator gesture included in the passenger compartment image, and the identified gesture matches the predetermined gesture stored in the storage unit A control unit that controls to display the captured image of the passenger compartment on the display unit when the identified gesture matches the predetermined gesture, and the imaging unit The unit is capable of changing a region to be imaged, and the storage unit further stores a gesture related to a change corresponding to the control of changing the region to be imaged. It is determined whether or not the gesture related to the change stored in the storage unit matches, and if the identified gesture and the gesture related to the change match, the change of the area to be captured corresponding to the gesture related to the change A control signal for performing control is output to the imaging unit.
By doing in this way, the position which an operator desires, for example, images, such as a right backseat and a left backseat of a passenger compartment, can be displayed on a display part. In addition to changes such as panning to move the imaged area in the horizontal direction and tilt to move the imaged area up and down, zooming in and out of the area to be imaged can be performed. Changes are also included.
Furthermore, it is desirable to provide an illuminating unit that illuminates the interior of the passenger compartment, and to place the image capturing unit at a position where the operator performs a gesture and captures the illuminated part illuminated by the illuminating unit. For example, it is desirable that the emission direction of the illumination light in the illumination unit and the imaging direction in the imaging unit are the same direction. By doing in this way, the brightness | luminance of the part which performs the gesture in the image of a vehicle interior becomes high, and the control part can identify an operator's gesture easily from the image of a vehicle interior.

It is desirable that the captured image of the passenger compartment is controlled by the control unit so that the display automatically ends when displayed on the display unit for a predetermined period.
By doing in this way, the operator does not need to perform an operation to end the display of the captured image of the passenger compartment, and the burden on the operator can be reduced. Note that the predetermined period is preferably a period longer than the time required for the operator to check the cabin image displayed on the display unit.

  The storage unit stores a gesture related to the end corresponding to the control to end the display of the passenger compartment image on the display unit, and the control unit recognizes that the gesture identified from the image signal matches the gesture related to the end. It is desirable to perform control to end the display of the image of the passenger compartment on the display unit when it is determined whether or not to match.

  That is, the operator can end the display of the cabin image on the display unit by his / her own intention. By doing so, it is possible to prevent the operator from overlooking the displayed vehicle image, as compared with the case where the display is automatically ended after the vehicle cabin image is displayed for a predetermined period.

  It is desirable to limit the operator whose gesture is identified in the control unit to the driver of the vehicle. In this way, by limiting the operator to be identified to the driver, it is sufficient to identify the gesture only in the region where the driver is sitting, as compared with the case where the operator is not limited. The area in which the gesture is identified in the image is narrowed. Therefore, even if gesture identification is performed using a control unit having the same processing capability, more detailed gesture identification can be performed, and the gesture identification rate can be improved. Furthermore, since the region for gesture identification is limited, the algorithm used for gesture identification can be simplified.

  It is desirable that the gesture identified by the control unit is a gesture by a hand closer to the imaging unit among the hands of the operator. By limiting the gesture to the gesture by the hand of the operator in this way, it is sufficient to identify the gesture only in the range in which the driver's hand moves, compared to the case of not limiting to the gesture of the hand. Can be improved. Furthermore, since the hand is a part that can greatly change its shape compared to other parts of the body, such as goo, choki, par, etc., by making gestures by hand to be identified, gesture misrecognition In addition to being able to suppress, an algorithm used for gesture identification can be simplified.

  In particular, by limiting to gestures by the hand closer to the imaging unit among the hands of the operator, the gesture identification rate is further improved compared to identifying gestures by the hand far from the imaging unit Can be made. That is, when performing a gesture with a hand far from the imaging unit, it is conceivable that the hand on the far side is brought close to the imaging unit so that the image is easily captured by the imaging unit. In this case, the far side hand crosses the front side of the driver's body, and then the far side hand makes a gesture. On the other hand, when a gesture is performed with the hand closer to the imaging unit, the hand on the closer side does not cross the front of the driver's body. When the hand performing the gesture crosses the front of the body, it becomes difficult to continue identifying the hand performing the gesture in the control unit, and the possibility of misidentifying the gesture increases. For this reason, the gesture recognition rate can be improved by making the hand that performs the gesture a hand closer to the imaging unit that does not cross the front of the body.

  Also, if a gesture is performed with the hand far from the imaging unit approaching the imaging unit, the hand performing the gesture will cross the front of the driver's body, which may hinder the driving operation of the vehicle . On the other hand, when a gesture is performed with a hand closer to the imaging unit, the closer hand can be brought closer to the imaging unit without crossing the front of the driver's body.

  Furthermore, it is desirable to provide an illuminating unit that illuminates the interior of the passenger compartment, and to place the image capturing unit at a position where the operator performs a gesture and captures the illuminated part illuminated by the illuminating unit. For example, it is desirable that the emission direction of the illumination light in the illumination unit and the imaging direction in the imaging unit are the same direction. By doing in this way, the brightness | luminance of the part which performs the gesture in the image of a vehicle interior becomes high, and the control part can identify an operator's gesture easily from the image of a vehicle interior.

  When an unauthorized intrusion detection unit that detects unauthorized intrusion into a vehicle and a transmission unit that transmits an image signal to a receiving unit installed outside the vehicle, the control unit detects unauthorized intrusion from the unauthorized intrusion detection unit. When a detection signal output upon detection is input, it is desirable to output a control signal for transmitting an image signal to the reception unit to the transmission unit. In this way, a person who has illegally entered the vehicle can be imaged by the imaging unit, and an image signal of the unauthorized intruder can be sent to a receiving unit installed outside the vehicle.

  When the detection signal output when the unauthorized intrusion is detected from the unauthorized intrusion detection unit is input, the control unit prohibits the vehicle image captured by the imaging unit from being displayed on the display unit. It is desirable to do. In this way, it is possible to prevent an unauthorized intruder from knowing that an image has been captured by the imaging unit.

  According to the in-vehicle situation display apparatus of the present invention, the image signal output from the imaging unit is processed in the control unit, and the operator's gesture is identified from the captured image of the passenger compartment. The control unit determines whether the predetermined gesture stored in the storage unit and the identified gesture match, and if they match, outputs the image signal to the display unit, and the captured image of the passenger compartment Is displayed on the display unit. Therefore, the operator can check the state of the passenger compartment, particularly the state of the rear seat, only by looking at the display unit without looking back. Furthermore, since the operator can perform a predetermined gesture to display the vehicle interior condition on the display unit, there is no need to separately provide a switch for displaying the vehicle interior condition on the display unit. That is, it is possible to control the displayed vehicle situation without increasing the number of switches used for the operation.

It is a schematic diagram explaining the structure of the backseat condition display apparatus which concerns on the 1st Embodiment of this invention. It is a figure explaining the state by which the camera of FIG. 1 is arrange | positioned on the ceiling of a vehicle interior. It is a schematic diagram explaining arrangement | positioning of the camera of FIG. 1, and an illumination part. It is a figure explaining the state in which the image inside a compartment is displayed on the display apparatus of FIG. It is a flowchart explaining the operation | movement in the backseat condition display apparatus of FIG. It is a figure which shows an example of the image of the compartment imaged with the camera of FIG. It is a figure which shows the example of the gesture performed with respect to the backseat condition display apparatus of FIG. It is a figure which shows the example of the gesture performed with respect to the backseat condition display apparatus of FIG. It is a figure which shows the example of the gesture performed with respect to the backseat condition display apparatus of FIG. It is a figure which shows the example of the gesture performed with respect to the backseat condition display apparatus of FIG. It is a figure which shows the example of the gesture which may be misrecognized. It is a figure which shows the example of the operation | movement which may be recognized accidentally as a gesture. It is a figure which shows the example of the operation | movement which may be recognized accidentally as a gesture. It is a figure which shows the other example of the gesture performed with respect to the backseat condition display apparatus of FIG. It is a schematic diagram explaining the other arrangement | positioning aspect in the display apparatus of FIG. It is a schematic diagram explaining schematic structure of the backseat condition display apparatus which concerns on the modification of the 1st Embodiment of this invention. It is a schematic diagram explaining the arrangement | positioning state of the display apparatus of FIG. It is a flowchart explaining the control in the backseat condition display apparatus which concerns on 2nd Embodiment of this invention. It is a flowchart explaining the control in the backseat condition display apparatus which concerns on the 3rd Embodiment of this invention. It is a schematic diagram explaining arrangement | positioning of the illumination part 230 in the backseat condition display apparatus 1 which concerns on the 4th Embodiment of this invention. It is a schematic diagram explaining the whole structure of the backseat condition display apparatus which concerns on the 5th Embodiment of this invention. It is a flowchart explaining control of the imaging region by the camera in the rear seat status display apparatus of FIG. It is a figure which shows the gesture which instruct | indicates a pan operation | movement to a camera.

[First Embodiment]
Hereinafter, the rear seat status display device 1 according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 15. FIG. 1 is a schematic diagram illustrating the configuration of a rear seat status display device 1 according to the present embodiment.

  A rear seat status display device (in-vehicle status display device) 1 according to the present embodiment is used when a passenger in a front seat, particularly a driver, grasps a vehicle interior condition in a vehicle, particularly a rear seat condition. Further, the rear seat status display device 1 of the present embodiment uses a security system that images a person who has illegally entered the vehicle and transmits it to the outside, and a person who owns the vehicle (a regular user) When not in use, it also works as a security system.

  As shown in FIG. 1, the rear seat status display device 1 includes an ECU (Electronic Control Unit) 10, a camera (imaging unit) 20, an illumination unit 30, and a display device (display unit) 40. A communication device (communication unit) 50 and an unauthorized intrusion detection device (unauthorized intrusion detection unit) 60 are mainly provided.

  The ECU 10 plays a central role in the rear seat status display device 1 and controls the display device 40 to display an image captured by the camera 20. Further, when the vehicle is not used by a regular user, the security system is also controlled. The ECU 10 is mainly provided with a CPU (control unit) 11, a GPU (control unit) 12, a memory (storage unit) 13, and a video output 14.

  A CPU (Central Processing Unit) 11 is an electronic circuit that performs signal calculation processing, performs calculations related to various controls in the rear seat status display device 1, and the vehicle is used by authorized users. If not, calculations related to various controls in the security system are performed.

  A GPU (Graphics Processing Unit) 12 is an electronic circuit that performs signal processing similar to the CPU 11, and performs image signal processing such as gesture extraction. Detailed control contents by the CPU 11 and the GPU 12 will be described later.

  The memory 13 is a semiconductor storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory), and stores a program used for arithmetic processing in the CPU 11 or the GPU 12. Further, the memory 13 stores in advance a table that associates various control contents in the rear seat status display device 1 with various gestures described later.

  The video output 14 is an interface that outputs an image signal output from the camera 20 and converted to a format that can be displayed on the display device 40 to the display device 40. Output of signals from the video output 14 to the display device 40 is controlled based on a control signal output from the CPU 11.

FIG. 2 is a diagram illustrating a state in which the camera 20 of FIG. 1 is arranged on the ceiling of the passenger compartment. FIG. 3 is a schematic diagram illustrating the arrangement of the camera 20 and the illumination unit 30 in FIG.
As shown in FIGS. 2 and 3, the camera 20 is arranged at the front center of the ceiling of the passenger compartment, and images the passenger compartment of the vehicle. In the present embodiment, the camera 20 is described as being applied to a camera having a fisheye lens having a wide-angle imaging range and capable of imaging the entire region including the front seat and the rear seat of the passenger compartment. Furthermore, the camera 20 is also used as an image pickup means of the security system, and also picks up an image of a person who has illegally entered the vehicle.

  As shown in FIGS. 1 and 3, the illumination unit 30 illuminates the interior of the vehicle and is disposed at the front center of the ceiling of the vehicle. The illumination unit 30 is arranged in a posture to emit illumination light toward the vehicle interior, and the direction of the illumination light emitted from the illumination unit 30 is the same as the optical axis direction of the camera 20 as shown in FIG. The direction from the front upper side to the rear lower side of the vehicle). The camera 20 captures at least an illumination area irradiated with illumination light in the left hand performing the gesture. The illumination light emitted from the illumination unit 30 may be visible light, infrared light, or may include both, and is not particularly limited.

FIG. 4 is a diagram illustrating a state in which an image inside the passenger compartment is displayed on the display device 40 of FIG.
As shown in FIGS. 1 and 4, the display device 40 displays an image of the interior of the passenger compartment captured by the camera 20. In the present embodiment, the display device 40 is a device arranged on the instrument panel of the vehicle (a so-called meter multi-screen). When the image inside the passenger compartment is not displayed, various display modes such as the driving state of the vehicle are provided. Description will be made by applying to an example in which information is displayed.

  The communication device 50 is a device mainly used in a security system, and transmits an image signal wirelessly to a receiving station (receiving unit) 51 installed outside the vehicle as shown in FIG. is there. Transmission of the image signal from the communication device 50 is controlled based on a control signal output from the CPU 11.

  The receiving station 51 transmits the received image signal to, for example, a service center of a company that provides security services for vehicles. The destination to which the image signal received by the receiving station 51 is transmitted may be arbitrarily set by a regular user of the vehicle, and is not particularly limited.

The unauthorized intrusion detection device 60 is a device mainly used in a security system. The unauthorized intrusion detection device 60 outputs a detection signal when detecting unauthorized intrusion into the vehicle.
Examples of the unauthorized intrusion detection device 60 include various sensors such as an infrared sensor, an ultrasonic sensor, an acceleration sensor, a pyroelectric sensor, a vibration sensor, an impact sensor, a sound pressure sensor, and a glass breakage detection sensor. The unauthorized intrusion detection device 60 uses any of these sensors to detect intrusion behavior such as opening a door or crushing a window.

  Between the ECU 10, the display device 40, the communication device 50, and the unauthorized intrusion detection device 60, an in-vehicle communication network 15 that enables communication of data and signals among them is provided. As the in-vehicle communication network 15, a known in-vehicle network such as CAN (Controller Area Network) can be used, and is not particularly limited.

Next, the operation of the rear seat status display device 1 having the above-described configuration will be described. FIG. 5 is a flowchart for explaining the operation of the rear seat status display device 1 of FIG.
The rear seat status display device 1 starts operating when a driver who is a regular user of the vehicle gets into the vehicle and the engine of the vehicle is started. Here, the timing at which the rear seat status display device 1 starts to operate may be when the vehicle engine is started, as described above, or until the position of the accessory ON is reached by inserting the engine key into the insertion slot. It may be when the vehicle is rotated, when the vehicle door is unlocked, when the door is opened or closed, or when the accelerator is stepped on It may be when the vehicle speed is no longer 0 (when driving is started), and is not particularly limited.

  Since the ECU 10 of the rear seat status display device 1 also controls the security system, electric power is supplied to the ECU 10 while the operation of the rear seat status display device 1 is stopped. Control is in progress.

  When the rear seat status display device 1 starts operating, as shown in FIG. 5, the CPU 11 of the ECU 10 performs control for starting imaging of the vehicle interior by the camera 20 (S11). The camera 20 outputs the captured image of the passenger compartment as an image signal, and the image signal is input to the ECU 10.

  The GPU 12 of the ECU 10 performs a process of recognizing the driver's gesture from the passenger compartment image captured by the camera 20 by performing arithmetic processing on the input image signal (S12). Then, it is determined whether or not the gesture has been recognized by the gesture recognition process by the GPU 12 (S13). If the gesture has not been recognized (in the case of NO), the process returns to S12 and the gesture recognition process is repeated.

  When the gesture is recognized by the gesture recognition process (in the case of YES), the CPU 11 determines whether or not the recognized gesture is a gesture for requesting the display of the rear seat of the vehicle (S14). The determination of the gesture is performed based on whether or not the gesture for requesting the display of the rear seat of the vehicle stored in the memory 13 matches the recognized gesture. When the recognized gesture does not match the gesture requesting the display of the rear seat of the vehicle (in the case of NO), the CPU 11 does not perform new control on the display device 40, that is, the display device 40 The displayed contents are displayed as they are (S15), and the process returns to S12 described above to repeat the same operation.

  When the recognized gesture matches the gesture requesting the display of the rear seat of the vehicle (in the case of YES), the CPU 11 performs control to display the vehicle interior image captured by the camera 20 on the display device 40 ( S16). Specifically, a control signal for stopping the output of the image signal is output to the display device 40 until the device that has output the signal of the image to be displayed, and from the video output 14 of the ECU 10. Then, the image signal of the passenger compartment imaged by the camera 20 is output. The signal of the passenger compartment image is input to the display device 40, and the passenger compartment image is displayed on the display device 40. Thereafter, the process returns to S12 described above, and the same operation is repeated.

Here, processing for recognizing a gesture in the GPU 12 will be described.
The GPU 12 recognizes a gesture by the driver's hand on the camera 20 side (hand on the center side of the vehicle, or left hand in the case of a right-hand drive vehicle) by processing the input image signal. More specifically, the shape of the finger is extracted from the passenger compartment image captured by the camera 20. Shape extraction methods include detecting and extracting finger edges in images, extracting them by matching with pre-stored finger shape templates, and matching with finger joint models. Various methods such as an extraction method can be used.

  In addition, as a method of determining whether the hand gestured by the driver is the right hand or the left hand, the finger shape template of the left hand is used as the finger shape template, and matching with the finger shape of the recognized hand is performed. The difference between the temporally adjacent images in the passenger compartment image captured by the camera 20 is obtained, and the determination is made based on which side of the driver the hand that made the gesture has moved. It is possible to use a method, a method of using an optical flow that represents a hand movement in a temporally continuous image as a vector, or the like.

FIG. 6 is a diagram illustrating an example of a passenger compartment image captured by the camera 20 of FIG. 7 to 10 are diagrams showing examples of gestures performed on the rear seat status display device 1 of FIG.
The passenger compartment image captured by the camera 20 is, for example, an image as shown in FIG. The driver is sitting on the left side and holding the steering wheel, and the passenger is sitting on the right side. The GPU 12 performs processing for recognizing any gesture shown in FIGS. 7 to 10 performed by the driver. One of the gestures shown in FIGS. 7 to 10 or a plurality of gestures may be stored in the memory 13 as a gesture for requesting the display of the rear seat of the vehicle.

  Here, what is shown in FIG. 7 is a gesture in which five fingers are opened (a so-called par state) and the palm is directed to the camera 20, and what is shown in FIG. It is a gesture that closes (so-called goo state) and points the fist toward the camera 20. What is shown in FIG. 9 is a gesture in which the index finger and middle finger are opened and the other fingers are closed (so-called choking state) and the palm side is directed to the camera 20, and FIG. A gesture in which the index finger, the middle finger, and the ring finger are opened, the other fingers are closed, and the opened finger is directed sideways toward the camera 20.

FIG. 11 is a diagram illustrating examples of gestures that may be erroneously recognized, and FIGS. 12 and 13 are diagrams illustrating examples of operations that may be erroneously recognized as gestures.
In addition, it is preferable that the gesture memorize | stored in the memory 13 excludes the gesture which approximates the other operation | movement which a driver | operator performs in a passenger compartment from a viewpoint of preventing misrecognition of a gesture. For example, the gesture shown in FIG. 11 in which the index finger is opened, the other fingers are closed, the index finger is directed upward, and the palm side is directed to the camera 20 is preferably excluded. As shown in FIG. 12, this gesture is similar to an operation in which a driver operates a switch arranged in front of the ceiling of the passenger compartment, such as a map lamp switch. Also, FIG. 12 shows an operation in which the driver reaches for the rearview mirror in order to adjust the direction of the rearview mirror used for checking the rear of the vehicle. It is preferable that the gesture approximated to the shape of the finger at this time is also excluded from the gesture stored in the memory 13.

  Further, as a method of preventing erroneous recognition of a gesture, a method of providing an interval period for stopping recognition of another gesture after the GPU 12 recognizes one gesture is conceivable. In this way, when the driver performs a gesture intended to input control to the rear seat status display device 1 and then returns to the driving operation, the shape of a finger corresponding to another gesture is accidentally formed. Even if it is made, the other gesture is not recognized by the GPU 12 of the rear seat status display device 1.

  According to said structure, the driver | operator who looks back on the display apparatus 40 displays the image (situation in a vehicle interior) imaged with the camera 20, especially the condition of a backseat on the display apparatus 40, and looks back. The situation in the passenger compartment, especially the situation in the rear seat, can be confirmed by simply looking at the display device 40.

  Furthermore, since the driver can make a predetermined gesture to display the vehicle interior condition on the display device 40, there is no need to separately provide a switch for displaying the vehicle interior condition on the display device 40. In other words, the state of the passenger compartment can be displayed on the display device 40 without increasing the number of switches used for operating the rear seat status display device 1. Further, since the driver does not need to operate a switch when displaying the vehicle interior state on the display device 40, the driver does not need to turn his / her line of sight toward the rear seat state display device 1 and can easily perform the rear seat state display device. 1 can be operated.

  As in the rear seat status display device 1 of the present embodiment, by limiting the operator to be identified to the driver, compared to the case where the operator is not limited, only in the region where the driver is sitting. Gesture identification may be performed, and the area in which the gesture identification is performed in the passenger compartment image is narrowed. Therefore, even if gesture identification is performed using the GPU 12 having the same processing capability, it is possible to perform more detailed gesture identification and improve the gesture identification rate. Furthermore, since the region for gesture identification is limited, the algorithm used for gesture identification can be simplified.

  By limiting the gestures identified by the GPU 12 to gestures by the driver's hand, it is sufficient to identify the gestures only within the range in which the driver's hand moves compared to the case where the gestures are not limited to hand gestures. The identification rate can be improved. Furthermore, since the hand is a part that can greatly change its shape compared to other parts of the body, such as goo, choki, par, etc., by making gestures by hand to be identified, gesture misrecognition In addition to being able to suppress, an algorithm used for gesture identification can be simplified.

  In particular, by limiting to gestures by the hand closer to the camera 20 among the driver's hands, the gesture identification rate is further improved as compared with the case of identifying the gesture by the hand far from the camera 20. Can be made. That is, when performing a gesture with a hand far from the camera 20, it is conceivable that the hand far from the camera 20 is brought close to the camera 20 so that the camera 20 can easily capture an image. In this case, the far side hand crosses the front side of the driver, and then the far side hand makes a gesture. If the hand performing the gesture crosses the front of the body, it becomes difficult to continue identifying the hand performing the gesture in the GPU 12, and the possibility of misidentifying the gesture increases. For this reason, the gesture recognition rate can be prevented from being lowered by making the hand performing the gesture a hand closer to the camera 20 that does not cross the front of the body.

  In addition, when a gesture is performed with a hand far from the camera 20 being brought close to the camera 20, the hand performing the gesture crosses in front of the driver's body, which may hinder the driving operation of the vehicle. . On the other hand, when performing a gesture with the hand closer to the camera 20, the hand on the closer side can be brought closer to the camera 20 without crossing the front of the driver's body, thereby hindering the driving operation. There is no.

  Since the emission direction of the illumination light in the illumination unit 30 and the imaging direction in the camera 20 are the same direction, the illumination part illuminated by the illumination unit 30 is a part of the hand that makes a gesture for the driver. Imaged. By doing in this way, since the brightness | luminance of the part of the hand which performs the gesture in a vehicle interior image becomes high, GPU12 can identify a driver | operator's gesture easily from a vehicle interior image.

FIG. 14 is a diagram illustrating another example of a gesture performed on the rear seat status display device 1 of FIG.
Note that, as in the above-described embodiment, the driver may perform a gesture with the left hand at a position that is moderately separated from the camera 20, or the camera 20 and the illumination unit 30 with the left hand as shown in FIG. It may be covered. In this way, the entire shape of the left hand cannot be captured by the camera 20, but the luminance of the left hand portion in the image captured by the camera 20 is particularly high. Therefore, the left hand part can be easily extracted from the image captured by the GPU 12. For example, every time the camera 20 and the illumination unit 30 are covered with the left hand, it is possible to start and end the predetermined control.

FIG. 15 is a schematic diagram for explaining another arrangement mode in the display device 40 of FIG.
The display device 40 may be provided on the instrument panel as shown in FIG. 4, or a head-up display arranged above the instrument panel as shown in FIG. (HUD) may be used and is not particularly limited.

  In the above-described embodiment, description has been made by applying to an example in which gesture identification is performed in the GPU 12, but the gesture identification may be performed by the CPU 11, an electronic circuit that performs other arithmetic processing, or the like, and is particularly limited. It is not a thing.

[Modification of First Embodiment]
Next, a modification of the first embodiment of the present invention will be described with reference to FIGS.

  The basic configuration of the rear seat status display device of this embodiment is the same as that of the first embodiment. However, the first embodiment further includes a car navigation system (hereinafter referred to as “car navigation”). The difference is that the rear seat status display device is integrated. In addition, about the component etc. which are the same as 1st Embodiment, the same code | symbol is attached | subjected and the description is abbreviate | omitted.

FIG. 16 is a schematic diagram illustrating a schematic configuration of the rear seat status display device 101 according to the present modification.
The rear seat status display device (in-vehicle status display device) 101 of this modified example uses a car navigation system of a vehicle, and functions as a car navigation system while not being used as the rear seat status display device 101. Note that the rear seat status display device 101 of this modification can also be used as a security system, as in the first embodiment. As shown in FIG. 16, the rear seat status display device 101 is mainly provided with a main body 110, a camera 20, a lighting unit 30, and an unauthorized intrusion detection device 60.

  The main body 110 plays a central role of the rear seat status display device 101 and displays an image captured by the camera 20 on the display device 140 having a built-in function. Further, when the rear seat status display device 101 is not used, it also functions as a car navigation system. The main body 110 mainly includes a CPU (control unit) 111, a GPU (control unit) 112, a memory (storage unit) 113, a display device (display unit) 140, and a communication device (communication unit) 150. Is provided.

  The CPU 111 is an electronic circuit that performs signal calculation processing, performs calculations related to various controls in the rear seat status display device 101, and when the rear seat status display device 101 is not used, Calculations related to control are performed.

  The GPU 112 is an electronic circuit that performs signal calculation processing in the same manner as the CPU 111, and performs image signal calculation processing such as gesture extraction. When the rear seat status display device 101 is not used, a calculation process relating to an image (for example, a map image) displayed when operating as a car navigation system is performed. Detailed control contents by the CPU 111 and the GPU 112 will be described later.

  The memory 113 is a semiconductor storage device such as a RAM or a ROM, and stores a program used for arithmetic processing when the CPU 111 or the GPU 112 operates as the rear seat status display device 101 or arithmetic processing when operating as a car navigation system. Is. Furthermore, the memory 113 stores in advance a table that associates various control contents in the rear seat status display device 101 with various gestures described later.

FIG. 17 is a schematic diagram illustrating an arrangement state of the display device 140 of FIG.
As shown in FIGS. 16 and 17, when the display device 140 functions as the rear seat status display device 101, the display device 140 displays an image inside the passenger compartment imaged by the camera 20, and functions as a car navigation system. When doing so, an image such as a map image is displayed. In the present embodiment, the display device 140 is configured integrally with the main body 110 and will be described as applied to an example in which the display device 140 is disposed in a center console located between a driver seat and a passenger seat as shown in FIG.

  The communication device 150 is a device mainly used for car navigation. As shown in FIG. 16, the communication device 150 wirelessly communicates with a receiving station 51 or a GPS satellite (not shown) installed outside the vehicle. Is used to transmit and receive image signals and various types of information.

Next, since the operation of the rear seat status display device 101 having the above-described configuration is the same as that of the first embodiment, the description thereof is omitted.
As described above, by integrating the rear seat status display device 101 with the car navigation system, it is possible to use calculation resources such as a CPU and a GPU mounted on the car navigation system for the rear seat status display device 101. In other words, the function as the rear seat status display device 101 can be realized without providing a dedicated ECU for the rear seat status display device 101, and the number of parts can be reduced.

  On the other hand, when the function as the rear seat status display device 1 is realized using a dedicated ECU as in the rear seat status display device 1 of the first embodiment, the rear seat is installed in a vehicle not equipped with a car navigation system. A status display device 1 can be provided. Furthermore, the rear seat status display device 1 can be provided later on a vehicle that is already equipped with a car navigation system.

[Second Embodiment]
Next, a second embodiment of the present invention will be described with reference to FIG.
The basic configuration of the rear seat status display device of this embodiment is the same as that of the first embodiment, but the control after displaying the passenger compartment image on the display device is different from the first embodiment. . Therefore, in this embodiment, the control in the rear seat status display device will be described with reference to FIG. 18, and description of other components and the like will be omitted.

FIG. 18 is a flowchart illustrating control in the rear seat status display device 1 according to the present embodiment.
Similarly to the first embodiment, in the rear seat status display apparatus 1 of the present embodiment, when a driver who is a regular user of the vehicle gets into the vehicle and the engine of the vehicle is started, the operation is started. From the control (S11) for starting the operation of the rear seat status display device 1 to start the imaging of the passenger compartment by the camera 20 (S11) to the control for displaying the passenger compartment image on the display device 40 (S16). Since it is the same as that of 1st Embodiment, the description is abbreviate | omitted.

  The CPU 11 of the ECU 10 outputs a control signal for starting the measurement of elapsed time to a timer (not shown) at the same time as outputting a control signal for displaying an image of the passenger compartment on the display device 40. Thereafter, the CPU 11 determines whether or not the elapsed time measured by the timer has passed a predetermined period stored in the memory 13 (S117). When the elapsed time measured by the timer does not pass the predetermined period (in the case of NO), the elapsed time measurement by the timer is continued.

  When the elapsed time measured by the timer has passed a predetermined period (in the case of YES), the CPU 11 outputs a control signal for terminating the display of the cabin image on the display device 40 to the video output 14. At the same time, a control signal for displaying the image displayed on the display device 40 on the display device 40 before displaying the image of the passenger compartment is output (S118). Thereafter, the process returns to S12 described above, and the same operation is repeated.

  According to the above configuration, the driver does not need to perform an operation to end the display of the passenger compartment image on the display device 40, and therefore, the driver's burden of an operation to end the display can be reduced. The predetermined period is preferably a period longer than the time required for the driver to check the passenger compartment image displayed on the display device 40. Specifically, a period of about 2 seconds to 3 seconds can be exemplified. Further, the predetermined period may be changed to a desired length by a driver who is an operator or the like.

[Third Embodiment]
Next, a third embodiment of the present invention will be described with reference to FIG.
The basic configuration of the rear seat status display device of this embodiment is the same as that of the first embodiment, but the control after displaying the passenger compartment image on the display device is different from the first embodiment. . Therefore, in the present embodiment, control in the rear seat status display device will be described using FIG. 19, and description of other components and the like will be omitted.

FIG. 19 is a flowchart illustrating control in the rear seat status display device 1 according to the present embodiment.
Similarly to the first embodiment, in the rear seat status display apparatus 1 of the present embodiment, when a driver who is a regular user of the vehicle gets into the vehicle and the engine of the vehicle is started, the operation is started. From the control (S11) for starting the operation of the rear seat status display device 1 to start the imaging of the passenger compartment by the camera 20 (S11) to the control for displaying the passenger compartment image on the display device 40 (S16). Since it is the same as that of 1st Embodiment, the description is abbreviate | omitted.

  When the GPU 12 of the ECU 10 is displayed as an image of the passenger compartment on the display device 40, the GPU 12 of the ECU 10 starts to recognize the gesture by the hand on the camera 20 side of the driver by processing the input image signal (S217). Then, it is determined whether or not the gesture is recognized by the gesture recognition process by the GPU 12 (S218). If the gesture is not recognized (in the case of NO), the process returns to S217, and the gesture recognition process is repeated.

  If the gesture is recognized by the gesture recognition process (in the case of YES), the CPU 11 determines whether or not the recognized gesture is a gesture for requesting the end of the display of the rear seat of the vehicle (S219). Here, the gesture for requesting the end of the display of the rear seat of the vehicle is a gesture different from the gesture for requesting the display of the rear seat of the vehicle.

  The determination of the gesture is performed based on whether or not the gesture for requesting the end of the display of the rear seat of the vehicle stored in the memory 13 matches the recognized gesture. If the recognized gesture does not match the gesture for requesting the end of the display of the rear seat of the vehicle (in the case of NO), the process returns to S217, and the gesture recognition process is repeated.

  When the recognized gesture matches the gesture requesting the end of the display of the rear seat of the vehicle (in the case of YES), the CPU 11 displays a control signal for terminating the display of the cabin image on the display device 40. Output to output 14. At the same time, a control signal for displaying the image displayed on the display device 40 on the display device 40 before displaying the image of the passenger compartment is output (S118). Thereafter, the process returns to S12 described above, and the same operation is repeated.

  According to said structure, a driver | operator can complete | finish the display of the image of the compartment on the display apparatus 40 by own intention. In this way, it is possible to prevent the driver from overlooking the displayed vehicle image, as compared with the case where the display is automatically ended after the vehicle cabin image is displayed for a predetermined period.

[Fourth Embodiment]
Next, a fourth embodiment of the present invention will be described with reference to FIG.
The basic configuration of the rear seat status display device of this embodiment is the same as that of the first embodiment, but the arrangement of the illumination units is different from that of the first embodiment. Therefore, in this embodiment, arrangement | positioning of an illumination part, etc. are demonstrated using FIG. 20, and description of other components is abbreviate | omitted.

FIG. 20 is a schematic diagram for explaining the arrangement of the illumination units 230 in the rear seat status display device 1 according to the present embodiment.
As shown in FIG. 20, the illumination unit 230 of the rear seat status display device 1 according to the present embodiment is disposed approximately in the center of the ceiling of the passenger compartment, and illuminates the passenger compartment, particularly the rear seat. is there. The illumination light emitted from the illumination unit 230 illuminates a region that is not captured by the camera 20 in the left hand that performs the driver's gesture. In other words, the camera 20 captures an image of a non-illuminated area that is not irradiated with illumination light, of the left hand that performs the gesture. The illumination light emitted from the illumination unit 230 may be visible light, infrared light, or may include both, and is not particularly limited. According to said structure, the brightness | luminance of the backseat part in a passenger compartment image becomes high, and a driver | operator becomes easy to grasp | ascertain the condition of a backseat.

[Fifth Embodiment]
Next, a fifth embodiment of the present invention will be described with reference to FIGS.
The basic configuration of the rear seat status display device of this embodiment is the same as that of the first embodiment, but is different from the first embodiment in that the imaging area by the camera is controlled. Therefore, in this embodiment, the change of the imaging area by the camera will be described with reference to FIGS. 21 to 23, and description of other components and the like will be omitted.

  FIG. 21 is a schematic diagram illustrating the overall configuration of the rear seat status display device according to the present embodiment. As shown in FIG. 21, the rear seat status display device 301 of the present embodiment includes an ECU 310, a camera (imaging unit) 320, an illumination unit 30, a display device 40, a communication device 50, and an unauthorized intrusion detection device. 60 is mainly provided.

  The ECU 310 plays a central role in the rear seat status display device 301 and controls the display device 40 to display an image captured by the camera 320. It is also what controls. The ECU 310 is mainly provided with a CPU (control unit) 311, a GPU 12, a memory (storage unit) 313, and a video output 14.

  The CPU 311 is an electronic circuit that performs signal calculation processing, performs calculations related to various controls in the rear seat status display device 301, and performs calculations related to control of an area captured by the camera 320.

  The memory 313 is a semiconductor storage device such as a RAM or a ROM, and stores a program used for arithmetic processing in the CPU 311 or the GPU 12. Furthermore, the memory 313 stores in advance a table that associates various control contents in the rear seat status display device 301, control of an area captured by the camera 320, which will be described later, and various gestures (gestures related to change). Has been.

  The camera 320 is arranged at the front center on the ceiling of the passenger compartment, and images the passenger compartment of the vehicle. In the present embodiment, the camera 320 performs panning for moving the imaging range in the left-right direction, tilt operation for moving up and down, and enlargement and reduction of the imaging range based on a control signal output from the CPU 311. The zoom operation to be performed can be performed.

  Next, control of the imaging area by the camera 320, which is a feature of the present embodiment, will be described. FIG. 22 is a flowchart for explaining control of the imaging region by the camera in the rear seat status display device of FIG. The control until the passenger compartment image captured by the camera 320 is displayed on the display device 40 is the same as in the first embodiment, the second embodiment, and the third embodiment. The description is omitted.

  When an image of the passenger compartment imaged by the camera 320 is displayed on the display device 40, the GPU 12 performs an arithmetic process on the input image signal, so that the driver's gesture is obtained from the passenger compartment image captured by the camera 320. Is started (S301). Then, it is determined whether or not the gesture has been recognized by the gesture recognition process by the GPU 12 (S302). If the gesture has not been recognized (in the case of NO), the process returns to S301, and the gesture recognition process is repeatedly performed.

  When the gesture can be recognized by the gesture recognition process (in the case of YES), the CPU 311 determines whether the recognized gesture is a gesture for instructing the camera 320 to control the pan operation. (S303). If it is determined that the recognized gesture is a gesture for instructing the pan operation (in the case of YES), the CPU 311 outputs a control signal for causing the camera 320 to perform the pan operation (S304).

  If the recognized gesture is not determined to be a gesture for instructing the pan operation (in the case of NO), the CPU 311 performs a gesture for instructing the camera 320 to control the tilt operation with respect to the camera 320. It is determined whether or not (S305). If the recognized gesture is determined to be a gesture for instructing a tilt operation (in the case of YES), the CPU 311 outputs a control signal for causing the camera 320 to perform a tilt operation (S304).

  If the recognized gesture is not determined as a gesture for instructing a tilt operation (in the case of NO), the CPU 311 performs a gesture for instructing the camera 320 to control the zoom operation. It is determined whether or not (S307). If the recognized gesture is determined to be a gesture for instructing a zoom operation (in the case of YES), the CPU 311 outputs a control signal for causing the camera 320 to perform a zoom operation (S308).

  When the recognized gesture is not determined to be a gesture for instructing the zoom operation (in the case of NO), after the output of the control signal for performing the pan operation, after the output of the control signal for performing the tilt operation, or the zoom operation After the output of the control signal for performing the above, the display of the passenger compartment image captured by the camera 320 on the display device 40 is continued (S309).

FIG. 23 is a diagram illustrating a gesture for instructing the camera 320 to perform a pan operation.
As a gesture for instructing the camera 320 to perform a panning operation, as shown in FIG. 23, a gesture in which the index finger is opened and the index finger is directed in the direction of panning the camera 320 can be exemplified. Further, as a gesture for instructing the camera 320 to perform a tilting operation, as shown in FIG. 11, a gesture in which an index finger is opened and the index finger is directed in a direction in which the camera 320 is tilted can be exemplified.

  Furthermore, as a gesture for instructing the camera 320 to perform a zoom operation, for example, a 5 × zoom operation, a gesture in which five fingers are opened can be exemplified as shown in FIG. 7. As a gesture for simultaneously instructing a pan operation and a three times zoom operation as a combination of these operations, as shown in FIG. 10, three fingers are opened, and the opened fingers are directed in the direction of panning the camera 320. Gestures can be exemplified.

According to the above configuration, an image of a position desired by the driver, for example, the right rear seat or the left rear seat of the passenger compartment can be displayed on the display device 40.
Note that the camera 320 itself may perform a pan operation, a tilt operation, and a zoom operation as in the above-described embodiment, and the camera 20 itself is fixed as in the first embodiment. The image displayed on the display device 40 by image processing may be panned, tilted, or zoomed, and is not particularly limited.

  DESCRIPTION OF SYMBOLS 1,101,301 ... Backseat situation display apparatus (in-vehicle situation display apparatus), 11, 111, 311 ... CPU (control part), 112, 112 ... GPU (control part), 13, 113, 313 ... Memory (storage part) ), 20, 320 ... Camera (imaging unit), 30, 230 ... Illumination unit, 40, 140 ... Display device (display unit), 50, 150 ... Communication device (communication unit), 60 ... Unauthorized intrusion detection device (unauthorized intrusion) Detection unit), 51... Receiving station (receiving unit)

Claims (9)

A storage unit in which a predetermined gesture corresponding to control for displaying on the display unit the image of the passenger compartment imaged on the imaging unit that images the passenger compartment of the vehicle;
An image signal relating to the passenger compartment output from the imaging unit is processed to identify an operator gesture included in the passenger compartment image, and the identified gesture is stored in the storage unit and the predetermined gesture Determine if they match,
A control unit that performs control to display the captured image of the passenger compartment on the display unit when the identified gesture matches the predetermined gesture;
An illumination unit which mainly illuminate a rear seat in the passenger compartment,
Is provided,
The in-vehicle situation display device, wherein the imaging unit is a portion that performs the gesture for the operator, and is disposed at a position for imaging a non-illuminated portion that is not illuminated by the illumination unit. A storage unit in which a predetermined gesture corresponding to control for displaying on the display unit the image of the passenger compartment imaged on the imaging unit that images the passenger compartment of the vehicle;
An image signal relating to the passenger compartment output from the imaging unit is processed to identify an operator gesture included in the passenger compartment image, and the identified gesture is stored in the storage unit and the predetermined gesture Determine if they match,
A control unit that performs control to display the captured image of the passenger compartment on the display unit when the identified gesture matches the predetermined gesture;
Is provided,
The imaging unit can change the area to be imaged,
The storage unit further stores a gesture related to the change corresponding to the control of the change of the area to be imaged,
The control unit determines whether the identified gesture matches the gesture related to the change stored in the storage unit;
When the identified gesture matches the gesture related to the change, a control signal for controlling the change of the area to be imaged corresponding to the gesture related to the change is output to the imaging unit. Vehicle status display device. An illumination unit for illuminating the interior of the passenger compartment is further provided,
The in-vehicle situation display device according to claim 2 , wherein the imaging unit is a portion that performs the gesture for the operator, and is disposed at a position for imaging an illumination portion illuminated by the illumination unit. . The control unit ends the display of the captured image of the passenger compartment on the display unit when a predetermined period has elapsed after starting control of displaying the image of the passenger compartment captured on the display unit. The in-vehicle situation display device according to any one of claims 1 to 3, wherein control is performed. The storage unit stores an end gesture corresponding to a control to end display of the captured image of the passenger compartment in the display unit,
The control unit determines whether the identified gesture matches the gesture related to the end stored in the storage unit,
If the gesture according to the termination and the gesture was identified match any of claims 1 to 3, characterized in that performs control to end the display of the image of the casing which is captured on the display unit The in-vehicle situation display device according to claim 1. Wherein the operator the gesture is identified in the control unit, vehicle status display device according to any one of claims 1 to 5, characterized in that the driver of the vehicle. The in-vehicle situation according to any one of claims 1 to 6 , wherein the gesture identified by the control unit is a gesture by a hand closer to the imaging unit among the hands of the operator. Display device. An unauthorized intrusion detector for detecting unauthorized intrusion into the vehicle;
A transmission unit for transmitting the image signal to a reception unit installed outside the vehicle;
Is further provided,
When the detection signal output when the unauthorized intrusion detection unit detects an unauthorized entry into the vehicle is input to the control unit, the control unit causes the transmission unit to transmit a control signal for transmitting the image signal to the reception unit. vehicle condition display device according to any one of 7 claim 1, characterized in that the output. When the control unit receives a detection signal that is output when the unauthorized intrusion detection unit detects unauthorized intrusion into the vehicle, the captured image of the passenger compartment is further displayed on the display unit. 9. The in-vehicle situation display apparatus according to claim 8, wherein control for prohibiting the operation is performed.