JP2017103627A - Vehicle rear view device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a vehicle rear view device capable of improving the difficulty of watching due to minute motions unintended by a driver.SOLUTION: A vehicle rear view device comprises: an in-vehicle camera for capturing the rear of the vehicle; a driver state detection unit for imaging the upper body of a driver; a control unit that, on the basis of the image signal captured by the driver state detection unit, removes tracking information accompanying movement of the head smaller than a predetermined moving speed from the tracking information of the movement of the head including the moving direction and the moving speed obtained by tracking the movement of the head of the driver, and on the basis of the video signal captured by the in-vehicle camera, generates a backward image reflecting the tracking information of the movement of the head after the removal, to output the image signal as a video signal; and a display unit that displays a backward image reflecting the movement of the driver's head on the basis of the video signal sent from the control unit.SELECTED DRAWING: Figure 4

Description

  Embodiments of the present invention relate to a vehicle rear display device that captures an image of the rear of a vehicle with an in-vehicle camera and allows a driver to visually recognize the image.

  When the driver checks the rear periphery of the vehicle with the rearview mirror, there are cases where it is desired to check not only the range reflected in the rearview mirror but also the surroundings of that range. In such a case, the driver needs to move the head and move the range reflected in the room mirror without changing the direction of the room mirror.

  On the other hand, in order to improve the visibility of the rear view without using the rear mirror, an in-vehicle camera that images the rear of the vehicle is installed, and the display device installed at a position where the driver inside the windshield can visually recognize the rear A vehicle rear display device that displays an image of a field of view has been proposed.

  In such a vehicle rear display device, the display range is fixed in order to display a rear image captured by the in-vehicle camera. For this reason, even if the driver moves the head to move the display range displayed on the vehicle rear display device, the display range does not move. This state is different from the confirmation movement of the rear view using a normal room mirror, which makes the driver feel uncomfortable.

  Therefore, it has also been proposed to detect the movement of the driver's head and move the display range displayed on the vehicle rear display device in accordance with the movement of the head.

  However, the driver's head continues to move in small increments due to the shaking of the vehicle and his unconscious movement. If the display range is moved in line with the movement of the driver's unconscious head as it is, there is a problem that the display contents move in small increments or display of the display range not intended by the driver occurs, making it difficult to see.

JP 58-209635 A JP 2010-64714 A JP 2008-141578 A JP 2011-188028 A

  The problem to be solved by the present invention is to provide a vehicle rear display device capable of improving the difficulty of seeing due to small movements not intended by the driver.

  The vehicle rear display device of the embodiment is based on an in-vehicle camera that images the rear of the vehicle, a driver state detection unit that images the upper body of the driver, and a video signal captured by the driver state detection unit. The tracking information associated with head movement smaller than a predetermined moving speed is removed from the head movement tracking information including the moving direction and moving speed obtained by tracking the movement of the head of the hand, and the vehicle is mounted on the vehicle. A control unit that generates a rear video reflecting the tracking information of the head movement after the removal based on a video signal captured by a camera and outputs it as a video signal, and a video signal sent from the control unit And a display unit for displaying a rear image reflecting the movement of the driver's head based on the above.

It is the schematic which shows the whole arrangement | positioning of the vehicle rear display apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the attachment position of a driver state detection part and a display part in a vehicle interior. It is a figure explaining the relationship between a movement of a driver | operator's head, and the movement of the back image | video displayed on a display part. It is a flowchart which shows the flow of the display process of the back image | video according to the motion of the driver | operator's head in the vehicle rear display apparatus which concerns on this embodiment. It is a flowchart which shows the flow of the display process of the back image according to the motion of the driver | operator's head in the vehicle rear display apparatus which concerns on the modification of this embodiment.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram showing the overall arrangement of a vehicle rear display device according to an embodiment of the present invention. As shown in FIG. 1, the vehicle rear display device 100 according to the embodiment of the present invention mainly includes an in-vehicle camera 10, a driver state detection unit 20, a control unit 30, and a display unit 40.

  In this embodiment, in order to image the back of the vehicle, the in-vehicle camera 10 is installed above the vertical center line of the rear window of the vehicle.

  It is desirable that the in-vehicle camera 10 can capture the rear view so that the driver feels the same distance as when looking at the back of the host vehicle. Accordingly, a wide-angle optical system in which the distance between the own vehicle and the following vehicle is different from the actual one is not suitable. For example, a camera having a focal length of about 50 mm is suitable. Furthermore, it is preferable to provide a high-sensitivity optical system so that the rear side of the vehicle can be imaged even at night.

  The in-vehicle camera 10 is preferably a camera that can obtain an HDR (High Dynamic Range) image in which the contrast between a bright place and a dark place is increased.

  Note that the in-vehicle camera 10 may be provided with a panning function for turning the in-vehicle camera 10 in the horizontal direction.

  The video signal obtained by the in-vehicle camera 10 is sent to the control unit 30.

  FIG. 2 is a diagram illustrating an example of attachment positions of the driver state detection unit 20 and the display unit 40 in the vehicle compartment. The driver state detection unit 20 recognizes the state of the driver from an image, and images the upper body of the driver. The driver state detection unit 20 can acquire information such as the position of the driver's head, the direction of the line of sight, the direction of the face, the inclination of the face, and the degree of eye opening as the driver state to be recognized. In the embodiment, information on the position of the driver's head is used.

  The driver state detection unit 20 is installed on, for example, a dashboard in front of the driver or embedded in the dashboard.

  For example, a near-infrared camera is suitable for the driver state detection unit 20 so that the driver's head position can be accurately detected even under the influence of sunlight or wearing of the driver's sunglasses. . It is desirable to increase the S / N ratio of the image during the daytime by incorporating a filter for cutting unnecessary wavelength light other than the wavelength of near infrared light inside the driver state detection unit 20. In addition, it is preferable to provide a visible light cut filter on the front surface of the driver state detection unit 20.

  The video signal obtained by imaging in the driver state detection unit 20 is supplied to the control unit 30.

  The control unit 30 performs various image processing, which will be described later, on the video signal captured and acquired by the driver state detection unit 20 and the video signal obtained by the in-vehicle camera 10, and generates a video signal to be displayed on the display unit 40. .

  The control unit 30 can also control the operation of other electronic devices mounted on the host vehicle. For example, turning on / off headlights, taillights, brake lights, room lights, and the like mounted on the host vehicle. It is also possible to further control the operation of an electronic device for performing a shift, an electronic device for performing a shift by automatic transmission, and the like.

  The control unit 30 includes, for example, a processing device such as a CPU or an MPU that performs arithmetic processing, a memory that stores data, such as an SRAM, a DRAM, or a flash memory, and a communication interface that performs communication with other electronic devices. be able to.

<Head position detection>
The control unit 30 applies a well-known image recognition technique to the video signal acquired by imaging the upper body of the driver in the driver state detection unit 20, and detects the position of the driver's head. The control unit 30 detects an outline that appears to be the upper body of the image obtained from the input video signal, binarizes the pixel value in the image to 0 or 1, and moves from the upper left to the lower right in the image. The shape of the contour is evaluated by scanning all the windows.

  As a detailed detection method for the upper body that follows, for example, a method using a luminance gradient direction co-occurrence (CoHOG) feature amount focusing on a feature amount in an image can be applied. In the image of the upper body, the driver's face candidate area is divided into multiple blocks, and for each block, the combinations of edge directions are histogrammed at two points, and how often they exist simultaneously (Co-occurrence) is examined among 30 points at four different distances. By using the CoHOG feature amount, it is possible to detect the driver's face area (contour) in detail.

  Detect facial feature points (parts) such as eyes, nose and mouth from the identified facial area (contour), and the head position including the face orientation and position from the geometric relationship between these facial areas and each part position Is detected.

<Head movement calculation (tracking)>
Next, the control unit 30 calculates (tracks) the movement of the head based on the continuous video signal. As a head tracking method in a moving image in the present embodiment, for example, an object tracking method using an algorithm called a particle filter can be applied.

  The particle filter is a sequential tracking algorithm that simultaneously detects and tracks an object, is a sequential tracking that does not require an end point of tracking in advance, and is light in calculation, so that real-time processing is possible. The particle filter sets the initial position, considers a number of next states that can occur from the current state as hundreds or thousands of particles, and calculates a weighted average based on the likelihood of all particles Tracking is performed while estimating the state.

  For the state estimation model, the moving direction and speed of the head are estimated using an optical flow obtained from the image. The optical flow uses a time-continuous image sequence to obtain an image velocity field (object velocity + camera velocity) and represents it as a vector set. For example, an optical flow is obtained by using a constraint equation represented by (luminance time / luminance gradient) such as a block matching gradient method and adding a constraint condition thereto. In the block matching method, an optical flow can be obtained by searching for a matching portion from the next time image using a block in the image as a template.

<Removal of head peristalsis>
The driver's head continues to move in small increments due to the shaking of the vehicle and the unconscious movement of the driver. If the display range to be displayed on the display unit 40 is moved in accordance with the movement of the driver's head, the display moves little by little, or a display movement unintended by the driver occurs, making it difficult to see.

  Therefore, in the present embodiment, the influence of the driver's unconscious movement of the head on the moving direction and speed of the head obtained in the tracking process of the head movement is removed. In general, the driver's unconscious movement of the head moves slowly, unlike when the driver sits back on the seat or rotates his neck. Therefore, the moving direction and speed of the head smaller than the predetermined moving speed are not reflected as tracking information of the head movement. It should be noted that the magnitude of the predetermined moving speed to be removed is preferably adjustable by the driver.

  There is a research report about the unintentional movement of the driver's head with a maximum moving speed of 50 mm / sec ("Study on automatic tracking and focus control device for eye camera" 2002 Niigata Institute of Industrial Technology Report No31). Therefore, for example, it is preferable to remove the head movement of the driver whose movement speed obtained in the tracking process of the head movement is 50 mm / sec or less from the tracking information of the head movement.

<Generation of rear video>
The control unit 30 generates a rear image in which the tracking information of the head movement from which the affected part due to the head movement is removed is reflected on the video signal acquired by the in-vehicle camera 10. FIG. 3 is a diagram for explaining the relationship between the movement of the driver's head and the movement of the rear image displayed on the display unit 40. The movement of the driver's head and the display content (range) of the rear image move in the opposite direction. For example, when the head moves to the right, the display content (range) of the rear video moves to the left.

  From the tracking information of the head movement, when the driver moves the head leftward from the previous position, the control unit 30 displays the rear image displayed on the display unit 40 rightward from the previous position. Generate to move. On the other hand, when the driver moves his / her head in the right direction from the previous position, the rear image to be displayed on the display unit 40 is generated so as to move leftward from the previous position.

  The video signal related to the rear video processed by the control unit 30 as described above is sent to the display unit 40. If the control unit 30 fails, the video signal sent from the in-vehicle camera 10 is sent to the display unit 40 as it is.

  The display unit 40 displays a rear image that reflects the movement of the head from which the affected part due to the movement of the head is removed based on the video signal sent from the control unit 30. The display unit 40 is installed in the vicinity of the driver's seat of the vehicle, for example, at the position of the room mirror inside the windshield.

  The display unit 40 is preferably an in-vehicle electronic mirror capable of switching and displaying a conventional rearview mirror and a rear image generated by the control unit 30.

  Since the in-vehicle electronic mirror displays the above rear image on the liquid crystal monitor built in the rearview mirror, the driver can easily confirm the rear situation.

  It is preferable that the in-vehicle electronic mirror has a structure that corrects difficulty in viewing due to the backlight of the sun, the light of the headlight, and the like.

  Next, the flow of the rear image display process corresponding to the movement of the driver's head in the vehicle rear display device configured as described above will be described.

  FIG. 4 is a flowchart showing a flow of a rear image display process corresponding to the movement of the driver's head in the vehicle rear display device according to the present embodiment.

  First, the driver's upper body is imaged by the in-vehicle camera of the driver state detection unit 20 (step S41). The video signal obtained by imaging in the driver state detection unit 20 is supplied to the control unit 30.

  Next, an image recognition technique is applied to the video signal obtained by imaging the upper body of the driver to detect the position of the driver's head (step S42).

  Next, the movement of the driver's head is tracked based on the continuous video signal (step S43).

  Subsequently, the influence of the driver's unconscious head movement on the head movement direction and speed obtained in the head movement tracking process is removed from the head movement tracking information ( Step S44).

  Next, a rear image reflecting the tracking information of the head movement from which the affected part due to the head movement is removed is generated (step S45) and displayed on the display unit 40 (step S46).

<Modification>
The rear image captured by the in-vehicle camera 10 is affected by the shaking of the host vehicle. In the above-described embodiment, the influence is not mentioned, but it is preferable to remove the influence caused by the shaking of the own vehicle with respect to the video signal obtained by the in-vehicle camera 10.

  As a method for removing the influence of the shaking of the host vehicle, for example, the position where the effective image is cut out from the image based on the video signal acquired by the in-vehicle camera 10 is moved to suppress the fine shaking of the subject on the effective image. it can.

  FIG. 5 is a flowchart showing a flow of a rear image display process according to the movement of the driver's head in the vehicle rear display device according to the modification of the present embodiment. Even in the modification, the flow of the basic rear image display process is the same as that of the present embodiment. For example, in step S54, the influence of the in-vehicle camera 10 due to the shaking of the host vehicle is removed.

  As described above, according to the present embodiment, it is easy to see because the driver's unintended movement is not reflected in the rear image of the host vehicle displayed on the display unit 40.

  Although several embodiments of the present invention have been described, these embodiments are presented by way of example and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 10 ... Car-mounted camera 20 ... Driver state detection part 30 ... Control part 40 ... Display part 100 ... Vehicle rear display apparatus

Claims (6)

An in-vehicle camera that images the back of the vehicle;
A driver state detector that images the upper body of the driver;
Based on the video signal imaged by the driver state detector, a predetermined movement speed is obtained from the head movement tracking information including the movement direction and movement speed obtained by tracking the movement of the driver's head. Remove tracking information associated with smaller head movements,
A control unit that generates a rear image reflecting the tracking information of the head movement after the removal based on a video signal imaged by the in-vehicle camera, and outputs it as a video signal;
A vehicle rear display device, comprising: a display unit configured to display a rear image reflecting the movement of the driver's head based on the video signal transmitted from the control unit. The controller is
Based on the video signal imaged by the driver state detection unit, the contour of the upper body of the driver is detected, the position of the driver's head is detected from the shape of the contour,
Detects the driver's face contour using the Coincidence of Coincidence of Gradient Gradient (CoHOG) feature for the driver's face candidate area,
The vehicle rear display device according to claim 1, wherein the position of the head including the orientation and position of the face is detected from the geometric relationship between the identified facial contour and facial feature points. The controller is
Using the particle filter which is a sequential tracking algorithm, the head movement is estimated as the next state from the initial position,
The vehicle rear display device according to claim 2, wherein the moving direction and moving speed of the head are estimated using an optical flow using a time-continuous image sequence.   4. The control unit according to claim 1, wherein the movement of the head of the driver whose predetermined moving speed is 50 mm / sec or less is removed from tracking information of the movement of the head. 5. Vehicle rear display device.   The vehicle rear display device according to any one of claims 1 to 4, wherein the predetermined movement speed is adjustable by a driver who is a user.   6. The apparatus according to claim 1, wherein a position where an effective image is cut out from an image is moved based on a video signal imaged by the in-vehicle camera, and an influence of shaking of the host vehicle is removed on the effective image. The vehicle rear display device described.

Images