JP6649063B2 - Vehicle rear display device and display control device - Google Patents

Description

The embodiment of the present invention relates to a vehicle rear display device and a display control device that allow a driver to visually recognize an image of the rear of a vehicle using an onboard camera.

  When the driver checks the area around the rear of the vehicle with the rearview mirror, the driver sometimes wants to check not only the area reflected on the rearview mirror but also the surrounding area. In such a case, the driver needs to move the head to move the area reflected on the rear-view mirror without changing the direction of the rear-view mirror.

  On the other hand, instead of using a rearview mirror, an in-vehicle camera that captures an image of the rear of the vehicle is installed as a means for improving the visibility of the rear view, and a display device installed at a position inside the windshield that can be visually recognized by a driver has a rear view. 2. Description of the Related Art A vehicle rear display device that displays an image of a field of view has been proposed.

  In such a vehicle rear display device, a display range is fixed in order to display a rear video captured by an onboard camera. Therefore, even if the driver moves his / her head to move the display range projected on the vehicle rear display device, the display range does not move. Since this state is different from the movement of confirming the rear view using the normal rearview mirror, the driver feels uncomfortable.

  Therefore, it has been proposed to detect the movement of the driver's head and move the display range projected on the vehicle rear display device in accordance with the movement of the head.

  However, the driver's head keeps moving little by little due to the shaking of the vehicle or his own unconscious movement. If the display range is moved in accordance with the unconscious movement of the driver's head as it is, the display content moves little by little, or the display range not intended by the driver may be displayed, making it difficult to see.

JP-A-58-209635 JP 2010-64714 A JP 2008-141578 A JP 2011-188028 A

The problem to be solved by the present invention is to provide a vehicle rear display device and a display control device that can reduce the difficulty in seeing due to unintended small movements of the driver.

The vehicle rear display device of the embodiment is a vehicle-mounted camera that images the rear of the vehicle, a driver state detection unit that images the upper body of the driver, and driving based on a video signal imaged by the driver state detection unit. From the tracking information of the head movement including the moving direction and the moving speed obtained by tracking the movement of the head of the hand, the tracking information accompanying the movement of the head smaller than a predetermined moving speed is removed. The rear image is moved so as to move the display range of the video signal displayed corresponding to the direction of the head movement of the driver in the tracking information of the head movement after the removal based on the video signal captured by the camera. A control unit that generates a video signal, and a display unit that displays to the driver a rearward video reflecting the movement of the driver's head based on the video signal transmitted from the control unit. Is provided.

1 is a schematic diagram illustrating an overall arrangement of a vehicle rear display device according to an embodiment of the present invention. It is a figure showing an example of a mounting position of a driver state detecting part and a display part in a vehicle interior. FIG. 4 is a diagram illustrating a relationship between a movement of a driver's head and a movement of a rear image displayed on a display unit. 5 is a flowchart illustrating a flow of a process of displaying a rear image according to the movement of the driver's head in the vehicle rear display device according to the embodiment. It is a flowchart which shows the flow of the display processing of the back image | video according to the movement of the driver's head in the vehicle back display device which concerns on the modified example of this embodiment.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a schematic diagram showing an overall arrangement of a vehicle rear display device according to an embodiment of the present invention. As shown in FIG. 1, the vehicle rear display device 100 according to the embodiment of the present invention mainly includes a vehicle-mounted camera 10, a driver state detection unit 20, a control unit 30, and a display unit 40.

  In the present embodiment, the on-vehicle camera 10 is installed above the vertical center line of the rear window of the vehicle in order to image the rear of the vehicle.

  It is desirable that the in-vehicle camera 10 can capture an image of the rear field of view so that the driver has the same sense of distance and distance when he looks behind the host vehicle. Therefore, a wide-angle optical system in which the distance between the host vehicle and a following vehicle looks different from the actual one is not suitable. For example, a camera having a focal length of about 50 mm is suitable. Further, it is preferable to provide a high-sensitivity optical system so that the rear of the vehicle can be imaged even at night.

  The in-vehicle camera 10 is preferably a camera that can obtain an HDR (High Dynamic Range) image in which the contrast between a bright place and a dark place is enhanced.

  In addition, the vehicle-mounted camera 10 may be provided with a panning function for turning the vehicle-mounted camera 10 in the horizontal direction.

  The video signal obtained by the on-board camera 10 is sent to the control unit 30.

  FIG. 2 is a diagram illustrating an example of a mounting position of the driver state detection unit 20 and the display unit 40 in the vehicle cabin. The driver state detection unit 20 recognizes the driver state by video, and captures an image of the upper body of the driver. The driver state detection unit 20 can acquire information such as the position of the head of the driver, the direction of the line of sight, the direction of the face, the inclination of the face, and the degree of eye opening as the state of the driver to be recognized. In the embodiment, information on the position of the driver's head is used.

  The driver state detection unit 20 is installed on, for example, a dashboard in front of the driver, or is embedded in the dashboard.

  For example, a near-infrared camera is suitable so that the driver state detection unit 20 can accurately detect the position of the driver's head even under the influence of sunlight or the driver wearing sunglasses. . It is desirable to increase the daytime image S / N ratio by incorporating a filter for cutting unnecessary wavelength light other than the wavelength of near-infrared light inside the driver state detection unit 20. Further, it is preferable that a visible light cut filter is provided on the front surface of the driver state detecting unit 20.

  The video signal obtained by imaging in the driver state detection unit 20 is supplied to the control unit 30.

  The control unit 30 performs various image processing described later on the video signal obtained by imaging in the driver state detection unit 20 and the video signal obtained by the vehicle-mounted camera 10, and generates a video signal to be displayed on the display unit 40. .

  The control unit 30 can also control the operation of other electronic devices mounted on the host vehicle, such as turning on / off a headlight, a tail light, a brake light, a room light, and the like mounted on the host vehicle. The operation of the electronic device for performing the shift operation, the electronic device for performing the automatic speed change, and the like may be further controlled.

  The control unit 30 includes, for example, a processing device such as a CPU or an MPU that performs arithmetic processing, an SRAM that stores data, a memory such as a DRAM or a flash memory, and a communication interface for communicating with other electronic devices. be able to.

<Head position detection>
The control unit 30 detects the position of the driver's head by applying a known image recognition technique to a video signal obtained by imaging the driver's upper body in the driver state detection unit 20. The control unit 30 detects an outline considered to be the upper body, binarizes the pixel value in the image to 0 or 1, and performs the binarization in the image from the upper left to the lower right on the image obtained from the input video signal. The contour shape is evaluated by scanning all windows.

  As a detailed method for detecting the upper body, for example, a method based on a luminance gradient direction co-occurrence (CoHOG) feature amount that focuses on a feature amount in an image can be applied. This is because, in an image of the upper body, the candidate area of the driver's face is divided into a plurality of blocks, and for each block, a histogram of combinations of edge directions is formed at two points, and at what frequency (Co-occurrence) is checked between 30 points at four different distances. By using the CoHOG feature amount, the driver's face area (contour) can be detected in detail.

  Face features (parts) such as eyes, nose, mouth, etc. are detected from the specified face area (contour), and the position of the head including the direction and position of the face is determined from the geometrical relationship between these face areas and the positions of each part. Is detected.

<Head motion calculation (tracking)>
Next, the control unit 30 calculates (tracks) the movement of the head based on the continuous video signal. As a method of tracking a head in a moving image in the present embodiment, for example, an object tracking method using an algorithm called a particle filter can be applied.

  The particle filter is a sequential tracking algorithm that simultaneously performs detection and tracking of an object, is a sequential tracking that does not require an end point of the tracking in advance, and has a light calculation, so that real-time processing is possible. The particle filter sets an initial position, considers many possible next states from the current state to hundreds or thousands of particles (particles), and calculates a weighted average based on the likelihood of all particles to the next. Tracking is performed while estimating the state.

  In the state estimation model, the moving direction and speed of the head are estimated using an optical flow obtained from the image. In the optical flow, a speed field of an image (object speed + camera speed) is obtained by using a time-continuous image sequence, and the obtained speed field is represented by a set of vectors. For example, an optical flow is obtained by using a constraint equation expressed by (luminance time / luminance gradient) such as a block matching gradient method and adding a constraint condition thereto. In the block matching method, an optical flow can be obtained by searching for a matching portion from an image at the next time using a certain block in the image as a template.

<Removal of head swing>
The driver's head keeps moving little by little due to the shaking of his own vehicle or his own unconscious movement. If the display range displayed on the display unit 40 is moved according to the movement of the driver's head as it is, the display moves little by little or a display movement not intended by the driver occurs, making it difficult to see.

  Thus, in the present embodiment, the influence of the driver's unconscious head swing on the head movement direction and speed obtained in the head movement tracking process is removed. In general, unlike the case where the driver sits back in the seat or turns his neck, the unconscious head movement of the driver moves slowly. Therefore, the moving direction and speed of the head smaller than the predetermined moving speed are not reflected as tracking information of the head movement. Preferably, the magnitude of the predetermined moving speed to be removed is adjustable by the driver.

  Regarding such unconscious driver's head movement, there is a research report that the maximum moving speed is 50 mm / sec ("Study on automatic tracking focus control device for eye camera", 2002 Niigata Prefectural Industrial Technology Research Institute of Industrial Technology Report No 31). Therefore, for example, it is preferable to remove from the head movement tracking information a head movement of a driver whose moving speed obtained in the head movement tracking process is 50 mm / sec or less.

<Generation of rear video>
The control unit 30 generates a rear image in which the tracking information of the movement of the head, in which the part affected by the swing of the head is removed, is reflected on the image signal acquired by the on-vehicle camera 10. FIG. 3 is a diagram illustrating the relationship between the movement of the driver's head and the movement of the rear image displayed on the display unit 40. The movement of the driver's head and the display content (range) of the rear image move in opposite directions. For example, when the head moves to the right, the display content (range) of the rear image moves to the left.

  From the tracking information of the movement of the head, when the driver is moving the head leftward from the previous position, the control unit 30 shifts the rearward video displayed on the display unit 40 rightward from the previous position. Generate to move. On the other hand, when the driver is moving the head to the right from the previous position, the rear video to be displayed on the display unit 40 is generated so as to move to the left from the previous position.

  The video signal related to the rear video processed by the control unit 30 as described above is sent to the display unit 40. When the control unit 30 fails, the video signal transmitted from the vehicle-mounted camera 10 is transmitted to the display unit 40 as it is.

  The display unit 40 displays a rear image in which the movement of the head is removed based on the video signal transmitted from the control unit 30 and the portion affected by the swing of the head is removed. The display unit 40 is installed near a driver's seat of the vehicle, for example, at a position of a room mirror inside a windshield.

  The display unit 40 is preferably a conventional room mirror and a vehicle-mounted electronic mirror capable of switching and displaying a rear image generated by the control unit 30.

  The in-vehicle electronic mirror displays the above-described rear image on a liquid crystal monitor built in the room mirror, so that the driver can easily check the situation behind.

  It is preferable that the in-vehicle electronic mirror has a structure that corrects the difficulty of viewing due to the backlight of the sun, the light of the headlight, or the like.

  Next, a flow of a process of displaying a rear image according to the movement of the driver's head in the vehicle rear display device configured as described above will be described.

  FIG. 4 is a flowchart illustrating a flow of a process of displaying a rear image according to the movement of the driver's head in the vehicle rear display device according to the present embodiment.

  First, the upper body of the driver is imaged by the on-board camera of the driver state detection unit 20 (step S41). The video signal captured and acquired by the driver state detection unit 20 is supplied to the control unit 30.

  Next, the position of the driver's head is detected by applying image recognition technology to the video signal obtained by imaging the upper body of the driver (step S42).

  Next, the movement of the driver's head is tracked based on the continuous video signal (step S43).

  Subsequently, the influence of the driver's unconscious head swing on the head movement direction and speed obtained in the head movement tracking process is removed from the head movement tracking information ( Step S44).

  Next, a rear image reflecting the tracking information of the movement of the head from which the part affected by the swing of the head has been removed is generated (step S45), and displayed on the display unit 40 (step S46).

<Modification>
The rear image captured by the on-board camera 10 is affected by the swing of the vehicle. In the embodiment described above, such influence is not mentioned, but it is preferable to remove the influence of the vibration of the own vehicle from the video signal obtained by the vehicle-mounted camera 10.

  As a method of removing the influence of the vibration of the own vehicle, for example, moving the position where an effective image is cut out from an image based on the video signal acquired by the on-vehicle camera 10 to suppress the minute vibration of the subject on the effective image. it can.

  FIG. 5 is a flowchart illustrating a flow of a display process of a rear image according to the movement of the driver's head in the vehicle rear display device according to the modified example of the embodiment. In the modified example, the basic flow of the display process of the rear image is the same as that of the present embodiment. For example, in step S54, the influence of the on-vehicle camera 10 due to the shaking of the own vehicle is removed.

  As described above, according to the present embodiment, the rearward video of the host vehicle displayed on the display unit 40 does not reflect the wiggle that the driver does not intend, so that it is easy to see.

  Although several embodiments of the present invention have been described, these embodiments are provided by way of example and are not intended to limit the scope of the invention. These new embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

DESCRIPTION OF SYMBOLS 10 ... In-vehicle camera 20 ... Driver state detection part 30 ... Control part 40 ... Display part 100 ... Vehicle rear display device

Claims (7)

An on-board camera for imaging the rear of the vehicle,
A driver state detector for imaging the upper body of the driver,
Based on the head signal tracking information including the moving direction and the moving speed obtained by tracking the head movement of the driver based on the video signal imaged by the driver state detection unit, a predetermined moving speed Removes tracking information associated with smaller head movements,
Based on the video signal captured by the on-board camera, the display range of the video signal displayed corresponding to the direction of the head movement of the driver in the tracking information of the head movement after the removal is moved. A control unit that generates a rear video and outputs the video as a video signal;
A display unit for displaying to the driver a rear image reflecting the movement of the driver's head based on the image signal transmitted from the control unit. The control unit includes:
Based on the video signal captured by the driver state detection unit, detects the contour of the driver's upper body, detects the position of the driver's head from the shape of the contour,
Using the luminance gradient direction co-occurrence (CoHOG) feature for the candidate face of the driver, the contour of the driver's face is detected,
The vehicle rear display device according to claim 1, wherein a position of a head including a direction and a position of the face is detected from a geometric relationship between the specified contour of the face and a feature point of the face. The control unit includes:
Using a particle filter that is a sequential tracking algorithm, the head movement is estimated as the next state from the initial position,
The vehicle rear display device according to claim 2, wherein the moving direction and the moving speed of the head are estimated using an optical flow using a time-continuous image sequence.   The said control part removes from the tracking information of the head movement about the head movement of the driver whose predetermined moving speed is 50 mm / sec or less, The Claims any one of Claims 1 thru | or 3 characterized by the above-mentioned. Vehicle rear display device.   The vehicle rear display device according to any one of claims 1 to 4, wherein the predetermined moving speed is adjustable by a driver as a user.   The method according to any one of claims 1 to 5, wherein a position at which an effective image is cut out from the image is moved based on a video signal captured by the on-vehicle camera, and the influence of the vibration of the own vehicle on the effective image is removed. The vehicle rear display device according to claim 1. From a signal obtained by imaging the movement of the driver's head, the tracking of the movement of the driver's head is performed, and tracking information of the movement of the head including the moving direction and the moving speed is obtained.
For a display unit that displays a video signal obtained by capturing an image of the rear of the vehicle to a driver, the moving speed is displayed on the display unit based on the tracking information except when the moving speed is lower than a predetermined moving speed. A display control device that controls movement of a display range of a video signal.

Images