JP2018093427A - On-vehicle projection apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce an operation load of an on-vehicle projection apparatus by a simple configuration.SOLUTION: An on-vehicle projection apparatus 10 includes a projector 12, an in-vehicle camera 14, an out-vehicle camera 16, and a controller 18. The controller 18 has a function as a direction adjustment section for adjusting an imaging direction of the in-vehicle camera 14, a calibration processing section acquiring correction data for an image projected to an image projection plane, by using an image captured when the imaging direction of the in-vehicle camera 14 is a direction toward the image projection plane by the projector 12, a detector for detecting a hand of an occupant as a detection object, based on the image captured when the imaging direction of the in-vehicle camera 14 is a direction toward a gesture recognition region 36, and a control section for controlling the on-vehicle projection apparatus 10 according to movement of the hand of the occupant detected by the detector.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an in-vehicle projection apparatus, and more particularly to an apparatus that performs distortion correction on a projection image.

  Research and development has been extensively conducted on a vehicle-mounted projection device that projects an image using a ceiling or the like of a passenger compartment as a screen. In this projection apparatus, an image of a situation outside the vehicle, an image of car navigation, an image of a television receiver, and the like are projected onto a screen by a projector. As the screen, a non-flat surface such as a wall or window glass is used in addition to the ceiling of the passenger compartment. Therefore, when projecting an image, distortion correction processing is performed on the image data. The distortion correction process is a process for giving a correction distortion according to the shape of the screen to the projected image so that the distortion seen from the passenger is suppressed. In order to obtain correction data used for the distortion correction process, calibration is executed in advance. In calibration, a reference image that has not been subjected to distortion correction processing is projected onto a screen, the reference image is photographed by a camera, and correction data is obtained using the reference image data obtained thereby.

  Patent Document 1 below describes a technique for correcting distortion of an image projected into a vehicle interior.

JP 2015-184409 A JP 2016-38621 A

  About the vehicle-mounted projection apparatus, it is preferable to reduce the operation burden on the passenger from the viewpoint of safe driving. Therefore, as described in Patent Document 2, it is conceivable to introduce a spatial input system that recognizes an occupant's hand gesture (gesture) with a camera and controls the in-vehicle projector according to the gesture. However, in addition to a camera for acquiring correction data, a gesture recognition camera is required, which complicates the configuration.

  An object of the present invention is to reduce the operation burden of a vehicle-mounted projection apparatus with a simple configuration.

  The present invention projects the image onto the image projection plane using a projector that projects an image into the vehicle interior and an image captured when the shooting direction of the camera in the vehicle interior is the direction toward the image projection plane. A calibration processing unit that acquires correction data for an image, and a control unit that controls the projector according to an image captured when the imaging direction deviates from a direction toward the image projection plane, The projector projects the image corrected by the correction data onto the image projection surface in accordance with the control of the control unit.

  The present invention is also directed to a projector for projecting an image in a vehicle interior, a camera that captures the interior of the vehicle, a direction adjustment unit that adjusts a capture direction of the camera, and a projection direction that projects the image. A calibration processing unit that acquires correction data for an image projected on the image projection plane using an image shot when the image is directed toward the plane, and the shooting direction deviates from the direction toward the image projection plane. A detection unit that detects a detection object based on an image captured when the image is captured, a control unit that controls the in-vehicle projection device according to the movement of the detection object detected by the detection unit, It is characterized by providing.

  Preferably, after the correction data is acquired, the direction adjusting unit sets the photographing direction in a direction toward the occupant.

  Desirably, a brightness adjusting unit that adjusts the brightness of an image projected into the vehicle interior based on an image captured by the camera.

  According to the present invention, it is possible to reduce the operation burden of the in-vehicle projector with a simple configuration.

It is a figure which shows the vehicle-mounted projection apparatus which concerns on embodiment of this invention. It is a figure which shows the modification of the vehicle-mounted projection apparatus which concerns on embodiment of this invention. It is a figure which shows the modification of the vehicle-mounted projection apparatus which concerns on embodiment of this invention. It is a figure which shows the modification of the vehicle-mounted projection apparatus which concerns on embodiment of this invention. It is a figure explaining the distortion correction process of a projection image. It is a flowchart of the process which a controller performs.

  FIG. 1 shows an in-vehicle projector 10 according to an embodiment of the present invention. The in-vehicle projection device 10 projects an image or the like outside the vehicle onto the ceiling 24 and executes a gesture operation process. When projecting a video, calibration is executed in advance, and correction data for correcting distortion of the projected image is acquired. An image subjected to the distortion correction processing by this correction data is projected onto the ceiling 24. The gesture operation process is a process for photographing a passenger's gesture and executing control according to the gesture.

  The in-vehicle projector 10 includes a projector 12, an in-vehicle camera 14, an out-of-vehicle camera 16, and a controller 18. Among these, components other than the vehicle exterior camera 16 may be provided in the passenger compartment 20.

  The vehicle exterior camera 16 is a camera disposed outside the vehicle, and photographs a landscape in a vertically upward direction, for example. The vehicle exterior camera 16 may be disposed in front of the vehicle. In that case, for example, the camera 16 outside the vehicle is disposed on the back surface of the rearview mirror 22 as viewed from the driver so as not to obstruct the driver's visual field. The vehicle exterior camera 16 is composed of, for example, a CCD camera or a CMOS camera. Image data taken by the camera 16 outside the vehicle is sent to the controller 18.

  The projector 12 is provided in the overhead module 30 in front of the passenger compartment, and projects an image on the ceiling 24. The projector 12 may be capable of projecting a moving image or a still image. The projector 12 can communicate with the controller 18 and receives various control signals from the controller 18. Further, the projector 12 may have a zoom function so that the size of the projection surface is variable.

  The projector 12 can project an image captured by the outside camera 16 via the controller 18. The camera 16 outside the vehicle captures a vertically upward landscape, and the projector 12 sets an optical path on the ceiling 24 of the passenger compartment 20 so that the vertically upward landscape outside the vehicle is projected onto the ceiling 24. As a result, the same effect as when the sunroof is opened is obtained, and the passenger gets a feeling of opening.

  Further, the projector 12 may project a television receiver video, a video disk recorder video, a car navigation video, a message to the passenger, and the like on the ceiling 24 in addition to the image taken by the camera 16 outside the vehicle.

  The in-vehicle camera 14 is disposed between the heads of the two rear seats 28. The in-vehicle camera 14 is composed of a CCD camera or a CMOS camera, for example. The in-vehicle camera 14 has a variable field of view according to control by the controller 18. That is, the photographing field of view is represented by the photographing direction and the viewing angle, and the in-vehicle camera 14 can adjust the photographing direction and the viewing angle according to control by the controller 18. The field of view of the in-vehicle camera 14 is rectangular, for example. In this case, the viewing angle is defined by a set of the vertical viewing angle and the horizontal viewing angle, and the photographing direction is defined as the center direction of the vertical viewing angle range and the horizontal viewing angle range.

  In the calibration, the in-vehicle camera 14 captures an image projected by the projector 12. In other words, the viewing angle is set so that the shooting direction of the in-vehicle camera 14 is directed to the direction of the projection image and the projection image enters the shooting field of view. The field of view during calibration is preferably set so as to include the optical path of the projector 12 (indicated by a broken line in FIG. 1).

  On the other hand, in the gesture operation process, the in-vehicle camera 14 captures the gesture recognition area 36 between the two rear seats 28. That is, the viewing angle is set so that the shooting direction of the in-vehicle camera 14 is directed in the direction of the gesture recognition region 36 and the gesture recognition region 36 is within the shooting field of view. The occupant sitting on the back seat holds his hand over the gesture recognition area 36 and moves the hand to express the gesture. Image data captured by the in-vehicle camera 14 is sent to the controller 18. The controller 18 controls the in-vehicle projector 10 based on the captured image data. The setting of the field of view of the in-vehicle camera 14 may be performed by a switch, dial, or the like provided on the front panel. In this case, the controller 18 sets the field of view of the in-vehicle camera 14 according to the operation of the switch or dial. The gesture recognition area may be another area in the vehicle. For example, the area between the driver's seat 32 and the passenger seat 34 may be used. In this case, an occupant sitting on the passenger seat 34 holds a hand over the gesture recognition area and moves the hand to express the gesture.

  The controller 18 receives photographed image data from the in-vehicle camera 14 and the outside camera 16 and transmits a control signal and image data to the projector 12. The controller 18 may be a part of an electronic control unit (ECU) that controls the vehicle, for example.

  The controller 18 uses a direction adjustment unit that adjusts the shooting direction of the in-vehicle camera 14 and an image that is captured when the shooting direction of the in-vehicle camera 14 is the direction toward the image projection plane of the projector 12. The calibration processing unit that obtains correction data for the projected image and the image captured when the imaging direction of the in-vehicle camera 14 is directed toward the gesture recognition area 36 (when the imaging direction deviates from the direction toward the image projection plane) A detection unit that detects an occupant's hand as a detection target based on the image, and a control unit that controls the in-vehicle projection device 10 based on the movement (gesture) of the occupant's hand detected by the detection unit It has a function. These components may be configured by executing a program, or may be configured individually by a digital circuit that is hardware.

  The projector 12 shown in FIG. 1 is provided in the overhead module 30 in front of the passenger compartment, but as shown in FIGS. 2 and 3, it is between the two rear seats 28 and the center console 26. It may be provided above. The in-vehicle camera 14 shown in FIG. 1 is provided between the two rear seats 28, but may be provided on the center console 26 as shown in FIG.

  Calibration performed by the controller 18 will be described with reference to FIG. When the ceiling 24 has a curved surface shape, the projection image viewed from the passenger is distorted if no treatment is performed on the image data to be projected. Therefore, the controller 18 executes the following calibration.

  The controller 18 controls the projector 12 to project the reference image onto the ceiling 24 as a screen, and further controls the in-vehicle camera 14 to photograph the reference image projected onto the ceiling 24. The reference image is, for example, an image on which a predetermined graphic is drawn or an image on which a test pattern is drawn. The controller 18 obtains correction data based on the captured image data of the reference image. The correction data is data for applying a distortion for correction according to the shape of the ceiling 24 to an image projected at normal time, that is, for moving each pixel to suppress distortion seen from the occupant. The correction data includes, for example, information that associates the position coordinates of each pixel before movement with the position coordinates of each pixel after movement, and information that represents the luminance of each pixel after movement.

  In the normal process after the calibration is executed, the controller 18 executes a distortion correction process on the image data to be projected using the correction data, and an image based on the image data subjected to the distortion correction process is given to the projector 12. Project.

  FIG. 6 shows a flowchart of processing executed by the controller 18. The controller 18 sets the imaging field of view of the in-vehicle camera 14 to the first field of view when the electric system is turned on, for example, when the engine start / stop button of the vehicle is pressed, or when the engine is started ( S101). The first visual field is a photographing visual field when calibration is executed, and is defined by a predetermined photographing direction and a viewing angle. That is, the controller 18 sets the viewing angle of the in-vehicle camera 14 so that the shooting direction of the in-vehicle camera 14 is directed to the direction of the projected image and the projected image enters the shooting field of view.

  After the shooting field of view of the in-vehicle camera 14 is set to the first field of view, the controller 18 executes calibration to acquire correction data (S102), and starts projecting an image by the projector 12 (S103). For the image projected on the ceiling 24, the controller 18 executes distortion correction processing on the image data using the correction data.

  After starting the projection of the image onto the ceiling 24, the controller 18 sets the photographing field of view of the in-vehicle camera 14 to the second field of view (S104). The second visual field is a photographing visual field when the gesture operation process is executed, and is defined by a predetermined photographing direction and a viewing angle. That is, the controller 18 sets the viewing angle of the in-vehicle camera 14 so that the shooting direction of the in-vehicle camera 14 is directed in the direction of the gesture recognition area 36 and the gesture recognition area 36 is within the shooting visual field. The process of setting the shooting field of view of the in-vehicle camera 14 to the second field of view may be performed by an operation of a switch, a dial or the like provided on the front panel.

  After setting the shooting field of view of the in-vehicle camera 14 to the second field of view, the controller 18 recognizes the gesture of the passenger's hand based on the captured image data sequentially output from the in-vehicle camera 14 over time (S105). The controller 18 stores information for determining an operation on the in-vehicle projection device 10 for each of a plurality of types of gestures. For the operation on the in-vehicle projection device 10, for example, an operation for selecting which of the image taken by the outside camera 16, the image of the television receiver, the image of the video disk recorder, the image of the car navigation, etc. is projected on the ceiling 24. There is. In addition, when a moving image is projected on the ceiling 24, there may be operations for playing, stopping, low-speed reproduction, high-speed reproduction, etc. of the moving image. Furthermore, when sound is reproduced together with image projection, the operation on the in-vehicle projector 10 may include an operation for adjusting the volume. The operation on the in-vehicle projector 10 may include an operation for ending the image projection.

  The controller 18 determines an operation for the in-vehicle projection device 10 according to the gesture recognized in step S105, and controls the in-vehicle projection device 10 according to the operation (S106).

  The controller 18 determines whether or not an operation for ending the processing of the in-vehicle projector 10 has been performed (S107). This operation may be performed by gesture recognition (S105) or may be performed by operating a button on the front panel. When the end operation is performed, the controller 18 stops the projection of the image by the projector 12 and ends the processing of the in-vehicle projection device 10 (S107). When the end operation is not performed, the controller 18 returns to the process of step S105 (S107).

  According to such a process, since the vehicle-mounted projection apparatus 10 is operated by a gesture, the operation burden on the passenger is reduced. The same in-vehicle camera 14 is used both when executing calibration and when executing gesture operation processing. As a result, two processes of calibration and gesture operation process are executed with a simple configuration.

  Furthermore, in each of the calibration and gesture operation processing, an appropriate shooting field of view is set individually, so that highly accurate correction data is obtained and the gesture is recognized with high accuracy.

  Note that the controller 18 may have a function as a luminance adjustment unit that adjusts the luminance of an image projected into the passenger compartment 20 based on the captured image. The controller 18 may execute a process of adjusting the brightness of the projection image in parallel with the gesture recognition (S105) and the control according to the gesture (S106). In this case, in step 104, the controller 18 sets the viewing angle of the in-vehicle camera 14 so that not only the gesture recognition area 36 but also the near-ceiling area 38 is within the imaging field of view of the in-vehicle camera 14. Here, the reason why the field of view is set so that not only the gesture recognition area 36 but also the ceiling vicinity area 38 is included is that the brightness of the image projected on the ceiling 24 is adjusted according to the brightness of the ceiling vicinity area 38. is there.

  The brightness adjustment process for the projected image will be described. The controller 18 obtains the luminance evaluation value of the image captured by the in-vehicle camera 14. The luminance evaluation value is an evaluation value having a larger value as the luminance of the projection image is higher. The controller 18 adjusts the brightness of the image projected by the projector 12 when the brightness evaluation value is outside the predetermined range. That is, when the luminance evaluation value exceeds the upper limit value of the predetermined range, the luminance of the image projected by the projector 12 is reduced, and when the luminance evaluation value falls below the lower limit value of the predetermined range, the image projected by the projector 12 Increase the brightness. On the other hand, when the luminance evaluation value is within the predetermined range, the controller 18 maintains the luminance of the image projected by the projector 12.

  A process in which the controller 18 obtains the luminance evaluation value of the captured image by the in-vehicle camera 14 will be specifically described. The brightness evaluation value of the captured image is a value obtained by statistical processing such as an average value, maximum value, minimum value, median value, median of the brightness of pixels included in the portion near the ceiling 38 in the captured image, for example. It is. In addition, the luminance evaluation value may be a moving average value of luminance evaluation values for a predetermined number of frames going back in the past with respect to the captured image data sequentially output from the in-vehicle camera 14 over time. That is, the luminance evaluation value is obtained by obtaining a luminance evaluation value for each of the photographed images having a predetermined number of frames retroactively, and finally obtaining an average value of the luminance evaluation values obtained for each of the photographed image data having the predetermined number of frames. It may be a value obtained as a typical luminance evaluation value. For example, the controller 18 newly obtains a luminance evaluation value every time the in-vehicle camera 14 outputs one frame of captured image data.

  According to such processing, the near-ceiling area 38 is photographed by the in-vehicle camera 14, and the brightness of the projection image is adjusted according to the brightness of the area on the image corresponding to the near-ceiling area 38. The same in-vehicle camera 14 is used for gesture recognition and acquisition of luminance evaluation values corresponding to the ceiling vicinity region 38. Thereby, both the adjustment of the brightness of the projected image and the recognition of the gesture are performed with a simple configuration.

  In the above description, the process of adjusting the brightness of the projected image when the brightness evaluation value is outside the predetermined range has been described. In addition to such processing, the controller 18 may execute processing for adjusting the luminance of the projected image when the change in the luminance evaluation value is outside the predetermined range. The change in the luminance evaluation value is obtained, for example, as a value obtained by subtracting the luminance evaluation value obtained before a predetermined frame from the luminance evaluation value obtained last.

  The embodiment in which the projector 12 projects an image on the ceiling 24 has been described above. The projector 12 may project an image on a window glass, a wall surface of the passenger compartment 20, or the like.

DESCRIPTION OF SYMBOLS 10 In-vehicle projector, 12 Projector, 14 In-vehicle camera, 16 Outside camera, 18 Controller, 20 Car compartment, 22 Room mirror, 24 Ceiling, 28 Rear seat, 30 Overhead module, 32 Driver's seat, 34 Passenger seat, 38 Near the ceiling region.

Claims (4)

A projector that projects an image into the passenger compartment;
A calibration processing unit that obtains correction data for an image projected on the image projection plane, using an image shot when the shooting direction of the camera in the vehicle interior is a direction toward the image projection plane;
A control unit that controls the projector according to an image taken when the shooting direction deviates from a direction toward the image projection plane,
The projector projects the image corrected by the correction data onto the image projection surface according to the control of the control unit. In an in-vehicle projector,
A projector that projects an image into the passenger compartment;
A camera for photographing the vehicle interior;
A direction adjustment unit for adjusting the shooting direction of the camera;
A calibration processing unit that obtains correction data for an image projected on the image projection plane, using an image shot when the shooting direction is a direction toward the image projection plane;
A detection unit that detects a detection object based on an image captured when the imaging direction deviates from a direction toward the image projection plane;
A control unit for controlling the in-vehicle projection device according to the movement of the detection object detected by the detection unit;
A vehicle-mounted projection device comprising: The in-vehicle projector according to claim 2,
After the correction data is acquired, the direction adjustment unit sets the shooting direction in a direction toward the occupant. The in-vehicle projector according to any one of claims 1 to 3,
A brightness adjustment unit that adjusts the brightness of an image projected into the vehicle interior based on an image captured by the camera;
A vehicle-mounted projection device comprising:

Images