WO2020016994A1 - Image distortion inspection device, image distortion inspection method, and program - Google Patents

Abstract

The present invention is provided with: an imaging unit (130) that captures, from the outside of a vehicle (101), an image formed by light outputted by an HUD, which is installed within the vehicle (101) so as to project a virtual image on the basis of an original image, in such a manner as to allow identification of a light position being the position where the light is inputted and of a light direction being the direction from which the light is inputted; a detection unit that detects, from the captured image, a light position and a light direction of light indicating an inspection point contained in the original image; a display position specifying unit which, from the detected light position and the detected light direction, specifies a virtual image display position where a point corresponding to the inspection point is to be displayed in the virtual image; and a distortion computation unit that computes, from a pixel position being the position of a pixel corresponding to the inspection point and the specified virtual image display position, a distortion degree indicating the degree of distortion of the original image in the virtual image.

Description

Image distortion inspection apparatus, image distortion inspection method, and program

The present invention relates to an image distortion inspection device, an image distortion inspection method, and a program.

In recent years, vehicles equipped with a head-up display (HUD) as an image display device have been sold in the general market. The HUD can display information without imposing a burden on the driver by projecting a virtual image in front of the driver and reducing movement of the driver's line of sight.

In an example of a vehicle-mounted HUD, a virtual image is displayed by projecting an image displayed on a HUD light source onto a concave half mirror called a combiner. Since light is projected from the HUD light source to the combiner through a plurality of lenses, an image displayed by the HUD light source is generally displayed as a virtual image as a distorted image.

Therefore, in order to display an image without distortion as a virtual image, it is necessary to perform image distortion correction. In the image distortion correction, an image distortion value which is a degree of distortion of a virtual image is inspected in advance, and a distortion correction value is calculated from the image distortion value. By deforming the display image of the HUD light source according to the calculated distortion correction value, a virtual image without distortion is displayed.

Image distortion values generally include a method of calculating from an optical design value of the HUD and a method of calculating a distortion inspection image projected on a combiner with a camera and calculating the degree of distortion of the captured image. Since optical members such as lenses constituting the HUD are often manufactured as optical members having optical characteristics deviating from optical design values, the image distortion value is calculated from an image obtained by capturing a distortion inspection image with a camera. A calculation method is desirable.

Normally, the distortion inspection image is captured by a camera installed in the driver's seat near the driver's viewpoint in the vehicle. For example, Patent Literature 1 discloses a technique in which a camera installed inside a vehicle is moved, a distortion inspection image is captured at several representative driver viewpoint positions, and an image distortion value and a distortion correction value are calculated. Is described.

JP 2017-47794 A

In the prior art, in order to calculate the image distortion value and the distortion correction value, it is necessary to install a camera inside the vehicle, take an image for image distortion inspection, inspect the degree of image distortion, and then remove the camera from the vehicle interior. It is. For this reason, there is a problem that the work load at the time of the image distortion inspection is high.

Accordingly, an object of one or more aspects of the present invention is to enable an imaging unit outside a vehicle to capture an image and inspect the degree of distortion of the image.

The image distortion inspection apparatus according to one aspect of the present invention is configured to project an image based on light output from an image display device installed inside a vehicle to project a virtual image based on an original image at a position where a light beam is input. An image pickup unit that picks up an image from the outside of the vehicle so that a light ray direction that is a direction in which the light ray is input, and an inspection point included in the original image from the picked-up image A detection unit that detects the light beam position and the light beam direction of the light beam, and a virtual image display in which a point corresponding to the inspection point is displayed in the virtual image from the detected light beam position and the detected light beam direction. A display position specifying unit that specifies a position, a pixel position that is a position of a pixel of the inspection point in the original image, and the specified virtual image display position, a degree of distortion of the original image in the virtual image. A distortion calculator for calculating a degree of distortion indicated, characterized in that it comprises a.

The image distortion inspection method according to one aspect of the present invention includes an image distortion inspection method that projects an image based on light output from an image display device installed inside a vehicle to project a virtual image based on an original image at a position where a light beam is input. A light beam indicating an inspection point included in the original image from the captured image so that a certain light beam position and a light beam direction which is a direction in which the light beam is input can be recognized. The light ray position and the light ray direction are detected, and from the detected light ray position and the detected light ray direction, in the virtual image, a virtual image display position at which a point corresponding to the inspection point is displayed is specified. In the original image, from the pixel position that is the position of the pixel of the inspection point, and from the specified virtual image display position, calculating a distortion degree indicating the degree of distortion of the original image in the virtual image, That.

A program according to one embodiment of the present invention is a program that causes a computer to receive light output from an image display device installed inside a vehicle in order to project a virtual image based on an original image, at a position where a light beam is input. In order to know a certain ray position and a ray direction that is a direction in which the ray is input, from an image taken from outside the vehicle, a ray position of a ray indicating a point for inspection included in the original image and A detection unit that detects a light beam direction, a display position specifying unit that specifies a virtual image display position at which a point corresponding to the inspection point is displayed in the virtual image from the detected light beam position and the detected light beam direction. And a distortion degree indicating a degree of distortion of the original image in the virtual image from a pixel position that is a position of a pixel of the inspection point in the original image and the specified virtual image display position. It characterized in that to function as a strain calculating unit, for calculating a household.

According to one or more aspects of the present invention, an image can be captured by an imaging unit outside the vehicle, and the degree of distortion of the image can be inspected.

FIG. 3 is a schematic diagram showing a usage state of an image distortion inspection system including the image distortion inspection device according to Embodiments 1 to 3. (A) and (b) are schematic diagrams for explaining an image distortion value. It is the schematic which shows the relationship between a virtual image and a virtual image projection image. It is a schematic diagram for explaining image distortion correction processing. (A) and (b) are schematic diagrams for explaining a distortion correction value. FIG. 3 is a block diagram illustrating a hardware configuration example of an image distortion inspection device and a HUD according to Embodiments 1 and 2. It is a schematic diagram for explaining a light beam space image. (A) And (b) is a schematic diagram for explaining an example of a light beam space camera using a micro lens-array. It is a schematic diagram for explaining an example which constitutes a beam space camera by arranging a plurality of normal cameras side by side vertically and horizontally. FIG. 2 is a block diagram schematically showing a functional configuration of the image distortion inspection device according to the first embodiment. It is the schematic which shows an example of the optical model of HUD. It is the schematic which shows the 1st example of the image for distortion inspections. FIG. 3 is a schematic diagram illustrating a relationship between a microlens-array and an imaging surface. FIG. 3 is a schematic diagram for explaining a relationship between a microlens and an imaging surface. It is a schematic diagram for explaining a virtual image display position. It is the schematic which shows the example which replaced the concave mirror of the combiner with the convex lens using the virtual HUD combiner. It is the schematic which shows an example of the information showing the correspondence of the virtual image display position, the light beam position of reflected light, and the light beam direction. It is the schematic which shows an example of the information which shows the correspondence of the reflected light and the transmitted light of a combiner. It is a schematic diagram showing an example of display position correspondence information. 5 is a flowchart illustrating an inspection method in the image distortion inspection device according to the first embodiment. FIG. 9 is a block diagram schematically showing a functional configuration of an image distortion inspection device according to a second embodiment. (A)-(c) is a schematic diagram showing a first example of a plurality of distortion inspection images. (A)-(c) is a schematic diagram showing a second example of a plurality of distortion inspection images. 9 is a flowchart illustrating an inspection method in the image distortion inspection device according to the second embodiment. FIG. 11 is a block diagram illustrating a hardware configuration example of an image distortion inspection device according to a third embodiment. FIG. 11 is a block diagram schematically showing a functional configuration of an image distortion inspection device according to a third embodiment. 9 is a flowchart illustrating an inspection method in the image distortion inspection device according to the third embodiment.

Embodiment 1 FIG.
FIG. 1 is a schematic diagram showing a usage state of an image distortion inspection system 100 including an image distortion inspection device 110 according to the first embodiment.
The image distortion inspection device 110 is arranged outside the vehicle 101, and inspects the degree of image distortion of an image to be displayed using an image display device (not shown) installed inside the vehicle 101. Here, for example, a HUD is used as the image display device. Hereinafter, the image display device is referred to as a HUD.

For example, the light projected from the HUD light source 102 of the HUD is divided into reflected light reflected by the combiner 103 and a small amount of transmitted light transmitted through the combiner 103. Here, the combiner 103 functions as a reflector that reflects part of the light projected from the HUD light source 102.

The transmitted light passes through the combiner 103 from the HUD light source 102 inside the vehicle 101, travels outside the vehicle 101, and reaches the imaging unit 130 of the image distortion inspection device 110.
The reflected light reaches the driver's viewpoint position, and the driver visually recognizes the virtual image 171 displayed at the virtual image display position in front of the vehicle 101 by the reflected light.
Since the light from the HUD light source 102 to the combiner 103 is projected through a plurality of lenses, the direction of the light beam (traveling direction) changes depending on the lens.

光学 Optical members such as lenses constituting the HUD are often manufactured as optical members having optical characteristics deviating from optical design values. Therefore, the direction of the light beam from the HUD light source 102 changes in a direction different from the assumed optical design value. As a result, both the transmitted light and the reflected light of the combiner 103 travel in directions different from the optical design values, and the driver who observes the reflected light visually recognizes the distorted image as the virtual image 171.

The image distortion inspection device 110 captures an image of the transmitted light of the combiner 103 by the imaging unit 130, and detects a light beam position where the light beam of the transmitted light is input and a light beam direction where the light beam is input. Thus, an image distortion value indicating the degree of HUD image distortion is calculated.

FIGS. 2A and 2B are schematic diagrams for explaining image distortion values.
A display image 170 shown in FIG. 2A is an image displayed on a liquid crystal panel (not shown) of the HUD light source 102. The display resolution of the display image 170 is determined by the liquid crystal panel. As an example, when using a Full-HD (High Definition) liquid crystal panel, the horizontal resolution is 1920 pixels and the vertical resolution is 1080 pixels.

A virtual image projection image 172 shown in FIG. 2A is obtained by converting a virtual image 171 (see FIG. 1) in a three-dimensional space displayed at a virtual image display position where a virtual image is displayed into an arbitrary two-dimensional plane. This is an image generated when projected on a space. The virtual image projection image 172 is equivalent to an image obtained when a virtual image in a three-dimensional space is captured by a virtual camera.

Here, in FIG. 1, the traveling direction of the front of the vehicle 101 is the Z axis, the height direction is the Y axis, and the lateral direction is the X axis.
Further, an arbitrary two-dimensional plane space on which a virtual image is projected is a plane space (X axis-Y axis) perpendicular to the Z axis. In FIG. 2, the horizontal direction of the liquid crystal panel is defined as a U axis, and the vertical direction is defined as a V axis.

FIG. 3 is a schematic diagram showing the relationship between the virtual image 171 and the virtual image projection image 172.
The arbitrary position shown in FIG. 3 is a position translated in the Z-axis direction from the point 171a at the virtual image display position formed by the center pixel of the input image input to the HUD to the driver's seat. This arbitrary position may be another position, but is preferably near the driver's viewpoint.

The virtual image projection image 172 assumes a plane perpendicular to the Z-axis direction (X-axis-Y-axis plane) between the virtual image display position and the arbitrary position, and a straight line from the virtual image display position to the arbitrary position intersects with the plane. The position to be calculated is calculated. If there is no HUD image distortion, the virtual image projection image 172 matches the display image 170 (see FIG. 2A).

As shown in FIG. 2A, the image distortion value indicates a pixel of the virtual image projection image 172 corresponding to each pixel of the display image 170. Specifically, as shown in FIG. 2B, the image distortion value is a matrix representing the positional relationship between each pixel of the display image 170 and each pixel of the virtual image projection image 172. For example, let the pixel at the upper left of the display image 170 be the pixel (1,1), and let the image distortion value corresponding to the pixel (1,1) be (U _1-1 , V _1-1 ). Similarly, let the image distortion value corresponding to the pixel (1, 2) of the display image 170 be (U _1-2 , V _1-2 ).

_{Here, U 1-1} represents a pixel position in the U-axis direction on the corresponding virtual projection image to a virtual image display position where the pixel (1,1) is made of a display image _{170, V 1-1,} the V-axis Represents the pixel position in the direction.
When the image resolution of the display image 170 is FULL-HD, the image distortion values are 1920 × 1080 matrices, and are two matrices representing a U-axis value and a V-axis value, respectively.

The HUD calculates a distortion correction value with reference to the image distortion value calculated by the image distortion inspection apparatus 110, and deforms the display image 170 displayed on the HUD light source 102 according to the distortion correction value, thereby performing an image distortion correction process. Is carried out.

FIG. 4 is a schematic diagram for explaining the image distortion correction processing.
An input image 173 shown in FIG. 4 is an image of a content to be displayed via the combiner 103. In the image distortion correction processing, the HUD deforms the input image 173 with reference to the distortion correction value to generate a distortion corrected image 174. The HUD displays the generated distortion-corrected image 174 on the liquid crystal panel on the HUD light source 102 and projects it on the combiner 103. Even when the display image displayed on the liquid crystal panel is distorted by the optical member, the HUD The distortions cancel each other out due to the image deformation, and an image 175 without distortion is displayed as a virtual image.

FIGS. 5A and 5B are schematic diagrams for explaining the distortion correction value.
As shown in FIG. 5A, the distortion correction value corresponding to the pixel position (1, 1) of the distortion corrected image 174 is (HU _1-1 , HV _1-1 ).

The distortion correction value indicates a pixel of the distortion correction image 174 corresponding to each pixel of the input image 173, as shown in FIG. Specifically, as shown in FIG. 5B, the distortion correction value is a matrix representing the positional relationship between each pixel of the input image 173 and each pixel of the distortion correction image 174. For example, assume that the pixel at the upper left of the distortion corrected image 174 is a pixel (1, 1), and a distortion correction value corresponding to the pixel (1, 1) is (HU _1-1 , HV _1-1 ). Similarly, let the distortion correction value corresponding to the pixel (1, 2) of the distortion correction image 174 be (HU _1-2 , HV _1-2 ).

In the image distortion correction processing, the HUD substitutes the pixel value of the pixel position (HU _1-1 , HV _1-1 ) of the input image 173 into the pixel value of the pixel position (1,1) of the distortion corrected image 174. Generates a distortion corrected image 174.

FIG. 6 is a block diagram illustrating a hardware configuration example of the image distortion inspection device 110 and the HUD.
The image distortion inspection apparatus 110 includes an image processing device 111, an imaging device 117, and a wireless communication device 118.

The image processing device 111 is a device that calculates an image distortion value from an image captured by the imaging device 117. The image processing device 111 is an interface for connecting a CPU (Central Processing Unit) 112 functioning as a processor, a RAM (Random Access Memory) 113 and a ROM (Read Only Memory) 114 functioning as a memory, and an imaging device 117. It comprises a sensor IO (Input-Output) 115 and a wireless communication IO 116 which is an interface for connecting a wireless communication device 118.

The functions executed by the image processing device 111 may be realized by the CPU 112 reading a program stored in the ROM 114 into the RAM 113 and executing the read program. Such a program may be provided through a network, or may be provided by being recorded on a recording medium. Such a program may be provided as a program product, for example. The image processing device 111 may be provided as a computer.
As the image processing device 111, a general computer may be used, or a device in which the above devices are integrated into an embedded device or an FPGA board may be used.

The imaging device 117 is a light space camera capable of acquiring a light space image.
The light ray space image is an image showing the position and direction of a light ray existing in an arbitrary three-dimensional space. As shown in FIG. 7, if the position and direction of the light beam that reaches the two-dimensional plane can be obtained, a light space image in a three-dimensional space can be calculated from the light space image.

A normal camera cannot record the direction of a light beam only, but it can record the direction of a light beam by using a light beam space camera.

FIGS. 8A and 8B are schematic diagrams for explaining an example of a light beam spatial camera using a microlens-array. FIG. 8A shows an optical model of a normal camera, and FIG. 8B shows an optical model of a ray space camera using a microlens-array.

As shown in FIG. 8A, in a normal camera, light from a subject is focused on an imaging surface 117b by a lens 117a to capture an image of the subject. For example, in FIG. 8A, light from the subject 1 is condensed on the imaging surface 117b by the lens 117a, and an image in which the subject 1 is in focus can be obtained. The light of the subject 2 is not converged on the imaging surface 117b and becomes an out-of-focus image.

As shown in FIG. 8B, in the light beam space camera, a microlens-array in which a plurality of microlenses 117c are arranged is provided between a lens 117a and an imaging surface 117b.
The imaging surface 117b is arranged at a position where the light beam transmitted through the micro lens 117c is focused.

Since the lens is a device that converts the direction of a light beam into the position of an image, the direction of the light beam that has reached the microlens 117c can be calculated from the position of the light beam that has reached the imaging surface 117b.

FIG. 8B shows an example in which a microlens-array is added between the lens 117a and the imaging surface 117b. However, as shown in FIG. Can also be configured by arranging the cameras 117d vertically and horizontally and horizontally.

Returning to FIG. 6, the wireless communication device 118 is a communication device for performing wireless communication with the HUD 150, and functions as a communication unit for performing communication with the HUD 150. As the wireless communication, for example, a wireless LAN (Local Area Network) such as Wi-Fi (registered trademark) may be used.

The HUD 150 includes an image processing device 151, an image display device 157, and a wireless communication device 158.

The image processing device 151 is a device that outputs a display image to the image display device 157. The image processing device 151 is an interface for connecting the CPU 152 that functions as a processor, the RAM 153 and the ROM 154 that function as memories, the display IO 155 that is an interface for connecting the image display device 157, and the wireless communication device 158. And a wireless communication IO 156.

At the time of the image distortion correction processing, the image processing device 151 deforms the input image with reference to the distortion correction value calculated from the image distortion value inspected by the image distortion inspection device 110, and performs distortion correction as the deformed input image. The image is output to the image display device 157.

As the image processing device 151, a general computer may be used, or a device in which the above devices are integrated into an embedded device or an FPGA board may be used.

The image display device 157 is an image projection device including the HUD light source 102 and the combiner 103. For example, the image display device 157 projects the distortion corrected image acquired from the image processing device 151 from the HUD light source 102 to the combiner 103 to display a virtual image.
The wireless communication device 158 is a communication device that performs wireless communication with the image distortion inspection device 110, and functions as a communication unit that communicates with the image distortion inspection device 110.

In FIG. 6, wireless communication is used for communication between the image distortion inspection device 110 and the HUD 150. However, wired communication may be performed using a wired cable such as a LAN cable or a CAN cable. In this case, although not shown, the image distortion inspection apparatus 110 and the HUD 150 may each include a wired communication device for performing wired communication.

(4) The image distortion value inspected by the image distortion inspection apparatus 110 may be stored in an arbitrary recording medium, and the recording medium may be read by the HUD 150 so that a hardware configuration without a wireless communication device or a priority communication device may be adopted. In this case, although not shown, the image distortion inspection apparatus 110 may include a writing device for writing data on a recording medium, and the HUD 150 may include a reading device for reading data from the recording medium.

FIG. 10 is a block diagram schematically showing a functional configuration of the image distortion inspection device 110.
The image distortion inspection device 110 includes an imaging unit 130, a light ray detection unit 131, a display position identification unit 132, a display position correspondence information storage unit 133, and an image distortion value calculation unit 134.
The hardware that implements the function of the imaging unit 130 is the imaging device 117 in FIG. The hardware that realizes the functions of the light ray detection unit 131, the display position identification unit 132, the display position correspondence information storage unit 133, and the image distortion value calculation unit 134 is the image processing device 111 in FIG.

The imaging unit 130 converts the image of the light output from the HUD 150 to project a virtual image into the vehicle 101 so that the position of the light beam and the direction of the light beam can be recognized. Image from outside. Specifically, when the HUD light source 102 displays a distortion inspection image as an original image, the imaging unit 130 captures a light space image output from the HUD light source 102 and transmitted by the combiner 103.

FIG. 11 is a schematic diagram illustrating an example of an optical model of the HUD 150 that is the image display device 157.
The light output from the HUD light source 102 reaches the combiner 103 and is split into reflected light and transmitted light. The light beam of the transmitted light is input to the imaging unit 130.

FIG. 12 is a schematic diagram illustrating an example of a distortion inspection image.
The distortion inspection image 176 shown in FIG. 12 is an image in which vertical and horizontal white lines are superimposed on a black background. The degree of distortion of the image is inspected from the degree of distortion of the white line. In other words, the pixels constituting the white line are used as inspection points, and the degree of image distortion is inspected using the inspection points. Note that the inspection point may be a pixel where white lines cross each other.

戻 り Returning to FIG. 10, the light beam detection unit 131 is a detection unit that detects the light beam position and the light beam direction of the light beam indicating the inspection point from the image captured by the imaging unit 130. In the following, an example will be described in which a light beam position and a light beam direction are detected when a light beam space camera using a microlens-array is used as the imaging device 117.

FIG. 13 is a schematic diagram showing the relationship between the microlens-array and the imaging surface.
The light beam that has reached the microlens 117c from the lens 117a is focused on the imaging surface 117b.

FIG. 14 is a schematic diagram for explaining the relationship between the micro lens 117c and the imaging surface 117b.
As shown in FIG. 14, one microlens 117c # 1 included in the microlens-array is assigned a pixel area 117b # 1 including a plurality of pixels included in the imaging surface 117b. ing. The pixel region 117b # 1 includes pixels in a range where light that has reached the microlenses 117c # 1 from the lens 117a is focused on the imaging surface 117b.

The ray position is detected from the position of the microlens-array that the ray has reached. The method of detecting the position of the light beam is almost the same as that of the normal camera. In the light space camera, one microlens corresponds to one pixel on the imaging surface of the normal camera. However, in the light beam space camera, the light beam that has reached the microlens is focused on the imaging surface, so that a light beam that is not focused can be acquired at the installation position of the microlens-array. Here, the position of the light beam indicating the inspection point is detected.

Since the lens is a device that converts the direction of a light beam into the position of an image, the position of the light beam that reaches the imaging surface 117b changes depending on the direction of the light beam that has reached the microlens 117c. For this reason, the direction of the light beam can be detected from the position where the light beam reaches the imaging surface 117b. Specifically, the direction of the light beam can be detected by the position where the light beam from one micro lens 117c # 1 is focused on the corresponding pixel area 117b # 1. Here, the direction of the light beam indicating the inspection point is detected. For example, when the ray position of a ray indicating a point for inspection is specified, the ray direction of the ray input to the ray position is detected.

Returning to FIG. 10, the display position specifying unit 132 specifies the virtual image display position of the HUD 150 from the light beam position and the light beam direction detected by the light beam detecting unit 131. Specifically, the display position specifying unit 132 corresponds to the inspection point in the virtual image projected by the HUD 150 from the ray position and the ray direction of the point corresponding to the inspection point detected by the ray detection unit 131. The virtual image display position which is the position where the point to be displayed is displayed is specified.

In specifying the virtual image display position, information indicating the correspondence between the virtual image display position, the light ray position and the light ray direction is created in advance, and stored in the display position correspondence information storage unit 133 as the display position correspondence information. Then, the display position specifying unit 132 specifies the virtual image display position corresponding to the light beam position and the light beam direction detected by the light beam detection unit 131 by referring to the display position correspondence information.

The display position correspondence information storage unit 133 is an information storage unit that stores display position correspondence information. The display position correspondence information is derived from the optical design values of the HUD 150.

FIG. 15 is a schematic diagram for explaining a virtual image display position.
The driver visually recognizes the virtual image at the virtual image display position by the reflected light of the combiner 103.
The reflecting surface of the combiner 103 has a concave shape, and the light from the HUD light source 102 installed inside the vehicle 101 is reflected by the concave mirror. The reflected light from the concave mirror has the same function as the transmitted light from the convex lens. According to the virtual HUD light source 102 # 1 in which the HUD light source 102 is virtually inverted to the outside of the vehicle 101, the concave mirror is , Can be replaced as a convex lens.

FIG. 16 is a schematic diagram showing an example in which the concave mirror of the combiner 103 is replaced with a convex lens 103 # 1 using the virtual HUD light source 102 # 1.
Assuming that the focal length of the convex lens 103 # 1 shown in FIG. 16 is f and the distance between the virtual HUD light source 102 # 1 and the convex lens 103 # 1 is a, the Z-axis direction from the convex lens 103 # 1 to the virtual image display position Satisfies the following relational expression (1).

The focal length f of the convex lens 103 # 1, which is the optical design value of the HUD 150, is the same as the focal length of the concave mirror of the combiner 103, and is known. The distance a between the virtual HUD light source 102 # 1 and the convex lens 103 # 1 is the same as the distance from the HUD light source 102 to the combiner 103 and is known. Therefore, the distance b in the Z-axis direction from the convex lens 103 # 1 to the virtual image display position can be calculated by Expression (1).

Also, based on the distance b calculated from the following equation (2) and the display position (X _S1, Y _S1, a) of each pixel of the virtual HUD light source 102 # 1 _, a virtual image display position (X _V1 , _YV1 , b) can also be calculated.

As described above, the virtual image display position can be calculated from the optical design value of the HUD 150.
Further, the position and direction of the light beam reflected from the HUD light source 102 and reflected by the combiner 103 are also specified based on the installation position of the HUD light source 102 and the optical characteristics of the combiner 103. This makes it possible to create information indicating the correspondence between the virtual image display position and the ray position and ray direction of the reflected light.

FIG. 17 is a schematic diagram illustrating an example of information indicating a correspondence relationship between a virtual image display position and a light ray position and a light ray direction of reflected light.
The virtual image display position is represented by one point in a three-dimensional space. (X _V1 , Y _V1 , Z _V1 ) in FIG. 17 indicates the position of one point of the X-axis value X _V1 , the Y-axis value Y _V1 , and the Z-axis value Z _V1 .
The ray position and ray direction of the reflected light are represented by one point on the three-dimensional space and the vector direction. In FIG. 17, (X _R1 , Y _R1 , Z _R1 , θ _R1 , φ _R1 ) indicates the position of one point of the X-axis value X _R1 , the Y-axis value of Y _R1 , and the Z-axis value of Z _R1. The angle of the Z axis plane is θ _R1 , and the angle between the Y axis and the Z axis direction is the vector direction of φ _R1 .

The information shown in Figure 17, ray position and beam direction _{(X R1, Y R1, Z} R1, θ R1, φ R1) a virtual image display position at which reflected light is made is _{(X V1, Y V1,} Z _V1), and the light ray position and ray direction _{(X R2, Y R2, Z} R2, θ R2, virtual image display position phi _R2) in which reflected light is made is _{(X V2, Y V2, Z} V2).

In FIG. 17, the light ray position and the light ray direction of the reflected light are represented by one point in the three-dimensional space and the vector direction. However, as shown in FIG. May be expressed by the position of intersection with the two-dimensional plane and the vector direction. In this case, for example, ray position and beam direction of FIG. _{17 (X R1, Y R1,} Z R1, θ R1, φ R1) is crossed position _{(X R1, Y} R1) and the vector direction (θ _R1, φ _R1) Can be represented by

The driver visually recognizes the virtual image by the reflected light of the combiner 103, but the light beam detector 131 detects the transmitted light of the combiner 103. For this reason, information indicating the correspondence between the reflected light and the transmitted light of the combiner 103 is calculated in advance. For example, the correspondence between the ray position and the ray direction of the reflected light and the ray position and the ray direction of the transmitted light may be specified with reference to the optical characteristics of the combiner 103.

FIG. 18 is a schematic diagram illustrating an example of information indicating the correspondence between the reflected light and the transmitted light of the combiner 103.
The light ray position and the light ray direction of the reflected light and the transmitted light of the combiner 103 are represented by one point on the three-dimensional space and the vector direction, respectively. For example, (X _R1 , Y _R1 , Z _R1 , θ _R1 , φ _R1 ) shown in FIG. 18 represent the ray position and ray direction of the reflected light, and (X _T1 , Y _T1 , Z _T1 , θ _T1). , Φ _T1 ) represents the ray position and ray direction of the transmitted light corresponding to the ray position and ray direction of the reflected light.

The display position correspondence information indicating the correspondence between the virtual image display position and the ray position and the ray direction of the transmitted light is information indicating the correspondence between the virtual image display position and the reflected light, and the correspondence between the reflected light and the transmitted light. It is created from the information to represent.
For example, FIG. 19 is a schematic diagram illustrating an example of the display position correspondence information.
The display position correspondence information shown in FIG. 19 is created based on the information shown in FIG. 17 and the information shown in FIG.

From the above, the display position correspondence information can be derived from the optical design values of the HUD 150. The display position correspondence information storage unit 133 shown in FIG. 10 stores such display position correspondence information, and the display position identification unit 132 stores the display position correspondence information stored in the display position correspondence information storage unit 133. , The virtual image display position can be specified from the light beam position and the light beam direction of the transmitted light detected by the light beam detection unit 131.

In the above description, an example in which the display position correspondence information is derived from the optical design values of the HUD 150 has been described. However, in order to create more accurate display position correspondence information, the transmission light of the combiner 103 and the HUD 150 are transmitted in advance. An image may be captured by the imaging unit 130 using both the reflected light and the light ray position and the light ray direction may be calculated to create the display position correspondence information.

The image distortion value calculation unit 134 is a distortion calculation unit that calculates an image distortion value indicating the degree of image distortion from the virtual image display position specified by the display position specification unit 132. Specifically, the image distortion value calculation unit 134 calculates a pixel position that is a position of a pixel of the inspection point in the distortion inspection image that is the original image and a virtual image display position corresponding to the inspection point. In the virtual image projected by the HUD 150, an image distortion value indicating a degree of distortion that is a degree of distortion of the distortion inspection image is calculated. Note that pixels other than the inspection point may be interpolated using the image distortion value calculated from the inspection point.

In the calculation of the image distortion value, a virtual camera is arranged at an arbitrary position shown in FIG. 3, and a pixel position of an image obtained when a virtual image is captured by the virtual camera is calculated. An example of the calculation formula is shown in formula (3).

Here, f _C represents a virtual camera focal length, and (X _V1 , Y _V1 , Z _V1 ) represents a virtual image display position in a three-dimensional space. (U, V) represents the pixel position of the image captured by the virtual camera, and this value is the image distortion value.

As described above, an image capable of inspecting the degree of distortion of an image by capturing an image for distortion inspection with the imaging unit 130 installed outside the vehicle 101 and measuring the ray position and ray direction of the HUD light source changed by the optical member of the HUD. A distortion inspection device can be provided.

FIG. 20 is a flowchart illustrating an inspection method in image distortion inspection apparatus 110 according to Embodiment 1.
The distortion inspection image 176 is displayed on the HUD light source 102, and the imaging unit 130 captures an image using the transmitted light transmitted through the combiner 103 (S10).

Next, the light beam detection unit 131 calculates the light beam position and the light beam direction of the transmitted light from the image captured in step S10 (S11). For example, the light ray detection unit 131 calculates a light ray position and a light ray direction from a position where transmitted light reaches an imaging surface of the light ray space camera by using a light ray space camera for the imaging unit 130.

Next, the display position specifying unit 132 specifies a virtual image display position from the light beam position and the light beam direction calculated in step S12 (S12). For example, the display position specifying unit 132 specifies the virtual image display position corresponding to the light ray position and the light ray direction calculated in step S12 with reference to the display position corresponding information stored in the display position corresponding information storage unit 133. .

Next, the image distortion value calculator 134 calculates an image distortion value from the virtual image display position specified in step S12 (S13). For example, the image distortion value calculation unit 134 arranges a virtual camera at an arbitrary position shown in FIG. 3 and obtains a pixel position of an image obtained when a virtual image is captured by the virtual camera. Calculate the image distortion value.

As described above, the distortion inspection image 176 is captured by the imaging unit 130, which is a light space camera installed outside the vehicle 101, and the light position and the light direction of the HUD light source 102 changed by the optical member of the HUD 150 are measured. And an image distortion inspection method capable of inspecting the degree of distortion.

Embodiment 2 FIG.
As shown in FIG. 1, the image distortion inspection system 200 according to the second embodiment includes an image distortion inspection device 210.
As shown in FIG. 6, the hardware configuration of the image distortion inspection device 210 according to the second embodiment is the same as the hardware configuration of the image distortion inspection device 110 according to the first embodiment. Also in the second embodiment, the same HUD 150 as in the first embodiment is used.

In the second embodiment, the image distortion inspection apparatus 210 and the image distortion inspection method that improve the inspection accuracy of the image distortion value by changing the distortion inspection image over time are provided.

FIG. 21 is a block diagram schematically showing a functional configuration of the image distortion inspection device 210 according to the second embodiment.
The image distortion inspection device 210 includes an imaging unit 130, a light ray detection unit 131, a display position identification unit 132, a display position correspondence information storage unit 133, an image distortion value calculation unit 134, an inspection image storage unit 235, A communication control unit 236 and a communication unit 237 are provided.

The imaging unit 130, the light beam detection unit 131, the display position specifying unit 132, the display position correspondence information storage unit 133, and the image distortion value calculation unit 134 in the second embodiment are the same as those of the first embodiment. This is the same as the display position specifying unit 132, the display position correspondence information storage unit 133, and the image distortion value calculation unit 134.
The hardware that implements the functions of the inspection image storage unit 235 and the communication control unit 236 is the image processing device 111 in FIG. The hardware that implements the function of the communication unit 237 is the wireless communication device 118 in FIG.

The inspection image storage unit 235 stores a plurality of distortion inspection images. It is assumed that the plurality of distortion inspection images are different from each other.
In the second embodiment, the distortion inspection image displayed on the HUD light source 102 is changed as time elapses. For example, among the distortion inspection images 276A to 276C shown in FIGS. 22A to 22C, the displayed image is changed over time, and the boundary between the black image region and the white image region is changed. Line distortion may be checked. In other words, the image distortion may be detected by using the pixels forming the boundary between the image area and the white image area as inspection points.

In addition, as in the distortion inspection images 276D to 276F shown in FIGS. 23A to 23C, only one pixel is displayed on a black background, and the position of the white point changes with time. May be scanned in the vertical and horizontal directions. In other words, the image distortion may be detected using the white point as the inspection point.

The communication control unit 236 controls communication performed with the HUD 150. For example, the communication control unit 236 sequentially selects one distortion inspection image from a plurality of distortion inspection images stored in the inspection image storage unit 235 with time, and selects the selected distortion inspection image. Is transmitted to the HUD 150 by the communication unit 237. For example, the communication control unit 236 may sequentially transmit one distortion inspection image to the communication unit 237 every time a predetermined time elapses.

The communication unit 237 performs communication with the HUD 150. Here, the communication unit 237 performs wireless communication.

The HUD 150 displays the distortion inspection image received by the wireless communication device 158 on the liquid crystal panel of the HUD light source 102 by the image processing device 151. Next, in order to notify the image distortion inspection device 210 that the image for distortion inspection has been displayed on the HUD 150, the image processing device 151 of the HUD 150 transmits the image display notification signal to the wireless communication device 158 to the image distortion inspection device 210. Send.

通信 The communication unit 237 of the image distortion inspection device 210 receives the image display notification signal. When the communication unit 237 receives the image display notification signal, the communication control unit 236 notifies the imaging unit 130 that such a signal has been received. After receiving the image display notification signal, the imaging unit 130 performs a process of capturing an image for distortion inspection.

As described above, the image distortion value of the HUD 150 can be inspected using the distortion inspection image that changes with time. By changing the distortion inspection image, it is possible to detect the light beam position and the light beam direction corresponding to each pixel of the liquid crystal panel of the HUD light source 102, and thus provide the image distortion inspection device 210 with improved image distortion value inspection accuracy. can do.

FIG. 24 is a flowchart illustrating an inspection method in the image distortion inspection device 210 according to the second embodiment.
First, the communication control unit 236 selects one distortion inspection image from the plurality of distortion inspection images stored in the inspection image storage unit 235, and reads the selected distortion inspection image (S20). For example, the inspection image storage unit 235 stores a plurality of distortion inspection images so that the selection order can be recognized. The communication control unit 236 selects one distortion inspection image in accordance with the order and reads the selected distortion inspection image.

Next, the communication control unit 236 causes the communication unit 237 to transmit the distortion inspection image read in step S20 to the HUD 150 in order to display the image on the liquid crystal panel of the HUD light source 102 (S21).

Next, the communication control unit 236 determines that the distortion inspection image transmitted in step S21 is displayed on the liquid crystal panel of the HUD light source 102 by the communication unit 237 receiving the image display notification signal transmitted from the HUD 150. Is confirmed (S22). Then, the process proceeds to step S23.

処理 The processing in steps S23 to S26 in FIG. 24 is the same as the processing in steps S10 to S13 in FIG. 20 of the first embodiment. However, after step S26, the process proceeds to step S27.

In step S27, the communication control unit 236 determines whether all of the plurality of distortion inspection images stored in the inspection image storage unit 235 have been displayed. If all the distortion inspection images have been displayed (Yes in S27), the process ends. If there are distortion inspection images that have not been displayed (No in S27), the process proceeds to step S20. Return to

According to the second embodiment, the imaging unit 130 installed outside the vehicle 101 captures an image for distortion inspection that changes with time, and the light beam position and light beam of the HUD light source 102 changed by the optical member of the HUD 150. It is possible to provide an image distortion inspection method capable of inspecting the degree of image distortion by measuring the direction.

Embodiment 3 FIG.
As shown in FIG. 1, an image distortion inspection system 300 according to the third embodiment includes an image distortion inspection device 310.
In the third embodiment, even if the relative position between image distortion inspection device 310 and vehicle 101 is not constant, the relative position between image distortion inspection device 310 and vehicle 101 is acquired and image distortion inspection device 310 is moved. Thus, the image distortion value of the HUD 150 can be inspected.
The HUD 150 according to the third embodiment is the same as the HUD 150 according to the first embodiment.

FIG. 25 is a block diagram illustrating a hardware configuration example of the image distortion inspection device 310 according to the third embodiment.
The image distortion inspection device 310 includes an image processing device 311, an imaging device 117, a wireless communication device 118, a position measurement device 320, and a moving mechanism device 321.
The imaging device 117 and the wireless communication device 118 according to the third embodiment are the same as the imaging device 117 and the wireless communication device 118 according to the first embodiment.

The image processing device 311 includes a CPU 112, a RAM 113, a ROM 114, a sensor IO 315, a wireless communication IO 116, and a movement control IO 319.
The CPU 112, the RAM 113, the ROM 114, and the wireless communication IO 116 according to the third embodiment are the same as the CPU 112, the RAM 113, the ROM 114, and the wireless communication IO 116 according to the first embodiment.

The sensor IO 315 is an interface for connecting the imaging device 117 and the position measurement device 320.

The movement control IO 319 is an interface for controlling the movement mechanism device 321 in order to change the position of the imaging device 117. For example, the CPU 112 controls the direction and rotation of the wheels of the movement mechanism device 321 via the movement control IO 319.

The position measuring device 320 measures the position of the vehicle 101. For example, the position measurement device 320 measures the position of the vehicle 101 using a laser sensor such as LiDAR (Light Detection and Ranging). Specifically, the position measuring device 320 measures the position of the vehicle 101 by irradiating the vehicle 101 with a laser from the position measuring device 320 and receiving the laser reflected from the vehicle 101.

The moving mechanism device 321 moves the image distortion inspection device 310 to change the position of the imaging device 117. For example, the moving mechanism device 321 includes four wheels (not shown) installed below the image distortion inspection apparatus 310, and a motor (not shown) for moving the wheels. The CPU 112 moves the image distortion inspection device 310 by controlling the direction and rotation of the wheels via the movement control IO 319.

FIG. 26 is a block diagram schematically showing a functional configuration of the image distortion inspection device 310 according to the third embodiment.
The image distortion inspection device 310 includes an imaging unit 130, a light ray detection unit 131, a display position identification unit 132, a display position correspondence information storage unit 133, an image distortion value calculation unit 134, a position detection unit 338, and a relative position. An acquisition unit 339, a movement control unit 340, and an imaging position change unit 341 are provided.

The imaging unit 130, the light beam detection unit 131, the display position specifying unit 132, the display position correspondence information storage unit 133, and the image distortion value calculation unit 134 according to the third embodiment are the same as those of the first embodiment. This is the same as the display position specifying unit 132, the display position correspondence information storage unit 133, and the image distortion value calculation unit 134.
In the third embodiment, similarly to the second embodiment, an inspection image storage unit 235, a communication control unit 236, and a communication unit 237 are provided, and a plurality of distortion inspection images are stored. Then, the distortion inspection images may be transmitted to the HUD 150 in order.

The hardware that implements the function of the position detection unit 338 is the position measurement device 320 in FIG. Hardware that implements the functions of the relative position acquisition unit 339 and the movement control unit 340 is the image processing device 311 in FIG. Hardware that implements the function of the imaging position changing unit 341 is the moving mechanism device 321 in FIG.

The position detection unit 338 detects a vehicle position that is the position of the vehicle 101. Specifically, the position detection unit 338 detects the vehicle position of the vehicle 101 by measuring the outer shape of the vehicle 101 using LiDAR.
The relative position acquisition unit 339 acquires a relative position, which is a relative position between the image distortion inspection device 310 and the vehicle 101, from the vehicle position detected by the position detection unit 338. It is assumed that the positional relationship between the vehicle 101 and the HUD 150 installed inside the vehicle 101 is known from design values when the HUD 150 is mounted. Thus, the relative position between the image distortion inspection device 310 and the vehicle 101 can be obtained.

Here, an example in which the position detection unit 338 is realized by the position measurement device 320 using LiDAR has been described, but the position detection unit 338 may be realized by the imaging device 117, for example. Specifically, the relative position acquisition unit 339 stores the image feature points of the vehicle 101 and the positions thereof in advance, and detects the image feature points from the image captured by the imaging device 117, thereby detecting the vehicle feature points. The relative position between 101 and the imaging device 117 may be calculated.

In addition, an in-vehicle sensor (not shown) installed in the vehicle 101 may be used for position measurement. A calibration board or the like is attached to the image distortion inspection device 310, and the position of the image distortion inspection device 310 is measured by measuring with a vehicle-mounted sensor. The in-vehicle sensor transmits the measurement result to the image distortion inspection device 310 wirelessly, for example. The relative position acquisition unit 339 can acquire the measurement result via the wireless communication device 118 and acquire the relative position between the vehicle 101 and the image distortion inspection device 310.

The movement control unit 340 calculates a movement amount from the relative position acquired by the relative position acquisition unit 339 to a position predetermined as a position for performing an inspection, and instructs the imaging position change unit 341 to calculate the amount of movement. The image distortion inspection device 310 is moved.
The imaging position changing unit 341 moves the image distortion inspection device 310 by the movement amount calculated by the movement control unit 340 in accordance with the instruction from the movement control unit 340, and thereby the installation position of the imaging device 117, in other words, A moving unit that changes an imaging position of the imaging device 117.

The movement amount of the image distortion inspection device 310 is obtained, for example, in advance by setting a relative position between the image distortion inspection device 310 and the vehicle 101 suitable for an image distortion value inspection process, and by the relative position acquisition unit 339. What is necessary is just to calculate from the relative position and the difference value between the relative position.

As described above, even if the relative position between the image distortion inspection device 310 and the vehicle 101 is not adjusted, the image distortion inspection device 310 automatically moves so as to be a relative position suitable for the image distortion value inspection processing. Become.

FIG. 27 is a flowchart illustrating an inspection method in the image distortion inspection device 310 according to the third embodiment.
First, the position detection unit 338 detects the position of the vehicle 101 by measuring the external shape of the vehicle 101 equipped with the HUD 150 for inspecting the image distortion value (S30).

Next, the relative position acquisition unit 339 acquires the relative position between the vehicle 101 and the image distortion inspection device 310 from the position of the vehicle 101 detected in step S30, and the movement control unit 340 sets the relative position to the acquired relative position. Then, a difference value from the previously stored relative position is obtained, and the movement amount of the image distortion inspection device 310 is calculated (S31).

Next, the imaging position changing unit 341 changes the imaging position by moving the image distortion inspection device 310 according to the movement amount calculated in step S31 (S32).

The processing in steps S33 to S36 in FIG. 27 is the same as the processing in steps S10 to S13 in FIG. 20 of the first embodiment.
As described above, even if the relative position between the image distortion inspection device 310 and the vehicle 101 is not constant, the HUD 150 is obtained by acquiring the relative position between the image distortion inspection device 310 and the vehicle 101 and moving the image distortion inspection device 310. Can be provided.

Although the first to third embodiments described above are examples in which the degree of image distortion of the HUD 150 is inspected as an image display device, the first to third embodiments are not limited to the above and depart from the spirit. It can be changed appropriately within a range not to be performed.

100, 200, 300 image distortion inspection system, {101} vehicle, {102} HUD light source, {103} combiner, {110, 210, 310} image distortion inspection device, {111, 311} image processing device, {112} CPU, {113} RAM, {114} ROM, {115} sensor IO, 116 wireless communication IO, {117} imaging device, {118} wireless communication device, {319} movement control IO, {320} position measurement device, {321} movement mechanism device, {130} imaging unit, {131} ray detection unit, {132} display position identification unit, {133} display position correspondence information storage Unit, {134} image distortion value calculation unit, {235} inspection image storage unit, {236} communication control unit, {237} communication unit, {338} position detection unit, {339} relative position acquisition unit, # 3 0 movement control unit, 341 imaging position change unit, 0.99 HUD, 151 image processing device, 152 CPU, 153 RAM, 154 ROM, 155 display IO, 156 wireless communication IO, 157 image display device, 158 a wireless communications device.

Claims (8)

1. In order to project a virtual image based on the original image, an image by light output from an image display device installed inside the vehicle is a light beam position where a light beam is input and a direction where the light beam is input. An imaging unit that captures an image from outside the vehicle so that a light beam direction can be understood;
   From the captured image, a detection unit that detects the ray position and ray direction of a ray indicating a point for inspection included in the original image,
   From the detected light beam position and the detected light beam direction, in the virtual image, a display position specifying unit that specifies a virtual image display position where a point corresponding to the inspection point is displayed.
   In the original image, a pixel position that is the position of the pixel of the inspection point, and the specified virtual image display position, a distortion calculation unit that calculates a distortion degree indicating the degree of distortion of the original image in the virtual image. An image distortion inspection device, comprising:
2. The imaging unit captures a ray space image as the virtual image,
   The said detection part detects the light beam position of the light beam which shows the said point for a test | inspection, and the light beam direction of the light beam which shows the said point for a test | inspection from the said light beam space image. Image distortion inspection equipment.
3. A plurality of light beam positions and a plurality of light beam directions, and a plurality of virtual image display positions, further comprising an information storage unit for storing display position correspondence information for associating,
   The display position specifying unit refers to the display position correspondence information to correspond to the detected light beam position and the virtual image display position associated with the detected light beam direction to the inspection point. The image distortion inspection apparatus according to claim 1, wherein the point is specified as a virtual image display position at which a point is displayed.
4. An inspection image storage unit that stores a plurality of distortion inspection images used as the original image,
   The image display device further includes: a communication unit that transmits each of the plurality of distortion inspection images to the image display device in a predetermined order as the original image. The image distortion inspection device according to any one of claims 1 to 3.
5. The image distortion inspection apparatus according to claim 4, wherein the communication unit transmits the plurality of distortion inspection images one by one to the image display device each time a predetermined time elapses. .
6. A position detection unit that detects a vehicle position that is the position of the vehicle,
   From the vehicle position, a relative position acquisition unit that acquires a relative position that is a relative position between the image distortion inspection device and the vehicle,
   A movement control unit that calculates a movement amount from the relative position to a predetermined position,
   The image distortion inspection device according to any one of claims 1 to 5, further comprising: a moving unit configured to move the image distortion inspection device to the predetermined position according to the movement amount.
7. In order to project a virtual image based on the original image, an image by light output from an image display device installed inside the vehicle is a light beam position where a light beam is input and a direction where the light beam is input. Image from the outside of the vehicle so that the direction of the light beam can be understood,
   From the captured image, to detect the ray position and ray direction of a ray indicating a point for inspection included in the original image,
   From the detected light beam position and the detected light beam direction, in the virtual image, specify a virtual image display position at which a point corresponding to the inspection point is displayed,
   From the pixel position of the pixel of the inspection point in the original image and the specified virtual image display position, calculate a degree of distortion indicating the degree of distortion of the original image in the virtual image. Image distortion inspection method.
8. Computer
   Receiving light output from an image display device installed inside the vehicle to project a virtual image based on the original image, a light beam position where a light beam is input and a direction where the light beam is input. A detection unit that detects a light ray position and a light ray direction of a light ray indicating a point for inspection included in the original image from an image captured from outside the vehicle so that the light ray direction can be understood.
   From the detected light beam position and the detected light beam direction, in the virtual image, a display position specifying unit that specifies a virtual image display position at which a point corresponding to the inspection point is displayed, and
   In the original image, a pixel position, which is the position of the pixel of the inspection point, and the specified virtual image display position, a distortion calculation unit that calculates a distortion degree indicating the degree of distortion of the original image in the virtual image. A program characterized by functioning as a program.

Images