JP2017047794A - Distortion correction method for head-up display and distortion correction device for head-up display using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a head-up display (HUD) image without distortion with small calculation amount regardless of the position of a viewpoint.SOLUTION: An installation position of an imaging section 40 placed to be directed in a prescribed direction is detected at the position of a viewpoint 52 of a driver, a projection image Ip of a reference pattern Ir projected on a front visual field of the driver is captured by the imaging section 40, and a first coordinate correspondence relation between the reference pattern Ir and a point corresponding to the projection image Ip is calculated. A second coordinate corresponding relation between the reference pattern Ir at the actually detected position of the viewpoint 52 and a point corresponding to a projection image Ip which is predicted to be captured from the viewpoint 52 is estimated using the first coordinate correspondence relations obtained at a plurality of installation positions of the imaging section 40, and coordinate conversion tables Mip_x and Mip_y are generated to deform the reference pattern Ir so that the projection image Ip can be observed without distortion from the viewpoint 52. On the basis of the generated coordinate conversion tables, preliminarily deformed image information is generated and projected so as to generate a head-up display (HUD) image Ii.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a vehicle head-up display device that displays a necessary information by forming a virtual image in front of a driver.

  Recently, a head that can project a display image on the front window glass of an aircraft, an automobile, etc. to form a virtual image in front of the driver or driver, and visually recognize the display image without diverting the line of sight from the front. Up-display devices have been put into practical use (for example, Patent Literature 1 and Non-Patent Literature 1).

  In such a head-up display device, since a virtual image is displayed on a reflective member called a combiner or a front window glass, the displayed virtual image is distorted. Therefore, for example, in Patent Document 1, the distortion of the virtual image is reduced by adjusting the optical parameter of the optical system that projects the virtual image. In Non-Patent Document 1, virtual image distortion correction is performed in response to viewpoint movement.

U.S. Pat. No. 3,723,805

Folker Wientapper, 3 others, "A Camera-Based Calibration for Automotive Augmented Reality Head-Up-Displays", IEEE International Symposium on Mixed and Augmented Reality 2013 Science and Technology Proceedings, 1-4 October 2013, Adelaide, SA, Australia

  However, the head-up display device described in Patent Document 1 does not support the movement of the observer's viewpoint position.

  Therefore, it is possible to observe a virtual image without distortion when the observer is at a predetermined position. However, when the observer's viewpoint position moves, distortion of the virtual image occurs and the distortion is corrected. There was a problem that I could not.

  In the head-up display device described in Non-Patent Document 1, a known dot pattern is displayed on the head-up display in advance, and an observation image is captured by moving the camera while moving the display image. The difference between the position of the point representing the dot pattern in this observed image and the position of the ideal display image without distortion is expressed by a cubic polynomial of the viewpoint position (camera position) and the coordinates of the camera image, The coefficient was obtained by the method of least squares.

  At that time, since it is necessary to take a large number of images (for example, 1000 or more) while moving the camera, there is a problem that work is required.

  The present invention has been made in view of such conventional problems, and an object of the present invention is to efficiently generate a projection image of a head-up display without distortion regardless of the viewpoint position with a small amount of calculation. .

  In order to solve the above problems, a head-up display distortion correction method according to the present invention is a head-up display distortion correction method for displaying a virtual image in the driver's forward field of view, wherein the position of the driver's viewpoint is An imaging unit installation process for installing the imaging unit toward the front of the vehicle at a position corresponding to the above, an observation direction adjustment process for adjusting the imaging unit so as to face a predetermined direction, and an installation position of the imaging unit are detected An image pickup unit position detection process, a reference pattern projection process for projecting a reference pattern into the driver's forward field of view to generate a projection image, and a projection image capturing for capturing the projection image by the image pickup unit. A first coordinate correspondence calculation process for calculating a first coordinate correspondence between the process, the coordinates of the points constituting the reference pattern, and the coordinates of the same point as the point in the captured image. The first coordinate correspondence calculation process at each installation position of the imaging unit and the first coordinate correspondence calculation process at each installation position of the imaging unit. Based on the first coordinate correspondence relationship, the first coordinate correspondence relationship storage process for storing the coordinate correspondence relationship, the driver viewpoint position detection process for detecting the position of the driver's viewpoint, and the first coordinate correspondence relationship, The coordinates of the points constituting the reference pattern, which are considered to be obtained when installed at the viewpoint position, and the points in the projected image of the reference pattern predicted to be imaged at the viewpoint position; Based on the second coordinate correspondence estimation step for estimating a second coordinate correspondence between the coordinates of the same point and the second coordinate correspondence estimated at the position of the viewpoint. An image information generation process of generating the image information, characterized the image information projecting process, in that it consists of generating a projection image by projecting the image information.

  According to the head-up display distortion correction method of the present invention, the above-described configuration enables projection to display a head-up display projection image with corrected distortion regardless of the position of the driver's viewpoint. Thus, it is possible to easily generate a deformation pattern distorted in the reverse direction in advance.

It is a functional block diagram which shows the function structure of the distortion correction apparatus of the head-up display using the distortion correction method of the head-up display which concerns on Example 1 which is one Embodiment of this invention. FIG. 3 is a functional block diagram showing only components necessary for calibration using the head-up display distortion correction apparatus shown in FIG. 1. FIG. 2 is a functional block diagram showing only components necessary for generating and displaying a projected image of a head-up display whose distortion has been corrected using the head-up display distortion correcting apparatus shown in FIG. 1. It is a figure explaining the content of the calibration implemented in Example 1, (a) is a figure which shows an example of a reference | standard pattern. (B) is a figure which shows an example of the projection image of a reference | standard pattern. (C) is a figure which shows an example of the camera image which imaged the projection image of the reference | standard pattern. It is a figure explaining the gray code utilized when calibrating. It is a figure explaining the corresponding map to produce, (a) is a figure showing an example of a prior deformation pattern formed in order to project a head-up display display image without distortion. (B) is a figure which shows an example of a head-up display display image without distortion. In Embodiment 1, it is a figure explaining the method of the alignment performed when moving an imaging part. It is a 1st figure explaining the method of carrying out the interpolation production | generation of the corresponding map in arbitrary viewpoint positions, and is a figure which shows an example of the several installation position of the moved imaging part. It is a figure explaining the 1st method of carrying out the interpolation production | generation of the corresponding | compatible map in arbitrary viewpoint positions, and is a figure explaining the method of expressing a corresponding | compatible map with a primary expression of a viewpoint position. It is a figure explaining the 2nd method of carrying out the interpolation production | generation of the corresponding map in arbitrary viewpoint positions, and is a figure explaining the method of expressing a correspondence map by the piecewise linear function of a viewpoint position. 6 is a first flowchart showing a flow of a series of calibration processes in the first embodiment. 6 is a second flowchart showing a flow of a series of calibration processes in the first embodiment. 6 is a flowchart illustrating a series of flows for projecting a distortion-free HUD display image in the first embodiment.

  Hereinafter, a specific embodiment of a head-up display distortion correction method of the present invention will be described with reference to the drawings.

  FIG. 1 is a functional block diagram showing a configuration of a head-up display distortion correction apparatus 100 using a head-up display distortion correction method according to an embodiment of the present invention. FIG. 2 is a functional block diagram showing only components necessary for calibration in the head-up display distortion correction apparatus 100 shown in FIG. FIG. 3 is necessary for displaying a head-up display projection image (hereinafter referred to as a HUD display image) in which distortion is actually corrected in the head-up display distortion correction apparatus 100 shown in FIG. It is a functional block diagram which shows only a structure part.

  The overall configuration of the head-up display distortion correction apparatus 100 will be described below with reference to FIGS. 1 to 3.

(Description of overall configuration of head-up display distortion correction device)
As shown in FIG. 1, a head-up display distortion correction apparatus 100 is mounted on a vehicle (not shown) in FIG. 1 and is projected onto a windshield 16 of the vehicle to generate a reference image / image information. Unit 10, vehicle information acquisition unit 12 that acquires vehicle information necessary for generating image information such as vehicle speed and guidance route information, and the image information generated by reference pattern / image information generation unit 10 as windshield 16 The information projection unit 14 that projects the image of the reference pattern to generate the projection image Ip or the HUD display image Ii of the reference pattern, the imaging unit 40 that observes the projection image Ip of the reference pattern, and the viewpoint of the imaging unit 40 or the driver 50 52, the installation position of the imaging unit 40 or the driver from the images observed by the imaging unit / viewpoint observation camera 60 and the imaging unit / viewpoint observation camera 60 An imaging unit installation position / viewpoint position detection unit 64 that detects the position of the zero viewpoint 52, a reference pattern generated by the reference pattern / image information generation unit 10, and a projected image Ip of the reference pattern captured by the imaging unit 40 A first coordinate correspondence calculation unit 70 for calculating a first coordinate correspondence that is a correspondence between coordinates representing the same point, and a correspondence map generated based on the calculated first coordinate correspondence The reference map projection estimated to be obtained at the position of the viewpoint 52 of the driver 50 based on the correspondence map storage unit 80 (first coordinate correspondence storage unit) that stores A second coordinate correspondence relationship, which is a correspondence relationship between coordinates representing the same point of the image Ip and a reference pattern for projecting the projection image Ip, is estimated, and the estimated second coordinate correspondence relationship is obtained. Based, vision A coordinate conversion table for generating a coordinate conversion table for generating a pre-deformed pattern Id (FIG. 4C (a)) deformed in advance so that a distortion-free HUD display image Ii (projected image) is observed at the position 52. A generation unit 90 (second coordinate correspondence estimation unit).

  When the head-up display distortion correction apparatus 100 is used, it is necessary to perform so-called calibration to collect information necessary for correcting distortion of the projected image in advance. Then, after calibration is completed, the position of the viewpoint 52 of the driver 50 is actually measured, and a distortion-free HUD display image Ii corresponding to the viewpoint position is generated and displayed. At this time, each component shown in FIG. 1 functions in accordance with each scene of “calibration” and “generation / display of HUD display image Ii”.

  FIG. 2 shows a functional configuration when calibration is performed using the distortion correction apparatus 100 of the head-up display.

  At the time of calibration, as shown in FIG. 2, only a part of the components shown in FIG. 1 exhibits a function necessary for calibration. That is, the reference pattern / image information generation unit 10 shown in FIG. 1 functions as a reference pattern generation unit 10a that generates a calibration reference pattern Ir (FIG. 4A). The reference pattern Ir projected from the information projection unit 14 is regularly reflected by the windshield 16 to generate a projected image Ip of the reference pattern.

  The imaging unit / viewpoint observation camera 60 illustrated in FIG. 1 functions as an imaging unit observation camera 60a that observes the imaging unit 40 installed to observe the projected image Ip of the reference pattern. The imaging unit installation position / viewpoint position detection unit 64 illustrated in FIG. 1 functions as an imaging unit installation position detection unit 64 a that detects the installation position of the imaging unit 40. In addition, the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit) generates and stores a coordinate conversion table for generating the pre-deformed pattern Id (FIG. 4C (a)).

  Note that the first coordinate correspondence calculation unit 70 and the correspondence map storage unit 80 (first coordinate correspondence storage unit) shown in FIG. 1 are used as they are.

  FIG. 3 shows a functional configuration when the HUD display image Ii is actually displayed using the head-up display distortion correction apparatus 100, that is, in actual operation.

  At the time of actual operation, as shown in FIG. 3, only a necessary part among the constituent parts shown in FIG. 1 exhibits a function necessary for generating and displaying the HUD display image Ii.

  That is, the imaging unit / viewpoint observation camera 60 illustrated in FIG. 1 functions as a viewpoint observation camera 60b that observes the viewpoint 52 of the driver 50. The imaging unit / viewpoint observation camera 60 is installed on a vehicle (not shown in FIG. 1), for example, on an instrument panel in a layout that allows the entire eye range, which is the range of viewpoints, to be observed. The imaging unit installation position / viewpoint position detection unit 64 illustrated in FIG. 1 functions as a viewpoint position detection unit 64 b that detects the position of the viewpoint 52.

  The coordinate conversion table generation unit 90 illustrated in FIG. 1 reads a coordinate conversion table corresponding to the position of the viewpoint 52 detected by the viewpoint position detection unit 64b, and stores the read coordinate conversion table in the image information generation unit 10b. provide. The image information generation unit 10b refers to the vehicle information acquired from the vehicle information acquisition unit 12, and uses the coordinate conversion table read from the coordinate conversion table generation unit 90 to obtain image information including vehicle information such as vehicle speed and guidance route. Generate. Then, the generated image information is projected from the information projection unit 14 and regularly reflected by the windshield 16 to generate the HUD display image Ii.

(Explanation of calibration)
An outline of calibration will be described with reference to FIGS. 4A to 4C. The correspondence between the point Q (p, q) on the projected image Ip of the reference pattern and the point R (u, v) on the reference pattern Ir shown in FIG.
[Formula 1]

  Mcp_x described in Expression 1 represents a horizontal coordinate correspondence map, and Mcp_x (x, y) represents a horizontal coordinate u of the reference pattern Ir corresponding to the point P (x, y) of the camera image Ic. . Mcp, y represents a vertical coordinate correspondence map, and Mcp_y (x, y) represents the vertical coordinate v of the reference pattern Ir corresponding to the point P (x, y) of the camera image Ic.

  4A (a) to 4A (c) are diagrams illustrating an example of the relationship among the reference pattern Ir, the projected image Ip of the reference pattern, and the camera image Ic. That is, in FIGS. 4A (a) to 4A (c), an arbitrary point R (u, v) on the reference pattern Ir is projected onto a point Q (p, q) on the projected image Ip of the reference pattern Ir. Thus, this point Q (p, q) is observed at the position of the point P (x, y) on the camera image Ic.

  4A (a) to 4A (c), the position of the point P (x, y) on the camera image Ic corresponds to the position of the point R (u, v) on the reference pattern Ir. That is, representing the same point is called a first coordinate correspondence in the present invention.

(Description of how to identify corresponding points)
Next, a method for specifying the point Q (p, q) on the projection image Ip of the reference pattern Ir corresponding to (matching with) the point P (x, y) on the camera image Ic will be described with reference to FIG. 4B. explain.

  FIG. 4B is a diagram illustrating an example of a reference pattern (Ir1, Ir2, Ir3,...). In the present embodiment, a plurality of different reference patterns (Ir1, Ir2, Ir3,...) Are sequentially projected in order to obtain the correspondence between the points on the projected image Ip of the reference pattern Ir and the points on the camera image Ic.

  The reference pattern (Ir1, Ir2, Ir3,...) Shown in FIG. 4B represents a coded data string called a Gray code in a two-dimensional pattern. The hatched area in FIG. 4B is formed with “black band”, and the non-hatched area is formed with “white band”. By sequentially projecting a plurality of patterns (Ir1, Ir2, Ir3,...) Shown in FIG. 4B and observing in what order the white and black bands are projected at each point of the projected image, a projected image is obtained. The coordinates of each point can be coded.

  For example, focus on point A shown in FIG. 4B. When the reference pattern Ir1 is projected, a black belt is projected at the point A. The point on which the black belt is projected in this way is represented by “0”. Similarly, when the reference pattern Ir2 is projected, a black belt is projected at the point A. That is, the point A is represented by “0”. When the reference pattern Ir3 is projected, a white band is projected at the point A. The point where the white belt is projected in this way is represented by “1”. Therefore, the coordinates of the point A shown in FIG. 4B are obtained by assigning the information of the previously projected reference pattern to the lower bits when the three reference patterns (Ir1, Ir2, Ir3) are sequentially projected, and the later projected reference pattern. By assigning the pattern information to the upper bits, it is possible to represent the information with a bit length of “100” of 3, that is, 3 bits.

  By preparing and sequentially projecting a plurality of reference patterns (Ir1, Ir2, Ir3,...) Shown in FIG. 4B to finer patterns, the coordinates of the area on which the reference pattern is projected can be set to a longer bit length, that is, It can be specified with higher resolution. For example, when n different reference patterns are sequentially projected, the coordinates of the point on which the reference pattern is projected can be represented by n bits.

  Next, a specific method for identifying corresponding points will be described. First, the projected image Ip of the projected reference pattern (Ir1, Ir2, Ir3,...) Is imaged by the imaging unit 40 (FIG. 1), and then a black belt is present in any region in the observed camera image Ic. It is recognized and it is recognized in which area the white belt is reflected.

  In this embodiment, in order to easily and reliably recognize black and white bands, in this embodiment, after projecting the reference pattern Ir1 shown in FIG. 4B, a pattern obtained by inverting the black and white of the reference pattern Ir1 (negative-positive inversion pattern) Project. That is, after projecting the reference pattern Ir1 and capturing the projection image Ip with the imaging unit 40, the negative / positive inversion pattern of the reference pattern Ir1 is projected and the imaging unit 40 captures the projection image Ip again.

  When the difference calculation is performed on the two images with negative and positive images captured in this manner, the difference value of the region on which the white band of the reference pattern Ir1 is projected becomes a positive value. On the other hand, the difference value of the area where the black belt of the reference pattern Ir1 is projected is a negative value. Therefore, it is possible to easily and reliably recognize the area where the white belt and the black belt are projected without being affected by the change in external brightness.

  By repeating the same process for each projected image of a plurality of different reference patterns, the reference patterns (Ir1, Ir2, Ir3, which have the same coded coordinates as the point (x, y) of the camera image Ic). The coordinates of the point (p, q) on which (...) is projected can be mechanically calculated without performing a complicated association calculation.

  For the sake of simplicity, FIG. 4B shows only the pattern in which the band extends in the vertical direction (vertical stripe), but actually projects the pattern in which the band extends in the horizontal direction (horizontal stripe). By specifying the coordinates in the same manner as described above, the two-dimensional coordinates of the projected image of the reference pattern can be specified with high resolution.

  In this way, the correspondence relationship between the coordinates (x, y) of the camera image Ic and the coordinates (p, q) of the projection image can be clarified. The coordinates (p, q) of the projection image and the coordinates (u, v) of the reference pattern Ir that is the source of the projection image Ip projected from the information projection unit 14 (FIG. 1) have a one-to-one correspondence. Therefore, the correspondence between the coordinates (x, y) of the camera image Ic and the coordinates (u, v) of the reference pattern Ir can also be clarified.

(Explanation of how to generate pre-deformed patterns)
Next, a method for generating image information to be projected from the information projection unit 14 (FIG. 1) in order to obtain a distortion-free head-up display (HUD) display image Ii using the result of calibration will be described with reference to FIG. ), And will be described with reference to FIG.

  This image information is generated in the reference pattern / image information generation unit 10 (FIG. 1). At this time, the correspondence maps Mcp_x and Mcp_y obtained by calibration and represented by Expression 1 are used as the coordinate conversion table.

  That is, since the correspondence relationship between the coordinates described in the correspondence maps Mcp_x and Mcp_y includes information on the distortion of the projected image generated when the reference pattern Ir is projected, coordinate conversion is performed using these correspondence maps Mcp_x and Mcp_y. By performing the above, image information having a distortion in the reverse direction (distorted in the reverse direction) can be generated in advance. The image information having the reverse distortion is hereinafter referred to as a pre-deformed pattern Id. Then, by projecting the pre-deformed pattern Id from the information projection unit 14 (FIG. 1), a distortion-free HUD display image Ii can be obtained.

That is, the relationship between an arbitrary point (x, y) of the HUD display image Ii and the point (u, v) in the pre-deformed pattern Id representing the same point as the point (x, y) is It becomes like this.
[Formula 2]

  Here, since the size of the projected image of the head-up display actually projected and the size of the projected image Ip of the reference pattern Ir projected at the time of calibration are generally different, the correspondence maps Mcp_x and Mcp_y obtained by the calibration are different. Are used after size adjustment (resizing). Note that when the correspondence maps Mcp_x and Mcp_y are resized, there is a possibility that discontinuity occurs in the content of the correspondence map between adjacent pixels of the correspondence map. For this reason, it is desirable to apply a smoothing process (smoothing process) to the resized correspondence map. Specifically, smoothing processing can be performed on the resized correspondence map by replacing the value of the target pixel with a weighted average value of the values of the pixel and neighboring pixels.

  In Expression 2, Mip_x is a correspondence map obtained by resizing and smoothing the correspondence map Mcp_x, and Mip_y is a correspondence map obtained by resizing and smoothing the correspondence map Mcp_y. These correspondence maps Mip_x and Mip_y are stored in the correspondence map storage unit 80 (FIG. 1) as a coordinate conversion table. Hereinafter, the correspondence maps Mip_x and Mip_y are referred to as coordinate conversion tables. FIG. 4C (a) is an example of a pre-deformed pattern Id generated using these coordinate conversion tables Mip_x and Mip_y.

  FIGS. 4C (a) and 4 (b) are explained by taking as an example the case of forming a pre-deformed pattern Id for displaying the vehicle speed (40 km / h) as the HUD display image Ii. Therefore, it is necessary to generate various information such as a guidance route and a warning in real time as the HUD display image Ii, not only the vehicle speed other than 40 km / h but also the vehicle speed. At this time, the correspondence maps Mip_x and Mip_y, which are coordinate conversion tables, can be used in common regardless of the type of information.

(Explanation of how to generate the correspondence map according to the viewpoint position)
Since the physique and seating position of the driver 50 (FIG. 1) are different depending on the individual, the HUD display image Ii shown in FIG. 1 is observed from the viewpoints 52 at various positions. When the position of the viewpoint 52 is different, the distortion generated in the projected image Ip of the reference pattern is also different. Therefore, each time the position of the viewpoint 52 moves, the correspondence map Mcp_x and Mcp_y at the position of the moved viewpoint 52 is used to generate the pre-deformed pattern Id. Need to be generated. For this purpose, it is necessary to install the imaging units 40 (FIG. 1) at a plurality of different positions and create a plurality of correspondence maps Mcp_x and Mcp_y respectively corresponding to the respective installation positions.

  However, since the correspondence maps Mcp_x and Mcp_y at the assumed viewpoint positions cannot be created in advance, in this embodiment, the imaging units 40 are installed at a plurality of necessary minimum installation positions. Corresponding maps Mcp_x and Mcp_y are generated, and the corresponding maps Mcp_x and Mcp_y at an arbitrary position of the viewpoint 52 are generated based on the generated corresponding maps Mcp_x and Mcp_y.

  At that time, the corresponding maps Mcp_x and Mcp_y obtained at the respective installation positions of the imaging unit 40 are resized and smoothed as described above, and the coordinate conversion tables Mip_x and Mip_y at the respective installation positions of the imaging unit 40 are obtained. Create Then, based on the coordinate conversion tables Mip_x and Mip_y at each installation position of the imaging unit 40, a coefficient of a regression equation of coordinate conversion corresponding to an arbitrary position of the viewpoint 52 is obtained, and the calculated coefficient is used as a coordinate conversion table generation unit. 90 (second coordinate correspondence estimation unit). Details will be described later.

  Then, the coefficients of the regression equation corresponding to the actually measured position of the viewpoint 52 are read out, coordinate conversion is performed, and the HUD display image Ii (projection image) without distortion is observed at the position of the viewpoint 52 in advance. By generating the pre-deformed pattern Id and projecting the generated pre-deformed pattern Id, the HUD display image Ii without distortion regardless of the position of the viewpoint 52 of the driver 50 (FIG. 4C (b)) Can be displayed in real time.

(Description of how to adjust the installation position of the imaging unit)
Next, a method for adjusting the installation position of the imaging unit 40 performed during calibration will be described with reference to FIG. The installation position of the imaging unit 40 is adjusted while monitoring the camera image Ic captured by the imaging unit 40. At that time, in order to adjust the installation position, as shown in FIG. 5, a reference line Ly (first reference marker) is drawn at the vertical center position of the camera image Ic, and the reference line Lx is set at the horizontal center position. (First reference marker) is drawn. Further, as shown in FIG. 5E, a reference line Mx (second reference marker) extending in the lateral direction (extending along the X-axis) is formed on the surface of the screen S on which the projection image Ip is displayed. The reference line My (second reference marker) extending in the vertical direction (extending along the Y axis) is displayed as a HUD projection image.

  The imaging part 40 shall adjust an installation position on XY plane of FIG. After moving the imaging unit 40 to a predetermined installation position (for example, the position indicated by the imaging units 40, 40a, 40b), the captured camera images (Ic, Ica, Icb) are respectively displayed on the screen S. Both the lines Mx and My (second reference marker) and the reference lines Lx and Ly (first reference marker) drawn directly on the camera image Ic are in a state of being reflected.

  At this time, in FIG. 5, when the orientations of the imaging units 40, 40a, and 40b are rotated around an axis parallel to the Y axis, that is, in the directions of the arrows Ca, Cb, and Cc in FIG. (Ica, Icb), the display positions of the reference lines Mx, My (second reference markers) move in the directions of arrows Da, Db, Dc in FIG. When the imaging units 40, 40a, and 40b face the direction of the point R on the screen S, the reference lines Mx and My completely overlap with the reference lines Lx and Ly (first reference markers), respectively. It will be in the state.

  In this way, it is confirmed that the reference line Mx and the reference line Lx and the reference line My and the reference line Ly are both overlapped, and the adjustment of the installation positions of the imaging units 40, 40a, and 40b is completed. Judge.

  By adjusting the orientations of the imaging units 40, 40a, and 40b in this way, as shown in FIG. 5, the imaging units 40, 40a, and 40b whose installation positions have been changed always have the reference line Mx and the reference line My. It is in a state facing the direction of the point R which is the intersection. That is, the orientation of the imaging unit 40 can be uniquely specified from the installation positions of the imaging units 40, 40a, and 40b.

  Further, by adjusting the orientations of the imaging units 40, 40 a, and 40 b in this way, the HUD display image Ii displayed at the position of the point R that is the intersection of the reference line Mx and the reference line My is displayed in the plurality of imaging units 40. It is possible to reproduce the state of viewing at the center of the field of view (the state of gazing) from the installation position.

(Description of interpolation generation method of correspondence map (coordinate conversion table) at arbitrary virtual installation position of imaging unit)
Next, using a plurality of correspondence maps Mcp_x and Mcp_y obtained by changing the installation position of the imaging unit 40, the virtual imaging unit 40v is set to an arbitrary installation position (virtual installation position) corresponding to the viewpoint position of the driver. Two methods for interpolating and generating a correspondence map (coordinate conversion table) at the virtual installation position when installed will be described with reference to FIGS. 6A to 6C. For simplicity of explanation, when the imaging unit 40 is installed at the installation position w, the correspondence maps at the installation position w are represented by Mcp_x (w) and Mcp_y (w), respectively.

  6A is a view of the imaging unit 40 as viewed from the direction of the arrow F in FIG. 6 after being moved to a plurality of installation positions on the XY plane shown in FIG. That is, the imaging unit 40 is moved to nine positions (1, 2, 3, 4, 5, 6, 7, 8, 9) on the XY plane. At this time, the imaging unit 40 is installed so that the vertical interval and the horizontal interval are as equal as possible.

  Circles indicating the imaging units 40, 40a, 40b, 40c, 40d, 40e, 40f, 40g, and 40h illustrated in FIG. 6A indicate lens surfaces provided in the imaging unit, respectively.

  FIG. 6B illustrates the imaging units 40 and 40a to 40h installed at nine positions (1 to 9) (FIG. 6A) on the XY plane, respectively, by the imaging unit observation camera 60a (FIG. 2). The state where the positions of the captured image capturing units 40 and 40a to 40h are superimposed on one image J (x, y) is shown.

  As shown in FIG. 6B, the imaging units 40, 40a to 40h are arranged at positions (1 ′, 2 ′, 3 ′, 4 ′, 5 ′, 6 ′, 7 ′, 8 'and 9').

  Here, in the image J (x, y), positions (1 ′ to 9 ′) at which the imaging unit is observed are respectively (xc (1), yc (1)), (xc (2), yc ( 2)), ..., (xc (9), yc (9)).

  The positions (xc (1), yc (1)), (xc (2), yc (2)), ..., (xc (9), yc (9)) where the imaging units 40, 40a to 40h are observed are From the image J (x, y), it can be obtained by detecting the center positions of the lenses of the imaging units 40, 40a to 40h (FIG. 6A). Since the lens is observed as a circular region, the center position of the lens can be obtained using, for example, a generalized Hough transform.

  Generalized Hough transform is an image recognition technique often used when detecting a specific shape from an image. When detecting a circular region, for example, an edge composing point is detected from the image J (x, y), a normal direction of the edge at the detected edge composing point is obtained, and a normal line is obtained from the edge composing point. A point located along the direction by a radius of the circle to be detected is considered as the center position of the circle, and the pixel value of the corresponding pixel in the prepared Hough space is incremented. When the same processing is performed on all edge constituent points, a point having a protruding pixel value can be detected as being the center position of the circle. In the case of the present embodiment, since the diameter of the lens mounted on the imaging unit 40 to be used is known in advance, the diameter of the lens observed by the imaging unit observation camera 60a (FIG. 2) can be predicted. That is, the radius of the circle to be detected can be predicted in advance.

  Next, a first method for interpolating a coordinate conversion table at an arbitrary viewpoint position will be described.

In FIG. 6B, when the virtual imaging unit 40v is at the viewpoint position H (xe, ye), the coordinate conversion tables Mip_x (xe, ye) and Mip_y (xe, ye) at the viewpoint position H (xe, ye) are respectively 6B, it can be approximated as a result of linear interpolation of the coordinate conversion table (resizing and smoothing each of the correspondence maps Mcp_x and Mcp_y) at the positions (1 ′ to 9 ′) of the respective imaging units in FIG. 6B. . That is, it can be obtained by the linear regression equation expressed by Equation 3 and Equation 4.
[Formula 3]
[Formula 4]

Ax (x, y), bx (x, y), cx (x, y) in Equation 3 and ay (x, y), by (x, y), cy (x, y) in Equation 4 are Each is a regression coefficient, and can be obtained by Equations 5 and 6 below.
[Formula 5]
[Formula 6]

  Equations 5 and 6 respectively correspond to plane approximation of the corresponding map at a plurality of installation positions of the imaging unit 40 for each pixel, that is, linear interpolation of the corresponding map for each pixel. These regression coefficients (ax (x, y), bx (x, y), cx (x, y), ay (x, y), by (x, y), cy (x, y)) Based on this, the coordinate conversion tables Mip_x (xe, ye) and Mip_y (xe, ye) at an arbitrary viewpoint position H (xe, ye) can be obtained. At this time, the calculated regression coefficients (ax (x, y), bx (x, y), cx (x, y), ay (x, y), by (x, y), cy (x, y) ) Is stored in the coordinate conversion table generation unit 90 (second coordinate correspondence relationship estimation unit) and is read as necessary to perform coordinate conversion.

  Next, a second method for interpolating and generating the coordinate conversion tables Mip_x and Mip_y at an arbitrary viewpoint position will be described.

  FIG. 6C shows a second interpolation generation method for interpolating the coordinate conversion tables Mip_x (xe, ye) and Mip_y (xe, ye) at an arbitrary viewpoint position H (xe, ye) where the virtual imaging unit 40v is installed. It is a figure explaining.

  In FIG. 6C, the coordinate conversion table at an arbitrary viewpoint position H (xe, ye) is the viewpoint position H (xe, ye) without using the coordinate conversion tables at all positions (1 ′ to 9 ′) of the imaging unit. It is possible to generate an interpolation using only the coordinate conversion table at the position of the imaging unit in the vicinity of.

  That is, the correspondence map at an arbitrary viewpoint position H (xe, ye) in the eye range is a minimum value and a maximum value of the horizontal coordinates of the positions (1 ′ to 9 ′) of the imaging units installed in the eye range. The rectangular region Rr having the minimum and maximum values of the vertical coordinate as vertices is divided into a predetermined number of divisions in the horizontal direction and the vertical direction, respectively, so that a plurality of rectangular partial regions (Pr1, Pr2, Pr3) are obtained. , Pr4), and interpolation is performed according to which partial region the viewpoint position H (xe, ye) belongs to among the partial regions (Pr1, Pr2, Pr3, Pr4) thus formed. Can be generated. For example, the positions (1 ′, 2 ′,. Interpolation can be performed using the coordinate conversion table in 4 ′, 5 ′). Thus, the correspondence map at an arbitrary viewpoint position H (xe, ye) can be represented by a piecewise linear function of the viewpoint position H (xe, ye). At this time, the regression coefficients (ax (x, y), bx (x, y), cx (x, y), ay (x, y), aforesaid for each partial region (Pr1, Pr2, Pr3, Pr4). by (x, y), cy (x, y)) are calculated and stored in the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit), respectively. Then, a regression coefficient corresponding to the detected viewpoint position H (xe, ye) of the driver 50 is read out and coordinate conversion is performed. When there are only two or less imaging units in the circular region T1, that is, when there are not a sufficient number of imaging units, there are at least three locations in order of increasing distance from the center position of the partial region. The position of the imaging unit may be selected.

  Interpolation generation of the coordinate conversion table at an arbitrary viewpoint position may be performed using either of the two methods described above, but an image representing the position of the imaging unit obtained when the installation position of the imaging unit is changed Of the coordinate conversion table obtained at each installation position of the imaging unit when the change in distortion of the imaging unit is large (in the image corresponding to FIG. 6B, when the difference in distortion of the positional relationship between adjacent imaging units is large depending on the location). If a plane is applied to each pixel, for example, when there is a large change in image distortion, the error may increase in linear interpolation. Therefore, the interpolation is generated using the second method, that is, the piecewise linear function of the viewpoint position. Is desirable.

(Description of process flow in embodiment 1)
The overall flow of processing in the first embodiment will be described with reference to FIGS. 7A, 7B, and 8. FIG. 7A and 7B are flowcharts showing a flow of a series of processing when calibration is performed. FIG. 8 is a flowchart showing a flow of a series of processes for projecting a distortion-free HUD display image Ii. First, a series of flow at the time of calibration will be described with reference to the flowcharts of FIGS. 7A and 7B and the functional block diagram of FIG.

  (Step S10) The imaging unit 40 is installed in the direction in which the projected image Ip of the reference pattern is projected.

  (Step S12) The reference lines Mx and My (second reference markers) shown in FIG. 5 are placed on the screen S placed at the position where the projected image Ip of the reference pattern is formed.

  (Step S14) The orientation of the imaging unit 40 is adjusted so that the second reference markers Mx and My are captured at predetermined positions in the camera image Ic.

  (Step S18) The position of the imaging unit 40 is detected by the imaging unit installation position detection unit 64a.

  (Step S20) The reference pattern generation unit 10a generates a gray code (positive) and projects it from the information projection unit 14 toward the windshield 16.

  (Step S <b> 22) The projected image of the gray code is captured by the imaging unit 40.

  (Step S24) The reference pattern generation unit 10a generates a gray code (negative) and projects it from the information projection unit 14 toward the windshield 16.

  (Step S <b> 26) The projected image of the gray code is captured by the imaging unit 40.

  (Step S28) The imaged gray code (positive) projection image is binarized. At that time, the projected image of the gray code (negative) taken as a threshold value for binarization.

  (Step S30) The result of the binarization process is stored in the first coordinate correspondence calculation unit 70.

  (Step S32) It is determined whether or not all gray codes (positive) have been projected. When all the gray codes have been projected, the process proceeds to step S34. Otherwise, the process returns to step S20 to project a different gray code (positive).

  (Step S34) In the first coordinate correspondence calculation unit 70, correspondence maps Mcp_x and Mcp_y are created. The created correspondence maps Mcp_x and Mcp_y are subjected to resizing and smoothing processing as necessary, and become correspondence maps (coordinate conversion tables) Mip_x and Mip_y. Then, the generated correspondence map Mip_x, Mip_y is stored in the correspondence map storage unit 80.

  (Step S38) It is determined whether the imaging unit 40 has been moved to all installation positions. When it is moved to all the installation positions, the process proceeds to step S40. Otherwise, the process returns to step S10, and the imaging unit 40 is moved to another installation position.

  (Step S40) Based on the correspondence maps Mip_x and Mip_y obtained at the respective installation positions of the imaging unit 40, a regression equation for coordinate conversion for generating a correspondence map at an arbitrary viewpoint position is calculated.

  (Step S42) The calculated regression equation coefficients are stored in the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit), and the processing of FIGS. 7A and 7B is terminated.

  Next, a flow of a series of processes for projecting a distortion-free HUD display image Ii will be described with reference to the flowchart of FIG. 8 and the functional block diagram of FIG.

  (Step S80) The position of the viewpoint 52 is detected by the viewpoint position detector 64b.

  (Step S84) The image information generation unit 10b reads the coefficient of the regression equation of the coordinate conversion at the detected position of the viewpoint 52 from the coordinate conversion table generation unit 90, and based on the read coefficient, the HUD display image Ii Image information to be projected for generation is subjected to coordinate transformation to create a pre-deformed pattern Id.

  (Step S86) The created preliminary deformation pattern Id is projected from the information projection unit 14 to generate the HUD display image Ii.

  (Step S88) It is determined whether a condition for turning off (erasing) the HUD display image Ii is satisfied. When the condition is satisfied, the HUD display is erased and the process is terminated. Otherwise, the process returns to step S80.

  As described above, according to the head-up display distortion correction method performed in the head-up display distortion correction apparatus 100 according to the first embodiment, the imaging unit installed forward at a position corresponding to the viewpoint 52 of the driver. After adjusting 40 to face in a predetermined direction, the installation position of the imaging unit 40 is detected. A projected image Ip of the reference pattern Ir is generated in the driver's forward field of view, and the projected image Ip is captured by the imaging unit 40. Among the captured images, the coordinates of the points constituting the reference pattern Ir A first coordinate correspondence between the coordinates representing the same point as that point is calculated. The same processing is repeated a plurality of times by changing the installation position of the imaging unit 40, and then the obtained first coordinate correspondence relationship is stored. Then, the coordinates of the points constituting the reference pattern Ir that are considered to be obtained at the detected position of the viewpoint 52 of the driver, and the projected image Ip of the reference pattern Ir that is predicted to be imaged at the position of the viewpoint 52 are obtained. A coordinate transformation that deforms the reference pattern Ir so that the projected image Ip can be observed without distortion from the viewpoint position (x, y) by estimating the second coordinate correspondence between the coordinates of the corresponding points of The tables Mip_x (x, y) and Mip_y (x, y) are generated. Then, based on the coordinate conversion tables Mip_x (x, y) and Mip_y (x, y), the image information deformed in advance is generated, and this image information is projected to generate the HUD display image Ii. Mip_x (x, y) and Mip_y (x, y), which are values for each pixel in the table, are expressed by a linear expression only for the position (x, y) of the viewpoint 52, and there is no distortion regardless of the viewpoint position. The display image Ii can be efficiently generated with a small amount of calculation.

  Furthermore, since the algorithm is simple, it is easy to implement and is suitable for hardware. Further, since the shape of the windshield 16 and the optical system parameters do not need to be measured in advance, distortion caused by various factors can be corrected collectively.

  And since it is not necessary to install the image pick-up part 40 in a fixed position, it is not necessary to learn work when performing calibration, and calibration can be performed easily even if it is not an expert. Furthermore, since the detection of the imaging unit 40 used for calibration and the detection of the viewpoint 52 when the HUD display image Ii is actually displayed can be performed by the same imaging unit / viewpoint observation camera 60, the imaging unit 40 at the time of calibration can be detected. There is no need to perform coordinate conversion between the position and the actual position of the viewpoint 52. In addition, since effects such as lens distortion are offset, calibration can be easily performed with little effort.

  Further, according to the head-up display distortion correction method performed in the head-up display distortion correction apparatus 100 according to the first embodiment, the observation direction adjustment process is performed by the imaging unit 40 among the images captured by the imaging unit 40. The image of the first reference marker (Lx, Ly) that is installed and fixed at the position regardless of the movement of the imaging unit 40, and the second reference marker (Mx, Lyr) displayed at a predetermined position in the driver's front field of view. Since it is performed by matching the position of the image My) with the position of the image, estimation of the posture of the imaging unit 40 is not necessary, and the amount of calculation at the time of calibration can be reduced.

  According to the head-up display distortion correction method performed in the head-up display distortion correction apparatus 100 according to the first embodiment, the minimum and maximum values of the horizontal coordinates of the plurality of installation positions of the imaging unit 40 and the vertical direction A rectangular region Rr having the minimum and maximum coordinate values as vertices is divided into a predetermined number of divisions in the horizontal and vertical directions to form a plurality of partial regions (Pr1, Pr2, Pr3, Pr4). Among the plurality of formed partial areas (Pr1, Pr2, Pr3, Pr4), the imaging unit located within a predetermined distance d from the center position G1 of the partial area Pr1 to which the driver's viewpoint position H (xe, ye) belongs In order to estimate the second coordinate correspondence at the viewpoint position H (xe, ye) based on the first coordinate correspondence at each of the 40 installation positions, Obtained when changing the location positions, even when the change of the distortion of an image representing a plurality of installation position of the imaging unit 40 is large, it is possible to perform highly accurate distortion correction.

  Furthermore, according to the head-up display distortion correction method performed in the head-up display distortion correction apparatus 100 according to the first embodiment, the second coordinate correspondence is expressed by a linear expression of the position (x, y) of the viewpoint 52. Therefore, the amount of calculation when performing calibration can be reduced. Furthermore, since the association for each pixel is performed when obtaining the second coordinate correspondence, a smooth projected image without distortion can be obtained.

  In addition, according to the head-up display distortion correction apparatus 100 using the head-up display distortion correction method according to the first embodiment, an image is installed at a position corresponding to the viewpoint 52 of the driver so as to face a predetermined position. The installation position of the unit 40 is detected by the imaging unit installation position detection unit 64a, and the information projection unit 14 projects the projection image Ip of the reference pattern Ir into the driver's front field of view. The first coordinate correspondence calculating unit 70 captures the image Ip, and the first coordinate correspondence calculation unit 70 includes a first coordinate between the coordinates of the points that form the reference pattern Ir and the coordinates that represent the same point in the captured image. After calculating the coordinate correspondence, the installation position of the imaging unit 40 is changed, the first coordinate correspondence is calculated at each installation position, and the correspondence map storage unit 80 (first coordinate correspondence storage unit) To remember.

  Then, the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit) performs coordinate conversion at an arbitrary position of the viewpoint 52 based on the first coordinate correspondences at the plurality of installation positions of the imaging unit 40. Interpolate and store regression coefficient coefficients.

  Next, the image information generation unit 10b determines the position (x, y) of the viewpoint 52 from the coefficients of the regression equation for coordinate conversion stored in the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit). In order to generate image information (preliminarily deformed pattern Id) that has been deformed in advance with reference to the coefficient of the regression equation according to the above, and to generate the HUD display image Ii by projecting this prior deformed pattern Id, The values Mip_x (x, y) and Mip_y (x, y) for each pixel are expressed by a linear expression only for the position (x, y) of the viewpoint 52, and a distortion-free HUD display image Ii is obtained regardless of the viewpoint position. It can be generated efficiently with a small amount of calculation.

  Then, according to the head-up display distortion correction apparatus 100 using the head-up display distortion correction method according to the first embodiment, the first coordinate correspondence calculation unit 70 includes the image captured by the imaging unit 40. An image of the first reference marker (Lx, Ly) that is installed in the imaging unit 40 and whose position is fixed regardless of the movement of the imaging unit 40 and a second position displayed at a predetermined position in the driver's front field of view. Since the first coordinate correspondence is calculated in a state where the position of the image of the reference marker (Mx, My) coincides with that of the reference marker (Mx, My), it is not necessary to install a texture pattern, a marker, or the like, and the labor for calibration is reduced. be able to.

  Furthermore, according to the head-up display distortion correction apparatus 100 using the head-up display distortion correction method according to the first embodiment, the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit) includes the imaging unit 40. A rectangular area Rr having the minimum and maximum horizontal coordinate values and the minimum and maximum vertical coordinate values at a plurality of installation positions (1 ′ to 9 ′) as vertices in advance in the horizontal and vertical directions in advance. A plurality of partial regions (Pr1, Pr2, Pr3, Pr4) are formed by dividing into a predetermined number of divisions, and the driver's viewpoint among the formed partial regions (Pr1, Pr2, Pr3, Pr4) Based on the first coordinate correspondence at each installation position of the imaging unit 40 within the predetermined distance d from the center position G1 of the partial region Pr1 to which the position H (xe, ye) belongs, the viewpoint position H (x , Yes) when the change in the distortion of the image representing the plurality of installation positions of the imaging unit 40 obtained when the installation position of the imaging unit 40 is changed in order to estimate the second coordinate correspondence relationship. However, highly accurate distortion correction can be performed.

  In addition, according to the head-up display distortion correction apparatus 100 using the head-up display distortion correction method according to the first embodiment, the coordinate conversion table generation unit 90 (second coordinate correspondence estimation unit) includes the second coordinates. Since the correspondence relationship is expressed by a linear expression only for the position (x, y) of the viewpoint 52, the HUD display image can be efficiently generated with a small amount of calculation. Furthermore, since the association for each pixel is performed when obtaining the second coordinate correspondence, a smooth projected image without distortion can be obtained.

  In the first embodiment, the direction of the imaging unit 40 is adjusted so that the first reference markers Lx, Ly and the second reference markers Mx, My, which are formed in a cross shape, coincide with each other. The shapes of Lx and Ly and the second reference markers Mx and My are not limited to crosshairs. That is, another shape may be used as long as the imaging unit 40 can be aligned in a predetermined direction.

  In the first embodiment described above, when the coordinate conversion table at an arbitrary viewpoint position is predicted, the imaging unit 40 is moved to nine locations to form four partial areas. The number of moving locations and the number of partial areas to be generated are not limited to these values. That is, calibration may be performed by moving the imaging unit 40 to more positions, and the number of partial areas may be increased. By increasing the number of partial areas in this way, a smoother approximation is possible. Further, by using the installation position (calibration point) of the imaging unit 40 used in a certain partial area also in another partial area, it is possible to increase the calibration points and perform interpolation more resistant to errors.

  Although the embodiments of the present invention have been described in detail with reference to the drawings, the embodiments are only examples of the present invention, and the present invention is not limited to the configurations of the embodiments. Needless to say, design changes and the like within a range not departing from the gist of the invention are included in the present invention.

10b... Image information generation unit 40... Imaging unit 52... View point 64. First coordinate correspondence calculation unit 80... Correspondence map storage unit (first coordinate correspondence storage unit)
90... Coordinate conversion table generation unit (second coordinate correspondence estimation unit)
DESCRIPTION OF SYMBOLS 100 ... Head-up display distortion correction apparatus Id ... Prior deformation pattern Lx, Ly ... 1st reference | standard marker (reference line)
Mx, My ... second reference marker (reference line)
Ir ··· Reference pattern Ip ··· Projection image of reference pattern Ii ··· HUD display image Ic ··· Observation image (camera image)
Mip_x, Mip_y ... correspondence map (coordinate conversion table)
w ... installation position

Claims (8)

A head-up display distortion correction method for displaying a virtual image in a driver's forward view,
An imaging unit installation process in which the imaging unit is installed toward the front of the vehicle at a position corresponding to the position of the driver's viewpoint;
An observation direction adjustment process for adjusting the imaging unit to face a predetermined direction;
An imaging unit position detection process for detecting an installation position of the imaging unit;
A reference pattern projection process for projecting a reference pattern into the driver's forward field of view to generate a projected image;
A projected image imaging process of capturing the projected image by the imaging unit to obtain a captured image;
A first coordinate correspondence calculation process for calculating a first coordinate correspondence between the coordinates of the points constituting the reference pattern and the coordinates of the same point as the point in the captured image;
An imaging unit position changing process for changing an installation position of the imaging unit;
A first coordinate correspondence storage process for storing each first coordinate correspondence obtained by repeatedly performing the first coordinate correspondence calculation process at each installation position of the imaging unit;
A driver viewpoint position detection process for detecting the position of the driver's viewpoint;
Based on each first coordinate correspondence, an image is captured at the coordinates of the points constituting the reference pattern and the position of the viewpoint, which is considered to be obtained when the imaging unit is installed at the position of the viewpoint. A second coordinate correspondence estimation process for estimating a second coordinate correspondence between the coordinates of the same point as the point in the projected image of the reference pattern predicted
A second coordinate correspondence storage process for storing the second coordinate correspondence;
An image information generation process for generating image information deformed in advance so that a projection image without distortion is projected based on the second coordinate correspondence estimated at the position of the viewpoint;
An image information projection process for projecting the image information to generate a projected image, and a distortion correction method for a head-up display.   The observation direction adjustment process includes an image of a first reference marker that is installed in the imaging unit and whose position is fixed regardless of the movement of the imaging unit among the images captured by the imaging unit, and driving The head-up display distortion correction method according to claim 1, wherein the correction is performed by matching the position of the second reference marker image displayed at a predetermined position in the person's front field of view.   The second coordinate correspondence relationship estimation process includes a minimum value and a maximum value of horizontal coordinates and a minimum value and a maximum value of vertical coordinates of a plurality of installation positions of the imaging unit installed in the imaging unit position change process. Are divided into a predetermined number of divisions respectively in the horizontal direction and the vertical direction to form a plurality of partial areas, and the viewpoint of the driver among the plurality of formed partial areas The second coordinate correspondence at the position of the viewpoint is estimated based on the first coordinate correspondence at each installation position of the imaging unit within a predetermined distance from the center position of the partial area to which the position belongs. The head-up display distortion correction method according to claim 1, wherein the head-up display distortion correction method is provided.   4. The head-up display distortion correction method according to claim 1, wherein the second coordinate correspondence relationship is expressed by a linear expression of the position of the viewpoint. 5. A head-up display distortion correction device that displays a virtual image in the driver's forward field of view,
An information projection unit that projects image information in front of the driver to generate a projection image;
An imaging unit installed at a position corresponding to the viewpoint of the driver so as to face a predetermined position of the image information projected by the information projection unit;
The projected image of the reference pattern projected as the image information is captured by the imaging unit, the captured observation image and the projected image are compared, and the first coordinate correspondence between the coordinates representing the same point is obtained. A first coordinate correspondence calculating unit that moves the imaging unit to a plurality of different installation positions and calculates each of the imaging units at each installation position;
A first coordinate correspondence storage unit for storing each of the first coordinate correspondences;
An imaging unit installation position / viewpoint position detection unit for detecting an installation position of the imaging unit or a driver's viewpoint; and
Based on each first coordinate correspondence, an image is captured at the coordinates of the points constituting the reference pattern and the position of the viewpoint, which is considered to be obtained when the imaging unit is installed at the position of the viewpoint. A second coordinate correspondence between the projected coordinates of the reference pattern predicted to be the same as the coordinates of the point and the coordinates of a point representing the same point is estimated, and a projected image without distortion at the position of the viewpoint is obtained. A coordinate conversion table generating unit that generates a coordinate conversion table for generating a pre-deformed pattern that has been deformed in advance so as to be observed;
An image information generation unit that generates pre-deformed image information based on the coordinate conversion table, and projects the image information from the information projection unit to generate a projection image. Head-up display distortion correction device.   The first coordinate correspondence calculation unit is installed in the imaging unit among images captured by the imaging unit, and a first reference marker whose position is fixed regardless of movement of the imaging unit. The first coordinate correspondence relationship is calculated in a state in which the position of the image and the image of the second reference marker displayed at a predetermined position in the driver's forward field of view coincide with each other. The distortion correction apparatus of the head-up display as described.   The second coordinate correspondence relationship estimation unit horizontally converts a rectangle having a minimum value and a maximum value of horizontal coordinates and a minimum value and a maximum value of vertical coordinates of the plurality of installation positions of the imaging unit, respectively. The imaging unit within a predetermined distance from the center position of the partial region to which the position of the viewpoint of the driver belongs among a plurality of partial regions formed by dividing the predetermined number of directions in the direction and the vertical direction. 7. The head-up according to claim 5, wherein the second coordinate correspondence relationship at the viewpoint position is estimated based on the first coordinate correspondence relationship at each installation position. Display distortion correction device.   The head according to any one of claims 5 to 7, wherein the second coordinate correspondence relationship estimation unit represents the second coordinate correspondence relationship by a linear expression of the position of the viewpoint. Up display distortion correction device.