JP6255797B2 - Image processing device - Google Patents

Description

The present invention relates to a display system or the like for vehicle using a head-up display, relates to a suitable image processing equipment to the display system that provides the user an image projected on the projection plane from the display device.

  As is well known, in a head-up display (hereinafter abbreviated as HUD) display system, for example, an image projected on a windshield of a car from a display screen of a display device on the dashboard of the car is shown to the driver. Here, the surface of the windshield is a curved surface. Therefore, in the HUD system, the image visually recognized by the driver is a distorted image obtained by inverting the image displayed on the display screen of the display device. As described above, in the HUD system, an image visually recognized by the driver is an image obtained by performing a kind of conversion process on the image displayed on the display device. Therefore, in the conventional technique, an image processing apparatus connected to the display device performs distortion correction processing corresponding to the inverse conversion processing of the conversion processing on the image data and displays the image data on the display device (see, for example, Patent Document 1). ).

JP 2008-102518 A

  By the way, paying attention to each pixel of the display screen (hereinafter referred to as screen A) of the display device described above, the light emitted from one pixel reaches the driver's vision through reflection by the windshield and is visually recognized by the driver. It becomes a pixel at a certain position in the screen (hereinafter referred to as screen B). In the HUD system, a change occurs between the position of the same pixel in the screen A and the position in the screen B. As a typical example, the upper left pixel in the screen A occupies the upper right position in the screen B. Further, the difference between the position of the same pixel in the screen A and the position in the screen B depends on the position of the pixel in the screen A or the position in the screen B. For example, as in the previous example, the pixel at the upper left corner of the screen A occupies the position of the upper right corner on the screen B. Therefore, in this case, the position of the pixel in the screen A and the position in the screen B The difference between them is a large value. However, since the center pixel of the screen A occupies almost the center position in the screen B, the difference between the position in the screen A and the position in the screen B in this case is a small value. . Therefore, in the conventional HUD display system, the above-described distortion correction processing is executed as follows. First, for each pixel in the screen B, the position occupied in the screen A is obtained, and correction data indicating the difference between the position in the screen B of the pixel and the position in the screen A is stored in the memory. Let Then, in the above-described distortion correction processing, the position of each pixel of the image that the driver wants to visually recognize (that is, the image on screen B) is moved according to the correction data corresponding to the pixel in the memory, and screen A is displayed on the display device. The image data is generated. Since this distortion correction processing is processing in units of pixels, the amount of calculation is large. Since the conventional image processing apparatus executes the distortion correction process in the drawing process, there is a problem that the amount of calculation of the drawing process increases. In addition, when the image processing apparatus is caused to execute such drawing processing including distortion correction processing with a large amount of calculation, it is necessary to mount a processor with high calculation processing capacity in the image processing apparatus, and the image processing apparatus becomes expensive. There was a problem.

  The present invention has been made in view of such problems, and an object thereof is to provide an image processing apparatus capable of generating image data to be displayed on a display device such as a HUD display system with a small amount of calculation. .

  The present invention relates to an image processing apparatus used for a head-up display that generates an image when a light beam emitted from a pixel on a display screen of a display device hits a projection plane and a reflected light beam passes through a predetermined viewpoint. Storage means for storing correction application image data obtained by applying correction processing for correcting the distortion generated in the path from the screen through the projection plane to the viewpoint for each pixel; There is provided an image processing apparatus comprising: a reading control unit that reads correction application image data; and a drawing unit that displays the correction application image data read by the reading control unit on the display screen.

  According to this invention, the image processing apparatus can generate correction application image data to be displayed on the display screen with a small amount of calculation.

It is a block diagram which shows the structure of the HUD display system which is one Embodiment of this invention. It is a figure which shows the principle of the HUD display in the embodiment. It is a figure which shows the relationship between the image visually recognized by a driver and the image displayed on a display apparatus in the embodiment. It is a figure which shows the preparation method of the correction data in other embodiment of this invention. It is a figure which shows the correction method of the image data in other embodiment of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a HUD display system according to an embodiment of the present invention. FIG. 2 is a diagram showing the principle of HUD display in the HUD display system. As shown in FIG. 2, the display device 50 in the present embodiment is fixed to a dashboard (not shown) in the vehicle in a posture in which the display screen faces the windshield 60 in the vehicle. In the HUD display system, an image irradiated from the display screen of the display device 50 is reflected by the windshield 60 and reaches the viewpoint 70 of the driver. Here, the image displayed on the display screen of the display device 50 is subjected to a certain conversion process in the process of being transmitted from the display screen to the driver's viewpoint 70, and as a result, is distorted with respect to the original image. It reaches the driver's viewpoint as an image. Hereinafter, for convenience, this conversion processing is referred to as distortion conversion processing. The HUD display system according to the present embodiment performs a distortion correction process corresponding to the inverse conversion of the distortion conversion process on the image data to be visually recognized by the driver, and causes the display device 50 to display the image data after the distortion correction process. Thus, the image data is visually recognized by the driver in a low distortion state.

  As shown in FIG. 1, the HUD display system according to the present embodiment includes a screen layout device 10, an image correction device 20, a correction data generation device 30, an image processing device 40, and a display device 50.

  The screen layout device 10 is a device that generates image data of an image material constituting a screen to be visually recognized by a driver and arrangement coordinate data indicating a position of the image material on the screen in accordance with an operation of an operation unit (not shown). is there. FIG. 3A is a diagram illustrating an example of a screen that is visually recognized by the driver. In the illustrated example, the vehicle speed of 109 km / h is displayed on the screen. To explain this speed display as an example, the screen layout device 10 generates image data of each numerical value (image material) from 0 to 9 representing each digit of speed based on the operation of an operation unit (not shown), and The position coordinate data indicating the display position of the numerical value at the hundredth position, the display position of the numerical value at the tenth position, and the display position of the numerical value at the first position is generated. FIG. 3B illustrates the image data of the image material “9” generated by the screen layout device 10.

  The image correction apparatus 20 is an apparatus that performs distortion correction processing and filter processing on the image data and arrangement coordinate data of the image material output by the screen layout apparatus 10 and outputs the corrected image data and correction application arrangement coordinate data. . The distortion correction processing is performed by the driver based on correction data indicating a positional shift between the position of each pixel in the display screen of the display device 50 and the position of each corresponding pixel in the screen viewed by the driver. This process reduces image distortion. The filtering process is to prevent deterioration in image quality due to the difference between the pixel density of the image displayed on the display screen of the display device 50 and the pixel density of the image when the image is visually recognized by the driver. It is processing.

  The distortion correction process and the filter process executed by the image correction apparatus 20 constitute an inverse conversion process of the above-described distortion conversion process. Here, the content of the distortion conversion process performed on the image irradiated from the display screen of the display device 50 includes the positional relationship between the display device 50, the windshield 60, and the viewpoint 70 of the driver, and the windshield 60. Depends on shape etc. Further, the content of the distortion conversion process performed on each image irradiated from the display screen of the display device 50 differs depending on the position of each image on the screen. Therefore, in the present embodiment, the correction data generation device 30 is provided, and the correction data storage unit 21 is provided in the image correction device 20.

  The correction data generation device 30 receives real machine arrangement information indicating the positional relationship between the display device 50, the windshield 60, and the viewpoint 70 of the driver, the shape of the windshield 60, and the like from the user, and based on the real machine arrangement information, This is a device that generates correction data and a filter coefficient for each pixel position in the screen to be visually recognized by the driver.

  More specifically, the correction data generation device 30 executes a simulation based on the ray tracing method based on the actual device arrangement information, and the light rays emitted from each pixel of the display screen of the display device 50 are reflected by the windshield 60 and are transmitted from the driver. A route to reach the viewpoint 70 is obtained. Then, the correction data generation device 30 generates correction data based on the ray path obtained for each pixel of the display screen of the display device 50. That is, as a result of the simulation by the ray tracing method, the light beam emitted from the pixel at, for example, the position VQ ′ in FIG. 2 in the display screen of the display device 50 reaches the viewpoint 70 of the driver through reflection by the windshield, and the driver visually recognizes it. For example, in the case of a pixel at the position VQ in the screen, the vector data indicating the difference between the position VQ and the position VQ ′ is used as correction data corresponding to the position VQ in the screen to make the driver visually recognize. Further, the correction data generation device 30 generates filter coefficients for the above-described filter processing based on the result of simulation by the ray tracing method. The correction data does not necessarily have to be generated using the ray tracing method. Based on the actual machine arrangement information, the vector data for each pixel position on the display screen may be calculated without using the ray tracing method. Alternatively, the calculation result may be stored in advance and the correction data stored in accordance with the input actual machine arrangement information may be used as appropriate.

  The correction data storage unit 21 in the image correction device 20 stores the correction data and filter coefficients generated by the correction data generation device 30. The image correction device 20 refers to the correction data and filter coefficients in the correction data storage unit 21 and performs distortion correction processing and filter processing on the image data and arrangement coordinate data of the image material output by the screen layout device 10. Then, correction application image data and correction application arrangement coordinate data are generated. For example, when the screen layout apparatus 10 outputs the image data and arrangement coordinate data of the numerical value “9” shown in FIG. 3A, the correction data and the filter coefficient associated with the pixel position indicated by the arrangement coordinate data are corrected data. Read from the storage unit 21. Then, a distortion correction process using the read correction data is performed on the image data and arrangement coordinate data of numerical value “9”. As a result, correction applied arrangement coordinate data is obtained from the arrangement coordinate data. Further, the image correction apparatus 20 performs the filter process using the read filter coefficient on the image data of the numerical value “9” after the distortion correction process. As a result, correction applied image data for the numerical value “9” is obtained. Then, the image correction device 20 outputs the correction application image data and the correction application arrangement coordinate data obtained in this way.

  FIG. 3D illustrates the content of the correction applied image data having the numerical value “9” in this case. FIG. 3C shows a display example when the correction application image data having the numerical value “9” is displayed at the position indicated by the correction application arrangement coordinate data on the display screen of the display device 50. An arrow drawn between FIG. 3A and FIG. 3C represents correction data associated with the position of the numerical value “9” in the screen (see FIG. 3A) to be visually recognized by the driver. ing.

  In FIG. 1, an image processing device 40 is a device that performs image processing for displaying a still image and a moving image on the display device 50. As shown in FIG. 1, the image processing apparatus 40 includes a still image related data storage unit 41, a read control unit 42, a correction data storage unit 43, a moving image correction unit 44, a drawing unit 46, and a display buffer 47. Have In a preferred embodiment, the still image related data storage unit 41 and the correction data storage unit 43 are configured by a non-volatile memory such as an EEROM (Electronic Erasable Programmable Read Only Memory). However, the still image related data storage unit 41 and the correction data storage unit 43 can also be configured by a volatile memory such as a DRAM.

  The still image related data storage unit 41 is means for storing the correction application image data and the correction application arrangement coordinate data output from the image correction apparatus 20. The readout control unit 42 reads out the correction application image data and the correction application arrangement coordinate data of all image materials to be displayed from the still image related data storage unit 41 in synchronization with the vertical scanning cycle of the display device 50, and the drawing unit 46 is a means to supply to 46.

  The correction data storage unit 43 is means for storing the correction data generated by the correction data generation device 30 and the filter coefficient, like the correction data storage unit 21 in the image correction device 20. The moving image correction unit 44 includes a moving image related data storage unit 45. The moving image related data storage unit 45 is a means for receiving and storing image data and arrangement coordinate data of moving image material whose display position changes from moment to moment from the host computer in synchronization with the vertical scanning period of the display device 50. is there. The moving image correction unit 44 reads out correction data and filter coefficients associated with the arrangement coordinate data of the image material in the moving image related data storage unit 45 from the correction data storage unit 43 in synchronization with the vertical scanning cycle of the display device 50. Then, using the read correction data and filter coefficient, the image data and the arrangement coordinate data of the image material in the moving image related data storage unit 45 are subjected to distortion correction processing and filter processing, and correction application image data and correction application are performed. Arrangement coordinate data is generated and supplied to the drawing unit 46.

  The display buffer 47 is a buffer that stores image data to be displayed on the display device 50. The display buffer 47 may be a frame buffer that stores image data for one screen of the display device 50 or a line buffer that stores image data for one line.

The drawing unit 46 receives the correction application image data and the correction application arrangement coordinate data of the image material supplied from the read control unit 42 and the moving image correction unit 44, and the correction application image data is determined in the display buffer 47 determined by the correction application arrangement coordinate data. Execute the drawing process to write to the area
The above is the details of the configuration of the HUD display system according to the present embodiment.

  In this HUD display system, the image processing device 40 and the display device 50 are mounted in a vehicle. The screen layout device 10, the image correction device 20, and the correction data generation device 30 are devices for generating data to be written in a large number of image processing devices, and need not be provided for each vehicle.

  In a preferred embodiment, the screen layout device 10 generates image data and arrangement coordinate data of various image materials to be displayed on the display device 50 by the image processing device 40, and gives them to the image correction device 20. In addition, the correction data generation device 30 includes a positional relationship between the display device 50 in the vehicle on which the image processing device 40 and the display device 50 are mounted, the windshield 60, and the viewpoint 70 of the driver, the shape of the windshield 60, and the like. The correction data and the filter coefficient are generated based on the actual machine arrangement information indicating Further, the image correction device 20 performs distortion correction processing and filter processing using the correction data generated by the correction data generation device on the image data and the arrangement coordinate data of the image material supplied from the screen layout device 10. Correction application image data and correction application arrangement coordinate data are generated. Then, the image correction device 20 writes the correction application image data and the correction application arrangement coordinate data in the still image related data storage unit 41 of the image processing device 40, and the correction data generation device 30 stores the correction data storage unit of the image processing device 40. The correction data and the filter coefficient are written in 43. In this way, the image processing apparatus 40 in which the correction application image data and the correction application arrangement coordinate data are written in the still image related data storage unit 41 and the correction data and the filter coefficient are written in the correction data storage unit 43 is, for example, a vehicle manufacturer. And mounted on the vehicle together with the display device 50.

  In another preferred embodiment, correction application image data and correction application arrangement coordinate data for various image materials generated by the image correction device 20 are written in a storage medium such as a CD-ROM or a semiconductor memory, and the image is stored in the storage medium. The data is transferred to the still image related data storage unit 41 of the processing device 40. The correction data and the filter coefficient generated by the correction data generation device 30 are also written in some storage medium and transferred from this storage medium to the correction data storage unit 43 of the image processing device 40.

  In still another preferred embodiment, the image processing device 40 downloads correction application image data and correction application arrangement coordinate data for various image materials from the image correction device 20 via the Internet, and also receives correction data from the correction data generation device 30. And the filter coefficient are downloaded and stored in the still image related data storage unit 41 and the correction data storage unit 43.

  In the image processing apparatus 40, the following operation is performed. First, an external signal indicating information to be displayed at a fixed position in the display screen of the display device 50 such as a vehicle speed value measured by a vehicle speed sensor is supplied to the read control unit 42 of the image processing device 40. The read control unit 42 selects an image material to be displayed according to the external signal supplied in this way, and the correction application image data and the correction application arrangement coordinate data of the image material are stored in the still image related data storage unit 41. The process of reading out the image data and supplying it to the drawing unit 46 is repeated in synchronization with the vertical scanning cycle.

  In the example shown in FIG. 3C, an external signal indicating a vehicle speed value of 109 km is supplied from the vehicle speed sensor to the read control unit. Therefore, the reading control unit 42 includes correction application arrangement coordinate data indicating a numerical display area at the 100th position of the vehicle speed value in the display screen of the display device 50, and correction application image data having a numerical value “1” associated therewith. Correction applied arrangement coordinate data indicating the numerical display area of the tenth place, correction application image data of numerical value “0” associated therewith, correction applied arrangement coordinate data indicating the numerical display area of the first place, and The associated correction applied image data with the numerical value “9” is read from the still image related data storage unit 41 and supplied to the drawing unit 46.

  The drawing unit 46 executes a drawing process for writing the correction application image data supplied in this way into an area in the display buffer 47 determined by the correction application arrangement coordinate data. As a result, as shown in FIG. 3C, an image with the mirror image inverted and the distorted vehicle speed value “109” is displayed on the display screen of the display device 50. The image of the vehicle speed value “109” reaches the driver's viewpoint through reflection by the windshield. As a result, as shown in FIG. 3A, the vehicle speed value “109” subjected to distortion correction is visually recognized by the driver.

  The vehicle speed value measured by the vehicle speed sensor changes every moment. The operation in that case is the same as described above, and the read control unit 42 stops the correction applied arrangement coordinate data indicating the numerical display area of each digit of the vehicle speed value and the correction applied image data of the numeric value of each digit of the vehicle speed value. The data is read from the image-related data storage unit 41 and supplied to the drawing unit 46. Then, the drawing unit 46 executes a drawing process for writing the correction application image data supplied in this way into an area in the display buffer 47 determined by the correction application arrangement coordinate data.

  As described above, in the present embodiment, the drawing unit 46 draws a still image using the correction application image data and the correction application arrangement coordinate data of the image material read from the still image related data storage unit 41 by the read control unit 42. Process. In this still image drawing process, no distortion correction process is performed. Therefore, the drawing unit 46 can perform a still image drawing process with a small amount of calculation.

  Further, in the present embodiment, with regard to a moving image, correction data stored in the correction data storage unit 43 with respect to image data and arrangement coordinate data of the moving image material written in the moving image related data storage unit 45 of the moving image correction unit 44. A distortion correction process using the filter and a filter process using the filter coefficient are performed, and the correction application image data and the correction application arrangement coordinate data obtained as a result are given to the drawing unit 46 and used for the drawing process. Therefore, regarding the moving image, the calculation amount of the moving image correction unit 44 is generated in addition to the calculation amount of the drawing unit 46. However, when the ratio of the still image and the moving image to the whole of the former screen is large, the calculation amount of the image processing apparatus 40 as a whole can be reduced. Accordingly, it is possible to realize HUD display in which distortion correction is performed for both a still image and a moving image with a small amount of calculation.

<Other embodiments>
As mentioned above, although embodiment of this invention was described, other embodiment can be considered to this invention. For example:

(1) In the above embodiment, the correction data storage unit 43 and the moving image correction unit 44 are provided to display a moving image on the display device 50. However, when only the still image is displayed on the display device 50, the correction data storage unit 43 and the moving image correction unit 44 are unnecessary.

(2) In the above embodiment, both the correction application image data and the correction application arrangement coordinate data of the image material are stored in the still image related data storage unit 41. However, only the correction application image data may be stored. For example, when the display position of the image material is standardized and the display position rarely changes, correction applied arrangement coordinate data for the display position is incorporated as fixed data in the drawing program executed by the drawing unit 46. It is conceivable to use such a mode.

(3) In the above-described embodiment, the correction application image data and the correction application arrangement coordinate data are generated for each image material to be displayed at various places in the display screen and stored in the still image related data storage unit 41. However, instead of generating correction application image data and correction application arrangement coordinate data for each image material unit, correction application image data is generated for an image of one screen to be displayed on the display screen, and still image related data storage is performed. You may memorize | store in the part 41. FIG. In this case, the correction application arrangement coordinate data is unnecessary.

(4) In the above embodiment, the correction data is created by executing a simulation based on the ray tracing method. However, the correction data may be created as follows. As shown in FIG. 4, a laser irradiation device 51 is placed at an arbitrary pixel position of the display device 50 at a position where the display device 50 is actually disposed in the actual vehicle, and a laser is matched to the irradiation direction of the display device 50. Irradiate. In addition, a camera 70A is disposed at the eye position of the driver. The laser light from the laser irradiation device 51 is reflected by the windshield 60 and imaged by the camera 70A. At this time, the position of the laser irradiation device 51 in the display image (XY coordinate system) of the display device 50 is adjusted so that the image of the laser beam appears at a desired pixel position on the imaging screen (UV coordinate system) of the camera 70A. The position (XY coordinate value) of the laser irradiation device 51 is recorded in association with the pixel position (UV coordinate value) on the imaging screen (UV coordinate system) of the camera 70A. In this case, the conversion data from the position of the laser beam image (UV coordinate value) to the position of the laser irradiation device 51 (XY coordinate value) is the correction data. By performing this operation for all the pixels on the display device 50, correction data for all pixel positions in the UV coordinate system can be obtained. This method is a method of creating correction data according to the actual product without using the ray tracing method, and does not require various data such as the curvature of the windshield, and does not require complicated arithmetic processing.

(5) FIGS. 5A to 5C are diagrams showing examples of image data correction. FIG. 5B shows a correction table used for the image data correction process of FIG. This correction table is a correction table in which the Y coordinate value is a row number and the X coordinate value is a column number. In this correction table, the matrix element corresponding to a certain X coordinate value (column number) and Y coordinate value (row number) is an image of the position of the display screen of the display device 50 corresponding to the XY coordinate value of the camera 70A. The position (UV coordinate value) occupied in (UV coordinate system) is shown. For example, the matrix element corresponding to the XY coordinate value (X, Y) = (3, 1) is (1, 1), and the UV coordinate value is (U, V) = (1, 1). Using such a correction table, it is possible to correct image data by the following procedure.
a. As shown in FIG. 5A, image data to be visually recognized by the driver is arranged in the UV coordinate system according to the arrangement coordinate data.
b. By referring to the correction table shown in FIG. 5B, the UV coordinate value of the pixel of the UV coordinate system associated with each pixel position of the XY coordinate system of the display device 50 is acquired, and the corresponding UV coordinate value is obtained. The obtained pixel value is acquired from the UV coordinate system (FIG. 5A).
c. Each pixel value acquired from the UV coordinate system is arranged as each correction application image data at each pixel position in the original XY coordinate system. This process is performed by combining conversion from image data to correction application image data and conversion from arrangement coordinate data to correction application arrangement coordinate data.
For example, the UV coordinates (U, V) = (5, 1) in FIG. 5A correspond to the XY coordinates (X, Y) = (7, 0) in FIG. 5B. Accordingly, the UV coordinates (U, V) = (5, 1) in FIG. 5B have pixels on the XY coordinates (X, Y) = (7, 0) in FIG. 5C.
The correction applied image data (FIG. 5C) arranged in the XY coordinate system in this way reaches the driver's viewpoint through reflection by the windshield, and as image data without distortion (FIG. 5A). Visible to the driver.
This aspect has an advantage that the correction of the image data and the arrangement coordinate data can be executed by a simple arithmetic process.
When the driver is allowed to visually recognize the moving image, the above a. ~ C. This process may be repeated.

(6) In the above aspect (4), the correction data is obtained for each UV coordinate value in the UV coordinate system. However, the UV coordinate value in the UV coordinate system corresponding to each XY coordinate value in the XY coordinate system is obtained to obtain the image. It may be used for data correction.

DESCRIPTION OF SYMBOLS 10 ... Screen layout apparatus, 20 ... Image correction apparatus, 30 ... Correction data generation apparatus, 40 ... Image processing apparatus, 50 ... Display apparatus, 60 ... Windshield, 70 ... Viewpoint, 70A ... Camera 21, 43... Correction data storage unit, 41... Still image related data storage unit, 42... Readout control unit, 44 ....... moving image correction unit, 45. 47 …… Display buffer.

Claims (3)

An image processing device used for a head-up display that generates an image by causing a light beam emitted from a pixel on a display screen of a display device to hit a projection surface and reflected light passes through a predetermined viewpoint,
Storage means for storing correction application image data obtained by performing correction processing on the image data to correct distortion generated for each pixel from the display screen through the projection plane to the viewpoint;
Read control means for reading out desired correction application image data from the storage means;
An image processing apparatus comprising: drawing means for displaying the correction applied image data read by the reading control means on the display screen. The storage means further stores correction application arrangement coordinate data for designating a display position of the correction application image data in the display screen,
The read control means reads desired correction application image data and correction application arrangement coordinate data,
The image processing apparatus according to claim 1, wherein the drawing unit displays the correction application image data at a position indicated by the correction application arrangement coordinate data in a display screen of the display device. 2. The reading control means selects one correction application image data from a plurality of correction application image data that can be displayed at the same display position in accordance with an external signal, and reads out the correction application image data from the storage means. Or the image processing apparatus of 2 .

Images