CN117557480A - HUD image distortion correction module, HUD image distortion correction method and HUD image processing display device - Google Patents

Abstract

The invention provides a HUD image distortion correction module, a HUD image distortion correction method and a HUD image processing display device, wherein the HUD image distortion correction module comprises the following components: the device comprises a preprocessing input interface unit, a preprocessing read-write control unit, a memory unit, a judging unit, a bus write interface unit, a bus read interface unit, a bus distortion control unit, a multi-line cache unit and a distortion correction unit; the judging unit is used for sending the data to the multi-line cache unit through the bus distortion control unit when the scene is in time; transmitting data to a bus write interface unit in a second scene; transmitting the data to a distortion correction unit at the time of scene three; the distortion correction unit is used for realizing distortion correction processing under different scenes. And a special module is selected for distortion correction processing, so that the resources of the main control chip are not occupied. Meanwhile, the HUD image distortion correction module can be configured in different working scenes according to different application scenes. The application range is wide.

Description

HUD image distortion correction module, HUD image distortion correction method and HUD image processing display device

Technical Field

The invention relates to the technical field of digital image signal processing, in particular to a HUD image distortion correction module and method and a HUD image processing display device.

Background

Head Up Display (HUD) was first used in the fighter field based on radar targeting and optical targeting. With the development of the automobile market, HUDs are also widely used in the automobile field. The HUD projects the instrument panel information and the surrounding environment information required by the driver onto the windshield in front of the driver, so that the driver is ensured not to need to lower the head to judge the information such as the speed, the oil quantity and the like, the potential safety hazard caused by frequent head lifting and head lowering is avoided, and the driving safety risk is greatly reduced.

In the vehicle-mounted HUD system, since the windshield is a curved surface with curvature, normal images are distorted and have curvature deformation in the process of displaying through the HUD screen, so that the watching of a driver is influenced, and therefore, HUD predistortion processing is required to be carried out on the images. In order to avoid distortion of the projected image, special requirements may be placed on the windshield manufacturing and mounting process, but this special customization is costly and not easy to implement. In addition, the optical device for transmitting the image can be adjusted, but the method also reduces the universality of the vehicle-mounted HUD system, improves the production cost and is not beneficial to the popularization of the HUD on automobiles.

Therefore, the method for correcting the distortion of the image is a choice with low cost, simple and convenient operation and wide application range. For HUD distortion correction, the current scheme is to process the HUD distortion correction in a main control chip of a vehicle machine through a CPU or a GPU, and because the HUD image correction processing data size is large and the real-time requirement is high, the main control chip resources are seriously occupied by HUD correction tasks, so that the main control chip cannot process other tasks in time. Resulting in poor user experience with the vehicle body.

How to provide a flexible and efficient HUD image processing system is a problem to be solved currently.

Disclosure of Invention

In view of the above, the present invention provides a HUD image distortion correction module, a HUD image distortion correction method, and a HUD image processing display device for solving the deficiencies of the prior art.

In order to achieve the above purpose, the invention adopts the following technical scheme:

the embodiment of the invention provides a HUD image distortion correction module, which comprises the following components: the device comprises a preprocessing input interface unit, a preprocessing read-write control unit, a memory unit, a judging unit, a bus write interface unit, a bus read interface unit, a bus distortion control unit, a time sequence generating unit, a multi-line cache unit and a distortion correcting unit;

the preprocessing input interface unit is used for generating a data signal which accords with the bit width of the bus after preset processing;

the preprocessing read-write control unit is used for writing the data signals output by the preprocessing input interface unit into the memory unit for storage; and the data reading unit is also used for reading the data of the memory unit after receiving the data reading request of the judging unit;

the storage unit is used for storing data;

the judging unit is used for controlling the execution of different data streams under different working scenes; the scene time judging unit reads the data stored in the memory unit through the preprocessing read-write control unit and sends the data to the multi-line cache unit through the bus distortion control unit; in the second scene, the judging unit reads the data stored in the memory unit through the preprocessing read-write control unit and sends the data to the bus write interface unit; the memory unit directly sends the data to the distortion correction unit after receiving the data reading request of the preprocessing read-write control unit in the third scene;

the bus write interface unit is used for sending the received data to a bus and finally writing the received data into a frame buffer;

the bus reading interface unit is used for reading frame buffer data from a bus;

the bus skew control unit is used for sending the frame buffer data acquired from the bus read interface unit to the multi-row buffer unit after sending the frame buffer request signal to the bus read interface unit;

the time sequence generating unit is used for receiving the time sequence output by the front end and the time sequence parameters configured by the system and generating the working time sequence required by the distortion correcting unit; the data processing unit is also used for generating a prefetched image signal and outputting the prefetched image signal to the bus distortion control unit so as to prefetch the data in the frame cache to the multi-line cache unit in advance;

the multi-line buffer unit is used for storing a plurality of lines of image data for parallel use by the distortion correction unit;

the distortion correction unit is used for realizing distortion correction processing under different scenes.

Further, the first scene is a scene which does not need frame rate conversion and performs distortion correction of which the glass curvature is smaller than a preset value; the second scene is a scene requiring frame rate conversion or distortion correction with glass curvature larger than or equal to a preset value; the third scene is a scene which does not need distortion correction.

Further, the preset process includes: and (5) mixing and storing YUV format data and performing horizontal mirroring processing.

Further, the module further comprises: an offset table unit for storing feature point information required for image distortion correction, including offsets in the horizontal direction and the vertical direction of the feature points; the offset table unit adopts a double-cache design; one set of caches is used for display and another set is used for system configuration changes.

Further, the HUD image distortion correction module further includes: the image preprocessing unit is used for receiving or generating a HUD image source, and sending the image to the preprocessing input interface unit after the image is subjected to scaling, clipping, format conversion and other processes;

and the image post-processing and outputting unit is used for carrying out gamma processing on the image corrected by the distortion correcting unit and outputting the image to the subsequent vehicle-mounted display unit for display.

Further, the distortion correction unit includes:

the mapping coordinate calculation unit is used for receiving the image time sequence and coordinate information of the time sequence generation unit, and obtaining accurate coordinate values mapped to the decimal part of the source image through calculation;

and the bilinear interpolation calculation unit is used for calculating the accurate coordinate value with the decimal part, taking 4 integer coordinates, which are adjacent to each other in the vertical direction and the horizontal direction, of the accurate coordinate value, using the integer coordinates as address information to read the image content stored in the multi-line cache, and carrying out interpolation calculation so as to obtain the pixel value after HUD distortion correction.

Further, the map coordinate calculation unit includes:

a scaling rotation subunit, configured to perform scaling and rotation processing on the input coordinate point;

the index weight acquisition subunit is used for receiving the coordinate information after scaling and rotation, determining the offset table information in a proper range in the offset table corresponding to the coordinate, and finally accurately positioning the index of the offset table block where the current point is located and the weights of the coordinate points relative to 4 intersection points in the block;

a source block index value obtaining subunit, configured to obtain an index value of a corresponding block in the source map according to the hit-selected block index;

and the accurate coordinate calculation subunit is used for calculating the accurate coordinate value of the decimal part of the current point in the source diagram by adopting a bilinear interpolation method according to the source block index value and the weight information.

Further, the index weight acquisition subunit includes: an offset table reading logic subunit, a plurality of block calculation subunits, and a block selection logic subunit;

an offset table reading logic subunit, configured to determine a column range and a row range of the coordinate point, and read intersection information stored in the offset table for use by the block calculation unit;

a block calculating subunit, configured to calculate whether the coordinate point is in a preset block defined by 4 points determined by the offset table, and mark the coordinate point as a block is selected when the coordinate point is in the block; judging whether the coordinate points are on adjacent points on the upper, lower, left and right sides of a curve equation formed by a preset block when the coordinate points are not in the block, and marking that the edges of the block are selected when the coordinate points are on the adjacent points on the upper, lower, left and right sides of the curve equation; when the weight is selected, calculating the corresponding weight;

and the block selection logic subunit is used for receiving the selected marks and weights of all parallel calculation of each block, selecting one block as an effective selected block if one block mark is selected in all the blocks, and selecting the block with the selected block edge as a hit block if one block mark is not selected in all the blocks but one block is selected as a block edge.

The embodiment of the invention also provides a HUD image processing display device, which comprises: the HUD image distortion correction module of any of the above.

The embodiment of the invention also provides a HUD distortion correction method, which comprises the following steps:

when frame rate conversion is not needed and distortion correction with glass curvature smaller than a preset value is carried out, image data are sent to a multi-line caching unit, and the distortion correction of the image is directly realized in a multi-line caching mode;

when frame rate conversion is needed or distortion correction with glass curvature larger than or equal to a preset value is carried out, firstly storing image data into a frame buffer through a bus interface, then reading the data from the frame buffer and sending the data to a multi-line buffer unit, and realizing the distortion correction of the image by utilizing the frame buffer and the multi-line buffer;

when the distortion correction is not required, the image can be subjected to translational adjustment output by the control of the timing generation unit.

Compared with the prior art, the invention has the following beneficial effects: the special module is selected for distortion correction processing, so that the resources of the main control chip are not occupied, and the main control is prevented from being seriously occupied by the HUD correction task to influence the experience effect of the vehicle machine; meanwhile, the HUD image distortion correction module can be configured in different working scenes according to different application scenes, and the application range is wide.

Drawings

In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a block diagram of a HUD distortion correction module provided by an embodiment of the invention;

fig. 2 is a block diagram of a HUD distortion correction module according to another embodiment of the present invention;

fig. 3 is a structural diagram of the distortion correction unit;

fig. 4 is a block diagram of the map coordinates calculating unit in fig. 3;

FIG. 5 is a block diagram of the indexing weight acquisition subunit of FIG. 4;

FIG. 6 is a schematic diagram of the distortion of the calibration plate;

FIG. 7 is a diagram of weights for a block selected;

FIG. 8 is a weight diagram of block edge selected.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

The HUD distortion correction is to perform distortion correction processing on a normal image input without distortion, that is, the image is pre-bent, so that the image after the pre-bending is output, and the image is a normal shape image when displayed on a vehicle-mounted windshield with curvature.

As shown in fig. 1, in an embodiment of the present invention, the HUD image distortion correction module includes a preprocessing input interface unit pre_yc_if, a preprocessing read/write control unit pre_wr_ctl, a decision unit pre_wr_arbiter, a memory unit pre_yc_mem, a bus write interface unit axi_wr_if, a bus read interface unit axi_read_if, a bus distortion control unit axi2warp_ctl, a timing generation unit timing_gen, a multi-line buffer unit lbuf_blk, and a distortion correction unit warp_engine.

The preprocessing input interface unit pre_yc_if receives signals output by the image preprocessing unit, and after the signals are stored into the memory unit pre_yc_mem through the preprocessing read-write control unit pre_wr_ctl, the judgment unit pre_wr_arbiter reads out from the memory unit pre_yc_mem according to the need, and the preprocessing input interface unit pre_yc_if comprises 3 working scenes:

when distortion correction is required, the following three scenarios are divided:

in scene 1, when frame change is not needed and the glass bending degree is smaller, data read in a memory unit is sent to a multi-line cache unit lbuf_blk through a preprocessing read-write control unit pre_wr_ctl, a judging unit pre_wr_arbiter and a bus distortion control unit axi2warp_ctl, and HUD distortion correction is directly realized in a line cache mode.

Scene 2, when frame change or glass bending degree is provided, the data read in the memory unit is sent to the bus write interface unit axi_wr_if through the preprocessing read-write control unit pre_wr_ctl and the judging unit pre_wr_arbiter, and the input image data is stored into an external frame buffer through the axi bus write interface; under the control of the timing sequence output by the timing sequence generating unit timing_gen, the bus distortion control unit axi2warp_ctl reads images from the frame buffer through the bus read interface unit axi_read_if, then the image data is buffered into the multi-line buffer unit lbuf_blk, and then the data read from the multi-line buffer unit lbuf_blk is sent to the distortion correction unit warp_engine for image distortion correction processing.

Scene 3, when no distortion correction is required, the data read in the memory unit is directly transmitted to the distortion correction unit warp_engine.

The HUD correction engine can be configured in different working scenes according to different application scenes, when the frame is not required to be changed and the glass bending degree is small, resources stored by external frames such as SDRAM or DDR RAM can be omitted, the distortion correction of the image can be directly realized by using a multi-line cache, the cost can be saved, and when the frame is required to be changed or the glass bending degree is large, the image can be stored and read through bus interfaces axi_wr_if and axi_read_if. When distortion correction is not needed, only translation adjustment of the image can be performed, and the image can be simply output.

The following describes the individual units in more detail to better understand the embodiments of the present invention.

The timing generation unit timing_gen receives the timing output by the front end and includes the field synchronizing signal vsyn_pre and the timing parameters of the system configuration, and generates the timing of the HUD correction engine to control the distortion correction unit to work according to the correct timing. According to the different working scenes of the HUD correction engine, the HUD correction engine flexibly works in different scenes. In the scene 3, distortion correction is not required, the frequency of the generated field synchronous signal vsyn_warp is the same as that of the input field synchronous vsyn_pre, the image effective signal can be different from the input image through the offset parameter, and the display position of the effective image on the screen can be adjusted through the offset of the effective time sequence. In scenario 2, after the HUD correction timing is enabled, the HUD correction engine timing is automatically generated according to configured timing parameters such as the number of points and the number of lines, and the timing frame rate can be different from the input timing frame rate, so that the display requirement of the output frame rate can be met. In the scene 1, the distortion correction of the image is realized in a multi-line buffer mode, the generation of correction time sequence is similar to that in the scene 3, the input and output frame rate is consistent, and the control of the balance of the data flow of the multi-line buffer correction processing is realized by adjusting effective offset, blanking and line frequency. Since HUD distortion correction is to be performed across lines, it is necessary to store many lines of image data while applying them to the distortion correction unit warp_engine, and when the correction timing of the distortion correction unit warp_engine arrives, it is necessary to read out the lines of image data from the frame buffer in advance for use, and the timing generation unit timing_gen generates a prefetch image signal vsyn_rd which is output to the bus distortion control unit axi2warp_ctl in addition to the correction timing, so that the lines of image data can be prefetched from the frame buffer into the lines of buffer unit which is convenient for parallel processing in advance. The usual procedure is to generate a trigger at the beginning of the vertical blanking, and the vsyn_rd signal here is flexibly configurable, can be generated at the beginning of the vertical blanking, or can be generated at any suitable position at the end of the previous frame, in particular, the pre-fetch signal vsyn_rd of the new frame is automatically generated when the horizontal and vertical timing counter is equal to this value, depending on the configurable line trigger position and the point trigger position. By generating vsyn_rd, the problem that the image error is caused by the fact that the time for reading a plurality of lines of images is not enough when the vertical blanking of one frame of images is short can be solved.

The preprocessing input interface unit pre_yc_if receives timing signals hsyn_i, vsyn_i, den_i, image data signal data_i, which are output from the front end, and software-configurable register parameters, such as: the source window signals rec_hst, rec_hsp, rec_vst, rec_vsp, the mirror signal hmirror and the data format signal data_format are subjected to preset processing to generate a data signal and a valid signal which accord with the bus bit width. After the source window signal is synchronized by vsyn_i, the source window signal is used to select the valid image data data_i to be data_use, and other image areas are endowed with black values. The mirror signal hmirror is used to control the interaction of the data_use high and low byte positions, which are swapped when mirroring is required. data_i is 24-bit data (including ycbcr three components) in the yuv444 format, and when data_format is 1, it indicates data stored in the yuv444 format, and when data_format is 0, it indicates data stored in the yuv422 format. The input signal data_i is arranged and selected as follows:

y0 cb0 cr0 y1 cb1 cr1 y2 cb2 cr2 y3 cb3 cr3 y4 cb4 cr4 y5 cb5 cr5 y6 cb6cr6 y7 cb7 cr7

data_format is 1, bus bit width is 64bit, 8 points occupy 3 address spaces in normal mode, store as:

y0 cb0 cr0 y1 cb1 cr1 y2 cb2

cr2 y3 cb3 cr3 y4 cb4 cr4 y5

cb5 cr5 y6 cb6 cr6 y7 cb7 cr7

the 8-bit address and the high-low byte are exchanged during mirroring and stored as

cr7 cb7 y7 cr6 cb6 y6 cr5 cb5

y5 cr4 cb4 y4 cr3 cb3 y3 cr2

cb2 y2 cr1 cb1 y1 cr0 cb0 y0

data_format is 0, 8 points occupy 2 address spaces in normal mode, and are stored as:

y0 cb0 y1 cr1 y2 cb2 y3 cr3

y4 cb4 y5 cr5 y6 cb6 y7 cr7

the mirroring requires that the 8-bit address and the high and low bytes are swapped, while the storage is swapped from cb and cr, stored as

y7 cb7 y6 cr6 y5 cb5 y4 cr4

y3 cb3 y2 cr2 y1 cb1 y0 cr0

When a plurality of points are combined into 64-bit data, according to the row counter, the operation is performed according to the rule, and when a 64-bit data signal is input, an effective signal is generated. The pre_yc_if processing can support the storage of the image format as yuv444 or yuv422, and the mode of mixing and storing the data with different yuv formats can facilitate the subsequent bus transmission to transmit with larger burst and higher efficiency; meanwhile, the horizontal mirror image function is supported, the mirror image processing of a follow-up optical component and a screen display optical path is not needed when the HUD is used for screen projection, more flexible selection is provided, and the display cost is reduced. The preprocessing input interface unit pre_yc_if outputs a line field timing signal of processing delay to the preprocessing read-write control unit pre_wr_ctl at the same time so as to control memory read-write.

The preprocessing read-write control unit pre_wr_ctl is used for receiving a time sequence signal, an effective signal and a data signal output by the preprocessing input interface unit pre_yc_if, writing image data into the memory unit pre_yc_mem by using a ping-pong mechanism under the control of the time sequence signal, combining the preprocessing input interface unit pre_yc_if, writing the first data into a tail address when mirroring is needed, and realizing mirror-written data mirroring in an address decrementing mode. When the memory unit pre_yc_mem stores the first row data, a first frame writing request signal is sent to the judging unit pre_wr_arbiter, and when the continuous rows are stored, a frame writing row request signal is sent to the judging unit pre_wr_arbiter, so that the efficiency of accessing data through a bus can be improved in a row processing mode.

The memory unit pre_yc_mem for storing image data is preferably designed as a memory of 2 rows of ping pong. By adopting 2 rows of ping-pong storage, seamless buffering of data can be effectively realized, the data processing efficiency is improved, and meanwhile, the buffering space of a memory unit can be effectively saved.

The judging unit pre_wr_arbiter is used for reading the data stored in the memory unit at a moment and sending the data to the multi-line cache unit through the bus distortion control unit; reading data stored in the memory unit and sending the data to the bus write interface unit in the second scene; reading data stored in the memory unit and transmitting the data to the distortion correction unit in the third scene;

scenario one is a situation where frame buffering is not needed, such as: no frame rate conversion is required and distortion correction with a glass curvature smaller than a preset value is performed. Scenario two is a situation requiring frame buffering, such as: frame rate conversion is required or distortion correction with a glass curvature equal to or greater than a preset value is performed. Scene three is a case where no distortion correction is required.

When receiving the write request signal from the preprocessing read/write control unit pre_wr_ctl, the processing read/write control unit pre_wr_ctl sends a write bus request signal to the bus write interface unit axi_wr_if, and when the bus interface allows, the processing read/write control unit pre_wr_ctl reads out the image data stored in the memory unit pre_yc_mem and sends the image data to the bus write interface unit axi_wr_if. In the case of scenario 1 and scenario 3, the decision unit pre_wr_arbiter finally transmits the data to the multi-line buffer unit lbuf_blk or the distortion correction unit warp_engine.

The bus write interface unit axi_wr_if is used for sending the data received from the decision unit pre_wr_arbiter to the bus according to the bus protocol and finally writing the data into the frame buffer.

The bus distortion control unit axi2warp_ctl is used for sending a request signal for reading the frame buffer to axi_read_if according to the trigger control signal vsyn_rd generated by the timing_gen of the timing generating unit, and writing the frame buffer data read by the interface from the bus into the multi-line buffer. When the rising edge of vsyn_rd is detected, the first line data of one frame of image starts to be read, a read request signal is sent to the bus read interface unit axi_read_if, and meanwhile, a read address, a read line length and the like are transmitted to the bus read interface unit axi_read_if together. In the initial stage of a frame, after one line of data is read and stored in the line cache, the next line of image data is repeatedly requested to be read until all line caches in the line cache are filled. When the working time sequence of the distortion correction unit warp_engine is in the image valid period, partial lines in the multi-line cache unit are calculated and read according to coordinates to be processed, the partial lines need to be read in parallel and simultaneously, and when the maximum value of the address of the partial lines is close to the address value written by the bus distortion control unit axi2warp_ctl to the multi-line cache unit. The bus skew control unit axi2warp_ctl issues a read next line image data request signal to axi_read_if, reads out new image contents from the frame buffer, and stores them in the multi-line buffer. By constantly reading and writing, the balance of the image data can be ensured, and the correction requirement of the distortion correction unit warp_engine can be satisfied.

The bus read interface unit axi_read_if is used for reading the data of the frame buffer from the bus according to the bus protocol by combining the read frame data request signal sent by the bus skew control unit axi2warp_ctl, and finally writing the data into the multi-line buffer unit lbuf_blk after being processed by the bus skew control unit axi2 warp_ctl.

And the multi-line caching unit lbuf_blk is used for storing multi-line image data for parallel use by the distortion correcting unit warp_engine.

And the distortion correction unit warp_engine is used for realizing distortion correction processing under different scenes.

In a preferred embodiment, as shown in fig. 2, the HUD image distortion correction module further includes: the offset table unit ofst_blk is used for storing an offset table required for image distortion correction. The distortion correction unit warp_engine performs image distortion correction based on an offset lookup table, and the offset table unit ofst_blk is used for storing feature point information required for correction, including offsets in the horizontal and vertical directions of the feature points. The contents of the offset table unit ofst_blk are configured in advance by the system, the offset table unit ofst_blk and the distortion correction unit warp_engine send a data request for acquiring an offset table to the offset table unit ofst_blk under the timing control signal output by the timing generation unit time_gen, and after receiving the request, the offset table unit ofst_blk sends corresponding offset table information to the distortion correction unit warp_engine so as to execute distortion correction processing. In order to ensure that the image correction coefficient can be adjusted without flashing when the image is normally displayed, the offset table unit ofst_blk is designed into double caches, one group of caches is used for displaying under the timing control of the timing generator unit timing_gen, the other group of caches can be changed by the system configuration, and in particular, if the idle offset caches are configured and an instruction for changing the offset table is received, the other group of offset tables can be switched and output in the field blanking period, so that the ofst_blk can be adjusted to attach different HUD display glasses even after leaving a factory for getting on a vehicle. Each set of offset tables of the offset table unit ofst_blk is designed in the form of a raster arrangement, the range of stored intersection characteristic values is sz1 row, sz2 column, sz1 and sz2 are natural numbers.

As shown in fig. 3, the distortion correction engine unit warp_engine includes: a mapping coordinate calculation unit mapcoord_cmpt and a bilinear interpolation calculation unit bilinear.

The mapping coordinate calculating unit Mapcoord_cmp is used for receiving the image time sequence and coordinate information generated by the time sequence generating unit time_gen, and obtaining the accurate coordinate value mapped to the decimal part of the source image through calculation.

The specific structure of the mapping coordinate calculating unit is shown in fig. 4, and mainly includes a scaling rotation subunit point_scale_rotation, an index weight obtaining subunit get_block_index, a source block index value obtaining subunit get_cross_points, and an accurate coordinate calculating subunit bilinear.

And the scaling and rotating subunit point_scale_rotate receives the time sequence coordinate information output by the time sequence generating module and the central Point, the scaling and rotating angle parameters of the system configuration, and performs scaling and rotating processing on the input coordinates. Let the input point coordinates be cordy_i, cordx_i, delay one beat of signal be cordy_d1, cordx_d1, when the horizontal active timing starts pulse, cordy_sel is selected as cordy_i, cordx_sel is selected as cordx_i, and other horizontal active periods cordy_sel is selected as cordy_d1, cordx_sel is selected as cordx_d1. And respectively subtracting the positions of the image center points from the cordy_d1 and the cordx_d1 to obtain coordinate values of the positions of the relative center points, and multiplying the coordinate values by a configurable scaling factor to obtain coordinates cordy_scal and cordx_scal of the horizontally and vertically scaled images. The unit also rotates the coordinate after scaling, and when the configurable rotation angle theta is greater than or equal to 0, the rotation is calculated as follows:

roti＝cos_angle(theta)*coordy_scal-sin_angle(theta)*coordx_scal；

rotj＝cos_angle(theta)*coordx_scal+sin_angle(theta)*coordy_scal；

otherwise, when index is less than 0,

roti＝cos_angle(theta)*coordy_scal+sin_angle(theta)*coordx_scal；

rotj＝cos_angle(theta)*coordx_scal-sin_angle(theta)*coordy_scal；

in particular, since the rotation angle is not changed in real time, the configuration is changed only when the rotation angle needs to be changed, when the unit calculates the forward rotation or cosine value of the rotation angle, the cos_angle (theta) and sin_angle (theta) are calculated first by using software, and then the configuration is carried out in a register mode, so that the cost of looking up a table to calculate the trigonometric function value can be saved by hardware. The hardware only needs to calculate the multiplication and addition operations in the rotation formula. Besides the calculation of the effective point, when the cordy_sel selects the cordy_i, the calculated result is additionally stored as a new for the next module to use, and the new and roti share calculation logic, and only the output storage is selected through time sequence control, so that the additional expenditure of the new is saved.

And the index weight acquisition subunit get_block_index receives coordinate information after scaling rotation, determines offset table information in a proper range in an offset table corresponding to the coordinate, and finally accurately positions the index of the offset table block where the current point is positioned and the weights of the coordinate points relative to 4 intersection points in the block. The specific structure of the subunit is as shown in fig. 5, and includes an offset table reading logic subunit, a plurality of block calculation subunits Blk00, blk01, blk02, blk03, blk04.

The offset table reads the logic subunits, first determining the column spacing col_interval, which is calculated as follows:

col_interval= (sz 2-1)/(warp dis_j (1, sz 2) -warp dis_j (1, 1)), where sz2 is the offset table horizontal maximum, warp dis_j (1, sz 2) is the horizontal coordinate of the first row sz2 point, and warp dis_j (1, 1) is the horizontal coordinate of the first row first point.

The column range where the current point is located is estimated approximately by using the coordinate information rotj and col_interval and warp dis_j (1, 1) input after scaling rotation, and kk is calculated first as follows:

kk= (rotj-col_offset) col_interval; the column offset col_offset is a parameter of the system configuration.

The column range col_range is determined according to kk, specifically:

when kk < = 3, col_range=1: 6, preparing a base material;

when kk > = sz2-3, col_range = sz2-5:sz2-0;

otherwise, col_range=kk-2:kk+3;

determining the line range is described next. The distorted image is smile-curve-shaped and stepped horizontally, and after rotation, as shown in fig. 6, the number of lines taken in the vertical direction can be smaller by sectioning the intersection point horizontally, so that the number of blkxy computing units is saved. Assuming a division into 2 segments, the row offset row_offset is determined first: when at the image center left Bian Shi, row_offset=min (ptsdis_i (1, 1), ptsdis_i (1, 0.5×sz2)); when on the right, row_offset=min (ptsdis_i (1, 0.5×sz2), ptsdis_i (1, sz2)).

The line spacing is calculated as follows:

row_interval＝(sz1-1)/(ptsDis_i(sz1,1)-ptsDis_i(1,1))；

rkk, rkk = (new-row_offset) ×row_interval is calculated according to the row offset and row spacing and the rotated row coordinates new output by the rotation module, and finally a row range is determined according to rkk, specifically as follows:

when rkk < = 3, row_range = 1:6, preparing a base material;

when rkk > = sz2-3, row_range = sz2-5:sz2-0;

otherwise, row_range= rkk-2:rkk+3;

the block range selected in the embodiment of the invention is 5x5, the read range is 6x6 intersection point, after the column range and the row range are determined, the offset table content of 6x6 stored in the ofst_blk is read according to row_range and col_range for calculating the block.

The blocks Blk00, blk01 and Blk02 until Blkxy, x and y are equal to 5, the calculation mode of each block calculation subunit is the same, after the offset table content of 6x6 is read out, all the blocks are calculated in parallel, and the result is obtained at the same time, so that the real-time image correction rate is achieved. As shown in fig. 7, the block Blkxy is enclosed by ABCD, the information of each intersection is read by the offset table unit ofst_blk, the curves AD and BC can be fitted to straight lines, because there is substantially no distortion in the vertical direction, and AB and CD can be fitted to smile curves represented by polynomials, and the parameters of the fitting polynomials can be configured by the system. And obtaining equations line_AD, line_BC, curve_AB and curve_DC of four curves according to the intersection point information, substituting the coordinates after rotation scaling into the equations, judging whether the current point P (x, y) is in the block Blkxy again, and if so, sending out a block selection signal blkxy_hit which is equal to 1, otherwise, 0. Meanwhile, when blkxy_hit is equal to 0, judging whether the accurate coordinates of the decimal part of the current point are on adjacent points on the upper, lower, left and right sides of the curve equation, and if yes, sending out a block edge selection signal blkxy_ehit to be equal to 1. As shown in fig. 7, the distance between P and 4 curve edges is taken as a weight output, the weights include x_ad, x_bc, y_ab, y_cd, and fig. 8 shows the case that blkxy_ehit is equal to 1, where y_ab may be a negative value, y_cd may be a slightly larger positive value, and the weighting of the signed number may be such that the image is smoothly excessive during edge point processing.

And a block selection logic subunit, for receiving all the block signals blkxy_hit, blkxy_ehit and weights x_ad, x_bc, y_ab, y_cd calculated in parallel from Blk00 to BLKxy, selecting one block of all blocks as a valid hit block if the block blkxy_hit is equal to 1, and selecting a block of blkxy_ehit equal to 1 as a hit block if the block blkxy_hit is equal to 0 but the block blkxy_ehit is equal to 1 in all blocks, so as to facilitate the image display edge smoothing process.

The source block index value obtaining subunit get_cross_points is that the block mapping is one-to-one, and the index value srcxy of the corresponding block in the source map is obtained according to the block index blkxy selected by hit, and the two are the same.

The accurate coordinate calculation subunit bilinear calculates the accurate coordinate value P' of the current point P with the decimal part in the source diagram by adopting a bilinear interpolation method according to the source block index value and the weight information calculated above.

And the bilinear interpolation calculation unit Bilineryuv calculates the accurate coordinate value of the decimal part of the source image, and then takes 4 integer coordinates which are adjacent to the accurate coordinate value in the upper, lower, left and right directions and serve as address information to read the image content in the multi-line cache, and interpolation calculation is carried out, so that the pixel value after HUD distortion correction is obtained. The module receives the source point coordinates to read the image content in the multi-line cache, calculates the weight required by image interpolation by utilizing the accurate coordinate values of the source image, wherein the accurate coordinate contains a decimal part, the smaller the weight of the decimal part is, the closer the decimal part is to the coordinate values of the integer part, the larger the decimal part is, the weight is, the closer the weight is to the coordinate values of the integer part after adding 1, and the calculation methods of the horizontal coordinate and the vertical coordinate are the same. And finally, performing bilinear interpolation by using the pixels with the weight of 4 integer coordinates to obtain pixel values after the distortion correction of the HUD, wherein the calculation method of each pixel component of the YUV is the same and is not described in detail.

As shown in fig. 2, in a preferred embodiment, the HUD image distortion correction module further includes: the image preprocessing unit is used for receiving or generating a HUD image source, and sending the image to the preprocessing input interface unit after the image is subjected to scaling, clipping, format conversion and other processes; the image source can be an image input by serial lvds and an image input by rgb in parallel, and can also be an image signal generated by self test time sequence. And the image is scaled, so that the display requirement on the HUD curved glass screen can be better met. The front end is added with the clipping function, so that the HUD correction processing can be performed by clipping part of the effective image area to be displayed in the image with larger resolution, and the rest image contents do not need to be output to a HUD follow-up module for processing, so that the efficiency of the HUD image processing system can be greatly improved. The HUD image distortion correction module may further include: and the image post-processing and output unit is used for carrying out gamma and other processing on the image after the image distortion correction and outputting the image to the subsequent vehicle-mounted unit for display. The nonlinear characteristics of the display device can be corrected by gamma processing, and the visual effect and viewing experience of the image are improved.

The embodiment of the invention also provides a HUD image processing display device which comprises the HUD image distortion correction module.

The embodiment of the invention also provides a HUD image distortion correction method which is applied to the HUD image processing display device, and comprises the following steps:

when frame rate conversion is not needed and distortion correction with glass curvature smaller than a preset value is carried out, image data are sent to a multi-line caching unit, and the distortion correction of the image is directly realized in a multi-line caching mode;

when frame rate conversion is needed or distortion correction with glass curvature larger than or equal to a preset value is carried out, firstly storing image data into a frame buffer through a bus interface, then reading the data from the frame buffer and sending the data to a multi-line buffer unit, and realizing the distortion correction of the image by utilizing the frame buffer and the multi-line buffer; when the distortion correction is not required, the image can be subjected to translational adjustment output by the control of the timing generation unit. HUD distortion correction can be configured in different working scenes according to different application scenes, when frame change is not needed and the glass bending degree is small, resources stored by external frames, such as SDRAM or DDR RAM, can be omitted, image distortion correction can be directly realized by using a multi-line cache, cost can be saved, and when frame change or glass bending degree is large, frame storage and reading access can be performed on the image through a bus interface. When no distortion correction is needed, only the translation adjustment of the image is performed, and the output is simple. Is configured in different working scenes. The application range is wide.

While the foregoing is directed to the preferred embodiment, other and further embodiments of the invention will be described in detail herein, with the understanding that the present disclosure is to be considered as an exemplification of the principles of the invention and is not intended to limit the invention to the specific embodiment illustrated and the embodiments illustrated, and any modifications, equivalents, improvements or changes which fall within the spirit and principles of the invention are intended to be included within the scope of the invention as set forth in the appended claims.

Claims (10)

1. A HUD image distortion correction module, the module comprising: the device comprises a preprocessing input interface unit, a preprocessing read-write control unit, a memory unit, a judging unit, a bus write interface unit, a bus read interface unit, a bus distortion control unit, a time sequence generating unit, a multi-line cache unit and a distortion correcting unit;

the preprocessing input interface unit is used for generating a data signal which accords with the bit width of the bus after preset processing;

the preprocessing read-write control unit is used for writing the data signals output by the preprocessing input interface unit into the memory unit for storage; and the data reading unit is also used for reading the data of the memory unit after receiving the data reading request of the judging unit;

the storage unit is used for storing data;

the judging unit is used for controlling the execution of different data streams under different working scenes; the scene time judging unit reads the data stored in the memory unit through the preprocessing read-write control unit and sends the data to the multi-line cache unit through the bus distortion control unit; in the second scene, the judging unit reads the data stored in the memory unit through the preprocessing read-write control unit and sends the data to the bus write interface unit; the memory unit directly sends the data to the distortion correction unit after receiving the data reading request of the preprocessing read-write control unit in the third scene;

the bus write interface unit is used for sending the received data to a bus and finally writing the received data into a frame buffer;

the bus reading interface unit is used for reading frame buffer data from a bus;

the bus skew control unit is used for sending the frame buffer data acquired from the bus read interface unit to the multi-row buffer unit after sending the frame buffer request signal to the bus read interface unit;

the time sequence generating unit is used for receiving the time sequence output by the front end and the time sequence parameters configured by the system and generating the working time sequence required by the distortion correcting unit; the data processing unit is also used for generating a prefetched image signal and outputting the prefetched image signal to the bus distortion control unit so as to prefetch the data in the frame cache to the multi-line cache unit in advance;

the multi-line buffer unit is used for storing a plurality of lines of image data for parallel use by the distortion correction unit;

the distortion correction unit is used for realizing distortion correction processing under different scenes.

2. The HUD image distortion correction module according to claim 1, wherein the first scene is a scene that does not require frame rate conversion and performs distortion correction with a glass curvature smaller than a preset value; the second scene is a scene requiring frame rate conversion or distortion correction with glass curvature larger than or equal to a preset value; the third scene is a scene which does not need distortion correction.

3. The HUD image distortion correction module according to claim 1, wherein the preset process includes: and (5) mixing and storing YUV format data and performing horizontal mirroring processing.

4. The HUD image distortion correction module of claim 1, further comprising: an offset table unit for storing feature point information required for image distortion correction, including offsets in the horizontal direction and the vertical direction of the feature points; the offset table unit adopts a double-cache design; one set of caches is used for display and another set is used for system configuration changes.

5. The HUD image distortion correction module of claim 1, further comprising: the image preprocessing unit is used for receiving or generating a HUD image source, and sending the image to the preprocessing input interface unit after the image is subjected to scaling, clipping, format conversion and other processes;

and the image post-processing and outputting unit is used for carrying out gamma processing on the image corrected by the distortion correcting unit and outputting the image to the subsequent vehicle-mounted display unit for display.

6. The HUD image distortion correction module according to claim 1, wherein the distortion correction unit includes:

the mapping coordinate calculation unit is used for receiving the image time sequence and coordinate information of the time sequence generation unit, and obtaining accurate coordinate values mapped to the decimal part of the source image through calculation;

and the bilinear interpolation calculation unit is used for calculating the accurate coordinate value with the decimal part, taking 4 integer coordinates, which are adjacent to each other in the vertical direction and the horizontal direction, of the accurate coordinate value, using the integer coordinates as address information to read the image content stored in the multi-line cache, and carrying out interpolation calculation so as to obtain the pixel value after HUD distortion correction.

7. The HUD image distortion correction module according to claim 6, wherein the map coordinate calculation unit includes:

a scaling rotation subunit, configured to perform scaling and rotation processing on the input coordinate point;

the index weight acquisition subunit is used for receiving the coordinate information after scaling and rotation, determining the offset table information in a proper range in the offset table corresponding to the coordinate, and finally accurately positioning the index of the offset table block where the current point is located and the weights of the coordinate points relative to 4 intersection points in the block;

a source block index value obtaining subunit, configured to obtain an index value of a corresponding block in the source map according to the hit-selected block index;

and the accurate coordinate calculation subunit is used for calculating the accurate coordinate value of the decimal part of the current point in the source diagram by adopting a bilinear interpolation method according to the source block index value and the weight information.

8. The HUD image distortion correction module of claim 7, wherein the index weight acquisition subunit comprises: an offset table reading logic subunit, a plurality of block calculation subunits, and a block selection logic subunit;

an offset table reading logic subunit, configured to determine a column range and a row range of the coordinate point, and read intersection information stored in the offset table for use by the block calculation unit;

a block calculating subunit, configured to calculate whether the coordinate point is in a preset block defined by 4 points determined by the offset table, and mark the coordinate point as a block is selected when the coordinate point is in the block; judging whether the coordinate points are on adjacent points on the upper, lower, left and right sides of a curve equation formed by a preset block when the coordinate points are not in the block, and marking that the edges of the block are selected when the coordinate points are on the adjacent points on the upper, lower, left and right sides of the curve equation; when the weight is selected, calculating the corresponding weight;

and the block selection logic subunit is used for receiving the selected marks and weights of all parallel calculation of each block, selecting one block as an effective selected block if one block mark is selected in all the blocks, and selecting the block with the selected block edge as a hit block if one block mark is not selected in all the blocks but one block is selected as a block edge.

9. A HUD image processing display device, comprising: the HUD image distortion correction module of any of claims 1-8.

10. A HUD image distortion correction method, comprising:

when frame rate conversion is not needed and distortion correction with glass curvature smaller than a preset value is carried out, image data are sent to a multi-line caching unit, and the distortion correction of the image is directly realized in a multi-line caching mode;

when frame rate conversion is needed or distortion correction with glass curvature larger than or equal to a preset value is carried out, firstly storing image data into a frame buffer through a bus interface, then reading the data from the frame buffer and sending the data to a multi-line buffer unit, and realizing the distortion correction of the image by utilizing the frame buffer and the multi-line buffer;

when the distortion correction is not required, the image can be subjected to translational adjustment output by the control of the timing generation unit.