TWI411967B - System and method for automated calibration and correction of display geometry and color - Google Patents

Abstract

Various embodiments are described herein for a system and method for calibrating a display device to eliminate distortions due to various components such as one or more of lenses, mirrors, projection geometry, lateral chromatic aberration and color misalignment, and color and brightness non-uniformity. Calibration for distortions that vary over time is also addressed. Sensing devices coupled to processors can be used to sense display characteristics, which are then used to compute distortion data, and generate pre-compensating maps to correct for display distortions.

Description

System and method for automatic calibration and correction of display geometry and color Priority claim

The present application claims priority to U.S. Provisional Patent Application Serial No. 60/836,940, filed on Aug.

Field of invention

Various embodiments are discussed with reference to calibration of display devices.

Background of the invention

Most image display devices exhibit some form of geometric or color distortion. Such distortions can have a variety of factors, such as geometric conditional backgrounds, non-ideal properties of various optical components in the system, under-alignment of various components, complex display surfaces and optical paths that result in geometric distortion, and flaws within the panel, and the like. Depending on the system, the amount of distortion can vary greatly, from undetectable to very annoying. The effects of the aforementioned distortions may also vary, and may result in changes in the color of the image, or changes in the shape or geometry of the image.

Summary of invention

In one feature, at least one embodiment described in this specification provides a display calibration system for use with a display device having a viewing surface. Such a display calibration system includes: at least one sensing device adapted to sense information relating to at least one of a shape, a scale, a boundary, and an orientation of the viewing surface; and at least one processor And coupled to the at least one sensing device, and adapted to calculate characteristics of the display device based on information sensed by the at least one sensing device.

In another feature, at least one embodiment described in this specification provides a display calibration system for use with a display device having a viewing surface. Such a display calibration system includes: at least one sensing device adapted to sense information of a test image displayed on the viewing surface; and at least one processor coupled to the at least one sensing device, The at least one processor is adapted to calculate display distortion based on the sensed information. These pre-compensation maps can be implemented by surface functions. When the pre-compensation map is applied to the input image data before display, a display image formed on the viewing surface is substantially free of distortion.

In another feature, at least one embodiment described in this specification provides a display calibration system for use with a display device having a viewing surface. The display calibration system includes: at least one image sensing device adapted to sense information from a test image displayed on the viewing surface; and at least one coupled to the at least one image sensing device The processor, the at least one processor, is adapted to calculate display distortion according to the sensed information, so that the viewing surface is divided into small pieces according to the strictness of display distortion in each chip, and the generated A pre-compensation pattern related to the display distortion in each patch, so that when the pre-compensation map is applied to the input image data before display, a display image formed on the viewing surface is substantially free of distortion.

In another feature, at least one embodiment described in this specification provides a display calibration system for use with a display device having a viewing surface. Such a display calibration system includes: at least one image sensing device adapted to independently sense color information relating to at least one color component of a test image displayed on the viewing surface; and at least one coupled to a processor of the at least one image sensing device, the at least one processor adapted to calculate color non-uniformity based on the sensed information, and to generate at least one color correction map related to the at least one color component, Therefore, when the at least one color correction map is applied to the input image material before the display, a display image formed on the viewing surface is substantially free of at least one color unevenness.

In another feature, at least one embodiment described in this specification provides a display calibration system for use with a display device having a viewing surface. Such a display calibration system includes: at least one image sensing device adapted to sense information of an individual color component test image displayed on the viewing surface; and at least one coupled to the at least one image sensing And a processor of the display device, the at least one processor adapted to independently calculate geometric display distortion based on the sensed information, and to independently generate at least one precompensation pattern associated with the at least one color component Therefore, when the at least one color correction map is applied to the input image data before the display, a display image formed on the viewing surface is substantially free of at least one color dependency geometric distortion.

In another feature, at least one embodiment described in this specification provides a display calibration method that can be used in a projection system having a curved viewing surface, the method comprising the steps of: using multiple projectors, Projecting a different portion of an image onto a corresponding portion of the curved viewing surface; and focusing each portion of the image substantially over a corresponding portion of the curved viewing surface to optimize the focus of the image, It is integrally formed on the curved viewing surface.

In another feature, at least one embodiment described in this specification provides a display calibration method that can be used in a projection system having a curved viewing surface, the method comprising the steps of: measuring a curved viewing surface a plurality of distances to a focus plane of the projected image; and shifting the focus plane until the functions of the plurality of distances are minimized to obtain an optimized focus.

Simple illustration

For a better understanding of the embodiments and/or related implementations of the present specification, and for the purpose of clarity of the disclosure, reference to the accompanying drawings, FIG. Embodiments and/or related implementations: Figure 1 is a simplified diagram of an exemplary embodiment of an automated calibration and calibration system; Figures 2a and 2b are illustrations of curved screen geometry; and Figure 3 is a geometric distortion An illustration of an overflow, underflow, and mismatch paradigm; Figure 4 is an illustration of an example of a calibration image test pattern; and Figure 5 is an illustration of a corrected geometry condition and various coordinate spaces involved; 6 is an illustration of an exemplary embodiment of a calibration data generator; Figure 7 is an illustration of an optimization of the scale and origin; and Figure 8 is an example of a multiple color calibration data generator. Illustrative diagram of an embodiment; FIG. 9 is an illustration of a setting related to color unevenness calibration; FIG. 10 is an illustration of an exemplary embodiment of a calibration data generator related to color unevenness correction Figure 11 is an illustration of an exemplary embodiment of a warpage data generator; Figure 12 is an illustration of a display for correcting segmentation of a tile; and Figure 13 is an exemplary embodiment of a digital warp unit Figure 14 is a schematic diagram showing the arrangement of the shape and relative orientation of the viewing surface; Figure 15 is an illustration of a defocusing test pattern; Figure 16 is an illustration of a focus test pattern; Figure is a partial illustration of an exemplary embodiment of a calibration system consisting of a multi-projector and a curved screen; Figure 18 is a multi-projector for displaying the focal planes of different projectors and Figure 17 And a partial illustration of an exemplary embodiment of a calibration system consisting of a curved screen; Figure 19 is an illustration of an example of a focusing technique that minimizes a distance function; and Figure 20 is another multi-projection Partial illustration of an exemplary embodiment of a calibration system consisting of a curved screen and an adjustable projection position to optimize image focus; Figure 21 is a multi-use Partial illustration of an exemplary embodiment of a camera calibration system; FIG. 22 is an exemplary embodiment of a rear projection television (RPTV) having an integrated calibration system that allows the display to self-calibrate and allow for dynamic distortion correction. Partial illustration of the exemplary embodiment; Figure 23 is a partial illustration of an exemplary embodiment of a calibration system of multiple projectors and multiple sensing devices; Figure 24 is a calibration using the physical edges and boundaries of the viewing surface described above. Partial illustration of an exemplary embodiment of the system; Figure 25 is a partial illustration of an exemplary embodiment of a calibration system that uses a focusing technique to determine the shape of a curved display screen; and Figure 26 A partial illustration of an exemplary embodiment of a calibration system that uses a focusing technique to determine the shape of a wavy display screen.

Detailed description of the preferred embodiment

It should be understood that, for simplicity and clarity of illustration, reference numbers may be repeated between the figures to indicate corresponding or similar elements. In addition, numerous specific details are set forth to provide a thorough understanding of the embodiments and/or embodiments disclosed herein. However, it will be understood by one of ordinary skill in the art that the embodiments and/or implementations described herein may not require such specific details. In other instances, some of the methods, procedures, and components of the present invention are not described in detail, and are not intended to obscure the embodiments and/or implementations described herein. In addition, the description should not be taken as limiting the scope of the embodiments described in the specification, but rather the structure and/or operation of the embodiments described herein.

Some of the important distortions associated with some display devices include: distortion caused by the lens assembly, distortion from the mirror (curved or planar) reflection group; distortion caused by projection geometry, such as off angle and rotation Projection (trapezoid, rotation) and projection on the surface of the screen; lateral chromatic aberrations and distortions of different colors, such as under-alignment and misconvergence in multiple microdisplay devices; color and luminance (brightness) are not Distortion caused by uniformity and optical focusing problems (spherical aberration, astigmatism, etc.).

The first group sees the geometric distortion in the final image, ie the input image shape is not preserved. Chromatic aberration is also a geometric distortion; however, distortion varies with each color component. Such distortions are common in projection (front or rear) display devices and collectively referred to as geometric distortion. Chromaticity and brightness non-uniformity affects all display devices, so a signal that means a fixed brightness or chromaticity is seen to vary across the surface of a display device, or is different from its intended feel. . This type of distortion may be caused by some varying brightness, varying optical path length across the display, and uneven sensor response in the panel (eg, LCD, LCOS, plasma display) sources. Focusing on the associated distortion blurs an image and is caused by focusing different points on the plane of the object on different image planes. In the exemplary embodiments of the present specification, there are certain issues related to focus and depth of focus.

Embodiments of the present specification describe a system and method for calibrating a display device to eliminate or reduce at least some of the distortions described above. These embodiments can automate both the calibration data and the resulting corrections and the application of the calibration. It is also aimed at the calibration related to the distortion of their long-term changes. The calibration phase (generating calibration data) relates to: characterizing the display; photographing the particular test pattern observed on the display device, for example, through a sensing device similar to a high resolution camera; and from this The image is taken out (ie, the calibration data). The correction phase involves pre-distorting the image via an electronic correction device to cause an undistorted image on the screen. Also proposed is a mechanism for achieving the best test focus and shooting test style.

1 is a simplified diagram showing an exemplary embodiment of an automated calibration and calibration system that corrects an image displayed on a viewing surface 16 of a display device. Such an automated calibration and calibration system includes a test image generator 14, a sensing device 11, a calibration data generator 12, a warpage generator 13, and a digital warping unit 15. The display device may be a television (rear projection television, LCD, plasma, etc.), a front projection system (ie, a projector with a screen), or any other system capable of presenting images, all of them Has a viewing surface. The viewing surface 16, typically has a border or border that distinguishes it from the background; this would normally be a physical frame surrounding the display screen (viewing surface). However, the boundary is not necessarily a frame or a physical shape. Typically, a boundary may be associated with any area of the solid viewing surface 16 that may be distinguished by a device from the background area. For example, a rectangular outline projected onto the display by the device and displayed inside the physical screen frame by the display can be confirmed as the boundary. In the exemplary embodiment recited in this specification, the viewing surface 16 is considered to be in the sense of calibration and correction, and is considered to be the actual display device located in the above-described, at least in some cases, possible identification of the frame itself. The area within the boundary. This boundary is also referred to as the viewing surface border shown above in FIG. 1 as surrounding the viewing surface 16.

For curved screens with varying depths, the display can take two main observation points. The viewing plane can be viewed as a focal plane in which the image is to be in the correct form, which may differ from the physical viewing surface 16 described above, or only a portion of the entity of the viewing surface 16. All points on the above-mentioned focal plane have the same depth of focus. In this case, the physical mark or field of view of the sensing device (i.e., the observer) will determine the focal plane boundary (see Figure 2a). The viewing surface bezel is used to determine the orientation of the camera relative to the viewing surface 16 when available.

Alternatively, the entire screen may be viewed such that the physical screen frame forms the curved boundary as described above (see Figure 2b). Here, the different points above the screen have different depths of focus. This calibration and calibration operation focuses on matching the final image to the curved boundary.

The two observation points can be combined to identify the different display areas required for the calibration and calibration. For example, the boundary can be viewed as a combination of the physical screen frames described above, except that the image features captured above are outside a particular focus plane. A curved border may also be forced onto a flat display by projecting a curved profile. This can be considered as a special case where the boundary is curved, but the screen itself is flat, that is, has an infinite radius of curvature.

For some distortions involving changes in shape or geometry, the image viewed on the viewing surface 16 (before correction) may not be fully included (overflow). This is shown in Figure 3. In case (a), the image ABCD overflows and may completely contain the viewed surface frame 18, while in case (b), the image is completely included (underflow). Case (c) is an intermediate condition (mismatch) in which the image portion is partially covered by the viewing surface 16 described above. All three situations may be caused by the front or rear projection system and may be corrected by the system.

The test image generator 14 provides images that contain special patterns designed for the calibration process; these images are also referred to as calibration test patterns. These calibration test styles that can be used and are most commonly used include: regular (non-joined) ruled line styles, circular, square, horizontal, and vertical styles, bars, lines, concentric styles, rectangles, circles Shape, and uniform gray and chromaticity. The color version described above can be used for lateral chromatic aberration correction and chromaticity unevenness correction. The shapes in these styles are also referred to as topography. Each style has its well-defined topographical characteristics, i.e., the number, location, scale, boundaries, color, and any other defined parameters of the topography are known.

Several exemplary correction patterns are shown in panels (a) through (m) of Figure 4. The criteria (central position, radius, etc.) that are used to display these characteristics are not part of the test pattern. The color and shape changes of these test styles can also be used to exchange black and white, replace black and white with color, use different colors related to different shapes within a style, combine different shapes within a style, and change grayscale. And chroma.

These versions using the style of the primary colors are used to correct lateral chromatic aberrations. An exemplary color style is displayed in panel (g), where the horizontal lines, vertical lines, and their intersections are all of a different color.

Each style presents certain definite features, the most notable of which are the central positions of the shapes and their boundaries, which are mathematically considered to be points and lines, respectively.

The sensing device 11 can record the calibration test pattern seen on the viewing surface 16. To correct for geometric distortion, the sensing device 11 may be a camera. The camera resolution and shooting format can be selected based on the accuracy required in the above calibration. The sensing device 11 may be a color analyzer (eg, a photometer or a spectrometer) when correcting for unevenness in chromaticity and brightness.

In this exemplary embodiment, to correct for geometric errors, the sensing device 11 can be placed at any position relative to the display device described above. Such a degree of freedom in the positioning sensing device 11 may be based on the fact that the captured images may be allowed to contain distortion components by the positioning of the sensing device 11. Unless the sensing device 11 directly views the viewing surface 16 (i.e., directly above), there will be a trapezoidal component due to the sensing device 11. Such deformation may occur in up to three axes, which are considered to be multi-axis trapezoidal distortion components.

Furthermore, since the optics of the sensing device 11, such as a camera, have its own distortion, there is also an optical distortion component that is taken into account. Other types of sensing devices 11 have other inherent distortions. The combined distortion introduced by the above camera or sensing device 11 will be referred to as camera distortion. Such camera distortion is determined and compensated for when the calibration data is generated.

To determine the camera distortion described above, in at least one exemplary embodiment, physical reference marks are used, and their orientation/shape without distortion is known. These markers are captured by the camera and the camera distortion described above can be determined by comparing their orientation/shape in the captured image to their undistorted orientation/shape. A natural mark is the border (boundary) itself, which is known to be a fixed orientation and shape (usually a rectangle without distortion in the real world). The frame is also a reference for the completion of the calibration operation. In other words, the corrected image should be linear with respect to the frame. Therefore, when correcting geometric distortion, the image captured by the camera should include the viewing screen boundaries (ie, the bezel 18).

In another exemplary embodiment in which the boundary is undetectable, the sensor in the camera is used to sense the signal from the transmitter above the screen to determine the camera relative to the viewing surface 16. distortion. The measurements of these achievements will produce a map of the viewing surface 16 as seen by the camera.

When correcting lateral chromatic aberration, the camera will take a K group of images, where K is the number of color components, for example, the three primary colors RGB. At least some of the test patterns in Figure 4 will be repeated for each color component.

The brightness and color (brightness and chrominance) corrections are complete, regardless of the geometric correction correlation. In some projection systems, such brightness and color correction are done after correction of geometric distortion. In a flat panel display device in which geometric distortion is not present, brightness and color correction are directly accomplished. In an exemplary embodiment, a sensing device, such as a color analyzer, is placed directly at or near the viewing surface 16 for capturing color information. In this case, the correction of the sensing device positioning described above is not necessary. The sensing device 11 may capture information of the entire image or a particular point. In the latter case, data from the grid lines above the screen needs to be taken. If the sensing device 11 is in a trapezoidal position relative to the viewing surface 16, it is similar to the camera above, and its correction due to positioning needs to be completed.

For some display devices with geometric distortion, brightness and color correction should be done after the geometric correction has been completed. This means that the display device is first corrected for geometric distortion including color dependence. The geometrically corrected color-related correction allows for any additional color distortion introduced by the geometric correction described above to be taken into account, and indeed only the region containing the last image (i.e., no background) is corrected.

In this exemplary embodiment, the calibration data generator 12 can analyze the images and capture calibration data in a format used by the warpage generator 13, which can then provide warpage data to the calibration data. Digital warping unit 15.

The digital warping operation can generally be described as applying a pre-compensation map to perform a mathematical transformation between the input image coordinates and the output image coordinates in accordance with equation (1).

In equation (1), i covers the input pixel coordinates, ( u _i , v _i ) gives the spatial coordinates of the input pixel, Given the color of the input pixel, ( x _i , y _i ) gives the spatial coordinates of the output pixel mapped to the output space, and The corresponding pixel output color is given. In the case of a three primary color system, Only one RGB value. Equation (1) is the expression of the above correction, and is in the form of a ruled line. It is difficult for a processor to directly use a grid format, where the correction must be applied in an instant manner, such as a 60 Hz frame rate for video. Therefore, the warp generator can convert equation (1) into a more hardware efficient format. The calibration data generator 12 is composed of three sub-generators for correcting geometric conditions, lateral colors, and color non-uniformities, respectively.

In the following, the calibration data of the above corrected geometric conditions will be discussed first. In the examples listed below, the preliminary test patterns analyzed are those having a ruled line style, such as those shown in panels (a) and (b) in FIG. When the intersection of the lines/lines provides a ruled line, the styles in panels (e) through (g) in Figure 4 can also be used.

These test images, which are similar to the grid style, provide a set of shapes centered on known locations within the input space described above. These centres can be specified as , where i covers the range of such shapes. There is a total of M × N shapes starting from the upper left corner and proceeding along the above test pattern, and W _T × H _T is the resolution of the test pattern. The test pattern resolution does not need to match the native resolution of the display device. When displayed, the center of the shape in the above test pattern will be transformed into some other The value indicated. These shapes will also be distorted, that is, a circle will be distorted into an ellipse, and so on. The coordinate systems define the display space at the origin relative to the upper left corner of the bezel 18 of the viewing surface 16 described above. Let W _D × H _D indicate the resolution of the display device (in the bezel 18) in an arbitrary measurement unit, and the coordinates Also within these same measurement units. This display space is equivalent to the real world or viewer space. In other words, the corrected image is likely to appear undistorted in the display space.

The camera can take an image of the above-described distorted grid pattern and transmit it to the calibration data generator 12. The resolution of the camera is indicated as W _C × H _C . In the embodiments recited in this specification, the camera resolution does not have to match the display, and in addition, the camera can be placed anywhere. The coordinates of the center of the camera space are The origin is defined as the upper left corner of the above-mentioned captured image.

The images taken are from the observation points of the above cameras, and the calibration operation is necessary to make observations from the observer in the above-mentioned real world observation points. Therefore, the calibration procedure is necessary to subtract the observation point of the above camera, which is also called the camera distortion. As discussed above, in an exemplary embodiment, this uses the viewing surface bezel 18 as a marker upon completion. Therefore, the camera image should also capture the viewing surface bezel 18. In the real world, the viewing surface border 18 is defined by coordinates: upper left corner: (0,0) upper right corner: ( W _D , 0) lower left corner: (0, H _D ) lower right corner: ( W _D , H _D ) (2)

In the camera image, the coordinates become: upper left corner: Top right corner: Lower left corner: Bottom right corner: (3)

Figure 5 illustrates various spatial and coordinate systems. Although the images are shown as black circles on a white background, all test styles can be colored and other shapes or topography used (see, for example, Figure 4). The three conditions displayed in the display and camera space correspond to: case (a) when the image overflows to completely cover the viewing surface bezel 18, condition (b) when the image is fully fit into the viewing surface When the bezel 18 is under or underflow, and condition (c) is an intermediate condition or mismatch, wherein the image is not completely within the viewing surface bezel 18. These conditions are referred to as projection geometry categories. It should be noted that when the input and camera space are defined in pixels, the display space may be pixels, millimeters, or some other unit.

The display distortion indicated by f _D above can be described as a map generated by equation (4) in a functional manner.

This means the correction ( ) is the inverse function generated in Equation 4, which is listed in Equation 5.

The digital warping unit 15 will apply the above correction to the input image So that it is warped (pre-distorted) before being displayed.

The above two maps are defined in the forward direction: the function domain is the input image, and the range is the output image. As you can see, an electronic correction circuit is more efficient and correct, using an inverse function architecture to generate images. In an anti-warping architecture, the output image of the circuit is generated by mapping the pixels in the output to the input via the correction map, and then performing color filtering (ie, assigning) in the input space. Color value). This also means that the correction map is expressed in the form of an inverse function, which will be denoted as f _W . Due to the correction in the inverse function form, the display distortion map itself ( ). The map or warpage data required by an inverse function architecture correction unit is only the display distortion map. Therefore, the above-described ruled line data to be generated by the calibration data generator 12 is defined in the equation (6).

It should be noted that the terms "grid line" and map are often used interchangeably. This information is taken from the images captured by the above cameras and they are located in the camera space. The captured images correspond to the maps defined in equation (7).

This will be referred to as a map of the complete image map, which can be considered as a combination of the display distortion map f _D and the camera distortion map f _C , the subtraction of which can produce the necessary person defined in equation (8) f _W.

The subtraction of f _C from f _D is only the link of the two maps (or function compound). In addition, when the display coordinate system scale and origin may not be applicable, the coordinates It is necessary to achieve the correct pixel scale and origin as described above. This will be discussed in more detail below.

An exemplary embodiment of the calibration data generator 12 described above is shown in FIG. A test style W _C × H _C camera image is first analyzed to capture the center of the shape This will produce f _F . The center of the shape in the camera space is the corresponding position of the shape center in the input space after being mapped by the display and the camera. For some image areas that overflow the viewing surface 16, such shapes will be unavailable. Such outward shapes, in a rear projection television, or in the case of a front projection system, will generally be invisible as they will lie in a background on a possibly different plane. Therefore, only the shape defined in the viewing surface 16 as EFGH (see Figure 5) will be analyzed.

These shape centers can be found using a variety of image processing algorithms. One method involves using a threshold mechanism to transform a captured image into a binary (black and white) image. The shape in the binary image allows the pixels to be identified and marked. The centroid of each group of classified pixels will then approximate the center of the shapes. The threshold can be automatically determined by analyzing the histogram of the image. The histogram may be the brightness or specific hue of the image taken above.

The images taken are also analyzed to capture the coordinates and boundaries of the viewing surface. This step may use a different image. To determine the camera distortion f _C , the border coordinates are required. If the camera is not optically distorted, the camera distortion will be marked as The perspective distortion, and the decision to f _C , is only necessary for the coordinate system defined in equations (3) of the four corners. If the camera is also optically distorted, additional marking is necessary. The border boundary EFGH, which may provide sufficient markers, can be parameterized by the line equation of its edges. The edge equation can also be used to determine the four corners and to determine which shape is within the viewing surface 16. a solid rectangular grid with known coordinates, such as in the display space Can also be attached to or projected onto the viewing surface 16 to provide additional indicia that would be imaged in the camera space as . This grid line can be considered as the camera calibration (CC) grid. The decision of the border coordinates and boundaries is also referred to as display characterization.

From the point of view of the sensing device, the live distortion of the camera lens and a curved screen is indistinguishable. In both cases, the markers and borders are imaged as curved. Therefore, a curved screen can also be addressed within a framework of camera distortion and a combined CC grid. Correcting the camera distortion will also ensure that the final image matches the curved border. For curved screen correction, the CC grid lines can be constructed by attaching markers to the frame 18 at regular distances (as measured on the screen); they can then be interpolated into the interior of the frame 18. They may also be attached to the interior of the bezel 18. It should be noted that although the screen is curved, it is a two-dimensional surface, so it can be calibrated via the above two-dimensional CC grid.

The edges (border 18 or additional CC grid lines) or markers can be detected using standard image processing methods such as edge detection. Knowing the position of the edges, a line equation can fit to the edge, and the intersection of the lines provides four corner and CC grid coordinates. The edge and CC grid coordinates can be defined as shown in equation (9), where _Ncc is the number of points in the camera calibration grid.

( l _Tx ( t ), l _Ty ( t )) → upper edge ( l _Rx ( t ), l _Ry ( t )) → right edge ( l _Bx ( t ), l _By ( t )) → lower edge ( l _Lx ( t ), l _Ly ( t )) → left edge , i =1... N _CC → Camera calibration grid (9)

For some display devices (such as those with curved screens), a CC grid from the physical mark may not be immediately available. In this case, the edge equations can be used to mathematically establish the CC grid. The degree of freedom that exists therein is how the points are arranged along the edges and how they are interpolated into the interior of the bezel 18. Regardless of the method chosen, the last image will match the border 18, provided that the domain coordinates (see discussion of sorting) are properly selected. One arrangement is to arrange the points equidistantly along the edges, which in turn can be linearly interpolated into their interior.

If the manufacturer provides the camera is marked as In terms of optical distortion specifications, these specifications can be combined with the above-described perspective distortion to replace or generate the camera calibration grid, which is shown in equation (10).

The camera's distorted optical component can be determined prior to calibration of the display because it is independent of camera position and orientation. The data in equations (3) and (9) will be collectively referred to as camera calibration data.

Once the coordinates have been captured, they need to be arranged in the correct order described above. Mathematically, sorting will be for each range coordinate Assign its corresponding domain coordinates . In order to establish the above complete image map f _F , the domain coordinates must be determined. The above extraction procedure does not provide any information about the coordinates of the domains. The centers will not necessarily be determined in an order that matches the ordering of the shapes in the input test style described above.

These test patterns, similar to those shown in panels (c) and (d) of Figure 4, can be used to sort the points. Some of the images captured from these test styles can be classified according to the barcodes to which they belong. These shape centers can also be arranged in this category. The horizontal and vertical bar codes to which the centers belong, for example ( r , s ), will determine the above-mentioned domain coordinates. Where I is defined in equation (11).

i =( r -1) N + s (11)

When sorting, it is important to determine which bar code and shape are within the viewing surface bezel 18. If the background area (the exterior of the viewing surface bezel 18) does not provide an image with a high contrast, then an appropriate threshold (in the step of capturing the shape coordinates) alone will ensure that only the viewing surface bezel is provided The shape and bar code within 18 are measured. If the outer shapes are also strongly imaged, the shape and bar code can be determined internally by comparison with the edge of the borders. The number of such bar codes is necessary to take into account any missing bar codes (the outside of the frame 18). A given number of bar codes can be flashed at a time to determine whether they are inside or outside the border. Bar codes of different colors can also be used to implicitly number them.

The camera calibration data also needs to be ordered, wherein the domain coordinates are within the display space. Here, however, the procedure is relatively simple, as all topographies (by definition) are located within the bezel 18. In most cases, the coordinate comparison is sufficient to determine the above ordering. In the case of the CC grid, the ranking will assign the above-mentioned coordinate network. These are the domain coordinates (in the display space) associated with the CC grid lines referred to above as the domain CC grid lines. The value of the CC grid line in the field will depend on whether the grid corresponds to the entity marker or whether it is mathematically established. In the former case, the known coordinates of the markers can produce the CC grid line of the domain. In the latter case, there is some degree of freedom in selecting the CC grid line of the domain. If the last image matches the above-mentioned frame 18 (i.e., geometric category (a)), the CC grid points above the edges are necessarily mapped to the corresponding edges above the rectangular EFGH. This means that the edges need to be mapped as follows: top edge Straight edge of the line passing {(0,0),( W _D ,0)} Straight edge of the line passing {( W _D ,0),( W _D , H _D )} The straight left edge of {(0, H _D ), ( W _D , H _D )} a line passing {(0,0),(0, H _D )}

In addition to these restrictions, these domain CC grid points can be selected in any reasonable manner. When the capture and sorting is completed, the map f _W can be found using equation (8).

The camera calibration data is used to first establish the inverse function camera distortion map f _C ^-1 . The most common reality of a pure perspective camera distortion (ie, ), only four corner points are needed.

This (reverse) perspective transformation is produced by Equation 13.

Here, ( x _d , y _d ) is the coordinates in the above display space, and ( x _c , y _c ) is the coordinates in the camera space. Using equation (12), eight linear equations are obtained, which can be used to find solutions for the coefficients { a , b , c , d , e , f , g , h } of the perspective transformation described above.

When the camera distortion includes an optical distortion component The edge equation or CC grid is used to determine the inverse function camera distortion map f _C ^-1 when it is or will be corrected for a curved border. One way is to use the CC grid because it provides information about the distortion at the internal point, not just the edge. The CC grid series is given in equation (10). The ruled line can be either interpolated (in the sense of least squares) or interpolated by a set of defined basis functions. One option is to use a spline substrate to obtain spline fit or interpolation for a ruled line as defined by equation (14).

, the matching or interpolation of the grid (14)

f _C ^-1 and the coordinates calculated during the step of capturing the camera calibration data described above , the map f _W is obtained by the link as follows among them, It is generated by equation (15).

The link can evaluate the camera anti-aliasing map using the full image mapping range for its domain.

The above obtained ruled line , corresponding to the middle diagram in Figure 5, and the map (in the form of an inverse function) needed to correct the distortion of the display. As mentioned above, the grid lines only contain their points within the viewing surface bezel 18 described above. In terms of the distortion of the overflow (cases (a) and (b)), a plurality of pixels (corresponding to the center of the shape) of the above-mentioned domain space (that is, an input image from the viewpoint of display distortion), in the grid line The defined display space does not have their coordinates. The electronic correction unit, which is the above-described digital warping unit 15 in this exemplary embodiment, will process all of the domain space pixels; the domain space associated with an inverse function architecture correction unit is actually the resulting output image. Therefore, the missing grid line data needs to be calculated, which is done by interpolation and re-sampling steps.

As in the calculation of camera distortion described above, the ruled line f _W can be either interpolated (in the sense of least squares) or interpolated by a basis function group of similar splines. The fit or interpolation Can be interpolated to determine the missing data. This function can also be used to resample at a higher rate, that is, to increase the domain points from M × N to ( nM - n +1) × ( nN - n +1), n = 2, 3, ..., making the correction grid more dense.

This calibration map can now be adopted as And the corrected ruled lines obtained by evaluating the function at any of the arrays of points above the input space, including the points at which they are missing. To maintain the original grid , The relevant interpolation form is used to define a grid array of new regular intervals as shown in Equation 16 above the input space.

{( x _i , y _i )}, i =1... × , the system contains the array (16)

The array can be denser and has > M row and > N lines. Evaluate the top of this array According to Equation 17, the corrected grid lines ( x _di , y _di ) can be generated as described above, including the points at which they are missing, and may be denser.

:( x _i , y _i ) → ( x _di , y _di ) If ( x _i , y _i )= inside the display frame with (17)

The combination of their cooperation and interpolation can also Used in order to provide interpolation related to missing data interpretation and internal data.

The final stage in the generation of the above calibration data is to fix the scale and origin. The calibration grid is within the display space and is defined above the upper right corner of the viewing surface bezel 18. The units (scales) of the above display space are arbitrary and may be different from those used in the input space. Before the data can be used by the warpage generator 13, the origins and scales need to be consistent with the input space. This can be considered as an optimization of the origin and scale.

Considering the middle diagram of Figure 5, when the correction is applied, the last corrected image should be rectangular relative to the viewing surface bezel 18. Referring to Fig. 7, such a rectangle containing the corrected image will be referred to as an active rectangle A'B'C'D'. This active rectangle is necessarily located within the optical envelope of the above image (ABCD) and within the viewing surface border (EFGH). The origin and scale need to be selected such that the upper left corner of the active rectangle corresponds to (0, 0), and the width of the rectangle multiplied by the height is W _T × H _T , which is the pixel of the input image. Resolution (see Figure 7).

It should be noted that the input space related to the above correction is actually an output image related to the electronic correction in an inverse function architecture, and once the calibration and shift have been completed, the input image related to the above correction is actually The display is spatially equivalent (i.e., corrects the associated output space).

If the upper left corner and the scale of the active rectangle are in the coordinate space of the display, respectively with Given, all grid coordinates need to be scaled and shifted as shown in equation (18).

The above may determine the value of W _D × H _{D of} the rectangular coordinate value, which may be selected as any integer value as long as they can maintain the aspect ratio of the viewing surface frame 18 described above. Using Equation (18), the display spatial scale (bottom map) can be transformed into the input image scale required for the correction (top graph) in Figure 7.

There is a degree of freedom in the determination of the active rectangle; however, adding a certain natural constraint can simplify the selection. To maximize the pixel resolution of the corrected image, the rectangle should be chosen to be as large as possible. If the corrected image is to have the same aspect ratio as the input image, the rectangle The aspect ratio should match the input image ( W _T / H _T ).

The various restrictions C1 to C4 series are as follows.

C1) The active rectangle is contained in the optical envelope ABCD.

C2) The active rectangle is included in the viewing surface border EFGH.

C3) The active rectangular area is maximized.

C4) The active rectangle aspect ratio is set to be equal to the input image ( ).

Solve the above active rectangle (ie, decide with These restrictions have become an issue in numerical optimization. All of the above limitations can be placed in a mathematical form that allows for the use of various optimization methods to solve the top problem.

One possible approach is to minimize the use of restrictions. This involves rewriting the constraints in the equation or inequality form and defining a function to be minimized (or maximized). The line equations for the edge of the border (see equation (9)) and the outermost grid point (see equation (17)) can be used to formulate their constraints C1 and C2 into the form of inequalities, ie Four rectangular corners are located within (<=) of the lines. The constraint C4 has been in the form of an equation, and the constraint C3 becomes a function of the above-mentioned maximization, that is, the region of the above-described active rectangle is maximized.

In the case of live (a) of Figure 5, wherein the image overflows to fill the viewing surface 16, the viewing surface bezel 18 provides a natural rectangle that automatically satisfies the constraints C1 through C3. By fixing the scale of the display to the test image, their parameters can be set according to equation (19).

The corrected image will be perfectly matched to the viewing surface bezel 18, which is ideal for viewing the entire viewing surface bezel 18. Therefore, in this case, the optimization step in Fig. 6 simply means that equation (19) is used, that is, the points do not need to be scaled or shifted.

This optimization step can also be used to achieve a change in the aspect ratio by modifying the constraint 4 as shown in equation (20).

Continuing with equation (18), the aspect ratio of the above corrected image will become α . The degree of freedom in the choice of such aspect ratio allows the image to be included in a letter-boxed or pillar-boxed image having different aspect ratios in a single display device. By adjusting the scale and shift, the image can also be easily overscanned (i.e., image overflow) and underscan (i.e., image underscan) on the viewing surface 16. Therefore, the use of the surface function is advantageous for the easy implementation of overscan and underscan.

The last calibration data generated by the calibration data generator 12 is the grid data given by equation (21). .

The above discussion focuses on the distortion in the case where the corrections for all primary colors are the same. In these cases, the same grid data indicates all color related corrections, which may be referred to as a single color correction. However, in terms of lateral color distortion, the ruled line data differs for all primary colors, and multiple color corrections are required, which may be referred to as multi-color correction. Any geometric distortion common to all of the primary colors can be included in the lateral correction; therefore, the previous implementation of the calibration data generator 12 described above can be considered a special case of the multiple color correction described below.

An exemplary implementation of the above-described calibration data generator 12 for lateral color correction is shown in FIG. As can be seen, this is similar to the implementation of the above-mentioned single color correction (see the previous section), which is repeated for K times, where K is the number of primary colors. These primary colors are labeled I _i , i =1... K . For the most common three primary colors RGB, ( I ₁ , I ₂ , I ₃ ) = ( R , G , B ).

The steps and details associated with calibrating each primary color are the same as described above for a single color correction, with the following modifications.

These test styles are now colored based on the primary colors being calibrated as described above. For example, when calibrating a red color, all test styles (see Figure 4, panels (a) through (j)) will have their appearance (circles, bars, etc.). These topographical features (number of circles, etc.) can vary depending on the color style above.

All image processing steps, such as capturing their centers and edges, will use color images. The threshold can be adjusted to handle the color being calibrated as described above. Once a binary image is obtained, the image processing is independent of the color.

Usually, due to the lateral color distortion in the camera lens itself, the camera calibration data is different for different primary colors, and it is necessary to calculate all the primary colors separately. The system can be configured to correct for lateral color distortion within the camera itself. Test image patterns from different primary colors, similar to those used to calibrate the display device, can be used to generate the camera calibration data. The (multi-color) calibration data for the above camera can be completed independently of the display calibration and only needs to be done once. In generating the camera calibration data, a display device having zero or very small (i.e., much less than the camera) lateral color distortion should be used. If such a display device is not available, some additive markings can be used to provide a solid grid with known coordinates. The final result associated with the multi-color camera calibration described above is an inverse function camera distortion that depends on the primary colors defined by equation (22) above.

, k =1... K , the fit and interpolation of the grid lines (twenty two)

The K-derived ruled lines (similar to equation (17)) are defined in equation (23) after any missing data has been calculated.

Here, the number of points per grid may vary depending on the test style used and any resampling done.

The test patterns associated with these primary colors may be subject to different projection geometry categories (see Figure 5). Some of the test patterns associated with the primary colors may completely overflow the viewing surface bezel 18 as in panel (a) of Figure 5, while others may be completely located in panel (b) of Figure 5 Inside the border. When the optimization is performed, the above-mentioned active rectangles are necessarily located in the viewing surface frame 16 and in the image envelope ABCD _k associated with each color; the spatial intersection of the image envelopes will be used. . What this means is that a single optimization is done, and constraint 1 takes into account the envelope ABCD _{k of} all primary colors. This optimization determines the coordinates associated with the active rectangles common to all primary colors. These coordinates are then used to calibrate and shift the grid lines according to equation (18).

The output of the optimization step is a grid line, which can produce calibration data relating to all primary colors as indicated in equation (24).

These data sets are used by the warp generator 13.

In the exemplary embodiment, the color and brightness, or simply the color non-uniformity calibration data generation, is performed after the geometric distortions (types 1 through 4) have been corrected. Color inhomogeneities may be caused by several sources, such as changes in the path length to the viewing surface 16 due to projection geometry (trapezoidal angle), flaws in the microdisplay panel, and the like.

In the case of a geometrically corrected display device, the test pattern image will appear within the bezel 18 as a rectangle (i.e., an active rectangle) that may match the scale. The origin adopts the upper left corner of the above-mentioned movable rectangle instead of the upper left corner of the above-mentioned viewing surface frame 18. The test patterns used are only the additive versions of the user of the single color geometry described above; that is, in terms of correcting the primary color K, the topography (circles, lines) will be colored. This is the same as that used to correct horizontal colors. In terms of brightness, gray values (maximum white, half maximum) can be used. The term "color" is generally used to identify any color component being corrected; it may be a component of luminance, RGB or YC _b C _r , or one in any other color space that can be measured by the sensing device 11 The weight of the.

The sensing device 11, which may be a camera or a color analyzer (ie, a spectrometer, a photometer, etc.). For greater accuracy, a photometer or spectrometer should be used. These color analyzers may capture the entire image (ie, multiple points) or data at a single point. The sensing device 11 is arranged such that it is as close as possible to the viewing surface 16. Their single point color analyzers will actually be placed at the known coordinates (ie, the center of the shapes) on the screen to obtain information about the coordinates. Although the multi-point color analyzer and camera can be placed at an arbitrary position, the accuracy of the boost is obtained by arranging them closer to the viewing surface 16 and as close as possible to the center. FIG. 9 illustrates an exemplary apparatus including a viewing surface 91, a single point color analyzer 92, and a multi-point color analyzer 93. The above-mentioned calibration data generator related to color unevenness is similar to that of correcting geometric distortion. Figure 10 is an exemplary embodiment of a calibration data generator 12' that illustrates color non-uniformity.

The data taken by the single-point color analyzer 92 described above includes the primary color values at all points to be measured. And corresponding spatial coordinates . Here, k =1... K identifies the color being analyzed. Above The original color values indicated are also known because their test patterns are clearly defined. This produces Equation (25), which is the above-described ruled line data for explaining color unevenness distortion, and is called a color distortion map.

It should be noted that these spatial coordinates are not subject to this color unevenness distortion. The original color value For a given test style, it will usually be a fixed value. This means that all non-background pixels are of the same color. There may be more than one set of measurements s = 1 ... S may be completed, where each set corresponds to a test pattern having a different fixed color value, such as a different level of saturation or gray scale. To simplify its notation, this single indicator will also cover different measurement sets across the equations shown in equation (26).

These spatial coordinates are the same for each group. The discussion below applies to each group (ie, test style).

In the case of the multi-point color analyzer 93 which can be a camera as described above, the above-mentioned photographed data corresponds to the entire image. In this case, some image processing systems need to be completed before the grid is obtained. The center of these shapes And their domain coordinates Will be calculated. This step is completed in the same way as the capture and sort steps used during geometry correction. In addition to these centers, the color values at the center of the above shape are also calculated. Correcting the color value can be obtained by averaging or filtering the pixel color values in the identified central neighbors in the captured image according to equation (27).

a _j = filter coefficient Neighbourhood (27)

Where C ' _kj is the color value in the captured image in the neighborhood of the above center. For the four points closest to the averaging, the filter coefficients are a _j = 1/4, j =1...4.

The final result is the ruled line data defined in equation (25). It should be noted that: (i) only the domain coordinates are needed because the color distortion does not change the spatial coordinates; (ii) there is no missing data because the image is not geometrically distorted, and Within the viewing surface 16; and (iii) there is no need to calculate the sensing device distortion and perform the chaining because there is no geometric correction done.

Depending on the type of sensing device used and the format of the captured material, a color space transformation may be required to direct the color data to the color space of the display. For example, a spectrometer may produce data based on chroma values, and the display device and the electronic correction unit (which is a processor) require RGB values. A color transformation may be performed by a matrix multiplication or by a more complex nonlinear equation. For a color space conversion, the grid data for all primary colors will be used. Typically, such a transformation takes the form shown in equation (28).

If no color distortion occurs, then for a fixed color test style, all coordinates The color value of the place should be measured as a constant . The measured constant may not be equal to the original fixed pixel value . For most displays, these measurements are proportional to the original values, where the proportional constant λ is fixed in the absence of color distortion and spatially in the presence of color distortion. Therefore, the color distortion map of the display can be represented as shown in equation (29).

Usually, the color value of the input and measurement will be The given known display color function f _{I is} associated with a parameter vector as shown in equation (30).

If there is color distortion, λ will have a spatial change. a predetermined coordinate The parameters at which it can be determined by analyzing the data of the different groups s =1 ... S as shown in equation (31), wherein the s indicator is clearly shown.

What is needed at each coordinate is a sufficient number of values. This analysis may approximate f _I by doing a match with the data. Similarly, its inverse function It can be calculated by analyzing the same data in the opposite direction as shown in equation (32).

The inverse function also depends on certain parameters called color correction parameters. It can be determined by the explicit form of the known f _I , or can be calculated from a combination of the inverse function data using a specific basis function such as a polynomial function. For a linear least squares fit, the inverse function map is in the form shown in equation (33).

Here, r =1... R gives the number of parameters used to define the inverse function color map, and B _r is the basis function. These parameters differ for each central coordinate space and for each primary color. usually, It will be determined by the expression used by the above-described electronic correction unit, which can be assumed to be in the form of a polynomial without loss of generality. The above expression also allows adjustment of the last fixed chromaticity, since in some cases it may be necessary or desirable to reduce the original at the output. value. Here, the parameters can be adjusted by a simple scaling factor to increase or decrease the value of the above inversion.

Once the inverse function (at each center coordinate) is known, the above corrected color map that corrects for the color unevenness distortion is given by equation (34).

The spatial variation and correction of the color distortion are respectively determined by the parameter And its inverse function Explain completely. Therefore, the above (base) calibration data, which is labeled as f _Wck for correction, can fully describe the above-described grid data associated with the color correction parameters according to equation (35).

In the most common case of equation (29), the parameters are generated according to equation (36).

The above-described ruled lines may be made denser by resampling with a suitable fit or interpolation function. The above-described new ruled line using a symbol similar to the geometric calibrator is given by equation (37).

This is the data output of the above calibration data generator 12'.

The complete data output of the calibration data generator 12' including all of the sub-generators (i.e., each of the rows in FIG. 10) is given by equation (38).

If there is no horizontal color, then the K grid lines The system is the same, that is, only one geometric correction grid is calculated and output. This calibration data is input to the warpage generator 13.

As mentioned earlier, this grid data is not directly used by the electronic calibration unit. Although a grid representation is the most common format, it is inefficient for a hardware implementation, mainly because it needs to store a large amount of data (coordinates related to each pixel), and cannot be easily Manipulation (such as in terms of scale changes). Some pre-existing technology systems that use a checklist are not optimal for the same reason. The warpage generator 13 can convert the ruled line definition defined in (38) into the warpage data, which is a type of expression of the above correction, and the form is in the application of the hardware. It is efficient. If the electronic correction unit can directly use the ruled line data, the above-described ruled line for re-sampling all pixels can be used, and it is no longer necessary to use the warp generator 13 to generate warpage data.

The warpage data is generated in accordance with the data requirements of the electronic correction unit described above. An electronic correction unit that applies geometry and color transformations using a variety of architectures. Most cell systems use geometric correction related inverse function maps, and the above-described grid lines are designed for an inverse function architecture. An efficient electronic correction architecture, such as the US Patent Application No. US 2006-0050074 A1, entitled "System and method for representing a general two dimensional transformation" (system and method for representing a general two-dimensional transformation) The presenter is based on the linear function expression of the above-mentioned grid data. The warpage generator 13 converts the ruled line data into a function expression. Fig. 11 illustrates an exemplary embodiment of the above warpage generator 13.

A general function representation of a two-dimensional grid line ( x _i , y _i ) → u _i can be written as shown in equation (39).

Equation (39) defines a two-dimensional surface function across the domain ( x , y ). It is a linear combination of the above-mentioned basis functions B _i ( x , y ), i =1... L , and the coefficient of this combination, called the surface coefficient, is given by a _i . These coefficients are constant and do not change across the domain. These basis functions do not have to be linear; only the combinations of them are linear. In at least some instances, the basis functions may be extremely non-linear; therefore, the form shown in equation (39) is sufficiently general to represent the correction grid. The number of such basis functions and their number is defined by the electronic correction unit. Because they are implemented and evaluated in hardware. The warpage generator 13 determines the necessary coefficients.

In an exemplary embodiment, the basis functions used in the hardware are of the polynomial type. Importing two indicators, a polynomial basis function and the corresponding surface, may be written as shown in equation (40).

Since these basis functions are known, the new data to be determined and stored above is the set of surface coefficients a _i . Moving a surface representation is a transformation from the ruled line value to the surface coefficient as shown in equation (41).

The benefit of the above expression is derived from the fact that in the case where a grid line value needs to be stored for each pixel, the surface coefficients can be allowed to span the group of pixels to calculate the grid line values; Thus, the surface coefficients that need to be stored are considerably smaller in number.

The number of such coefficients determines how accurately the original grid values can be represented. By increasing the number of coefficients, that is, by using more basis functions, the added accuracy can be obtained. Alternatively, if the field is distinguished as small pieces, a smaller number of basis functions can be used, with each slice using a different surface function. The die structure is established based on the stringency of display distortion within each tile. Such a solution allows for greater flexibility in matching the complexity of the bonded surface to the distortion. For example, the more complex a distortion, the more small pieces are used. The coefficient of the small piece p =1... P is marked as . Among the following, without loss of generality, a polynomial form of the token will be used, which can be easily adapted to another substrate. The entire surface will take the form indicated in equation (42).

i =0... L _x , j =0... L _y p =1... P ( x , y ) Small piece p (42)

A single surface system corresponds to a single patch that is equal to the entire output image (domain) described above. An exemplary patch is shown in Figure 12.

The tile distinction can be initialized to a certain starting structure, such as 16 patches arranged in a 4x4 symmetric arrangement. The arrangement of the patches (i.e., the number of patches and the boundaries of each patch) is referred to as the patch geometry D, which is in the form specified in equation (43).

, p =1... P small piece (43)

Given a piece of geometric condition, the coefficients can be calculated using the linear least squares fit of the above data according to equation (38). This fit should be limited to ensure that the surface is continuous at the boundary of the die. Once the surface is determined, an error analysis is performed and, as shown in equation (44), the coordinate network values are compared to the calculated values.

Errpr _i =| u _i - u ( x _i , y _i )| (44)

The error value is compared to an allowable level E _max . If the maximum error is less than or equal to the tolerance level, that is, The surface coefficients are retained and output from the warpage generator 13 as the warpage data described above. If the maximum error is large, the small piece geometry will be further subdivided and refined, and the coefficients will be recalculated and the error analysis repeated.

The surface representation in equation (38) can be written as shown in equation (45).

It should be noted that the ( i , j ) indicator in the above-mentioned grid representation is no longer needed because the function form is defined for the entire space and not just for a discrete coordinate group. The indicators ( i , j ) can now be given the indices or, more generally, the basis functions described above. The indicator k identifies the primary colors, and the indicator p identifies the aforementioned patches. The surface is evaluated for the patch in which the domain coordinates are located. The number of patch arrangements and basis functions may vary with respect to the primary colors. Additional variations of the above format can be obtained, for example, by changing the basis function of each tile. The domain space associated with the above geometric correction has been marked as ( x , y ), and its corresponding to the output image space (in an inverse function architecture), and the range space has been relabeled as ( u , v ) And its corresponding to the input image space.

For this color correction, the domain space has been relabeled as ( u , v ). This color correction operates for a geometrically correct image. This means that the color correction potential must be applied before the input image having the coordinate space ( u , v ) has been warped for geometric correction. If the electronic correction unit applies the color correction before the input image has been warped for geometric correction, then the above coefficients need to be adjusted for the new order corrected by the application, that is, a need is needed. The steps to reorder. In this case, the color parameters are defined in the ( x , y ) space. First, you can get a new grid It is defined in the space ( x , y ) from the above surface as shown in equation (46).

The grid lines can then be matched as mentioned above, and the coefficients are calculated, which is now the output image space described above. These color correction surface coefficients use the same mark. This error analysis will now use the above-mentioned reordered grid lines.

The final output of the warpage generator 13 is the set of coefficients in equation (47) (ordered to adjust if necessary), which collectively form the above warpage data.

Project D ^k system contains all the information defining the geometry of a small piece of the related primary color k. The ( a , b ) data is the geometric warpage data or transformation of the above calibratable types 1 to 4, and The color warping or transformation of the above-described correctable type 5 distortion.

The digital warp unit 15 is a processor and an electronic correction unit that functions as the system. The term "electronic correction unit" is used interchangeably with the term "digital warpage unit". In actual use, the digital warping unit 15 can apply warpage data to the digital input image (video) to predistort or warp the input images. These input images are warped in both spatial and color spaces. The spatial warp is based on the geometric warpage, and the color warpage is completed, depending on the color warp. The pre-distortion can be established to eliminate distortion of the display to produce an undistorted image displayed on the viewing surface 16.

An exemplary embodiment of the digital warping unit 15 of the above-described correctable geometry and color non-uniformity is shown in Figure 13 (the index is omitted in this figure). The digital warpage unit 15 includes two main blocks: a first block to which the geometric warpage is applied (ie, geometrically warping the input images), and one can be made only for color unevenness Correcting and warping the second warped block of the input image in the color space. Here, the color correction occurs after the geometric correction, however, this discussion can be easily adopted for the reverse order. When a particular correction is not needed, both blocks can be bypassed. Each block complex has two components: a surface evaluation component that evaluates the surface defined for each primary color ( x _i , y _i ) for each primary color (this indicator is omitted) in equation (45). a polynomial by which the necessary coordinates { u _i , v _i , And a pixel-generating component that actually uses the necessary coordinates to calculate the pixel color value C _i . For the geometric correction, the pixel generation is a filtering step in which a filter having a pre-computed coefficient designated w _j , j =1... W is applied to the current pixel being processed as described above. ( u _i , v _i ) some neighbors of pixels around.

In at least some instances, the filter coefficients are calculated external to the system and loaded into the digital warp unit 15. For this color unevenness correction, the pixel produces a pixel value that will take the image from the geometric warp described above, and applies equation (33) to determine its new color value. The pixel generation step is summarized in equation (48) , Γ = ( u _i , v _i ) neighborhood , r =1... R (48)

These steps are performed for each primary color. The It is the intermediate color value after geometric correction.

The details of these filtering and color correction equations depend on the architecture of the above hardware. A simple filter may simply average four nearest neighbors, in which case w _j = 1/4. A complex filter, possibly using an elliptical neighbor, whose shape depends on the local Jacobian of the surface, and the filter coefficients can be obtained using a complex filter to generate the algorithm. In this case, the neighbor coordinates There may be a need to be used to evaluate Jacobian. For the same reason, a simple color correction involves using only one linear correction as defined in equation (49).

Alternatively, a complex color correction may be used that uses a cubic polynomial as defined in equation (50).

The color parameters ( The surface coefficient is calculated and the architectural details of the digital warpage unit 15 are known.

The final result of the digital warp unit 15 is mathematically corrected by using a vector symbol used to indicate all primary color components to be overwritten by equation (1) in equation (51) below.

The warped or pre-compensated output image is input to the display device (not shown), wherein it is projected onto the viewing surface 16 without visual distortion, thereby completing the automated calibration and correction described above. Once the calibration and calibration procedure is complete, normal (no test style) images and video can be transmitted to the display device.

This multi-color geometric calibration and correction has been discussed in conjunction with lateral color correction. However, it can be used to calibrate and correct any distortion in which the primary color components have geometric distortion. Other applications include: distortion due to under-alignment; and multiple microdisplay devices having optical components that are arranged opposite each other or in a rear projection display device relative to the chassis or housing plus different magnifications for the color components The resulting under convergence.

In the projection system, this color calibration and correction is done for a geometrically corrected image. This means that the color correction also takes into account any non-uniformity introduced by the geometric warping itself. A geometrically warped image will have different areas containing different color or brightness content due to calibration and filtering procedures. In fact, the more a region is calibrated, the greater the change in brightness and color. This is automatically compensated for by color correction made after geometric warping. Therefore, the system can automatically compensate for the color unevenness caused by the above geometric warping program.

In another aptamer, the system can be integrated into a single circuit to obtain a digital calibration and warpage unit. The calibration data and warpage generators 12 and 13 are components that can be executed on any processor. The test image generator 14 can also be replaced by a set of stored images output by the processor. Using a built-in processor in the above hardware, a single circuit solution can be given for the entire calibration and calibration procedure described above. In addition, the hardware can be integrated with a camera to integrate a display device to obtain a self-calibrating display device. In this aptamer, only one processor is needed to receive sensing information from at least one image sensing device and calculate the display distortion to generate some pre-compensation maps, ie, warp maps And color maps (also known as geometric warpage and color warping), and applying the pre-compensation maps to their input image data so that the displayed image on the viewing surface is substantially No distortion. However, in other cases, using more than one processor may be more efficient. Thus, the implementation of the embodiments described herein requires at least one processor.

Various types of sensors can be integrated into the display device (rather than or in conjunction with the cameras) to act as the sensing device 11. In an exemplary embodiment shown in Fig. 14, a sensor 143 is a distance sensing device that is used independently or used with a camera 142 to measure the viewing surface 141. The distance of a certain number of points. This plane does not need to be flat. The relative angles of the cameras 142 to the viewing surface 141 are calculated from the measured distances and the angles at which the sensing distances are relative to each other. Further, the above-described shape which is not a flat screen can be calculated by such a method. In the example shown in Figure 14, the denser lines on the right side of the screen will indicate that the sensor 143 is closer to the normal view of the screen, and the less dense pattern on the left side, Indicates that it is far from the general observation above the left side. Various types of sensors 143 can be used including infrared sensors, and the like. In this exemplary embodiment, the display screen (i.e., viewing surface 141) is depicted, and does not require a physical structure, and the camera 142 can be arbitrarily arranged.

Another exemplary embodiment constitutes a self-calibrating display device with dynamic calibration and calibration so that the calibration and calibration procedure can be performed at any time to correct for distortion without the need for external resources. This may allow for correction of distortions that may change over time, such as a projector-related trapezoidal distortion, or some field calibration of a rear projection display device such as an RPTV. The calibration system is disposed within the housing or chassis of the RPTV to provide self-calibration in this case. Other important distortions of long-term changes are changes in the optical components due to physical motion, aging, and temperature. For example, in a rear projection display device, the curvature of a mirror may vary slightly due to its weight or temperature, which would require dynamic calibration and correction. The calibration and calibration system is executed when the display device is turned on, or a change in the distortion is detected.

In the field of fixed display devices, such as television systems, where no sensing devices are available, dynamic calibration and calibration become particularly important. Here, after the initial calibration and correction, the future distortion is caused by a small amount of variation in the components in the long term. In a controlled conditional context, such as a manufacturing plant, the digital warping unit can be used to simulate various long-term distortions expected in the field, i = 1...N. These distortions can then be calibrated and corrected to use the system described in the exemplary embodiments mentioned above; however, two electronic correction units may be used, one for analog distortion and one for testing These automatically generated calibration data. The correction related warpage data relating to the N test conditions can be stored in the display device. In the field and for a long time, with the development of a small amount of distortion, one of the N warpage corrections, one that best corrects the distortion will be selected. Therefore, the entire system is not necessary, only the digital warping unit needs to be built in the display device because the calibration system is completed during manufacturing, and the N sets of calibration data are stored in the display device. . To automate the selection of appropriate calibration data described above, the sensors within the display frame can be used to detect their particular test patterns. The image test pattern associated with the above-described best detection of distortion is thus loaded. This procedure can be performed when the display device is turned on for dynamic correction and calibration.

As shown in Figures 15 and 16, in an exemplary embodiment, the calibration system is adapted to find the best projector focus on a viewing surface. This is done by displaying a test pattern on the viewing surface, such as a specific set of parallel lines. The image is then taken and scanned by the electronic correction unit to find a comparison of dark areas and bright areas in the test patterns. The projector is focused and then shifted, and the contrast is re-measured. This will continue until the largest comparison is found. This maximum contrast corresponds to the best focus. This is displayed on the viewing surface 151 with a poor focus and is displayed on the viewing surface 161 with better focus. This same technique can be used to focus the sensing device. Some physical markers with sharp edges, such as the screen frame of the display screen (i.e., the viewing surface), are taken and analyzed for maximum contrast. If necessary, a properly colored test pattern can be displayed to improve the contrast between the markers and the background. The sensing device is focused and then shifted, and the contrast is re-measured. The setting of the maximum contrast provides the best focus associated with the sensing device. The sensing device focuses the focus prior to focusing the display device.

In another exemplary embodiment, some of which are shown in Figures 17 and 18, the calibration system is used in a display device having curved screens 171 and 181 and multiple projectors 1 through 3, respectively. The projectors span the entire area of the curved screens 171 and 181 described above and are controlled by the same electronic components. The geometric calibration is done for each of the projectors 1 through 3, and is mapped to corresponding regions of the screens 171 and 181. In addition, the geometric calibration rotates and translates each projector image to align with an adjacent projector image. In particular, in the overlapping regions, the corresponding pixels are overlaid on top of each other. It should be noted that the maps from the different projectors 1 through 3 on the screens 171 and 181 have different angles of incidence and are varied according to the curves of the screens 171 and 181. The electronic components having or having the maps represented by the curved screens 171 and 181, such as warpage data, correct for angular variations across the screens 171 and 181.

In addition to the geometric calibration, the color calibration of each of the projectors 1 through 3 is done to ensure visually identical color characteristics across all of the projector areas. The electronic component is also adapted to distinguish pixel color and brightness in or between projectors 1 through 3 such that a uniform brightness and color mapping is achieved across the curved screens 171 and 181. It should be noted that any number of individual projectors can be used, and such overlapping regions can be shared among many projectors when applying the same calibration techniques.

This focusing problem is always critical in terms of projection on a curved screen. This stems from the fact that a projector has a flat focal plane and the screen is curved, and as such, different portions of the screen have different distances from any of the focal planes. Looking at a portion of the screen, the image may appear to be more focused than another portion of the screen. To overcome this problem, a technique can be used to minimize the out-of-focus when projecting with a single projector, an exemplary embodiment of which is shown in FIG. In this case, the calibration system can minimize the sum of the squares of the distances from a series of normals from the curved screen 191 to the focal plane 193 in the manner in which the projected focus plane is arranged. If the screen wishes its center to be more focused than the side, the segments that connect the central portion of the screen to the focal plane give a greater weight.

In this case, the optimal focus plane can be pre-calculated based on the known shape of the screen described above. The intersection of the best focus plane and the screen described above produces a point on the screen with the best focus image, and then a maximum contrast is obtained. At the calculated and known best plane and maximum contrast point, an image test pattern similar to that used in Figure 16 is projected onto the screen, and the image is then taken and analyzed for comparison. If the maximum contrast position of the captured image is consistent with the previously determined maximum contrast point, and within a certain tolerance, the projected image is located on the optimal focus plane. If the maximum contrast point does not coincide with the previously determined maximum contrast point, the projector focus will be adjusted and the program will be repeated again and again until a match is obtained. It should be noted that this technique can be applied to some screens that are one-dimensionally curved (eg, cylindrical, zero-space curvature, etc.) or two-dimensionally curved (eg, spherical, non-zero spatial curvature, etc.) .

In another exemplary embodiment shown in Fig. 20, in addition to the calibration already explained, the focus problem is addressed by projecting images at different angles by multiple projectors. As shown in this figure, the defocus problem can be substantially eliminated by the projectors that are illuminated at a particular angle above a particular area of the curved screen 201. The angles are such that each of the projection axes is substantially normal to its corresponding portion of the screen portion, and each of the focal planes is nearly tangent to the center of the covered portion of the curved screen 201. To optimize the focus of each segment, the same technique as shown in Figure 19 can be used. Alternatively, the center of each focus segment can be kept tangential to the screen. In this exemplary embodiment, the calibration system can match the focus of the overlapping regions of the multiple projectors plus pixel geometry, brightness, and color to produce a smooth, seamless and focused image on the screen 201. . As a result of this technique, the warpage will become much less severe as the angle between the focus plane and the screen tangent decreases.

A system for calibrating a sensing device for multiple color geometries has been discussed. Similarly, the system can be used to calibrate color (non-geometric) distortion in the sensing device described above. Using a calibrated and calibrated display device, some fixed color patterns are displayed on the screen and recorded by the sensing device; the same pattern used to calibrate the color distortion of the display can be used. Knowing the original color value, you can get a camera color map similar to equation (25). From the color map, the color correction parameters related to the above camera can be determined, and there will be a spatial change in the presence of color distortion. The model related to the correction, for example, can be a linear least squares fit. These correction parameters fully characterize the calibration data associated with the color distortion of the camera.

This color correction has been rendered in terms of primary colors and brightness. The system can be adapted to handle correction and adjustment of an arbitrary color. These test patterns or various colors (not only primary colors or gray scales) can be used to obtain a color map of the above display in a manner similar to equation (31), which is shown in equation (52).

Here, each A color vector can be generated that has all of the components and not just a particular primary color component. The color used by the set can be selected as some sort of resampling of the vectors in the entire color space described above. The inverse function map is now represented by equation (53).

Here, each color parameter is a vector of length K (the number of primary colors). Based on the previous token:

However, this is not only the rearrangement of the color parameters into a single equation, since the basis functions are now defined over the entire color space and not only in a one-dimensional color space (i.e., a primary color). In the form of a polynomial, the basis functions are defined in equation (55).

The λ parameters can be further generalized by introducing a K-dimensional patch structure of Q patches as shown in equation (56) in the color space. , q =1... Q small piece (56)

This can be added to the color parameters as shown in equation (57).

This produces a general transformation (the center of the shapes) at each spatial grid point within the color space. This calibration color data is now defined by equation (58).

In the absence of any distortion, the ruled line will be one unit at each coordinate. The warp generator converts this into a surface function of the form specified in equation (59).

Finally, the digital warp unit will evaluate this polynomial and the color correction can be applied using equation (53).

At each spatial coordinate, there is a general color map that allows for the correction of any one of the colors at either coordinate. This includes performing general color adjustments such as white point adjustment, contrast adjustment, and tone adjustment, regardless of the different areas of the display. All such adjustments are some of the specific functions in the color space, and thus can be approximated via a function such that the general form specified above in equation (53) is achieved. Selective color correction can also be done with additional features that distinguish between small pieces in the color space. This correction can be limited to some specific colors by forcing the unit cell lines outside the color patch to be corrected, leaving the others unchanged. This includes selective tone calibration where specific tones are corrected and other tones are not touched. With the general color calibration and calibration of the system, high color accuracy will be achieved in the display device.

The system can also be used for color customization by providing some custom color parameters λ ' _ikrq , which can be calculated outside of the system and can be input to the warp generator 13 . Similarly, some custom geometric effects (specific effects) can be provided by providing some custom geometry lines. This warpage generator 13 is achieved.

In another exemplary embodiment shown in Fig. 21, two cameras Cm1 and Cm2 are mounted on a projector 213. An input image is provided to the projector 213, which in turn produces a corresponding projected image pattern on a viewing surface 211. The two cameras Cm1 and Cm2 are used to capture the projected image pattern on the viewing surface 211. The system further includes a processor (not shown but previously described). The relative positions of the two cameras Cm1 and Cm2 are known to the processor. The two cameras Cm1 and Cm2 may be staggered horizontally, vertically, or both horizontally and vertically with respect to the projector 213. The processor can determine the distortion parameters including the angle of the projector 213 relative to the viewing surface 211 based on a comparison of the two captured images from the two cameras Cm1 and Cm2. The electronic correction unit described above (not shown but previously described) then applies a warp transformation to the input image to correct the distortion.

The projected image of the above achievements is largely undistorted. The system and method can be used in a rear projection television (RPTV), for example, where one or more cameras are mounted to the RPTV as seen in the exemplary embodiment shown in FIG. Fixed position and orientation. These cameras can also be installed in other ways. These cameras can capture some of the patterns projected onto the RPTV screen above. The view of the RPTV screen from the perspective of the above camera may have some trapezoidal distortion associated with it. However, where the calibration system is part of the display device described above, the display can be self-calibrated as discussed above.

In another exemplary embodiment shown in Fig. 23, a plurality of projectors P1 to P3 are used to project an image onto a curved screen 231. At the same time, there are several cameras Cm1 to Cm3, which are used to capture the image projected by each of the projectors P1 to P3. The number of such cameras Cm1 to Cm3, and the number of such projectors P1 to P3, are arbitrary in this embodiment. In one case, each camera Cm1 to Cm3 can be used to capture images from all of the projectors P1 to P3. The cameras Cm1 to Cm3 may be staggered with respect to each other horizontally and vertically. Each of the projectors P1 to P3 is adapted to project a known pattern or test image onto the curved screen 231 for calibration. Based on the images captured by the cameras Cm1 through Cm3, a processor (not shown but previously described) can calculate the distortion parameters including the shape and relative orientation of the curved screen 231 described above. These parameters can then be used by the processor to generate a warp transform that can be applied to the input images provided to each of the projectors P1 through P3 during normal use. The warp transformation of each of the projectors P1 to P3 is such that it pre-compensates for display distortion suffered by the particular projector. In addition, the brightness associated with each of the projectors P1 through P3 can be analyzed such that the total brightness of the projected image on the viewing surface 231 is consistent. In addition, the processor can align the pixels in the overlapping regions and distribute the brightness of the overlapping pixels between the different projectors in a seamless image quality.

In a further embodiment of the system of Fig. 23, the brightness and color data can also be taken by the cameras Cm1 to Cm3. The data is then used by the processor to match and blend the side edges of the different contiguous images by adjusting the intensity associated with each pixel. The total brightness and color of all of the projectors P1 to P3 can also be normalized by the processor.

In another exemplary embodiment, partially shown in Figure 24, the sensing device (in this case the camera) is used to capture a projected projection with or without a pattern. image. At this point, the camera is also used to detect the shape, scale, relative orientation, and boundaries of a viewing surface 241. The boundary side edges may be the side edges of a drop-down viewing surface (ie, a telescopic projector screen), or the corners of a room, and the like. A processor (not shown but previously described) can then analyze the direction of the side edges of the image and the pattern of the test image to calculate characteristics of the viewing surfaces, such as shape, scale, boundary, and relative orientation. Under this calculation, the display distortion can be determined. Based on the complexity of the pattern of the images projected and then captured, the electronic correction unit (i.e., the processor) can determine the distortion parameters. In the simplest form, the electronic correction unit determines the angle of projection relative to the viewing surface. In a more complex form, the electronic correction unit can determine the shape of the viewing surface, for example, a curved or irregular viewing surface. The electronic correction unit may also determine distortion parameters associated with the lens ,, such as pin cushion or barrel distortion. Once the distortion parameters are collected, the appropriate pre-compensated warp map is applied to the input image data to correct the distortion and the resulting image is visually undistorted.

In one alternative embodiment, the system of Figure 24 is further adapted to correct projection onto a flat surface in the absence of any physical indicia or side edges. Distortion from the above projections may include both trapezoidal and lens distortion. In this system, a camera is attached to the projector in a fixed position and orientation. This calibration and calibration is performed in a two-step procedure. In the first step, a complete calibration procedure, using some of the test image styles, can be used to store images of the styles captured by the camera at some of the known keystone distortion angles and lens distortion parameters including the zoom level. In addition, any additional information necessary for the correction, such as warpage data, may also be stored. This step can be performed in a factory that assembles the projector and can be considered factory calibration. This second step occurs in the field where the projector is used. The projector can project the same pattern used in the first step described above, which can then be taken by the camera. The pattern of such live shots is such that the styles captured by the factories are compared to the stored distortion parameters of the factory to determine the distortion parameters of the projector in the field. Knowing the distortion parameters in the scene, the calibration warp can be retrieved if it has already been stored, or established in real time to correct the trapezoidal distortion and lens distortion of the projector. Since the comparison is done with previously stored information (images), the true side edges or marks (such as screen frames) are not needed. The data stored in the above factory does not need to be a complete image, but it may be grid data, or other parameters that characterize the different distortion levels.

In another alternative embodiment, the camera will be used to correct keystone distortion using a simple grid pattern of only 4 points. In this case, the test pattern is as in Figure 2a or 2b, consisting of a 2x2 grid (only 4 points are required). In the case of keystone distortion, 4 points are sufficient to determine the distortion without any lens distortion. These four points can be placed anywhere, since simply knowing their position (before and after projection) is sufficient to determine the keystone correction. This method can also incorporate any of the projector lens shift adjustments, which are simple translations of the four points. For a projector with a telescopic lens that may or may not have lens distortion, the calibration may first be performed on the axis (i.e., without keystone distortion) for different scaling levels and stored corrected warps. This corrected warp (in terms of proper scaling level and lens distortion) can then be applied, and the calibration can be repeated over the trapezoidal correction using only 4 points. The keystone correction can be phase-coupled or functionally combined with the telescopic lens to obtain a final map that corrects for all projector distortion. This lens correction only needs to be calculated and stored once during a factory calibration procedure. This keystone correction is then performed using the camera in the field and is composed of lens correction.

Another exemplary embodiment, shown in Figure 25, is related to the projection of a curved screen 251. To determine the map of shape and distance contained in the curved screen 251, a two-dimensional image pattern, for example, a plaid image pattern, is projected onto the viewing surface. The camera is used to capture the above projected image. The electronic correction unit (i.e., the processor not shown but previously described) is then adapted to calculate the contrast introduced by each of the above-described plaid patterns. By constantly changing the focus, the best contrast at each point above the pattern is found to be a function of the focal length described above. In this manner, the surface map of the curved surface 251 described above is determined. The accuracy and detail of the map depends on the complexity of the projected style and the number of focal lengths attempted. It should also be noted that this technique also produces the angle of the camera, and thus the angle of the projector at each point relative to the normal to the viewing surface. Once the electronic correction unit, the distortion parameters associated with the shape, scale, and angle of the viewing surface at each point are calculated, which in turn calculates a warp transformation, or uses an appropriate one that has been stored. The warp transformation, when applied to the input image data described above, results in a non-visually distorted image that matches the characteristics of the viewing surface.

Another exemplary embodiment, shown in Figure 26, is related to the case of a wavy screen 261. The techniques described above in the embodiments associated with Figure 25 can also be used to determine the shape and relative orientation of the undulating viewing surface at each point. This example shows that any irregular viewing surface can be used in the display device. Once the map of the viewing surface is prepared, the electronic correction unit (not shown but previously described), the map can be used to configure the warp transformation applied to the input image. Once this warp transformation is applied to the input image, the projected image is visually distorted and matches the characteristics of the viewing surface.

While the above description is provided by way of example embodiments, it is understood that certain features and/ or functions of the embodiments of the embodiments may be Modification. Accordingly, the above description is intended to be illustrative and not limiting, and the skilled in the art will understand that other modifications and modifications can be made without departing from the practice. The scope defined in the scope of the appended patent application.

1-3. . . Projector

11. . . Sensing device

12. . . Calibration data generator

12'. . . Calibration data generator

13. . . Warpage generator

14. . . Test image generator

15. . . Digital warping unit

16. . . Watching the surface

91. . . Watching the surface

92. . . Single point color analyzer

93. . . Multi-point color analyzer

141. . . Watching the surface

142. . . camera

143. . . Sensor

151. . . Watching the surface

161. . . Watching the surface

171,181. . . Curved screen

191. . . Curved screen

193. . . Focus plane

211. . . Watching the surface

213. . . Projector

231. . . Curved screen

241. . . Watching the surface

251. . . Curved screen

261. . . Wavy screen

Figure 1 is a simplified diagram of an exemplary embodiment of an automated calibration and calibration system; Figures 2a and 2b are illustrations of curved screen geometry; Figure 3 is an overflow, underflow, and mismatch in geometric distortion An illustration of an example; Figure 4 is an illustration of an example of a calibration image test pattern; Figure 5 is an illustration of a calibration geometry and various coordinate spaces involved; and Figure 6 is an example of a calibration data generator. Illustrative diagram of a preferred embodiment; Figure 7 is an illustration of an optimization of the scale and origin; Figure 8 is an illustration of an exemplary embodiment of a multiple color calibration data generator; An illustration of a setting related to color unevenness calibration; FIG. 10 is an illustration of an exemplary embodiment of a calibration data generator related to color unevenness correction; and FIG. 11 is an example of a warpage data generator Illustrative diagram of an embodiment; FIG. 12 is an illustration of a display segmentation correction for a display; FIG. 13 is an illustration of an exemplary embodiment of a digital warpage unit; and FIG. 14 is a view of a viewing surface The shape and relative orientation determine the schematic diagram of the relevant settings; Figure 15 is an illustration of a out-of-focus test pattern; Figure 16 is an illustration of a focus test pattern; Figure 17 is a multi-projector and a curved screen A partial illustration of an exemplary embodiment of a calibration system; an illustration of a calibration system consisting of a multi-projector and a curved screen of Figure 17 for displaying the focus planes of different projectors Partial illustration of an embodiment; Figure 19 is an illustration of an example of a focusing technique that minimizes a distance function; Figure 20 is another adjustable projector that consists of a multi-projector and a curved screen. Partial illustration of an exemplary embodiment of a calibration system that optimizes image focus optimization; FIG. 21 is a partial illustration of an exemplary embodiment of a calibration system using multiple cameras; and FIG. 22 is a Partial illustration of an exemplary embodiment of a rear projection television (RPTV) that allows the display to self-calibrate and allow dynamic distortion correction of the integrated calibration system; Figure 23 is a Partial illustration of an exemplary embodiment of a calibration system of multiple projectors and multiple sensing devices; Figure 24 is a partial illustration of an exemplary embodiment of a calibration system using physical edges and boundaries of the viewing surface described above Figure 25 is a partial illustration of an exemplary embodiment of a calibration system that uses a focusing technique to determine the shape of a curved display screen; and Figure 26 illustrates the use of a focusing technique to determine a wavy display. A partial illustration of an exemplary embodiment of a calibration system for the shape of a screen.

11. . . Sensing device

12. . . Calibration data generator

13. . . Warpage generator

14. . . Test image generator

15. . . Digital warping unit

16. . . Watching the surface

18. . . Surface border

Claims (26)

A display calibration system for use with a display device having a viewing surface, the display calibration system comprising: at least one sensing device adapted to sense a shape, a scale, a boundary, and At least one related information of the orientation; and at least one processor coupled to the at least one sensing device and adapted to calculate characteristics of the display device based on information sensed by the at least one sensing device. The display calibration system of claim 1, wherein the at least one sensing device is further adapted to sense a test image displayed on the viewing surface, and wherein the at least one processor is further It is adapted to calculate the distortion of the display based on the sensed test images and the characteristics of the display devices. The display calibration system of claim 2, wherein the at least one processor is further adapted to generate some pre-compensation maps based on the display distortions, so that when the pre-compensation maps are applied, When the input image data before the display is displayed, the display image formed on the viewing surface is substantially free from distortion. The display calibration system of claim 3, wherein the display distortion varies over time, and wherein the display calibration system is adapted to dynamically calibrate the display device to pre-compensate for varying distortion. The display calibration system of claim 3, wherein the at least one processor is adapted to correct at least one of the following: an overflow condition, wherein a displayed image is greater than the viewing surface; An underflow condition, wherein the displayed image is smaller than the viewing surface; and a mismatched condition in which a portion of the displayed image overflows the viewing surface and other portions of the displayed image underflow Watch the surface. A display calibration system according to claim 3, wherein the display device is a rear projection display device having a housing, and wherein the display calibration system is disposed within the housing. The display calibration system of claim 3, wherein the at least one sensing device is further adapted to sense at least one brightness information and color information, and wherein the at least one processor is further adapted Equipped to precompensate at least one brightness non-uniformity and color unevenness, respectively. The display calibration system of claim 3, wherein the display system further includes some optical components with additional distortion, and wherein the at least one processor is further adapted to make the additional distortion Linking to these display distortions to pre-compensate for these additional distortion and display distortion. The display calibration system of claim 2, wherein the display distortion comprises at least one of geometric distortion, optical distortion, under convergence, under-alignment, and lateral chromatic aberration. The display calibration system of claim 1, wherein the at least one sensing device is adapted to sense the viewing surface to a majority The distance of the points, and wherein the at least one processor is adapted to calculate the relative position and relative orientation of the viewing surface based on the equal distances. The display calibration system of claim 2, wherein the at least one sensing device is adapted to sense different portions of a test image on the viewing surface at various focal lengths, and wherein the at least one a processor adapted to determine a highest contrast in different portions of the test image and to calculate a distance to a different portion of the viewing surface based on the highest contrast of the determination to calculate a shape and relative orientation of the viewing surface . The display calibration system of claim 2, wherein the at least one sensing device has some sensor distortion, and wherein the at least one processor is further adapted to calculate the sensor distortion And when calculating the distortion of the displays, the sensor distortion is taken into account. The display calibration system of claim 12, wherein the sensor distortion is caused by at least one sensing device having an axis that is not parallel to a normal direction of the viewing surface. The display calibration system of claim 2, wherein the at least one image sensing device comprises a plurality of image sensing devices disposed at different locations known by the at least one processor, and wherein At least one processor adapted to compare different sensed test images from different sensing devices and to calculate the displays based on the different sensed images and the locations of the different sensing devices distortion. The display calibration system of claim 2, wherein the at least one image sensing device is adapted to sense information about a test image having four marks on the viewing surface, and wherein At least one processor is adapted to calculate keystone distortion based on the sensed information. The display calibration system of claim 3, wherein the at least one sensing device is further adapted to sense at least one of the brightness information and the color information, and wherein the at least one processor, The method is further adapted to correct at least one of brightness non-uniformity and color non-uniformity caused by the pre-compensation maps. A display calibration system for use with a display device having a viewing surface, the display calibration system comprising: at least one sensing device adapted to sense information of a test image displayed on the viewing surface And at least one processor coupled to the at least one sensing device, the at least one processor adapted to calculate display distortion based on the sensed information, and generate some pre-compensation maps to compensate for the compensation The display is distorted, and the pre-compensation map can be implemented by a surface function, so that when the pre-compensation map is applied to the input image data before display, a display image formed on the viewing surface is substantially free from distortion. The display calibration system of claim 17, wherein the at least one processor is further adapted to link various distortions and to generate surface functions that pre-compensate for distortion of the links. The display calibration system of claim 17, wherein the surface functions are polynomials. The display calibration system of claim 17, wherein the at least one processor is further adapted to adjust the surface functions to further compensate for at least one of an overscan condition and an underscan condition. A display calibration system for use with a display device having a viewing surface, the display calibration system comprising: at least one image sensing device adapted to sense a test image displayed from the viewing surface And at least one processor coupled to the at least one image sensing device, the at least one processor adapted to calculate display distortion based on the sensed information to cause distortion according to display within each patch The severity of the viewing surface is divided into small pieces, and pre-compensation patterns related to the display distortion in each of the small pieces are generated, so that when the pre-compensation patterns are applied to the input image data before display, one is viewed there. The displayed image on the surface is substantially free of distortion. A display calibration system for use with a display device having a viewing surface, the display calibration system comprising: at least one image sensing device adapted to independently sense a test displayed on the viewing surface Color information relating to at least one color component of the image; and at least one processor coupled to the at least one image sensing device, the at least one processor adapted to sense the color Information for calculating color non-uniformity and causing at least one color correction map associated with generating at least one color component such that when the at least one color correction map is applied to the input image material prior to display, one on the viewing surface The display image is substantially free of at least one color unevenness. A display calibration system for use with a display device having a viewing surface, the display calibration system comprising: at least one image sensing device adapted to sense individual color components displayed on the viewing surface Testing at least one color component independently of the at least one image sensor and the display device Corresponding geometric display distortion, and at least one pre-compensation map relating to independently generating at least one color component, such that when the at least one color correction map is applied to the input image material before display, one is formed on the viewing surface The display image generally has no geometric distortion of at least one color dependency. A display calibration method that can be used in a projection system having a curved viewing surface, the method comprising the steps of: projecting different portions of an image onto a corresponding portion of the curved viewing surface using a plurality of projectors; And causing each portion of the image to be substantially focused on a corresponding portion of the curved viewing surface such that the image is optimally focused and integrally formed on the curved viewing surface. The method of claim 24, wherein the method further comprises the step of independently positioning and orienting each of the plurality of projectors such that a projection axis of each of the projectors is substantially perpendicular to the curved view The corresponding part of the surface is used to optimize focus and minimize geometric distortion. A display calibration method that can be used in a projection system having a curved viewing surface, the method comprising the steps of: measuring a majority distance from the curved viewing surface to a focal plane of the projected image; and offsetting the focal plane until The functions of these majority distances are minimized to get the focus of optimization.