JP4293191B2 - Pixel position acquisition method, image processing apparatus, program for executing pixel position acquisition method on a computer, and computer-readable recording medium on which the program is recorded - Google Patents

Description

  The present invention relates to a pixel position acquisition method, an image processing apparatus, a program for executing the pixel position acquisition method on a computer, and a computer-readable recording medium on which the program is recorded.

In a fixed pixel type display such as a liquid crystal projector, by performing a pixel characteristic value (brightness, chromaticity) correction process on each pixel, the display quality improvement of images and videos of the fixed pixel type display is greatly improved.
Since the correction processing method and correction amount of such pixel characteristic value correction processing are set based on the display principle of the fixed pixel type display, the light emission characteristic / transmission characteristic of the pixel due to the manufacturing process and manufacturing conditions, etc. A technique for accurately acquiring pixel characteristic values (luminance and chromaticity appearing as output of pixel emission characteristics and pixel transmission characteristics) of a fixed pixel display is important.
In general, an appropriate image / video is displayed on a fixed pixel display, the display screen is photographed by an imaging device such as a CCD camera, and the pixel characteristic value is obtained by analyzing the photographed data.

In such a method for acquiring the pixel characteristic value of the fixed pixel type display, it is necessary to accurately acquire the characteristic value of each pixel.
There are various factors that affect the accuracy of acquiring the characteristic value of each pixel, but it is particularly important to accurately acquire the position where each pixel of the fixed pixel display is mapped on the shooting data. It is.
For this reason, conventionally, a state in which a plurality of markers having a specific distribution are displayed on the display screen is photographed by the imaging device, and the correspondence between the display screen and the photographed data is calculated from the association of the markers. A method for acquiring a position where each pixel position on the display screen is mapped on the shooting data has been proposed (for example, see Patent Document 1).
Also, it obtains the contour of the image displayed on the image display device as a straight line, calculates the projective transformation between the display image and the shooting data from the intersection of the straight lines of the contour (that is, the four corners of the display image), and the projective transformation There has been proposed a method of performing geometric correction of photographing data using the method (for example, see Patent Document 2).

JP 2001-356005 A (FIG. 3) Japanese Patent Laying-Open No. 2005-122323 (FIGS. 10 and 19)

However, in the technique described in the patent document, the correspondence between each pixel position of the display image and the position mapped on the photographing data is calculated based on the dot-shaped marker and the four corner points of the display image. However, it is hard to say that the correspondence can be derived with high accuracy.
Further, in the techniques described in these patent documents, when trying to obtain the relationship between the pixel position and the position mapped on the shooting data with high accuracy, the number of sampling points for deriving the correspondence increases. Since projective transformation or the like must be performed for each of them, there is a problem in that the computation processing becomes complicated.

  An object of the present invention is to provide a pixel position acquisition method, an image processing apparatus, a program for executing a pixel position acquisition method on a computer, and a program for acquiring a position where each pixel of a display is mapped on photographing data at high speed and accurately. Is to provide a computer-readable recording medium on which is recorded.

The pixel position acquisition method according to the present invention includes:
A display image displayed on the screen is picked up by an image pickup device, and based on the picked-up image pickup data, the pixel position of the image generation device projected on the screen and the pixel position are mapped onto the image pickup data. A pixel position acquisition method for acquiring a correspondence relationship with a determined position,
Displaying a first line image composed of a plurality of straight lines extending along a vertical direction of the image generating device on the screen;
A procedure of capturing a display image based on the first line image by the imaging device and acquiring the captured first imaging data;
Displaying a second line image consisting of a plurality of straight lines extending in the horizontal direction of the image generating device on the screen;
A procedure of capturing a display image based on the second line image by the imaging device and acquiring captured second imaging data;
A procedure of cutting out a plurality of imaging lines mapped on the first imaging data in units of areas including the respective imaging lines;
A procedure for setting an approximation function representing each imaging line in the first imaging data based on the area of each extracted imaging line;
A procedure of cutting out a plurality of imaging lines mapped on the second imaging data in units of areas including the respective imaging lines;
A procedure for setting an approximate function representing each imaging line in the second imaging data based on the area of each extracted imaging line;
Based on the approximate function set by the first imaging data and the second imaging data, an association between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data is obtained. And a procedure.

According to the present invention as described above, the first line image and the second line image serving as the test pattern are configured by a plurality of straight lines extending in the vertical or horizontal direction, and the straight lines are mapped on the imaging data. By setting an approximation function that represents the image, and by associating the pixel position of the image generation device with the position where the pixel position is mapped on the imaging data, an image display composed of the screen and the image generation device is performed. Projected on the screen because the position of the imaging device with respect to the device and the imaging data distortion caused by the optical elements such as lenses constituting the imaging device can be correlated in the form of an approximate function. The correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data can be acquired with high accuracy.
Also, the correspondence between the pixel position of the image generation device projected on the screen simply by displaying the first line image and the second line image and the position where the pixel position is mapped on the imaging data is high. Since it can be executed with high accuracy, the correspondence can be acquired at high speed.

In the present invention,
The first line image and the second line image are configured to include at least four markers that give correspondence with respective imaging data,
The procedure of cutting out a plurality of imaging lines mapped on the first or second imaging data as an area including each imaging line,
Setting a conversion formula between the first line image and the first imaging data based on a correspondence relationship between the first line image and the marker of the first imaging data;
Setting a conversion formula between the second line image and the second imaging data based on a correspondence relationship between the second line image and the marker of the second imaging data;
Setting an area surrounding each straight line in the first or second line image;
Converting a region surrounding each straight line in the first or second line image by the set conversion equation;
Preferably, the method includes a step of cutting out each imaging line of the first imaging data or the second imaging data based on the converted area.
Here, the setting of the conversion formula between the first line image and the first imaging data, or the second line image and the second imaging data can be easily performed using, for example, a projective conversion formula. Specifically, when the vector that gives the mapping position of the imaging data is (x, y) ^T and the vector that gives the pixel position of the display image is (X, Y) ^T , the following conversion equation (1) can be used. it can.

  According to this invention, if the area for extracting each imaging line is set by the first line image or the second line image by the conversion formula set by the marker, the first imaging data or the second imaging is set. Since a region to be cut out on the data can be easily obtained by a conversion formula, it is possible to cut out each imaging line mapped on the first imaging data or the second imaging data without performing complicated calculation processing. . In particular, conversion can be performed more easily by using a projective transformation matrix such as the conversion formula (1).

In the present invention,
The procedure for setting the approximate function representing each imaging line is as follows. When the vertical direction of the first imaging data or the second imaging data is the y axis and the horizontal direction is the x axis,
Each imaging line mapped on the first imaging data is set as an approximate function in which x is expressed as a polynomial of y,
Each imaging line mapped on the second imaging data is preferably set as an approximate function in which y is expressed as a polynomial of x.

  According to the present invention, since each imaging line mapped on the first imaging data or the second imaging data is set as an approximate function given by a polynomial, each imaging line mapped on the imaging data is Even if it is a curve, the correspondence between the first and second line images and the first and second imaging data can be taken with high accuracy. In addition, by setting as an approximate function using a polynomial corresponding to the imaging line, it is possible to obtain the corresponding relationship without requiring a complicated calculation process to obtain the approximate function.

In the present invention,
The procedure for obtaining the correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data is as follows:
Correspondence between each straight line on the first line image and each imaging line on the first imaging data, and correspondence between each straight line on the second line image and each imaging line on the second imaging data The step of attaching,
Obtaining intersections of a plurality of imaging lines on the first imaging data and a plurality of imaging lines on the second imaging data based on an approximate function set in accordance with each of the associated imaging lines;
Based on each obtained intersection point, an arbitrary point in an area surrounded by two adjacent imaging lines on the first imaging data and two adjacent imaging lines on the second imaging data is determined. And preferably setting an interpolation function to be provided.

  According to the present invention, since the interpolation function can be set based on the imaging line whose correspondence with the straight line on the line image is determined with high accuracy by the approximation function, the arbitrary pixel position of the image generation device and The correspondence relationship between the pixel position and the position mapped on the imaging data can be obtained with high accuracy.

In the present invention,
After the procedure for obtaining the correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data,
The correspondence between the pixel position of the image generation device already calculated and the position where the pixel position is mapped on the imaging data, and the pixel position of the image generation device calculated again and the pixel position are mapped on the imaging data. Provided with a procedure for performing pass / fail evaluation based on the correspondence with the determined position,
If not, from the procedure of displaying the first line image again on the screen to the procedure of finding the correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data Is preferably repeated.

  According to the present invention, the pass / fail evaluation is performed to evaluate the correspondence between the pixel position and the position where the pixel position is mapped on the imaging data. By obtaining the correspondence between the pixel position and the position where the pixel position is mapped on the imaging data, the correspondence can be obtained with higher accuracy.

In the present invention,
It is preferable that at least one of the procedure of displaying the first line image and the procedure of displaying the second line image at the time of repetition displays the updated line image with the interval of each straight line narrower than the previous time.
According to the present invention, the correspondence relationship is obtained again using the line image updated with the interval between the straight lines reduced, so that the two adjacent imaging lines mapped on the first imaging data and the second Since the area surrounded by two adjacent imaging lines mapped on the imaging data is smaller than the previous area, any pixel position in that area and the area where the pixel position is mapped on the imaging data Correspondence with the position of can be obtained with higher accuracy.

In the present invention,
It is preferable that at least one of the procedure of displaying the first line image and the procedure of displaying the second line image at the time of repetition displays a line image in which each straight line is arranged at a position different from the previous time.
According to the present invention, since the imaging line mapped on the imaging data can be associated again with the imaging line being arranged at a position different from the previous time, the distortion of the imaging data that could not be detected in the previous time is detected. The possibility increases, and a correspondence between an arbitrary pixel position in the area and a position in the area where the pixel position is mapped on the imaging data can be obtained with high accuracy. Further, since the number of imaging lines in the line image is the same, the calculation process is not complicated, and the association can be obtained with the same load as the previous time.

  The present invention is not limited to the pixel position acquisition method described above, but also executes an image processing apparatus and a computer that execute the respective steps constituting the method according to each of the inventions. A program to be executed, and a computer-readable recording medium on which the program is recorded. According to these inventions, an image processing apparatus and a computer that can enjoy the same operations and effects as those described above Can be obtained.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[First Embodiment]
FIG. 1 shows a pixel characteristic value acquisition system 1 for performing the pixel position acquisition method according to the first embodiment of the present invention. The pixel characteristic value acquisition system 1 acquires pixel characteristic values, such as luminance and chromaticity, for each pixel of a fixed pixel type display such as a liquid crystal panel constituting the projector 100, and based on the acquired pixel characteristic values. Thus, correction data for each pixel is generated, and the image pickup apparatus 2 and the computer 3 are provided.
The imaging device 2 employs a CCD camera, a CMOS sensor, or the like, is placed at a fixed point position with respect to the screen 101, and takes image data acquired by imaging a projection image based on the image data projected on the screen 101. 3 is output.

  The computer 3 includes an arithmetic processing unit and a storage device, processes the imaging data output from the imaging device 2, and a pixel position on a display image displayed by a liquid crystal panel or the like constituting the projector 100, and the pixel position Is determined to correspond to the position mapped on the imaging data, and a pixel characteristic value for each pixel is obtained. Specifically, the computer 3 includes a display image capturing unit 31, a display image forming unit 32, a mapping area acquisition unit 33, an approximate function setting unit 34, a pixel mapping calculation unit 35, and a pixel characteristic value acquisition unit 36. In addition to the vertical line image data storage unit 37, the horizontal line image data storage unit 38, and the characteristic value acquisition image data storage unit 39 prepared in advance, the storage device stores data acquired or generated by each functional means. The vertical line imaging data storage unit 40, the horizontal line imaging data storage unit 41, the characteristic value acquisition imaging data storage unit 42, the vertical line mapping region data storage unit 43, the horizontal line mapping region data storage unit 44, and the vertical line mapping are stored. Data storage unit 45, horizontal line mapping data storage unit 46, pixel mapping data storage unit 47, and pixel characteristic value data storage unit 48 are secured. To have. Hereinafter, processing performed by each of these functional means will be described in detail.

[1] Processing in Display Image Capturing Unit 31 and Display Image Forming Unit 32 The display image capturing unit 31 and the display image forming unit 32 output predetermined image data to the projector 100 that is a pixel characteristic value acquisition target. The projector 100 is a means for displaying a predetermined display image on the screen 101, and the display image displayed on the screen 101 is picked up by the image pickup apparatus 2 and captured as image pickup data. A series of processes as shown in FIG.
(1) First, the display image forming unit 32 reads and acquires the vertical line image data stored in the vertical line image data storage unit 37 (process S1), and transmits the acquired data to the projector 100 (process S2). . Here, as shown in FIG. 3, the vertical line image data is an image A1 in which a plurality of straight lines extending in the vertical direction on the screen are arranged at predetermined intervals in the horizontal direction. In the vertical line image data storage unit 37 as metadata A2 including data width W, height H, 0th vertical line position w ₀ , 1st vertical line position w ₁ , 2nd vertical line position w ₂ . It is recorded.

(2) The display image forming unit 32 notifies the display image capturing unit 31 that the projector 100 has started to display vertical line image data (step S3). When the display image capturing unit 31 acquires information indicating that the image display from the display image forming unit 32 is started (processing S4), the display image capturing unit 31 captures the display image by the imaging device 2 (processing S5), and the vertical line image is acquired. The captured image data is taken into the computer 3 and stored in the vertical line image captured data storage unit 40 (processing S6).
(3) When the storage of the imaging data is completed, the display image capturing unit 31 notifies the display image forming unit 32 to that effect (processing S7). When the display image forming unit 32 acquires the information indicating the end of imaging (processing S8), it starts acquiring the next horizontal line image data (processing S9).

(4) Thereafter, the display image forming unit 32 repeats the acquisition and display of the horizontal line image data and the pixel characteristic value acquisition image data, and the display image capturing unit 31 sequentially captures the image data of each image accordingly. The horizontal line image data storage unit 41 and the characteristic value acquisition data storage unit 42 are stored. As shown in FIG. 4, the horizontal line image data is an image A3 in which a plurality of straight lines extending in the horizontal direction are arranged at predetermined intervals in the vertical direction, and are stored in the horizontal line image data storage unit 38. The metadata A4 includes the width W, the height H, the zeroth horizontal line position h ₀ , the first horizontal line position h ₁ , the second horizontal line position h ₂ . Although not shown, the pixel characteristic value acquisition image data includes a luminance signal, a color signal, and the like that display a certain luminance, color, etc. on all the pixels. For example, the voltage transmission characteristic (VT) of each pixel is displayed. -Γ) is stored in the characteristic value acquisition image data storage unit 39 as data that can be understood.

(5) The display image capturing means 31 captures a display image displayed based on the horizontal line image data with the imaging device 2, captures it as horizontal line image captured data, stores it in the horizontal line image captured data storage unit 41, and pixel characteristics. A display image displayed based on the image data for value acquisition is captured by the imaging device 2, captured as characteristic value acquisition image data, and stored in the characteristic value acquisition image data storage unit 42.
(6) The display image forming unit 32 determines whether or not the image has been displayed on the projector 100 based on all the image data described above (processing S9). Is displayed to the display image capturing means 31 (process S10), and the process is terminated.
(7) The display image capturing unit 31 determines whether or not the information indicating that the image display has been completed is acquired from the display image forming unit 32 (processing S11). finish.

[2] Processing in Mapping Area Acquisition Unit 33 The mapping area acquisition unit 33 includes the above-described vertical line image data A1 and horizontal line image data A3, and vertical line image imaging data and horizontal line images obtained by capturing these display images with the imaging device 2. This is means for obtaining a rough correspondence with the imaging data and cutting out each imaging line on the vertical line image imaging data and the horizontal line image imaging data based on this, and specifically, the processing shown in FIG. 5 is performed. .
(1) First, the mapping area acquisition unit 33 acquires the vertical line image data A1 stored in the vertical line image data storage unit 37 and the horizontal line image data A3 stored in the horizontal line image data storage unit 38 (processing S12). ). When the acquired vertical line image data and horizontal line image data are superimposed, a lattice-shaped image A5 is obtained as shown in FIG.

(2) Next, the mapping area acquisition unit 33 acquires the vertical line image imaging data stored in the vertical line image imaging data storage unit 40 and the horizontal line image imaging data stored in the horizontal line image imaging data storage unit 41. (Process S13). When the acquired vertical line image capturing data and horizontal line image capturing data are superimposed, a grid-like image A5 is acquired as an image A6 deformed according to the image capturing position with respect to the screen 101 of the image capturing apparatus 2, as shown in FIG. Is done.
(3) the mapping region acquiring unit 33, the marker C _A of the four corners of the image A5 as a preset vertical line image data and the horizontal line image _data, C B, _{C C,} acquires the position of the C _D (step S14 ).
(4) continued mapping region acquiring unit 33, the marker _C A of the image _{A5, C B, C C,} C D mapping positions are mapped on the image A6 as the imaging data _{c A, c B, c C} , c _D is acquired (step S15).

(5) The mapping area acquisition means 33 calculates a projective transformation matrix based on the correspondence between the markers C _A , C _B , C _C , C _D and the mapping positions c _A , c _B , c _C , c _D ( Process S16). Specifically, for the vector (x, y) ^T , the coordinate axis is increased by one and an arbitrary constant w is set. When the vector (wx, wy, w) ^T is used, the projective transformation is expressed by the following equation (2). It can be expressed by the conversion formula shown.

When this conversion formula (2) is arranged, the mapping position coordinates (x, y) of the mapping positions c _A , c _B , c _C , c _D are expressed by the following formula (3) according to the marker coordinates (X, Y). Is given by equation (4).

(6) When the correspondence between the mapping position coordinates (x, y) and the marker coordinates (X, Y) is obtained, the mapping area acquisition unit 33, for example, on the image A7 representing the vertical line image data shown in FIG. Is surrounded by points R _A , R _B , R _C , and R _D to cut out from the vertical line image data. Further, the coordinates (X of each of the points R _A , R _B , R _C , and R _D are performed. , Y), the coordinates (x, y) of the points r _A , r _B , r _C , r _D mapped on the image A8 that is the vertical line image pickup data are obtained from the equations (3) and (4). Based on the obtained coordinates, the imaging line on the image A8 is cut out to obtain vertical line mapping area data (processing S17). Vertical line mapping region data acquired, for example, if the three first vertical line dimension w ₂ from the left end of the vertical line image data of FIG. 7, as shown in the image A9, on the vertical line image data The position w ₂ , the upper left position (x ₂ , y ₂ ) of the area including the imaging line generated by mapping to the vertical line image imaging data, the width Δx _{2 of} the area including the imaging line, the height of the area including the imaging line Δy ₂ , and imaging data of an area surrounding the imaging line by points r _A , r _B , r _C , and r _D , and stored in the vertical line mapping area data storage unit 43 for each imaging line. The horizontal line mapping area is also obtained in the same procedure as that for the vertical line, and is stored in the horizontal line mapping area data storage unit 44 with the same data structure.

[3] Processing in Approximate Function Setting Unit 34 The approximate function setting unit 34 represents each imaging line based on the mapping area data stored in the vertical line mapping area data storage unit 43 and the horizontal line mapping area data storage unit 44. For example, in the case of vertical line mapping area data, the approximation function is set by performing the processing shown in the flowchart shown in FIG.
(1) First, the approximate function setting unit 34 acquires vertical line mapping area data from the vertical line mapping area data storage unit 43 (processing S18).

(2) Next, the approximate function setting unit 34 acquires the position of the imaging line generated by mapping the vertical line of the vertical line image data to the vertical line image imaging data (processing S19). As shown in FIG. 9, the acquisition of the position of the imaging line is performed by obtaining the extent of the horizontal position x of the vertical line image imaging data at a certain vertical position y for the vertical line mapping area data such as the image A10. Based on this, the representative value of the horizontal position x at each vertical position y is obtained. In the present embodiment, as shown in the image A11 in FIG. 9, bright spots indicating the imaging ray with respect to a certain vertical position y _{j + 17} has a spread of up to x _{i + 7 ~x i + 11} in the horizontal direction Therefore, the horizontal position x with respect to the vertical position y _{j + 17} is an average value of these five points, and this is calculated for all the vertical positions y in the vertical line image data, and an image line such as A12 in FIG. (X, y) data group is obtained.

(3) Then, the approximate function setting unit 34 determines the position of the imaging line generated by mapping the vertical line of the vertical line image data in the vertical line mapping area data to the vertical line image imaging data, and the vertical line image imaging data. Is applied to an approximation function that represents the horizontal position X of the vertical line image pickup data with a polynomial related to the vertical direction Y (process S20). Specifically, this approximate function expresses the horizontal position X with a polynomial of the vertical position Y, the degree of the polynomial is M, the number of position coordinates on the imaging line is N, and the imaging data horizontal component of the position coordinates on the imaging line X, y is the vertical component of the imaging data at the position coordinate on the imaging line, μx is the average value of the horizontal component of the imaging data at the position coordinate on the imaging line, and μy is the average value of the vertical component of the imaging data on the imaging line If the standard deviation of the horizontal component of the imaging data at the position coordinates on the imaging line is σx and the standard deviation of the vertical component of the imaging data on the imaging line is σy, the relationship between the horizontal position X and the vertical position Y is as follows: It is given by equation (5).

(4) Finally, the approximate function setting unit 34 acquires the obtained approximate expression (5) as vertical line mapping data together with the vertical line position w on the vertical line image data, and stores it in the vertical line mapping data storage unit 45. (Processing S21).
In the case of the horizontal line mapping area data, the horizontal line mapping data is acquired by the same procedure as the vertical line mapping area data. In this case, when acquiring the data group corresponding to the image A10, each horizontal position x The vertical position Y is expressed by a polynomial of the horizontal position X by obtaining the range of the spread of the vertical position y with respect to and calculating the average value of the vertical positions y. Specifically, the approximate expression (6) given by the following expression (6) is stored in the horizontal line mapping data storage unit 46 together with the horizontal line position h of the horizontal line image data.

[4] Processing in Pixel Mapping Calculation Unit 35 The pixel mapping calculation unit 35 is displayed on the liquid crystal panel or the like constituting the projector 1 based on the vertical line mapping data and the horizontal line mapping data obtained by the approximate function setting unit 34. This is a part for calculating where the pixel position of the display image is mapped in the imaging data obtained by imaging the pixel position. Specifically, the processing shown in the flowchart of FIG. 10 is performed.
(1) First, the pixel mapping calculation unit 35 acquires the vertical line mapping data stored in the vertical line mapping data storage unit 45 (processing S22), and subsequently the horizontal line mapping data stored in the horizontal line mapping data storage unit 46. Is acquired (step S23).

(2) Next, the pixel mapping calculating means 35 calculates the intersection of the vertical line mapping data and the horizontal line mapping data (processing S24), and from the horizontal line mapping data, the vertical line mapping data, and the intersection of the two data, A position where each pixel position of the display image displayed by the liquid crystal panel of the fixed pixel type display is mapped on the imaging data within the quadrangular region constituted by the mapping data and the horizontal line mapping data is calculated (processing S25). . Specifically, in the present embodiment, as shown in FIG. 11, the intersection c _aa of the imaging lines p _a and p _{b on} the horizontal line imaging data and the imaging lines q _a and q _b on the vertical imaging data. , C _ab , c _ba , c _bb (image A13), horizontal lines P _a , P _b on the horizontal line image data displayed by the liquid crystal panel of the fixed pixel type display, etc., vertical lines Q _a on the vertical line image data, Q _b of intersection _{C aa, C ab, C ba} , C bb performs association (image A14) (A15), position of the imaging data from the correspondence between each intersection (x, y) is the pixel position of the display image The interpolation function to be mapped to (X, Y) is calculated (A16).

(3) The interpolation function is calculated by the pixel mapping calculating means 35. Specifically, the coordinates (x, y) of an arbitrary point c _αβ in the quadrilateral region in the image A13 are the coordinates of the intersection c _aa (x _aa , _{y aa),} intersection _{c ab} coordinates _{(x ab, y ab),} intersection _{c ba} coordinates _{(x ba, y ba),} the coordinates of the intersection point _{c bb (x bb,} using _{y bb)} the following formula ( 7) and can be expressed by equation (8).

Here, the coefficients N _aa , N _ba , N _ab , and N _bb are the horizontal distance between the intersection points C _aa and C _ba in the quadrilateral region of the display image, and the vertical direction distance between the intersection points C _ba and C _bb. Is given by the following equations (9) to (12). That is, as a result, the coordinates (x, y) of an arbitrary point c _αβ of the quadrilateral area mapped on the imaging data shown in the image A13 correspond to the quadrilateral area on the display image shown in the image A14. It can be expressed as the coordinates (X, Y) of the point C _αβ , and an interpolation function that gives a correspondence between the pixel position on the display image and the mapping position where the pixel position is mapped on the imaging data can be calculated.

(4) The pixel mapping calculation means 35 performs an arbitrary mapping position in all quadrilateral areas mapped on the imaging data and an arbitrary pixel position in all quadrilateral areas on the display image by the above procedure. An interpolation function that gives a correspondence with is calculated and stored in the pixel mapping data storage unit 47 as pixel mapping data (processing S26).

[5] Process in Pixel Characteristic Value Acquisition Unit 36 The pixel characteristic value acquisition unit 36 stores the characteristic value acquisition imaging data stored in the characteristic value acquisition imaging data storage unit 42 and the pixel mapping data storage unit 47. This is a part for acquiring the characteristic value of each pixel on the display image based on the pixel mapping data, and the pixel characteristic value is acquired in the following procedure.
(1) First, the pixel characteristic value acquisition unit 36 acquires characteristic value acquisition imaging data from the characteristic value acquisition imaging data storage unit 42, and acquires characteristic values on the imaging data. As the characteristic value, for example, numerical data such as a luminance value and color unevenness is given.

(2) Next, the pixel characteristic value acquisition unit 36 acquires pixel mapping data from the pixel mapping data storage unit 47, and each pixel position on the display image and a mapping position where the pixel position is mapped on the imaging data For each pixel position on the display image, characteristic value data such as a luminance value and color unevenness at the position where the pixel position is mapped on the imaging data is acquired.
(3) Finally, the pixel characteristic value acquisition unit 36 accumulates characteristic value data corresponding to all pixel positions on the display image in the pixel characteristic value data accumulation unit 48 as pixel characteristic value data.

[6] Operation of this embodiment Next, the overall flow of the pixel characteristic value acquisition method implemented by the above-described functional means will be described based on the flowchart shown in FIG.
(1) The display image forming unit 32 causes the projector 100 to sequentially display the image based on the vertical line image data, the horizontal line image data, and the characteristic value acquisition image data (processing S27), and displays the image data on the screen 101. The displayed display image is picked up by the image pickup device 2, and sequentially taken as picked-up data by the display image pick-up means 31 (processing S28), the vertical line image pick-up data storage unit 40, the horizontal line image pick-up data storage unit 41, and the characteristic value acquisition Recorded and stored in the imaging data storage unit 42 (process S29).
(2) When it is determined by the display image capturing means 31 that the capturing of all the imaging data has been completed (processing S30), the mapping area acquiring means 33 acquires the vertical line image data and the horizontal line image data (processing S31). In addition, vertical line image capturing data and horizontal line image capturing data are acquired (step S32).

(3) The mapping area acquisition unit 33 first grasps where the marker position set on the vertical line image data is mapped on the vertical line image imaging data, and determines the correspondence between the marker position and the mapping position. Expressions (3) and (4) are calculated (step S33). Then, when Expression (3) and Expression (4) are calculated, the mapping area acquisition unit 33 sets an area surrounding each straight line set on the vertical line image data, and based on this, Expression (3) Each imaging line is cut out by the area mapped on the imaging data converted by the equation (4) (processing S34), and the obtained vertical line mapping area data is stored in the vertical line mapping area data storage unit 43. (Processing S35).
(4) Similarly, with respect to horizontal line image data and horizontal line image imaging data, each imaging line in the imaging data is cut out based on Expression (3) and Expression (4), and is stored as horizontal line mapping area data. The process is repeated until mapping area data is acquired for all the imaging lines of the horizontal line (process S36).

(5) When the mapping area data acquisition by the mapping area acquisition unit 33 is completed, the approximate function setting unit 34 acquires the vertical line mapping area data and the horizontal line mapping area data (processing S37).
(6) The mapping area acquisition means 33 acquires the position of the imaging line that gives each imaging line stored as the vertical line mapping area data (processing S38), and subsequently approximates this imaging line by Expression (5). Fitting of an approximate function of a given polynomial is performed (processing S39).
(7) The obtained approximate function is stored in the vertical line mapping data storage unit 45 as vertical line mapping data (processing S40).
(8) Based on the horizontal line mapping area data, the mapping area acquisition unit 33 sets an approximation function in the same procedure using the equation (6) for applying the imaging line in the horizontal line mapping data, and sets all mapping areas. When the approximation function setting and the mapping data accumulation for the imaging line on the data are completed (processing S41), the processing is terminated.

(9) Next, the pixel mapping calculation means 35 acquires the vertical line mapping data and the horizontal line mapping data (processing S42), and based on the approximate function that gives each vertical line mapping data and the approximate function that gives each horizontal line mapping data. The intersection of the vertical line imaging line and the horizontal line imaging line is calculated (processing S43).
(10) The pixel mapping calculating means 35 is configured to select an arbitrary point in the quadrilateral area generated by the vertical line mapping data and the horizontal line mapping data, and the quadrilateral area by the vertical line image data and the horizontal line image data corresponding to this point. An interpolation function that gives a correspondence relationship with the point is calculated based on the equations (7) and (8) (processing S44).
(11) The pixel mapping calculation means 35 accumulates the calculated interpolation function as pixel mapping data (process S45), and repeats until the interpolation functions for all the vertical line and horizontal line areas are obtained (process S46).

(12) When the calculation of the interpolation function by the pixel mapping calculation unit 35 is completed, the pixel characteristic value acquisition unit 36 acquires the characteristic value acquisition imaging data from the characteristic value acquisition imaging data storage unit 42 (processing S47). Pixel mapping data is acquired from the mapping data storage unit 47 (processing S48), each pixel on the display image is associated with the characteristic value (processing S49), and stored in the pixel characteristic value data storage unit 48 as pixel characteristic value data. (Processing S50).
According to the present embodiment, since vertical line data and horizontal line data are employed as image data for pixel position acquisition, the pixel position on the display image and the pixel position caused by the imaging device 2 are imaged. It is possible to accurately grasp the complex distortion that occurs between the position mapped on the data in the form of an approximate function and enhance the correspondence between the pixel position of the display image and the position where the pixel position is mapped to the imaging data. It can be obtained with accuracy.
Further, since the high-accuracy correspondence can be grasped by the above-described equations (1) to (12), it is not necessary to perform complicated arithmetic processing, and the high-precision correspondence can be easily obtained.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the following description, the description of the same parts as those already described is omitted.
In the first embodiment described above, the pixel mapping value is obtained by the pixel characteristic value obtaining unit 36 immediately after the pixel mapping data is calculated by the pixel mapping calculating unit 35.
On the other hand, in the pixel characteristic value acquisition system 4 according to the second embodiment, as shown in FIG. 13, the processing end determination unit 49 is provided in the computer 4, and it is determined that the pixel mapping data is not appropriate. In this case, image data is displayed again from the display, pixel mapping data is calculated by the pixel mapping calculation means 50, and the processing is repeated until appropriate pixel mapping data is obtained.

In the first embodiment described above, the pixel mapping calculation unit 35 calculates the coordinates of an arbitrary point in the quadrilateral region using only the intersection of the vertical line and the horizontal line. On the other hand, as shown in FIG. 14, the pixel mapping calculation unit 50 according to the second embodiment calculates the coordinates of an arbitrary point in the quadrilateral area. The difference is that points are used.
Hereinafter, these differences will be described in detail.

First, as shown in FIG. 14, the pixel mapping calculation unit 50 calculates an arbitrary point c _αβ in the quadrilateral region on the imaging data shown in the image A18 and the image data shown in the image A19 corresponding thereto. When calculating the interpolation function that gives the corresponding relationship with the point C _αβ , the intersection point c _aa , c _ab , c _ba , c _bb (A20) as in the first embodiment, and the middle point c _{pa of} each intersection point , C _pb , c _qa , c _qb are used, and the interpolation function is calculated after _obtaining the corresponding relationship with the points C _pa , C _pb , C _pb , C _qb on the corresponding image data. (A21).

Here, the x coordinate of the point c _pa is a value that equally divides the x coordinates of the intersections c _aa and c _ba , the y coordinate is calculated from an approximate function of the horizontal line mapping data, and the points c _pb , c _qa , and c _qb are also It is calculated similarly.
Such intersection and any point c _.alpha..beta coordinates of a quadrilateral area in the captured data using the midpoint (x, y) is the coordinate value of each point _{c ij (x ij, y ij} ) and Then, the interpolation function is given by the following equations (13) and (14).

The coefficients N _aa , N _ba , N _ab , N _bb , N _pa , N _pb , N _qa , N _qb in the equations (13) and (14) are the intersection points C _aa , C _{ba of the} image data shown in the image A19. When the distance between them is ΔW, C _aa , and the distance between C _ab is ΔH, the following equations (15) to (22) are obtained.

  The pixel mapping calculation means 50 obtains an interpolation function that gives a correspondence between an arbitrary position in the quadrilateral area mapped on the imaging data in this way and the pixel position of the display image, thereby increasing the number of sample points to be compared. Therefore, it is possible to obtain the correspondence between arbitrary positions with higher accuracy.

  Next, the processing end determination unit 49 is a part that generates pixel mapping data again when it is determined that the accuracy of the pixel mapping data obtained by the pixel mapping calculation unit 50 is insufficient. The end of processing is determined based on the evaluation value Error given by equation (23). The evaluation value Error may be performed for all pixels, but N representative points may be set in advance, and it may be determined how the evaluation value changes at the representative point.

Here, the evaluation value Error is calculated as a difference between the pixel mapping data acquired a plurality of times, and the processing end determination unit 49 determines whether the evaluation value Error has become sufficiently small or the evaluation value Error after the previous processing. If the difference from the latest evaluation value Error after processing becomes sufficiently small, the processing is terminated.
On the other hand, when the process end determination unit 49 determines that the evaluation value Error is not sufficiently small and the difference from the previous time is large, the process end determination unit 49 displays the first vertical line display as shown in FIG. Image data in which the vertical line pitch of the image A22 is set to double as in the image A23 and image data in which the vertical line pitch is set to unequal intervals as in the image A24 are reconstructed and input to the projector 100. Then, a display image based on this is displayed. The horizontal line image data is similarly reconstructed.

Next, the operation of the pixel characteristic value acquisition system 4 according to the second embodiment will be described based on the flowchart shown in FIG.
(1) First, the display image forming unit 32 inputs the vertical line image data, the horizontal line image data, and the characteristic value acquisition image data to the projector 100 and displays each display image on the screen 101 (processing S51). ) The display image capturing means 31 captures and captures each display image (processing S52).

(2) Next, the mapping area acquisition unit 33 acquires a mapping area (process S53), and based on the acquired mapping area, the approximate function setting unit 34 acquires mapping data (process S54). .
(3) Then, the pixel mapping calculating unit 50 calculates pixel mapping data by the above-described processing (processing S55). Up to this point, the procedure is the same as in the first embodiment.
(4) The processing end determination means 49 determines whether or not the previously calculated pixel mapping data exists (processing S56). If there is no processing, the reconfiguration of the image data such as the image A23 or the image A24 is performed. (Step S57), this is input to the projector 100, and the above-described Steps S51 to S55 are repeated.

(5) The processing end determination means 49 uses the pixel mapping data based on the reconstructed image data obtained by repeating the processes S51 to S55 and the evaluation value Error based on the equation (23) based on the previous pixel mapping data. Is calculated (process S58), and it is determined whether or not the evaluation value Error is equal to or less than a predetermined threshold (process S59).
(6) If it is determined that the evaluation value Error is not less than or equal to the predetermined threshold value, the processing end determination unit 49 reconstructs image data such as the image A23 or the image A24 and supplies it to the projector 100. The input is repeated and processing S51 to S55 is repeated to calculate new pixel mapping data. On the other hand, if it is determined that the threshold value is equal to or less than the predetermined threshold value, the process ends, and the pixel characteristic value acquisition unit 36 starts acquiring pixel characteristic values based on the latest pixel mapping data (process S60).

[Modification of Embodiment]
It should be noted that the present invention is not limited to the above-described embodiments, and modifications, improvements, and the like within the scope that can achieve the object of the present invention are included in the present invention.
In the above-described embodiment, the pixel position acquisition method according to the present invention is used to acquire the pixel characteristic value of the display image by the projector 100. However, the present invention is not limited to this, and a direct-view type liquid crystal display, plasma display, organic display, etc. In order to acquire pixel characteristic values of an EL display or the like with high accuracy, the pixel position acquisition method according to the present invention may be employed.
In addition, the specific structure, shape, and the like when implementing the present invention may be other structures as long as the object of the present invention can be achieved.

The schematic diagram showing the structure of the pixel characteristic value acquisition system which concerns on 1st Embodiment of this invention. 6 is a flowchart showing processing in a display image capturing unit and a display image forming unit in the embodiment. The schematic diagram showing the structure of vertical line image data. The schematic diagram showing the structure of horizontal line image data. The flowchart showing the process in a mapping area | region acquisition means. The schematic diagram for demonstrating the process by a mapping area | region acquisition means. The schematic diagram for demonstrating the process by a mapping area | region acquisition means. The flowchart showing the process in an approximate function setting means. The schematic diagram for demonstrating the process by an approximate function setting means. The flowchart showing the process in a pixel mapping calculation means. The schematic diagram for demonstrating the process by a pixel mapping calculation means. The flowchart showing the effect | action of the said embodiment. The schematic diagram showing the structure of the pixel characteristic value acquisition system which concerns on 2nd Embodiment of this invention. The schematic diagram for demonstrating the process by the pixel mapping calculation means in the said embodiment. The schematic diagram for demonstrating the reconstruction of the image data in the said embodiment. The flowchart showing the effect | action of the said embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Pixel characteristic value acquisition system, 31 ... Display image capture means, 32 ... Display image formation means, 33 ... Mapping area acquisition means, 34 ... Approximation function setting means, 35, 50 ... Pixel mapping calculation means, 49 ... Processing end Judgment means

Claims (10)

A display image displayed on the screen is picked up by an image pickup device, and based on the picked-up image pickup data, the pixel position of the image generation device projected on the screen and the pixel position are mapped onto the image pickup data. A pixel position acquisition method for acquiring a correspondence relationship with a determined position,
Displaying a first line image composed of a plurality of straight lines extending along a vertical direction of the image generating device on the screen;
A procedure of capturing a display image based on the first line image by the imaging device and acquiring the captured first imaging data;
Displaying a second line image consisting of a plurality of straight lines extending in the horizontal direction of the image generating device on the screen;
A procedure of capturing a display image based on the second line image by the imaging device and acquiring captured second imaging data;
A procedure of cutting out a plurality of imaging lines mapped on the first imaging data in units of areas including the respective imaging lines;
A procedure for setting an approximation function representing each imaging line in the first imaging data based on the area of each extracted imaging line;
A procedure of cutting out a plurality of imaging lines mapped on the second imaging data in units of areas including the respective imaging lines;
A procedure for setting an approximate function representing each imaging line in the second imaging data based on the area of each extracted imaging line;
Based on the approximate function set by the first imaging data and the second imaging data, an association between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data is obtained. And a pixel position acquisition method. The pixel position acquisition method according to claim 1,
The first line image and the second line image are configured to include at least four markers that give correspondence with respective imaging data,
The procedure of cutting out a plurality of imaging lines mapped on the first or second imaging data as an area including each imaging line,
Based on the correspondence between the first line image and the marker of the first imaging data , the pixel position of the first line image of the first line image and the first imaging data is represented by the imaging data. Setting a conversion expression to be converted to the mapping position of
Based on the correspondence between the second line image and the marker of the second imaging data , the pixel position of the second line image of the second line image and the second imaging data is the imaging data. Setting a conversion expression to be converted to the mapping position of
Setting an area surrounding each straight line in the first or second line image;
Converting a region surrounding each straight line in the first or second line image by the set conversion equation;
And a step of cutting out each imaging line of the first imaging data or the second imaging data based on the converted area. In the pixel position acquisition method according to claim 1 or 2,
The procedure for setting an approximation function representing each imaging line is as follows.
When setting an xy coordinate system in which the vertical direction of the first imaging data or the second imaging data is the y-axis and the horizontal direction is the x-axis,
Each imaging line mapped on the first imaging data is set as an approximate function in which x is expressed as a polynomial of y,
Each imaging line mapped on the second imaging data is set as an approximate function in which y is expressed as a polynomial of x, and a pixel position acquisition method. In the pixel position acquisition method according to any one of claims 1 to 3,
The procedure for obtaining the correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data is as follows:
Correspondence between each straight line on the first line image and each imaging line on the first imaging data, and each imaging on each straight line on the second line image and the second imaging data Associating with a line;
Obtaining intersections of a plurality of imaging lines on the first imaging data and a plurality of imaging lines on the second imaging data based on an approximation function set in accordance with each associated imaging line; ,
Based on each obtained intersection point, an arbitrary area within a region surrounded by two adjacent imaging lines on the first imaging data and two adjacent imaging lines on the second imaging data And a step of setting an interpolation function for giving a point. In the pixel position acquisition method according to any one of claims 1 to 4,
After the procedure for obtaining the correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data,
The correspondence between the pixel position of the image generation device already calculated and the position where the pixel position is mapped on the imaging data, and the pixel position and the pixel position of the image generation device calculated again are the imaging. Based on the correspondence with the position mapped on the data, it has a procedure to evaluate pass / fail,
If not, from the procedure for displaying the first line image again on the screen to the procedure for obtaining the correspondence between the pixel position of the image generation device and the position where the pixel position is mapped on the imaging data The pixel position acquisition method characterized by repeating. In the pixel position acquisition method according to claim 5,
At least one of the procedure of displaying the first line image and the procedure of displaying the second line image at the time of the repetition displays the updated line image by narrowing the interval of each straight line from the previous time. A pixel position acquisition method. In the pixel position acquisition method according to claim 5,
At least one of the procedure of displaying the first line image and the procedure of displaying the second line image at the time of repetition is characterized by displaying a line image in which each straight line is arranged at a position different from the previous time. Pixel position acquisition method. A display image displayed on the screen is picked up by an image pickup device, and based on the picked-up image pickup data, the pixel position of the image generation device projected on the screen and the pixel position are mapped onto the image pickup data. An image processing apparatus for acquiring a correspondence relationship with a position,
An image processing apparatus that executes each procedure constituting the pixel position acquisition method according to claim 1. A display image displayed on the screen is picked up by an image pickup device, and based on the picked-up image pickup data, the pixel position of the image generation device projected on the screen and the pixel position are mapped onto the image pickup data. A program for executing on a computer a pixel position acquisition method for acquiring a correspondence relationship with a position,
A program for executing the steps of the pixel position acquisition method according to any one of claims 1 to 7.   A computer-readable recording medium on which the program according to claim 9 is recorded.

Images