CN104168467B - Use the method that time series structured light technique realizes Projection Display geometric correction - Google Patents
Use the method that time series structured light technique realizes Projection Display geometric correction Download PDFInfo
- Publication number
- CN104168467B CN104168467B CN201410442211.0A CN201410442211A CN104168467B CN 104168467 B CN104168467 B CN 104168467B CN 201410442211 A CN201410442211 A CN 201410442211A CN 104168467 B CN104168467 B CN 104168467B
- Authority
- CN
- China
- Prior art keywords
- image
- original image
- projector
- cache
- projection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 31
- 238000012937 correction Methods 0.000 title claims abstract description 28
- 238000013507 mapping Methods 0.000 claims abstract description 37
- 238000005516 engineering process Methods 0.000 claims description 9
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 2
- 230000001360 synchronised effect Effects 0.000 claims 1
- 230000001788 irregular Effects 0.000 abstract description 7
- 238000001514 detection method Methods 0.000 description 12
- 239000011159 matrix material Substances 0.000 description 7
- 230000010363 phase shift Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 241000764238 Isis Species 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000013139 quantization Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000002411 adverse Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000001314 profilometry Methods 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000011426 transformation method Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
Landscapes
- Image Processing (AREA)
- Projection Apparatus (AREA)
- Transforming Electric Information Into Light Information (AREA)
- Controls And Circuits For Display Device (AREA)
Abstract
The present invention be directed to a kind of method using time series structured light technique to realize Projection Display geometric correction of irregular projection surface. time series structured light technique is used to obtain the coordinate corresponding relation of camera image and scialyscope cache image, camera image is determined projection effective coverage, final image viewing area is calculated in projection effective coverage, solve the homography relation of final image viewing area and original image, coordinate corresponding relation according to this homography relation and camera image and scialyscope cache image sets up the original image nonlinear discrete mapping relations to scialyscope cache image, according to Discrete Mapping point, original image and scialyscope buffer memory plane net are formatted, last mapping at corresponding net region employing texture transforms to scialyscope buffer memory plane original image. the present invention is useful for the Projection Display geometric correction of irregular surface. the present invention has the advantage that geometric correction precision is high, computational efficiency is high, easy and simple to handle.
Description
Technical Field
The invention relates to a projection display geometric correction technology in multi-projector splicing, in particular to geometric correction aiming at an irregular projection surface based on a time series structured light technology.
Background
When the projection is carried out on the irregular three-dimensional surface, the display image can generate geometric distortion under the influence of the three-dimensional shape of the projection surface and the arrangement position and orientation of the projector. The computer vision technology is utilized to carry out geometric correction on the projector cache image, so that human eyes can see a projected image without geometric distortion when observing the projected image at a specific position and a specific visual angle. The technique can be used for geometric correction problems in projection stitching. The geometric correction technology of three-dimensional surface projection enables the projection environment to get rid of the limitation of a special projection screen, and the geometric correction technology is used as a key technology of multi-projection splicing and widely applied to high-resolution, large-screen and immersive display, such as airport tower simulation systems, remote video conference display, office interactive display environments and the like. Roselli et al in the paper "feature point detection and geometry correction in a multi-projector autostereoscopic display system [ J ]. Journal of electronics, 2010, 38 (8): 1729-. Firstly, projecting and capturing a circular characteristic point array; detecting the characteristic points of the shot characteristic point array, detecting a characteristic circle by adopting a circle detection method based on Hough transformation, and then correcting the positions of the characteristic points by using a least square straight line fitting method; establishing grids in one-to-one correspondence between the camera space and the projector cache space by using the detected feature points; and finally, mapping the expected image texture to the projector cache space according to the one-to-one mapping relation among the corresponding grids to realize geometric correction. The geometric correction method based on the projected feature pattern represented by the above method has the following disadvantages: firstly, the method for detecting the feature points is greatly interfered by a projection environment, common interferences such as a display surface with inconsistent reflectivity, background light and irrelevant objects in the projection environment can all cause adverse effects on the detection of the feature points, and the interferences can cause false detection and missed detection of the feature points, thereby bringing inconvenience to subsequent steps such as grid construction. Secondly, as the established mapping grid is finer, the mapping relation between the camera and the projector is more accurate, the geometric correction precision is higher, however, the number of the characteristic points obtained by characteristic point detection is limited, so that the compactness degree of the obtained mapping grid is not high, and the geometric correction precision is directly influenced. Thirdly, the deformed non-standard circle shot by the Hough transformation method is detected, and although the correction is carried out in the later processing, the position of the obtained circle center can not avoid the existence of errors. Fourthly, the method for constructing the grid by using the characteristic points is complex in operation and large in calculation amount, comprises the operations of sequencing, straight line fitting, judging and removing false detection points, inserting missed detection points and the like, and is not beneficial to the rapidity of projection geometric correction. The key technical problem can be solved by applying the scheme provided by the invention, and the defects of low detection accuracy of the characteristic points, poor anti-interference performance, insufficiently compact mapping grid, complex operation in the gridding process and low calculation efficiency of the existing method are overcome.
Disclosure of Invention
The invention aims to provide a projection display geometric correction method using a time series structured light technology, which aims at the defects of low detection accuracy of characteristic points, poor anti-interference performance, insufficiently compact mapping grids, complex operation in a gridding process and low calculation efficiency of the conventional projection display geometric correction method.
The purpose of the invention is realized by adopting the following technical scheme:
projecting a structured light pattern sequence in the horizontal direction and the vertical direction to a display surface by using a projection device, synchronously shooting by using a camera device, obtaining a projector cache image coordinate corresponding to each pixel point of a camera image by decoding, determining a projection effective area in the camera image, and finding a maximum inscribed rectangle with the same horizontal-vertical ratio as an original image in the projection effective area as a final image display areaSolving forHomography with original imageAccording toAnd establishing a nonlinear discrete mapping relation from an original image to a projector cache image according to the coordinate corresponding relation between the camera image and the projector cache image, dividing the original image coordinate plane into square grids by taking the upper, lower, left and right adjacent 4 mapping points as a group according to the characteristic that the discrete points mapped into the original image are arranged at equal intervals, determining the corresponding quadrilateral grid division in the projector cache image according to the discrete mapping relation from the original image to the projector cache image, and transforming the original image to the projector cache plane by adopting a texture mapping method in the corresponding grid area.
Compared with the prior art, the invention has the following advantages:
because the corresponding point of each pixel point of the final rectangular display area in the camera space is used as a node to construct a mapping grid, under the condition of a certain number of camera pixels, the grid division method is the finest division mode theoretically, and can achieve high geometric correction precision; compared with the gridding mapping method based on the characteristic points, the grid established by the method is higher in fineness degree, the grid establishing process is simpler, more convenient and more direct, the complex operations of sequencing, straight line fitting, judgment, removal of false detection points, insertion of missed detection points and the like are not needed, and higher computing efficiency and accuracy can be achieved.
Drawings
FIG. 1 is a diagram of a projection geometry correction system according to the present invention.
FIG. 2 is a flowchart illustrating a method for correcting projection geometry according to the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
Fig. 1 is a diagram of a projection display geometry correction system, in which 101 is a camera, 102 is a projector, 103 is a computer, and 104 is an irregular projection display surface. The adopted device has 1 CASIOXJ-M140 projector, and the buffer frame size of the projector isThe gray quantization level of the pixel is 8 bits, and the maximum output frequency of the projector is 150 frames/s; 1 ProsilicaGC650 industrial camera with resolution ofThe pixel, the grey scale quantization level is 8bit, and the maximum capture frequency of the camera is 62 frames/s. 1 computer with Corei33530CPU, 4GB memory. An irregular curved surface formed by twisting large-size paper is used as a projection display surface. And writing a main program by adopting C + + in a Microsoft visual studio2010 environment, and finishing texture mapping by applying OpenGL. Fig. 2 is a flowchart of a projection display geometry correction method according to this embodiment. The camera is fixed with the projector, and the camera can catch the whole projection area on the display surface. The size of the original image is known. Projector buffer image size is expressed asHaving pixel coordinates of(ii) a The camera image has pixel coordinates of. The embodiment aims at the geometric correction of the single projection, and the specific implementation steps are as follows:
(1) scanning the display surface horizontally and vertically by Phase Measurement Profilometry (PMP) to obtain the horizontal phaseAnd vertical phaseAnd brightness modulation。
The PMP principle is described in detail in the article "LiuK, WangY, LauDL, et. Dual-frequency authentication for high-speed 3-Dshapemeassurement [ J ]. Opt. express,2010,18(5): 5229-:
projecting a sinusoidal structured light pattern with a projector onto a display surface, the projected pattern being represented as:
(1)
(2)
wherein,andscanning the pattern in the horizontal and vertical directions, respectivelyWhere f is the spatial frequency of the structured light pattern, N is the total number of phase shifts per direction, N is the phase shift coefficient,andis constant and satisfies. The parameters of the sinusoidal structured light pattern of this example areSpatial frequency f =64, and the total number of phase shifts N =8 per direction.
Synchronously capturing a structured light image with a camera, the captured image being represented as:
（3）
the variable i is a structured light image sequence corresponding to phase shift in the horizontal direction and the vertical direction respectively when 1 and 2 are taken;、、andrespectively show the sequence of the shot imagesBrightness value, average brightness value, brightness modulation and initial phase.
Finally, the corresponding pixel of the camera image is obtained by decoding the shot structured light image sequenceHorizontal phase ofAnd vertical phaseAnd brightness modulation. The decoding formula is:
（4）
（5）
(2) using brightness modulation mapsAnd determining a projection effective area in the camera image.
In the brightness modulation diagramIn the middle, the value of the pixel point having the corresponding relation with the projector cache image is larger, and the values of other pixel points are close to 0. Without loss of generality, we modulate the brightness of the vertical phase-shift structured light patternAnd performing threshold segmentation to determine a region which has a mapping relation with the projector in the camera image, namely a projection effective region. I.e. setting a threshold valueHere, theTake a value ofArea of camera coordinatesAnd judging as a projection effective area.
(3) Determining a final image display area within a projected effective area in a camera image。
The effective projection area in the camera image is an irregular polygon, and a rectangular area needs to be determined in the area for displaying the final projection image. And finding a maximum inscribed rectangle with the same aspect ratio as the original image in the projection effective area by a heuristic method. The above heuristic is described in the paper "van BaarJ, RaskarR, BaarJ, et]Berlin, German: Springer,2002,3 (3)'. The maximum inscribed rectangle area is represented asIs provided withHas coordinates of the upper left vertex ofWidth ofHeight of。
(4) Solving for rectangular regions in camera spaceHomography relation with the original image S.
Due to the fact thatHas a scaling and translation relation with the original image, soThe transformation relationship to the original image can be represented by a homography matrix, with homogeneous coordinates in the form:
（6）
wherein,is the coordinates of the original image and is,is the coordinates of the image of the camera,is a homogeneous coordinate scale factor and is,is an element of a homography matrix S, and. Using 4 vertex coordinates and rectangular regions known from the original imageDomainThe homography transformation matrix S can be calculated from the 4 corresponding vertex coordinates.
(5) Building rectangular regionsThe abscissa lookup tableAnd ordinate look-up table。
Coordinate lookup tableAndare all high asWide isCan be regarded as size and areaThe same graph. Lookup tableAndthe setup procedure of (a) can be expressed as:
（7）
（8）
wherein the integer isSatisfy the requirement of. Is provided withHas relative coordinates of the upper left vertex ofThen, thenAndrespectively store areasInner relative positionThe corresponding abscissa and ordinate in the camera image.
(6) By using、Establishing a rectangular region with a homography relation SMapping point abscissa lookup table in original imageAnd ordinate look-up table。
Lookup tableAndare all high asWide isIs used for the two-dimensional matrix of (1). The setup procedure can be expressed as:
（9）
wherein the integer isSatisfy the requirement of。Andrespectively store areasInner relative positionAnd mapping the horizontal coordinate and the vertical coordinate of the original image.
(7) By using、And horizontal phaseVertical phase ofBuilding rectangular regionsMapping point abscissa lookup table in projector cache imageAnd ordinate look-up table。
Lookup tableAndare all high asWide isIs used for the two-dimensional matrix of (1). The setup procedure can be represented by the following equation:
（10）
（11）
wherein the integer isSatisfy the requirement of。Andrespectively store areasInner relative positionAnd mapping to the abscissa and the ordinate of the projector after caching the image.
(8) From a look-up table、、、And obtaining a nonlinear discrete mapping relation from the original image to the projector cache image.
、Stores a rectangular display area in the camera imageCoordinates of corresponding positions of all pixel points in the original image; whileAndis arranged atAnd all the pixel points correspond to the point coordinates in the projector cache image. Therefore, the nonlinear discrete mapping relation from the original image to the projector cache image is indirectly obtained, and the coordinate mapping relation is expressed as:
（12）
wherein the integer isSatisfy the requirement ofTo co-ordinateThe mapped points.
(9) And gridding the original image and the projector cache plane according to the discrete mapping point.
The coordinate plane of the original image is divided into a square grid by using the characteristic that discrete points mapped into the original image are arranged at equal intervals, and the 4 mapping points which are adjacent from top to bottom, left to right form a group. The meshing of the original image may be represented by a set of four vertex coordinates of a single square lattice block. For a single grid blockOf (2) matrixExpressed as:
（13）
wherein, integer coordinateRepresenting a two-dimensional number of a lattice block, satisfies。Lines 1, 2,3, and 4 of (a) indicate the top left, top right, bottom right, and bottom left vertex coordinates of a grid block, respectively.
And determining the corresponding quadrilateral meshing division in the projector cache plane by utilizing the discrete mapping relation from the original image to the projector cache image. Similarly, a corresponding single quadrilateral lattice in the projector's cache plane may be represented asOf (2) matrix：
（14）
Similarly, integer coordinatesRepresenting a two-dimensional number of a lattice block, satisfies。The row of (b) indicates the vertex coordinates of the lattice block.
Then, the corresponding grid block between the original image plane and the projector buffer plane can be expressed as:
（15）
wherein,。
(10) and transforming the original image to a projector cache plane by adopting a texture mapping method corresponding to the grid area.
And taking the original image as a two-dimensional texture, and assigning texture coordinates to each grid block of the original image. Square grid blockThe corresponding 4 texture coordinates are respectively、、、WhereinAndrepresenting the width and height of the original image, respectively. And mapping pixels in each grid block of the original image into a corresponding quadrilateral grid area of a projector cache plane by using OpenGL in a texture mapping mode, and finally realizing pre-distortion from the original image to the projector cache image.
Obtaining the corresponding grid area with the mapping relation between the original image plane and the projector cache plane by the correction algorithm. When a new image with unchanged size is to be projected, the pre-correction of the new cached image of the projector can be achieved only by performing texture mapping on the pixels of the new image grid area to the grid area corresponding to the cache plane of the projector.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (1)
1. A projection display geometric correction method using time series structured light technology is characterized in that a projection device is used for projecting a sinusoidal structured light pattern sequence in the horizontal direction and the vertical direction to a display surface, a camera device is used for synchronous shooting, projector cache image coordinates corresponding to each pixel point of a camera image are obtained through decoding, a projection effective area is determined through threshold segmentation on brightness modulation of sinusoidal structured light in the camera image, and a maximum inscribed rectangle with the same horizontal-vertical ratio as an original image is found in the projection effective area to serve as a final image display areaSolving forHomography with original imageAccording toAnd establishing a nonlinear discrete mapping relation from an original image to a projector cache image according to the coordinate corresponding relation between the camera image and the projector cache image, dividing the original image coordinate plane into square grids by taking the upper, lower, left and right adjacent 4 mapping points as a group according to the characteristic that the discrete points mapped into the original image are arranged at equal intervals, determining the corresponding quadrilateral grid division in the projector cache image according to the discrete mapping relation from the original image to the projector cache image, and transforming the original image to the projector cache plane by adopting a texture mapping method in the corresponding grid area.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410442211.0A CN104168467B (en) | 2014-09-02 | 2014-09-02 | Use the method that time series structured light technique realizes Projection Display geometric correction |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410442211.0A CN104168467B (en) | 2014-09-02 | 2014-09-02 | Use the method that time series structured light technique realizes Projection Display geometric correction |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN104168467A CN104168467A (en) | 2014-11-26 |
| CN104168467B true CN104168467B (en) | 2016-06-15 |
Family
ID=51912072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410442211.0A Active CN104168467B (en) | 2014-09-02 | 2014-09-02 | Use the method that time series structured light technique realizes Projection Display geometric correction |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104168467B (en) |
Families Citing this family (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105554486A (en) * | 2015-12-22 | 2016-05-04 | Tcl集团股份有限公司 | Projection calibration method and device |
| CN106162124B (en) * | 2016-08-02 | 2018-06-08 | 上海唱风信息科技有限公司 | Projector exports the calibration method of image |
| CN106507077B (en) * | 2016-11-28 | 2018-07-24 | 江苏鸿信系统集成有限公司 | Preventing collision method is corrected and blocked to projecting apparatus picture based on image analysis |
| JP6702171B2 (en) * | 2016-12-22 | 2020-05-27 | カシオ計算機株式会社 | Projection control device, projection control method and program |
| CN110857855B (en) * | 2018-08-22 | 2022-09-02 | 杭州海康机器人技术有限公司 | Image data acquisition method, device and system |
| CN110996081B (en) * | 2019-12-06 | 2022-01-21 | 北京一数科技有限公司 | Projection picture correction method and device, electronic equipment and readable storage medium |
| CN112261396B (en) * | 2020-10-26 | 2022-02-25 | 成都极米科技股份有限公司 | Projection method, projection device, projection equipment and computer readable storage medium |
| JP7318669B2 (en) * | 2021-01-27 | 2023-08-01 | セイコーエプソン株式会社 | Display method and display system |
| CN115150595B (en) * | 2021-03-31 | 2023-09-26 | 成都极米科技股份有限公司 | Method and device for determining display area of projection picture and projector |
| CN114363600B (en) * | 2022-03-15 | 2022-06-21 | 视田科技（天津）有限公司 | Remote rapid 3D projection method and system based on structured light scanning |
| CN115314691B (en) * | 2022-08-09 | 2023-05-09 | 北京淳中科技股份有限公司 | Image geometric correction method and device, electronic equipment and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6536907B1 (en) * | 2000-02-08 | 2003-03-25 | Hewlett-Packard Development Company, L.P. | Aberration compensation in image projection displays |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103209311A (en) * | 2012-01-14 | 2013-07-17 | 复旦大学 | Multi-projection autocorrection display system based on camera |
| CN103019643B (en) * | 2012-12-30 | 2015-12-23 | 中国海洋大学 | A kind of large screen projection automatic calibration of plug and play and splicing display method |
-
2014
- 2014-09-02 CN CN201410442211.0A patent/CN104168467B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6536907B1 (en) * | 2000-02-08 | 2003-03-25 | Hewlett-Packard Development Company, L.P. | Aberration compensation in image projection displays |
Also Published As
| Publication number | Publication date |
|---|---|
| CN104168467A (en) | 2014-11-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104168467B (en) | Use the method that time series structured light technique realizes Projection Display geometric correction | |
| CN111192552B (en) | Multi-channel LED spherical screen geometric correction method | |
| CN104268922B (en) | A kind of image rendering method and image rendering device | |
| TWI486551B (en) | Method and system for three-dimensional data acquisition | |
| TWI493500B (en) | Image processing method and related apparatus for rendering two-dimensional image to show three-dimensional effect | |
| CN110830781B (en) | Automatic projected image correction method and system based on binocular vision | |
| CN104155765B (en) | The method and apparatus of revision for 3-D image in spliced integration imaging display | |
| CN108592884B (en) | A kind of general linear array satellite core line image generating method | |
| WO2017206222A1 (en) | Intelligent internet high-definition scanner with laser correction | |
| CN104036475A (en) | High-robustness geometric correction method adapted to random projector group and projection screen | |
| CN111882530B (en) | Sub-pixel positioning map generation method, positioning method and device | |
| CN106558017B (en) | Spherical display image processing method and system | |
| JP2015060012A (en) | Image processing system, image processing device, image processing method and image processing program as well as display system | |
| WO2020098721A1 (en) | Image presentation method and apparatus for virtual reality device, device and computer storage medium | |
| CN104111038A (en) | Method for using phase fusion algorithm to repair phase error caused by saturation | |
| CN109474814A (en) | Two-dimensional calibration method, projector and the calibration system of projector | |
| JP2013025077A (en) | Projector and control method for projector | |
| CN110008779A (en) | A kind of stereoscopic two-dimensional code processing method and processing device | |
| CN112734860A (en) | Arc-screen prior information-based pixel-by-pixel mapping projection geometric correction method | |
| US20090195758A1 (en) | Meshes for separately mapping color bands | |
| CN114697623A (en) | Projection surface selection and projection image correction method and device, projector and medium | |
| JP4751084B2 (en) | Mapping function generation method and apparatus, and composite video generation method and apparatus | |
| JP4554231B2 (en) | Distortion parameter generation method, video generation method, distortion parameter generation apparatus, and video generation apparatus | |
| CN106683057A (en) | Method for automatically geometrically correcting multi-projection display walls with optional continuous screens with curved surfaces | |
| CN109587463A (en) | Calibration method, projector and the calibration system of projector |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |