CN116264611A - Projection correction method, projection correction device, electronic equipment and storage medium - Google Patents
Projection correction method, projection correction device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN116264611A CN116264611A CN202111518634.2A CN202111518634A CN116264611A CN 116264611 A CN116264611 A CN 116264611A CN 202111518634 A CN202111518634 A CN 202111518634A CN 116264611 A CN116264611 A CN 116264611A
- Authority
- CN
- China
- Prior art keywords
- projection
- normalized
- dimensional information
- preset
- corner points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012937 correction Methods 0.000 title claims abstract description 123
- 238000000034 method Methods 0.000 title claims abstract description 61
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 21
- 230000009466 transformation Effects 0.000 claims description 21
- 238000004590 computer program Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 5
- 238000011084 recovery Methods 0.000 claims description 4
- 238000006243 chemical reaction Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims 1
- 238000004519 manufacturing process Methods 0.000 abstract description 4
- 238000004891 communication Methods 0.000 description 12
- 238000010586 diagram Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- 230000001788 irregular Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000003672 processing method Methods 0.000 description 2
- 238000007792 addition Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
- H04N9/3185—Geometric adjustment, e.g. keystone or convergence
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/02—Affine transformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/18—Image warping, e.g. rearranging pixels individually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/60—Rotation of whole images or parts thereof
- G06T3/608—Rotation of whole images or parts thereof by skew deformation, e.g. two-pass or three-pass rotation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
- G06T7/579—Depth or shape recovery from multiple images from motion
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Software Systems (AREA)
- Transforming Electric Information Into Light Information (AREA)
Abstract
The application discloses a projection correction method, a projection correction device, electronic equipment and a storage medium. According to the embodiment of the application, the m preset features are extracted from n projection scene images respectively by acquiring the images obtained by shooting the projection pictures in the physical space by the camera from different shooting angles, and then the m preset features are three-dimensionally reconstructed by the motion restoration structure three-dimensional reconstruction method, so that the normalized three-dimensional information of the m preset features in the physical space can be obtained, the normalized three-dimensional information of the projection pictures in the physical space is determined, correction parameters of the projection images are generated, after the correction parameters are sent to the projection equipment, the projection equipment can be enabled to project deformed projection images, so that the edge contours of the projection pictures are corrected to the preset shapes, the projection pictures can be corrected by the images shot by the camera of a third party, devices such as cameras are not required to be installed on the projection equipment, and the manufacturing cost of projection correction is reduced.
Description
Technical Field
The application belongs to the technical field of projection, and particularly relates to a projection correction method, a projection correction device, electronic equipment and a storage medium.
Background
The projection screen of the projection device is affected by the placement position of the projection device, and the shape of the projection screen may be trapezoidal, so that in order to correct the projection screen into a rectangle, a manual trapezoidal correction method or an automatic trapezoidal correction method is generally adopted in the prior art. The manual trapezoid correction is that a user manually adjusts the projection equipment, and the projection inclination angle is manually input, or the final display screen shape is manually adjusted to finish the calibration, so that the teaching is complex. In the prior art, devices such as a Time of flight (TOF) camera and the like, a projection light structure and the like are usually added on a projection device, three-dimensional reconstruction is performed through projection light, and the inclination angle of a projection picture is automatically measured, so that an automatic trapezoid correction function is realized. Therefore, a low cost projection correction method is needed.
Disclosure of Invention
The embodiment of the application provides a projection correction method, a projection correction device, electronic equipment and a storage medium, which can reduce the cost of projection correction.
In one aspect, an embodiment of the present application provides a projection correction method, including:
acquiring images obtained by a camera shooting projection pictures in a physical space from different shooting angles, and obtaining n projection scene images; the projection picture is a picture displayed on a projection plane by projecting a projection image comprising m preset features onto the projection plane by projection equipment, wherein n and m are positive integers greater than or equal to 2;
respectively extracting m preset features from n projection scene images;
based on m preset features extracted from n projection scene images, carrying out three-dimensional reconstruction on the m preset features by a motion recovery structure three-dimensional reconstruction method to obtain normalized three-dimensional information of the m preset features in a physical space;
combining the positions of m preset features in the projected image and the normalized three-dimensional information of the m preset features to determine the normalized three-dimensional information of the projected image in the physical space;
generating correction parameters for the projection image according to the normalized three-dimensional information of the projection picture; the correction parameters are used for deforming the projection image so as to enable the edge contour of the projection picture to be corrected to a preset shape;
and controlling the projection device to project the deformed projection image according to the correction parameters.
Optionally, generating correction parameters for the projection image according to the normalized three-dimensional information of the projection screen includes:
calculating the angle of a projection plane in a physical space according to the normalized three-dimensional information of four corner points of a projection picture;
according to the angle of the projection plane in the physical space, converting the normalized three-dimensional information of the four corner points of the projection plane into normalized two-dimensional information of the four corner points of the projection picture;
and generating correction parameters for the projection image according to the normalized two-dimensional information of the four corner points of the projection picture.
Optionally, generating correction parameters for the projection image according to the normalized two-dimensional information of the four corner points of the projection picture includes:
calculating normalized two-dimensional information of four corner points of the maximum inscribed rectangle of the projection picture according to the normalized two-dimensional information of the four corner points of the projection picture;
generating affine transformation parameters of the projection image, which enable the four corner points of the projection picture to be transformed to the four corner points of the maximum inscribed rectangle, according to the normalized two-dimensional information of the four corner points of the projection picture and the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle, and obtaining correction parameters of the projection image.
Optionally, after obtaining the correction parameters for the projection image, further comprising:
according to the angle of the projection plane in the physical space, converting the normalized three-dimensional information of m preset features into normalized two-dimensional information;
calculating ideal two-dimensional information of m preset features in a projection plane according to the normalized two-dimensional information of four corner points of a projection picture and the positions of the m preset features in the projection image;
calculating flatness deviation parameters of the projection plane at each preset feature according to deviation between the normalized two-dimensional information and the ideal two-dimensional information of each preset feature;
and correcting affine transformation parameters at each preset feature based on the flatness deviation parameters at each preset feature, and interpolating based on the affine transformation parameters corrected at each preset feature to obtain correction parameters of the whole projection image at different positions.
In another aspect, an embodiment of the present application provides a projection correction apparatus, including:
the acquisition unit is used for acquiring images obtained by the camera shooting projection pictures in the physical space from different shooting angles to obtain n projection scene images; the projection picture is a picture displayed on a projection plane by projecting a projection image comprising m preset features onto the projection plane by projection equipment, wherein n and m are positive integers greater than or equal to 2;
the extraction unit is used for extracting m preset features from n projection scene images respectively;
the three-dimensional reconstruction unit is used for carrying out three-dimensional reconstruction on the m preset features through a motion recovery structure three-dimensional reconstruction method based on the m preset features extracted from the n projection scene images to obtain normalized three-dimensional information of the m preset features in a physical space;
the determining unit is used for combining the positions of the m preset features in the projected image and the normalized three-dimensional information of the m preset features to determine the normalized three-dimensional information of the projected image in the physical space;
the generating unit is used for generating correction parameters for the projection image according to the normalized three-dimensional information of the projection picture; the correction parameters are used for deforming the projection image so as to enable the edge contour of the projection picture to be corrected to a preset shape;
and the control unit is used for controlling the projection device to project the deformed projection image according to the correction parameters.
Optionally, the generating unit includes:
the first calculating subunit is used for calculating the angle of the projection plane in the physical space according to the normalized three-dimensional information of the four corner points of the projection picture;
the first generation subunit is used for converting the normalized three-dimensional information of the four corner points of the projection plane into the normalized two-dimensional information of the four corner points of the projection picture according to the angles of the projection plane in the physical space;
and the second generation subunit is used for generating correction parameters for the projection image according to the normalized two-dimensional information of the four corner points of the projection picture.
Optionally, the second generating subunit includes:
the second calculating subunit is used for calculating the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle of the projection picture according to the normalized two-dimensional information of the four corner points of the projection picture;
the first generation subunit is configured to generate affine transformation parameters of the projection image that enable the four corner points of the projection image to be transformed to the four corner points of the maximum inscribed rectangle according to the normalized two-dimensional information of the four corner points of the projection image and the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle, so as to obtain correction parameters for the projection image.
Optionally, the apparatus further comprises:
the conversion unit is used for converting the normalized three-dimensional information of m preset features into normalized two-dimensional information according to the angle of the projection plane in the physical space;
the first calculation unit is used for calculating ideal two-dimensional information of m preset features in a projection plane according to the normalized two-dimensional information of four corner points of the projection picture and the positions of the m preset features in the projection image;
the second calculation unit is used for calculating flatness deviation parameters of the projection plane at each preset feature according to the deviation between the normalized two-dimensional information and the ideal two-dimensional information of each preset feature;
the correction unit is used for correcting the affine transformation parameters at each preset feature based on the flatness deviation parameters at each preset feature, and interpolating based on the affine transformation parameters corrected at each preset feature to obtain correction parameters of the whole projection image at different positions.
In still another aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the projection correction method as provided by embodiments of the present application.
In yet another aspect, embodiments of the present application provide a storage medium having stored thereon computer program instructions that, when executed by a processor, implement a projection correction method as provided by embodiments of the present application.
According to the projection correction method, the device, the electronic equipment and the storage medium, the images obtained by shooting the projection pictures in the physical space from different shooting angles by the camera are obtained, m preset features are extracted from n projection scene images respectively, and then the m preset features are subjected to three-dimensional reconstruction by the motion restoration structure three-dimensional reconstruction method, so that the normalized three-dimensional information of the m preset features in the physical space can be obtained, the normalized three-dimensional information of the projection pictures in the physical space is determined, correction parameters of the projection images are generated, after the correction parameters are sent to the projection equipment, the projection equipment can be enabled to project the deformed projection images, so that the edge contour of the projection pictures can be corrected to the preset shape, the projection pictures can be corrected by the images shot by the camera of a third party, devices such as the camera are not required to be installed on the projection equipment, and the manufacturing cost of projection correction is reduced.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments of the present application will be briefly described, and it is possible for a person skilled in the art to obtain other drawings according to these drawings without inventive effort.
FIG. 1 is a flow chart of a projection correction method according to one embodiment of the present application;
FIG. 2 is a schematic view of a projection correction apparatus according to another embodiment of the present disclosure;
fig. 3 is a schematic structural diagram of an electronic device according to another embodiment of the present application.
Detailed Description
Features and exemplary embodiments of various aspects of the present application are described in detail below to make the objects, technical solutions and advantages of the present application more apparent, and to further describe the present application in conjunction with the accompanying drawings and the detailed embodiments. It should be understood that the specific embodiments described herein are intended to be illustrative of the application and are not intended to be limiting. It will be apparent to one skilled in the art that the present application may be practiced without some of these specific details. The following description of the embodiments is merely intended to provide a better understanding of the present application by showing examples of the present application.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises the element.
In order to solve the problems in the prior art, embodiments of the present application provide a projection correction method, a projection correction device, an electronic device, and a storage medium. The projection correction method provided in the embodiment of the present application will be first described below.
Fig. 1 is a flow chart of a projection correction method according to an embodiment of the present application. As shown in fig. 1, the method comprises the following steps:
The camera is a camera configured by the executive party of the embodiment of the application, or can communicate with the executive party of the embodiment of the application, and after the camera shoots images from different angles, n Zhang Touying scene images shot by the camera are acquired.
The projection scene image is an image in which a projection screen is captured. The projection screen is a screen in which the projection device projects a projection image including m preset features onto a projection plane, where n and m are positive integers greater than or equal to 2, that is, at least two projection scene images are captured, and at least two preset features are included in an image (that is, a projection image) projected by the projection device.
The preset feature is a known feature, alternatively the preset feature may be a feature point. For example, the projected image may comprise a predetermined pattern, such as a pentagram, triangle, cross, etc., each corner point of the pattern may be considered a feature point.
The extraction of the preset features may use a feature extraction method among image processing methods of the related art. According to the type of the preset features, the corresponding features can be extracted through an image processing method for extracting different types of features such as straight lines, circles, corner points and the like. For example, in the case where the preset feature is a corner, the corner feature in the image may be extracted by Harris corner detection algorithm. The foregoing is illustrative only and is not to be construed as limiting the embodiments of the present application.
And 103, carrying out three-dimensional reconstruction on the m preset features by a motion restoration structure three-dimensional reconstruction method based on the m preset features extracted from the n projection scene images to obtain normalized three-dimensional information of the m preset features in a physical space.
The motion restoration structure (Structure from motion, SFM) three-dimensional reconstruction method is a three-dimensional reconstruction method for realizing 3D reconstruction from motion, namely, calculating 3D information from time series 2D images.
The input of the SFM algorithm is that at least two images of the same scene are shot at different angles, after the characteristic points in each image are extracted, the characteristic point matching is executed (for example, the characteristic points can be matched through SIFT and SURF algorithms), any information of a camera is not needed, the parameters of the camera are calculated through the characteristic point matching among the images, and therefore three-dimensional reconstruction of the characteristics in the images is completed, and two-dimensional characteristic points are converted into a three-dimensional lattice to obtain three-dimensional coordinates.
Because the n Zhang Touying scene images shot in the step 101 are different in shooting angle, three-dimensional reconstruction of m preset features can be completed by using an SFM algorithm, and normalized three-dimensional information of the m preset features in a physical space is obtained.
It should be noted that, the three-dimensional coordinate obtained by the SFM algorithm is not a physical real coordinate, but a coordinate of a missing scale, which is a normalized three-dimensional coordinate capable of representing a relative positional relationship between points.
And 104, combining the positions of the m preset features in the projected image and the normalized three-dimensional information of the m preset features to determine the normalized three-dimensional information of the projected image in the physical space.
After obtaining the normalized three-dimensional information of the m preset features in the physical space, since the pattern of the projected image is known, that is, the positions of the m preset features in the projected image are known, the normalized three-dimensional information of the projected image can be determined in combination with the positions of the m preset features in the projected image.
By way of example, the normalized three-dimensional coordinates of the four corner points of the projection screen can be calculated by combining the relative positional relationships between the m preset features and the four corner points of the projection image.
In the embodiment of the application, the projection device for correcting the projection picture is to realize the correction of the projection picture through affine transformation of the projection image. That is, the projection apparatus digitally corrects the projection screen. The correction parameters are used for deforming the projection image so as to correct the edge contour of the projection picture to a preset shape.
The preset shape is generally rectangular, and since the projection apparatus is not corrected, the projection screen is generally an irregular quadrangle, and thus it is necessary to correct the projection screen from the irregular quadrangle to the rectangle. The correction parameter is a parameter for deforming a projection image projected by the projection apparatus so that the contour of the projection screen is corrected to a preset shape.
When the correction parameters for the projection image are generated according to the normalized three-dimensional information of the projection image, the angle of the projection plane where the projection image is located in the physical space (the coordinate system of the physical space is the physical coordinate system of the photographed electronic device) can be determined specifically through the normalized three-dimensional information of the projection image, and the angle can be expressed by a normal vector. Furthermore, according to the angle of the projection plane in the physical space, the normalized three-dimensional information of the four corner points of the projection plane can be converted into the normalized two-dimensional information of the four corner points of the projection picture.
Then, according to the normalized two-dimensional information of the four corners of the projection picture, the normalized two-dimensional information of the four corners of the maximum inscribed rectangle of the projection picture can be calculated, so that affine transformation parameters of the projection image, which enable the four corners of the projection picture to be transformed to the four corners of the maximum inscribed rectangle, are generated according to the normalized two-dimensional information of the four corners of the projection picture and the normalized two-dimensional information of the four corners of the maximum inscribed rectangle, and correction parameters of the projection image are obtained.
The above is an alternative embodiment for calculating correction parameters. In other embodiments, different approaches may be used.
For example, the correction parameter may be an inclination angle of a plane of the projection screen relative to the projection apparatus, so that the projection apparatus may correct the projection image based on the inclination angle of the plane of the projection screen relative to the projection apparatus. Alternatively, the correction parameter sent to the projection apparatus may also be an affine transformation parameter calculated based on the inclination angle of the projection screen with respect to the projection apparatus, that is, the execution party that calculates the affine transformation parameter is the execution party of the embodiment of the present application, instead of the projection apparatus.
As another example, the correction parameter may be a parameter that enables the projection device to determine the contour of the current projected picture. For example, the coordinates of four sides of the projection screen or the coordinates of four corner points of the projection screen may be obtained. In this way, the projection device can adjust the current projection screen to a preset shape based on this calculation.
For another example, the correction parameter may be a parameter that enables the projection apparatus to determine that the contour of the current projection screen is transformed into a contour of a preset shape. Illustratively, taking the preset shape of the adjustment target as a rectangle as an example, the correction parameter may be coordinates of four corner points of the projected picture.
And step 106, controlling the projection device to project the deformed projection image according to the correction parameters.
In one example, the projection correction method provided in the embodiment of the present application may be executed by an electronic device (for example, a mobile phone, a tablet computer, etc.) that captures an image, so that the calculated correction parameter may be sent through communication (for example, wireless communication such as bluetooth, a mobile cellular network, WIFI, etc., or wired communication such as direct connection of a USB data line) between the projection device and the calculated correction parameter may be sent to the projection device, so that the projection device projects a deformed projection image according to the correction parameter, and thus corrects an edge profile of a projection image to a preset shape.
In another example, the projection correction method provided in the embodiment of the present application may be performed by the projection apparatus, so that after the correction parameters are calculated, the projected image may be directly deformed.
Optionally, the projection correction method provided in the embodiment of the present application may be integrated in an application program APP of the mobile terminal, so that an image may be captured by a camera of the mobile terminal, the APP may be operated, a communication connection may be established between the communication module of the mobile terminal and the projection device, and after correction parameters are obtained, the communication connection may be sent to the projection device.
According to the projection correction method, the images obtained by shooting the projection pictures from different shooting angles by the camera are obtained, m preset features are extracted from n projection scene images respectively, then the m preset features are subjected to three-dimensional reconstruction by the motion restoration structure three-dimensional reconstruction method, normalized three-dimensional information of the m preset features in a physical space can be obtained, the normalized three-dimensional information of the projection pictures is determined, correction parameters of the projection images are generated, after the correction parameters are sent to the projection equipment, the projection equipment can be enabled to project deformed projection images, the edge contours of the projection pictures can be corrected to the preset shapes according to the correction parameters, the projection pictures can be corrected by the images shot by the camera of a third party, devices such as cameras are not required to be installed on the projection equipment, and the manufacturing cost of projection correction is reduced.
In one example, the projected image may also be fine-tuned based on the flatness of the projected plane, reducing the effect of irregularities in the projected plane medium on the projected image.
Specifically, the normalized three-dimensional information of m preset features can be converted into normalized two-dimensional information according to the angle of the projection plane in the physical space, and the ideal two-dimensional information of m preset features in the projection plane can be calculated according to the normalized two-dimensional information of four corner points of the projection picture and the positions of the m preset features in the projection image.
The principle is that in theory, the projection of m preset features in the projected image should be in a plane on the projection plane, however, since the projection plane in the physical space is uneven, for example, the projection plane may be a wall or a projection curtain, etc., the positions of the m preset features in the physical space are deformed due to the unevenness and are not in the same plane. Thus, when calculating the projection plane based on the normalized three-dimensional information of m preset features in the physical space, the uneven errors are removed, and a unique ideal projection plane is calculated by removing the errors (such as a least square method). Then, according to the normalized two-dimensional information of the four corner points of the projection picture and the positions of the m preset features in the projection image, the ideal positions of the m preset features projected on the ideal projection plane, namely the ideal two-dimensional information, can be calculated.
Furthermore, the flatness deviation parameter of the projection plane at each preset feature can be calculated in turn according to the deviation between the normalized two-dimensional information and the ideal two-dimensional information of each preset feature. The flatness deviation parameter may be regarded as a parameter for adjusting the projection screen based on the unevenness of the projection plane.
Based on the flatness deviation parameter at each preset feature, the affine transformation parameter at each preset feature can be corrected, and interpolation is performed based on the affine transformation parameter corrected at each preset feature, so that the correction parameters of the whole projection image at different positions are obtained. Therefore, the projected image is deformed based on the flatness of the projection plane in the physical space at the corresponding position, and the projection picture can better show the image to be projected.
In this embodiment of the application, the function of correcting the projection picture of the projection device is realized through the electronic device provided with the camera, so that the cost of the projection device can be reduced, the application is more convenient, the realization difficulty is lower, and the method can be also suitable for the projection device with larger projection angles such as laser projection.
It should be noted that, in the projection correction method provided in the embodiment of the present application, the execution subject may be a projection correction apparatus, or a control module of the projection correction apparatus for executing the projection correction method. In the embodiments of the present application, a method for performing projection correction by a projection correction apparatus is taken as an example, and the projection correction apparatus provided in the embodiments of the present application is described.
As shown in fig. 2, the projection correction apparatus provided in the embodiment of the present application includes an acquisition unit 21, an extraction unit 22, a three-dimensional reconstruction unit 23, a determination unit 24, a generation unit 25, and a control unit 26.
The acquiring unit 21 is configured to acquire images obtained by capturing projection pictures in a physical space from different capturing angles by using a camera, so as to obtain n projection scene images; the projection picture is a picture displayed on a projection plane by projecting a projection image comprising m preset features onto the projection plane by projection equipment, wherein n and m are positive integers greater than or equal to 2;
the extracting unit 22 is configured to extract m preset features from n projection scene images respectively;
the three-dimensional reconstruction unit 23 is configured to perform three-dimensional reconstruction on the m preset features by using a motion restoration structure three-dimensional reconstruction method based on the m preset features extracted from the n projection scene images, so as to obtain normalized three-dimensional information of the m preset features in a physical space;
the determining unit 24 is configured to determine normalized three-dimensional information of the projection screen in the physical space by combining positions of m preset features in the projection image and the normalized three-dimensional information of the m preset features;
the generating unit 25 is used for generating correction parameters for the projection image according to the normalized three-dimensional information of the projection picture; the correction parameters are used for deforming the projection image so as to enable the edge contour of the projection picture to be corrected to a preset shape;
the control unit 26 is used for controlling the projection device to project the deformed projection image according to the correction parameters.
Alternatively, the generating unit 25 may include:
the first calculating subunit is used for calculating the angle of the projection plane in the physical space according to the normalized three-dimensional information of the four corner points of the projection picture;
the first generation subunit is used for converting the normalized three-dimensional information of the four corner points of the projection plane into the normalized two-dimensional information of the four corner points of the projection picture according to the angles of the projection plane in the physical space;
and the second generation subunit is used for generating correction parameters for the projection image according to the normalized two-dimensional information of the four corner points of the projection picture.
Optionally, the second generating subunit may include:
the second calculating subunit is used for calculating the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle of the projection picture according to the normalized two-dimensional information of the four corner points of the projection picture;
the first generation subunit is configured to generate affine transformation parameters of the projection image that enable the four corner points of the projection image to be transformed to the four corner points of the maximum inscribed rectangle according to the normalized two-dimensional information of the four corner points of the projection image and the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle, so as to obtain correction parameters for the projection image.
Optionally, the apparatus may further include:
the conversion unit is used for converting the normalized three-dimensional information of m preset features into normalized two-dimensional information according to the angle of the projection plane in the physical space;
the first calculation unit is used for calculating ideal two-dimensional information of m preset features in a projection plane according to the normalized two-dimensional information of four corner points of the projection picture and the positions of the m preset features in the projection image;
the second calculation unit is used for calculating flatness deviation parameters of the projection plane at each preset feature according to the deviation between the normalized two-dimensional information and the ideal two-dimensional information of each preset feature;
the correction unit is used for correcting the affine transformation parameters at each preset feature based on the flatness deviation parameters at each preset feature, and interpolating based on the affine transformation parameters corrected at each preset feature to obtain correction parameters of the whole projection image at different positions.
According to the projection correction device, the images obtained by shooting the projection pictures from different shooting angles by the camera are obtained, m preset features are extracted from n projection scene images respectively, then the m preset features are subjected to three-dimensional reconstruction by the motion restoration structure three-dimensional reconstruction method, normalized three-dimensional information of the m preset features in a physical space can be obtained, the normalized three-dimensional information of the projection pictures is determined, correction parameters of the projection images are generated, after the correction parameters are sent to the projection equipment, the projection equipment can be enabled to project deformed projection images, the edge contours of the projection pictures can be corrected to the preset shapes according to the correction parameters, the projection pictures can be corrected by the images shot by the camera of a third party, devices such as cameras are not required to be installed on the projection equipment, and the manufacturing cost of projection correction is reduced.
The embodiment of the application also provides electronic equipment. The electronic device may be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, a tablet computer, a notebook computer, a palm computer, or a personal digital assistant, and the non-mobile electronic device may be a server, a personal computer, a television, and the like, which are not limited in this embodiment.
Fig. 3 shows a schematic hardware structure of an electronic device according to an embodiment of the present application.
A processor 301 and a memory 302 storing computer program instructions may be included in an electronic device.
In particular, the processor 301 may include a Central Processing Unit (CPU), or an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), or may be configured to implement one or more integrated circuits of embodiments of the present application.
The memory may include Read Only Memory (ROM), random Access Memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical/tangible memory storage devices. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software comprising computer-executable instructions and when the software is executed (e.g., by one or more processors) it is operable to perform the operations described with reference to a method according to an aspect of the present application.
The processor 301 implements any of the projection correction methods of the above embodiments by reading and executing computer program instructions stored in the memory 302.
In one example, the electronic device may also include a communication interface 303 and a bus 310. As shown in fig. 3, the processor 301, the memory 302, and the communication interface 303 are connected to each other by a bus 310 and perform communication with each other.
The communication interface 303 is mainly used to implement communication between each module, device, unit and/or apparatus in the embodiments of the present application.
It should be clear that the present application is not limited to the particular arrangements and processes described above and illustrated in the drawings. For the sake of brevity, a detailed description of known methods is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method processes of the present application are not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps, after appreciating the spirit of the present application.
The functional blocks shown in the above-described structural block diagrams may be implemented in hardware, software, firmware, or a combination thereof. When implemented in hardware, it may be, for example, an electronic circuit, an Application Specific Integrated Circuit (ASIC), suitable firmware, a plug-in, a function card, or the like. When implemented in software, the elements of the present application are the programs or code segments used to perform the required tasks. The program or code segments may be stored in a machine readable medium or transmitted over transmission media or communication links by a data signal carried in a carrier wave. A "machine-readable medium" may include any medium that can store or transfer information. Examples of machine-readable media include electronic circuitry, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio Frequency (RF) links, and the like. The code segments may be downloaded via computer networks such as the internet, intranets, etc.
It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, the present application is not limited to the order of the above-described steps, that is, the steps may be performed in the order mentioned in the embodiments, may be different from the order in the embodiments, or several steps may be performed simultaneously.
Aspects of the present application are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such a processor may be, but is not limited to being, a general purpose processor, a special purpose processor, an application specific processor, or a field programmable logic circuit. It will also be understood that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware which performs the specified functions or acts, or combinations of special purpose hardware and computer instructions.
In the foregoing, only the specific embodiments of the present application are described, and it will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, modules and units described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein. It should be understood that the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive various equivalent modifications or substitutions within the technical scope of the present application, which are intended to be included in the scope of the present application.
Claims (10)
1. A projection correction method, comprising:
acquiring images obtained by a camera shooting projection pictures in a physical space from different shooting angles, and obtaining n projection scene images; the projection picture is a picture displayed on a projection plane by projecting a projection image comprising m preset features onto the projection plane by projection equipment, wherein n and m are positive integers greater than or equal to 2;
respectively extracting m preset features from the n Zhang Touying scene images;
based on the m preset features extracted from the n Zhang Touying scene image, carrying out three-dimensional reconstruction on the m preset features by a motion restoration structure three-dimensional reconstruction method to obtain normalized three-dimensional information of the m preset features in a physical space;
combining the positions of the m preset features in the projected image and the normalized three-dimensional information of the m preset features to determine the normalized three-dimensional information of the projected image in a physical space;
generating correction parameters for the projection image according to the normalized three-dimensional information of the projection picture; the correction parameters are used for deforming the projection image so as to enable the edge contour of the projection picture to be corrected to a preset shape;
and controlling the projection equipment to project the deformed projection image according to the correction parameters.
2. The projection correction method according to claim 1, wherein the generating correction parameters for the projection image from the normalized three-dimensional information of the projection screen includes:
calculating the angle of the projection plane in the physical space according to the normalized three-dimensional information of the four corner points of the projection picture;
according to the angle of the projection plane in the physical space, converting the normalized three-dimensional information of the four corner points of the projection plane into the normalized two-dimensional information of the four corner points of the projection picture;
and generating correction parameters for the projection image according to the normalized two-dimensional information of the four corner points of the projection picture.
3. The projection correction method according to claim 2, wherein the generating correction parameters for the projection image based on normalized two-dimensional information of four corner points of the projection screen includes:
calculating normalized two-dimensional information of four corner points of the maximum inscribed rectangle of the projection picture according to the normalized two-dimensional information of the four corner points of the projection picture;
generating affine transformation parameters of the projection image, which enable the four corner points of the projection picture to be transformed to the four corner points of the maximum inscribed rectangle, according to the normalized two-dimensional information of the four corner points of the projection picture and the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle, so as to obtain correction parameters of the projection image.
4. A projection correction method according to claim 3, further comprising, after obtaining correction parameters for the projected image:
according to the angle of the projection plane in the physical space, converting the normalized three-dimensional information of the m preset features into normalized two-dimensional information;
calculating ideal two-dimensional information of the m preset features in the projection plane according to the normalized two-dimensional information of the four corner points of the projection picture and the positions of the m preset features in the projection image;
calculating flatness deviation parameters of the projection plane at each preset feature according to deviation between normalized two-dimensional information and ideal two-dimensional information of each preset feature;
correcting the affine transformation parameters at each preset feature based on the flatness deviation parameters at each preset feature, and interpolating based on the affine transformation parameters corrected at each preset feature to obtain correction parameters of the whole projection image at different positions.
5. A projection correction apparatus, comprising:
the acquisition unit is used for acquiring images obtained by the camera shooting projection pictures in the physical space from different shooting angles to obtain n projection scene images; the projection picture is a picture displayed on a projection plane by projecting a projection image comprising m preset features onto the projection plane by projection equipment, wherein n and m are positive integers greater than or equal to 2;
the extraction unit is used for extracting the m preset features from the n Zhang Touying scene images respectively;
the three-dimensional reconstruction unit is used for carrying out three-dimensional reconstruction on the m preset features through a motion recovery structure three-dimensional reconstruction method based on the m preset features extracted from the n Zhang Touying scene image to obtain normalized three-dimensional information of the m preset features in a physical space;
the determining unit is used for determining the normalized three-dimensional information of the projection picture in a physical space by combining the positions of the m preset features in the projection image and the normalized three-dimensional information of the m preset features;
a generating unit, configured to generate correction parameters for the projection image according to the normalized three-dimensional information of the projection screen; the correction parameters are used for deforming the projection image so as to enable the edge contour of the projection picture to be corrected to a preset shape;
and the control unit is used for controlling the projection equipment to project the deformed projection image according to the correction parameters.
6. The projection correction apparatus according to claim 5, wherein the generation unit includes:
the first calculating subunit is used for calculating the angle of the projection plane in the physical space according to the normalized three-dimensional information of the four corner points of the projection picture;
the first generation subunit is used for converting the normalized three-dimensional information of the four corner points of the projection plane into the normalized two-dimensional information of the four corner points of the projection picture according to the angles of the projection plane in the physical space;
and the second generation subunit is used for generating correction parameters for the projection image according to the normalized two-dimensional information of the four corner points of the projection picture.
7. The projection correction apparatus of claim 6, wherein the second generation subunit comprises:
the second calculating subunit is used for calculating the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle of the projection picture according to the normalized two-dimensional information of the four corner points of the projection picture;
and the first generation subunit is used for generating affine transformation parameters of the projection image, which are used for transforming the four corner points of the projection picture to the four corner points of the maximum inscribed rectangle, according to the normalized two-dimensional information of the four corner points of the projection picture and the normalized two-dimensional information of the four corner points of the maximum inscribed rectangle, so as to obtain correction parameters of the projection image.
8. The projection correction apparatus of claim 7, wherein the apparatus further comprises:
the conversion unit is used for converting the normalized three-dimensional information of the m preset features into normalized two-dimensional information according to the angle of the projection plane in the physical space;
the first calculation unit is used for calculating ideal two-dimensional information of the m preset features in the projection plane according to the normalized two-dimensional information of the four corner points of the projection picture and the positions of the m preset features in the projection image;
the second calculation unit is used for calculating flatness deviation parameters of the projection plane at each preset feature according to the deviation between the normalized two-dimensional information and the ideal two-dimensional information of each preset feature;
the correction unit is used for correcting the affine transformation parameters at each preset feature based on the flatness deviation parameters at each preset feature, and interpolating based on the affine transformation parameters corrected at each preset feature to obtain correction parameters of the whole projection image at different positions.
9. An electronic device, the electronic device comprising: a processor and a memory storing computer program instructions;
the processor, when executing the computer program instructions, implements the projection correction method as claimed in any one of claims 1-4.
10. A storage medium having stored thereon computer program instructions which, when executed by a processor, implement the projection correction method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111518634.2A CN116264611A (en) | 2021-12-13 | 2021-12-13 | Projection correction method, projection correction device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111518634.2A CN116264611A (en) | 2021-12-13 | 2021-12-13 | Projection correction method, projection correction device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116264611A true CN116264611A (en) | 2023-06-16 |
Family
ID=86723367
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111518634.2A Pending CN116264611A (en) | 2021-12-13 | 2021-12-13 | Projection correction method, projection correction device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116264611A (en) |
-
2021
- 2021-12-13 CN CN202111518634.2A patent/CN116264611A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102170182B1 (en) | System for distortion correction and calibration using pattern projection, and method using the same | |
| JP6351238B2 (en) | Image processing apparatus, imaging apparatus, and distance correction method | |
| CN108107571B (en) | Image processing apparatus and method, and non-transitory computer-readable recording medium | |
| US9344695B2 (en) | Automatic projection image correction system, automatic projection image correction method, and non-transitory storage medium | |
| EP3340170A1 (en) | Multi-depth image fusion method and apparatus | |
| CN107316326B (en) | Edge-based disparity map calculation method and device applied to binocular stereo vision | |
| US9491452B2 (en) | Camera calibration | |
| KR20180060977A (en) | Image-processing device and method for foreground mask correction for object segmentation | |
| US20150109418A1 (en) | Method and system for three-dimensional data acquisition | |
| US9613404B2 (en) | Image processing method, image processing apparatus and electronic device | |
| US8531505B2 (en) | Imaging parameter acquisition apparatus, imaging parameter acquisition method and storage medium | |
| US11523056B2 (en) | Panoramic photographing method and device, camera and mobile terminal | |
| GB2536429A (en) | Image noise reduction | |
| CN112041884A (en) | Object segmentation in a sequence of color image frames by background image and background depth correction | |
| CN114693760A (en) | Image correction method, device and system and electronic equipment | |
| CN112381887B (en) | Multi-depth camera calibration method, device, equipment and medium | |
| CN112470192A (en) | Dual-camera calibration method, electronic device and computer-readable storage medium | |
| CN111340042B (en) | Object contour recognition method, device, equipment and storage medium | |
| CN113902853A (en) | Face three-dimensional reconstruction method and device, electronic equipment and storage medium | |
| CN116264612A (en) | Projection correction method, projection correction device, electronic equipment and storage medium | |
| CN107370952B (en) | Image shooting method and device | |
| CN116264611A (en) | Projection correction method, projection correction device, electronic equipment and storage medium | |
| CN110852934A (en) | Image processing method and apparatus, image device, and storage medium | |
| CN113850100A (en) | Method and device for correcting two-dimensional code | |
| US20150262028A1 (en) | Image processing apparatus, image processing method and image processing program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |