CN110933391B - Calibration parameter compensation method and device for projection system and readable storage medium - Google Patents
Calibration parameter compensation method and device for projection system and readable storage medium Download PDFInfo
- Publication number
- CN110933391B CN110933391B CN201911325236.1A CN201911325236A CN110933391B CN 110933391 B CN110933391 B CN 110933391B CN 201911325236 A CN201911325236 A CN 201911325236A CN 110933391 B CN110933391 B CN 110933391B
- Authority
- CN
- China
- Prior art keywords
- projection
- acquisition device
- image
- preset
- feature points
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
- H04N9/3179—Video signal processing therefor
Abstract
The embodiment of the invention relates to the technical field of projection system calibration, and particularly discloses a calibration parameter compensation method and device for a projection system and a readable storage medium. The method comprises the steps of projecting a preset image to a projection surface by controlling a projection device, and simultaneously controlling an acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image; the method for calculating the actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device solves the problem that the projection effect is influenced by the failure of pre-calibrated parameters caused by the relative displacement of the projection device and the acquisition device in the use process of a user, adopts an algorithm to compensate the attitude variation between the projection device and the acquisition device, reduces the cost of a projection system, and ensures the use stability of the projection system.
Description
Technical Field
The invention relates to the technical field of projection system calibration, in particular to a calibration parameter compensation method and device of a projection system and a readable storage medium.
Background
In a projection system, when depth measurement is performed by a structured light and phase shift method, a projector and an acquisition device system need to be strictly calibrated. After calibration, the relative position of the acquisition device and the projector, i.e. the focusing state, cannot be changed, otherwise the accuracy of subsequent measurement is affected. In the process of transportation, use and the like of the projector, the projector and the acquisition device system are likely to move relatively, and the final result is affected. The existing scheme comprises a more complex structure, so that the relative position relation between the projector and the acquisition device is firmer, and the like. This approach is more complex in design and more costly.
Disclosure of Invention
In view of this, the present application provides a calibration parameter compensation method and apparatus for a projection system, and a readable storage medium, which can solve the problem of failure caused by relative displacement during the use of factory-side fine calibration through an algorithm.
In order to solve the above technical problems, the technical solution provided by the present invention is a calibration parameter compensation method for a projection system, where the projection system includes a projection device and an acquisition device, and the calibration parameter includes: the compensation method comprises the following steps of measuring internal parameters of the projection device, internal parameters of the acquisition device, initial attitude parameters of the projection device and the acquisition device, and the compensation method comprises the following steps:
controlling a projection device to project a preset image onto a projection surface, wherein the preset image at least comprises 9 feature points with known pixel coordinates, and controlling an acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
and calculating the actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device.
Preferably, the method for controlling the projection device to project a preset image onto the projection surface, where the preset image at least includes 9 feature points with known pixel coordinates, and controlling the acquisition device to shoot a projection picture on the projection surface to obtain the projection surface image includes:
and controlling a projection device to project preset images to a projection plane from different angles, wherein the preset images at least comprise 9 feature points with known pixel coordinates, and controlling an acquisition device to shoot projection pictures on the projection plane to obtain a plurality of corresponding projection plane images.
Preferably, the method for calculating the actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device, and the initial attitude parameters of the projection device and the acquisition device comprises:
extracting and matching feature points of a preset image and a projection plane image;
performing polar line correction on the matched characteristic points according to initial attitude parameters of the projection device and the acquisition device, internal parameters of the projection device and internal parameters of the acquisition device;
calculating the coordinate difference amount, namely parallax, between the matched feature points after epipolar line correction in a preset first direction;
obtaining three-dimensional coordinates of each characteristic point according to the parallax and the initial attitude parameters of the projection device and the acquisition device;
obtaining a fitting plane by using a least square method according to the three-dimensional coordinates of each characteristic point;
calculating the average value of the distance values from each characteristic point to the fitting plane to obtain the average value of the distances;
calculating the average value of coordinate difference values in a preset second direction between the matched feature points after epipolar line correction to obtain a difference average value;
and taking the accumulated result of the distance average value and the difference average value as an evaluation function, and optimizing the initial attitude parameters of the projection device and the acquisition device by adopting an optimization algorithm to obtain the actual attitude parameters of the projection device and the acquisition device.
Preferably, before the step of controlling the projection device to project a preset image onto the projection surface, where the preset image at least includes 9 feature points with known pixel coordinates, and controlling the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image, the method further includes: and judging whether the trapezoidal correction effect meets the preset requirement, and if not, entering the subsequent steps.
Preferably, the method for determining whether the effect of the trapezoidal correction meets the preset requirement includes:
controlling a projection device to project a preset image onto a projection surface, wherein the preset image at least comprises 9 feature points with known pixel coordinates;
controlling an acquisition device to shoot a projection picture on the projection plane to obtain a projection plane image;
extracting and matching feature points of a preset image and a projection plane image;
performing polar line correction on the matched feature points, and calculating an average value of coordinate difference values in a preset second direction between the matched feature points after the polar line correction;
and entering a calibration parameter compensation step when the average value of the coordinate difference values is larger than a preset threshold value.
The invention also provides a calibration parameter compensation device of the projection system, the projection system comprises a projection device and an acquisition device, and the calibration parameters comprise: projection arrangement internal parameter, collection system internal parameter, projection arrangement and collection system initial attitude parameter, compensation arrangement includes:
the projection acquisition control module is used for controlling the projection device to project a preset image to a projection surface, wherein the preset image at least comprises 9 characteristic points with known pixel coordinates, and simultaneously controlling the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
and the attitude parameter calculation module is used for calculating the actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device.
Preferably, the projection acquisition control module is further configured to control the projection device to project preset images onto the projection surface from different angles, where the preset images at least include 9 feature points with known pixel coordinates, and control the acquisition device to shoot a projection picture on the projection surface to obtain multiple corresponding projection surface images.
Preferably, the attitude parameter calculation module includes:
the characteristic point matching unit is used for extracting and matching the characteristic points of the preset image and the projection plane image;
the polar line correction unit is used for carrying out polar line correction on the matched characteristic points according to the initial attitude parameters of the projection device and the acquisition device, the internal parameters of the projection device and the internal parameters of the acquisition device;
the parallax calculation unit is used for calculating the coordinate difference amount in the preset first direction between the matched characteristic points after epipolar line correction, namely parallax;
the coordinate calculation unit is used for obtaining three-dimensional coordinates of each characteristic point according to the parallax and the initial attitude parameters of the projection device and the acquisition device;
the plane fitting unit is used for obtaining a fitting plane by utilizing a least square method according to the three-dimensional coordinates of each characteristic point;
the distance calculation unit is used for calculating the average value of the distance values from each characteristic point to the fitting plane to obtain the distance average value;
the difference calculating unit is used for calculating the average value of coordinate difference quantities in a preset second direction between the matched characteristic points after epipolar line correction to obtain a difference average value;
and the parameter optimization unit is used for optimizing the initial attitude parameters of the projection device and the acquisition device by adopting an optimization algorithm by taking the accumulated result of the distance average value and the difference average value as an evaluation function so as to obtain the actual attitude parameters of the projection device and the acquisition device.
Preferably, the calibration parameter compensation device of the projection system further includes a trapezoidal correction judgment module, which is used for judging whether the trapezoidal correction effect meets the preset requirement, and if not, the projection acquisition control module is entered.
Preferably, the trapezoidal correction determining module includes:
the image projection control unit is used for controlling the projection device to project a preset image to the projection surface, wherein the preset image at least comprises 9 characteristic points with known pixel coordinates;
the image acquisition control unit is used for controlling the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
the parameter compensation judging unit is used for extracting and matching the feature points of the preset image and the projection plane image; performing polar line correction on the matched feature points, and calculating the coordinate difference value average value in a preset second direction between the matched feature points after the polar line correction; and when the average value of the coordinate difference values is greater than a preset threshold value, entering a projection acquisition control module.
The invention also provides a calibration parameter compensation device of the projection system, which comprises:
a memory for storing a computer program;
a processor for executing the computer program to implement the steps of the calibration parameter compensation method of the projection system as described above.
The invention further provides a readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the calibration parameter compensation method of the projection system.
Compared with the prior art, the beneficial effects of the method are detailed as follows: the method comprises the steps of projecting a preset image to a projection surface by controlling a projection device, and simultaneously controlling an acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image; the method for calculating the actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device solves the problem that the projection effect is influenced by the failure of pre-calibrated parameters caused by the relative displacement of the projection device and the acquisition device in the use process of a user, adopts an algorithm to compensate the attitude variation between the projection device and the acquisition device, reduces the cost of a projection system, and ensures the use stability of the projection system.
Drawings
In order to illustrate the embodiments of the present invention more clearly, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings can be obtained by those skilled in the art without inventive effort.
Fig. 1 is a schematic flowchart of a calibration parameter compensation method for a projection system according to an embodiment of the present invention;
fig. 2 is a schematic diagram of a preset image according to an embodiment of the present invention;
fig. 3 is a schematic diagram of a flow of a method for calculating actual attitude parameters of a projection apparatus and an acquisition apparatus according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a calibration parameter compensation apparatus of a projection system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without any creative work belong to the protection scope of the present invention.
In order to make the technical solutions of the present invention better understood by those skilled in the art, the present invention will be further described in detail with reference to the accompanying drawings and specific embodiments.
As shown in fig. 1, an embodiment of the present invention provides a calibration parameter compensation method for a projection system, where the projection system includes a projection device and a collection device, and the pre-stored calibration parameters include: the internal parameters of the projection device, the internal parameters of the acquisition device, the initial attitude parameters of the projection device and the acquisition device, and the compensation method comprises the following steps:
s11: controlling a projection device to project a preset image onto a projection surface, wherein the preset image at least comprises 9 feature points with known pixel coordinates, and controlling an acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
s12: and calculating the actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device.
Specifically, the projection system includes a projection device and an acquisition device, the projection device refers to a device, a device or a module having a projection function, and may be a projector, or the like, the acquisition device refers to a device, a device or a module having a photographing function, and may be an acquisition device, a camera, or the like, the projection device and the acquisition device may be assembled integrally or separately, for example, on a screen-less television with a camera, the projector and the camera constitute a projection system.
Before the projection system leaves a factory, the projection device and the acquisition device are calibrated at the factory end, so that accurate system calibration parameters under a fixed distance are obtained, wherein the system calibration parameters comprise attitude parameters R and T between the projection device and the acquisition device, R represents a rotation parameter between the two optical devices, T represents an offset between the two optical devices, and the system calibration parameters also comprise a projection device intrinsic parameter Kc and an acquisition device intrinsic parameter Kp. After the projection system leaves a factory, the relative offset of the projection system is caused by the conditions of transportation, use and the like, but the internal parameters of the projection device and the acquisition device are not changed, so that the parameters can still be continuously maintained during dynamic compensation, and the compensation mainly aims at the posture between the projection device and the acquisition device.
The method of step S11 includes:
and controlling the projection device to project preset images to the projection surface from different angles, wherein the preset images at least comprise 9 feature points with known pixel coordinates, and controlling the acquisition device to shoot projection pictures on the projection surface to obtain a plurality of corresponding projection surface images.
Specifically, during compensation, the parameter to be compensated is R, T, the two parameters are represented by 3 parameters respectively, and during projection, the projection surface can be represented by 3 parameters, so that at least 9 feature points are required to solve the 9 parameters. In special cases, such as forward projection, the projection plane can be represented by only 1 parameter, and then only 7 parameters are needed to solve the parameters. Here we require at least 9 feature points in order to reduce the requirement for placement when the user is throwing.
Specifically, as shown in fig. 2, in order to obtain a better compensation effect, a user may be required to project preset images onto a projection plane at different angles, and an acquisition device is used to acquire a projection image; the specific process is as follows: (1) the user is prompted to place the projection range on the same projection plane, and a preset image containing a large number of feature points with known pixel coordinates is projected to the projection plane (the number of the feature points in the preset image is more than or equal to 9), wherein the feature points can be checkerboard corner points, circular centers, annular centers, orb features and the like, and matching and sub-pixel precision features are easy to achieve. (2) Controlling an acquisition device to acquire the projection image; (3) and (3) prompting a user whether to continue to collect, repeating the steps (1) and (2) if the collection is continued, and completing the collection process and entering the subsequent calibration parameter compensation process if the collection is not continued.
Specifically, if only one projection plane image is collected, the included angle between the optical axis of the optical machine of the projection device and the projection plane is generally about 90 degrees, the more the collected projection plane images are, the better the compensation effect is, but the longer the calculation time is. If the collection is more than or equal to 12, and the included angle between the optical axis of the optical machine of the projection device and the projection surface is between 45 degrees and 135 degrees, better compensation effect can be obtained when the collection is uniformly distributed to obtain the projection image.
The method of step S12 includes:
s121: extracting and matching feature points of a preset image and a projection plane image;
s122: performing polar line correction on the matched characteristic points according to initial attitude parameters of the projection device and the acquisition device, internal parameters of the projection device and internal parameters of the acquisition device;
s123: calculating the coordinate difference amount, namely parallax, between the matched feature points after epipolar line correction in a preset first direction;
s124: obtaining three-dimensional coordinates of each characteristic point according to the parallax and the initial attitude parameters of the projection device and the acquisition device;
s125: obtaining a fitting plane by using a least square method according to the three-dimensional coordinates of each characteristic point;
s126: calculating the average value of the distance values from each characteristic point to the fitting plane to obtain the average value of the distances;
s127: calculating the average value of coordinate difference values in a preset second direction between the matched feature points after epipolar line correction to obtain a difference average value;
s128: and taking the accumulated result of the distance average value and the difference average value as an evaluation function, and optimizing the initial attitude parameters of the projection device and the acquisition device by adopting an optimization algorithm to obtain the actual attitude parameters of the projection device and the acquisition device.
Wherein, the preset first direction means: the preset second direction is the direction consistent with the arrangement direction of the acquisition device and the projection device: and if the preset first direction is the horizontal direction, the preset second direction is the vertical direction, and if the preset first direction is the vertical direction, the preset second direction is the horizontal direction.
Specifically, as shown in fig. 3, the three-dimensional coordinates of the projected feature points in the coordinate system of the projection device and the coordinate system of the acquisition device are respectively calculated according to the pre-calibrated internal parameters of the device, that is, according to the internal parameters of the acquisition device and the internal parameters of the projection device. And establishing a reasonable constraint relation according to the matched characteristic points, and optimizing R and T. The specific process comprises the following steps:
(1) and extracting and matching feature points of the preset image and the projection plane image acquired in the previous step.
Specifically, assuming that the angular points of the checkerboard are used as the feature points, software obtains two groups of sub-pixel angular point coordinates with the same arrangement sequence by using an angular point detection algorithm carried by an openCV open source library on a preset image and an acquired projection plane image respectively, and the coordinates are relative to respective image coordinate systems. Because the sequence of the corner point coordinates is the same, in each group of corner point coordinates, the corner points with the same index number correspond to the same feature point, and the process of extraction and matching is completed.
(2) And carrying out epipolar correction on the characteristic points according to the initial attitude parameters R and T of the projection device and the acquisition device, the internal parametric distortion of the acquisition device and the internal parametric distortion of the projection device.
Specifically, an epipolar correction function in the openCV is used, and pre-calibrated device internal parameters, distortion and attitude parameters are input to obtain the coordinates of the characteristic points after epipolar correction.
(3) And if the acquisition device and the projection device are arranged in the horizontal direction, calculating the coordinate difference amount in the horizontal direction between the matched characteristic points after epipolar line correction, namely parallax. If the alignment is in the vertical direction, the coordinate difference between the matched feature points after epipolar line correction in the vertical direction is calculated, and this embodiment is exemplified by the acquisition device and the projection device being in the horizontal direction (the preset first direction).
The specific calculation method comprises the following steps: the polar line corrected feature point coordinates in the step (2) are still x and y two-dimensional, due to polar line constraint, the matched feature points should only have difference in a certain coordinate direction, and if the difference in the x direction exists, the parallax is the difference of the polar line corrected coordinates of the two matched points in the x direction.
(4) According to the parallax and the initial attitude parameters R, T of the projection device and the acquisition device, and according to the triangulation method of stereovision, the software can obtain the three-dimensional coordinates of the characteristic points, P1(x, y, z), P2(x, y, z) and the like. However, since R and T do not match the actual situation, the three-dimensional coordinates are not accurate.
Specifically, the three-dimensional coordinates z ═ b × f/d, x ═ z × xl/d, and y ═ z × yl/f of the final three-dimensional points are obtained by using a triangulation method, where b and f are the base length and the focal length, respectively, and these two parameters are obtained during the epipolar line correction calculation. d is the parallax, and xl, yl are the coordinates of the projection device feature points corresponding to the stereo point after epipolar line correction.
(5) According to the three-dimensional coordinates of the extracted characteristic points, a fitting plane is obtained by using a least square method, and the plane form is as follows: ax + by + c ═ z. When the characteristic points are actually collected, the characteristic points are located on a plane, and the software optimizes R and T by using the characteristic as an evaluation function. First, a plane fitting is performed on the calculated three-dimensional feature points.
Specifically, all the feature points in step (4) are put into the above equation according to the least square method, and are expressed as Ax ═ B in a matrix form, where a ═ is (x1, y1, 1; x2, y2,1.. xn, yn, 1); b ═ B (z 1; z 2;. zn); and obtaining x according to the matrix pseudo-inverse and multiplication, wherein x is (a, B, c) which is a parameter that the software wants to solve.
(6) The distances of the feature points to the plane are calculated and averaged.
Specifically, d ═ ax + by + c-z |/(sqrt (a ^2+ b ^2+1)) is obtained according to a point-to-plane calculation formula.
(7) An average value of the differences in the vertical direction (preset second direction) after the epipolar line correction is calculated.
The specific calculation method comprises the following steps: the feature point coordinates after polar line correction in step (2) are still x, y two-dimensional, and due to polar line constraint, the matched feature points should have parallax only in a certain coordinate direction, and assuming that the parallax is in the x direction at this time, the difference in the y direction should theoretically be 0. The software represents the part of the difference by absolute value or square of the difference, and calculates the average value of the part of the difference.
(8) And (5) accumulating the results of the step (6) and the step (7) as an evaluation function of the optimization algorithm. When R and T are correct, the evaluation function should be 0. Optionally, the weights of the two accumulation factors may be adjusted.
Specifically, the nonlinear optimization method requires an evaluation function, which is the accumulation described herein, and optimization factors, which are R and T.
(9) And optimizing R and T by using an LM optimization algorithm or a Gauss-Newton method optimization algorithm to obtain the actual attitude parameters of the projection device and the acquisition device.
It should be noted that the calibration parameter compensation method for the projection system further includes step S10: and judging whether the trapezoidal correction effect meets the preset requirement, and if not, entering the subsequent steps.
Specifically, the method of step S10 includes:
s101: controlling a projection device to project a preset image onto a projection surface, wherein the preset image at least comprises 9 feature points with known pixel coordinates;
s102: controlling an acquisition device to shoot a projection picture on a projection plane to obtain a projection plane image;
s103: extracting and matching feature points of a preset image and a projection plane image;
s104: performing polar line correction on the matched feature points, and calculating an average value of coordinate difference values in a preset second direction between the matched feature points after the polar line correction;
s105: and entering a calibration parameter compensation step when the average value of the coordinate difference values is larger than a preset threshold value.
Specifically, after the user turns on the projection system, the projection system automatically performs the keystone correction, the projection system detects the effect of the keystone correction, and if the effect is not as expected, it is determined that the parameter compensation is required. The specific effect confirmation method confirms by epipolar constraint: the two optical devices after epipolar line correction only have parallax in one direction, at the moment, software calculates the average value of the difference in the other direction, and when the average value is greater than a certain threshold value, the software considers that calibration needs to be carried out again, or an external input signal triggers a recalibration process.
The epipolar constraint calculation process is as follows:
(1) the projection device projects a preset image containing the characteristic points to a projection surface;
(2) the acquisition device shoots the projection picture to obtain a projection plane image, and the feature points are matched according to a preset image and the projection plane image;
(3) performing epipolar line correction on the matched characteristic points, and calculating the difference average value of the matched characteristic points in the direction vertical to the epipolar line;
the specific calculation method comprises the following steps: the polar line corrected feature point coordinates in the step (2) are still x and y two-dimensional, due to polar line constraint, the matched feature point should have parallax only in a certain coordinate direction, and assuming that the parallax is in the x direction, the polar line corrected feature point coordinates are the difference of the two matched point polar lines in the y direction, and theoretically should be 0.
(4) And when the average value is larger than the preset threshold value, the system considers that the calibration needs to be carried out again.
In conclusion, the technical scheme of the application enables the pre-calibration parameters to meet the use requirements of different projection distances by adopting a method of dynamically optimizing the pre-calibration parameters when the projection system is used, is simple to operate and is suitable for wide popularization.
As shown in fig. 4, an embodiment of the present invention further provides a calibration parameter compensation apparatus for a projection system, where the projection system includes a projection apparatus and an acquisition apparatus, and the calibration parameter includes: projection arrangement internal parameter, collection system internal parameter, projection arrangement and collection system initial attitude parameter, compensation arrangement includes:
the projection acquisition control module 21 is configured to control the projection device to project a preset image onto the projection surface, where the preset image at least includes 9 feature points with known pixel coordinates, and control the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
and the attitude parameter calculation module 22 is configured to calculate actual attitude parameters of the projection device and the acquisition device according to the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device, and the initial attitude parameters of the projection device and the acquisition device.
It should be noted that the projection acquisition control module 21 is further configured to control the projection apparatus to project preset images onto the projection surface from different angles, where the preset images at least include 9 feature points with known pixel coordinates, and control the acquisition apparatus to shoot a projection picture on the projection surface to obtain multiple corresponding projection surface images.
Note that the attitude parameter calculation module 22 includes:
the characteristic point matching unit is used for extracting and matching the characteristic points of the preset image and the projection plane image;
the polar line correction unit is used for carrying out polar line correction on the matched characteristic points according to the initial attitude parameters of the projection device and the acquisition device, the internal parameters of the projection device and the internal parameters of the acquisition device;
the parallax calculation unit is used for calculating the coordinate difference amount in the preset first direction between the matched characteristic points after epipolar line correction, namely parallax;
the coordinate calculation unit is used for obtaining three-dimensional coordinates of each characteristic point according to the parallax and the initial attitude parameters of the projection device and the acquisition device;
the plane fitting unit is used for obtaining a fitting plane by utilizing a least square method according to the three-dimensional coordinates of each characteristic point;
the distance calculation unit is used for calculating the average value of the distance values from each characteristic point to the fitting plane to obtain the distance average value;
the difference calculating unit is used for calculating the average value of coordinate difference quantities in a preset second direction between the matched characteristic points after epipolar line correction to obtain a difference average value;
and the parameter optimization unit is used for optimizing the initial attitude parameters of the projection device and the acquisition device by adopting an optimization algorithm by taking the accumulated result of the distance average value and the difference average value as an evaluation function so as to obtain the actual attitude parameters of the projection device and the acquisition device.
It should be noted that the calibration parameter compensation apparatus of the projection system further includes a trapezoidal correction determining module 20, which is configured to determine whether the trapezoidal correction effect meets a preset requirement, and if not, enter the projection acquisition control module.
The trapezoidal correction determining module 20 includes:
the image projection control unit is used for controlling the projection device to project a preset image to the projection surface, wherein the preset image at least comprises 9 characteristic points with known pixel coordinates;
the image acquisition control unit is used for controlling the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
the parameter compensation judging unit is used for extracting and matching the feature points of the preset image and the projection plane image; performing polar line correction on the matched feature points, and calculating the coordinate difference value average value in a preset second direction between the matched feature points after the polar line correction; and when the average value of the coordinate difference values is greater than a preset threshold value, entering a projection acquisition control module.
The embodiment of the present invention further provides a calibration parameter compensation apparatus for a projection system, including: a memory for storing a computer program; a processor for executing a computer program to implement the steps of the calibration parameter compensation method of the projection system as described above.
The embodiment of the invention further provides a readable storage medium, which stores a computer program, and the computer program, when executed by a processor, implements the steps of the calibration parameter compensation method of the projection system.
For the description of the features in the embodiment corresponding to fig. 4, reference may be made to the related description of the embodiments corresponding to fig. 1 to fig. 3, which is not repeated here.
The calibration parameter compensation method, the calibration parameter compensation device and the readable storage medium of the projection system provided by the embodiments of the present invention are described in detail above. The embodiments are described in a progressive manner in the specification, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The device disclosed by the embodiment corresponds to the method disclosed by the embodiment, so that the description is simple, and the relevant points can be referred to the method part for description. It should be noted that, for those skilled in the art, it is possible to make various improvements and modifications to the present invention without departing from the principle of the present invention, and those improvements and modifications also fall within the scope of the claims of the present invention.
Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative components and steps have been described above generally in terms of their functionality in order to clearly illustrate this interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in Random Access Memory (RAM), memory, Read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
Claims (10)
1. A calibration parameter compensation method for a projection system, the projection system comprising a projection device and an acquisition device, the calibration parameter comprising: the compensation method comprises the following steps of measuring internal parameters of the projection device, internal parameters of the acquisition device, initial attitude parameters of the projection device and the acquisition device, and the compensation method comprises the following steps:
controlling a projection device to project a preset image onto a projection surface, wherein the preset image at least comprises 9 feature points with known pixel coordinates, and controlling an acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
and according to the preset image, the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device, extracting and matching feature points of the preset image and the projection plane image, respectively calculating the three-dimensional coordinates of the feature points in a projection device coordinate system and an acquisition device coordinate system, establishing a constraint relation according to the matching relation of the feature points and the corresponding three-dimensional coordinates, optimizing the initial attitude parameters of the projection device and the acquisition device, and obtaining the actual attitude parameters of the projection device and the acquisition device.
2. The calibration parameter compensation method of the projection system according to claim 1, wherein the method for controlling the projection device to project a preset image onto the projection plane, the preset image at least includes 9 feature points with known pixel coordinates, and the method for controlling the acquisition device to capture the projection image on the projection plane to obtain the projection plane image comprises:
and controlling a projection device to project preset images to a projection plane from different angles, wherein the preset images at least comprise 9 feature points with known pixel coordinates, and controlling an acquisition device to shoot projection pictures on the projection plane to obtain a plurality of corresponding projection plane images.
3. The calibration parameter compensation method of the projection system according to claim 1, wherein the method for extracting and matching feature points of the preset image and the projection plane image according to the preset image, the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device, and the initial pose parameters of the projection device and the acquisition device, respectively calculating the three-dimensional coordinates of the feature points in the projection device coordinate system and the acquisition device coordinate system, respectively, establishing a constraint relationship according to the matching relationship of the feature points and the corresponding three-dimensional coordinates, and optimizing the initial pose parameters of the projection device and the acquisition device to obtain the actual pose parameters of the projection device and the acquisition device comprises:
extracting and matching feature points of a preset image and a projection plane image;
performing polar line correction on the matched characteristic points according to initial attitude parameters of the projection device and the acquisition device, internal parameters of the projection device and internal parameters of the acquisition device;
calculating the coordinate difference amount, namely parallax, between the matched feature points after epipolar line correction in a preset first direction;
obtaining three-dimensional coordinates of each characteristic point according to the parallax and the initial attitude parameters of the projection device and the acquisition device;
obtaining a fitting plane by using a least square method according to the three-dimensional coordinates of each characteristic point;
calculating the average value of the distance values from each characteristic point to the fitting plane to obtain the average value of the distances;
calculating the average value of coordinate difference values in a preset second direction between the matched feature points after epipolar line correction to obtain a difference average value;
and taking the accumulated result of the distance average value and the difference average value as an evaluation function, and optimizing the initial attitude parameters of the projection device and the acquisition device by adopting an optimization algorithm to obtain the actual attitude parameters of the projection device and the acquisition device.
4. The calibration parameter compensation method of a projection system according to claim 1, wherein the step of controlling the projection device to project a preset image onto the projection plane, the preset image at least includes 9 feature points with known pixel coordinates, and before the step of controlling the acquisition device to capture the projection image on the projection plane to obtain the projection plane image, the method further comprises: judging whether the trapezoidal correction effect meets the preset requirement, and if not, entering the subsequent step; the method for judging whether the trapezoidal correction effect meets the preset requirement comprises the following steps:
controlling a projection device to project a preset image onto a projection surface, wherein the preset image at least comprises 9 feature points with known pixel coordinates;
controlling an acquisition device to shoot a projection picture on the projection plane to obtain a projection plane image;
extracting and matching feature points of a preset image and a projection plane image;
performing polar line correction on the matched feature points, and calculating an average value of coordinate difference values in a preset second direction between the matched feature points after the polar line correction;
and entering a calibration parameter compensation step when the average value of the coordinate difference values is larger than a preset threshold value.
5. A calibration parameter compensation device for a projection system, the projection system comprising a projection device and an acquisition device, the calibration parameter comprising: projection arrangement internal parameter, collection system internal parameter, projection arrangement and collection system initial attitude parameter, compensation arrangement includes:
the projection acquisition control module is used for controlling the projection device to project a preset image to a projection surface, wherein the preset image at least comprises 9 characteristic points with known pixel coordinates, and simultaneously controlling the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
and the attitude parameter calculation module is used for extracting and matching feature points of the preset image and the projection plane image according to the preset image, the projection plane image, the internal parameters of the projection device, the internal parameters of the acquisition device and the initial attitude parameters of the projection device and the acquisition device, respectively calculating the three-dimensional coordinates of the feature points in the projection device coordinate system and the acquisition device coordinate system, establishing a constraint relation according to the matching relation of the feature points and the corresponding three-dimensional coordinates, and optimizing the initial attitude parameters of the projection device and the acquisition device to obtain the actual attitude parameters of the projection device and the acquisition device.
6. The calibration parameter compensation device of the projection system according to claim 5, wherein the projection acquisition control module is further configured to control the projection device to project preset images onto the projection surface from different angles, the preset images at least include 9 feature points with known pixel coordinates, and the acquisition device is controlled to capture the projection image on the projection surface to obtain a plurality of corresponding projection surface images.
7. The calibration parameter compensation device of a projection system as claimed in claim 5, wherein the attitude parameter calculation module comprises:
the characteristic point matching unit is used for extracting and matching the characteristic points of the preset image and the projection plane image;
the polar line correction unit is used for carrying out polar line correction on the matched characteristic points according to the initial attitude parameters of the projection device and the acquisition device, the internal parameters of the projection device and the internal parameters of the acquisition device;
the parallax calculation unit is used for calculating the coordinate difference amount in the preset first direction between the matched characteristic points after epipolar line correction, namely parallax;
the coordinate calculation unit is used for obtaining three-dimensional coordinates of each characteristic point according to the parallax and the initial attitude parameters of the projection device and the acquisition device;
the plane fitting unit is used for obtaining a fitting plane by utilizing a least square method according to the three-dimensional coordinates of each characteristic point;
the distance calculation unit is used for calculating the average value of the distance values from each characteristic point to the fitting plane to obtain the distance average value;
the difference calculating unit is used for calculating the average value of coordinate difference quantities in a preset second direction between the matched characteristic points after epipolar line correction to obtain a difference average value;
and the parameter optimization unit is used for optimizing the initial attitude parameters of the projection device and the acquisition device by adopting an optimization algorithm by taking the accumulated result of the distance average value and the difference average value as an evaluation function so as to obtain the actual attitude parameters of the projection device and the acquisition device.
8. The calibration parameter compensation device of the projection system as claimed in claim 5, further comprising a trapezoidal correction judgment module for judging whether the trapezoidal correction effect meets a preset requirement, and if not, entering the projection acquisition control module; the trapezoidal correction judging module comprises:
the image projection control unit is used for controlling the projection device to project a preset image to the projection surface, wherein the preset image at least comprises 9 characteristic points with known pixel coordinates;
the image acquisition control unit is used for controlling the acquisition device to shoot a projection picture on the projection surface to obtain a projection surface image;
the parameter compensation judging unit is used for extracting and matching the feature points of the preset image and the projection plane image; performing polar line correction on the matched feature points, and calculating the coordinate difference value average value in a preset second direction between the matched feature points after the polar line correction; and when the average value of the coordinate difference values is greater than a preset threshold value, entering a projection acquisition control module.
9. A calibration parameter compensation apparatus for a projection system, comprising:
a memory for storing a computer program;
a processor for executing said computer program for carrying out the steps of the calibration parameter compensation method for a projection system as claimed in any one of claims 1 to 4.
10. A readable storage medium, characterized in that the readable storage medium stores a computer program, which when executed by a processor implements the steps of the calibration parameter compensation method for a projection system according to any of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911325236.1A CN110933391B (en) | 2019-12-20 | 2019-12-20 | Calibration parameter compensation method and device for projection system and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911325236.1A CN110933391B (en) | 2019-12-20 | 2019-12-20 | Calibration parameter compensation method and device for projection system and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110933391A CN110933391A (en) | 2020-03-27 |
| CN110933391B true CN110933391B (en) | 2021-11-09 |
Family
ID=69864528
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911325236.1A Active CN110933391B (en) | 2019-12-20 | 2019-12-20 | Calibration parameter compensation method and device for projection system and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110933391B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111487043B (en) * | 2020-05-07 | 2021-11-09 | 北京的卢深视科技有限公司 | Method for determining calibration parameters of speckle projector of monocular speckle structured light system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103047943A (en) * | 2012-12-31 | 2013-04-17 | 吉林大学 | Method for detecting vehicle door outer plate shape and size based on single-projection encoding structured light |
| CN103234482A (en) * | 2013-04-07 | 2013-08-07 | 哈尔滨工程大学 | Structured light measuring system calibration method based on sinusoidal grating |
| CN103942796A (en) * | 2014-04-23 | 2014-07-23 | 清华大学 | High-precision projector and camera calibration system and method |
| CN108269288A (en) * | 2017-12-28 | 2018-07-10 | 中国科学院自动化研究所 | Intelligent abnormal projects contactless interactive system and method |
| CN108362228A (en) * | 2018-02-11 | 2018-08-03 | 西安知象光电科技有限公司 | A kind of hybrid three-dimensional measuring apparatus of finishing tool grating and measurement method based on double ray machines |
| JP2019007995A (en) * | 2017-06-20 | 2019-01-17 | パナソニックＩｐマネジメント株式会社 | Calibration apparatus of projection display device, calibration method, program, projection display device, and projection display system |
| CN110099267A (en) * | 2019-05-27 | 2019-08-06 | 广州市讯码通讯科技有限公司 | Trapezoidal correcting system, method and projector |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TWI592020B (en) * | 2016-08-23 | 2017-07-11 | 國立臺灣科技大學 | Image correction method of projector and image correction system |
| US20190052851A1 (en) * | 2017-08-11 | 2019-02-14 | Hilti Aktiengesellschaft | System and Method for Recalibrating a Projector System |
| CN107680159B (en) * | 2017-10-16 | 2020-12-08 | 西北工业大学 | Space non-cooperative target three-dimensional reconstruction method based on projection matrix |
| CN108171759A (en) * | 2018-01-26 | 2018-06-15 | 上海小蚁科技有限公司 | The scaling method of double fish eye lens panorama cameras and device, storage medium, terminal |
-
2019
- 2019-12-20 CN CN201911325236.1A patent/CN110933391B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103047943A (en) * | 2012-12-31 | 2013-04-17 | 吉林大学 | Method for detecting vehicle door outer plate shape and size based on single-projection encoding structured light |
| CN103234482A (en) * | 2013-04-07 | 2013-08-07 | 哈尔滨工程大学 | Structured light measuring system calibration method based on sinusoidal grating |
| CN103942796A (en) * | 2014-04-23 | 2014-07-23 | 清华大学 | High-precision projector and camera calibration system and method |
| JP2019007995A (en) * | 2017-06-20 | 2019-01-17 | パナソニックＩｐマネジメント株式会社 | Calibration apparatus of projection display device, calibration method, program, projection display device, and projection display system |
| CN108269288A (en) * | 2017-12-28 | 2018-07-10 | 中国科学院自动化研究所 | Intelligent abnormal projects contactless interactive system and method |
| CN108362228A (en) * | 2018-02-11 | 2018-08-03 | 西安知象光电科技有限公司 | A kind of hybrid three-dimensional measuring apparatus of finishing tool grating and measurement method based on double ray machines |
| CN110099267A (en) * | 2019-05-27 | 2019-08-06 | 广州市讯码通讯科技有限公司 | Trapezoidal correcting system, method and projector |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110933391A (en) | 2020-03-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110677634B (en) | Trapezoidal correction method, device and system for projector and readable storage medium | |
| CN109887087B (en) | SLAM mapping method and system for vehicle | |
| CN103776419B (en) | A kind of binocular distance measurement method improving measurement range | |
| KR102085228B1 (en) | Imaging processing method and apparatus for calibrating depth of depth sensor | |
| CN100442141C (en) | Image projection method and device | |
| TWI520098B (en) | Image capturing device and method for detecting image deformation thereof | |
| WO2020228694A1 (en) | Camera pose information detection method and apparatus, and corresponding intelligent driving device | |
| CN104820978B (en) | A kind of origin reference location method of CCD camera | |
| CN102436660A (en) | Automatic correction method and device of 3D camera image | |
| CN102661717A (en) | Monocular vision measuring method for iron tower | |
| KR101342393B1 (en) | Georeferencing Method of Indoor Omni-Directional Images Acquired by Rotating Line Camera | |
| CN111540004A (en) | Single-camera polar line correction method and device | |
| CN109410140B (en) | Distortion correction method, device and system and computer readable storage medium | |
| CN104836960A (en) | Image collection system and method | |
| CN104089628A (en) | Self-adaption geometric calibration method of light field camera | |
| CN103017666A (en) | Method and assembly for determining the position of a measurement point in geometric space | |
| CN110933391B (en) | Calibration parameter compensation method and device for projection system and readable storage medium | |
| Patel et al. | Distance measurement system using binocular stereo vision approach | |
| US20200260029A1 (en) | Image processing system for measuring depth and operating method of the same | |
| US11956537B2 (en) | Location positioning device for moving body and location positioning method for moving body | |
| CN106813595B (en) | Three-phase unit characteristic point matching method, measurement method and three-dimensional detection device | |
| CN109084959B (en) | Optical axis parallelism correction method based on binocular distance measurement algorithm | |
| Chatterjee et al. | A nonlinear Gauss–Seidel algorithm for noncoplanar and coplanar camera calibration with convergence analysis | |
| CN111698467A (en) | Intelligent tracking method and system based on multiple cameras | |
| CN116363226A (en) | Real-time multi-camera multi-projector 3D imaging processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |