CN111311682A - Pose estimation method and device in LED screen correction process and electronic equipment - Google Patents

Pose estimation method and device in LED screen correction process and electronic equipment Download PDF

Info

Publication number
CN111311682A
CN111311682A CN202010113353.8A CN202010113353A CN111311682A CN 111311682 A CN111311682 A CN 111311682A CN 202010113353 A CN202010113353 A CN 202010113353A CN 111311682 A CN111311682 A CN 111311682A
Authority
CN
China
Prior art keywords
coordinate system
led screen
image
camera
acquisition module
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010113353.8A
Other languages
Chinese (zh)
Inventor
何志民
王利文
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Colorlight Shenzhen Cloud Technology Co Ltd
Original Assignee
Colorlight Shenzhen Cloud Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Colorlight Shenzhen Cloud Technology Co Ltd filed Critical Colorlight Shenzhen Cloud Technology Co Ltd
Priority to CN202010113353.8A priority Critical patent/CN111311682A/en
Publication of CN111311682A publication Critical patent/CN111311682A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30121—CRT, LCD or plasma display

Abstract

The invention relates to the technical field of LED display screens, and particularly discloses a pose estimation method, a pose estimation device and electronic equipment in an LED screen correction process, wherein the method comprises the steps of acquiring a picture of an LED screen; marking image lamp points in the picture of the LED screen, establishing an image coordinate system by using the picture of the LED screen, and determining the coordinates of the image lamp points in the image coordinate system; establishing a lamp point coordinate system by using the LED screen; establishing a camera coordinate system by taking the optical center of the camera as an origin; establishing a conversion matrix between a lamp point coordinate system and a camera coordinate system; establishing a relation equation of converting the screen light points from the light point coordinate system to the pixel coordinate system by combining the conversion matrix; and obtaining a conversion matrix and the like. According to the method and the device for estimating the pose in the LED screen correction process and the electronic equipment, the LED screen pose can be estimated only by a single picture of a single camera, hardware equipment required by the LED screen pose estimation is reduced, and the operation steps of correction engineers for correcting the LED screen are simplified.

Description

Pose estimation method and device in LED screen correction process and electronic equipment
Technical Field
The invention relates to the technical field of LED display screens, in particular to a pose estimation method and device in an LED screen correction process and electronic equipment.
Background
The correction of the LED display screen is mainly divided into whole screen correction and single box correction. In the whole-screen correction, because the CCD area of the camera is limited, the screen needs to be divided into a plurality of blocks for collection, the position relations between different blocks and the camera are different, and the position relations are usually ignored or simply estimated, so that the block difference of the corrected screen occurs, and the correction effect is seriously influenced; after the factory correction, the field can be assembled again, the position relation between the boxes can be changed, the two boxes at original different positions become two adjacent boxes, the brightness difference can occur between the adjacent boxes, and the correction effect is seriously influenced. In single-box correction, due to artificial difference, the box body and the camera cannot be completely aligned, a slight inclination angle exists, and after correction, the splicing of the multiple box bodies has gradient gradual change, so that the correction effect is seriously influenced.
In order to improve the correction effect, a technician needs to perform pose estimation on the LED screen. In the traditional pose estimation, a plurality of cameras need to be calibrated together, a chessboard is usually required to be used as a calibration plate, and the calibration precision can be improved only by shooting a plurality of chequers. In the correction process of the LED screen body, the on-site calibration is difficult, and the calibration needs to be carried out again every time, which cannot be realized in the actual operation.
Disclosure of Invention
Aiming at the technical problems in the prior art, the invention provides a pose estimation method and device in an LED screen correction process and electronic equipment.
The invention provides a pose estimation method in an LED screen correction process, which comprises the following steps:
acquiring a picture of the LED screen;
marking image lamp points in the picture of the LED screen, establishing an image coordinate system by using the picture of the LED screen, and determining the coordinates of the image lamp points in the image coordinate system;
establishing a lamp point coordinate system by using the LED screen, and determining the coordinates of the lamp points on the screen in the lamp point coordinate system;
establishing a camera coordinate system by taking the optical center of the camera as an origin;
establishing a conversion matrix between a lamp point coordinate system and a camera coordinate system;
corresponding image light points in the LED screen picture to screen light points in the LED screen, and establishing a relation equation for converting the screen light points from a light point coordinate system to a pixel coordinate system by combining a conversion matrix;
and solving a conversion matrix according to the relation equation, and outputting the conversion matrix as an LED screen position and posture estimation result.
Further, the transformation matrix includes a rotation matrix and a translation vector.
Further, an image coordinate system where the picture of the LED screen is located is represented as O-xy, a pixel coordinate system is represented as uv, and a lamp point coordinate system is represented as Ow-XwYwZwThe camera coordinate system is represented as Oc-XcYcZc；
The relation equation is:
Figure BDA0002390751930000021
wherein: zcThe distance between the camera and the LED screen,
Figure BDA0002390751930000022
is the coordinates of the image light points in the pixel coordinate system,
Figure BDA0002390751930000023
for the camera internal reference, fxRepresenting the number of pixels of the focal length f of the camera on the x-axis of the image coordinate system, fyThe number of pixels representing the focal length f of the camera on the y-axis of the image coordinate system,
Figure BDA0002390751930000024
for the transformation matrix, R is the rotation matrix, T is the translation vector,
Figure BDA0002390751930000025
the coordinates of the screen light point in the light point coordinate system.
Further, camera reference
Figure BDA0002390751930000026
Wherein d isxRepresenting the width of each row of the pixel coordinate system in the image coordinate system, dyWhich represents the height of each line of the pixel coordinate system in the image coordinate system, and f is the camera focal length.
Further, the moment of conversionMatrix of
Figure BDA0002390751930000027
Rotation matrix
Figure BDA0002390751930000028
Figure BDA0002390751930000031
Translation vector
Figure BDA0002390751930000032
Further, the relationship equation is transformed into
Figure BDA0002390751930000033
Further, determining coordinates of the image light point in an image coordinate system includes:
carrying out binarization processing on the LED screen picture;
searching a connected domain;
and taking the central coordinate of the connected domain as the coordinate of the image lamp point.
The invention also provides a pose estimation device in the LED screen correction process, which comprises the following steps: the system comprises an LED screen picture acquisition module, an image coordinate acquisition module, a pixel coordinate acquisition module, a lamp point coordinate acquisition module, a camera coordinate acquisition module and an LED screen posture estimation module; wherein the content of the first and second substances,
the LED screen picture acquisition module is connected with the image coordinate acquisition module and used for acquiring the picture of the LED screen;
the image coordinate acquisition module is connected with the LED screen image acquisition module, the pixel coordinate acquisition module and the LED screen position and posture estimation module and used for marking image lamp points in the image of the LED screen, establishing an image coordinate system by using the image of the LED screen and determining the coordinates of the image lamp points in the image coordinate system;
the pixel coordinate acquisition module is connected with the image coordinate acquisition module and the LED screen position and posture estimation module and used for establishing a pixel coordinate system by using the picture of the LED screen and determining the coordinates of the image lamp points in the pixel coordinate system;
the lamp point coordinate acquisition module is connected with the LED screen position and posture estimation module and used for establishing a lamp point coordinate system by using the LED screen and determining the coordinates of the screen lamp points in the lamp point coordinate system;
the camera coordinate acquisition module is connected with the LED screen position and attitude estimation module and used for establishing a camera coordinate system by taking the optical center of the camera as an origin;
and the LED screen position and posture estimation module is connected with the image coordinate acquisition module, the pixel coordinate acquisition module, the lamp point coordinate acquisition module and the camera coordinate acquisition module and used for establishing a conversion matrix between a lamp point coordinate system and a camera coordinate system, corresponding image lamp points in the LED screen picture to screen lamp points in the LED screen, establishing a relation equation for converting the screen lamp points from the lamp point coordinate system to the pixel coordinate system by combining the conversion matrix, and solving the conversion matrix according to the relation equation to serve as an LED screen position and posture estimation result and outputting the LED screen position and posture estimation result.
The invention also provides electronic equipment which estimates the pose of the LED screen by the pose estimation method.
Further, the electronic device is a computer.
The invention also provides an electronic device comprising a processor and a memory,
a memory storing computer instructions;
a processor configured to execute computer instructions to implement the correction method described above.
According to the method and the device for estimating the pose in the LED screen correction process and the electronic equipment, the pose of the LED screen can be estimated only by a single picture of a single camera, a specific calibration plate is not required to be additionally added for calibrating the position relation between the LED screen and the camera, hardware equipment required by the estimation of the pose of the LED screen is reduced, and the operation steps of correcting the LED screen by a correction engineer are simplified. On the other hand, the method of the embodiment realizes the estimation of the LED screen position and posture, and further improves the final effect of LED screen correction.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flowchart illustrating the steps of a pose estimation method in an LED screen calibration process according to an embodiment of the present invention;
FIG. 2 is a diagram illustrating a position relationship between an image coordinate system and a pixel coordinate system in the pose estimation method in the LED screen calibration process according to the embodiment of the present invention;
FIG. 3 is a diagram illustrating a relationship between an image coordinate system and a camera coordinate system in the pose estimation method in the LED screen calibration process according to the embodiment of the present invention;
FIG. 4 is a flowchart illustrating the steps of a pose estimation method in an LED screen calibration process according to another embodiment of the present invention;
FIG. 5 is a schematic structural diagram of a pose estimation device in the LED screen correction process according to an embodiment of the present invention;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The pose estimation method in the LED screen correction process, disclosed by the embodiment of the invention, as shown in FIG. 1, comprises the following steps:
step S101: and acquiring a picture of the LED screen.
In the embodiment of the invention, the picture of the LED screen is obtained by shooting the LED screen to be detected by a camera, and the picture of the LED screen is used as the basis for estimating the position posture in the method. The acquisition mode of the picture of the LED screen can be realized by connecting the LED screen with a camera in a wired communication manner or in a wireless communication manner.
Step S102: marking image lamp points in the picture of the LED screen, establishing an image coordinate system and a pixel coordinate system by using the picture of the LED screen, and determining the coordinates of the image lamp points in the image coordinate system and the pixel coordinate system respectively.
The image coordinate system can be expressed as o-xy with the origin at the center point of the image in mm.
The pixel coordinate system can be expressed as uv with the origin in the upper left corner of the image in pixels.
The pixel coordinate system and the image coordinate system are both on the imaging plane, except for the respective origin and measurement unit. The origin of the image coordinate system is the focal point of the camera optical axis and the imaging plane, typically the midpoint of the imaging plane. The unit of the image coordinate system is mm, which belongs to the physical unit, and the unit of the pixel coordinate system is pixel, and usually, the way of describing one pixel point is several lines and several columns, as shown in fig. 2, the conversion between the image coordinate system o-xy and the pixel coordinate system uv is:
Figure BDA0002390751930000051
therefore, the first and second electrodes are formed on the substrate,
Figure BDA0002390751930000052
wherein d isxIndicates how much mm, d, the width of each row of the pixel coordinate system isyRepresenting how many mm the height of each row of the pixel coordinate system is, i.e. 1pixel dxmm，1pixel＝dymm。
Step S103: and establishing a lamp point coordinate system by using the LED screen, and determining the coordinates of the screen lamp points in the lamp point coordinate system.
The lamp point coordinate system may be represented as Ow-XwYwZwThe lamp point coordinate system may also be referred to as a world coordinate system, and belongs to a three-dimensional coordinate system, and the unit is m.
Step S104: and establishing a camera coordinate system by taking the optical center of the camera as an origin.
The camera coordinate system can representIs Oc-XcYcZcThe optical center of the camera is the origin in m.
Step S105: and establishing a conversion matrix between the lamp point coordinate system and the camera coordinate system.
The conversion between the lamp point coordinate system and the camera coordinate system belongs to rigid body conversion, namely, an object does not deform, so that the lamp point coordinate system can be converted into the camera coordinate system only by rotating and translating the lamp point coordinate system, and the conversion matrix of the embodiment of the invention comprises a rotation matrix and a translation vector.
For the rotation matrix, rotating different angles around different coordinate axes to obtain corresponding rotation matrix, taking the rotation of θ degrees around the Z axis as an example, the coordinate of the point p (x, y, Z) is transformed into p '(x', y ', Z'), and the relationship between them is: x 'cos θ -y' sin θ; y ═ x 'sin θ + y' cos θ; z ═ z';
therefore, the first and second electrodes are formed on the substrate,
Figure BDA0002390751930000061
similarly, rotating around X-axis and Y-axis respectively
Figure BDA0002390751930000062
Degrees and ω degrees, one can obtain:
Figure BDA0002390751930000063
Figure BDA0002390751930000064
then, the rotation matrix R ═ R1R2R3Let the translation vector be T,
point P (X) in the lamp point coordinate systemw,Yw,Zw) The coordinates in the camera coordinate system are:
Figure BDA0002390751930000065
therefore, the first and second electrodes are formed on the substrate,
Figure BDA0002390751930000066
r is a 3 x 3 matrix which can be written
Figure BDA0002390751930000067
T is a 3 x 1 matrix which can be written as
Figure BDA0002390751930000068
Convert the matrix into
Figure BDA0002390751930000069
Can be written
Figure BDA00023907519300000610
Step S106: and corresponding image light points in the LED screen picture to screen light points in the LED screen, and establishing a relation equation for converting the screen light points from a light point coordinate system to a pixel coordinate system by combining the conversion matrix.
The relationship between the lamp point coordinate system and the pixel coordinate system is established and is related to the camera coordinate system and the image coordinate system, and firstly, the conversion from the camera coordinate system to the image coordinate system is explained.
From the camera coordinate system to the image coordinate system, belonging to perspective projection relation, from 3D to 2D. As shown in FIG. 3, from the similar triangles, Δ ABO is knownc～ΔoCOc,ΔPBOc～ΔPCOcTherefore, the temperature of the molten steel is controlled,
Figure BDA0002390751930000071
Figure BDA0002390751930000072
therefore, it is
Figure BDA0002390751930000073
The relationship equation for converting the screen light point from the light point coordinate system to the pixel coordinate system is:
Figure BDA0002390751930000074
wherein: zcThe distance between the camera and the LED screen,
Figure BDA0002390751930000075
is the coordinates of the image light points in the pixel coordinate system,
Figure BDA0002390751930000076
for the camera internal reference, fxRepresenting the number of pixels of the focal length f of the camera on the x-axis of the image coordinate system, fyThe number of pixels representing the focal length f of the camera on the y-axis of the image coordinate system. Internal reference of camera
Figure BDA0002390751930000077
Figure BDA0002390751930000078
dxRepresenting the length of each row of the pixel coordinate system in the image coordinate system, dyIndicating the length of each line of the pixel coordinate system in the image coordinate system.
Figure BDA0002390751930000079
For the transformation matrix, R is the rotation matrix, T is the translation vector,
Figure BDA00023907519300000710
the coordinates of the screen light point in the light point coordinate system.
Step S107: and solving a conversion matrix according to the relation equation, and outputting the conversion matrix as an LED screen position and posture estimation result.
Camera internal reference in relational equations
Figure BDA00023907519300000711
Distortion factors are not considered for the present embodiment for known parameters of the camera. Coordinates of image light points in a pixel coordinate system
Figure BDA00023907519300000712
As already derived in step S102, is a known quantity; coordinates of screen lamp points in lamp point coordinate system
Figure BDA0002390751930000081
Which has been derived in step S104 as a known quantity. In summary, the distance Z between the camera and the LED screencFor unknowns, the matrix is transformed
Figure BDA0002390751930000082
For unknown quantities, rotating the matrix
Figure BDA0002390751930000083
With three degrees of freedom, translation matrix
Figure BDA0002390751930000084
With three degrees of freedom, four corner points (total 8 degrees of freedom) of the LED screen are used for calculating and solving the unknown quantity ZcAnd
Figure BDA0002390751930000085
and the LED screen position and posture estimation result is output.
The solving process for the transformation matrix can be performed as follows:
the relation equation:
Figure BDA0002390751930000086
the expansion is as follows:
Figure BDA0002390751930000087
the matrix size corresponds to: [3 × 1] - [3 × 4] [4 × 1 ].
Removing all zero terms to obtain:
Figure BDA0002390751930000088
the matrix size corresponds to: [3 × 1] - [3 × 3] [3 × 4] [4 × 1 ].
In actual use, Z of lamp point coordinate systemwDefaults to 0, takes Z ascMoving to the right side of the equation, we get:
Figure BDA0002390751930000089
the matrix size corresponds to: [3 × 1] - [3 × 3] [3 × 4] [4 × 1 ].
Respectively substituting four angular points of the LED screen to calculate ZcAnd
Figure BDA0002390751930000091
specifically, as shown in fig. 4, in step S102, determining coordinates of the image light point in the image coordinate system specifically includes:
step S1021: and carrying out binarization processing on the LED screen picture.
The binarization process can adopt a Otsu threshold binarization mode, which belongs to a common means of a person skilled in the art and is not described herein.
Step S1022: and searching a connected domain.
Step S1023: and taking the central coordinate of the connected domain as the coordinate of the image lamp point.
According to the pose estimation method in the LED screen correction process, the LED screen pose can be estimated only by a single camera, the position relation between the LED screen and the camera is calibrated without additionally adding a specific calibration plate, hardware equipment required by the LED screen pose estimation is reduced, and the operation steps of correction engineers for correcting the LED screen are simplified. On the other hand, the method of the embodiment realizes the estimation of the LED screen position and posture, and further improves the final effect of LED screen correction.
The invention also provides a pose estimation device in the process of correcting the LED screen, as shown in FIG. 5, comprising: the system comprises an LED screen picture acquisition module 101, an image coordinate acquisition module 102, a pixel coordinate acquisition module 103, a lamp point coordinate acquisition module 104, a camera coordinate acquisition module 105 and an LED screen posture estimation module 106; the LED screen image acquisition module 101 is connected with the image coordinate acquisition module 102 and is used for acquiring an LED screen image; the image coordinate acquisition module 102 is connected with the LED screen image acquisition module 101, the pixel coordinate acquisition module 103 and the LED screen position and posture estimation module 106, and is used for marking image light points in the LED screen image, establishing an image coordinate system by using the LED screen image, and determining coordinates of the image light points in the image coordinate system; the pixel coordinate acquisition module 103 is connected with the image coordinate acquisition module 102 and the LED screen posture estimation module 106, and is used for establishing a pixel coordinate system by using the picture of the LED screen and determining the coordinates of the image lamp points in the pixel coordinate system; the lamp point coordinate acquisition module 104 is connected with the LED screen position and posture estimation module 106 and used for establishing a lamp point coordinate system by using the LED screen and determining the coordinates of the screen lamp points in the lamp point coordinate system; the camera coordinate acquisition module 105 is connected with the LED screen position and attitude estimation module 106 and used for establishing a camera coordinate system by taking the optical center of the camera as an origin; and the LED screen position and posture estimation module 106 is connected to the image coordinate acquisition module 102, the pixel coordinate acquisition module 103, the lamp point coordinate acquisition module 104 and the camera coordinate acquisition module 105, and is configured to establish a conversion matrix between the lamp point coordinate system and the camera coordinate system, correspond image lamp points in the LED screen picture to screen lamp points in the LED screen, establish a relationship equation for converting the screen lamp points from the lamp point coordinate system to the pixel coordinate system by combining the conversion matrix, and obtain the conversion matrix according to the relationship equation to serve as an LED screen position and posture estimation result and output the result.
The method for estimating the position and attitude by the device 100 of the embodiment of the invention is realized by the following steps:
step S101: and acquiring a picture of the LED screen.
Step S102: marking image lamp points in the picture of the LED screen, establishing an image coordinate system and a pixel coordinate system by using the picture of the LED screen, and determining the coordinates of the image lamp points in the image coordinate system and the pixel coordinate system respectively.
Step S103: and establishing a lamp point coordinate system by using the LED screen, and determining the coordinates of the screen lamp points in the lamp point coordinate system.
Step S104: and establishing a camera coordinate system by taking the optical center of the camera as an origin.
Step S105: and establishing a conversion matrix between the lamp point coordinate system and the camera coordinate system.
Step S106: and corresponding image light points in the LED screen picture to screen light points in the LED screen, and establishing a relation equation for converting the screen light points from a light point coordinate system to a pixel coordinate system by combining the conversion matrix.
Step S107: and solving a conversion matrix according to the relation equation, and outputting the conversion matrix as an LED screen position and posture estimation result.
Specifically, step S102 specifically includes:
step S1021: and carrying out binarization processing on the LED screen picture.
Step S1022: and searching a connected domain.
Step S1023: and taking the central coordinate of the connected domain as the coordinate of the image lamp point.
For the description of the execution of each step, reference may be directly made to the embodiments of the foregoing method, which are not described herein again.
According to the pose estimation device in the LED screen correction process, the LED screen pose can be estimated only by a single picture of a single camera, the position relation between the LED screen and the camera is calibrated without additionally adding a specific calibration plate, hardware equipment required by the LED screen pose estimation is reduced, and the operation steps of correcting the LED screen by a correction engineer are simplified. On the other hand, the method of the embodiment realizes the estimation of the LED screen position and posture, and further improves the final effect of LED screen correction.
The invention further provides an electronic device 200, and the electronic device estimates the pose of the LED screen through the pose estimation method. The electronic device in this embodiment may be a computer.
Specifically, as shown in fig. 6, the electronic device 200 includes a memory 201 and a processor 202, where the memory 201 stores computer instructions; a processor 202 configured to execute computer instructions to implement the pose estimation method in the above embodiments.
The method for realizing pose estimation of the electronic device 200 is specifically realized according to the following steps:
step S101: and acquiring a picture of the LED screen.
Step S102: marking image lamp points in the picture of the LED screen, establishing an image coordinate system and a pixel coordinate system by using the picture of the LED screen, and determining the coordinates of the image lamp points in the image coordinate system and the pixel coordinate system respectively.
Step S103: and establishing a lamp point coordinate system by using the LED screen, and determining the coordinates of the screen lamp points in the lamp point coordinate system.
Step S104: and establishing a camera coordinate system by taking the optical center of the camera as an origin.
Step S105: and establishing a conversion matrix between the lamp point coordinate system and the camera coordinate system.
Step S106: and corresponding image light points in the LED screen picture to screen light points in the LED screen, and establishing a relation equation for converting the screen light points from a light point coordinate system to a pixel coordinate system by combining the conversion matrix.
Step S107: and solving a conversion matrix according to the relation equation, and outputting the conversion matrix as an LED screen position and posture estimation result.
Specifically, step S102 specifically includes:
step S1021: and carrying out binarization processing on the LED screen picture.
Step S1022: and searching a connected domain.
Step S1023: and taking the central coordinate of the connected domain as the coordinate of the image lamp point.
For the description of the execution of each step, reference may be directly made to the embodiments of the foregoing method, which are not described herein again.
The present invention has been further described with reference to specific embodiments, but it should be understood that the detailed description should not be construed as limiting the spirit and scope of the present invention, and various modifications made to the above-described embodiments by those of ordinary skill in the art after reading this specification are within the scope of the present invention.

Claims (10)

1. A pose estimation method in an LED screen correction process is characterized by comprising the following steps:
acquiring a picture of the LED screen;
marking image light points in the picture of the LED screen, establishing an image coordinate system and a pixel coordinate system by using the picture of the LED screen, and determining the coordinates of the image light points in the image coordinate system and the pixel coordinate system respectively;
establishing a lamp point coordinate system by using an LED screen, and determining the coordinates of screen lamp points in the lamp point coordinate system;
establishing a camera coordinate system by taking the optical center of the camera as an origin;
establishing a conversion matrix between the lamp point coordinate system and the camera coordinate system;
corresponding image light points in the LED screen picture to screen light points in the LED screen, and establishing a relation equation for converting the screen light points from the light point coordinate system to the pixel coordinate system by combining the conversion matrix;
and solving the conversion matrix according to the relation equation, and outputting the conversion matrix as an LED screen position and posture estimation result.
2. The method according to claim 1, wherein the transformation matrix comprises a rotation matrix and a translation vector.
3. The pose estimation method in the LED screen correction process according to claim 2, wherein an image coordinate system in which the picture of the LED screen is located is represented as O-xy, the pixel coordinate system is represented as uv, and the light point coordinate system is represented as Ow-XwYwZwSaid camera coordinate system being denoted as OC-XcYcZc；
The relation equation is as follows:
Figure FDA0002390751920000011
wherein: zcThe distance between the camera and the LED screen,
Figure FDA0002390751920000012
is the coordinates of the image light points in the pixel coordinate system,
Figure FDA0002390751920000013
for the camera internal reference, fxRepresenting the number of pixels of the focal length f of the camera on the x-axis of the image coordinate system, fyThe number of pixels representing the focal length f of the camera on the y-axis of the image coordinate system,
Figure FDA0002390751920000021
for the transformation matrix, R is the rotation matrix, T is the translation vector,
Figure FDA0002390751920000022
the coordinates of the screen light point in the light point coordinate system.
4. The pose estimation method in the LED screen correction process according to claim 3, wherein the camera internal parameters
Figure FDA0002390751920000023
Wherein d isxRepresenting the width of each row of the pixel coordinate system in the image coordinate system, dyRepresenting the height of each line of the pixel coordinate system in the image coordinate system, f is the camera focal length.
5. The method for estimating pose in LED screen correction process according to claim 4, wherein the transformation matrix
Figure FDA0002390751920000024
The rotation matrix
Figure FDA0002390751920000025
The translation vector
Figure FDA0002390751920000026
6. The pose estimation method in the LED screen correction process of claim 5, wherein the relation equation is transformed into
Figure FDA0002390751920000027
7. The method of claim 6, wherein determining the coordinates of the image light point in the image coordinate system comprises:
carrying out binarization processing on the LED screen picture;
searching a connected domain;
and taking the central coordinate of the connected domain as the coordinate of the image lamp point.
8. A device for estimating pose in an LED screen correction process is characterized by comprising: the system comprises an LED screen picture acquisition module, an image coordinate acquisition module, a pixel coordinate acquisition module, a lamp point coordinate acquisition module, a camera coordinate acquisition module and an LED screen posture estimation module; wherein the content of the first and second substances,
the LED screen picture acquisition module is connected with the image coordinate acquisition module and is used for acquiring the picture of the LED screen;
the image coordinate acquisition module is connected with the LED screen image acquisition module, the pixel coordinate acquisition module and the LED screen position and posture estimation module and is used for marking image lamp points in the image of the LED screen, establishing an image coordinate system by using the image of the LED screen and determining the coordinates of the image lamp points in the image coordinate system;
the pixel coordinate acquisition module is connected with the image coordinate acquisition module and the LED screen position and posture estimation module and used for establishing a pixel coordinate system by using the picture of the LED screen and determining the coordinates of the image lamp points in the pixel coordinate system;
the lamp point coordinate acquisition module is connected with the LED screen position and posture estimation module and used for establishing a lamp point coordinate system by using the LED screen and determining the coordinates of the screen lamp points in the lamp point coordinate system;
the camera coordinate acquisition module is connected with the LED screen position and attitude estimation module and used for establishing a camera coordinate system by taking the optical center of the camera as an origin;
the LED screen position and posture estimation module is connected with the image coordinate acquisition module, the pixel coordinate acquisition module, the lamp point coordinate acquisition module and the camera coordinate acquisition module and used for establishing a conversion matrix between the lamp point coordinate system and the camera coordinate system, corresponding image lamp points in the LED screen picture to screen lamp points in the LED screen, establishing a relation equation for converting the screen lamp points from the lamp point coordinate system to the pixel coordinate system by combining the conversion matrix, and solving the conversion matrix according to the relation equation to serve as an LED screen position and posture estimation result and output.
9. An electronic device characterized in that the electronic device estimates the pose of an LED screen by the pose estimation method of any one of claims 1 to 7.
10. An electronic device as claimed in claim 9, wherein the electronic device is a computer.
CN202010113353.8A 2020-02-24 2020-02-24 Pose estimation method and device in LED screen correction process and electronic equipment Pending CN111311682A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010113353.8A CN111311682A (en) 2020-02-24 2020-02-24 Pose estimation method and device in LED screen correction process and electronic equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010113353.8A CN111311682A (en) 2020-02-24 2020-02-24 Pose estimation method and device in LED screen correction process and electronic equipment

Publications (1)

Publication Number Publication Date
CN111311682A true CN111311682A (en) 2020-06-19

Family

ID=71148409

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010113353.8A Pending CN111311682A (en) 2020-02-24 2020-02-24 Pose estimation method and device in LED screen correction process and electronic equipment

Country Status (1)

Country Link
CN (1) CN111311682A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112233570A (en) * 2020-12-16 2021-01-15 卡莱特（深圳）云科技有限公司 Arc screen correction method and device, computer equipment and storage medium
CN112363682A (en) * 2020-11-19 2021-02-12 北京华建纵横科技有限公司 Image display processing method, device and system for spliced display screen and computer readable storage medium
CN113627118A (en) * 2021-10-12 2021-11-09 广州中大中鸣科技有限公司 Method, device, equipment and storage medium for automatically extracting coordinates of lamp
CN113662669A (en) * 2021-08-30 2021-11-19 华南理工大学 Optical power fusion tail end clamp holder and positioning control method thereof
CN114038410A (en) * 2021-11-26 2022-02-11 卡莱特云科技股份有限公司 LED display screen automatic correction method and device based on stepping motor
CN114882023A (en) * 2022-07-07 2022-08-09 苏州小牛自动化设备有限公司 Battery string position and posture correction method, device, control equipment and system

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101673397A (en) * 2009-09-30 2010-03-17 青岛大学 Digital camera nonlinear calibration method based on LCDs
CN105931229A (en) * 2016-04-18 2016-09-07 东北大学 Wireless camera sensor network position and posture calibration method for wireless camera sensor network
CN108198220A (en) * 2018-01-19 2018-06-22 长春希达电子技术有限公司 A kind of optical camera image surfaces method for precisely marking

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101673397A (en) * 2009-09-30 2010-03-17 青岛大学 Digital camera nonlinear calibration method based on LCDs
CN105931229A (en) * 2016-04-18 2016-09-07 东北大学 Wireless camera sensor network position and posture calibration method for wireless camera sensor network
CN108198220A (en) * 2018-01-19 2018-06-22 长春希达电子技术有限公司 A kind of optical camera image surfaces method for precisely marking

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
刘瑞芳等: "《程序设计实践》", 31 March 2015 *
夏军营: "空间目标的单目视觉位姿测量方法研究", 《中国博士学位论文全文数据库信息科技辑》 *
王宏业: "LED显示单元检测系统", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *
赵连军: "基于目标特征的单目视觉位置姿态测量技术研究", 《中国博士学位论文全文数据库信息科技辑》 *
马慧彬: "《基于机器学习的乳腺图像辅助诊断算法研究》", 31 August 2016 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112363682A (en) * 2020-11-19 2021-02-12 北京华建纵横科技有限公司 Image display processing method, device and system for spliced display screen and computer readable storage medium
CN112363682B (en) * 2020-11-19 2024-01-30 北京华建纵横科技有限公司 Spliced display screen image display processing method, device and system and computer readable storage medium
CN112233570A (en) * 2020-12-16 2021-01-15 卡莱特（深圳）云科技有限公司 Arc screen correction method and device, computer equipment and storage medium
CN112233570B (en) * 2020-12-16 2021-04-02 卡莱特（深圳）云科技有限公司 Arc screen correction method and device, computer equipment and storage medium
CN113662669A (en) * 2021-08-30 2021-11-19 华南理工大学 Optical power fusion tail end clamp holder and positioning control method thereof
CN113627118A (en) * 2021-10-12 2021-11-09 广州中大中鸣科技有限公司 Method, device, equipment and storage medium for automatically extracting coordinates of lamp
CN113627118B (en) * 2021-10-12 2022-02-22 广州中大中鸣科技有限公司 Method, device, equipment and storage medium for automatically extracting coordinates of lamp
CN114038410A (en) * 2021-11-26 2022-02-11 卡莱特云科技股份有限公司 LED display screen automatic correction method and device based on stepping motor
CN114882023A (en) * 2022-07-07 2022-08-09 苏州小牛自动化设备有限公司 Battery string position and posture correction method, device, control equipment and system

Similar Documents

Publication Publication Date Title
CN111311682A (en) Pose estimation method and device in LED screen correction process and electronic equipment
CN107633536B (en) Camera calibration method and system based on two-dimensional plane template
CN110057295B (en) Monocular vision plane distance measuring method without image control
CN111243033B (en) Method for optimizing external parameters of binocular camera
CN110830781A (en) Automatic projected image correction method and system based on binocular vision
CN111667536A (en) Parameter calibration method based on zoom camera depth estimation
CN111145269B (en) Calibration method for external orientation elements of fisheye camera and single-line laser radar
CN110136068B (en) Sound membrane dome assembly system based on position calibration between bilateral telecentric lens cameras
CN112802124A (en) Calibration method and device for multiple stereo cameras, electronic equipment and storage medium
CN111340888B (en) Light field camera calibration method and system without white image
CN113870366B (en) Calibration method and calibration system of three-dimensional scanning system based on pose sensor
CN115861445B (en) Hand-eye calibration method based on three-dimensional point cloud of calibration plate
US10628968B1 (en) Systems and methods of calibrating a depth-IR image offset
CN111461963A (en) Fisheye image splicing method and device
CN113450416B (en) TCSC method applied to three-dimensional calibration of three-dimensional camera
CN105374067A (en) Three-dimensional reconstruction method based on PAL cameras and reconstruction system thereof
CN115830103A (en) Monocular color-based transparent object positioning method and device and storage medium
CN116051659A (en) Linear array camera and 2D laser scanner combined calibration method
CN114998447A (en) Multi-view vision calibration method and system
CN112665517B (en) Multi-camera large-view-field surface shape measurement calibration method
CN111462245A (en) Zoom camera attitude calibration method and system based on rectangular structure
BR112021008558A2 (en) apparatus, disparity estimation method, and computer program product
JP2005244861A (en) Imaging apparatus and imaging system parameter correction method
CN112258581A (en) On-site calibration method for multi-fish glasses head panoramic camera
CN115880369A (en) Device, system and method for jointly calibrating line structured light 3D camera and line array camera

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 518000 room 3801, block a, block 8, block C, Wanke Yuncheng phase III, Liuxin 4th Street, Xili community, Xili street, Nanshan District, Shenzhen City, Guangdong Province (37-39 floors)

Applicant after: Carlette cloud Technology Co.,Ltd.

Address before: 518000 101, integrated circuit Industrial Park, 1089 chaguang Road, Xili street, Shenzhen City, Guangdong Province

Applicant before: COLORLIGHT (SHENZHEN) CLOUD TECHNOLOGY Co.,Ltd.

RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200619