CN116580105A - Plane mirror-based light source positioning method, device, equipment and readable storage medium - Google Patents

Plane mirror-based light source positioning method, device, equipment and readable storage medium Download PDF

Info

Publication number
CN116580105A
CN116580105A CN202310457009.4A CN202310457009A CN116580105A CN 116580105 A CN116580105 A CN 116580105A CN 202310457009 A CN202310457009 A CN 202310457009A CN 116580105 A CN116580105 A CN 116580105A
Authority
CN
China
Prior art keywords
light source
plane mirror
camera
image
equation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310457009.4A
Other languages
Chinese (zh)
Inventor
迟健男
何标涛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Science and Technology Beijing USTB
Original Assignee
University of Science and Technology Beijing USTB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Science and Technology Beijing USTB filed Critical University of Science and Technology Beijing USTB
Priority to CN202310457009.4A priority Critical patent/CN116580105A/en
Publication of CN116580105A publication Critical patent/CN116580105A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B17/00—Systems with reflecting surfaces, with or without refracting elements
    • G02B17/02—Catoptric systems, e.g. image erecting and reversing system
    • G02B17/06—Catoptric systems, e.g. image erecting and reversing system using mirrors only, i.e. having only one curved mirror
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00—Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/18—Mountings, adjusting means, or light-tight connections, for optical elements for prisms; for mirrors
    • G02B7/182—Mountings, adjusting means, or light-tight connections, for optical elements for prisms; for mirrors for mirrors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/11—Complex mathematical operations for solving equations, e.g. nonlinear equations, general mathematical optimization problems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration by the use of local operators
    • G06T5/30—Erosion or dilatation, e.g. thinning
    • G06T5/70—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Abstract

The invention discloses a plane mirror-based light source positioning method, a device, equipment and a readable storage medium, wherein the method comprises the following steps: acquiring a fixed camera, a fixed light source and a movable plane mirror, wherein the plane mirror is provided with at least three non-collinear targets; determining a camera view of a camera, and adjusting a plane mirror according to the camera view so that a virtual light source image of a light source appears on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view; shooting a plane mirror by a camera to obtain a detection image, and acquiring a positioning image corresponding to the target and a positioning image corresponding to the virtual image of the light source in the detection image; and calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image. The invention utilizes the plane mirror specular reflection principle and the space circular target positioning technology to realize the positioning of the light source which is not in the field of view of the camera, and has accuracy and instantaneity.

Description

Plane mirror-based light source positioning method, device, equipment and readable storage medium
Technical Field
The invention relates to the field of computer aided design, in particular to a plane mirror-based light source positioning method, a plane mirror-based light source positioning device, plane mirror-based light source positioning equipment and a readable storage medium.
Background
The human can acquire 80% -90% of information visually, and the sight direction of the human also reflects a lot of information. A gaze tracking system is a system that estimates the current gaze direction or gaze point position of the human eye from the eye movement characteristics of the human eye. The gaze tracking technology can be applied to the fields of psychology, medical research, man-machine interaction and the like. The technical principle of gaze tracking can be divided into a two-dimensional gaze tracking method and a three-dimensional gaze tracking method, wherein the three-dimensional gaze tracking method based on the head-mounted device has higher precision and less calibration procedures, but requires accurate camera coordinate system coordinates of the light source, and the light source is not in the camera field of view. The light source positioning is a part of system calibration of a sight tracking system, and essentially belongs to the field of vision positioning.
Visual positioning methods can be classified into monocular visual positioning, binocular visual positioning, and multiview visual positioning. The monocular vision positioning method has the advantages of low hardware cost and wide application without calibration between two cameras. Monocular vision positioning is basically divided into two types, namely a positioning method based on single-frame images and a positioning method based on multi-frame images. The positioning method based on the single frame image is commonly used and is divided into positioning based on point characteristics, positioning based on linear characteristics and positioning based on curve characteristics. Monocular visual localization based on Point features, also called PnP (Perselect-n-Point) problem, solves the camera coordinate system position of n light sources by shooting n light sources known in the object coordinate system, which requires more than three points; constructing a plurality of non-parallel straight lines based on the positioning of the straight line characteristics, and solving a plurality of nonlinear equations according to the geometric relation of projection so as to position; positioning based on curve features generally requires solving complex nonlinear systems and calculation of accurate curve equations. The positioning method based on the multi-frame images is that the camera shoots the light source in the moving process, and the coordinates of the light source are calculated according to the coordinate changes of the camera coordinate system, so that the positioning method does not depend on artificial marks.
However, the existing visual positioning method has the following defects: the binocular vision positioning and multi-vision positioning hardware has high cost, a plurality of cameras are needed, the accurate calibration difficulty of a rotation matrix and a translation matrix of a coordinate system between the cameras is high, the positioning visual field is narrow, and the precision is insufficient. Monocular vision positioning based on point characteristics requires solving a plurality of points, can not solve a single point, and has a complex calculation process. Monocular vision positioning based on linear features also has the defects of complex calculation and poor robustness. The positioning method based on the curve features is high in positioning speed, but cannot directly solve the problem that the light source is not in the field of view, and needs accurate curve contour positioning, so that the robustness is poor. The positioning method based on the multi-frame images is low in positioning speed, requires complex space geometric calculation, requires repeated iterative operation and is not high in instantaneity.
Aiming at the technical problems that the prior art can not accurately position the light source position outside the field of view of a camera and the robustness is poor due to complex calculation process, no effective solution exists at present.
Disclosure of Invention
The invention aims to provide a plane mirror-based light source positioning method, a plane mirror-based light source positioning device, plane mirror-based light source positioning equipment and a plane mirror-based light source positioning readable storage medium, which can solve the technical problems that the light source position outside the field of view of a camera cannot be accurately positioned and the robustness is poor due to complex calculation process in the prior art.
One aspect of the present invention provides a plane mirror-based light source positioning method, which includes: acquiring a fixed camera, a fixed light source and a movable plane mirror, wherein the plane mirror is provided with at least three non-collinear targets; determining a camera view of a camera, and adjusting a plane mirror according to the camera view so that a virtual light source image of a light source appears on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view; shooting a plane mirror by a camera to obtain a detection image, and acquiring a positioning image corresponding to the target and a positioning image corresponding to the virtual image of the light source in the detection image; and calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image.
Optionally, calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image includes: calculating the position information of the target and the position information of the virtual light source image through the position information of all the positioning images; determining a plane equation of the plane mirror according to the position information of the target; the actual position information of the light source is calculated according to the plane equation of the plane mirror and the position information of the virtual image of the light source.
Optionally, calculating the position information of the target and the position information of the virtual light source image from the position information of all the positioning images includes: acquiring point coordinates of all positioning images in an image coordinate system and point coordinates of a camera coordinate system; acquiring a position calculation formula of a target, wherein the position calculation formula of the target comprises an initial elliptic general equation and an initial elliptic conical surface equation, the initial elliptic general equation and the initial elliptic conical surface equation comprise a plurality of unknown coefficients, and the coefficients of the initial elliptic general equation and the coefficients of the initial elliptic conical surface equation have an association relationship; substituting the point coordinates of the image coordinate system into an initial ellipse general equation to obtain coefficients of the initial ellipse general equation; obtaining the coefficient of an initial elliptic conical surface equation through the coefficient of the initial elliptic general equation and the point coordinate calculation of a camera coordinate system, and substituting the coefficient of the initial elliptic conical surface equation into the initial elliptic conical surface equation to obtain the elliptic conical surface equation; and converting the elliptic conical surface equation into a first matrix, and solving the position information of the target and the position information of the virtual light source image through the first matrix.
Optionally, determining a plane equation of the plane mirror according to the position information of the target includes: acquiring position information of a target and an initial plane equation, wherein the initial plane equation comprises a plurality of unknown coefficients; converting the initial plane equation into a second matrix; and solving the coefficient of the initial plane equation through the second matrix, and substituting the coefficient into the initial plane equation to obtain the plane equation of the plane mirror.
Optionally, calculating the actual position information of the light source according to the plane equation of the plane mirror and the position information of the virtual image of the light source includes: acquiring a plane equation of a plane mirror and an actual position of a virtual image of a light source; calculating the normal vector of the plane mirror through a plane equation of the plane mirror; substituting a normal vector of a plane equation of the plane mirror and an actual position of a virtual image of the light source into a preset light source position calculation formula to obtain actual position information of the light source, wherein the preset light source position calculation formula is as follows:
x 0 ′ information of actual position of light source, x 0 And w is a normal vector of the plane mirror, and D is a coefficient of a plane equation of the plane mirror.
Optionally, before determining the camera field of view of the camera, the method further comprises: calibrating the camera and determining an internal reference matrix and distortion parameters of the camera.
Optionally, after capturing the detected image by the camera, the method further comprises: converting an image coordinate system of the detected image into a camera coordinate system through an internal reference matrix, and performing de-distortion operation under the camera coordinate system; converting the camera coordinate system into an image coordinate system again, and interpolating the pixel points of the detection image after the de-distortion operation through the original pixel values of the detection image; performing preprocessing operation, morphological operation and binarization operation on the interpolated detection image, wherein the preprocessing operation at least comprises any one of the following steps: digitization, geometric transformation, normalization, smoothing, restoration and enhancement, morphological operations include at least any one of: corrosion, expansion, open operation, close operation, morphological gradient, top hat operation, and black hat operation.
Another aspect of the present invention provides a plane mirror based light source positioning device, comprising: the device comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring a fixed camera, a fixed light source and a movable plane mirror, and the plane mirror is provided with at least three non-collinear targets; the adjusting module is used for determining the camera view of the camera, and adjusting the plane mirror according to the camera view so that a virtual light source image of the light source appears on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view; the identification module is used for shooting the plane mirror through the camera to obtain a detection image, and acquiring a positioning image corresponding to the target and a positioning image corresponding to the virtual light source image in the detection image; and the calculating module is used for calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image.
Yet another aspect of the present invention provides a computer device comprising: the light source positioning method based on the plane mirror is characterized in that the processor executes the computer program.
Yet another aspect of the present invention provides a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the flat mirror based light source positioning method of any of the above embodiments. Further, the computer-readable storage medium may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created from the use of blockchain nodes, and the like.
The invention utilizes the plane mirror specular reflection principle and the space circular target positioning technology to realize the positioning of the light source which is not in the field of view of the camera, and has accuracy and instantaneity. The method can locate the plurality of light sources on a single frame image, does not need to determine the relative position information among the light sources, reduces the complexity of data processing, reduces the influencing factors of the light source location, and improves the robustness of the light source location.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flowchart showing an alternative method for positioning a plane mirror-based light source according to an embodiment of the present invention;
FIG. 2 is a schematic view showing an alternative structure of a light source positioning according to a first embodiment of the present invention;
FIG. 3 is a schematic diagram showing an alternative configuration of a circular target imaging according to a first embodiment of the present invention;
fig. 4 shows a block diagram of a light source positioning device based on a plane mirror according to a second embodiment of the present invention; and
fig. 5 shows a block diagram of a computer device adapted to implement a plane mirror based light source positioning method according to a third embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Example 1
The present embodiment provides a plane mirror-based light source positioning method, fig. 1 shows a flowchart of the plane mirror-based light source positioning method, and as shown in fig. 1, the plane mirror-based light source positioning method may include steps S101 to S104, where:
step S101, a fixed camera, a fixed light source and a movable plane mirror are obtained, wherein the plane mirror is provided with at least three targets which are not collinear;
in an actual visual positioning scene, the camera and the light source are both fixed devices, and the plane mirror is a movable device. Meanwhile, at least three non-collinear targets are arranged on the plane mirror, the number of the targets is set according to the actual scene requirement, for example, when the plane mirror is far away from the camera, more targets can be set; as the flat mirror is closer to the camera, a smaller number of targets may be provided. In particular, the mirror may be a front coated mirror and the target may be a retro-reflective circular target. The front coating reflecting mirror reduces errors caused by light refraction generated by the thickness of the plane mirror, and the imaging of the front coating reflecting mirror is clearer. The round target with back light reflection enables the imaging elliptical contour to be positioned more accurately, and the robustness of the space round target positioning method is improved.
Besides the back light reflection round target is arranged on the plane mirror, the back light reflection round target can be arranged on the light source at the same time, and a small hole is punched in the center of the target, so that light rays penetrate through the small hole in the center of the target, and the round target on the plane mirror and the virtual image position of the round target of the light source in the plane mirror are positioned by utilizing the space round target positioning principle. And then solving a plane equation of the plane mirror, and finally solving the actual position of the light source by utilizing the mirror symmetry principle of the plane mirror.
Step S102, determining a camera view of the camera, and adjusting the plane mirror according to the camera view so that a virtual light source image of the light source is displayed on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view;
in particular, the present invention aims to solve the problem of positioning the light source in the field of view of the camera by using the plane mirror, so that the position of the plane mirror needs to be adjusted before calculating the position of the light source, so as to accurately position the light source. Firstly, determining a camera view field of a camera, controlling the movement of a plane mirror through background equipment, enabling all the characteristics of the adjusted plane mirror to be in the camera view field, and simultaneously, enabling a virtual light source image of a light source to be displayed on the plane mirror. In the whole adjustment process, the camera is always positioned between the light source and the plane mirror, and the adjusted plane mirror is in the field of view of the camera, and accordingly, the light source is out of the field of view of the camera.
FIG. 2 shows a schematic view of a light source positioning structure, wherein the camera is positioned between the light source and the plane mirror, and the plane mirror is positioned in the direction of the camera's line of sight, i.e. in the camera's field of view, as shown in FIG. 2; the light source is in the opposite direction of the camera's line of sight, i.e., outside the camera's line of sight. In particular, the imaging position of the image plane is determined by the position of the plane mirror, i.e. the position is not fixed, and the image plane position of fig. 2 is only the imaging position of the plane mirror at this time; meanwhile, the number of the light sources is not unique, and is determined by an actual application scene. The embodiment can realize the positioning of a plurality of light sources on a single frame image, and does not need to determine the relative position information among the light sources, thereby reducing the complexity of data processing and facilitating the rapid acquisition of the positions of the light sources. The drawings are merely illustrative of specific scenarios and do not limit the scope of the present invention.
Preferably, before determining the camera field of view of a camera, the camera is calibrated and the internal reference matrix and distortion parameters of the camera are determined.
The internal parameters of the camera describe the inherent properties of the camera itself, affecting the captured image quality, including parameters such as focal length, pixel spacing, etc., typically represented by an internal reference matrix. These parameters determine the shape and size of the two-dimensional image that the camera acquires from the three-dimensional scene. The distortion parameter is a degree of distortion of an image formed by the optical system on the object with respect to the object itself, and causes only distortion of the image, and does not affect the sharpness of the image.
Specifically, the camera calibration may be a positive calibration method or a conventional marking method, which is not particularly limited herein. Through calibrating the camera, a conversion relation is established among a world coordinate system, a camera coordinate system and an image coordinate system, so that the accurate positioning of the light source is facilitated.
Step S103, shooting the plane mirror through the camera to obtain a detection image, and acquiring a positioning image corresponding to the target and a positioning image corresponding to the virtual light source image in the detection image;
after the plane mirror is adjusted, a detection image is obtained by shooting the plane mirror through a camera, wherein the detection image is an integral image formed by all the features which can be seen in the field of view of the camera, and correspondingly, the detection image also comprises a positioning image corresponding to the target and a positioning image corresponding to the virtual image of the light source, and the positioning image refers to the shape of things on the plane mirror through the camera.
Step S104, calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image.
And extracting the position information of all the positioning images on the detection image, and carrying out integration calculation to finally obtain the actual position information of the light source.
Preferably, step S104 may include steps S1041 to S1043, wherein:
step S1041, calculating position information of the target and position information of the virtual light source image according to the position information of all the positioning images;
each positioning image has a target or a virtual light source corresponding to the positioning image, so that the position information of each target and the position information of the virtual light source can be obtained by respectively analyzing and calculating all the positioning images. The position information of each target and the position information of the virtual light source image may be obtained through hough algorithm calculation, or the same type of recognition algorithm may be adopted, which is not limited herein.
Preferably, step S1041 may include steps A1 to A5, wherein:
a1, acquiring point coordinates of all positioning images in an image coordinate system and point coordinates of a camera coordinate system;
wherein the corresponding features/positions in the overall layout of the point coordinates selected by the image coordinate system and the point coordinates selected by the camera coordinate system are identical. The point coordinates of the image coordinate system and the point coordinates of the camera coordinate system may be selected arbitrarily, or key point extraction may be performed, or other extraction methods may be performed, which is not limited herein.
A2, acquiring a position calculation formula of the target, wherein the position calculation formula of the target comprises an initial elliptic general equation and an initial elliptic conical surface equation, the initial elliptic general equation and the initial elliptic conical surface equation both comprise a plurality of unknown coefficients, and the coefficients of the initial elliptic general equation and the coefficients of the initial elliptic conical surface equation have an association relationship;
step A3, substituting the point coordinates of the image coordinate system into the initial ellipse general equation to obtain the coefficients of the initial ellipse general equation;
step A4, calculating and obtaining the coefficient of the initial elliptic conical surface equation through the coefficient of the initial elliptic general equation and the point coordinates of the camera coordinate system, and substituting the coefficient of the initial elliptic conical surface equation into the initial elliptic conical surface equation to obtain an elliptic conical surface equation;
and A5, converting the elliptic conical surface equation into a first matrix, and solving the position information of the target and the position information of the virtual light source image through the first matrix.
For a detailed explanation of the above solutions of step A1 to step A5, an example is cited, which does not limit the scope of protection of the present invention.
Fig. 3 shows a schematic structural diagram of a circular target imaging, as shown in fig. 3, a circle is a common curve graph, a space circle curve is projected on an image plane through a camera to form an ellipse, and the center coordinates, rotation angles and radius of the space circle can correspond to an ellipse curve of the image plane.
On the imaging surface, the initial ellipse general equation obtained by the projection of the circular target is:
a 2 +v 2 +uv+u+ev+=0, where (u, v) is the point coordinates of the image coordinate system, a, b, c, d, e and f are coefficients of any elliptic equation;
the initial elliptic cone equation is:
A 2 +y 2 +Cxy+Dxz+Eyz+Fz 2 =0, where (x, y, z) is the point coordinates of the world coordinate system, a=af 0 2 ，B＝bf 0 2 ，C＝cf 0 2 ，D＝df 0 ，E＝ef 0 ，F＝ 0 A, B, C, D, E and F are coefficients of any elliptic cone equation, F 0 Is constant.
The first matrix form is:
wherein (lambda) 1 λ 2 λ 3 ) The eigenvalues of matrix Q are represented, P being the eigenvector of Q. The matrix P can be written as:
the three-dimensional coordinates of the center of the target are obtained by converting a coordinate system:
wherein R is the radius of the target. The algorithm generates two solutions, the plus sign is taken after the equation when t is 1, the minus sign is taken after the equation when t is 2, and finally the average value of the two solutions can be taken as the final solution.
Step S1042, determining a plane equation of the plane mirror according to the position information of the target;
because the targets are all positioned on the surface of the plane mirror, the plane equation of the plane mirror can be calculated and obtained by the position information of the targets obtained through the calculation.
Preferably, step S1042 may include steps B1 to B3, wherein:
step B1, acquiring position information of the target and an initial plane equation, wherein the initial plane equation comprises a plurality of unknown coefficients;
the position information of the target, which is the corresponding position of the target in the camera coordinate system, is the value solved in step A5.
Step B2, converting the initial plane equation into a second matrix;
and B3, solving the coefficient of the initial plane equation through the second matrix, and substituting the coefficient into the initial plane equation to obtain the plane equation of the plane mirror.
The technical solutions of the steps B1 to B3 may be implemented by following ways: the initial plane equation is first defined as: ax+by+cz+d=0 (c+.0), let The equation can be changed to: z= 0 x+a 1 y+a 2 。
Written in a second matrix form: ax=b, wherein:
so x= (a) can be solved by the normal equation set T A) -1 A T b, substituting intoThree or more mirror circular target positions (x i ，y i ，z i ) And converting the coefficients, and finally obtaining a plane equation of the plane mirror.
Step S1043, calculating the actual position information of the light source according to the plane equation of the plane mirror and the position information of the virtual image of the light source.
Preferably, step S1043 may include steps C1 to C3, wherein:
step C1, acquiring the plane equation of the plane mirror and the actual position of the virtual light source image;
step C2, calculating the normal vector of the plane mirror through the plane equation of the plane mirror;
step C3, substituting the normal vector of the plane equation of the plane mirror and the actual position of the virtual light source image into a preset light source position calculation formula to obtain the actual position information of the light source, wherein the preset light source position calculation formula is as follows:
x 0 ′ actual position information of the light source, x 0 And w is the normal vector of the plane mirror, and D is the coefficient of the plane equation of the plane mirror.
The technical solutions of the steps C1 to C3 are implemented as follows: obtaining the solved plane equation ax+by+cz+d=0 (c+.0) of the plane mirror and the virtual image position x of the light source 0 The plane normal vector w= [ A B C of the plane mirror can be made according to the plane mirror reflection principle] T Solving to obtain the actual position x of the light source 0 ′ The method comprises the following steps:
preferably, after the detected image is obtained by photographing the plane mirror with the camera, the method further includes steps D1 to D3:
step D1, converting an image coordinate system of the detection image into a camera coordinate system through the internal reference matrix, and performing de-distortion operation under the camera coordinate system;
step D2, converting the camera coordinate system into an image coordinate system again, and interpolating the pixel points of the detection image after the de-distortion operation through the original pixel values of the detection image;
step D3, preprocessing operation, morphological operation and binarization operation are carried out on the interpolated detection image, wherein the preprocessing operation at least comprises any one of the following steps: digitizing, geometric transformation, normalization, smoothing, restoration and enhancement, the morphological operations comprising at least any one of: corrosion, expansion, open operation, close operation, morphological gradient, top hat operation, and black hat operation.
The image quality of the visual sense of the shot detection image can be improved by carrying out de-distortion, image preprocessing, morphological operation and image binarization processing on the shot detection image, the accuracy of image feature recognition is ensured, and the accuracy of light source positioning is further improved. The above image processing operation can be randomly selected according to the actual application scene, and there is no limitation.
The embodiment utilizes the plane mirror specular reflection principle and the space circular target positioning technology, realizes the positioning of the light source which is not in the field of view of the camera, and has accuracy and instantaneity. The method can locate the plurality of light sources on a single frame image, does not need to determine the relative position information among the light sources, reduces the complexity of data processing, reduces the influencing factors of the light source location, and improves the robustness of the light source location.
Example two
The second embodiment of the present invention further provides a plane mirror-based light source positioning device, which corresponds to the plane mirror-based light source positioning method provided in the first embodiment, and corresponding technical features and technical effects are not described in detail in this embodiment, and reference is made to the first embodiment for relevant points. Specifically, fig. 4 shows a block diagram of the structure of the plane mirror-based light source positioning device. As shown in fig. 4, the plane mirror based light source positioning device 400 includes an acquisition module 401, an adjustment module 402, an identification module 403, and a calculation module 404, wherein:
an acquisition module 401 for acquiring a fixed camera, a fixed light source, and a movable plane mirror, wherein the plane mirror is provided with at least three non-collinear targets;
the adjusting module 402 is connected with the obtaining module 401, and is used for determining a camera view of the camera, adjusting the plane mirror according to the camera view so that a virtual light source image of the light source appears on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view;
the recognition module 403 is connected with the adjustment module 402, and is used for obtaining a detection image by shooting the plane mirror through the camera, and obtaining a positioning image corresponding to the target and a positioning image corresponding to the virtual light source image in the detection image;
the calculating module 404 is connected to the identifying module 403, and is configured to calculate the actual position information of the light source according to the position information of all the positioning images acquired on the detected image.
Optionally, the computing module includes: the first calculating unit is used for calculating the position information of the target and the position information of the virtual light source image through the position information of all the positioning images; the second calculation unit is used for determining a plane equation of the plane mirror according to the position information of the target; and a third calculation unit for calculating actual position information of the light source according to the plane equation of the plane mirror and the position information of the virtual image of the light source.
Optionally, the first computing unit is specifically configured to: acquiring point coordinates of all positioning images in an image coordinate system and point coordinates of a camera coordinate system; acquiring a position calculation formula of a target, wherein the position calculation formula of the target comprises an initial elliptic general equation and an initial elliptic conical surface equation, the initial elliptic general equation and the initial elliptic conical surface equation comprise a plurality of unknown coefficients, and the coefficients of the initial elliptic general equation and the coefficients of the initial elliptic conical surface equation have an association relationship; substituting the point coordinates of the image coordinate system into an initial ellipse general equation to obtain coefficients of the initial ellipse general equation; obtaining the coefficient of an initial elliptic conical surface equation through the coefficient of the initial elliptic general equation and the point coordinate calculation of a camera coordinate system, and substituting the coefficient of the initial elliptic conical surface equation into the initial elliptic conical surface equation to obtain the elliptic conical surface equation; and converting the elliptic conical surface equation into a first matrix, and solving the position information of the target and the position information of the virtual light source image through the first matrix.
Optionally, the second computing unit is specifically configured to: acquiring position information of a target and an initial plane equation, wherein the initial plane equation comprises a plurality of unknown coefficients; converting the initial plane equation into a second matrix; and solving the coefficient of the initial plane equation through the second matrix, and substituting the coefficient into the initial plane equation to obtain the plane equation of the plane mirror.
Optionally, the third computing unit is specifically configured to: acquiring a plane equation of a plane mirror and an actual position of a virtual image of a light source; calculating the normal vector of the plane mirror through a plane equation of the plane mirror; substituting a normal vector of a plane equation of the plane mirror and an actual position of a virtual image of the light source into a preset light source position calculation formula to obtain actual position information of the light source, wherein the preset light source position calculation formula is as follows:
x 0 ′ information of actual position of light source, x 0 And w is a normal vector of the plane mirror, and D is a coefficient of a plane equation of the plane mirror.
Optionally, the device further comprises a calibration module for calibrating the camera and determining an internal reference matrix and distortion parameters of the camera.
Optionally, the apparatus further comprises an image processing module for: converting an image coordinate system of the detected image into a camera coordinate system through an internal reference matrix, and performing de-distortion operation under the camera coordinate system; converting the camera coordinate system into an image coordinate system again, and interpolating the pixel points of the detection image after the de-distortion operation through the original pixel values of the detection image; performing preprocessing operation, morphological operation and binarization operation on the interpolated detection image, wherein the preprocessing operation at least comprises any one of the following steps: digitization, geometric transformation, normalization, smoothing, restoration and enhancement, morphological operations include at least any one of: corrosion, expansion, open operation, close operation, morphological gradient, top hat operation, and black hat operation.
Example III
Fig. 5 shows a block diagram of a computer device adapted to implement a plane mirror based light source positioning method according to a third embodiment of the present invention. In this embodiment, the computer camera device 500 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack-mounted server, a blade server, a tower server, or a rack-mounted server (including a stand-alone server or a server cluster formed by a plurality of servers) for executing programs, and so on. As shown in fig. 5, the computer device 500 of the present embodiment includes at least, but is not limited to: a memory 501, a processor 502, and a network interface 503 that may be communicatively coupled to each other via a system bus. It is noted that FIG. 5 only shows computer device 500 having components 501-503, but it is understood that not all of the illustrated components are required to be implemented, and that more or fewer components may alternatively be implemented.
In this embodiment, the memory 503 includes at least one type of computer-readable storage medium, including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 501 may be an internal storage unit of the computer device 500, such as a hard disk or memory of the computer device 500. In other embodiments, the memory 501 may also be an external storage device of the computer device 500, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card) or the like, which are provided on the computer device 500. Of course, memory 501 may also include both internal storage units of computer device 500 and external storage devices. In the present embodiment, the memory 501 is generally used to store an operating system and various types of application software installed on the computer device 500, such as program codes of a plane mirror-based light source positioning method.
The processor 502 may be a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 502 is generally used to control the overall operation of the computer device 500. Such as performing control and processing related to data interaction or communication with the computer device 500. In this embodiment, the processor 502 is configured to execute the program code for executing the steps of the plane mirror based light source positioning method stored in the memory 501.
In this embodiment, the plane mirror based light source positioning method stored in the memory 501 may also be divided into one or more program modules and executed by one or more processors (the processor 502 in this embodiment) to complete the present invention.
The network interface 503 may include a wireless network interface or a wired network interface, the network interface 503 typically being used to establish a communication link between the computer device 500 and other computer devices. For example, the network interface 503 is used to connect the computer device 500 to an external terminal through a network, establish a data transmission channel and a communication link between the computer device 500 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, abbreviated as GSM), wideband code division multiple access (Wideband Code Division Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc.
Example IV
The present embodiment also provides a computer readable storage medium including a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a Read Only Memory (ROM), an Electrically Erasable Programmable Read Only Memory (EEPROM), a Programmable Read Only Memory (PROM), a magnetic memory, a magnetic disk, an optical disk, a server, an App application store, etc., having stored thereon a computer program that when executed by a processor implements the steps of a plane mirror based light source positioning method.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the invention described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or a plurality of modules or steps in them may be fabricated into a single integrated circuit module. Thus, embodiments of the invention are not limited to any specific combination of hardware and software.
It should be noted that, the embodiment numbers of the present invention are only for description, and do not represent the advantages and disadvantages of the embodiments.
From the above description of the embodiments, it will be clear to those skilled in the art that the above-described embodiment method may be implemented by means of software plus a necessary general hardware platform, but of course may also be implemented by means of hardware, but in many cases the former is a preferred embodiment.
The foregoing description is only of the preferred embodiments of the present invention, and is not intended to limit the scope of the invention, but rather is intended to cover any equivalents of the structures or equivalent processes disclosed herein or in the alternative, which may be employed directly or indirectly in other related arts.

Claims (10)

1. A method for positioning a light source based on a planar mirror, the method comprising:
acquiring a fixed camera, a fixed light source and a movable plane mirror, wherein the plane mirror is provided with at least three non-collinear targets;
determining a camera view of the camera, and adjusting the plane mirror according to the camera view so that a virtual light source image of the light source is displayed on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view;
shooting the plane mirror through the camera to obtain a detection image, and acquiring a positioning image corresponding to the target and a positioning image corresponding to the virtual light source image in the detection image;
and calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image.
2. The method of claim 1, wherein calculating the actual position information of the light source from the position information of all the positioning images acquired on the detection image comprises:
calculating the position information of the target and the position information of the virtual light source image through the position information of all the positioning images;
determining a plane equation of the plane mirror according to the position information of the target;
and calculating the actual position information of the light source according to the plane equation of the plane mirror and the position information of the virtual image of the light source.
3. The method according to claim 2, wherein the calculating the position information of the target and the position information of the virtual light source image from the position information of the all positioning images includes:
acquiring point coordinates of all the positioning images in an image coordinate system and point coordinates of a camera coordinate system;
acquiring a position calculation formula of the target, wherein the position calculation formula of the target comprises an initial elliptic general equation and an initial elliptic conical surface equation, the initial elliptic general equation and the initial elliptic conical surface equation comprise a plurality of unknown coefficients, and the coefficients of the initial elliptic general equation and the initial elliptic conical surface equation have an association relationship;
substituting the point coordinates of the image coordinate system into the initial ellipse general equation to obtain coefficients of the initial ellipse general equation;
calculating and obtaining the coefficient of the initial elliptic conical surface equation through the coefficient of the initial elliptic general equation and the point coordinates of the camera coordinate system, and substituting the coefficient of the initial elliptic conical surface equation into the initial elliptic conical surface equation to obtain an elliptic conical surface equation;
and converting the elliptic conical surface equation into a first matrix, and solving the position information of the target and the position information of the virtual light source image through the first matrix.
4. A method according to claim 3, wherein said determining a plane equation for the plane mirror based on the positional information of the target comprises:
acquiring position information of the target and an initial plane equation, wherein the initial plane equation comprises a plurality of unknown coefficients;
converting the initial plane equation into a second matrix;
and solving the coefficient of the initial plane equation through the second matrix, and substituting the coefficient into the initial plane equation to obtain the plane equation of the plane mirror.
5. The method of claim 4, wherein the calculating the actual position information of the light source from the plane equation of the plane mirror and the position information of the virtual image of the light source comprises:
acquiring the plane equation of the plane mirror and the actual position of the virtual image of the light source;
calculating the normal vector of the plane mirror through the plane equation of the plane mirror;
substituting a normal vector of the plane equation of the plane mirror and the actual position of the virtual light source image into a preset light source position calculation formula to obtain actual position information of the light source, wherein the preset light source position calculation formula is as follows:
x 0 ′ actual position information of the light source, x 0 And w is the normal vector of the plane mirror, and D is the coefficient of the plane equation of the plane mirror.
6. The method of claim 1, wherein prior to the determining the camera field of view of the camera, the method further comprises: calibrating the camera, and determining an internal reference matrix and distortion parameters of the camera.
7. The method of any of claims 1-6, wherein after capturing the inspection image by the camera, the method further comprises:
converting an image coordinate system of the detection image into a camera coordinate system through the internal reference matrix, and performing de-distortion operation under the camera coordinate system;
converting the camera coordinate system into an image coordinate system again, and interpolating the pixel points of the detection image after the de-distortion operation through the original pixel values of the detection image;
performing preprocessing operation, morphological operation and binarization operation on the interpolated detection image, wherein the preprocessing operation at least comprises any one of the following steps: digitizing, geometric transformation, normalization, smoothing, restoration and enhancement, the morphological operations comprising at least any one of: corrosion, expansion, open operation, close operation, morphological gradient, top hat operation, and black hat operation.
8. A planar mirror based light source positioning device, the device comprising:
the device comprises an acquisition module, a control module and a control module, wherein the acquisition module is used for acquiring a fixed camera, a fixed light source and a movable plane mirror, and the plane mirror is provided with at least three non-collinear targets;
the adjusting module is used for determining a camera view of the camera, adjusting the plane mirror according to the camera view so that a light source virtual image of the light source is displayed on the plane mirror, wherein the camera is positioned between the light source and the plane mirror, and the adjusted plane mirror is positioned in the camera view;
the identification module is used for shooting the plane mirror through the camera to obtain a detection image, and acquiring a positioning image corresponding to the target and a positioning image corresponding to the virtual light source image in the detection image;
and the calculation module is used for calculating the actual position information of the light source according to the position information of all the positioning images acquired on the detection image.
9. A computer device, the computer device comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method of any of claims 1 to 7 when executing the computer program.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1 to 7.
CN202310457009.4A 2023-04-25 2023-04-25 Plane mirror-based light source positioning method, device, equipment and readable storage medium Pending CN116580105A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310457009.4A CN116580105A (en) 2023-04-25 2023-04-25 Plane mirror-based light source positioning method, device, equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310457009.4A CN116580105A (en) 2023-04-25 2023-04-25 Plane mirror-based light source positioning method, device, equipment and readable storage medium

Publications (1)

Publication Number Publication Date
CN116580105A true CN116580105A (en) 2023-08-11

Family

ID=87538778

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310457009.4A Pending CN116580105A (en) 2023-04-25 2023-04-25 Plane mirror-based light source positioning method, device, equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN116580105A (en)

Similar Documents

Publication Publication Date Title
CN111179358B (en) Calibration method, device, equipment and storage medium
CN107705333B (en) Space positioning method and device based on binocular camera
CN109416744B (en) Improved camera calibration system, object and process
CN110717942B (en) Image processing method and device, electronic equipment and computer readable storage medium
US10726580B2 (en) Method and device for calibration
KR101666959B1 (en) Image processing apparatus having a function for automatically correcting image acquired from the camera and method therefor
CN109003311B (en) Calibration method of fisheye lens
CN105451012B (en) 3-D imaging system and three-D imaging method
CN110689581A (en) Structured light module calibration method, electronic device and computer readable storage medium
JP2012088114A (en) Optical information processing device, optical information processing method, optical information processing system and optical information processing program
CN111028205B (en) Eye pupil positioning method and device based on binocular distance measurement
CA2850937A1 (en) Using videogrammetry to fabricate parts
WO2019232793A1 (en) Two-camera calibration method, electronic device and computer-readable storage medium
CN116433737A (en) Method and device for registering laser radar point cloud and image and intelligent terminal
CN112257713A (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
CN111279393A (en) Camera calibration method, device, equipment and storage medium
CN111383264B (en) Positioning method, positioning device, terminal and computer storage medium
Santana-Cedrés et al. Estimation of the lens distortion model by minimizing a line reprojection error
CN112802114A (en) Multi-vision sensor fusion device and method and electronic equipment
CN114511608A (en) Method, device, terminal, imaging system and medium for acquiring depth image
CN111160233B (en) Human face in-vivo detection method, medium and system based on three-dimensional imaging assistance
CN107527323B (en) Calibration method and device for lens distortion
CN116580105A (en) Plane mirror-based light source positioning method, device, equipment and readable storage medium
CN115661258A (en) Calibration method and device, distortion correction method and device, storage medium and terminal
CN112330726B (en) Image processing method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination