CN117647198A

CN117647198A - Multi-frequency phase shift structure light processing method and device, data processing equipment and medium

Info

Publication number: CN117647198A
Application number: CN202311675009.8A
Authority: CN
Inventors: 张宜庆; 姚浩东; 周建华
Original assignee: Optical Micro Information Technology Hefei Co ltd
Current assignee: Optical Micro Information Technology Hefei Co ltd
Priority date: 2023-12-01
Filing date: 2023-12-01
Publication date: 2024-03-05

Abstract

The embodiment of the specification provides a multi-frequency phase shift structure light processing method and device, data processing equipment and medium, wherein the multi-frequency comprises low frequency, intermediate frequency and high frequency, and the multi-frequency phase shift structure light processing method comprises the following steps: determining a first phase corresponding to a low-frequency stripe pattern of a target object to be detected according to a preset virtual depth, an internal reference of sampling equipment and a preset phase-height depth model; determining a second phase corresponding to the medium-frequency stripe pattern and a third phase corresponding to the high-frequency stripe pattern of the object to be detected, which are acquired by the sampling equipment; determining an absolute phase corresponding to the high-frequency fringe pattern of the object to be detected according to the first phase corresponding to the low-frequency fringe pattern, the second phase corresponding to the medium-frequency fringe pattern and the third phase corresponding to the high-frequency fringe pattern of the object to be detected; and determining the depth of the object to be measured according to the absolute phase and the phase height depth model corresponding to the high-frequency stripe pattern of the object to be measured. By adopting the technical scheme, the reconstruction efficiency of the three-dimensional shape can be improved.

Description

Multi-frequency phase shift structure light processing method and device, data processing equipment and medium

Technical Field

The embodiment of the specification relates to the technical field of three-dimensional measurement, in particular to a multi-frequency phase shift structure light processing method and device, data processing equipment and medium.

Background

The high-speed, high-precision and non-contact three-dimensional shape measurement has wide application in the fields of mechanical engineering, industrial detection, computer vision, virtual reality, biomedicine and the like, wherein the stripe projection contour method is widely applied to three-dimensional shape measurement by the characteristics of high precision, high efficiency and good stability.

At present, the most stable of the fringe projection contour methods is a phase shift fringe method, in the three-dimensional reconstruction of phase shift contour operation, time phase expansion is a necessary step for recovering a definite absolute phase, and the absolute phase obtained by multi-frequency time domain phase expansion has the best reliability and the best precision.

However, when the phase shift fringe method is adopted, a plurality of projection fringe images in low frequency, intermediate frequency and high frequency states need to be acquired and processed, and the reconstruction efficiency is low.

Disclosure of Invention

In view of this, embodiments of the present disclosure provide a method and apparatus for processing light with a multi-frequency phase shift structure, a data processing device, and a medium, which can improve the efficiency of reconstructing a three-dimensional shape.

First, an embodiment of the present disclosure provides a method for processing light of a multi-frequency phase shift structure, where the multi-frequency phase shift structure includes a low frequency, an intermediate frequency, and a high frequency, and the method includes:

determining a first phase corresponding to a low-frequency stripe pattern of a target object to be detected according to a preset virtual depth, an internal reference of sampling equipment and a preset phase-height depth model;

determining a second phase corresponding to the medium-frequency stripe pattern and a third phase corresponding to the high-frequency stripe pattern of the object to be detected, which are acquired by the sampling equipment;

determining an absolute phase corresponding to the high-frequency fringe pattern of the object to be detected according to the first phase corresponding to the low-frequency fringe pattern, the second phase corresponding to the medium-frequency fringe pattern and the third phase corresponding to the high-frequency fringe pattern of the object to be detected;

and determining the depth of the object to be detected according to the absolute phase corresponding to the high-frequency stripe pattern of the object to be detected and the phase-height depth model.

Optionally, the determining the first phase corresponding to the low-frequency fringe pattern of the target object to be detected according to the preset virtual depth, the internal parameter of the sampling device and the preset phase-height depth model includes:

determining second dimension information and third dimension information of the object to be detected according to the preset virtual depth and the internal reference of the sampling equipment;

And determining a first phase corresponding to the low-frequency stripe pattern of the object to be detected according to the second dimension information and the third dimension information of the object to be detected and a preset phase-to-high depth model.

Optionally, the internal parameters of the sampling device and the calibration parameters of the phase-high depth model are determined in the following manner:

projecting a low-frequency stripe, a medium-frequency stripe, a high-frequency stripe and a full-bright grating to the calibration plate respectively;

acquiring a low-frequency stripe image, a medium-frequency stripe image, a high-frequency stripe image and a white image corresponding to the calibration plate through the sampling equipment, and taking the low-frequency stripe image, the medium-frequency stripe image, the high-frequency stripe image and the white image as a group of calculation images;

changing the pose of the calibration plate for multiple times, and respectively acquiring calculated images of the calibration plate under different poses through the sampling equipment;

determining internal parameters of the sampling equipment according to white images in each group of calculated images of the calibration plate under different poses;

and determining the calibration parameters of the phase-high depth model according to each group of calculated images of the calibration plate under different poses.

Optionally, the determining the calibration parameters of the phase-high depth model according to each set of calculated images of the calibration plate under different poses includes:

Determining the absolute phase of the calibration plate according to the low-frequency stripe image, the medium-frequency stripe image and the high-frequency stripe image in each group of calculation images of the calibration plate under different poses;

determining the absolute phase of a center angular point corresponding to each group of calculated images of the calibration plate under different poses according to the white images;

and establishing an overdetermined equation according to the absolute phase of the calibration plate, the absolute phase of the center angular point corresponding to each group of calculated images of the calibration plate under different poses and the phase-height depth model, and determining the calibration parameters of the phase-height depth model.

Optionally, the determining the second phase corresponding to the intermediate frequency fringe pattern of the object to be measured, which is acquired by the sampling device, includes:

and determining a second phase corresponding to the intermediate frequency fringe pattern of the object to be detected by adopting the first three steps in the four-step phase shift method.

Optionally, the formulas corresponding to the first three steps in the four-step phase shift method include:

wherein tan is ^-1 Representing an arctangent function; i ₀ A phase shift fringe distribution representing a first medium frequency fringe pattern; i ₁ A phase shift fringe distribution representing a second medium frequency fringe pattern; i ₂ A phase shift fringe distribution representing a third medium frequency fringe pattern; phi represents the intermediate frequency phase shift fringe phase.

Optionally, the determining the absolute phase corresponding to the high-frequency fringe pattern of the object to be measured according to the first phase corresponding to the low-frequency fringe pattern, the second phase corresponding to the medium-frequency fringe pattern, and the third phase corresponding to the high-frequency fringe pattern of the object to be measured includes:

determining an absolute phase corresponding to the intermediate frequency fringe pattern of the object to be detected according to a first phase corresponding to the low frequency fringe pattern of the object to be detected and a second phase corresponding to the intermediate frequency fringe pattern of the object to be detected;

and according to the absolute phase corresponding to the intermediate frequency fringe pattern of the object to be detected and the third phase corresponding to the high frequency fringe pattern of the object to be detected, adopting time phase expansion to determine the absolute phase corresponding to the high frequency fringe pattern of the object to be detected.

Optionally, the determining the absolute phase corresponding to the intermediate frequency fringe pattern of the object to be measured according to the first phase corresponding to the low frequency fringe pattern of the object to be measured and the second phase corresponding to the intermediate frequency fringe pattern of the object to be measured includes:

according to a first phase corresponding to the low-frequency fringe pattern of the object to be detected and a second phase corresponding to the medium-frequency fringe pattern of the object to be detected, a preset first calculation formula is adopted to obtain a step parameter corresponding to the medium-frequency fringe pattern of the object to be detected;

And determining the absolute phase corresponding to the intermediate frequency fringe pattern of the object to be detected by adopting a preset second calculation formula according to the step parameter corresponding to the intermediate frequency fringe pattern of the object to be detected and the second phase corresponding to the intermediate frequency fringe pattern of the object to be detected.

Optionally, the multi-frequency phase shift structure light processing method further includes:

and adjusting the value of the preset virtual depth until the obtained step parameter is equal to the set step parameter.

Accordingly, embodiments of the present disclosure further provide a multi-frequency phase shift structure light processing device, including:

the projection equipment is suitable for projecting medium-frequency stripes and high-frequency stripes to the object to be detected;

the sampling equipment is suitable for acquiring a medium-frequency stripe pattern and a high-frequency stripe pattern of the object to be detected;

the processing equipment is suitable for determining a first phase corresponding to a low-frequency stripe pattern of the object to be detected according to a preset virtual depth, an internal reference of the sampling equipment and a preset phase-to-phase depth model, determining a second phase corresponding to an intermediate-frequency stripe pattern of the object to be detected and a third phase corresponding to a high-frequency stripe pattern of the object to be detected, determining an absolute phase corresponding to a high-frequency stripe pattern of the object to be detected according to the first phase corresponding to the low-frequency stripe pattern of the object to be detected, the second phase corresponding to the intermediate-frequency stripe pattern and the third phase corresponding to the high-frequency stripe pattern, and determining the depth of the object to be detected according to the absolute phase corresponding to the high-frequency stripe pattern of the object to be detected and the phase-to-phase depth model.

The embodiments of the present disclosure also provide a data processing apparatus, which includes a memory and a processor, wherein the memory is adapted to store one or more computer instructions, and the processor executes the multi-frequency phase shift structured light processing method according to any one of the foregoing embodiments when executing the computer instructions.

The embodiments of the present specification also provide a computer readable storage medium, wherein computer instructions are stored, which when executed, perform the multi-frequency phase shift structured light processing method according to any of the previous embodiments.

According to the multi-frequency phase shift structure light processing method provided by the embodiment of the specification, according to the preset virtual depth, the internal reference of the sampling equipment and the preset phase-height depth model, the first phase corresponding to the low-frequency fringe pattern of the object to be detected can be determined, and further according to the first phase corresponding to the low-frequency fringe pattern of the object to be detected, the second phase corresponding to the medium-frequency fringe pattern and the third phase corresponding to the high-frequency fringe pattern, the absolute phase corresponding to the high-frequency fringe pattern of the object to be detected can be determined, and the depth of the object to be detected can be determined through the absolute phase corresponding to the high-frequency fringe pattern of the object to be detected and the phase-height depth model, so that the three-dimensional shape reconstruction process is completed. In the process, the low-frequency fringe pattern of the object to be detected is not required to be acquired, and compared with the existing method for acquiring a plurality of projection fringe images in low-frequency, medium-frequency and high-frequency states, the method can reduce the time required for acquiring and processing the low-frequency fringe pattern and improve the reconstruction efficiency of the three-dimensional shape.

Further, by changing the pose of the calibration plate for multiple times, the computing images of the calibration plate under different poses can be obtained through the sampling device, so that multiple groups of computing images are obtained, any group of computing images can comprise a low-frequency stripe image, a medium-frequency stripe image, a high-frequency stripe image and a white image corresponding to the calibration plate, namely, the influence of the internal parameters of the sampling device and the calibration parameters of the relatively high depth model on the computing images can be reflected more accurately and comprehensively through obtaining the images for different parameter characterization, and accordingly, the calibration parameters of the internal parameters of the sampling device and the calibration parameters of the relatively high depth model can be accurately determined according to the obtained computing images, and the accuracy is improved.

Further, by adopting the first three steps in the four-step phase shift method, the second phase corresponding to the intermediate frequency fringe pattern of the object to be detected is determined, and compared with the prior 4 intermediate frequency fringe patterns, only 3 intermediate frequency fringe patterns are needed now, so that the processing time of the intermediate frequency fringe patterns can be reduced, and the reconstruction efficiency is further improved.

Further, by adopting time phase expansion of the absolute phase corresponding to the medium-frequency fringe pattern and the third phase corresponding to the high-frequency fringe pattern of the object to be detected, ambiguity among phases extracted by a plurality of fringes can be eliminated, clear absolute phases can be recovered, the precision of the obtained high-frequency absolute phases can be improved, and further, the depth precision can be improved.

Further, considering that the first phase corresponding to the low-frequency fringe pattern in the embodiment of the present disclosure is obtained by inversion, since the obtaining of the fringe pattern is a dynamic process, there may be a slight difference between the first phase obtained by inversion and the actual first phase, and by adjusting the value of the preset virtual depth until the obtained step parameter is equal to the set step parameter, the influence of the difference may be eliminated, and an unambiguous absolute phase in high frequency may be obtained.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings can be obtained according to the provided drawings without inventive effort for a person skilled in the art.

FIG. 1 shows a schematic diagram of a multi-frequency phase shift processing method;

FIG. 2 is a flow chart of a method of optical processing of a multi-frequency phase shift structure in an embodiment of the present disclosure;

FIG. 3 shows a flow chart for determining calibration parameters of the high depth model and internal parameters of the sampling device in one embodiment of the present disclosure;

Fig. 4 is a flowchart of a method for determining an absolute phase corresponding to a high-frequency fringe pattern of an object to be measured according to an embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a multi-frequency phase shift processing method according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram showing the comparison between the precision ball depth data obtained by the multi-frequency phase shift structure optical processing method and the depth data difference of the original scheme in the embodiment of the present disclosure;

FIG. 7 is a schematic diagram of a multi-frequency phase shift structure light processing device according to an embodiment of the present disclosure;

fig. 8 shows a block diagram of a data processing apparatus in an embodiment of the present specification.

Detailed Description

As described in the background art, when the phase shift fringe method is adopted, a plurality of projection fringe images in low frequency, intermediate frequency and high frequency states need to be acquired and processed, and the reconstruction efficiency is low.

In order to better understand the problems of the prior art, a brief description of the prior art is provided below in connection with a multi-frequency phase shift processing scheme.

Referring to a schematic diagram of a multi-frequency phase shift processing method shown in fig. 1, as shown in fig. 1, a four-step phase shift method is used to process a low-frequency (e.g. 1 Hz) fringe pattern, a medium-frequency (e.g. 6 Hz) fringe pattern, and a high-frequency (e.g. 54 Hz) fringe pattern, respectively, so as to complete a reconstruction process of a 3D shape of an object to be measured.

Specifically, 4 low-frequency fringe patterns, 4 medium-frequency fringe patterns and 4 high-frequency fringe patterns are respectively obtained, four-step phase shift modulation is respectively carried out on the 12 fringe patterns, a low-frequency 4-step modulation chart, a medium-frequency 4-step modulation chart and a high-frequency 4-step modulation chart shown in fig. 1 are respectively obtained, and the 4-step modulation charts are respectively unfolded to obtain a low-frequency phase corresponding to the low-frequency fringe patterns, a medium-frequency phase corresponding to the medium-frequency fringe patterns and a high-frequency phase corresponding to the high-frequency fringe patterns.

Obtaining an intermediate frequency absolute phase according to the low frequency phase and the intermediate frequency phase, determining a final high frequency absolute phase according to the intermediate frequency absolute phase and the high frequency phase, and calculating the depth of a high frequency absolute phase image according to a phase height model or an inverse camera model to finish the reconstruction of the three-dimensional shape.

However, the above scheme requires too many projection fringe patterns (for example, 12 projection fringe patterns in the above example), resulting in a large calculation amount, an increase in time required for the processing procedure, and low reconstruction efficiency.

Through a great deal of exercise and practice, the inventor finds that when three-dimensional reconstruction is carried out by adopting multi-frequency phase shift structured light, high-frequency stripes projected to an object to be detected are determined to play a role in determining the depth precision, and low-frequency stripes and medium-frequency stripes only play a role in assisting in acquiring phases, so that the number of used stripes can be reduced under the condition that the depth precision is not influenced.

Based on this, the embodiment of the present disclosure provides a multi-frequency phase shift structure light processing method, which obtains, through inversion, a first phase corresponding to a low-frequency fringe pattern of a target object to be measured, so that the low-frequency fringe pattern of the target object to be measured does not need to be obtained, and thus the time required for obtaining and processing the low-frequency fringe pattern can be reduced, and the three-dimensional shape reconstruction efficiency is improved.

In addition, due to the reduction of the number of the stripe patterns, errors caused by movement in a real-time dynamic scanning process can be reduced, and the reconstruction accuracy of the three-dimensional shape can be improved.

In order to better understand the inventive concepts, principles and advantages of the embodiments of the present disclosure, the following detailed description of the optical processing scheme of the multi-frequency phase shift structure in the embodiments of the present disclosure is provided.

Referring to fig. 2, as shown in a flowchart of a method for processing light with a multi-frequency phase shift structure in the embodiment of the present disclosure, in some embodiments of the present disclosure, the multi-frequency may include a low frequency, a medium frequency, and a high frequency, and accordingly, the processing may be specifically performed according to the following steps:

s11, determining a first phase corresponding to a low-frequency fringe pattern of the object to be detected according to a preset virtual depth, an internal reference of the sampling device and a preset phase-high depth model.

The internal parameters of the sampling device may refer to an internal matrix of the sampling device, which describes inherent properties of the sampling device itself, including parameters such as focal length, pixel spacing, and the like. These parameters determine the parameters (e.g., shape and size) of the two-dimensional image acquired by the sampling device from the three-dimensional scene and are important inputs for image processing and geometric transformations.

The phase-height depth model is an improved phase-height conversion mapping model, and a one-to-one mapping model from phase to height is built in a sampling device coordinate system by building a virtual sampling device coordinate system (e.g., a camera coordinate system) and analyzing the conversion relationship of fringe information between the projection device coordinate system and the sampling device coordinate system.

The preset virtual depth can be determined according to the nearest distance which can be modeled by a platform constructed by the sampling equipment and the projection equipment, and can represent the information of the first dimension of the object to be measured.

As an optional example, when determining the preset virtual depth, the second dimension information and the third dimension information of the object to be measured may be determined according to the preset virtual depth and the internal reference of the sampling device, and then the second dimension information and the third dimension information of the object to be measured may be substituted into the preset phase-high depth model, and inversion operation may be performed according to the second dimension information and the third dimension information of the object to be measured and the preset phase-high depth model, so that the first phase corresponding to the low-frequency stripe pattern of the object to be measured may be determined, and the step of acquiring and processing the low-frequency stripe pattern may be replaced.

As an example, if the virtual depth (i.e. the information of the first dimension of the object to be measured) is Z, the second dimension information X and the third dimension information Y of the object to be measured can be obtained through an internal reference matrix (see the description below) of the sampling device, and then the first phase corresponding to the low-frequency stripe pattern can be obtained by inversion after determining the calibration parameters in the high-phase depth model.

In some alternative examples, the sampling device may include a camera, a video camera, a CCD, or the like, with photographing capabilities, and the embodiments of the present disclosure do not limit the type of sampling device.

S12, determining a second phase corresponding to the medium-frequency stripe pattern and a third phase corresponding to the high-frequency stripe pattern of the object to be detected, which are acquired by the sampling equipment.

Specifically, the sampling device may capture the intermediate frequency fringe pattern and the high frequency fringe pattern projected onto the object to be measured in sequence, and by performing decoding on the intermediate frequency fringe pattern and the high frequency fringe pattern, phase information can be extracted from the deformed fringe pattern (intermediate frequency to high frequency) to obtain a second phase corresponding to the intermediate frequency fringe pattern and a third phase corresponding to the high frequency fringe pattern.

In some embodiments, decoding the mid-band and high-band fringe patterns may be performed in a variety of ways. For example, a four-step phase shift method may be employed to perform decoding operations on the middle and high frequency fringe patterns.

S13, determining the absolute phase corresponding to the high-frequency stripe pattern of the object to be detected according to the first phase corresponding to the low-frequency stripe pattern, the second phase corresponding to the medium-frequency stripe pattern and the third phase corresponding to the high-frequency stripe pattern of the object to be detected.

Specifically, the first phase corresponding to the low-frequency fringe pattern, the second phase corresponding to the medium-frequency fringe pattern, and the third phase corresponding to the high-frequency fringe pattern of the object to be measured may represent an actual profile of the object to be measured in a two-dimensional space, and based on the three phases, an absolute phase corresponding to the high-frequency fringe pattern of the object to be measured may be determined.

S14, determining the depth of the object to be detected according to the absolute phase corresponding to the high-frequency stripe pattern of the object to be detected and the phase-height depth model.

Specifically, according to the absolute phase and the phase height depth model corresponding to the high-frequency stripe pattern, a preset calculation mode is adopted to obtain the depth of the object to be detected.

It can be understood that in the embodiment of the present disclosure, a specific calculation manner for obtaining the depth of the object to be measured is not limited, as long as the depth of the object to be measured can be obtained according to the absolute phase and the phase-height depth model corresponding to the high-frequency fringe pattern.

By adopting the multi-frequency phase shift structure light processing method in the embodiment of the specification, the first phase corresponding to the low-frequency fringe pattern of the object to be detected is obtained through inversion, so that the low-frequency fringe pattern of the object to be detected does not need to be obtained, the time required for obtaining and processing the low-frequency fringe pattern can be reduced, and the reconstruction efficiency of the three-dimensional shape can be improved.

In order to better understand and implement the embodiments of the present disclosure, some specific examples are given below for specific implementations of the optical processing method of the multi-frequency phase shift structure in the embodiments of the present disclosure.

In order to improve the measurement accuracy of the fringe projection three-dimensional measurement system, internal parameters of the sampling device and calibration parameters of a preset phase-height depth model can be predetermined.

As an example, referring to a flowchart for determining the internal parameters of the sampling device and the calibration parameters of the high-phase depth model in the embodiment of the present disclosure shown in fig. 3, the following steps may be specifically performed as shown in fig. 3:

s21, respectively projecting a low-frequency stripe, a medium-frequency stripe, a high-frequency stripe and a full-brightness grating to the calibration plate.

In some embodiments, the calibration plate may be projected with low frequency fringes, medium frequency fringes, and high frequency fringes by using a grating projection method.

The low frequency, medium frequency and high frequency fringes in the embodiments of the present disclosure are merely relative values, which are used to illustrate that fringes with different frequency values can be projected onto a calibration plate or an object to be measured, so as to obtain a deformed fringe pattern.

In some embodiments, the calibration plate may be a tooling piece with known shape and specification parameters. For example, the calibration plate may be selected from regularly shaped objects, such as square, round or diamond shaped target plates.

S22, acquiring a low-frequency stripe image, a medium-frequency stripe image, a high-frequency stripe image and a white image corresponding to the calibration plate through the sampling equipment, and taking the low-frequency stripe image, the medium-frequency stripe image, the high-frequency stripe image and the white image as a group of calculation images.

In some embodiments, a white image may refer to an image acquired when a Quan Liang grating is projected onto a calibration plate by a projection device (e.g., projector).

S23, changing the pose of the calibration plate for multiple times, and respectively acquiring the calculated images of the calibration plate under different poses through the sampling equipment.

Specifically, by changing the pose (e.g., at least one of the position or the pose) of the calibration plate, different low-frequency streak images, medium-frequency streak images, high-frequency streak images, and white images can be acquired, so that a plurality of sets of calculation images can be obtained.

S24, determining internal parameters of the sampling equipment according to white images in each group of calculated images of the calibration plate under different poses.

Specifically, in step S23, a plurality of sets of calculated images can be acquired, and a plurality of white images can be obtained, and since the plurality of white images are acquired with the calibration plate positioned in different positions, the internal reference of the sampling device can be determined according to the plurality of white images.

The method specifically comprises the following steps: the white images of the calibration plate under different poses are obtained, the coordinates of the center angular points of the white images under each pose are extracted, so that a coordinate system can be established according to the number of the circle centers in the row and column directions of the obtained circular calibration and the circle center distance between the two circle centers (for example, the circle center distance can be determined when the calibration plate is designed), the coordinate system is matched with the extracted coordinates of the center angular points, and the re-projection errors are minimized through iteration, so that the internal parameters (for example, distortion coefficients, focal lengths, principal point coordinates and the like) of the camera are obtained.

S25, determining the calibration parameters of the high-depth model according to each group of calculated images of the calibration plate under different poses.

Specifically, in step S23, a plurality of sets of calculated images can be obtained, and thus a plurality of low-frequency fringe images, a plurality of intermediate-frequency fringe images, a plurality of high-frequency fringe images, and a plurality of white images can be obtained.

Through changing the pose of the calibration plate many times, the computing images used for representing that the calibration plate is located under different poses can be obtained through the sampling equipment respectively, a plurality of groups of computing images are obtained, and any group of computing images can comprise a low-frequency stripe image, a medium-frequency stripe image, a high-frequency stripe image and a white image corresponding to the calibration plate, namely, the influence of the internal parameters of the sampling equipment and the calibration parameters of the phase-height depth model on the computing images can be reflected more accurately and comprehensively through obtaining the images used for representing different parameters, so that the calibration parameters of the internal parameters of the sampling equipment and the calibration parameters of the phase-height depth model can be determined accurately according to the obtained computing images, and the precision is improved.

It should be noted that, in the above embodiment, some steps do not have a necessary sequence, and may be executed synchronously or sequentially on the premise that contradiction cannot be generated, and the sequence may be changed. For example, in actually performing the steps of the optical processing method for a multi-frequency phase shift structure provided in the present specification, after the step S23 can be performed, the steps S24 and S25 may be performed synchronously; alternatively, step S25 is performed first and then step S24 is performed, and the order of steps is not particularly limited in the embodiment of the present disclosure.

In some examples, calibration parameters for the phase-high depth model may be obtained as follows:

1) And determining the absolute phase of the calibration plate according to the low-frequency stripe image, the medium-frequency stripe image and the high-frequency stripe image in each group of calculated images of the calibration plate under different poses.

The method specifically comprises the following steps: and determining the absolute phase of the intermediate frequency fringe image according to the low frequency fringe image and the intermediate frequency fringe image in each group of calculation images, and then determining the absolute phase of the calibration plate under the corresponding pose according to the absolute phase of the intermediate frequency fringe image and the high frequency fringe image.

2) And determining the absolute phase of the center angular point corresponding to each group of calculated images of the calibration plate under different poses according to the white images.

Specifically, when the pose of the calibration plate changes, the phase of the corresponding center point changes along with the change, and for each group of calculation images, the pose is fixed, and according to the low-frequency fringe image, the medium-frequency fringe image and the high-frequency fringe image in each group of calculation images, the absolute phase under the full resolution can be calculated, and as can be known from the above, the white image can extract the center point coordinate of the center point, so that the absolute phase of the center point coordinate under the full resolution can be determined through the center point coordinate of the center point.

3) And establishing an overdetermined equation according to the absolute phase of the calibration plate, the absolute phase of the center angular point corresponding to each group of calculated images of the calibration plate under different poses and the phase-height depth model, and determining the calibration parameters of the phase-height depth model.

Specifically, the absolute phase of the calibration plate and the absolute phase of the center angular point corresponding to each group of calculated images of the calibration plate under different poses can be substituted into the phase-height depth model, so that an overdetermined equation corresponding to each group of calculated images can be established, and the calibration parameters of the phase-height depth model can be determined by solving the overdetermined equation.

To facilitate an understanding of the above-described process for determining calibration parameters for the reference and phase-high depth models of the sampling device, a detailed description is provided below by way of example.

The circular calibration plate is projected with 4 low frequency fringes, 4 medium frequency fringes, 4 high frequency fringes and one Bai Tu in sequence, so that the camera can capture 13 images and serve as a first set of calculated images. Any group of calculated images can be represented by P0 for the image corresponding to the white image, P1 to P4 for the image corresponding to the low-frequency stripe, P5 to P8 for the image corresponding to the intermediate-frequency stripe, and P9 to P12 for the image corresponding to the high-frequency stripe.

Changing the pose of the circular calibration plate, and acquiring 13 images again to be used as a second group of calculated images; and the pose of the circular calibration plate is transformed again, so that a plurality of groups of calculated images can be obtained.

Adopting an image P0 corresponding to the white image in each group of calculated images to determine the internal parameters of the sampling equipment; the images P0 and P1 to P12 in each group are used for calculating absolute phases, and the absolute phases of the center angular points of the calibration plates of each posture in the images are calculated.

For example, according to the coordinate value of the image P0, the camera internal parameters can be determined using the following formula:

where Ac represents an internal reference of the sampling apparatus, m and n represent pixel coordinates of images (e.g., a low-frequency fringe image, a medium-frequency fringe image, a high-frequency fringe image, and a white image), ρ represents a magnification, X _C 、Y _C And Z _C Representing three-dimensional parameters (e.g., length, width, and height) of one pixel.

For example, the round calibration plate has 6*7 =42 rounded corner points, and then the corresponding 10 gestures of the round calibration plate are 420 rounded corner points, and then the round calibration plate is substituted into the phase-height depth model, so that 420 overdetermined equations can be obtained, and further calibration parameters of the phase-height depth model can be obtained according to the 420 overdetermined equations.

In some examples, the phase-high depth model may be:

wherein phi represents absolute phase, a ₁ To a ₈ Calibration parameters representing a phase-high depth modelA number.

Thus, by adopting the formulas (1) and (2), the internal reference Ac of the sampling equipment and the calibration parameter a of the phase-height depth model can be determined ₁ To a ₈ 。

In the reconstruction process of the three-dimensional image, the inventor further finds that the reconstruction efficiency can be further improved by reducing the number of intermediate frequency fringe patterns used in the reconstruction process without reducing the depth accuracy.

In some examples, the first three steps of the four-step phase shift method may be used to determine the second phase corresponding to the intermediate frequency fringe pattern of the object to be measured, so that compared with the case where 4 intermediate frequency fringe patterns are required, only 3 intermediate frequency fringe patterns are required, thereby reducing the processing time of the intermediate frequency fringe patterns and improving the reconstruction efficiency.

In some examples, a second phase corresponding to the intermediate frequency fringe pattern of the object to be measured may be determined using a formula corresponding to the first three steps of the four-step phase shift method.

Wherein, the corresponding formulas of the first three steps in the four-step phase shift method can comprise:

In some examples, the corresponding phase shift fringe distribution may be obtained using the following formula:

I _n (x，y)＝A(x，y)+B(x，y)cos(φ+nΠ/2) (4)

wherein I is _n (x, y) represents a phase shift fringe distribution; a (x, y) represents the intensity component of the phase-shifted fringe distribution; b (x, y) represents the amplitude component of the phase shift fringe distribution; n represents the code of the stripe pattern, for example,n may be 0,1,2,3.

It should be noted that, the process of obtaining the third phase corresponding to the high-frequency fringe pattern may refer to the calculation method in the existing scheme, or the formulas (3) and (4) may be adopted, which is different in that when the third phase corresponding to the high-frequency fringe pattern is calculated, n may be equal to 3.

By adopting the scheme in the example, the first phase corresponding to the low-frequency stripe pattern of the object to be measured can be inversely determined according to the preset virtual depth, the internal reference of the sampling equipment and the preset phase-height depth model, the second phase corresponding to the medium-frequency stripe pattern and the third phase corresponding to the high-frequency stripe pattern of the object to be measured can be determined according to the four-step phase shift method, and then the absolute phase corresponding to the high-frequency stripe pattern of the object to be measured can be determined according to the first phase corresponding to the low-frequency stripe pattern, the second phase corresponding to the frequency stripe pattern and the third phase corresponding to the high-frequency stripe pattern of the object to be measured.

As a specific example, referring to a flowchart of a method for determining an absolute phase corresponding to a high frequency fringe pattern of an object to be measured in the embodiment of the present disclosure shown in fig. 4, in some embodiments, as shown in fig. 4, the method may specifically be performed as follows:

s31, determining the absolute phase corresponding to the intermediate frequency fringe pattern of the object to be detected according to the first phase corresponding to the low frequency fringe pattern of the object to be detected and the second phase corresponding to the intermediate frequency fringe pattern of the object to be detected.

Specifically, as described above, the low-frequency fringes and the intermediate-frequency fringes only serve as auxiliary phases, so that when the first phase corresponding to the low-frequency fringe pattern and the second phase corresponding to the intermediate-frequency fringe pattern are acquired, the two phases can be processed to acquire the absolute phase corresponding to the intermediate-frequency fringe pattern of the object to be measured.

In some examples, the absolute phase corresponding to the intermediate frequency fringe pattern of the object to be measured may be determined as follows:

a1 According to the first phase corresponding to the low-frequency fringe pattern of the object to be detected and the second phase corresponding to the medium-frequency fringe pattern of the object to be detected, a preset first calculation formula is adopted to obtain the step parameter corresponding to the medium-frequency fringe pattern of the object to be detected;

As an example, the first calculation formula may be:

wherein T may represent different frequency values; k (K) _{In (a)} (x, y) a step parameter corresponding to the intermediate frequency fringe pattern; round may represent a rounding function.

In some examples, T may represent low, medium, and high frequencies.

A2 According to the step parameter corresponding to the intermediate frequency fringe pattern of the object to be measured and the second phase corresponding to the intermediate frequency fringe pattern of the object to be measured, determining the absolute phase corresponding to the intermediate frequency fringe pattern of the object to be measured by adopting a preset second calculation formula.

As an example, the second calculation formula may be:

Φ _{in (a)} (x,y)＝φ _{In (a)} (x,y)+2ΠK _{In (a)} (x,y) (7)

Therefore, by adopting the steps A1) and A2), the first phase corresponding to the low-frequency fringe pattern and the second phase corresponding to the medium-frequency fringe pattern can be unfolded by adopting the preset first calculation formula and the second calculation formula, the precision of the absolute phase corresponding to the obtained medium-frequency fringe pattern of the object to be detected is improved, and the accuracy of the depth of the object to be detected is further improved.

In some examples, considering that the first phase corresponding to the low-frequency fringe pattern in the embodiments of the present disclosure is obtained by inversion, since the obtaining of the fringe pattern is a dynamic process, there may be a slight difference between the first phase obtained by inversion and the actual first phase, so as to reduce the influence caused by the difference, the step parameter obtained by inversion may be made to be equal to the set step parameter.

As an optional example, the method for processing the multi-frequency phase shift structure light provided in the embodiments of the present disclosure may further include: and adjusting the value of the preset virtual depth until the obtained step parameter is equal to the set step parameter.

Specifically, by adjusting the preset virtual depth value, under the condition that the internal parameter of the sampling device and the preset phase-height depth model remain unchanged, the first phase corresponding to the low-frequency fringe pattern can be changed, and then the step parameter in the step A1) can be changed until the step parameter is equal to the set step parameter, so that the influence of difference can be eliminated, and the unambiguous absolute phase in high frequency can be obtained.

The ambiguity means that each fringe pattern phase value can find the phase height corresponding to the pixel point on the image.

In some examples, the set step parameters may be obtained by projecting a low-frequency fringe and a medium-frequency fringe onto the object to be measured and using a first calculation formula and a second calculation formula.

S32, according to the absolute phase corresponding to the medium frequency fringe pattern of the object to be detected and the third phase corresponding to the high frequency fringe pattern of the object to be detected, determining the absolute phase corresponding to the high frequency fringe pattern of the object to be detected by adopting time phase expansion.

Specifically, the sampling device captures the deformed fringe pattern in turn, and when performing the decoding operation, phase information (e.g., the first phase, the second phase, and the third phase in the previous example) can be extracted from the deformed fringe pattern, and the range of the single fringe phase is [ -pi, pi ], and the extracted phases of the plurality of fringes are ambiguous, so that it is necessary to recover a definite absolute phase through time phase unwrapping.

In some embodiments, the absolute phase calculation may be obtained by performing a corresponding adjustment calculation according to the descriptions of the foregoing formulas (1) to (7), and will not be described herein.

And when determining the absolute phase corresponding to the high-frequency stripe pattern of the object to be detected, calculating the depth according to the phase-high depth model or the inverse camera model, thereby completing the reconstruction process of the three-dimensional shape.

For a better understanding and implementation of the method of optical processing of a multi-frequency phase shift structure provided by the embodiments of the present disclosure, a brief description of the method is provided below in connection with a multi-frequency phase shift processing scheme.

Referring to a schematic diagram of a multi-frequency phase shift processing method shown in fig. 5, as shown in fig. 5, a four-step phase shift method is used to process a middle frequency (e.g. 6 Hz) stripe pattern and a high frequency (e.g. 54 Hz) stripe pattern, respectively, so as to complete a reconstruction process of a 3D shape of a target object to be measured.

Specifically, 4 intermediate frequency stripes and 4 high frequency stripes are projected to a target object to be measured respectively to obtain 4 intermediate frequency stripe patterns and 4 high frequency stripe patterns, phase shift modulation is performed on the 8 stripe patterns respectively, for example, the first 3 steps of a four-step phase shift method are adopted to obtain an intermediate frequency first 3 steps of modulation diagram shown in fig. 5, a four-step phase shift method is adopted to obtain a high frequency 4 steps of modulation diagram, the intermediate frequency first 3 steps of modulation diagram and the high frequency 4 steps of modulation diagram are respectively expanded to obtain a second phase (i.e. an intermediate frequency phase) corresponding to the intermediate frequency stripe patterns and a third phase (i.e. a high frequency phase) corresponding to the high frequency stripe patterns, formulas (1), (2) and virtual depth (for example, virtual depth z can be 1100 mm) are adopted, and the internal reference of sampling equipment and a preset phase-high depth model are inverted to obtain a first phase (i.e. a low frequency phase) corresponding to the low frequency stripe patterns.

By adopting a preset first calculation formula and a preset second calculation formula (see the above examples specifically), the low-frequency phase and the intermediate-frequency phase can be processed to obtain an intermediate-frequency absolute phase, then the final high-frequency absolute phase is determined by adopting time phase expansion according to the intermediate-frequency absolute phase and the high-frequency phase, and then the depth of the high-frequency absolute phase image is calculated according to a phase-height model or an inverse camera model, so that the reconstruction of the three-dimensional shape is completed.

For the multi-frequency phase shift structure light processing method in the embodiment of the specification, on the premise of not affecting absolute phase, low-frequency stripes are removed, only medium-frequency stripes and high-frequency stripes are reserved, and when the medium-frequency stripes are processed, only 3 steps in a four-step phase shift method are adopted, and the 3 steps are not the traditional three-step phase shift method, so that the effect of the traditional 12 stripe patterns can be realized by totally needing 7 stripe patterns.

For example, referring to fig. 6, a schematic diagram of comparing the difference between the depth data of the precision ball and the depth data of the original scheme is obtained by using the optical processing method of the multi-frequency phase shift structure in the embodiment of the present specification, and as can be seen from fig. 6, the difference 0 between the depth data of the precision ball obtained by using the conventional scheme and the scheme provided by the embodiment of the present specification, it can be considered that the lossless depth can be obtained by using the dual-frequency (intermediate frequency and high frequency) scheme in the embodiment of the present specification.

As an example, the standard diameter of the precision ball may be 80.8015mm, and the average precision of 10 times measured by the multi-frequency phase shift structure light processing method provided by the embodiment of the present disclosure reaches 0.0286mm, that is, the standard diameter of the precision ball may be considered to be measured accurately.

Moreover, due to the reduction of the number of the stripe patterns, on one hand, the efficiency of the scanning system can be effectively improved, and on the other hand, the error influence caused by movement in the real-time dynamic scanning process can be reduced.

It should be noted that the frequency values and the virtual depth values illustrated in fig. 5 are only exemplary, and are used to illustrate that the three-dimensional shape of the object to be measured can be reconstructed according to the stripes and the virtual depth values having different frequencies, which is not to be construed as limiting the present invention. In some embodiments, the frequency value and the virtual depth value may also be other values.

It will be appreciated that while the embodiments provided herein have been described above with respect to various embodiments, the various alternatives identified by the various embodiments may be combined with each other and cross-referenced without conflict, thereby extending what is believed to be the embodiments disclosed and disclosed herein.

The present disclosure further provides a multi-frequency phase shift structure light processing device corresponding to the multi-frequency phase shift structure light processing method, and the detailed description will be made with reference to the accompanying drawings by specific embodiments.

It should be noted that the optical processing device with the multi-frequency phase shift structure described below may be considered as a functional module required to implement the optical processing method with the multi-frequency phase shift structure provided in the present specification; the contents of the multi-frequency phase shift structure light processing device described below may be referred to in correspondence with the contents of the multi-frequency phase shift structure light processing method described above.

Referring to fig. 7, which is a schematic structural diagram of a multi-frequency phase shift structure light processing device in an embodiment of the present disclosure, in some embodiments, the multi-frequency phase shift structure light processing device 100 may include:

a projection device 110 adapted to project intermediate frequency fringes and high frequency fringes towards an object S to be measured;

the sampling device 120 is adapted to acquire a medium frequency stripe pattern and a high frequency stripe pattern of the object S to be measured;

the processing device 130 is adapted to determine a first phase corresponding to a low-frequency fringe pattern of the object S to be measured according to a preset virtual depth, an internal reference of the sampling device 120, and a preset phase-high depth model, and determine a second phase corresponding to an intermediate-frequency fringe pattern and a third phase corresponding to a high-frequency fringe pattern of the object S to be measured according to the obtained first phase corresponding to the low-frequency fringe pattern, the second phase corresponding to the intermediate-frequency fringe pattern, and the third phase corresponding to the high-frequency fringe pattern of the object S to be measured, determine an absolute phase corresponding to the high-frequency fringe pattern of the object S to be measured, and determine a depth of the object S to be measured according to the absolute phase corresponding to the high-frequency fringe pattern of the object S to be measured and the phase-high depth model.

The operation of the processing device 130 may be referred to in the foregoing examples, and will not be described here.

In a specific implementation, the projection device 110 may project the intermediate frequency stripe and the high frequency stripe onto the object S to be detected, and the sampling device 120 may capture the intermediate frequency stripe and the high frequency stripe, so as to obtain an intermediate frequency stripe pattern and a high frequency stripe pattern of the object S to be detected.

On the one hand, by performing the decoding operation on the intermediate frequency fringe pattern and the high frequency fringe pattern, the processing apparatus 130 may determine the second phase corresponding to the obtained intermediate frequency fringe pattern and the third phase corresponding to the high frequency fringe pattern of the object S to be measured; on the other hand, the processing device 130 may determine the first phase corresponding to the low-frequency fringe pattern of the object S to be measured according to the preset virtual depth, the internal reference of the sampling device 120, and the preset phase-to-phase depth model, so as to determine the absolute phase corresponding to the high-frequency fringe pattern of the object S to be measured according to the first phase corresponding to the low-frequency fringe pattern of the object S to be measured, the second phase corresponding to the medium-frequency fringe pattern, and the third phase corresponding to the high-frequency fringe pattern, and determine the depth of the object S to be measured according to the absolute phase corresponding to the high-frequency fringe pattern of the object S to be measured and the phase-to-phase depth model, thereby completing the reconstruction process of the three-dimensional shape of the object S to be measured.

By adopting the multi-frequency phase shift structure light processing device in the example, the low-frequency fringe pattern of the object to be detected is not required to be acquired, and compared with the prior art that a plurality of projection fringe images in low-frequency, medium-frequency and high-frequency states are required to be acquired, the time required for acquiring and processing the low-frequency fringe pattern can be reduced, and the reconstruction efficiency of the three-dimensional shape is improved.

For example, with the multi-frequency phase shift structured light processing device in the embodiment of the present specification, the effect of the conventional 12 stripe patterns can be achieved by applying to 7 stripe patterns (3 intermediate frequency stripe patterns and 4 high frequency stripe patterns).

In some examples, the processing device may be implemented by a central processing unit (Central Processing Unit, CPU), field programmable gate array (Field Programmable Gate Array, FPGA), or the like processing chip, as well as by an integrated circuit (Application Specific Integrated Circuit, ASIC) or one or more integrated circuits configured to implement embodiments of the present description.

In a specific implementation, as shown in fig. 8, a block diagram of a data processing apparatus is provided in an embodiment of the present disclosure. In fig. 8, data processing device 200 may include a memory 210 and a processor 220, with communication between memory 210 and processor 220 being possible via a communication bus 230; the memory 210 stores computer instructions capable of being executed on the processor 220, and when the processor 220 executes the computer instructions, the steps of the multi-frequency phase shift structure light processing method described in any of the above embodiments may be executed, and detailed descriptions thereof will be omitted herein.

In implementations, the processor may include a Central Processing Unit (CPU), a graphics processor (Graphics Processing Unit, GPU), a Field Programmable Gate Array (FPGA), or the like.

The Memory may include random access Memory (Random Access Memory, RAM), read-Only Memory (ROM), nonvolatile Memory (NVM), and the like.

In particular implementations, the computer instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, and the like, implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.

In an implementation, as shown in fig. 8, the data processing apparatus 200 may further include a display interface 240 and a display 250 connected through the display interface 240. The display interface 240 may communicate with the memory 210 and the processor 220 via the communication bus 230. The display 250 may display results from the processor 220 performing the multi-frequency phase shift structured light processing methods provided in embodiments of the present disclosure.

The present invention also provides a computer readable storage medium, on which computer instructions are stored, where the computer instructions can execute the steps of the optical processing method with a multi-frequency phase shift structure according to any of the foregoing embodiments of the present invention, and the specific reference may be made to the foregoing related content, which is not described herein.

Wherein the computer-readable storage medium may include any suitable type of memory unit, memory device, memory article, memory medium, storage device, storage article, storage medium, and/or storage unit. Such as memory, removable or non-removable media, erasable or non-erasable media, writeable or re-writeable media, digital or analog media, hard disk, floppy disk, compact Disk Read Only Memory (CDROM), compact disk recordable (CD-R), compact disk Rewriteable (CD-RW), optical disk, magnetic media, magneto-optical media, removable memory cards or disks, various types of Digital Versatile Disks (DVDs), a tape, a cassette, or the like.

Moreover, computer instructions may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, encrypted code, and the like, implemented using any suitable high-level, low-level, object-oriented, visual, compiled and/or interpreted programming language.

Although the embodiments of the present specification are disclosed above, the present invention is not limited thereto. Various changes and modifications may be made by one skilled in the art without departing from the spirit and scope of the invention, and the scope of the invention should be assessed accordingly to that of the appended claims.

Claims

1. A method for processing light of a multi-frequency phase shift structure, wherein the multi-frequency comprises a low frequency, an intermediate frequency and a high frequency, and wherein the method for processing light of the multi-frequency phase shift structure comprises the steps of:

2. The method for processing the multi-frequency phase shift structure according to claim 1, wherein determining the first phase corresponding to the low-frequency fringe pattern of the object to be measured according to the preset virtual depth, the internal reference of the sampling device, and the preset phase-height depth model comprises:

3. The method of claim 1, wherein the calibration parameters of the internal reference and the phase-height depth model of the sampling device are determined by:

4. A method of multi-frequency phase shift structured light processing according to claim 3, wherein said calculating images from each set of said calibration plate in different poses comprises:

5. The method for processing the light with the multi-frequency phase shift structure according to claim 1, wherein determining the second phase corresponding to the intermediate frequency fringe pattern of the object to be detected obtained by the sampling device includes:

6. The method of claim 5, wherein the formulas corresponding to the first three steps of the four-step phase shift method comprise:

7. The method according to claim 1, wherein determining the absolute phase corresponding to the high-frequency fringe pattern of the object according to the first phase corresponding to the low-frequency fringe pattern, the second phase corresponding to the medium-frequency fringe pattern, and the third phase corresponding to the high-frequency fringe pattern of the object comprises:

8. The method for optical processing of a multi-frequency phase shift structure according to claim 7, wherein determining the absolute phase corresponding to the intermediate frequency fringe pattern of the object according to the first phase corresponding to the low frequency fringe pattern of the object and the second phase corresponding to the intermediate frequency fringe pattern of the object comprises:

9. The method of multi-frequency phase shift structured light processing of claim 8, further comprising:

10. A multi-frequency phase shift structured light processing apparatus, comprising:

11. A data processing apparatus comprising a memory and a processor, wherein the memory is adapted to store one or more computer instructions which, when executed by the processor, perform the multi-frequency phase shift structured light processing method of any of claims 1 to 9.

12. A computer readable storage medium having stored thereon computer instructions which, when executed, perform the multi-frequency phase shift structured light processing method of any of claims 1 to 9.