CN115876124A - High-light-reflection surface three-dimensional reconstruction method and device based on polarized structured light camera - Google Patents

Abstract

The invention discloses a three-dimensional reconstruction method and a three-dimensional reconstruction device for a high-reflection surface based on a polarization structured light camera. The method comprises the following steps: the method comprises the steps that a polarizer is arranged in front of a lens of a projector, the projector penetrates through the polarizer to project linearly polarized light stripes of an object to be measured into the field of view of a camera, and the camera is a polarization camera; the camera vertically images a plurality of polarization state images aiming at the object to be detected, the polarization angle of the projected linearly polarized light stripe is not perpendicular to the polarization states, and then a plurality of projection images with different brightness are obtained, wherein the number of the projection images is consistent with that of the polarization state images; for the plurality of projection images, obtaining a plurality of corresponding point clouds through point cloud reconstruction; and performing point cloud fusion on the plurality of pieces of point clouds to obtain a three-dimensional reconstruction result of the object to be detected. The method improves the efficiency and effect of three-dimensional reconstruction of the object, and is particularly suitable for the surface of a high-reflectivity object.

Description

High-reflectivity surface three-dimensional reconstruction method and device based on polarization structured light camera

Technical Field

The invention relates to the technical field of machine vision, in particular to a three-dimensional reconstruction method and a three-dimensional reconstruction device for a high-reflection surface based on a polarized structured light camera.

Background

With the rapid development of computer vision technology, visual sensors capable of three-dimensional measurement are increasingly widely applied in industry. The structured light three-dimensional reconstruction technology based on the projector and the camera has the advantages of rapid imaging, high speed, high precision and the like in one-time scanning. However, for some objects with reflective surfaces, overexposure is easily caused by scanning with a projector, which seriously affects the imaging of a camera, resulting in the loss of three-dimensional reconstruction point cloud and noise generation. The point cloud missing and the noise are all irreversible information loss in the measurement process actually, and the fitting information obtained by various technologies has unreliability, so that the precision of industrial measurement can be reduced. The vibration direction of the electromagnetic field of the common light is random, the light which is irregularly transmitted can be filtered by utilizing the polarization component, and only the polarized light which is regularly changed along with time is transmitted through the electromagnetic field. The polarized light technology can be used for filtering randomly disordered scattered light and only obtaining effective light rays in a specified polarization state.

In recent years, with the rapid development of three-dimensional reconstruction techniques in computer vision, more and more manufacturers have started to introduce relevant three-dimensional scanning apparatuses. Structured light 3D camera based on projecting apparatus and camera is as one of them extremely important three-dimensional visual sensor, can realize high accuracy quick scanning and rebuild. However, in industrial manufacturing, a large number of samples with highly reflective surfaces, such as metal workpieces, exist, and the highly reflective phenomenon can seriously affect the accuracy of three-dimensional reconstruction, resulting in the situation of irreversible loss of measurement information. Therefore, how to suppress and eliminate the high light reflection phenomenon becomes a difficult problem to be solved in the three-dimensional reconstruction of the structured light. The existing methods usually involve removal of the high reflectance light around the techniques of reducing light intensity and polarized light to achieve the integrity of the structured light three-dimensional reconstruction.

For example, patent application CN113554575A provides a method for removing high-reflection object surface highlights based on a polarization principle, which obtains and synthesizes imaging pictures of a high-reflection object under multiple polarization angles according to an optimal polarization angle principle, thereby achieving the purposes of improving an image signal-to-noise ratio and weakening highlights. Meanwhile, a normalization weighting algorithm is provided, highlight images under multiple exposure times are collected and synthesized, and surface information in the highlight area is recovered.

Patent application CN115235377A provides a three-dimensional measurement method based on polarization-optimized projection intensity. The method can directly estimate the required optimal projection intensity by establishing a camera response function under a polarization system, and is used for compensating the image light intensity reduced by the additional polaroid. This method does not use a rotating polarizer and multiple exposure times. In addition, the best stripe image is obtained by applying an image fusion algorithm.

Patent application CN113237435A provides a three-dimensional vision measuring system and method for high reflective surface, which includes: establishing a model between the light intensity of a projector and the imaging gray of a camera, projecting a uniform saturated gray image, collecting the image under high and low exposure time, marking saturated pixels to obtain an imaging area of a high light reflection area on the surface of an object in the camera image, and calculating the low gray projection intensity which enables the imaging of the low gray projection intensity not to be saturated; judging the absolute phase of a camera pixel, obtaining one or more camera pixels corresponding to each pixel to be adjusted in a projection image, and calculating the gray level of the projection pixel and the average gray level of the camera pixel corresponding to the projection pixel to form a gray level matching pair; fitting projection intensity model parameters by taking the matched projection pixel gray level and the camera pixel gray level as samples; calculating the optimal projection intensity according to the projection intensity model to generate a self-adaptive sinusoidal image; the projector projects a self-adaptive sinusoidal image, an industrial camera acquires the image, the multi-frequency heterodyne phase-solving method solves the phase, and the three-dimensional shape of the object is obtained according to the fringe projection profilometry visual measurement model.

Patent application CN108645354A provides a structured light three-dimensional imaging method and system for highly reflective object surface. The method comprises the following steps: projecting a plurality of groups of binary phase shift coding patterns with the same frequency and different illumination intensities onto the surface of a measured object, generating a plurality of modulation picture groups under different illumination intensities according to the reflected images, and arranging the modulation picture groups in a descending order according to the illumination intensities; the method comprises the steps of obtaining the illumination saturation intensity of each pixel point in a group with the maximum illumination intensity, obtaining saturated pixel points, determining a saturated area, obtaining replacement pixel points which correspond to the saturated pixel points in the corresponding areas of other groups in a one-to-one mode and have the minimum illumination saturation intensity and the maximum illumination intensity, calculating the phase, and replacing the phase of the saturated pixel points with the phase of the replacement pixel points to obtain a three-dimensional image after the detected object is repaired.

Through analysis, the prior art mainly has the following technical defects:

1) For the existing structured light 3D camera, due to the phenomena of point cloud loss, noise and the like caused by overexposure, for some simple regular surfaces, a global or local fitting method is usually adopted to calculate the loss information. However, the fitting information has a large uncertainty, and if a more complex surface is encountered, the method tends to generate fitting errors and is difficult to use for accurate measurement.

2) The existing structured light 3D camera can also adopt a multi-exposure method to image in different exposure time, and several pieces of point clouds are obtained through reconstruction, so that actual information can also be obtained at a highlight position, and finally point cloud fusion is carried out to obtain complete point clouds. The method has a good reconstruction effect in practical application, but objects with stronger light reflection phenomena cannot be completely reconstructed; the multiple exposure scan time increases, the introduced noise may also increase, and the efficiency may decrease.

3) The existing method for eliminating high reflection by combining structured light 3D reconstruction with a polarized light technology is mainly to install a polarizer on a camera lens, obtain a plurality of polarization states of reconstructed point clouds by adjusting angles for a plurality of times, and obtain a complete point cloud by point cloud fusion. The method needs to manually rotate the polarizer to adjust the angle, and is low in efficiency in practical application.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a method and a device for three-dimensional reconstruction of a high-reflectivity surface based on a polarized structured light camera.

According to the first aspect of the invention, a three-dimensional reconstruction method of a high-reflectivity surface based on a polarized structured light camera is improved. The method comprises the following steps:

the method comprises the steps that a polarizer is arranged in front of a lens of a projector, the projector penetrates through the polarizer to project linearly polarized light stripes of an object to be measured into the field of view of a camera, and the camera is a polarization camera;

the camera vertically images a plurality of polarization state images aiming at the object to be detected, the polarization angle of the projected linearly polarized light stripe is not perpendicular to the polarization states, and then a plurality of projection images with different brightness are obtained, wherein the number of the projection images is consistent with that of the polarization state images;

for the plurality of projection images, acquiring a plurality of corresponding point clouds through point cloud reconstruction;

and carrying out point cloud fusion on the plurality of pieces of point clouds to obtain a three-dimensional reconstruction result of the object to be detected.

According to a second aspect of the invention, a three-dimensional reconstruction device of a high-light-reflection surface based on a polarized structured light camera is provided. The device includes projecting apparatus, camera, polarizer, point cloud rebuilds unit and point cloud and fuses the unit, the polarizer sets up before the camera lens of projecting apparatus, the camera is polarization camera, wherein:

the projector transmits the polarizer to project linearly polarized light stripes of the object to be measured into the camera view;

the camera vertically images a plurality of polarization state images aiming at an object to be detected, the polarization angle of the projected linearly polarized light stripe is not vertical to the polarization states, and then a plurality of projection images with different brightness are obtained, wherein the number of the projection images is consistent with that of the polarization state images;

the point cloud reconstruction unit is used for acquiring a plurality of corresponding point clouds for the plurality of projection images through point cloud reconstruction;

the point cloud fusion unit is used for performing point cloud fusion on the plurality of pieces of point clouds to obtain a three-dimensional reconstruction result of the object to be detected.

Compared with the prior art, the method has the advantages that the purpose of eliminating the high reflection influence to obtain the complete point cloud is achieved aiming at the problem that the high reflection influence to the three-dimensional reconstruction of the structured light, and the method has the advantages of high reliability, short imaging time, convenience in operation and the like.

Other features of the present invention and advantages thereof will become apparent from the following detailed description of exemplary embodiments thereof, which proceeds with reference to the accompanying drawings.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description, serve to explain the principles of the invention.

FIG. 1 is a flow chart of a three-dimensional reconstruction method of a highly reflective surface based on a polarized structured light camera according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of a polarized structured light 3D system apparatus according to one embodiment of the present invention;

fig. 3 is a schematic diagram of the principle of a polarizer according to an embodiment of the present invention;

fig. 4 is a schematic illustration of a bayer array, according to one embodiment of the present disclosure;

FIG. 5 is a schematic diagram of a polarized camera pixel array according to one embodiment of the invention;

FIG. 6 is a diagram of a mathematical model of a structured light system according to one embodiment of the present invention;

FIG. 7 is a schematic illustration of a polarized projection image according to one embodiment of the invention;

FIG. 8 is a schematic diagram of reconstruction of a point cloud and point cloud fusion according to one embodiment of the present invention;

FIG. 9 is a comparison diagram of point cloud reconstruction by different methods according to one embodiment of the invention.

Detailed Description

Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that: the relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless it is specifically stated otherwise.

The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate.

In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be discussed further in subsequent figures.

The invention provides a method for removing high reflection by utilizing a polarized light technology to realize complete reconstruction of structured light. In short, the method adopts a polarization camera capable of imaging four polarization states at one time, a linear polarizer is additionally arranged in front of a projector lens to enable projection light to be linearly polarized light with a fixed angle, four point clouds are reconstructed by utilizing images of four different polarization states, and point cloud fusion is carried out to obtain complete point cloud. The method can reconstruct the point cloud of four polarization states at the same time through one-time projection, and meanwhile, because the projection light is the polarized light, the light intensity of the image received by each polarization state is different, and the point cloud information of the highlight and dim positions can be well compensated. It should be understood that although described based on four polarization state images, the invention is not limited to the number of polarization state images imaged, the number of point clouds reconstructed, etc.

Referring to fig. 1, the provided three-dimensional reconstruction method for the highly reflective surface based on the polarized structured light camera includes the following steps:

in step S110, the projector projects linear polarized light stripes into the field of view of the polarized camera through a polarizer disposed in front of the lens.

Referring to fig. 2, the polarized structured light 3D device adopted by the present invention includes a projector, a polarizer and a polarized camera, wherein the polarizer is disposed in front of a lens of the projector, and the projector transmits the polarizer to project linearly polarized light fringes of an object to be measured to a field of view of the camera. In practical applications, various types of commercially available or specialized polarizers, projectors, and polarization cameras may be employed. For example, a linear polarizer is currently used, but in practice, the light intensity is attenuated to some extent, and therefore, a circular polarizer can be used to achieve the same function and maintain the original light intensity.

The principle of the polarizer is shown in fig. 3. Specifically, light is an electromagnetic wave, and the directions of the electric field and the magnetic field are perpendicular to the propagation direction of light. It can be assumed that the polarizer consists of a regular array of thin metal wires parallel to each other, where the thin metal wires are sufficiently thin to approximately say that electrons can move in one direction only. Therefore, only the electromagnetic field component in the same direction as the polarizer does work on the electrons, and the energy of the light is attenuated to a certain degree, so that the light is vibrated in one plane. The polarizer is arranged in front of a projector lens, and projector light which vibrates randomly is changed into linearly polarized light with a fixed polarization angle through the polarizer and is projected into the field of view of the camera.

And step S120, vertically imaging images of four polarization states by the polarization camera, wherein the polarization angle of the projection stripe is not vertical to the four polarization states, and further obtaining four images with different brightness.

A common color RGB camera converts a gray-scale picture into a color picture by saving different color channel intensities through a bayer array as shown in fig. 4. In a similar way, as shown in fig. 5, the polarization camera stores pixels of one polarization state in each of four adjacent pixels, which are 0 °, 45 °, 90 °, and 135 °, respectively. After image acquisition is finished, pixels corresponding to polarization states are taken out, four pictures are segmented and synthesized, so that four images with different polarization states can be obtained by photographing once, and each image is one fourth of the resolution of an original image.

And step S130, reconstructing four pieces of point clouds according to the obtained four projection images.

The three-dimensional reconstruction technology based on the structured light is an active three-dimensional reconstruction method, a projector is used for projecting a coding stripe pattern to the surface of a target in the visual field of a camera, the camera shoots the target and decodes the coding stripe pattern, and then the three-dimensional information of an object can be obtained through a triangulation principle. In one embodiment, the coding stripe pattern used is 18 stripe patterns made by gray code plus line shift coding.

Based on the imaging model of the camera, the relationship between the plane coordinates of the camera and the three-dimensional points of the camera can be quickly established. In the structured light reconstruction technique, the camera is regarded as a reverse-path camera, so that the relevant relation can be established by using a camera model. As shown in FIG. 6, M is a point on the object to be measured, and M is M at the corresponding point of the projector image _p ＝(u _p ,v _p ) ^T At the corresponding point of the camera image, m _c ＝(u _c ,v _c ) ^T 。

Suppose the camera focal length is f _c According to the projective relationship:

the coordinate of the point M in the space coordinate system of the projector is M _p ＝(x _p ,y _p ,z _p ) ^T The coordinate in the camera space coordinate system is M _c ＝(x _c ,y _c ,z _c ) ^T The two types of coordinates are as follows:

M _p ＝R·M _c +T(2)

where R and T are the rotation matrix and spatial translation vector between the projector and camera coordinate systems, respectively.

According to the camera model, the image coordinate system and the space coordinate system have the following relations:

wherein K is an internal reference of the camera model.

By using the principle of triangulation, depth information z can be obtained _c ：

Finally, the three-dimensional coordinate M of the space point M under the camera coordinate system can be obtained _c Comprises the following steps:

M _c ＝u _c z _c /f _c ,v _c z _c /f _c ,z _c ) ^T (6)

the projector projects linear polarization stripes, the polarization camera is triggered to shoot 18 stripe patterns, the stripe patterns in four polarization states are obtained through decomposition, and 72 stripe patterns are actually obtained. The point cloud under four polarization states can be reconstructed by using the structured light three-dimensional reconstruction method.

And step S140, performing point cloud fusion on the four obtained point clouds based on the image brightness to obtain a three-dimensional reconstruction result of the object to be detected.

For the reconstructed point cloud, the point cloud is lost due to overexposure and overexposure of image pixels, and therefore, corresponding fusion operation needs to be performed by judging whether the pixel value of the gray level image and the point cloud are lost. Meanwhile, traversing the gray level image pixels in four polarization states, firstly judging whether the corresponding pixels have the overexposure condition, if so, judging whether the gray level value of the pixels exceeds 255, and then checking whether the four point clouds of the corresponding pixels have points or not.

Specifically, if the gray value exceeds 255 and the point cloud is missing, the point cloud is considered to be missing due to overexposure, and the fusion strategy is to select the polarization-state point cloud with the lowest gray value for completion;

if the gray value does not exceed 255 and point cloud loss does not exist, taking the average value of the point location coordinates of the four pieces of point clouds as the point location coordinate of the pixel;

if the gray value does not exceed 255 but the point cloud is missing, the point cloud is considered to be missing due to over-darkness, and the fusion strategy is to select the polarization-state point cloud with the highest gray value for completion.

Correspondingly, the invention further provides a high-reflectivity surface three-dimensional reconstruction device based on the polarized structured light camera, and the device is used for realizing one or more aspects of the method. For example, the apparatus comprises a projector, a camera, a polarizer, a point cloud reconstruction unit, and a point cloud fusion unit, the polarizer being disposed in front of a lens of the projector, the camera being a polarization camera, wherein: the projector transmits the polarizer to project linearly polarized light stripes of the object to be measured into the camera view; the camera vertically images a plurality of polarization state images aiming at an object to be detected, the polarization angle of the projected linearly polarized light stripe is not vertical to the polarization states, and then a plurality of projection images with different brightness are obtained, wherein the number of the projection images is consistent with that of the polarization state images; the point cloud reconstruction unit is used for acquiring a plurality of corresponding point clouds for the plurality of projection images through point cloud reconstruction; the point cloud fusion unit is used for performing point cloud fusion on the plurality of pieces of point clouds to obtain a three-dimensional reconstruction result of the object to be detected. In the device, the point cloud reconstruction unit, the point cloud fusion unit and the like can be realized by a processor, an FPGA (field programmable gate array) or special hardware.

In order to further verify the effect of the invention, experimental verification is carried out. The experimental sample is a free-form surface metal raised table, the photographing reconstruction is respectively carried out by adopting the existing structured light 3D camera with the same visual field and the device provided by the invention, and the single exposure, the multiple exposure and the reconstruction effect of the invention are compared. Referring to fig. 7 to 9, fig. 7 is a polarization projection image, fig. 8 is a schematic diagram of reconstructed point cloud and point cloud fusion, fig. 9 is a schematic diagram of reconstructed point cloud of the present invention under single exposure and multiple exposure, and the lower right corner picture corresponds to the present invention. As can be seen from fig. 9, the point cloud missing exists due to the overexposure in a single exposure, and the multiple exposure can repair the missing part of the point cloud to a certain extent, but the reconstructed point cloud has a complete effect.

In summary, the invention adopts a structured light 3D system constructed by a polarization camera, a projector and a polarizer, wherein the polarizer is installed in front of a lens of the projector; and the constructed structured light 3D system is used for projecting to obtain images in different polarization states, high reflection is eliminated, and complete reconstruction of point cloud is realized.

Compared with the prior art, the invention has the following advantages:

1) Compared with the existing scheme that the missing information is solved by a fitting method and the influence of high light reflection is eliminated to obtain complete point cloud, the method is more reliable and credible and has practical significance for industrial detection.

2) Compared with multi-exposure point cloud fusion, the method can obtain four pieces of point clouds for fusion by one-time imaging, does not need to shoot and project for many times, needs short time, and reduces the probability of introducing noise in the shooting process.

3) Compared with the method that the polarizer is arranged on the camera lens, and the polarization angle is manually adjusted to obtain images with different light intensities, the method and the device have the advantages that the polarizer is arranged in front of the projector lens to project linearly polarized light, the polarization camera capable of obtaining four polarization states is adopted to obtain the images with different light intensities, manual adjustment is not needed, four images can be imaged at one time, and the use is convenient.

The present invention may be a system, method and/or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions embodied therewith for causing a processor to implement various aspects of the present invention.

The computer-readable storage medium may be a tangible device that can hold and store the instructions for use by the instruction execution device. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, semiconductor memory device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a Static Random Access Memory (SRAM), a portable compact disc read-only memory (CD-ROM), a Digital Versatile Disc (DVD), a memory stick, a floppy disk, a mechanical coding device, such as punch cards or in-groove projection structures having instructions stored thereon, and any suitable combination of the foregoing. Computer-readable storage media as used herein is not to be interpreted as a transitory signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., optical pulses through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or to an external computer or external storage device via a network, such as the internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. The network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in the respective computing/processing device.

The computer program instructions for carrying out operations of the present invention may be assembler instructions, instruction Set Architecture (ISA) instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C + +, python, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider). In some embodiments, aspects of the present invention are implemented by personalizing an electronic circuit, such as a programmable logic circuit, a Field Programmable Gate Array (FPGA), or a Programmable Logic Array (PLA), with state information of computer-readable program instructions, which can execute the computer-readable program instructions.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable medium storing the instructions comprises an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer, other programmable apparatus or other devices implement the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, by software, and by a combination of software and hardware are equivalent.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen in order to best explain the principles of the embodiments, the practical application, or technical improvements to the market, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims (10)

1. A three-dimensional reconstruction method of a high-reflectivity surface based on a polarization structure light camera comprises the following steps:

the method comprises the steps that a polarizer is arranged in front of a lens of a projector, the projector penetrates through the polarizer to project linearly polarized light stripes of an object to be measured into the field of view of a camera, and the camera is a polarization camera;

the camera vertically images a plurality of polarization state images aiming at the object to be detected, the polarization angle of the projected linearly polarized light stripe is not perpendicular to the polarization states, and then a plurality of projection images with different brightness are obtained, wherein the number of the projection images is consistent with that of the polarization state images;

for the plurality of projection images, obtaining a plurality of corresponding point clouds through point cloud reconstruction;

and performing point cloud fusion on the plurality of pieces of point clouds to obtain a three-dimensional reconstruction result of the object to be detected.

2. The method of claim 1, wherein the number of polarization state images and the projected images is four, and each projected image is obtained according to the steps of:

for the imaging of the camera, a pixel in a polarization state is stored in each of four adjacent pixels, and the pixel is respectively 0 degree, 45 degrees, 90 degrees and 135 degrees;

the camera acquires pixels corresponding to polarization states after finishing image acquisition, and four pictures are segmented and synthesized, so that four images with different polarization states are acquired by taking a picture at one time, and each image is one fourth of the resolution of an original image.

3. The method of claim 1, wherein the projector projecting a linearly polarized light stripe of the object to be measured through the polarizer into the camera field of view comprises: and projecting the coding stripe pattern to the surface of the object to be detected in the field of view of the camera through the projector, shooting a target and decoding the coding stripe pattern by the camera, and obtaining three-dimensional information of the object through a triangulation principle.

4. The method of claim 3, wherein the coded stripe pattern is an 18 stripe pattern made by gray code plus line shift coding.

5. The method according to claim 1, wherein in the point cloud reconstruction process, for a point on the object to be measured, the corresponding three-dimensional coordinates are obtained according to the following steps:

focal length f for camera _c In this case, the projective relationship is expressed as:

wherein M is a point on the object to be measured, and M is at the corresponding point of the projector imagem _p ＝(u _p ,v _p ) ^T At the point corresponding to the camera image, m _c ＝(u _c ,v _c ) ^T And the coordinate of M in the space coordinate system of the projector is M _p ＝(x _p ,y _p ,z _p ) ^T The coordinate in the camera space coordinate system is M _c ＝(x _c ,y _c ,z _c ) ^T ；

The relationship between the projector space coordinate system and the camera space coordinate system is expressed as:

M _p ＝R·M _c +T

wherein R and T are a rotation matrix and a spatial translation vector between the projector coordinate system and the camera coordinate system, respectively;

according to the camera model, the image coordinate system and the space coordinate system have the following relations:

depth information z is obtained by utilizing the principle of triangulation _c Expressed as:

obtaining the three-dimensional coordinate M of the point M under the camera coordinate system _c Expressed as:

M _c ＝u _c z _c /f _c ,v _c z _c /f _c ,z _c ) ^T

wherein K is an internal reference of the camera model.

6. The method of claim 2, wherein point cloud fusing the plurality of pieces of point clouds comprises:

traversing the gray level image pixels in four polarization states, firstly judging whether the corresponding pixels have the overexposure condition, if so, checking whether four pieces of point cloud of the corresponding pixels have points, wherein the gray level value of the pixels exceeds 255:

if the gray value exceeds 255 and the point location is missing, the point cloud is considered to be missing due to overexposure, and the polarization state point cloud with the lowest gray value is selected for completing;

if the gray value does not exceed 255 and no point cloud is lost, taking the average value of the point location coordinates of the four pieces of point clouds as the point location coordinate of the pixel;

if the gray values do not exceed 255 but the point cloud is missing, the point cloud is considered to be missing due to over-darkness, and the polarization state point cloud with the highest gray value is selected for compensation.

7. The method of claim 1, wherein the polarizer comprises a regular array of thin metal wires parallel to each other.

8. The method of claim 1, wherein the polarizer is mounted in front of the projector lens, and wherein randomly oscillating projector light passes through the polarizer to become linearly polarized light of a fixed polarization angle for projection into the camera field of view.

9. The utility model provides a three-dimensional reconstruction device of high reflection of light surface based on polarized structure light camera, includes projecting apparatus, camera, polarizer, point cloud rebuilds unit and point cloud and fuses the unit, the polarizer sets up before the camera lens of projecting apparatus, the camera is polarization camera, wherein:

the projector transmits the polarizer to project linearly polarized light stripes of the object to be measured into the camera view;

the camera vertically images a plurality of polarization state images aiming at an object to be detected, the polarization angle of the projected linearly polarized light stripe is not vertical to the polarization states, and then a plurality of projection images with different brightness are obtained, wherein the number of the projection images is consistent with that of the polarization state images;

the point cloud reconstruction unit is used for acquiring a plurality of corresponding point clouds for the plurality of projection images through point cloud reconstruction;

the point cloud fusion unit is used for performing point cloud fusion on the plurality of pieces of point clouds to obtain a three-dimensional reconstruction result of the object to be detected.

10. A computer-readable storage medium, on which a computer program is stored, wherein the computer program, when being executed by a processor, realizes the steps of the method according to any one of claims 1 to 8.

Images