CN112669438A

CN112669438A - Image reconstruction method, device and equipment

Info

Publication number: CN112669438A
Application number: CN202011622241.1A
Authority: CN
Inventors: 陈波; 陈素芳
Original assignee: Hangzhou Hikrobot Technology Co Ltd
Current assignee: Hangzhou Hikrobot Technology Co Ltd
Priority date: 2020-12-31
Filing date: 2020-12-31
Publication date: 2021-04-16

Abstract

The application provides an image reconstruction method, an image reconstruction device and image reconstruction equipment, wherein the method comprises the following steps: acquiring a plurality of initial line structure light images with different brightness levels; determining a light bar central line of the light image with the initial line structure, and determining a reliability index of the light bar central line based on characteristic information of the light bar central line; selecting a line structured light image to be fused, and fusing the light strip center line of the initial line structured light image and the light strip center line of the line structured light image to be fused based on the reliability index of the light strip center line of the initial line structured light image and the reliability index of the light strip center line of the line structured light image to be fused to obtain a target light strip center line; replacing the light strip central line of the initial line structured light image by the target light strip central line to obtain a target line structured light image; and performing three-dimensional reconstruction based on a plurality of target line structure light images corresponding to the plurality of initial line structure light images. According to the scheme, the light strip center line of the line structure light image is accurately acquired, and the three-dimensional reconstruction effect of complex materials is improved.

Description

Image reconstruction method, device and equipment

Technical Field

The present application relates to the field of image processing technologies, and in particular, to an image reconstruction method, apparatus, and device.

Background

The line structured light measuring system consists of a laser and a camera, the laser is used for projecting line structured light to the surface of a measured target (namely a measured object), and the camera is used for shooting the measured target to obtain an image with the line structured light, namely a line structured light image. After the line structured light image is obtained, the light strip center line of the line structured light image can be obtained, and the light strip center line is converted according to the pre-calibrated sensor parameters, so that the space coordinate (namely, the three-dimensional coordinate) of the measured target at the current position is obtained. Based on the space coordinates of the measured object at the current position, three-dimensional reconstruction (i.e. three-dimensional reconstruction) of the measured object can be realized.

It can be seen from the above contents that obtaining the central line of the light stripe of the line structured light image is an important link for realizing three-dimensional reconstruction, and how to accurately obtain the central line of the light stripe of the line structured light image directly affects the three-dimensional reconstruction effect. However, in the related art, the manner of acquiring the central line of the light bar of the line structured light image depends on the image quality of the line structured light image, and the image quality depends on the intensity of the ambient light and the surface material property of the measured object, and once the image quality is poor, the central line of the light bar of the line structured light image cannot be accurately acquired, and thus the three-dimensional reconstruction effect is poor.

Disclosure of Invention

The application provides an image reconstruction method, which comprises the following steps:

acquiring a plurality of initial line structure light images with different brightness levels;

for each initial line structured light image, then:

determining a light bar central line of the initial line structured light image, and determining a reliability index of the light bar central line based on characteristic information of the light bar central line; selecting a line structure light image to be fused from the plurality of initial line structure light images; fusing the light strip center line of the initial line-structured light image and the light strip center line of the line-structured light image to be fused based on the reliability index of the light strip center line of the initial line-structured light image and the reliability index of the light strip center line of the line-structured light image to be fused to obtain a target light strip center line; replacing the light strip central line of the initial line structured light image by the target light strip central line to obtain a target line structured light image corresponding to the initial line structured light image;

and performing three-dimensional reconstruction based on a plurality of target line structure light images corresponding to the plurality of initial line structure light images.

Illustratively, the acquiring a plurality of initial line-structured light images of different brightness levels includes:

the control line structured light measuring system adopts different imaging parameters to obtain a plurality of initial line structured light images with different brightness levels; the line structured light measuring system at least comprises a laser and a camera, the imaging parameters are parameters for determining the brightness level of an image, and the imaging parameters comprise at least one of the following parameters: gain of the camera, exposure time of the camera, laser intensity of the laser.

Illustratively, the determining the light bar centerline of the initial line structured light image comprises:

for each dimensionality of the initial line structured light image, utilizing a segmentation threshold to segment pixel points of the dimensionality into a light bar area and a background area, wherein the gray value of the pixel points in the light bar area is larger than the segmentation threshold, and the gray value of the pixel points in the background area is not larger than the segmentation threshold; wherein each dimension is each row or each column of the initial line structured light image;

determining, for each light bar region, a center point of the light bar region;

if a central point exists in the dimension, determining the central point as the central point of the light bar of the dimension; if the dimension has at least two central points, clustering each central point to obtain the clustering point number of each central point, and determining the central point with the largest clustering point number as the light strip central point of the dimension; when the central points are clustered, searching the central points communicated with the central points in other dimensions except the dimensions by a neighborhood method, and determining the total number of the communicated central points and the total number of the central points as clustering points;

determining a light bar centerline of the initial line structured light image based on light bar center points for all dimensions.

Illustratively, the light bar centerline comprises a plurality of light bar center points, the characteristic information of the light bar centerline comprises characteristic information of the plurality of light bar center points; the determining a reliability indicator for the light bar centerline based on the characteristic information of the light bar centerline comprises:

determining a reliability index of the light strip central point based on the clustering point number of the light strip central point and the gray characteristic value of the light strip central point aiming at each light strip central point included by the light strip central line; the reliability index of the light strip central point is positively correlated with the clustering point number of the light strip central point, and the reliability index of the light strip central point is positively correlated with the gray characteristic value of the light strip central point;

and determining the reliability index of the central line of the light strip based on the reliability indexes of the central points of all the light strips of the central line of the light strip, wherein the reliability index of the central line of the light strip comprises the reliability indexes of the central points of all the light strips.

For example, the fusing the central line of the light bar of the initial line-structured light image and the central line of the light bar of the line-structured light image to be fused based on the reliability index of the central line of the light bar of the initial line-structured light image and the reliability index of the central line of the light bar of the line-structured light image to be fused to obtain the central line of the target light bar includes:

for each dimensionality of the initial line structured light image, judging whether a reliability index of a light strip central point of the dimensionality in the initial line structured light image is larger than a preset threshold value;

if so, determining the light bar central point of the dimensionality in the initial line structured light image as the target light bar central point of the dimensionality; if not, selecting the light strip central points to be fused from the light strip central points of the dimensionality in all the light images of the line structure to be fused based on the reliability index of the light strip central points of the dimensionality in the light images of the line structure to be fused, interpolating the light strip central points to be fused to obtain interpolated light strip central points, and determining the interpolated light strip central points as the target light strip central points of the dimensionality;

determining a target light bar centerline of the initial line structured light image based on target light bar center points for all dimensions of the initial line structured light image.

The line structure light image to be fused corresponding to the initial line structure light image is an initial line structure light image with the brightness level different from that of the initial line structure light image; or a target line structured light image corresponding to an initial line structured light image with a brightness level different from that of the initial line structured light image, wherein the selection of the light strip center points to be fused from the light strip center points of the dimensionality in all the line structured light images to be fused based on the reliability index of the light strip center points of the dimensionality in the line structured light image to be fused comprises:

for each line structured light image to be fused, if the line structured light image to be fused is an initial line structured light image, when the reliability index of the light strip center point of the dimensionality in the line structured light image to be fused is larger than a preset threshold value, selecting the light strip center point of the dimensionality in the line structured light image to be fused as the light strip center point to be fused; if not, selecting the light strip central point of the dimensionality in the light image of the line structure to be fused as the light strip central point to be fused; and if the line structured light image to be fused is the target line structured light image, selecting the light strip central point of the dimensionality in the line structured light image to be fused as the light strip central point to be fused.

Exemplarily, interpolating the central points of all the optical strips to be fused to obtain interpolated central points of the optical strips, including: performing polynomial interpolation on the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips; or carrying out cubic spline interpolation on the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips.

The line structured light measuring system at least comprises a laser and a camera, and the process of acquiring a plurality of initial line structured light images comprises the following steps: in the process that an object to be detected moves along the movement direction, the laser device is used for projecting line structured light to the surface of the object to be detected, and the camera is used for shooting the object to be detected to obtain a plurality of initial line structured light images; the object to be measured horizontally moves under the linear structured light measurement system along the movement direction, and in the movement process of the object to be measured, each position between the starting position and the ending position of the object to be measured sequentially passes through the designated position under the linear structured light measurement system, and the laser projects linear structured light to the designated position.

The present application provides an image reconstruction apparatus, the apparatus comprising:

the acquisition module is used for acquiring a plurality of initial line structure light images with different brightness levels;

the determination module is used for determining a light bar central line of the initial line structured light image aiming at each initial line structured light image and determining a reliability index of the light bar central line based on the characteristic information of the light bar central line;

the selection module is used for selecting a line structured light image to be fused from the plurality of initial line structured light images;

the processing module is used for fusing the light strip center line of the initial line structure light image and the light strip center line of the line structure light image to be fused based on the reliability index of the light strip center line of the initial line structure light image and the reliability index of the light strip center line of the line structure light image to be fused to obtain a target light strip center line; replacing the light strip central line of the initial line structured light image by the target light strip central line to obtain a target line structured light image corresponding to the initial line structured light image; and the reconstruction module is used for performing three-dimensional reconstruction on the basis of a plurality of target line structure light images corresponding to the plurality of initial line structure light images.

The present application provides an image reconstruction apparatus, comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor;

the processor is configured to execute machine executable instructions to perform the steps of:

for each initial line structured light image, then:

As can be seen from the above technical solutions, in the embodiments of the present application, multiple initial line structured light images with different brightness levels may be obtained, for each initial line structured light image, a target light bar center line of the initial line structured light image may be obtained based on the fusion of the initial line structured light image and a line structured light image to be fused, a target line structured light image corresponding to the initial line structured light image may be obtained based on the target light bar center line, and then three-dimensional reconstruction may be performed based on the multiple target line structured light images. In the above manner, the target light strip center line is obtained by fusing the plurality of line-structured light images with different brightness levels, so that the light strip center line of the line-structured light image is accurately obtained, and the three-dimensional reconstruction effect is improved. The method for obtaining the light strip center line of the line structure light image does not depend on the image quality of the line structure light image, and even if the image quality of the line structure light image is poor, the light strip center line of the line structure light image can be accurately obtained, so that the three-dimensional reconstruction effect is good, the reliability of the light strip center line is improved, and the three-dimensional reconstruction of the line structure light with high frame rate is favorably realized.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art according to the drawings of the embodiments of the present application.

FIG. 1 is a schematic diagram of a linear structured light measurement system collecting a linear structured light image;

FIG. 2 is a block diagram of the architecture in one embodiment of the present application;

FIG. 3 is a schematic flow chart diagram illustrating an image reconstruction method according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram illustrating cluster-based center point denoising in one embodiment of the present application;

FIG. 5 is a schematic diagram of an image reconstruction apparatus according to an embodiment of the present application;

fig. 6 is a hardware configuration diagram of an image reconstruction apparatus according to an embodiment of the present application.

Detailed Description

The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in this application and the claims, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein is meant to encompass any and all possible combinations of one or more of the associated listed items.

It should be understood that although the terms first, second, third, etc. may be used in the embodiments of the present application to describe various information, the information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present application. Depending on the context, moreover, the word "if" as used may be interpreted as "at … …" or "when … …" or "in response to a determination".

The image reconstruction method provided in the embodiment of the application can be applied to image reconstruction equipment, and the image reconstruction equipment can be a line structured light measurement system, a management equipment (such as a back-end server) and the like, and is not limited thereto, as long as the image reconstruction can be realized. If the method is applied to a linear structured light measuring system, the linear structured light measuring system can collect a linear structured light image, and the image reconstruction method is executed based on the linear structured light image. If the method is applied to the management equipment, the line-structured light measurement system can collect line-structured light images, send the line-structured light images to the management equipment, and the management equipment executes the image reconstruction method based on the line-structured light images.

Referring to fig. 1, a schematic diagram of a line structured light measuring system for collecting line structured light images is shown, the line structured light measuring system at least includes a laser and a camera (such as an analog camera or a web camera), and the laser and the camera are not shown in fig. 1. In a moving process (which may be uniform) of an object to be measured (also referred to as a measured object or a measured object, and subsequently, taking the measured object as an example) along a moving direction, a laser is used to project line structured light onto a surface of the measured object, and a camera is used to shoot the measured object, so as to obtain a line structured light image. After the line structure light image is obtained, the light strip center line of the line structure light image can be obtained, and the light strip center line is converted to obtain the space coordinate of the measured target at the current position. Based on the spatial coordinates of the measured object at the current position, three-dimensional reconstruction (i.e. three-dimensional reconstruction) of the measured object can be achieved.

The obtaining of the light strip center line of the line structured light image is an important link for realizing three-dimensional reconstruction, and how to accurately obtain the light strip center line of the line structured light image directly influences the three-dimensional reconstruction effect. In view of the above findings, an embodiment of the present application provides a method for dynamically fusing center lines of light images of a multi-brightness line structure, which is shown in fig. 2 and is a structural block diagram of the embodiment of the present application, and in a moving process of a detected target, linear structure light images of multiple brightness levels are collected by adjusting imaging parameters of a linear structure light measurement system, and light bar center line extraction and reliability evaluation are performed on the linear structure light images of different brightness levels to obtain light bar center lines and reliability indexes of the linear structure light images. Aiming at each line-structured light image, the light strip center line of the line-structured light image can be subjected to fusion processing based on the light strip center line and the reliability index, so that a more complete and accurate target light strip center line corresponding to the structured light image is obtained and is used for the subsequent three-dimensional reconstruction of the measured target.

In the above manner, the target light strip center line is obtained by fusing the plurality of linear structured light images with different brightness levels, so that the light strip center line of the linear structured light image is accurately obtained, and the three-dimensional reconstruction effect is improved. The method for obtaining the light strip center line of the line structure light image does not depend on the image quality of the line structure light image, and even if the image quality of the line structure light image is poor, the light strip center line of the line structure light image can be accurately obtained, so that the three-dimensional reconstruction effect is good, and the reliability of the light strip center line is improved.

The following describes an image reconstruction method according to an embodiment of the present application with reference to specific embodiments.

Referring to fig. 3, a flow chart of an image reconstruction method is schematically shown, and the method may include:

step 301, a plurality of initial line structured light images with different brightness levels are obtained.

Illustratively, the line structured light measurement system may include at least a laser and a camera, and the process of acquiring a plurality of initial line structured light images may include, but is not limited to: in the process that an object to be detected moves along the movement direction, line structured light is projected to the surface of the object to be detected through a laser, and the object to be detected is shot through a camera to obtain a plurality of initial line structured light images; the object to be measured performs horizontal movement (such as uniform movement) along the movement direction at the lower side of the linear structured light measurement system, and during the movement process of the object to be measured, each position between the starting position and the ending position of the object to be measured sequentially passes through the designated position at the lower side of the linear structured light measurement system, and the laser projects linear structured light to the designated position.

The specified position may be a position right under the linear structured light measurement system, or a position left from the position right under the linear structured light measurement system, or a position right from the position right under the linear structured light measurement system, and the specified position is not limited as long as the specified position is located under the linear structured light measurement system, and the laser can project the linear structured light to the specified position.

The start position of the object to be measured may also be referred to as the head position, i.e., the foremost position, of the object to be measured, and the end position of the object to be measured may also be referred to as the tail position, i.e., the rearmost position, of the object to be measured.

When the object to be measured horizontally moves along the moving direction under the linear structured light measurement system, the starting position of the object to be measured firstly passes through the designated position under the linear structured light measurement system, the ending position of the object to be measured finally passes through the designated position under the linear structured light measurement system, and each position between the starting position and the ending position of the object to be measured sequentially passes through the designated position under the linear structured light measurement system.

In the movement process of the object to be detected, each position of the object to be detected sequentially passes through the designated position on the lower side of the linear structure light measurement system, taking the position a as an example, when the position a passes through the designated position on the lower side of the linear structure light measurement system, the laser can project linear structure light to the designated position, namely, the linear structure light is projected to the position a of the object to be detected, so that when the camera is used for shooting the object to be detected to obtain an initial linear structure light image, the initial linear structure light image comprises the object to be detected, the position a of the object to be detected has a line, and the line is generated when the linear structure light is projected to the position a of the object to be detected and is also the basis of subsequent data processing.

In the state that the measured target performs uniform translational motion relative to the linear structured light measuring system, the linear structured light measuring system adopts different imaging parameters to obtain a plurality of linear structured light images with different brightness levels, and the linear structured light image obtained by the linear structured light measuring system is called as an initial linear structured light image for convenient distinguishing.

For example, the imaging parameter may be a parameter for determining the brightness level of the image, and based on this, a plurality of initial line-structured light images with different brightness levels may be obtained by controlling the line-structured light measuring system to use different imaging parameters. For example, when the online structured light measurement system adopts the imaging parameter 1, the initial line structured light image with the brightness level of 1 may be obtained, and when the online structured light measurement system adopts the imaging parameter 2, the initial line structured light image with the brightness level of 2 may be obtained, and so on.

The linear structured light measuring system at least comprises a laser and a camera, and in the process of uniform translational motion of a measured target, the laser is used for projecting linear structured light to the surface of the measured target, and the camera is used for shooting the measured target to obtain an initial linear structured light image. In summary, it can be seen that the brightness level of the initial line structured light image is related to the laser and the camera, and therefore, the imaging parameter may be an imaging parameter of the laser and/or an imaging parameter of the camera, for example, the imaging parameter may include, but is not limited to, at least one of the following: gain of the camera, exposure time of the camera, laser intensity of the laser. For convenience of description, the gain of the camera, the exposure time of the camera, and the laser intensity of the laser are used as examples in the following.

Illustratively, during the uniform translational motion of the measured object, the line structured light measuring system can acquire a line structured light image sequence from dark to bright by alternately changing a set of low-to-high imaging parameters, wherein the line structured light image sequence comprises a plurality of initial line structured light images with different brightness levels. Alternatively, a sequence of line structured light images from light to dark is acquired by alternately shifting a set of high to low imaging parameters. Of course, other ways may also be adopted to collect the line-structured light image sequence, as long as the line-structured light image sequence may include a plurality of initial line-structured light images with different brightness levels, and the collection way is not limited.

Line structured light image sequence IMG ═ IMG₁,IMG₂,…,IMGT,IMG_T+1,IMG_T+2,…,IMG_2T,…IMG_NT represents the cycle of imaging parameters, and N represents the initial periodTotal number of originating line structured light images.

For example, assuming that there are a total of T imaging parameters (a combination of gain 1, exposure time 1 and laser intensity 1 as one imaging parameter, a combination of gain 1, exposure time 1 and laser intensity 2 as another imaging parameter, and so on), the linear structured light measurement system obtains the initial linear structured light image IMG when the first imaging parameter is adopted₁And obtaining an initial line structured light image IMG when the line structured light measurement system adopts a second imaging parameter₂By analogy, the linear structured light measurement system obtains the initial linear structured light image IMG when adopting the T-th imaging parameter_TAt this time, the cycle of the imaging parameters is completed for one time, and the cycle of the imaging parameters is performed for the next time. Obtaining initial line structured light image IMG when line structured light measurement system adopts first imaging parameter_T+1And obtaining an initial line structured light image IMG when the line structured light measurement system adopts a second imaging parameter_T+2By analogy, the linear structured light measurement system obtains the initial linear structured light image IMG when adopting the T-th imaging parameter_2TAt this time, the cycle of one imaging parameter is completed, and the cycle of the next imaging parameter is performed, and so on.

In summary, according to the round-robin period T of the imaging parameters, the gain of the camera, the exposure time of the camera, the laser intensity of the laser, and the like are alternately changed, so that the initial line structured light images with a plurality of brightness levels can be obtained, the brightness levels of the initial line structured light images are all different in each round-robin period T, and the round-robin period T represents the type of the imaging parameters, that is, there are T types of imaging parameters, and the T types of imaging parameters correspond to the T brightness levels.

In a possible embodiment, when the linear structured light measurement system adopts different imaging parameters, the round-robin manner of the imaging parameters may be a single variable adjustment mode, and the single variable adjustment mode refers to round-robin of a single parameter. For example, the gain of the camera is transformed, and a group of gain sequences G from low to high is set as { G1, G2, …, GT }; changing the exposure time of the camera, and setting a group of short-to-long exposure time sequences E ═ E1, E2, … and ET; and thirdly, converting the laser intensity of the laser, and setting a group of laser intensity sequences I from weak to strong as { I1, I2, …, IT }.

For example, assuming that the gains of the cameras are G1 and G2, the exposure times of the cameras are E1 and E2, and the laser intensities of the lasers are I1 and I2, the gains of the cameras and the exposure times of the cameras are kept unchanged, and the laser intensities of the lasers are changed, for example, the linear structure light image IMG with the brightness level of 1 is obtained by using G1, E1 and I1 as the linear structure light measurement system₁The linear structured light measurement system adopts G1, E1 and I2 to obtain a linear structured light image IMG with the brightness level of 2₂To this end, when a cycle of imaging parameters is completed, the linear structured light measurement system adopts G1, E1 and I1 again to obtain a linear structured light image IMG with a brightness level of 1₃And so on. Or, the gain of the camera and the laser intensity of the laser are ensured to be unchanged, and the exposure time of the camera is changed. Or, the gain of the camera is changed by ensuring that the exposure time of the camera and the laser intensity of the laser are unchanged.

In another possible embodiment, when the linear structured light measurement system adopts different imaging parameters, the round-robin manner of the imaging parameters may be a multivariable adjustment mode, and the multivariable adjustment mode refers to the round-robin of at least two parameters cooperatively. For example, the gain of the camera and the exposure time of the camera are transformed, or the gain of the camera and the laser intensity of the laser are transformed, or the exposure time of the camera and the laser intensity of the laser are transformed, or the gain of the camera, the exposure time of the camera, the laser intensity of the laser are transformed,

for example, assuming that the gains of the cameras are G1 and G2, the exposure times of the cameras are E1 and E2, the laser intensities of the lasers are I1 and I2, and the linear structured light measurement system adopts G1, E1 and I1, a linear structured light image IMG with brightness level 1 is obtained₁The linear structured light measurement system adopts G1, E1 and I2 to obtain a linear structured light image IMG with the brightness level of 2₂The linear structured light measurement system adopts G1, E2 and I1 to obtain a linear structured light image IMG with the brightness level of 3₃The linear structured light measurement system adopts G1, E2 and I2 to obtain a linear structured light image IMG with the brightness level of 4₄Linear light measuring systemThe system adopted G2, E1 and I1 to obtain a line-structured light image IMG with the brightness level of 5₅The linear structured light measurement system adopts G2, E1 and I2 to obtain a linear structured light image IMG with the brightness level of 6₆The linear structured light measurement system adopts G2, E2 and I1 to obtain a linear structured light image IMG with the brightness level of 7₇The linear structured light measurement system adopts G2, E2 and I2 to obtain a linear structured light image IMG with the brightness level of 8₈To this end, when a cycle of imaging parameters is completed, the linear structured light measurement system adopts G1, E1 and I1 again to obtain a linear structured light image IMG with a brightness level of 1₉And so on.

Step 302, for each initial line structured light image, a light bar centerline of the initial line structured light image is determined, and a reliability index of the light bar centerline is determined based on characteristic information of the light bar centerline.

For example, the light bar centerline may include a plurality of light bar center points, i.e., the plurality of light bar center points constitute the light bar centerline, and the light bar centerline may be a row of light bar centerlines of the initial line structured light image, or the light bar centerline may be a column of light bar centerlines of the initial line structured light image, and the column or row of light bar centerlines may depend on whether the installation position of the camera and the laser is vertical or horizontal, without limitation.

In one possible embodiment, the following steps may be used to determine the light bar centerline, although the following determination is merely exemplary and not limiting as long as the light bar centerline is available.

Step a1, for each dimension of the initial line structured light image, a segmentation threshold is used to segment the pixel points of the dimension into a light bar region and a background region, the gray value of the pixel points in the light bar region is greater than the segmentation threshold, and the gray value of the pixel points in the background region is not greater than the segmentation threshold. Illustratively, each dimension of the initial line structured light image is each row of the initial line structured light image, or each dimension of the initial line structured light image is each column of the initial line structured light image. Regarding whether each dimension of the initial line structured light image is each row or each column, it relates to the direction of the installation position of the camera and the laser, for example, if the installation position of the camera and the laser is a vertical direction (indicating that the line structured light projected onto the surface of the object is a vertical direction), each dimension of the initial line structured light image is each column of the initial line structured light image, and if the installation position of the camera and the laser is a horizontal direction (indicating that the line structured light projected onto the surface of the object is a horizontal direction), each dimension of the initial line structured light image is each row of the initial line structured light image.

For example, if the light bar center line is a line of light bar center lines of the initial line structured light image, each dimension of the initial line structured light image is each column of the initial line structured light image, that is, a light bar center point is obtained for each column, and the light bar center points of each column of the initial line structured light image may form a line of light bar center lines.

If the light bar center line is a row of light bar center lines of the initial line structured light image, each dimension of the initial line structured light image is each row of the initial line structured light image, that is, light bar center points are obtained for each row, and the light bar center points of each row of the initial line structured light image may form a row of light bar center lines.

Illustratively, for any initial line structured light image IMG of N ∈ N_nThe initial line structured light image IMG can be aligned using a segmentation threshold K_nThe pixel point of each dimension is divided, the pixel point area with the gray value larger than K is divided into light bar areas, and the pixel point area with the gray value not larger than K is divided into background areas.

Obviously, the IMG is directed to the initial line structured light image_nMay or may not have one, at least two, or none of the light bar regions. If there is a light bar region in the dimension, the light bar region may include at least one pixel point. In addition, if the dimension does not have the light bar area, the gray values of all the pixels of the dimension are not larger than the segmentation threshold value K.

For example, the value of the segmentation threshold K may be configured according to experience, or the value of the segmentation threshold K may be determined by the tsui method or the histogram method, and the value of the segmentation threshold K is not limited.

Step a2, for each light bar region, determines a center point of the light bar region.

Exemplary, light image IMG for initial line structure_nMay or may not have one, at least two, or none of the light bar regions. If the dimension has the light strip region, determining a central point of the light strip region aiming at each light strip region of the dimension, namely performing central coordinate calculation on the light strip region to obtain the central point of the light strip region, wherein the central point of the light strip region is the central point of all pixel points of the light strip region, and the determination mode is not limited as long as the central point of the light strip region can be obtained. For example, the center point of the light bar region may be determined by using a gray scale centroid method or a geometric center method. Of course, the above algorithm is just two examples.

Illustratively, the light image IMG is structured with an initial line_nIs the initial line structured light image IMG_nFor example, for the initial line structured light image IMG_nIf the gray value of the pixel point of the first row is greater than the segmentation threshold, the pixel point belongs to the light bar area, and if the gray value of the pixel point of the first row is not greater than the segmentation threshold, the pixel point belongs to the background area. And aiming at all the pixel points with the gray values larger than the segmentation threshold, forming adjacent pixel points into the same light strip area, and forming nonadjacent pixel points into different light strip areas. For example, the 1 st to 5 th pixel points in the first column belong to the light bar region 1, the 6 th to 11 th pixel points belong to the background region, the 12 th to 20 th pixel points belong to the light bar region 2, and so on. For each light bar region (such as the light bar region 1 and the light bar region 2) in the first row, a central point of the light bar region may be determined, for example, the central point of the light bar region 1 is a3 rd pixel point (central point of 1-5 pixel points) in the first row, and the central point of the light bar region 2 is a 16 th pixel point (central point of 12-20 pixel points) in the first row.

A3, for each dimension of the initial line structured light image, if there is a central point in the dimension, determining the central point as the central point of the light bar of the dimension; if the dimension has at least two central points, clustering each central point to obtain the clustering point number of each central point, and determining the central point with the most clustering points as the light strip central point of the dimension. So far, the light bar central point of the dimension can be obtained. Illustratively, when the central point is clustered, the central point communicated with the central point in other dimensions except the dimension is searched by a neighborhood method, and the communicated central point and the total number of the central points are determined as the clustering point number.

Exemplary, light image IMG for initial line structure_nIf there is no light bar region in the dimension, there is no center point in the dimension, that is, there is no light bar center point in the dimension.

IMG for initial line structured light images_nIf a light bar area exists in the dimension, a central point exists in the dimension, and the central point is directly determined as the light bar central point of the dimension.

IMG for initial line structured light images_nIf there are at least two optical stripe regions in the dimension, there are at least two central points in the dimension, and there is a noise central point in the at least two central points, so that the noise central point needs to be removed, and only one central point is reserved as the optical stripe central point of the dimension.

Referring to fig. 4, a schematic diagram of cluster-based center point denoising is shown, which is directed to an initial line structured light image IMG_nThe fifth dimension (i.e., the fifth column) of (1) has a central point p1 (the central point on the upper side of the fifth column) and a central point p2 (the central point on the lower side of the fifth column), and the central point p1 is clustered to obtain the clustering point number of the central point p1, i.e., the number of all central points (i.e., gray pixel points) communicated with the central point p1, including the central point p1 itself, obviously, the clustering point number of the central point p1 is 14; the central point p2 is clustered to obtain the clustering point 6 of the central point p2, that is, the number of all the central points (i.e., gray pixels) communicated with the central point p2, and the central point p2 itself is included. Then, the central point p1 with the largest clustering points is determined as the fifth dimensionThe center point of the light bar. For the tenth dimension (tenth column), a central point p3 (central point on the upper side of the tenth column) and a central point p4 (central point on the lower side of the tenth column) exist, and the central point p3 is clustered to obtain a clustering point 14 of the central point p 3; and clustering the central point p4 to obtain the clustering point 6 of the central point p4, and then determining the central point p3 with the largest clustering point as the light strip central point of the tenth dimension.

The processing procedure for other dimensions is similar, and the description is not repeated here.

After each dimension is processed, the light strip central point of the dimension, the initial line structured light image IMG can be obtained_nThere is at most one light bar center point per dimension, see fig. 4.

For example, when the central point of one dimension is clustered, the central point connected to the central point in other dimensions except the dimension may be searched through a neighborhood method, and the total number of the connected central point and the central point, that is, the number of cluster points of the central point, is counted.

Referring to fig. 4, the pixel points marked in the dark color are extracted center points (the row direction is sub-pixel precision), two or more center points may exist in each column between two dotted lines of the left image, and a right image result is obtained after a cluster point denoising strategy is used to ensure that at most one light bar center point exists in each column.

Step a4, for each dimension of the initial line-structured light image, after obtaining the light bar center points of the dimension, the light bar center line of the initial line-structured light image may be determined based on the light bar center points of all dimensions, that is, the light bar center points of all dimensions are combined into the light bar center line of the initial line-structured light image.

In a possible embodiment, the light bar center line includes a plurality of light bar center points, the reliability indicator of the light bar center line includes a plurality of reliability indicators of the light bar center points, that is, each light bar center point corresponds to one reliability indicator, and the characteristic information of the light bar center line includes characteristic information of the plurality of light bar center points. On the basis, the reliability index of the light bar central line is determined based on the characteristic information of the light bar central line, which may include but is not limited to: and determining the reliability index of the central point of the light strip based on the clustering point number of the central point of the light strip and the gray characteristic value of the central point of the light strip aiming at the central point of each light strip included by the central line of the light strip. The reliability index of the light strip central point is positively correlated with the clustering point number of the light strip central point, and the reliability index of the light strip central point is positively correlated with the gray characteristic value of the light strip central point. Then, the reliability index of the central line of the light bar is determined based on the reliability indexes of the central points of all the light bars of the central line of the light bar, for example, the reliability index of the central line of the light bar may include the reliability indexes of the central points of all the light bars. For example, assuming that the central line of the light bar includes a light bar central point 1 and a light bar central point 2, the reliability index of the light bar central point 1 is determined based on the cluster point number of the light bar central point 1 and the gray characteristic value of the light bar central point 1, and the reliability index of the light bar central point 2 is determined based on the cluster point number of the light bar central point 2 and the gray characteristic value of the light bar central point 2.

For example, the reliability index is used for reliability evaluation of light bar center points, and each light bar center point may correspond to one reliability index. For example, the reliability index of the light bar center point can be determined by the following formula: r is KxF_grayIn the above formula, R is the reliability index of the central point of the optical strip, K is the clustering point number of the central point of the optical strip, and F_grayThe gray characteristic value of the central point of the light bar. The gray feature value may include, but is not limited to, at least one of: contrast, gaussian fit confidence, saturation.

Step 303, for each initial line structured light image, selecting a line structured light image to be fused from the plurality of initial line structured light images, that is, a line structured light image to be fused corresponding to the initial line structured light image.

For example, for a line structure light image to be fused corresponding to the initial line structure light image, the line structure light image to be fused may be the initial line structure light image with a brightness level different from that of the initial line structure light image. Alternatively, the line structured light image to be fused may be a target line structured light image corresponding to an initial line structured light image having a luminance level different from that of the initial line structured light image. Of course, the line structured light image to be fused may also be an initial line structured light image having the same brightness level as that of the initial line structured light image. Alternatively, the line structured light image to be fused may be a target line structured light image corresponding to an initial line structured light image having the same brightness level as the initial line structured light image.

Illustratively, the number of the line structured light images to be fused can be configured empirically, and is not limited to, for example, 1, 2, etc., in this embodiment, the number of the line structured light images to be fused is taken as T.

In one possible embodiment, the line-structured light image sequence IMG ═ IMG₁,IMG₂,…,IMG_T,IMG_T+1,IMG_T+2,…,IMG_2T,…IMG_NFor initial line structured light image IMG_nCan be applied to IMG_nIMG adjacent and containing all brightness levels_n-T/2,…,IMG_n,…,IMG_n+T/2The center line of the light strip is subjected to subsequent fusion processing, thereby IMG_n-T/2,…,IMG_n,…,IMG_n+T/2Middle removing IMG_nAnd taking other initial line structure light images as the line structure light images to be fused. Of course, in practical applications, if IMG is used_nThe number of previous images is less than T/2, then the IMG can be selected_nAnd then supplementing the initial line structure light images until the number of the line structure light images to be fused is T. If IMG_nThe number of subsequent images is less than T/2, then the IMG can be selected_nThe initial line structured light image is supplemented in advance until the number of line structured light images to be fused is T.

In another possible embodiment, the line-structured light image sequence IMG ═ IMG₁,IMG₂,…,IMG_T,IMG_T+1,IMG_T+2,…,IMG_2T,…IMG_NFor initial line structured light image C_i,nCan be paired with C_i,nP adjacent and containing all brightness levels_i＝{D_i,n-T/2,…,D_i,n-1,C_i,n,C_i,n+1,…,C_i,n+T/2The center line of the light bar is subjected to subsequent fusion processing, so that P can be converted into a light bar_i＝{D_i,n-T/2,…,D_i,n-1,C_i,n,C_i,n+1,…,C_i,n+T/2Removing C in_i,nAnd taking the other line structure light images as the line structure light images to be fused.

Exemplary, C_i,nIs IMG ═ IMG₁,IMG₂,…,IMG_T,IMG_T+1,IMG_T+2,…,IMG_2T,…IMG_NAny one of the initial line structured light images, { D } of_i,n-T/2,…,D_i,n-1May be IMG ═ IMG₁,IMG₂,…,IMG_T,IMG_T+1,IMG_T+2,…,IMG_2T,…IMG_NIn C_i,nFor the target line structured light image corresponding to the previous initial line structured light image, the following embodiments may be referred to regarding the manner of obtaining the target line structured light image. { C_i,n+1,…,C_i,n+T/2Is IMG ═ IMG₁,IMG₂,…,IMG_T,IMG_T+1,IMG_T+2,…,IMG_2T,…IMG_NIn C_i,nFollowed by an initial line structured light image.

Obviously, since there are T kinds of imaging parameters in total, that is, there are T kinds of brightness levels, the line-to-be-fused structured light image may include line-to-be-fused structured light images of all brightness levels.

Step 304, for each initial line structured light image, based on the reliability index of the central line of the light stripe of the initial line structured light image and the reliability index of the central line of the light stripe of the line structured light image to be fused (i.e. the line structured light image to be fused corresponding to the initial line structured light image), the central line of the light stripe of the initial line structured light image and the central line of the light stripe of the line structured light image to be fused are fused to obtain the central line of the target light stripe.

Illustratively, the light bar centerline of the initial line structured light image comprises a plurality of light bar center points, and the reliability indicator for the light bar centerline comprises a reliability indicator for the plurality of light bar center points. The light strip central line of the structural light image to be fused comprises a plurality of light strip central points, and the reliability index of the light strip central line comprises the reliability indexes of the light strip central points. Based on the reliability index of the light strip central point of the initial line structured light image and the reliability index of the light strip central point of the line structured light image to be fused, the light strip central point of the initial line structured light image and the light strip central point of the line structured light image to be fused are fused to obtain a target light strip central point, and all the target light strip central points form a target light strip central line corresponding to the initial line structured light image.

In one possible embodiment, the following steps may be adopted to obtain the target light bar centerline, and the following determination is only an example and is not limited as long as the target light bar centerline can be obtained.

Step b1, for each dimension (e.g. row or column) of the initial line structured light image, determining whether the reliability indicator of the central point of the light bar of the dimension in the initial line structured light image is greater than a preset threshold (which may be configured empirically). If so, step b2 may be performed, and if not, step b3 may be performed.

And b2, determining the light bar center point of the dimension in the initial line structured light image as the target light bar center point of the dimension. For example, for a first dimension of the initial line structured light image, if the reliability index of the central point of the first dimension of the light stripe in the initial line structured light image is greater than the preset threshold, the central point of the first dimension of the light stripe is directly determined as the central point of the first dimension of the target light stripe. For the second dimension of the initial line-structured light image, if the reliability index of the central point of the light bar of the second dimension in the initial line-structured light image is not greater than the preset threshold, step b3 and the subsequent steps are executed to obtain the central point of the target light bar of the second dimension.

Step b3, selecting the light strip central points to be fused from the light strip central points of the dimension in all the light images of the line structure to be fused based on the reliability index of the light strip central points of the dimension in the light images of the line structure to be fused.

For example, referring to step 303, the line-structure light image to be fused corresponding to the initial line-structure light image may be an initial line-structure light image with a brightness level different from that of the initial line-structure light image; alternatively, the line-structured light image to be fused corresponding to the initial line-structured light image may also be a target line-structured light image corresponding to the initial line-structured light image with a brightness level different from that of the initial line-structured light image, and on this basis, the step b3 may include, but is not limited to:

for each line-structure light image to be fused, if the line-structure light image to be fused is an initial line-structure light image with a brightness level different from that of the initial line-structure light image, then: and if the reliability index of the light strip central point of the dimension in the light image of the line structure to be fused is greater than a preset threshold value, selecting the light strip central point of the dimension in the light image of the line structure to be fused as the light strip central point to be fused. If the reliability index of the light strip center point of the dimension in the to-be-fused line structure light image is not greater than the preset threshold, the light strip center point of the dimension in the to-be-fused line structure light image can be forbidden to be selected as the light strip center point to be fused.

For each line-structured light image to be fused, if the line-structured light image to be fused is a target line-structured light image corresponding to an initial line-structured light image with a brightness level different from that of the initial line-structured light image, directly selecting the light strip center point of the dimension in the line-structured light image to be fused as the light strip center point to be fused, and not comparing the size relationship between the reliability index and the preset threshold value.

For example, IMG is directed to an initial line structured light image_nSuppose IMG_nCorresponding to T to-be-fused line structured light images, wherein the to-be-fused line structured light images are IMGs_n-T/2,…,IMG_n-1,IMG_n+1…,IMG_n+T/2。

For IMG_nThe k-th dimension of (a), determining the IMG_nWhether the reliability index of the light bar center point of the kth dimension is greater than a preset threshold (the preset threshold is denoted as thread, and the preset threshold can be configured according to experience). If not, IMG_nThe light bar center point of the k-th dimension of (1) is not taken as a standby pointAnd fusing light bar central points. And the number of the first and second groups,

structured light image for each line to be fused (in IMG)_n-T/2For example, the processing of the light image of other line structures to be fused is the same), if the IMG is the same_n-T/2Is with IMG_nDifferent target line structured light images corresponding to other initial line structured light images directly connect the IMG_n-T/2The light strip center point of the k-th dimension is used as the light strip center point to be fused. If IMG_n-T/2Is with IMG_nDifferent other initial line structured light images (i.e., not the target line structured light image), the IMG is further determined_n-T/2Whether the reliability index of the light strip center point of the kth dimension is greater than a preset threshold value. If so, IMG_n-T/2The light strip central point of the k-th dimension is taken as the light strip central point to be fused, if not, the IMG_n-T/2The light strip center point of the k-th dimension of (1) is not used as the light strip center point to be fused.

In summary, the initial line structured light image IMG can be obtained_nThe number of the light stripe center points to be fused in the kth dimension may be multiple. Similarly, IMG can also be obtained_nOf other dimensions, i.e. the IMG can be obtained_nThe light bar center point to be fused for each dimension.

Step b4, for each dimension of the initial line structured light image, after obtaining the central point of the light bar to be fused of the dimension, interpolating the central points of all the light bars to be fused of the dimension to obtain the interpolated central point of the light bar, and determining the interpolated central point of the light bar as the central point of the target light bar of the dimension.

For example, for IMG_nThe k dimension of (2) may be obtained by interpolating the light bar center points to be fused of the k dimension, and determining the interpolated light bar center point as the target light bar center point of the k dimension.

For example, the interpolation method may include, but is not limited to, polynomial interpolation, cubic spline interpolation, and the like. If polynomial interpolation is adopted, polynomial interpolation can be carried out on the central points of all the optical strips to be fused in the dimension to obtain the interpolated optical strip central points, and the polynomial interpolation process is not limited. If cubic spline interpolation is adopted, cubic spline interpolation can be carried out on all the light strip central points to be fused of the dimensionality to obtain interpolated light strip central points, and the cubic spline interpolation process is not limited.

Of course, the above are only a few examples of interpolation manners, and the interpolation is not limited to this, as long as the interpolation can be performed on all the light bar center points to be fused in the dimension, and the interpolated light bar center points are obtained.

For example, considering that the measured target moves in one dimension (e.g., moves at a constant speed), the change of the surface height of the measured target during the movement process may cause the change of the coordinates of the central points of the light bars between adjacent frames, and the coordinates of the central points of the light bars are only related to the height of the object, therefore, a central point motion compensation strategy based on interpolation is designed, that is, the central points of all light bars to be fused in the dimension are interpolated to implement motion compensation.

Step b5, after obtaining the target optical strip center point of each dimension of the initial line structured light image, determining the target optical strip center line of the initial line structured light image based on the target optical strip center points of all dimensions of the initial line structured light image, that is, combining the target optical strip center points of all dimensions of the initial line structured light image to obtain the target optical strip center line of the initial line structured light image.

In a possible implementation manner, for each dimension of the initial line-structured light image, the process of step b2 may be used to obtain the target light bar center point of the dimension, or the processes of step b3 and step b4 may be used to obtain the target light bar center point of the dimension, and after obtaining the target light bar center points of all dimensions of the initial line-structured light image, the target light bar center points of all dimensions of the initial line-structured light image may be combined to obtain the target light bar center line of the initial line-structured light image.

In the above embodiments, the light image IMG is for the initial line structure_nSuppose IMG_nCorresponding T line-to-be-fused structured light images are IMGs_n-T/2,…,IMG_n-1,IMG_n+1…,IMG_n+T/2，IMG_nThe reliability index of the light strip center point of the kth dimension is not more than a preset threshold value, and the IMG in the structured light image of the line to be fused_n-T/2,IMG_n-1,IMG_n+1And IMG_n+T/2The reliability index of the light strip center point of the k dimension is larger than a preset threshold, and the reliability index of the light strip center point of the k dimension of the rest image in the line structured light image to be fused is not larger than the preset threshold, the IMG in the line structured light image is determined_n-T/2,IMG_n-1,IMG_n+1And IMG_n+T/2The light strip center point of the k dimension is selected as the light strip center point to be fused. In the following examples, IMG was performed_n-T/2Is denoted as s1, IMG_n-1Is denoted as s2, IMG_n+1Is denoted as s3, IMG_n+T/2The light bar center point of the k-th dimension is denoted as s 4.

Then, interpolation may be performed on all the light bar center points to be fused in the k-th dimension, that is, interpolation is performed on s1, s2, s3 and s4, so as to obtain an interpolated light bar center point, which is subsequently denoted as s5, and the interpolated light bar center point s5 is determined as the target light bar center point in the k-th dimension.

Furthermore, IMG is performed on the initial line structured light image_nIs processed as described above, so that an initial line structured light image IMG can be obtained_nThen the initial line structure light image IMG can be generated_nThe central points of all the target light bars form the central line of the target light bars.

In the above embodiments, the interpolation of s1, s2, s3 and s4 is involved, for example, the interpolation of s1, s2, s3 and s4 is performed by polynomial interpolation, cubic spline interpolation or the like. Regarding how to interpolate s1, s2, s3 and s4, taking cubic spline interpolation as an example, coordinates (e.g., when the k-th dimension is the k-th column, the coordinates are abscissa values) of a plurality of center points (e.g., s1, s2, s3 and s4) and frame numbers (indicating that the center points are the frames of the T images of the line-to-be-fused structured light, such as the IMG where s1 is located) of a plurality of center points are known_n-T/2Is the IMG of frame 1 image, s2_n-1Is the T/2-1 frame image, and so on), a functional relationship can be fitted (i.e., the functional relationship is fitted by a cubic spline interpolation algorithm), the input of the functional relationship is the frame number of the center point, and the output of the functional relationship is the coordinates of the center point. After the functional relation is obtained, the initial line structured light image can be IMG_nSubstituting the frame number into the function relationship to obtain the initial line structured light image IMG_nThe center point of the bar, i.e., the interpolated point (e.g., s5), is not limited to this interpolation process.

Illustratively, for the ith dimension of the initial line-structured light image, assume that the light bar center point sequence of the ith dimension is C_i＝{C_i,n-T/2,…,C_i,n,…,C_i,n+T/2The reliability index sequence corresponding to the light strip central point of the ith dimension is R_i＝{R_i,n-T/2,…,R_i,n,…,R_i,n+T/2}. Reliability index R based on ith dimension of initial line structured light image_i,nDetermining whether the light strip center point sequence C of the ith dimension is matched with the size of a preset threshold value_iPerforming a fusion treatment if R_i,nIf the value is larger than the preset threshold value, the fusion processing is not carried out, and C is directly reserved_i,nThe central point of the target light bar in the ith dimension if R_i,nIf not greater than the preset threshold, performing fusion processing, i.e. from C_i＝{C_i,n-T/2,…,C_i,n,…,C_i,n+T/2And selecting the light strip central point (namely the central point of the light strip to be fused) which is larger than the preset threshold value, interpolating the selected central point of the light strip to be fused, and taking the interpolated central point of the light strip as the central point of the target light strip of the ith dimension. And obtaining the target light bar central point of the ith dimension of the initial line structured light image.

Step 305, replacing the central line of the optical stripe of the initial line structured light image by the central line of the target optical stripe of the initial line structured light image to obtain a target line structured light image corresponding to the initial line structured light image.

For example, after obtaining a target light bar center line of the initial line structured light image (i.e., coordinates of the target light bar center line), the target light bar center line may be drawn in the initial line structured light image, and the previous light bar center line may be deleted, so that the initial line structured light image marked with the target light bar center line is referred to as a target line structured light image, and thus, a target line structured light image corresponding to the initial line structured light image is obtained.

Step 306, performing three-dimensional reconstruction based on the plurality of target line structured light images corresponding to the plurality of initial line structured light images, that is, performing three-dimensional reconstruction based on the plurality of target line structured light images. For example, the central line of the target light bar in the target line structured light image is converted according to the pre-calibrated sensor parameters to obtain the spatial coordinate of the measured target at the current position, and the three-dimensional reconstruction of the measured target can be realized based on the spatial coordinate.

For example, for each initial line structured light image, steps 302-305 can be performed to obtain a target line structured light image corresponding to the initial line structured light image. After the target line structured light image corresponding to each initial line structured light image is obtained, three-dimensional reconstruction can be performed based on a plurality of target line structured light images. For example, assuming that M initial line structured light images are acquired, each initial line structured light image corresponds to one target line structured light image, and M target line structured light images can be obtained.

For example, the execution sequence is only an example given for convenience of description, and in practical applications, the execution sequence between the steps may also be changed, and the execution sequence is not limited. Moreover, in other embodiments, the steps of the respective methods do not have to be performed in the order shown and described herein, and the methods may include more or less steps than those described herein. Moreover, a single step described in this specification may be broken down into multiple steps for description in other embodiments; multiple steps described in this specification may be combined into a single step in other embodiments.

As can be seen from the above technical solutions, in the embodiments of the present application, multiple initial line structured light images with different brightness levels may be obtained, for each initial line structured light image, a target light bar center line of the initial line structured light image may be obtained based on the fusion of the initial line structured light image and a line structured light image to be fused, a target line structured light image corresponding to the initial line structured light image may be obtained based on the target light bar center line, and then three-dimensional reconstruction may be performed based on the multiple target line structured light images. In the above manner, the target light strip center line is obtained by fusing the plurality of line-structured light images with different brightness levels, so that the light strip center line of the line-structured light image is accurately obtained, and the three-dimensional reconstruction effect is improved. The method for obtaining the light strip center line of the line structure light image does not depend on the image quality of the line structure light image, and even if the image quality of the line structure light image is poor, the light strip center line of the line structure light image can be accurately obtained, so that the three-dimensional reconstruction effect is good, the reliability of the light strip center line is improved, and the three-dimensional reconstruction of the line structure light with high frame rate is favorably realized. The method can improve the reliability of the light absorption material and the light reflection material on the light strip center line, enhance the adaptability of the line structured light measurement system to complex materials, avoid complex operation in the whole processing flow and have higher efficiency.

Based on the same application concept as the method, an image reconstruction apparatus is provided in the embodiment of the present application, and as shown in fig. 5, is a schematic structural diagram of the image reconstruction apparatus, and the apparatus includes:

an obtaining module 51, configured to obtain a plurality of initial line-structured light images with different brightness levels; a determining module 52, configured to determine, for each initial line structured light image, a light bar centerline of the initial line structured light image, and determine a reliability indicator of the light bar centerline based on feature information of the light bar centerline; a selecting module 53, configured to select a line structured light image to be fused from the multiple initial line structured light images; a processing module 54, configured to fuse the light strip centerline of the initial line-structured light image and the light strip centerline of the line-structured light image to be fused based on the reliability index of the light strip centerline of the initial line-structured light image and the reliability index of the light strip centerline of the line-structured light image to be fused, so as to obtain a target light strip centerline; replacing the light strip central line of the initial line structured light image by the target light strip central line to obtain a target line structured light image corresponding to the initial line structured light image; and a reconstructing module 55, configured to perform three-dimensional reconstruction based on a plurality of target line structured light images corresponding to the plurality of initial line structured light images.

For example, when the obtaining module 51 obtains a plurality of initial line-structured light images with different brightness levels, it is specifically configured to: the control line structured light measuring system adopts different imaging parameters to obtain a plurality of initial line structured light images with different brightness levels; the line structured light measuring system at least comprises a laser and a camera, the imaging parameters are parameters for determining the brightness level of an image, and the imaging parameters comprise at least one of the following parameters: gain of the camera, exposure time of the camera, laser intensity of the laser.

For example, the determining module 52 is specifically configured to determine the central line of the light bar of the initial line-structured light image: for each dimensionality of the initial line structured light image, utilizing a segmentation threshold to segment pixel points of the dimensionality into a light bar area and a background area, wherein the gray value of the pixel points in the light bar area is larger than the segmentation threshold, and the gray value of the pixel points in the background area is not larger than the segmentation threshold; wherein each dimension is each row or each column of the initial line structured light image;

determining, for each light bar region, a center point of the light bar region;

Illustratively, the light bar centerline comprises a plurality of light bar center points, the characteristic information of the light bar centerline comprises characteristic information of the plurality of light bar center points; the determining module 52 is specifically configured to determine the reliability indicator of the light bar centerline based on the characteristic information of the light bar centerline:

For example, the processing module 54 is configured to fuse the central line of the light bar of the initial line-structured light image and the central line of the light bar of the line-structured light image to be fused based on the reliability indicator of the central line of the light bar of the initial line-structured light image and the reliability indicator of the central line of the light bar of the line-structured light image to be fused, so as to obtain the target central line of the light bar specifically:

For example, the line structure light image to be fused corresponding to the initial line structure light image is an initial line structure light image with a brightness level different from that of the initial line structure light image; or a target line structured light image corresponding to an initial line structured light image having a luminance level different from that of the initial line structured light image; the processing module 54 is specifically configured to select the light strip center point to be fused from the light strip center points of the dimension in all the light images of the line structure to be fused based on the reliability index of the light strip center point of the dimension in the light image of the line structure to be fused:

For example, the processing module 54 interpolates the central points of all the optical bars to be fused, and when obtaining the interpolated central points of the optical bars, the processing module is specifically configured to: performing polynomial interpolation on the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips; or carrying out cubic spline interpolation on the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips.

Illustratively, the line structured light measuring system at least includes a laser and a camera, and the acquiring module 51 is specifically configured to, when acquiring a plurality of initial line structured light images: in the process that an object to be detected moves along the movement direction, the laser device is used for projecting line structured light to the surface of the object to be detected, and the camera is used for shooting the object to be detected to obtain a plurality of initial line structured light images; the object to be measured horizontally moves under the linear structured light measurement system along the movement direction, and in the movement process of the object to be measured, each position between the starting position and the ending position of the object to be measured sequentially passes through the designated position under the linear structured light measurement system, and the laser projects linear structured light to the designated position.

Based on the same application concept as the method, an image reconstruction apparatus is provided in the embodiment of the present application, and as shown in fig. 6, the image reconstruction apparatus includes: a processor 61 and a machine-readable storage medium 62, the machine-readable storage medium 62 storing machine-executable instructions executable by the processor 61; the processor 61 is configured to execute machine executable instructions to perform the following steps:

for each initial line structured light image, then:

Based on the same application concept as the method, embodiments of the present application further provide a machine-readable storage medium, where several computer instructions are stored, and when the computer instructions are executed by a processor, the image reconstruction method disclosed in the above example of the present application can be implemented.

The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, and the like. For example, the machine-readable storage medium may be: a RAM (random Access Memory), a volatile Memory, a non-volatile Memory, a flash Memory, a storage drive (e.g., a hard drive), a solid state drive, any type of storage disk (e.g., an optical disk, a dvd, etc.), or similar storage medium, or a combination thereof.

The systems, devices, modules or units illustrated in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer, which may take the form of a personal computer, laptop computer, cellular telephone, camera phone, smart phone, personal digital assistant, media player, navigation device, email messaging device, game console, tablet computer, wearable device, or a combination of any of these devices.

For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.

As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

Furthermore, these computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

The above description is only an example of the present application and is not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method of image reconstruction, the method comprising:

for each initial line structured light image, then:

2. The method of claim 1,

the acquiring of a plurality of initial line-structured light images with different brightness levels includes:

3. The method of claim 1,

the determining a light bar centerline of the initial line structured light image comprises:

determining, for each light bar region, a center point of the light bar region;

4. The method of claim 1, wherein the light bar centerline comprises a plurality of light bar center points, the characteristic information for the light bar centerline comprising characteristic information for the plurality of light bar center points; the determining a reliability indicator for the light bar centerline based on the characteristic information of the light bar centerline comprises:

5. The method according to claim 4, wherein the fusing the light bar center line of the initial line-structure light image and the light bar center line of the line-structure light image to be fused based on the reliability indicator of the light bar center line of the initial line-structure light image and the reliability indicator of the light bar center line of the line-structure light image to be fused to obtain a target light bar center line comprises:

6. The method of claim 5,

the line structure light image to be fused corresponding to the initial line structure light image is an initial line structure light image with the brightness level different from that of the initial line structure light image; or a target line structured light image corresponding to an initial line structured light image having a luminance level different from that of the initial line structured light image;

selecting the light strip central points to be fused from the light strip central points of the dimensionality in all the light images of the line structure to be fused based on the reliability index of the light strip central points of the dimensionality in the light images of the line structure to be fused, and the method comprises the following steps:

7. The method of claim 5,

the interpolation of the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips comprises the following steps:

performing polynomial interpolation on the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips; or carrying out cubic spline interpolation on the central points of all the optical strips to be fused to obtain the interpolated central points of the optical strips.

8. The method of any of claims 1-7, wherein the line structured light measurement system includes at least a laser and a camera, and wherein the process of obtaining a plurality of initial line structured light images comprises:

in the process that an object to be detected moves along the movement direction, the laser device is used for projecting line structured light to the surface of the object to be detected, and the camera is used for shooting the object to be detected to obtain a plurality of initial line structured light images; the object to be measured horizontally moves under the linear structured light measurement system along the movement direction, and in the movement process of the object to be measured, each position between the starting position and the ending position of the object to be measured sequentially passes through the designated position under the linear structured light measurement system, and the laser projects linear structured light to the designated position.

9. An image reconstruction apparatus, characterized in that the apparatus comprises:

the processing module is used for fusing the light strip center line of the initial line structure light image and the light strip center line of the line structure light image to be fused based on the reliability index of the light strip center line of the initial line structure light image and the reliability index of the light strip center line of the line structure light image to be fused to obtain a target light strip center line; replacing the light strip central line of the initial line structured light image by the target light strip central line to obtain a target line structured light image corresponding to the initial line structured light image;

and the reconstruction module is used for performing three-dimensional reconstruction on the basis of a plurality of target line structure light images corresponding to the plurality of initial line structure light images.

10. An image reconstruction apparatus, characterized by comprising: a processor and a machine-readable storage medium storing machine-executable instructions executable by the processor;

for each initial line structured light image, then: