CN110857855B

CN110857855B - Image data acquisition method, device and system

Info

Publication number: CN110857855B
Application number: CN201810958863.8A
Authority: CN
Inventors: 杨少鹏; 孙元栋; 李光辉
Original assignee: Hangzhou Hikrobot Technology Co Ltd
Current assignee: Hangzhou Hikrobot Co Ltd
Priority date: 2018-08-22
Filing date: 2018-08-22
Publication date: 2022-09-02
Anticipated expiration: 2038-08-22
Also published as: CN110857855A

Abstract

The embodiment of the invention provides an image data acquisition method, a device and a system, wherein the method comprises the following steps: the projector projects measurement structured light according to different exposure time aiming at an object to be measured; a camera collects a grating image of the projected structured light each time; the electronic equipment determines an effective area with a brightness value meeting a condition in each raster image; and splicing each determined effective area to obtain effective image data. Assuming that grating images are acquired simultaneously for an object A with large surface reflectivity and an object B with small surface reflectivity, because the exposure time of the projector is different, the image area A with clear image A and the image area B with clear image B are spliced, and the definition of the image A and the definition of the image B in the obtained image data are both high.

Description

Image data acquisition method, device and system

Technical Field

The invention relates to the technical field of three-dimensional measurement, in particular to an image data acquisition method, device and system.

Background

The schemes for three-dimensional measurements using structured light generally include: projecting structured light by a projector, wherein the structured light irradiates an object to be measured to form a projection grating; the camera collects the projection grating to obtain a grating image; the computer calculates the measurement data of the object to be measured according to the grating image.

The reflectivity of the surfaces of different materials is different, and if the reflectivity of the surface of the object to be measured is larger, the exposure time of the corresponding projector is shorter, so that a clearer grating image can be obtained; if the reflectivity of the surface of the object to be measured is small, the exposure time of the corresponding projector should be longer to acquire a clearer grating image.

However, if the grating image is acquired for both an object having a large surface reflectance and an object having a small surface reflectance, a clear grating image cannot be obtained.

Disclosure of Invention

The embodiment of the invention aims to provide an image data acquisition method, device and system so as to obtain a clear raster image.

In order to achieve the above object, an embodiment of the present invention provides an image data acquiring method, including:

acquiring N first grating images; wherein N is a positive integer greater than 1, and the first grating image is: the projector projects images corresponding to the measurement structured light according to different exposure time;

for each first raster image, determining a first effective area with a brightness value meeting a condition in the first raster image;

and splicing each determined first effective area to obtain effective image data.

Optionally, the method further includes:

acquiring the N second grating images, wherein the second grating images are as follows: the projector projects images corresponding to the structured light with single brightness value according to the different exposure time;

respectively determining an area with a brightness value meeting the condition in each second raster image as a second effective area;

the determining a first effective area with a brightness value satisfying a condition in the first raster image includes:

and mapping a second effective area in a second raster image which has the same exposure time as the projector corresponding to the first raster image to obtain a first effective area.

Optionally, the determining, as the second effective area, an area in each second raster image whose luminance value satisfies a condition includes:

determining an image to be segmented in the second grating image which is not segmented; the second undivided raster image is: a second raster image in which a second effective area is not determined;

and taking the area of which the brightness value meets the preset condition in the image to be segmented as a second effective area.

Optionally, the determining an image to be segmented in the second grating image that is not segmented includes:

determining the second grating image with the minimum exposure time as an image to be segmented; and/or

Selecting the second grating images which are not divided except the second grating image with the minimum exposure time according to the sequence of the exposure time from small to large; and determining the selected second raster image as an image to be segmented after removing the area corresponding to the second effective area in other second raster images.

Optionally, the step of taking the region of which the brightness value satisfies the preset condition in the image to be segmented as a second effective region includes:

determining a brightness threshold corresponding to the image to be segmented;

and determining the area of which the brightness value is greater than the brightness threshold value in the image to be segmented as a second effective area.

Optionally, the determining a first effective area in the first raster image whose luminance value satisfies a condition includes:

inputting the first raster image to a neural network model obtained by pre-training, and determining a first effective area according to an output result of the neural network model; the neural network model is obtained by taking a grating image sample as input and taking the coordinates of an image area segmented in the grating image sample as supervision and training.

Optionally, the acquiring N first raster images includes:

respectively determining the optimal exposure time corresponding to each object to be measured;

controlling the projector to project the measurement structured light according to the determined optimal exposure time respectively;

and acquiring a grating image corresponding to the measurement structure light projected each time as a first grating image.

Optionally, the respectively determining the optimal exposure time corresponding to each object to be measured includes:

acquiring a plurality of third grating images corresponding to each object to be measured; wherein the third raster image is: the projector sequentially projects images corresponding to the structured light to the object to be measured according to the set exposure time;

determining brightness parameters in the third raster images, wherein the brightness parameters are parameters representing brightness distribution ranges;

and determining the optimal exposure time corresponding to the object to be measured in the set exposure time according to the determined brightness parameter.

Optionally, the determining the brightness parameters in the plurality of third raster images includes:

calculating an effective brightness range of the structured light corresponding to the set initial exposure time as a first brightness parameter;

determining the slope of the brightness value of the third grating image and the brightness value of the structured light corresponding to the third grating image as a second brightness parameter;

determining the optimal exposure time corresponding to the object to be measured in the set exposure time according to the determined brightness parameter, wherein the optimal exposure time comprises the following steps:

judging whether the maximum value of the effective brightness range of the structured light corresponding to the initial exposure time is larger than a first preset threshold value or not;

if the exposure time is larger than the preset initial exposure time, determining the exposure time corresponding to the inflection point of the slope as the optimal exposure time corresponding to the object to be measured.

Optionally, the calculating the effective brightness range of the structured light corresponding to the set initial exposure time includes:

sequentially selecting the brightness values of the structured light to be compared in the projected structured light;

determining the change condition of the brightness value of the grating image corresponding to the brightness value of the structure to be compared under the set initial exposure time as a brightness significance coefficient to be compared;

judging whether the brightness significance coefficient to be compared is larger than a global brightness significance coefficient or not; the global brightness significance coefficient is a ratio of a second brightness difference to a first brightness difference, the first brightness difference is a difference between a maximum brightness value and a minimum brightness value of the structured light, and the second brightness difference is a difference between a raster image brightness value corresponding to the structured light with the maximum brightness value and a raster image brightness value corresponding to the minimum brightness value in the initial exposure time;

and if so, determining that the structured light brightness value to be compared belongs to the structured light effective brightness range in the initial exposure time.

for each object to be measured, acquiring a plurality of fourth grating images corresponding to the object to be measured, wherein the fourth grating images are as follows: the projector sequentially projects images corresponding to the structured light to the object to be measured according to the set exposure time;

determining quality parameters in the fourth grating images, wherein the quality parameters are parameters representing the imaging quality of the grating images;

and determining the projector exposure time corresponding to the fourth grating image with the optimal quality parameter as the optimal exposure time corresponding to the object to be measured.

Optionally, the acquiring multiple fourth grating images corresponding to the object to be measured includes:

acquiring a plurality of groups of fourth grating images corresponding to the object to be measured; the encoding algorithms corresponding to the fourth grating images in each group are the same, and the encoding parameters of the measurement structured light corresponding to the fourth grating images in each group change according to a preset rule;

the determining quality parameters of the plurality of fourth grating images includes:

for each group of fourth grating images, decoding the group of fourth grating images to obtain decoded data; based on the decoded data, a quality parameter of the set of fourth raster images is determined.

Optionally, before determining the first effective region in the first raster image whose luminance value satisfies the condition, the method further includes:

acquiring a plurality of fifth grating images with the same exposure time;

selecting two grating images to be processed from the acquired fifth grating image;

calculating the brightness difference between the two grating images to be processed;

determining a region with the brightness difference larger than a second preset threshold value as a candidate region;

and determining a first effective area with the brightness value meeting the condition in the candidate area of the first raster image.

In order to achieve the above object, an embodiment of the present invention further provides an image data acquiring apparatus, including:

the first acquisition module is used for acquiring N first grating images; wherein N is a positive integer greater than 1, and the first grating image is: the projector projects images corresponding to the measurement structured light according to different exposure time;

the first determining module is used for determining a first effective area with a brightness value meeting a condition in each first raster image;

and the splicing module is used for splicing each determined first effective area to obtain effective image data.

Optionally, the apparatus further comprises:

a second obtaining module, configured to obtain the N second grating images, where the second grating image is: the projector projects images corresponding to the structured light with single brightness value according to the different exposure time;

the second determining module is used for determining an area with a brightness value meeting the condition in each second raster image as a second effective area;

the first determining module is further configured to map a second effective area in a second raster image with the same exposure time as the projector corresponding to the first raster image, so as to obtain the first effective area.

Optionally, the second determining module includes:

the first determining submodule is used for determining an image to be segmented in the second grating image which is not segmented; the second undivided raster image is: a second raster image in which the second effective area is not determined;

and the second determining submodule is used for taking the area of which the brightness value meets the preset condition in the image to be segmented as a second effective area.

Optionally, the first determining submodule is specifically configured to:

Optionally, the second determining submodule is specifically configured to:

determining a brightness threshold corresponding to the image to be segmented; and determining the area of which the brightness value is greater than the brightness threshold value in the image to be segmented as a second effective area.

Optionally, the first determining module is further configured to:

Optionally, the first obtaining module includes:

the third determining submodule is used for respectively determining the optimal exposure time corresponding to each object to be measured;

the control sub-module is used for controlling the projector to project the measurement structured light according to the determined optimal exposure time;

and the first acquisition sub-module is used for acquiring a grating image corresponding to the measurement structured light projected each time as a first grating image.

Optionally, the third determining sub-module includes:

the first acquisition unit is used for acquiring a plurality of third grating images corresponding to each object to be measured; wherein the third raster image is: the projector sequentially projects images corresponding to the structured light to the object to be measured according to the set exposure time;

a first determining unit, configured to determine a luminance parameter in the plurality of third raster images, where the luminance parameter is a parameter indicating a luminance distribution range;

and the second determining unit is used for determining the optimal exposure time corresponding to the object to be measured in the set exposure time according to the determined brightness parameter.

Optionally, the first determining unit is specifically configured to:

calculating an effective brightness range of the structured light corresponding to the set initial exposure time as a first brightness parameter; determining the slope of the brightness value of the third grating image and the brightness value of the structured light corresponding to the third grating image as a second brightness parameter;

the second determining unit is specifically configured to:

judging whether the maximum value of the effective brightness range of the structured light corresponding to the initial exposure time is larger than a first preset threshold value or not; if the exposure time is larger than the preset initial exposure time, determining the exposure time corresponding to the inflection point of the slope as the optimal exposure time corresponding to the object to be measured.

Optionally, the first determining unit is further configured to:

Optionally, the third determining sub-module includes:

a second obtaining unit, configured to obtain, for each object to be measured, multiple fourth grating images corresponding to the object to be measured, where the fourth grating images are: the projector sequentially projects images corresponding to the structured light to the object to be measured according to the set exposure time;

a third determining unit, configured to determine quality parameters in the plurality of fourth grating images, where the quality parameters are parameters representing grating image imaging quality;

and the fourth determining unit is used for determining the projector exposure time corresponding to the fourth grating image with the optimal quality parameter as the optimal exposure time corresponding to the object to be measured.

Optionally, the second obtaining unit is specifically configured to:

the third determining unit is specifically configured to:

Optionally, the apparatus further comprises:

the third acquisition module is used for acquiring a plurality of fifth grating images with the same exposure time;

the selection module is used for selecting two grating images to be processed from the acquired fifth grating image;

the calculation module is used for calculating the brightness difference between the two grating images to be processed;

the third determining module is used for determining a region with the brightness difference larger than a second preset threshold value as a candidate region;

the first determining module is further configured to:

In order to achieve the above object, an embodiment of the present invention further provides an electronic device, including a processor and a memory;

a memory for storing a computer program;

and a processor for implementing any one of the above image data acquisition methods when executing the program stored in the memory.

In order to achieve the above object, an embodiment of the present invention further provides a three-dimensional measurement system, including: projectors, cameras and computers;

the projector is used for projecting the measurement structured light according to different exposure time;

the camera is used for carrying out image acquisition on the measurement structure light projected by the projector to obtain a first grating image and sending the first grating image to the computer;

the computer is used for receiving a plurality of first grating images sent by the camera; for each first raster image, determining a first effective area with a brightness value meeting a condition in the first raster image; and splicing each determined first effective area to obtain effective image data.

In the embodiment of the invention, a projector projects measurement structured light according to different exposure time aiming at an object to be measured; a camera collects a grating image of the projected structured light each time; the electronic equipment determines an effective area with a brightness value meeting a condition in each raster image; and splicing each determined effective area to obtain effective image data. Assuming that grating images are acquired simultaneously for an object A with large surface reflectivity and an object B with small surface reflectivity, because the exposure time of the projector is different, the A in a part of the grating images is clear, the B in a part of the grating images is clear, the image area with the clear A and the image area with the clear B are spliced, and the A and B in the obtained image data have higher definition.

Of course, not all of the advantages described above need to be achieved at the same time in the practice of any one product or method of the invention.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flowchart of a first image data acquiring method according to an embodiment of the present invention;

FIGS. 2a-2c are schematic diagrams of grating images at different exposure times according to an embodiment of the present invention;

FIGS. 3a-3b are statistical histograms of luminance distributions provided by embodiments of the present invention;

FIG. 4 is a second flowchart illustrating an image data obtaining method according to an embodiment of the present invention;

fig. 5 is a schematic structural diagram of an image data acquiring apparatus according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

fig. 7 is a schematic structural diagram of an image data acquiring system according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

In order to solve the above technical problems, embodiments of the present invention provide an image data acquisition method, an apparatus, and a system, where the method and the apparatus may be applied to a computer in a three-dimensional measurement system, or applied to a projector, a camera, or other electronic devices, and are not limited specifically. First, a detailed description will be given of an image data acquisition method according to an embodiment of the present invention. For convenience of description, the execution main body is referred to as an electronic apparatus in the following embodiments.

Fig. 1 is a schematic flowchart of a first method for acquiring image data according to an embodiment of the present invention, including:

s101: and acquiring N first raster images.

Wherein N is a positive integer greater than 1, and the first grating image is: the projector projects images corresponding to the measurement structured light according to different exposure times. In this embodiment, for the purpose of distinguishing the description, the raster image used for stitching is referred to as a first raster image.

In one embodiment, the optimal exposure time corresponding to each object to be measured can be determined separately; controlling the projector to project the measurement structured light according to the determined optimal exposure time respectively; and acquiring a grating image corresponding to the measurement structure light projected each time as a first grating image.

For example, assume that there are three objects to be measured A, B and C, where an optimal exposure time corresponding to a is determined to be 20ms, an optimal exposure time corresponding to B is determined to be 60ms, and an optimal exposure time corresponding to C is determined to be 140 ms; the projector can project the measurement structured light three times respectively according to the exposure time of 20ms, 60ms and 140ms, and three first grating images are obtained. Each first raster image comprises the three objects to be measured.

The optimal exposure time corresponding to the object to be measured may be an empirical value, or may be determined in other ways. For example, the projector projects structured light at various exposure times for an object to be measured; a camera collects a corresponding grating image; the electronic device (executing body) selects the image with the highest definition or the most suitable brightness value from the raster images, and the projector exposure time corresponding to the selected image is the optimal exposure time corresponding to the object to be measured.

In one embodiment, determining the optimal exposure time for each object to be measured may include:

acquiring a plurality of third grating images corresponding to each object to be measured; wherein the third raster image is: the projector sequentially projects images corresponding to the structured light to the object to be measured according to the set exposure time; determining brightness parameters in the third raster images, wherein the brightness parameters are parameters representing brightness distribution ranges; and determining the optimal exposure time corresponding to the object to be measured in the set exposure time according to the determined brightness parameter.

For the purpose of distinguishing the description, a raster image acquired during the determination of the optimum exposure time from the luminance parameter is referred to as a third raster image. For example, the brightness parameter may include a slope of the brightness value of the third raster image and the brightness value of the structured light corresponding to the third raster image, so that the exposure time corresponding to the inflection point of the slope may be determined as the optimal exposure time in the set exposure time.

Assuming that the exposure time is t, the response relationship between the brightness value of the third grating image and the brightness value of the structured light is: y ═ k × x + b; where y represents the third raster image brightness value, x represents the structured light brightness value, k represents the slope, and b represents the intercept.

It is understood that there is an optimum value of the exposure time, and if the set initial exposure time is a smaller value, k increases with the increase of the exposure time before the optimum value is reached and k does not increase with the increase of the exposure time after the optimum value is reached in the process of increasing the exposure time on the basis of the initial exposure time. Therefore, the exposure time corresponding to the inflection point of k can be determined as the optimal exposure time.

As another example, the brightness parameter may include an effective brightness range of the structured light corresponding to the set initial exposure time. For convenience of description, the maximum value of the effective luminance range is denoted as S _ max, and the preset threshold value is denoted as T _ max. The maximum value S _ max of the effective luminance range may be compared with a preset threshold value T _ max, which may be a large value, such as 250. If S _ max is not greater than T _ max, it indicates that the raster image is too bright, resulting in annihilation of the portion between S _ max and T _ max. In general, the larger the exposure time, the smaller S _ max.

In one case, the initial exposure time may be a small value; in this case, if S _ max of the structured light corresponding to the initial exposure time is not greater than T _ max, the initial exposure time may be determined as the optimal exposure time.

As described above, if S _ max is not greater than T _ max, it means that the raster image is too bright, resulting in annihilation of the portion between S _ max and T _ max, in which case the exposure time should not be increased; the initial exposure time is already a small value, and the exposure time should not be reduced; therefore, the initial exposure time of the projector can be directly determined as the optimal exposure time.

As another example, the effective brightness range of the structured light corresponding to the set initial exposure time may be calculated as the first brightness parameter; determining the slope of the brightness value of the third grating image and the brightness value of the structured light corresponding to the third grating image as a second brightness parameter; that is, the luminance parameter includes both the slope and the effective luminance range. Thus, whether the maximum value S _ max of the effective brightness range of the structured light corresponding to the initial exposure time is larger than a first preset threshold value T _ max is judged; if the exposure time is larger than the preset initial exposure time, determining the exposure time corresponding to the inflection point of the slope as the optimal exposure time corresponding to the object to be measured.

If S _ max is larger than T _ max, the raster image is not too bright, the exposure time can be continuously increased, and in the process of continuously increasing the exposure time, the exposure time corresponding to the slope inflection point is used as the optimal exposure time corresponding to the object to be measured.

Calculating the effective brightness range of the structured light corresponding to the initial exposure time may include:

determining the change condition of the grating image brightness value corresponding to the structure light brightness value to be compared in the initial exposure time as a brightness significance coefficient to be compared;

For example, a global luma saliency coefficient may be first computed:

assuming that 0 to 255 structured lights are projected in sequence in the initial exposure time, the maximum luminance value of the structured lights is 255 and the minimum luminance value is 0, the first luminance difference is 255, and assuming that the raster image luminance value corresponding to the structured light with the luminance value of 255 is Imax and the raster image luminance value corresponding to the structured light with the luminance value of 0 is Imin in the initial exposure time, the global luminance saliency coefficient is (Imax-Imin)/255.

Alternatively, the global luminance saliency coefficient may be set empirically.

The structured light brightness values of 0-255 may be selected in sequence as the structured light brightness values to be compared. The change situation of the grating image brightness value corresponding to the structure brightness value to be compared is that: and under the initial exposure time, the brightness value of the delta grating image/the brightness value of the delta structured light to be compared. In other words, assuming that a curve (or a straight line) is generated by using the structure light brightness value as the independent variable x and the grating image brightness value as the dependent variable y, the change condition is the slope at the structure light brightness value to be compared.

The brightness value of the structure to be compared is recorded as Scur, the slope at Scur is recorded as eta, eta is the significance coefficient of the brightness to be compared, and eta can be any one of the following:

η ═ I _ cur-I _ low)/(S _ cur-S _ low), or η ═ I _ high-I _ cur)/(S _ high-S _ cur), or η ═ I _ high-I _ low)/(S _ high-S _ low;

where Sdelta is a preset luminance interval, Slow ═ Scur-Sdelta, and Shigh ═ Scur + Sdelta, and the raster image luminance value corresponding to Scur is denoted as Icur, the raster image luminance value corresponding to Slow is denoted as Ilow, and the raster image luminance value corresponding to Shigh is denoted as Ihigh.

Sdelta may be 10, or may be other, and is not particularly limited. Assuming that the preset luminance interval Sdelta is 10, η ═ I _ high-I _ cur)/(S _ high-S _ cur), Scur sequentially increases from 0; scur for the first selection is 0 and Shigh is 10; scur for the second selection is 1, Shigh is 11; the third selection has a Scur of 2 and a Shigh of 12 … …. The following examples are given with Scur of 50 and Shigh of 60:

assuming that, in the initial exposure time, the raster image brightness value corresponding to the structured light S50 with a brightness value of 50 is I50, and the raster image brightness value corresponding to the structured light S60 with a brightness value of 60 is I60, the brightness saliency coefficient to be compared corresponding to Scur is:

η＝(I_60-I_50)/(S_60-S_50)。

and comparing the calculated eta with the global brightness significance coefficient, and if the eta is greater than the global brightness significance coefficient, judging that the Scur belongs to the effective brightness range of the structured light. Similarly, the above processing is performed on each selected structured light brightness value to be compared, so that the effective structured light brightness range can be determined.

For example, the above processing may be performed on each luminance value in ascending order (Scur is 1, Scur is 2 … …) starting from Scur is 0, and when η is smaller than the global luminance saliency coefficient, the last value of Scur may be determined as the minimum value of the structured light effective luminance range; then, starting from Scur-255, the above-mentioned processing is sequentially performed on each luminance value in descending order (Scur-254, Scur-253 … …), and when η is smaller than the global luminance saliency coefficient, the last value of Scur is determined as the maximum value of the structured light effective luminance range; thus, the effective brightness range of the structured light at the initial exposure time is determined.

In another embodiment, determining the optimal exposure time for each object to be measured may include:

aiming at each object to be measured, acquiring a plurality of fourth grating images corresponding to the object to be measured, wherein the fourth grating images are as follows: the projector sequentially projects images corresponding to the structured light to the object to be measured according to the set exposure time; determining quality parameters in the fourth grating images, wherein the quality parameters are parameters representing the imaging quality of the grating images; and determining the projector exposure time corresponding to the fourth grating image with the optimal quality parameter as the optimal exposure time corresponding to the object to be measured.

For the purpose of distinguishing the description, a grating image acquired during the determination of the optimal exposure time according to the quality parameter is referred to as a fourth grating image.

In this embodiment, the acquiring the plurality of fourth grating images corresponding to the object to be measured may include: acquiring a plurality of groups of fourth grating images corresponding to the object to be measured; and the coding algorithms corresponding to the fourth grating images in each group are the same, and the coding parameters of the measurement structured light corresponding to the fourth grating images in each group are changed according to a preset rule.

Determining quality parameters of the plurality of fourth grating images may include: for each group of fourth grating images, decoding the group of fourth grating images to obtain decoded data; based on the decoded data, a quality parameter of the set of fourth raster images is determined.

In brief, structured light is used for carrying out gridding digital coding on a spatial position, for example, stripe structured light is used for carrying out gridding digital coding on a two-dimensional plane, and iPhone X structured light is used for carrying out gridding digital coding on a three-dimensional space. The spatial positions are digitally encoded in a grid, i.e. each discrete spatial position is encoded with a unique value. Correspondingly, the spatial coding value can be decoded through each pixel point of the raster image.

The encoding process can be understood as: projecting structured light for carrying out gridding coding on the spatial position by a projector; the decoding process can be understood as: restoring the space coding value of each pixel point by calculating a group of grating images; the spatial encoding values are in accordance with a distribution rule, and one encoding algorithm may correspond to one distribution rule. Therefore, the proportion of the pixel points which accord with the distribution rule in the decoding data can be counted and used as the quality parameter of the group of grating images. The larger the proportion of the pixel points conforming to the distribution rule is, the better the quality parameter of the group of grating images is.

Taking the exposure time t as an example, the projector may be controlled to project a group of structured light under the condition that the exposure time is t, the group of structured light has the same corresponding encoding algorithm, and the encoding parameter changes according to a preset rule, so as to obtain a group of grating images corresponding to the group of structured light. The coding algorithm of the structured light may be a phase shift method, a binary code method, or a gray code method, etc. If the encoding algorithm is a phase shift method, the encoding parameter may be a phase, that is, the phase of the set of structured light is changed according to a preset rule. If the encoding algorithm is a binary code method or a gray code method, the encoding parameter may be a stripe width or other parameters for changing the structured light pattern, which is not limited specifically.

For example, if the encoding algorithm is a phase-shift method, the structured light may be sine or cosine light. The directional distribution of the structured light may be a transverse distribution, a longitudinal distribution, and the like, and is not particularly limited. In this embodiment, the encoding algorithm and the directional distribution of the same group of structured light are the same, and the encoding parameters of the same group of structured light are different.

For example, the projector may be controlled to project three times of structured light with a longitudinal sinusoidal distribution at each set exposure time; or, the projector can be controlled to project three times of structured light with transverse sinusoidal distribution under each set exposure time; etc., are not to be enumerated. The encoding algorithm, the direction distribution and the encoding parameter change rule of the structured light under each exposure time are the same, so that the subsequent quality parameters are more reasonable.

For another example, the projector may be controlled to project multiple sets of structured light at each set exposure time, for example, three times of structured light with longitudinal sinusoidal distribution (as one set of structured light) and three times of structured light with transverse sinusoidal distribution (as another set of structured light) may be projected, so that one exposure time corresponds to multiple sets of grating images, which is also possible.

Alternatively, the projector may be controlled to project multiple groups of structured light with different coding algorithms at each set exposure time, for example, three times of structured light with longitudinal sinusoidal distribution (as one group of structured light) and three times of structured light coded by a binary code method (as another group of structured light) may be projected, so that one exposure time corresponds to multiple groups of grating images, which is also possible.

The number of raster images per group is not limited. For example, if the encoding algorithm is a phase shift method, 3, 4, or 5 raster images may be included in a group of raster images, and if the encoding algorithm is a gray code method, the number of raster images in each group may be related to the resolution of the raster images, for example, in the case of 1440 × 900, 10 raster images may be included in a group of raster images. The specific values in this example are merely illustrative and do not limit the embodiments of the present invention.

It is understood that the decoded data of the raster image includes a position mark corresponding to each pixel point. There are various decoding methods, such as a decoding method using gray code, a decoding method using phase shift method, etc., which are not listed.

One group of raster images corresponds to one piece of decoding data; as described above, if the projector projects a set of structured light at each set exposure time, one exposure time corresponds to one piece of decoded data; if the projector projects multiple sets of structured light at each set exposure time, one exposure time corresponds to multiple sets of decoded data.

It is to be understood that the encoding algorithm corresponds to the decoding algorithm (or decoding mode). For example, if the adopted coding algorithm is a phase shift method, a decoding mode corresponding to the phase shift method is correspondingly adopted during decoding, in this case, the decoded data is phase data which is periodically distributed transversely or longitudinally (or the distribution rule corresponding to the phase shift method is that the decoded data is periodically distributed transversely or longitudinally), and the larger the proportion of the pixels which are sequentially and normally distributed in the decoded data is, the better the imaging quality of the raster image is.

For another example, if the adopted coding algorithm is a gray code method, a decoding manner corresponding to the gray code method is correspondingly adopted during decoding, in this case, the decoded data is position data distributed in a horizontal sequence or a vertical sequence (or the distribution rule corresponding to the gray code method is that the decoded data is distributed in a horizontal sequence or a vertical sequence), and similarly, the larger the proportion of pixels distributed normally in sequence in the decoded data is, the better the imaging quality of the raster image is.

Alternatively, in the present embodiment, the quality parameter of the raster image may be determined based on the width of the stripe in the raster image.

For example, if structured light is projected according to the encoding algorithm of gray code, the resulting raster image typically includes black stripes and white stripes; moreover, in the case of a bright image, the white stripes are wider than the black stripes; in the case of a dark image, the black stripes are wider than the white stripes. Therefore, the imaging quality can be judged from the change of the width of the stripe, for example, the number of the deviated pixels of each stripe width can be used as the quality parameter, so that the smaller the number of the deviated pixels, the better the quality parameter is represented.

And determining the exposure time of the projector corresponding to the grating image with the optimal quality parameter as the optimal exposure time of the projector.

S102: for each first raster image, a first effective area with a brightness value satisfying a condition is determined in the first raster image.

The brightness value satisfies the condition, which can be understood as a moderate brightness value (neither too bright nor too dark) and a high sharpness. For distinguishing the description, a region satisfying the luminance condition in the first raster image is referred to as a first effective region.

In one embodiment, before S102, N second raster images may be acquired, where the second raster images are: the projector projects images corresponding to the structured light with single brightness value according to the different exposure time; respectively determining an area with a brightness value meeting the condition in each second raster image as a second effective area; thus, S102 is: and mapping a second effective area in a second raster image which corresponds to the first raster image and has the same projector exposure time to the first raster image to obtain a first effective area.

For the purpose of distinguishing descriptions, a raster image corresponding to a single luminance value structured light is referred to as a second raster image, and an area satisfying a luminance condition in the second raster image is referred to as a second effective area.

The exposure time corresponding to the first grating image and the exposure time corresponding to the second grating image are the same set of exposure time. Continuing with the above example, the projector projects the measurement structured light three times for the objects to be measured A, B and C with exposure times of 20ms, 60ms, and 140ms, respectively, resulting in three first grating images each including the three objects to be measured. Correspondingly, the projector projects the structured light with a single brightness value three times for the objects to be measured A, B and C according to the exposure time of 20ms, 60ms and 140ms, respectively, to obtain three second grating images, and each second grating image includes the three objects to be measured.

The single brightness value structured light corresponding to the N second raster images may be the same or different. For example, if N is 3, the single-luminance-value structured light corresponding to the first second raster image may be 250, the single-luminance-value structured light corresponding to the second raster image may be 251, and the single-luminance-value structured light corresponding to the first second raster image may be 252. Or, the three second raster images all correspond to the single-brightness-value structured light of 250. The specific brightness values of the structured light are merely illustrative and do not limit the present invention.

Because the structured light corresponding to the second grating image is a single brightness value, the brightness value of each area of the image caused by different reflectivity of the object to be measured can be reflected more accurately in the second grating image. Further, the second effective region can be determined more accurately in the second raster image.

For example, assume that there are three objects to be measured, the leftmost egg tray, the middle dark cloth, and the rightmost white paper, wherein the white paper has a larger reflectivity, the dark cloth has a second order, and the egg tray has a smaller reflectivity. Assuming that the optimal exposure time corresponding to the object to be measured is determined as described above, the optimal exposure time corresponding to the white paper is determined to be 20ms, the optimal exposure time corresponding to the dark cloth is determined to be 60ms, and the optimal exposure time corresponding to the egg tray is determined to be 140 ms.

Assuming that the projector projects structured light with a single brightness value within an exposure time of 20ms, fig. 2a is obtained, in which the white paper is clearer in fig. 2 a; projecting structured light with a single brightness value by a projector within the exposure time of 60ms to obtain a dark color cloth in the graph 2 b; the projector projects a single intensity value of structured light at an exposure time of 140ms, resulting in fig. 2c, in which the egg tray is clearer.

There are various ways to determine the second active area in the second raster image. For example, a luminance section may be set, and a region in which the luminance value is located in the section may be determined as the second effective region. As shown in fig. 2a-2c, the brightness of the white paper in fig. 2a, the dark cloth in fig. 2b, and the egg tray in fig. 2c are all moderate, so that a brightness interval can be set for the moderate brightness to select a proper effective area.

For another example, the image to be segmented may be determined in the second raster image that is not segmented; the second undivided raster image is: a second raster image in which the second effective area is not determined; and taking the area of which the brightness value meets the preset condition in the image to be segmented as a second effective area.

Wherein determining the image to be segmented in the second raster image that is not segmented may include: determining the second grating image with the minimum exposure time as an image to be segmented; and/or selecting the second grating images which are not divided except the second grating image with the minimum exposure time according to the sequence of the exposure time from small to large; and determining the selected second raster image as an image to be segmented after removing the area corresponding to the second effective area in other second raster images.

The "region whose luminance value satisfies the condition" may be: areas with luminance values greater than a threshold value. The threshold value corresponding to each second raster image may be the same or different, and may be specified by the user, or may be calculated by using a global adaptive algorithm. The global adaptive algorithm is not limited, such as OTSU (the ohs method or the maximum inter-class variance method), Kittler (a global binarization method), and the like.

In one case, regarding a region in the image to be segmented whose luminance value satisfies a preset condition as the second effective region, the method may include: determining a brightness threshold corresponding to the image to be segmented; and determining the area of which the brightness value is greater than the brightness threshold value in the image to be segmented as a second effective area. In this case, the luminance threshold values for different images are different. The brightness threshold corresponding to each image may be specified by the user, or the threshold may be calculated using the various global adaptive algorithms described above.

How to "determine the region whose luminance value satisfies the condition in each second raster image" is described below with reference to fig. 2a to 2c and fig. 3a to 3 b:

the second grating image pattern 2a with the minimum exposure time is determined as the image to be segmented. Referring to fig. 3a, fig. 3a is a luminance distribution statistical histogram of fig. 2 a. In the luminance distribution histogram, the horizontal axis coordinate represents the luminance value, and the vertical axis coordinate represents the number of pixels of the corresponding luminance value. Three wave crests in fig. 3a respectively indicate an area where the egg tray is located, an area where the dark cloth is located, and an area where the white paper is located from left to right (the area where the egg tray is located has the smallest brightness value, the area where the dark cloth is located has the next lowest brightness value, and the area where the white paper is located has the largest brightness value), a threshold corresponding to fig. 2a is set to be 100, that is, an area where the brightness value is greater than 100 is taken as a second effective area, and the second effective area in fig. 2a, that is, the area where the white paper is located is marked as an area a.

Then according to the sequence of the exposure time from small to large, selecting a second grating image which is not divided except the second grating image with the minimum exposure time; and determining the selected second raster image as an image to be segmented after removing the area corresponding to the second effective area in other second raster images.

The process is performed by selecting fig. 2b in the order of increasing exposure time, and the region corresponding to the region a in fig. 2a is removed in fig. 2 b. It can be understood that the positions of the projector, the object to be measured, and the camera are all fixed, and therefore, the positions of the pixel points in the obtained grating image are also unchanged, that is to say, the coordinates of the pixel points in the grating image are all consistent. The coordinates of area a identified in fig. 2a are also applicable to other raster images, and the area corresponding to area a may be removed in fig. 2b based on the coordinates of area a.

And determining the image to be segmented from the image shown in the figure 2b without the area corresponding to the area A, and continuously determining a second effective area in the image to be segmented. Referring to fig. 3B, fig. 3B is the statistical histogram of the luminance distribution in fig. 2B after the area corresponding to the area a is removed, two peaks in fig. 3B respectively indicate, from left to right, the area where the egg tray is located and the area where the dark color cloth is located (the luminance value of the area where the egg tray is located is the smallest, the area where the dark color cloth is located is the next time, and the area where the white paper is located is removed), the threshold value corresponding to fig. 2B is set to 119, that is, the area where the luminance value is greater than 119 is taken as a second effective area, and the second effective area in fig. 2B, that is, the area where the dark color cloth is located is marked as an area B.

The exposure time is selected from fig. 2c in order of increasing exposure time. As described above, the positions of the pixels in the raster image are also unchanged, that is, the coordinates of the pixels in the raster image are consistent, so that the areas corresponding to the area a in fig. 2a and the area B in fig. 2B can be removed from fig. 2 c. The remaining area of fig. 2C after removing these two parts of the area, i.e. the area where the egg tray is located, is denoted as area C.

Similarly, if other second raster images which are not divided exist, selecting the second raster images which are not divided except the second raster image with the minimum exposure time according to the sequence of the exposure time from small to large; and determining the selected second raster image as an image to be segmented after removing the area corresponding to the second effective area determined in other second raster images, and no further description is given.

In this embodiment, an image segmentation algorithm may be used to determine the second effective region in the second raster image, and the imaging quality does not need to be corresponded pixel by pixel, thereby reducing the amount of calculation.

As described above, the coordinates of the pixel points in the raster image are all the same, so that the second effective region can be mapped to the first raster image to obtain the first effective region. As described above, the exposure time corresponding to the first raster image and the exposure time corresponding to the second raster image are the same set of exposure times, and the "mapping" herein refers to coordinate mapping between images having the same exposure time.

For example, the projector projects the measurement structured light according to an exposure time of 20ms to obtain a first grating image, which is denoted as I1; projecting measurement structured light by a projector according to the exposure time of 60ms to obtain a first grating image, and recording as I2; the projector projects measurement structured light with an exposure time of 140ms, resulting in a first grating image, denoted I3. The sequence of the projection of the structured light with single brightness value and the projection of the structured light for measurement by the projector is not limited.

After the coordinates of the a-region are determined in fig. 2a, the coordinates of the a-region are mapped to I1, resulting in a first effective region in I1; after the coordinates of the B region are determined in fig. 2B, the coordinates of the B region are mapped to I2, resulting in a first effective region in I2; after the coordinates of the C region are determined in fig. 2C, the coordinates of the C region are mapped to I3, resulting in the first valid region in I3.

In another embodiment, determining the first effective region in the first raster image, where the luminance value satisfies the condition, may include: inputting a first grating image into a pre-trained neural network model, and determining a first effective area according to an output result of the neural network model; the neural network model is obtained by taking a grating image sample as input and taking the coordinates of an image area segmented in the grating image sample as supervision and training.

For example, a hierarchical structure of the neural network may be preset, and a plurality of raster image samples and a segmentation result of the raster image samples may be obtained; the segmentation result may be the coordinates of the segmented image region in the raster image sample. Inputting the obtained grating image sample into the neural network with a preset structure, taking the segmentation result as supervision, and performing iterative training on the neural network; and obtaining the trained neural network model until the iteration condition is met. The iteration condition may be that the number of iterations reaches a threshold, or that a deviation of the output result from supervision is smaller than a threshold, and the like, and is not limited specifically.

Then, the first raster image can be input into the trained neural network model, the output result of the neural network model can be the coordinate value of the first effective area, and the first effective area is determined from the first raster image according to the output coordinate value.

S103: and splicing each determined first effective area to obtain effective image data.

Continuing with the above example, if the first effective region is determined in I1, I2, and I3, respectively, the first effective region determined in I1, the first effective region determined in I2, and the first effective region determined in I3 are spliced to obtain effective image data.

As described above, the coordinates of the pixel points in the raster image are all consistent, and the image data in the first effective region in I1 can be assigned to the corresponding position in I4, the image data in the first effective region in I2 can be assigned to the corresponding position in I4, and the image data in the first effective region in I3 can be assigned to the corresponding position in I4; i4 is the valid image data obtained by stitching.

Alternatively, S103 can also be understood as: and for each first raster image, dividing a first effective area from the first raster image, and splicing the divided first effective areas to obtain effective image data.

In the effective image data, the image data of the three parts of the first effective areas are clear, namely, the three objects to be measured are clear, and the three-dimensional measurement is carried out by utilizing the effective image data, so that the accuracy is high.

In the three-dimensional measurement scheme, if a plurality of grating images are required to be compared, measurement data are obtained according to the comparison result, the embodiment of the invention can be executed for a plurality of times to obtain a plurality of effective image data, one effective image data is a clear grating image, and the measurement data are obtained based on the comparison result of the plurality of effective image data.

For example, if three-dimensional measurement is performed by using a three-step phase shift method, three horizontal-stripe grating images and three vertical-stripe grating images can be acquired, and the measurement data can be obtained by using the comparison results of the six grating images. The three horizontal stripe grating images have different phases, and the three vertical stripe grating images have different phases. Assuming that the three horizontal-stripe grating images are I11, I12, and I13, respectively, and the three vertical-stripe grating images are I21, I22, and I23, respectively, the embodiment shown in fig. 1 is performed for the six grating images, respectively.

Taking I11 as an example, continuing the above example, assuming that there are three objects to be measured, the projector projects measurement structured light with phases corresponding to I11 according to exposure times of 20ms, 60ms, and 140ms to obtain three first grating images, determines first effective regions in the three first grating images, respectively, and concatenates the determined first effective regions to obtain effective image data corresponding to I11. Similarly, effective image data corresponding to I12, I13, I21, I22 and I23 are obtained, that is, 6 parts of effective image data are obtained in total, and the measurement data are obtained based on the 6 parts of effective image data by using a three-step phase shift method.

As an embodiment, before S102, a plurality of fifth raster images with the same exposure time may be acquired; selecting two grating images to be processed from the acquired fifth grating image; calculating the brightness difference between the two grating images to be processed; determining a region with the brightness difference larger than a second preset threshold value as a candidate region; in this case, S102 is: and determining a first effective area with the brightness value meeting the condition in the candidate area of the first raster image.

For the purpose of distinguishing the description, the raster image in the process of determining the candidate region is referred to as a fifth raster image. The "candidate region" in this embodiment may be understood as a non-background region, and it may be understood that a background region exists in the grating image in addition to the object to be measured, and it is not necessary to process pixel points in the background region. In the embodiment, the candidate region in the raster image, that is, the non-background region, is identified first, and only the pixel points of the candidate region are processed, so that the processing efficiency is improved.

For example, a raster image with a higher luminance value and a raster image with a lower luminance value may be selected from the acquired fifth raster image. For example, the raster image with the highest brightness value and the raster image with the lowest brightness value may be selected, and the specific selection manner is not limited. For convenience of description, the two selected raster images are referred to as to-be-processed raster images, the to-be-processed raster image with a higher luminance value is referred to as I10, and the to-be-processed raster image with a lower luminance value is referred to as I20.

As described above, the positions of the pixels in the raster image are not changed, so that I10 corresponds to the pixels in I20 one-to-one. The corresponding pixel points in I10 and I20 are referred to as pixel point pairs, and if the difference between the luminance values of a pair of pixel point pairs is greater than a preset threshold, the pixel point pair is considered to belong to a candidate region.

It can be understood that, when the object to be measured is irradiated with the structured light with different luminance values in the same exposure time, the luminance values of the projection grating regions are greatly different, and the luminance values of the background region are not greatly different, so that the pixel point pairs with the larger luminance value difference belong to the candidate regions.

As described above, the positions of the pixel points in the raster image do not change, and therefore, the positions of the candidate regions do not change. After the candidate region is determined in the to-be-processed raster image, the same position in other raster images may also be determined as the candidate region. Therefore, the first effective region whose luminance value satisfies the condition can be determined in the candidate region of the first raster image, and thus, the effective region is determined not in the entire first raster image but only in the region (candidate region) where the projection raster is located, and the processing efficiency can be improved.

By applying the embodiment of the invention, the projector projects the measurement structured light according to different exposure time aiming at the object to be measured; a camera collects a grating image of the projected structured light each time; the electronic equipment determines an effective area with a brightness value meeting a condition in each raster image; and splicing each determined effective area to obtain effective image data. Assuming that grating images are acquired simultaneously for an object A with large surface reflectivity and an object B with small surface reflectivity, because the exposure time of the projector is different, the A in a part of the grating images is clear, the B in a part of the grating images is clear, the image area with the clear A and the image area with the clear B are spliced, and the A and B in the obtained image data have higher definition.

On the other hand, in the above embodiment, the structured light with a single brightness value is projected to obtain the second grating image, the effective area is determined in the second grating image, and then the determined effective area is mapped to the first grating image corresponding to the measurement structured light; because the structured light has the same brightness value, the second grating image can accurately reflect the different brightness values of all areas of the image caused by the different reflectivity of the object to be measured, and the determined effective area is more accurate.

Fig. 4 is a schematic flowchart of a second method for acquiring image data according to an embodiment of the present invention, including:

s401: and acquiring a plurality of fifth grating images with the same exposure time.

S402: and selecting two raster images to be processed from the acquired fifth raster image.

S403: and calculating the brightness difference between the two raster images to be processed.

S404: and determining the area with the brightness difference larger than a second preset threshold value as a candidate area.

For the purpose of distinguishing the description, the raster image in the process of determining the candidate region is referred to as a fifth raster image. The "candidate region" in the present embodiment may be understood as a non-background region. It can be understood that, a background area exists in the grating image except for the object to be measured, and it is not necessary to process the pixel points in the background area. In the embodiment, the candidate region in the raster image, that is, the non-background region, is identified first, and only the pixel points of the candidate region are processed, so that the processing efficiency is improved.

For example, a raster image with a higher luminance value and a raster image with a lower luminance value may be selected from the acquired fifth raster image. For example, the raster image with the highest brightness value and the raster image with the lowest brightness value may be selected, and the specific selection manner is not limited. For convenience of description, the two selected raster images are referred to as to-be-processed raster images, the to-be-processed raster image with a higher luminance value is denoted as I10, and the to-be-processed raster image with a lower luminance value is denoted as I20.

It can be understood that the positions of the projector, the object to be measured, and the camera are all fixed, and therefore, the positions of the pixel points in the obtained grating image are also unchanged, that is to say, the coordinates of the pixel points in the grating image are all consistent. Therefore, I10 corresponds to the pixels in I20 one by one. The corresponding pixel points in I10 and I20 are referred to as pixel point pairs, and if the difference between the luminance values of a pair of pixel point pairs is greater than a preset threshold, the pixel point pair is considered to belong to a candidate region.

S405: and acquiring N second grating images. Wherein N is a positive integer greater than 1, and the second grating image is: and the projector projects images corresponding to the single-brightness-value structured light according to the different exposure time.

For example, assume that there are three objects to be measured, the leftmost egg tray, the middle dark cloth, and the rightmost white paper, wherein the white paper has a larger reflectivity, the dark cloth has a second order, and the egg tray has a smaller reflectivity. The assumption is that the optimal exposure time corresponding to the white paper is 20ms, the optimal exposure time corresponding to the dark cloth is 60ms, and the optimal exposure time corresponding to the egg tray is 140 ms.

S406: and respectively determining an area with the brightness value meeting the condition in the candidate area of each second raster image as a second effective area.

As described above, the positions of the pixel points in the raster image do not change, and therefore, the positions of the candidate regions do not change. After the candidate region is determined in the to-be-processed raster image, the same position in other raster images may also be determined as the candidate region. The second effective region whose luminance value satisfies the condition may be determined in the candidate region of the second raster image, and thus, the effective region is determined not in the entire second raster image but only in the region (candidate region) where the projection raster is located, and the processing efficiency may be improved.

The brightness value satisfies the condition, which can be understood as a moderate brightness value (neither too bright nor too dark) and a high sharpness. For distinguishing the description, a region satisfying the luminance condition in the second raster image is referred to as a second effective region.

The "region whose luminance value satisfies the condition" may be: regions with luminance values greater than a threshold value. The threshold value corresponding to each second raster image may be the same or different, and may be specified by the user, or may be calculated by using a global adaptive algorithm. The global adaptive algorithm is not limited, such as OTSU (the ohs method or the maximum inter-class variance method), Kittler (a global binarization method), and the like.

How to "determine the regions whose luminance values satisfy the condition in each of the second raster images, respectively" is explained below with reference to fig. 2a to 2c and fig. 3a to 3 b:

the second grating image with the minimum exposure time in fig. 2a is determined as the image to be segmented. Referring to fig. 3a, fig. 3a is a luminance distribution statistical histogram of fig. 2 a. In the luminance distribution histogram, the horizontal axis coordinate represents the luminance value, and the vertical axis coordinate represents the number of pixels of the corresponding luminance value. Three wave crests in fig. 3a respectively indicate an area where the egg tray is located, an area where the dark cloth is located, and an area where the white paper is located from left to right (the area where the egg tray is located has the smallest brightness value, the area where the dark cloth is located has the next lowest brightness value, and the area where the white paper is located has the largest brightness value), a threshold corresponding to fig. 2a is set to be 100, that is, an area where the brightness value is greater than 100 is taken as a second effective area, and the second effective area in fig. 2a, that is, the area where the white paper is located is marked as an area a.

Then according to the sequence of the exposure time from small to large, selecting the second grating image which is not divided except the second grating image with the minimum exposure time; and determining the selected second raster image as an image to be segmented after removing the area corresponding to the second effective area in other second raster images.

In this embodiment, the effective region is determined in the region (candidate region) where the projection grating is located, that is, the background region is removed in fig. 2a, 2b, and 2 c. For fig. 2C, the remaining area of fig. 2C excluding the background area, the area corresponding to the area a, and the area corresponding to the area B may be referred to as the second effective area of fig. 2C, that is, the area where the egg tray is located, and is denoted as the area C.

S407: and acquiring N first raster images. The first raster image is: the projector projects images corresponding to the measurement structured light according to different exposure times.

The exposure time corresponding to the first grating image and the exposure time corresponding to the second grating image are the same set of exposure time. Continuing the example above, assume that there are three objects to be measured, an egg tray, a dark cloth, and a white paper, the optimal exposure time for the white paper is 20ms, the optimal exposure time for the dark cloth is 60ms, and the optimal exposure time for the egg tray is 140 ms. In this case, in S407, the projector projects three times of measurement structured light for the three objects to be measured A, B and C with exposure times of 20ms, 60ms, and 140ms, respectively, to obtain three first grating images.

S408: and mapping a second effective area in a second raster image with the same projector exposure time as that of each first raster image to the first raster image to obtain a first effective area.

S409: and splicing each determined first effective area to obtain effective image data.

Alternatively, S409 may also be understood as: and for each first raster image, dividing a first effective area from the first raster image, and splicing the divided first effective areas to obtain effective image data.

For example, if three-dimensional measurement is performed by using a three-step phase shift method, three horizontal stripe grating images and three vertical stripe grating images can be acquired, and the measurement data can be obtained by using the contrast results of the six grating images. The three horizontal stripe grating images have different phases, and the three vertical stripe grating images have different phases. Assuming that the three horizontal-stripe grating images are I11, I12, and I13, respectively, and the three vertical-stripe grating images are I21, I22, and I23, respectively, the embodiment shown in fig. 1 is performed for the six grating images, respectively.

With the embodiment of fig. 4 of the present invention, on the first hand, the sharpness of each part of the obtained image data is high, and on the second hand, the structured light with a single brightness value is projected to obtain a second grating image, an effective area is determined in the second grating image, and then the determined effective area is mapped to the first grating image corresponding to the measured structured light; because the structured light brightness values are the same, the second grating image can accurately reflect that the brightness values of all areas of the image are different due to different reflectivity of an object to be measured, and the determined effective area is more accurate.

Corresponding to the above method embodiment, an embodiment of the present invention further provides an image data acquiring apparatus, as shown in fig. 5, including:

a first obtaining module 501, configured to obtain N first raster images; wherein N is a positive integer greater than 1, and the first grating image is: the projector projects images corresponding to the measuring structured light according to different exposure time;

a first determining module 502, configured to determine, for each first raster image, a first effective area in the first raster image, where a luminance value satisfies a condition;

a stitching module 503, configured to stitch each determined first effective area to obtain effective image data.

As an embodiment, the apparatus further comprises: a second obtaining module and a second determining module (not shown in the figure), wherein,

the first determining module 502 is further configured to map a second effective area in a second raster image, which has the same projector exposure time as that of the first raster image, to the first raster image, so as to obtain a first effective area.

As an embodiment, the second determining module includes: a first determination submodule and a second determination submodule (not shown in the figure), wherein,

As an embodiment, the first determining submodule is specifically configured to:

As an embodiment, the second determining submodule is specifically configured to:

As an embodiment, the first determining module is further configured to:

As an implementation manner, the first obtaining module 502 includes: a third determination submodule, a control submodule, and a first acquisition submodule (not shown), wherein,

the control submodule is used for controlling the projector to project the measurement structured light according to the determined optimal exposure time;

As an embodiment, the third determining sub-module includes: a first acquisition unit, a first determination unit, and a second determination unit (not shown in the figure), wherein,

As an embodiment, the first determining unit is specifically configured to:

calculating an effective brightness range of the structured light corresponding to the set initial exposure time as a first brightness parameter; determining the slope of the brightness value of the third grating image and the brightness value of the corresponding structured light as a second brightness parameter;

the second determining unit is specifically configured to:

As an embodiment, the first determining unit is further configured to:

As an embodiment, the third determining sub-module includes: a second acquisition unit, a third determination unit, and a fourth determination unit (not shown in the figure), wherein,

a third determining unit, configured to determine quality parameters in the fourth grating images, where the quality parameters are parameters representing grating image imaging quality;

As an embodiment, the second obtaining unit is specifically configured to:

the third determining unit is specifically configured to:

As an embodiment, the apparatus further comprises: a third obtaining module, a selecting module, a calculating module and a third determining module (not shown in the figure), wherein,

the first determining module 502 is further configured to:

With the embodiment of the invention shown in fig. 5, the projector projects measurement structured light according to different exposure times for an object to be measured; a camera collects a grating image of the projected structured light each time; the electronic equipment determines an effective area with a brightness value meeting a condition in each raster image; and splicing each determined effective area to obtain effective image data. Assuming that grating images are acquired simultaneously for an object A with large surface reflectivity and an object B with small surface reflectivity, because the exposure time of the projector is different, the A in a part of the grating images is clear, the B in a part of the grating images is clear, the image area with the clear A and the image area with the clear B are spliced, and the A and B in the obtained image data have higher definition.

An embodiment of the present invention further provides an electronic device, as shown in fig. 6, including a processor 601 and a memory 602,

a memory 602 for storing a computer program;

the processor 601 is configured to implement any of the image data acquisition methods described above when executing the program stored in the memory 602.

The Memory mentioned in the above electronic device may include a Random Access Memory (RAM) or a Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the processor.

The Processor may be a general-purpose Processor, including a Central Processing Unit (CPU), a Network Processor (NP), and the like; but also Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components.

An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the computer program implements any one of the above-mentioned image data acquisition methods.

An embodiment of the present invention further provides a three-dimensional measurement system, as shown in fig. 7, including: projectors, cameras and computers;

The computer may also perform any of the image data acquisition methods described above.

Alternatively, in another embodiment, the camera may determine, for each first raster image, a first effective region in the first raster image whose luminance value satisfies a condition; and splicing each determined first effective area to obtain effective image data. The camera may also perform any of the image data acquisition methods described above.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on the differences from the other embodiments. In particular, the embodiment of the image data acquiring apparatus shown in fig. 5, the embodiment of the electronic device shown in fig. 6, and the embodiment of the image data acquiring system shown in fig. 7 are substantially similar to the embodiment of the image data acquiring method shown in fig. 1 to 4, so that the description is relatively simple, and relevant points can be obtained by referring to the partial description of the embodiment of the image data acquiring method shown in fig. 1 to 4.

The above description is only for the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims

1. An image data acquisition method characterized by comprising:

splicing each determined first effective area to obtain effective image data;

wherein, the acquiring of the N first grating images includes:

acquiring a grating image corresponding to the measurement structured light projected each time as a first grating image;

the method further comprises the following steps:

and mapping a second effective area in a second raster image which corresponds to the first raster image and has the same projector exposure time to the first raster image to obtain a first effective area.

2. The method according to claim 1, wherein the determining, as the second effective area, an area in each second raster image whose luminance value satisfies a condition includes:

determining an image to be segmented in the second grating image which is not segmented; the second undivided raster image is: a second raster image in which the second effective area is not determined;

3. The method according to claim 2, wherein determining an image to be segmented in the second non-segmented raster image comprises:

4. The method according to claim 2, wherein the taking, as the second effective region, a region in the image to be segmented whose luminance value satisfies a preset condition includes:

determining a brightness threshold corresponding to the image to be segmented;

5. The method according to claim 1, wherein determining the first effective area in the first raster image whose luminance values satisfy the condition comprises:

6. The method according to claim 1, wherein the separately determining an optimal exposure time for each object to be measured comprises:

7. The method of claim 6, wherein the determining the brightness parameters in the plurality of third raster images comprises:

8. The method of claim 7, wherein the calculating the effective brightness range of the structured light corresponding to the set initial exposure time comprises:

determining the change condition of the brightness value of the grating image corresponding to the brightness value of the structural light to be compared under the set initial exposure time as a brightness significance coefficient to be compared;

9. The method according to claim 1, wherein the separately determining an optimal exposure time for each object to be measured comprises:

10. The method according to claim 9, wherein the acquiring a plurality of fourth grating images corresponding to the object to be measured comprises:

11. The method according to claim 1, further comprising, before determining the first effective region in the first raster image whose luminance value satisfies the condition:

acquiring a plurality of fifth grating images with the same exposure time;

12. An image data acquisition apparatus characterized by comprising:

the splicing module is used for splicing each determined first effective area to obtain effective image data;

wherein, the first obtaining module comprises:

the first acquisition submodule is used for acquiring a grating image corresponding to the measurement structured light projected each time as a first grating image;

the device further comprises:

13. The apparatus of claim 12, wherein the second determining module comprises:

the first determining submodule is used for determining an image to be segmented in the second grating image which is not segmented; the second undivided raster image is: a second raster image in which a second effective area is not determined;

14. The apparatus of claim 13, wherein the first determining submodule is specifically configured to:

15. The apparatus according to claim 13, wherein the second determining submodule is specifically configured to:

16. The apparatus of claim 12, wherein the first determining module is further configured to:

17. The apparatus of claim 12, wherein the third determining submodule comprises:

18. The apparatus according to claim 17, wherein the first determining unit is specifically configured to:

the second determining unit is specifically configured to:

19. The apparatus of claim 18, wherein the first determining unit is further configured to:

20. The apparatus of claim 12, wherein the third determining submodule comprises:

21. The apparatus according to claim 20, wherein the second obtaining unit is specifically configured to:

the third determining unit is specifically configured to:

22. The apparatus of claim 12, further comprising:

the first determining module is further configured to:

23. A three-dimensional measurement system, comprising: projectors, cameras and computers;

the projector is used for projecting the measurement structured light according to the determined optimal exposure time and projecting the structured light with a single brightness value according to different exposure time;

the camera is used for carrying out image acquisition on the measurement structured light projected by the projector according to the determined optimal exposure time to obtain a first grating image and sending the first grating image to the computer; carrying out image acquisition on the single-brightness-value structured light projected by the projector according to different exposure times to obtain a second grating image, and sending the second grating image to the computer;

the computer is used for determining the optimal exposure time corresponding to each object to be measured; receiving a plurality of first grating images and a plurality of second grating images sent by the camera; respectively determining an area with a brightness value meeting the condition in each second raster image as a second effective area; for each first raster image, mapping a second effective area in a second raster image with the same exposure time of a projector corresponding to the first raster image to obtain a first effective area; and splicing each determined first effective area to obtain effective image data.