WO2019113968A1 - Image content-based structured light projection method , depth detection method and structured light projection apparatus - Google Patents

Abstract

An image content-based structured light projection method , comprising: firstly acquiring an image of an object that is not affected by external light; projecting first structured light, wherein same comprises projecting a first grayscale gradient strip onto a non-edge region of the object, and projecting a second grayscale gradient strip onto an edge region; and acquiring an image of the object after the first structured light is projected. The depth variation of the non-edge region of the object and the depth contour of the edge of the object can be obtained, and the obtained depth image is more accurate, thereby also making it convenient to classify the obtained coded image blocks during calculation. Further disclosed are a depth detection method and a structured light projection apparatus using same. According to the method, very fine structured light can be obtained, the calculation difficulty for the structured light is not great, there are not many projection frames, and the interference caused by external light can be effectively eliminated.

Description

 Projected structured light method based on image content, depth detecting method and structured light projection device

[Technical Field]

The present application relates to the field of image processing and machine vision technology, and in particular, to a projected structured light method and a depth detecting method based on image content.

 【Background technique】

Vision is the most direct and important way of human observation and cognitive world. We live in a three-dimensional world. Human vision can not only sense the brightness, color, texture information, motion of the surface of the object, but also its shape, space and spatial position (depth, distance). How to make machine vision obtain high-precision 3D depth information in real time and improve the intelligence level of the machine is the difficulty of current machine vision system research.

Depth sensing technology and devices In the industrial field, high-resolution, high-precision 3D depth information has a wide range of applications in automotive-assisted safe driving, high-speed machine tool processing, industrial modeling, 3D printing, medical imaging, and IoT 3D visual perception. demand. In the field of consumer electronics, deep sensing technology and devices help to improve the intelligence level and interaction capabilities of electronic products, and bring new human-computer interaction experience to users, enabling innovative applications in smart TVs, smart phones, home appliances, tablet PCs, etc. .

Depth sensing technology can be roughly divided into passive and active. Traditional binocular stereo vision ranging is a passive ranging method, which is greatly affected by ambient light and complicated in stereo matching process. Active ranging methods mainly include structured optical coding and ToF. The active visual mode based on the structured optical coding can acquire the image depth information more accurately. The principle of detecting the depth image by projecting the structured light is as shown in FIG. 1. The structured light projection module 110 projects the structured light and is reflected by the lens. 120 is incident, the CCD photosensitive element 130 detects the reflected light, taking the n-th ray 101 of the structured light as an example. As shown in the figure, the light exit angle a1 is known, the distance d between the reference plane and the lens 120 is known, and the CCD photosensitive element 130 The incident point x of the light reflected by the reference plane is detected, and the incident point x' of the reflected light of the measured object is obtained, so that the incident angle a2 of the reflected light of the measured object entering the lens 120 can be obtained, so that the distance d' of the measured object can be calculated. The above is the basic principle of measuring the depth of the object by the structured light projection, and analyzing the structured light ID (number) of the light bar or the scatter point, and knowing the ID to know the incident angle, the surface of the object corresponding to the reflected light can be calculated according to the triangular principle. The depth (distance) is actually different depending on the structural light used, the lens, the photosensitive element, and the like. In the prior art method of using stripe structured light, the stripe structure light is projected to have a certain width, and the stripe width should be as narrow as possible to capture a finer depth variation. But the thing is that the stripes are denser and denser and harder to distinguish. The general practice is to project multiple frames, from coarse to fine, such as Greencode. However, the detection period is greatly extended and the accuracy is not accurate to the pixel level. There are also some methods of scanning the object to be inspected with the gradient stripe, but it is highly susceptible to external light interference. When there is other background light interference, the analysis of the above ID is easily disturbed, and the depth of the solution is problematic. Since the structured light is often disturbed by the background light, the matching effect of the structured light is greatly reduced.

Therefore, it is necessary to provide a structured light matching projection method and a depth detecting method for reducing background light interference.

 [Summary of the Invention]

The purpose of the present application is to provide an image content-based projection structure light method, a depth detection method, and a structure light projection device that reduce background light interference.

To achieve the above objective, the present application provides a projected structured light method based on image content, including:

Obtain an image of an object that is not affected by external light;

The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

Projecting the first structured light includes projecting a first grayscale gradient strip to the non-edge region of the object and a second grayscale gradient strip to the edge region.

The method for projecting structured light based on image content in the present application first acquires an image of an object that is not affected by external light, and after obtaining an image of the object, obtaining an edge of the object and a non-edge region of the object, projecting a corresponding first grayscale gradient strip and The second grayscale gradient strip. By this method, very fine structured light can be obtained, which is based on image content, is not affected by external light, and the calculation of structured light is not difficult, the number of projected frames is not much, and the interference against external light is also good.

The present application also provides a depth detecting method using the image structure-based projected structured light method, including:

Obtain an image of an object that is not affected by external light;

The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

Projecting the first structured light, including respectively projecting a first grayscale gradient strip to the non-edge region of the object, and projecting a second grayscale gradient strip to the edge region;

Obtaining an object image after projecting the first structured light, and performing depth solution on the object image.

The depth detecting method uses different structured light for the edge of the object and the non-edge region of the object, and can obtain the depth change of the non-edge region of the object and the depth contour of the edge of the object as much as possible, and the obtained depth image is more accurate and can be easily solved. The acquired coded image blocks are classified.

In order to obtain finer structured light, in a preferred embodiment, the analyzing the image of the object further comprises obtaining a highlight region of the image of the object, wherein the projected first structured light, wherein the non-edge region of the object is projected The first grayscale gradient strip includes: a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-highlight region, and the highlight portion structured light projected to the highlight region is projected to the highlight region from the previous time The light is dark.

Here, as long as the highlight portion structured light projected to the highlight region is darker than the light projected to the highlight region in the previous time, the image pixels for the highlight region can be reduced, and the difference may be different in different embodiments.

In a preferred embodiment, the high-light partial structure light projected on the highlight region is darker than the light projected to the highlight region in the previous time, and is less than half the brightness of the light projected from the previous highlight region. Light. For example, if it is detected that the image obtained after the previous projection of the structured light has a highlight region, the brightness of the projected light that projects the highlight portion of the highlight portion again is reduced to 128; for example, the brightness of the light projected to the highlight region for the previous time. At 128, when the acquired image still detects that the pixel brightness is 255, the brightness of the structured light of the highlight portion is reduced to 64, and then continues until the detected image brightness is less than 255, and the projected brightness is no longer lowered. When the structured light is projected again or again, the second gradation gradient strip projected on the edge region of the object and the structural light of the object portion of the first gradation gradient strip projected on the non-edge region of the object may be simultaneously reduced Brightness, or the same as the previous brightness, can be.

In a preferred embodiment, the step of analyzing an image of an object that is not affected by external light to obtain a highlight region of the image of the object includes a highlight overflow detection, and the highlight overflow detection may have multiple criteria for judging The requirements are set.

In some embodiments, the blooming detection includes: determining a gray value of a pixel, where the number n of adjacent pixels having a gray value of 255 is greater than or equal to a preset threshold x, determining the gray value The area where the adjacent pixels of 255 are located is a highlight area. In a particular embodiment, the predetermined threshold x=5, or x=1, or other value.

In other embodiments, the blooming detection includes determining the gray value of the pixel, and the ratio of the number of adjacent pixels n of the gray value to 255 to the total number of pixels of the object image exceeds a preset threshold. y, it is determined that the area where the adjacent pixel of the gray value is 255 is a highlight area. In a specific embodiment, the preset threshold y=5%, or other values.

The present invention adopts the depth detection method of the image structure-based projected structured light method, and analyzes the object image that is not affected by external light, and further includes obtaining an object edge of the object image, a non-edge region of the object, and a highlight region, according to Obtaining the structured image, the structured light includes: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the non-edge region of the object is projected A grayscale gradient strip includes: a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-highlight region, and the highlight portion structured light projected to the highlight region is projected to the highlight region from the previous time. The light is dark. The image of the object obtained by the method greatly reduces the influence of external light, and the constraint is greatly reduced by the edge, the highlight, and the like.

In order to further obtain finer structured light, based on the foregoing embodiments of the structured light method of the present application, a preferred embodiment includes: after the projecting the first structured light, the method further comprises:

Obtaining an object image after projecting the previous frame of structured light,

And analyzing the acquired image of the object after the previous frame of the structured light, determining whether the object image includes a highlight region, and if the image includes a highlight region, projecting the next frame of structured light, and the projected frame of the structured light The method includes: projecting a first grayscale gradient strip on the non-edge region of the object, wherein the first grayscale gradient strip projected on the non-edge region of the object includes a highlight portion structured light projected to the highlight region and an object portion projected to the non-highlight region The structured light, the highlight portion of the light projected onto the highlight region is darker than the light projected to the highlight region from the previous time.

The present application adopts the depth detection method of the projected structure light method based on the image content, and after analyzing the current image, performs strategy adjustment on the projected structured light and the number of projections, and uses the structured light method to project the non-edge region of the object. a first grayscale gradient strip, projecting a second grayscale gradient strip to the edge region, wherein the first grayscale gradient strip projected on the non-edge region of the object includes a highlight portion structured light and a non-highlight region projected to the highlight region The projected object partially structs light, and the highlight portion of the projected light projected onto the highlight region is darker than the light projected to the highlight region from the previous time.

The structured light projection method may also have other implementation manners, such as analyzing the current image. If the edge of the obtained object is large and complicated, the second grayscale gradient strip of the frame is separately projected on the edge of the object, or in the object. The second grayscale gradient strip is projected a few times at the edge, and the obtained multi-frame image is superimposed to solve the depth of the edge.

The number of times the structured light is projected may be determined according to the condition of the previous frame image, and the multi-frame structured light may be projected, and the structured light of each frame is darker than the structure light of the previous frame to obtain a more accurate object image.

In a preferred embodiment, the first grayscale gradient stripe and the second grayscale gradient stripe have different grayscale value regions. For example, the gray value of the second grayscale gradient stripe is 128-256, and the grayscale value of the first grayscale gradient stripe is 0-128, or other different value regions. In a preferred embodiment, the first grayscale gradient strip and the second grayscale gradient strip have different gradient stripe arrangement manners, and the first grayscale gradient strip and the second grayscale gradient strip The gradation of the pixels in the adjacent area is inconsistent. The first grayscale gradient strip and the second grayscale gradient strip can be distinguished from each other, so that the two can be more accurately calculated for the edge of the object and the depth of the surface of the object.

In a preferred embodiment, the analyzing the acquired image of the object and determining whether the highlight image includes the highlight region includes: determining a gray value of the pixel, and selecting a neighboring pixel with a gray value of 255 If the number n is greater than or equal to the preset threshold x, it is determined that the area where the adjacent pixel of the gray value is 255 is a highlight area; or the pixel of the gray value is judged, and the number of adjacent pixels of the gray value is 255 When the ratio of the number n of the total number of pixels of the object image exceeds a preset threshold y, it is determined that the area of the adjacent pixel whose gradation value is 255 is a highlight area.

The image of the object that is not affected by external light includes:

Projecting white light globally to acquire the first frame image;

Does not project any light, and acquires the image of the second frame;

The obtained two frames of images are subtracted to obtain an image of an object that is not affected by external light.

Specifically, the subtracting the acquired two frames of images to obtain an object image that is not affected by external light includes: subtracting the pixel-by-pixel grayscale of the image of the second frame by pixel-by-pixel grayscale The pixel gradation after subtraction.

Wherein, the first grayscale gradient strip is projected on the non-edge region of the object, and the entire strip of the first grayscale gradient strip has no repeating texture along the grayscale gradient direction, and the gray between adjacent pixels along the grayscale gradient direction The degree is different, and the gray level of adjacent pixels increases or decreases along the gray level gradient direction.

The second grayscale gradient strip is projected along an edge of the object, and the width of the second grayscale gradient strip is D and the width spans the edge of the object, and the grayscale gradient direction of the second grayscale gradient stripe and the edge of the object Consistently, there is no repeating texture along the grayscale gradient direction of the entire strip of the second grayscale gradient strip, and the gray scales of adjacent pixels along the grayscale gradient direction are different, and the gray scales of adjacent pixels are increased along the grayscale gradient direction or Decrement. The first grayscale gradient stripe and the second grayscale gradient stripe do not coincide with pixel grayscales in adjacent regions.

The present application is based on a projected structured light method of an image content and a depth detecting method. In other embodiments, the image detecting of the object tracking may be applied, after the first structured light is projected, and the object image of the first structured light is acquired. And projecting the first structured light to the object again, acquiring the image of the object after the first structured light is projected, repeatedly projecting the first structured light, acquiring a plurality of the image of the object, and obtaining an image continuously tracked by the object.

The application also provides a structured light projection device, comprising:

a processor adapted to implement each instruction;

A storage device adapted to store a plurality of instructions adapted to be loaded and executed by the processor:

Obtain an image of an object that is not affected by external light;

Obtain an image of an object that is not affected by external light;

The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

Projecting the first structured light includes projecting a first grayscale gradient strip to the non-edge region of the object and a second grayscale gradient strip to the edge region.

The image of the object that is not affected by external light includes:

Projecting white light globally to acquire the first frame image;

Does not project any light, and acquires the image of the second frame;

The obtained two frames of images are subtracted to obtain an image of an object that is not affected by external light.

In some embodiments, the analyzing the image of the object that is not affected by external light, further comprising obtaining a highlight region of the image of the object, wherein the first structured light is projected, wherein the first gray projected on the non-edge region of the object The gradient strip includes a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-high beam region, and the highlight portion structured light projected to the highlight region is darker than the previous projected light to the highlight region .

The highlight portion structured light projected on the highlight region is darker than the light projected to the highlight region in the previous time, and is a light that is projected to be half the brightness of the light projected from the highlight region in the previous time.

The step of analyzing the image of the object that is not affected by the external light to obtain a highlight region of the object image includes a highlight overflow detection, and the highlight overflow detection is to determine the gray value of the pixel, and the gray value is If the number n of adjacent pixels of 255 is greater than or equal to the preset threshold x, it is determined that the area where the adjacent pixel of the gray value is 255 is a highlight area; or the gray value of the pixel is determined, and the gray value is When the ratio of the number n of adjacent pixel points of 255 to the total number of pixel points of the object image exceeds a preset threshold y, it is determined that the area of the adjacent pixel whose gradation value is 255 is a highlight area.

In some embodiments, the instructions stored by the storage device further include after projecting the first structured light:

Obtaining an object image after projecting the previous frame of structured light,

And analyzing the acquired image of the object after the previous frame of the structured light, determining whether the object image includes a highlight region, and if the image includes a highlight region, projecting the next frame of structured light, and the projected frame of the structured light The method includes: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale gradient strip projected on the non-edge region of the object includes a highlight projected on the highlight region Partially structured light and part of the structured light projected onto the non-highlighted region, the highlight portion of the structured light projected onto the highlight region is darker than the light projected to the highlight region from the previous time.

The step of determining whether the object image includes the highlight region in the object image after the acquired object image of the previous frame structured light includes: determining the gray value of the pixel, and the gray value is 255. If the number n of adjacent pixels is greater than or equal to the preset threshold x, it is determined that the area where the adjacent pixel of the gray value is 255 is a highlight area; or the gray value of the pixel is judged, and the gray value is 255. When the ratio of the number of adjacent pixels to the total number of pixels of the object image exceeds a preset threshold y, it is determined that the region where the adjacent pixel of the gradation value is 255 is a highlight region.

The subtracting the acquired two frames of images to obtain an object image that is not affected by external light includes: subtracting the pixel-by-pixel grayscale of the first frame image from the pixel-by-pixel grayscale of the second frame image to obtain a phase Subtracted pixel grayscale.

Wherein, the first grayscale gradient strip is projected on the non-edge region of the object, and the entire strip of the first grayscale gradient strip has no repeating texture along the grayscale gradient direction, and the gray between adjacent pixels along the grayscale gradient direction The degree is different, and the gray level of adjacent pixels increases or decreases along the gray level gradient direction.

Wherein the second grayscale gradient strip is projected along the edge of the object, and the width of the second grayscale gradient strip is D and the width spans the edge of the object, and the grayscale gradient direction of the second grayscale gradient strip is The edges of the object are consistent, and the entire strip of the second grayscale gradient strip has no repeating texture along the grayscale gradient direction, and the grayscale between adjacent pixels along the grayscale gradient direction is different, and the adjacent pixel grayscale along the grayscale gradient direction Increment or decrement.

The first grayscale gradient stripe and the second grayscale gradient stripe have different gradient stripe arrangement manners, and the first grayscale gradient stripe is adjacent to the second grayscale gradient stripe The pixel gradation of the area is inconsistent.

Advantageous Effects: Different from the prior art, the present application adopts the image structure-based projection structure light method and the depth detection method, firstly acquiring an object image that is not affected by external light, and differently using the object edge from the non-edge area of the object. The structured light, after projecting the first structured light, acquires an image of the object after projecting the first structured light, and obtains a depth change of the non-edge region of the object and a depth profile of the edge of the object as much as possible, and the obtained depth image is more accurate. It is also convenient to classify the obtained coded image blocks when solving. The method of the present application can obtain very fine structured light, and the calculation of the structured light is not difficult, and the number of projection frames is not much, which can effectively eliminate the interference of external light.

 [Description of the Drawings]

Figure 1 is a basic schematic diagram of depth detection;

2 is a schematic diagram of an embodiment of a method for projecting structured light based on image content of the present application;

FIG. 3 is a schematic diagram of the comparison between the projected light and the image obtained after light transmission in the present application.

【Detailed ways】

To enable those skilled in the art to better understand the technical solutions of the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

Referring to FIG. 2 and FIG. 3, the present application is based on an image structure of a projected structure light method embodiment, including:

(1) Projecting white light globally to acquire the image of the first frame;

(2) not projecting any light, acquiring the image of the second frame;

(3) subtracting the acquired two frames of images to obtain an image of the object that is not affected by external light;

(4) analyzing the image of the object to obtain an edge of the object, a non-edge region of the object, and a highlight region;

(5) projecting the first structured light, comprising: respectively projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale projected on the non-edge region of the object The gradient strip includes: a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-high beam region, and the highlight portion structured light projected to the highlight region is darker than the previous projected light to the highlight region . Here, the high-light portion structured light projected to the highlight region is darker than the global projected white light, that is, darker than the brightness of the previous projected light.

Since the light of the object in the highlight area is highlighted, the brightness of the highlight area has reached the highest level. If the light of the same intensity as the previous time is continued, only the same highlight result is detected, and the actual depth image of the object cannot be obtained. The high-light partial structure light projected on the highlight region is darker than the brightness of the previous projection light to obtain a more accurate depth image after projecting the structured light, and can effectively eliminate interference from external light.

In a specific embodiment, a high-light partial structure light of 128 luminance is projected on the highlight region for a pixel point having a grayscale value of 255 in the highlight region.

The step of subtracting the acquired two frames of images to obtain an object image that is not affected by the light includes: subtracting the pixel-by-pixel grayscale I2 (x, y) of the second frame image by the pixel-by-pixel grayscale of the first frame image I1(x,y)=I2(x,y) – I1(x, y), the pixel gradation after subtraction is obtained.

The edge detection uses a Canny edge detection algorithm, referred to as the Canny algorithm. The Canny algorithm is John F. Canny. 1986 A multi-level edge detection algorithm developed in the year. In general, the purpose of edge detection is to significantly reduce the data size of an image while preserving the original image attributes. There are a variety of algorithms for edge detection. Although the Canny algorithm is a long-standing method, it can be said to be a standard algorithm for edge detection and is still widely used in research. Canny The goal is to find an optimal edge detection algorithm. The meaning of optimal edge detection is:

(1) Optimal detection: the algorithm can identify the actual edges in the image as much as possible, and the probability of missing the true edge and the probability of false detection of the non-edge are as small as possible;

(2) Optimal positioning criterion: the position of the detected edge point is closest to the position of the actual edge point, or the extent to which the detected edge deviates from the true edge of the object due to the influence of noise;

(3) The detection point corresponds one-to-one with the edge point: the edge point detected by the operator should have a one-to-one correspondence with the actual edge point.

In order to meet these requirements, a variational method is used, which is a method of finding a function that optimizes a specific function. The optimal detection is represented by four exponential function terms, but it is very similar to the first derivative of the Gaussian function.

The Canny edge detection algorithm can be divided into the following five steps:

1. Apply Gaussian filtering to smooth the image in order to remove noise;

2. Find the intensity gradient of the image;

3. Apply non-maximum suppression techniques to eliminate edge false detections;

4. Apply a dual threshold method to determine possible boundaries;

5. Use hysteresis techniques to track boundaries.

The step of analyzing the image of the object to obtain an edge of the object and a highlight region includes performing edge detection and highlight overflow detection.

The highlight overflow detection includes: determining a gray value of a pixel, where the number n of adjacent pixel points whose gray value is 255 is greater than or equal to a preset threshold x, determining that the adjacent pixel whose gray value is 255 is located The area is a highlight area. In a specific embodiment, the preset threshold x=5, or x=1, or other values. Alternatively, it is determined whether the pixel gray value is 255, and the ratio of the number n of adjacent pixel points of the gray value to 255 to the total number of pixel points of the object image exceeds a preset threshold y, and if the threshold value y is exceeded, the The area where the adjacent pixel of the gradation value is 255 is the highlight area. In a specific embodiment, the preset threshold y=5%, or other values.

The first grayscale gradient strip and the second grayscale gradient strip adopt a grayscale structured light strip. The first grayscale gradient strip is projected on the non-edge region of the object, and the entire strip of the first grayscale gradient strip has no repeating texture along the grayscale gradient direction, that is, in the uniform direction along the grayscale gradient, there is no arbitrary The gray levels of the pixels are the same, and the gray levels between adjacent pixels along the gray level gradient direction are different. This difference may be manifested by increasing or decreasing the gray level of adjacent pixels along the gray level gradient direction.

The second grayscale gradient strip is projected along the edge of the object, and the width of the second grayscale gradient strip is D and the width spans the edge of the object, that is, the second grayscale gradient stripe projected along the edge of the object. Over the edge line; the gray-graded direction of the second gray-graded strip is consistent with the extending direction of the object edge, and the entire strip of the second gray-graded strip has no repeating texture along the gray-graded direction, that is, along the gray In the uniform direction of the gradual gradient, the gradation of any pixel is the same, and the gradation of the adjacent pixels along the gradation direction of the gradation is different. The difference may be represented by the adjacent pixel gray along the gradation direction. The degree is incremented or decremented.

The first grayscale gradient strip is different from the grayscale value range of the second grayscale gradient strip. For example, the gray value of the second grayscale gradient stripe is 128-256, and the grayscale value of the first grayscale gradient stripe is 0-128, or other different value regions. In a preferred embodiment, the first grayscale gradient strip and the second grayscale gradient strip have different gradient stripe arrangement manners, and the first grayscale gradient strip and the second grayscale gradient strip The gradation of the pixels in the adjacent area is inconsistent. The first grayscale gradient strip and the second grayscale gradient strip can be distinguished from each other, so that the two can be more accurately calculated for the edge of the object and the depth of the surface of the object.

When the present application is applied to object tracking detection, the above step (5) may be repeated, and the first structured light is projected a plurality of times to obtain a plurality of images of the object at different time points.

The application is applied to perform high-precision depth detection on an object, and further optimize the structured light on the basis of the above step (5), and then project the structured light again, further acquire an image, obtain a plurality of images, and perform superposition operation. Get the depth of the object. The number of times the structured light is projected can be judged based on the image obtained by projecting the structured light from the previous time, and the specific implementation is as follows.

Specifically, after the first structured light is projected in the above step (5), acquiring the third frame image, and further comprising determining whether to project the second structured light according to the third frame image to acquire the fourth frame image.

Determining whether to project the second structured light to acquire the fourth frame image according to the third frame image includes determining whether the highlight image is included in the third frame image, and if the third frame image includes a highlight region, projecting a second structured light, the projected second structured light comprising: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale projected on the non-edge region of the object The gradient strip includes a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-high beam region, and the highlight portion structured light projected to the highlight region is projected to the highlight region in the first structured light The intensity of the structured light in the highlight portion is dark. In a specific embodiment, for a pixel point in the image of the third frame where the gray value of the highlight region is equal to 255, the highlight portion of the second structured light structures the light of 128 brightness.

Determining whether the highlight region is included in the image of the third frame includes determining whether there is a pixel having a gray value of 255 in the image of the third frame, and determining whether the number n of pixels having a gray value of 255 is greater than 1, To include a highlight region; or to determine that if the number n of adjacent pixel points whose gray value is 255 is greater than or equal to a preset threshold x, it is determined that the region where the adjacent pixel of the gray value is 255 is a highlight region. In a specific embodiment, the preset threshold x=5, or x=1, or other values. Alternatively, it is determined whether the ratio of the number of adjacent pixel points n of the RGB value in the image of the third frame to the total number of pixel points of the object image exceeds a preset threshold y, and if the threshold value y is exceeded, the gray value is determined. The area where the adjacent pixels of 255 are located is a highlight area. In a specific embodiment, the preset threshold y=5%, or other values.

Further, the projecting the second structured light further includes determining, according to the fourth frame image, whether to project the third structured light to acquire the fifth frame image.

Determining whether to project the third structured light to acquire the fifth frame image according to the fourth frame image includes determining whether the highlight image is included in the fourth frame image, and if the fourth frame image includes a highlight region, projecting a third structured light, the projected third structured light comprising: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale projected on the non-edge region of the object The gradient strip includes a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-high beam region, and the highlight portion structured light projected to the highlight region is projected to the highlight region in the second structured light The intensity of the structured light in the highlight portion is dark. In a specific embodiment, for the pixel point of the fourth frame image in which the highlight value is equal to 255, the highlight portion of the third structured light structures the light of 64 brightness.

Determining whether the highlight region is included in the image of the fourth frame includes determining whether there is a pixel having a gray value of 255 in the image of the fourth frame, and determining whether the number n of pixels having a gray value of 255 is greater than 1, To include a highlight region; or to determine that if the number n of adjacent pixel points whose gray value is 255 is greater than or equal to a preset threshold x, it is determined that the region where the adjacent pixel of the gray value is 255 is a highlight region. In a specific embodiment, the preset threshold x=5, or x=1, or other values. Alternatively, it is determined whether the ratio of the number of adjacent pixel points n of the RGB value in the fourth frame image to the total number of pixel points of the object image exceeds a preset threshold y, and if the threshold value y is exceeded, the gray value is determined. The area where the adjacent pixels of 255 are located is a highlight area. In a specific embodiment, the preset threshold y=5%, or other values.

In this way, until the highlight region is no longer detected in the acquired image, the projected light intensity of the projected highlight portion can be no longer reduced.

The method of the present application can obtain very fine structured light, and the calculation of the structured light is not difficult; the number of projected frames is not much; the interference against external light is also good.

The scope of application of the present application is that the projection light source is controllable (ie, any pattern can be controlled to be output, and most of the structured light sensors adopt this solution; the projection light source is generally a general projector such as a laser projector or a DMD).

The application also provides a structured light projection device, comprising:

a processor adapted to implement each instruction;

A storage device adapted to store a plurality of instructions adapted to be loaded and executed by the processor:

Obtain an image of an object that is not affected by external light;

Obtain an image of an object that is not affected by external light;

The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

Projecting the first structured light includes projecting a first grayscale gradient strip to the non-edge region of the object and a second grayscale gradient strip to the edge region.

The application also provides a depth detecting device, comprising:

a processor adapted to implement each instruction;

A storage device adapted to store a plurality of instructions adapted to be loaded and executed by the processor:

Obtain an image of an object that is not affected by external light;

The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

Projecting the first structured light, including respectively projecting a first grayscale gradient strip to the non-edge region of the object, and projecting a second grayscale gradient strip to the edge region;

Obtaining an object image after projecting the first structured light, and performing depth solution on the object image.

The above is only the embodiment of the present application, and thus does not limit the scope of patents of the present application, and the equivalent structure or equivalent process transformation made by using the specification and the contents of the drawings, or directly or indirectly applied to other related technical fields, The same applies to the scope of patent protection of this application.

Claims (31)

1. A method for projecting structured light based on image content, comprising:

   Obtain an image of an object that is not affected by external light;

   The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

   Projecting the first structured light includes projecting a first grayscale gradient strip to the non-edge region of the object and a second grayscale gradient strip to the edge region.
2. The image structure-based projection structure light method according to claim 1, wherein the acquiring an object image that is not affected by external light comprises:

   Projecting white light globally to acquire the first frame image;

   Does not project any light, and acquires the image of the second frame;

   The obtained two frames of images are subtracted to obtain an image of an object that is not affected by external light.
3. The method for projecting structured light based on image content according to claim 1, wherein the analyzing the image of the object that is not affected by external light further comprises obtaining a highlight region of the image of the object, the projection being first Structured light, wherein the first grayscale gradient strip projected onto the non-edge region of the object comprises a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-highlight region, and a highlight portion structure projected to the highlight region The light is darker than the light projected to the highlight region the previous time.
4. The image structure-based projection structure light method according to claim 3, wherein the highlight portion structured light projected to the highlight region is darker than the light projected from the previous highlight region, and is a projection ratio. The light that was projected to the highlight area for the previous time was reduced by half the light.
5. The method for projecting structured light based on image content according to claim 3, wherein the step of analyzing the image of the object that is not affected by external light to obtain a highlight region of the image of the object includes a high-gloss detection. The high light overflow detection is performed by determining the gray value of the pixel, and the number n of adjacent pixel points whose gray value is 255 is greater than or equal to the preset threshold x, and determining the area of the adjacent pixel where the gray value is 255 For the highlight region; or for determining the gray value of the pixel, if the ratio of the number of adjacent pixels of the gray value of 255 to the total number of pixels of the object image exceeds a preset threshold y, the gray scale is determined. The area where the adjacent pixel is 255 is the highlight area.
6. The image structure-based projection structure light method according to any one of claims 3 to 5, wherein the projecting the first structured light further comprises:

   Obtaining an object image after projecting the previous frame of structured light,

   And analyzing the acquired image of the object after the previous frame of the structured light, determining whether the object image includes a highlight region, and if the image includes a highlight region, projecting the next frame of structured light, and the projected frame of the structured light The method includes: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale gradient strip projected on the non-edge region of the object includes a highlight projected on the highlight region Partially structured light and part of the structured light projected onto the non-highlighted region, the highlight portion of the structured light projected onto the highlight region is darker than the light projected to the highlight region from the previous time.
7. The method for projecting structured light based on image content according to any one of claims 1 to 5, wherein the grayscale value of the first grayscale gradient strip and the second grayscale gradient stripe are The area is different.
8. The image structure-based projection structure light method according to claim 6, wherein the step of analyzing the acquired object image after the previous frame structured light is acquired, and determining whether the object image includes a highlight region The method includes: determining the gray value of the pixel, and determining the number of adjacent pixels of the gray value of 255 to be greater than or equal to the preset threshold x, determining that the area of the adjacent pixel whose gray value is 255 is a highlight area Or determining the gray value of the pixel, and the ratio of the number of adjacent pixels of the gray value of 255 to the total number of pixel points of the object image exceeds a preset threshold y, and the gray value is determined to be 255. The area where the adjacent pixels are located is a highlight area.
9. The image structure-based projection structure light method according to claim 2, wherein the subtracting the acquired two frames of images to obtain an object image that is not affected by external light includes: The pixel-by-pixel gradation in the image of the first frame is subtracted pixel by pixel, and the pixel gradation after subtraction is obtained.
10. The method for projecting structured light based on image content according to any one of claims 1 to 5, wherein the first gradation gradient strip is projected on a non-edge region of the object, and the whole of the first gradation gradient strip There is no repeating texture in the grayscale gradient direction along the stripe, and the gray scales between adjacent pixels along the grayscale gradient direction are different, and the gray scales of adjacent pixels are increased or decreased along the grayscale gradient direction.
11. The method for projecting structured light based on image content according to any one of claims 1 to 5, wherein the second grayscale gradient strip is projected along an edge of the object, and the width of the second grayscale gradient strip is D and width across the edge of the object, the grayscale gradient direction of the second grayscale gradient strip is consistent with the edge of the object, and the entire strip of the second grayscale gradient strip has no repeating texture along the grayscale gradient direction, along the gray The gray level of adjacent pixels in the gradient direction is different, and the gray level of adjacent pixels is increased or decreased along the gray level gradient direction.
12. The image structure-based projection structure light method according to any one of claims 1 to 5, wherein the first grayscale gradient stripe and the second grayscale gradient stripe have different gradient stripe arrangement manners. The first grayscale gradient stripe and the second grayscale gradient stripe are inconsistent with pixel grayscales in adjacent regions.
13. A depth detecting method, comprising:

    Obtain an image of an object that is not affected by external light;

    The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

    Projecting the first structured light, including respectively projecting a first grayscale gradient strip to the non-edge region of the object, and projecting a second grayscale gradient strip to the edge region;

    Obtaining an object image after projecting the first structured light, and performing depth solution on the object image.
14. The depth detecting method according to claim 13, wherein the acquiring an image of an object that is not affected by external light comprises:

    Projecting white light globally to acquire the first frame image;

    Does not project any light, and acquires the image of the second frame;

    The obtained two frames of images are subtracted to obtain an image of an object that is not affected by external light.
15. The depth detecting method according to claim 14, wherein the analyzing the object image further comprises obtaining a highlight region of the object image, wherein the first structured light is projected, wherein the first object is projected on the non-edge region The grayscale gradient strip includes: a highlight portion structured light projected to the highlight region and a portion of the structured light projected to the non-highlight region, and the highlight portion projected light projected to the highlight region is light projected from the previous highlight region. Be dark.
16. The depth detecting method according to claim 15, wherein the projecting the first structured light to obtain an image of the object after the first structured light is projected, further comprising:

    For analyzing an image of the object acquired after projecting the previous frame of structured light, determining whether the object image includes a highlight region, and if the image includes a highlight region, projecting the next frame of structured light, and the projected frame is structured light. The method includes: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale gradient strip projected on the non-edge region of the object includes a highlight portion projected on the highlight region The structured light and the portion of the structured light projected onto the non-highlight region, the highlight portion of the structured light projected to the highlight region is darker than the light projected to the highlight region from the previous time. .
17. The depth detecting method according to claim 15 or 16, wherein the step of obtaining a highlight region of the object image comprises a highlight overflow detection, the highlight overflow detection comprising: determining a grayscale value of the pixel, and grayscale If the number n of adjacent pixel points with a value of 255 is greater than or equal to the preset threshold x, it is determined that the region where the adjacent pixel of the gray value is 255 is a highlight region; or the gray value of the pixel is determined, and the gray scale is determined. When the ratio of the number n of adjacent pixel points having the value of 255 to the total number of pixel points of the object image exceeds a preset threshold y, it is determined that the area of the adjacent pixel whose gradation value is 255 is a highlight area.
18. The depth detecting method according to any one of claims 13 to 15, wherein the first grayscale gradation strip is projected on a non-edge region of the object, and the entire gradation of the first gradation gradient strip is grayscaled. There is no repeating texture in the gradient direction, and the gray level between adjacent pixels along the gray level gradient direction is different, and the gray level of adjacent pixels is increased or decreased along the gray level gradient direction.
19. The depth detecting method according to claim 13 or 14, wherein the second gradation gradation strip is projected along an edge of the object, and the width of the second gradation gradation strip is D and the width spans the edge of the object. The grayscale gradient direction of the second grayscale gradient strip is consistent with the edge of the object, and the entire strip of the second grayscale gradient strip has no repeating texture along the grayscale gradient direction, and the adjacent pixels are grayed along the grayscale gradient direction. The degree is different, and the gray level of adjacent pixels increases or decreases along the gray level gradient direction.
20. The depth detecting method according to claim 13 or 14, wherein the first grayscale gradient strip and the second grayscale gradient strip have different gradient stripe arrangement manners, and the first grayscale gradient The stripe and the second grayscale gradient stripe are inconsistent with pixel gray levels in adjacent regions, and the first grayscale gradient stripe is different from the grayscale value stripping region of the second grayscale gradient stripe .
21. A structured light projection device, comprising:

    a processor adapted to implement each instruction;

    A storage device adapted to store a plurality of instructions adapted to be loaded and executed by the processor:

    Obtain an image of an object that is not affected by external light;

    Obtain an image of an object that is not affected by external light;

    The image of the object that is not affected by external light is analyzed to obtain an edge of the object and a non-edge area of the object;

    Projecting the first structured light includes projecting a first grayscale gradient strip to the non-edge region of the object and a second grayscale gradient strip to the edge region.
22. The structured light projection device according to claim 21, wherein the acquiring an image of an object that is not affected by external light comprises:

    Projecting white light globally to acquire the first frame image;

    Does not project any light, and acquires the image of the second frame;

    The obtained two frames of images are subtracted to obtain an image of an object that is not affected by external light.
23. The structured light projection device according to claim 21, wherein the analyzing the image of the object that is not affected by external light further comprises obtaining a highlight region of the image of the object, wherein the first structured light is projected, wherein The first grayscale gradation strip projected on the non-edge region of the object includes a highlight portion structured light projected to the highlight region and an object portion structured light projected to the non-highlight region, and the highlight portion structured light projected to the highlight region is prior to the previous time The light projected to the highlight area is dark.
24. The structured light projection device according to claim 23, wherein the highlight portion of the light projected onto the highlight region is darker than the light projected from the highlight region in the previous time, and is a projection ratio of the previous time. The light emitted by the highlight area is reduced by half the light.
25. The structured light projection device according to claim 23, wherein the step of analyzing the image of the object that is not affected by external light to obtain a highlight region of the image of the object includes highlight overflow detection, the highlight overflow detection For determining the gray value of the pixel, if the number n of adjacent pixels of the gray value is 255 is greater than or equal to the preset threshold x, determining that the area of the adjacent pixel whose gray value is 255 is a highlight area; Or determining the gray value of the pixel, if the ratio of the number of adjacent pixels of the gray value of 255 to the total number of pixels of the object image exceeds a preset threshold y, the gray value is determined to be 255. The area where adjacent pixels are located is a highlight area.
26. The structured light projection device according to any one of claims 23 to 25, wherein the instruction stored by the storage device further comprises: after projecting the first structured light:

    Obtaining an object image after projecting the previous frame of structured light,

    And analyzing the acquired image of the object after the previous frame of the structured light, determining whether the object image includes a highlight region, and if the image includes a highlight region, projecting the next frame of structured light, and the projected frame of the structured light The method includes: projecting a first grayscale gradient strip on the non-edge region of the object, and projecting a second grayscale gradient strip on the edge region, wherein the first grayscale gradient strip projected on the non-edge region of the object includes a highlight projected on the highlight region Partially structured light and part of the structured light projected onto the non-highlighted region, the highlight portion of the structured light projected onto the highlight region is darker than the light projected to the highlight region from the previous time.
27. The structured light projection device according to claim 26, wherein the step of analyzing the acquired object image after the previous frame structured light is captured, and determining whether the object image includes a highlight region comprises: The pixel gray value is determined, and the number n of adjacent pixel points whose gray value is 255 is greater than or equal to the preset threshold x, and the area where the adjacent pixel of the gray value is 255 is determined to be a highlight area; or The pixel gray value is determined. If the ratio of the number of adjacent pixel points n of the gray value to 255 to the total number of pixel points of the object image exceeds a preset threshold y, the adjacent pixel whose gray value is 255 is determined. The area is the highlight area.
28. The structured light projection device according to claim 22, wherein the subtracting the acquired two frames of images to obtain an object image that is not affected by external light includes: gray-by-pixel gradation of the second frame image The pixel-by-pixel gradation in the image of the first frame is subtracted, and the pixel gradation after subtraction is obtained.
29. The structured light projection device according to any one of claims 21 to 25, wherein the first gray scaled strip is projected on a non-edge region of the object, and the inner strip of the first gray scale strip is inscribed There is no repeating texture in the grayscale gradient direction, and the grayscales between adjacent pixels along the grayscale gradient direction are different, and the grayscales of adjacent pixels are increased or decreased along the grayscale gradient direction.
30. The structured light projection device according to any one of claims 21 to 25, wherein the second grayscale gradient strip is projected along an edge of the object, and the width of the second grayscale gradient strip is D and the width is horizontal. Across the edge of the object, the grayscale gradient direction of the second grayscale gradient strip is consistent with the edge of the object, and the entire strip of the second grayscale gradient strip has no repeating texture along the grayscale gradient direction, along the grayscale gradient direction. The gray level varies between adjacent pixels, and the gray level of adjacent pixels increases or decreases along the gray level gradient direction.
31. The structured light projection device according to any one of claims 21 to 25, wherein the first gradation gradient strip and the second gradation gradient strip have different gradation strip arrangement patterns, A grayscale gradient strip is inconsistent with the pixel grayscale of the second grayscale gradient strip in the adjacent region.