CN212567304U - Depth data measuring head - Google Patents

Abstract

Disclosed is a depth data measuring head comprising: the structured light projection device comprises at least two structured light generators which are arranged in a staggered manner in the baseline direction of the connecting line of the two infrared light image sensors and are respectively used for projecting structured light to a measured space so as to form infrared textures which are randomly distributed on an object to be detected in the measured space; the two infrared light image sensors positioned on two sides of the structured light projection device are used for respectively imaging the measured space, so that two infrared texture images are formed; the controller is respectively connected with the structured light projection device and the two infrared light image sensors and is used for controlling the two infrared light image sensors to synchronously shoot the infrared textures; and a housing for receiving the above device and fixing the relative position between the devices. Therefore, the influence of the high light reflection phenomenon on the depth data measurement is avoided by arranging the structure light projection devices which are staggered up and down. In other embodiments, the above functions may also be achieved by arranging staggered pairs of imaging devices.

Description

Depth data measuring head

Technical Field

The utility model relates to a three-dimensional detection technology field especially relates to be used for degree of depth data measuring head and structured light projection arrangement thereof.

Background

In recent years, three-dimensional imaging techniques have been developed vigorously. Currently, a binocular detection scheme based on structured light can perform three-dimensional measurement on the surface of an object in real time. Fig. 1 shows an example of a binocular head. The measuring head comprises a structured light projection device 1, and two image acquisition devices 2 and 3. Briefly, the scheme includes that firstly, a projection device 1 is utilized to project a two-dimensional laser texture pattern with coded information to the surface of a natural body, such as a discretized speckle pattern, two image acquisition devices 2 and 3 with relatively fixed positions are used for continuously acquiring laser textures, a processing unit uses a sampling window to sample two images acquired by the two image acquisition devices simultaneously, matched laser texture patterns in the sampling window are determined, the depth distance of each laser texture sequence segment projected to the surface of the natural body is calculated according to the difference between the matched texture patterns, and three-dimensional data of the surface of an object to be measured is further obtained through measurement.

As shown in the figure, the light projection device 1 is generally disposed on the base line of the two image pickup devices 2 and 3 due to the structure for the sake of compactness or the like. At this time, when scanning a mirror surface perpendicular to the ground or other highly reflective surfaces, a total reflection or high light reflection phenomenon occurs, which results in that the depth image and the three-dimensional data cannot be well acquired. Fig. 2 shows a schematic representation of a binocular head experiencing total reflection (a schematic representation of a plane in the direction perpendicular to the y-axis, where S is the projection source and O is the imaging position).

For this reason, there is a need for a depth data measuring head capable of improving the phenomenon of high light reflection.

SUMMERY OF THE UTILITY MODEL

The utility model provides a degree of depth data measuring head, this measuring head avoid the influence of high light reflection phenomenon to degree of depth data measurement through stagger arranging structured light projection arrangement and two mesh image device from the angle of total reflection. Specifically, it is possible to avoid projection or imaging in the direction of total reflection by arranging pairs of structured light projection devices or imaging devices staggered up and down and switching when high light reflection is detected.

According to a first aspect of the present disclosure, there is provided a depth data measuring head characterized by comprising: the structured light projection device comprises at least two structured light generators which are arranged in a staggered manner in the baseline direction of the connection line of the two infrared light image sensors, wherein the at least two structured light generators are respectively used for projecting structured light to a measured space so as to form randomly distributed infrared textures on an object to be detected in the measured space; the two infrared light image sensors are positioned on two sides of the structured light projection device and used for respectively imaging the measured space so as to form two infrared texture images, and a preset relative spatial position relationship exists between the two infrared light image sensors, so that the depth data of the infrared texture relative to the two infrared image sensors can be determined based on the position difference of texture fragment images correspondingly formed by the same texture fragment in the infrared texture in the two infrared texture images and the preset relative spatial position relationship; the controller is respectively connected with the structured light projection device and the two infrared light image sensors and is used for controlling the two infrared light image sensors to synchronously shoot the infrared textures; and a housing for accommodating the structured light projection device, the two infrared light image sensors and the controller and fixing the relative positions of the structured light projection device and the two infrared light image sensors. Therefore, the influence of the high light reflection phenomenon on the depth data measurement is avoided through the staggered structured light projection device.

Optionally, the at least two structured light generators are arranged at different positions on a perpendicular bisector of a connecting line of the two infrared light image sensors.

Optionally, the at least two structured light generators project structured light at different times towards the measured space. The controller may switch one or more structured light generators projecting structured light to be at a different baseline height when structured light projected by the one or more structured light generators is detected as high brightness reflections.

Optionally, the depth data measurement head may have processing capabilities built in, and then the depth data measurement head may further comprise: a processor to: determining depth data of the infrared texture relative to the two infrared image sensors based on position difference of texture fragment images correspondingly formed by the same texture fragment in the two infrared texture images and the preset relative spatial position relation; and detecting that structured light projected by one or more structured light generators is detected to transmit a high brightness reflection.

Alternatively, the depth data measuring head may be externally connected to other processing equipment, and then, the depth data measuring head may include: an interface device for: the two infrared light image sensors are sent to an external processor and an indication that a high brightness reflection is detected is obtained.

Optionally, the at least two structured light generators comprise: lie in the multiunit structure light generator of different baseline heights, every group structure light generator includes: a plurality of structured light generators for emitting non-overlapping structured light; a fixing structure for fixing the plurality of structured light generators; and a driving device at least partially connected to the fixed structure and used for changing the emergent direction of the plurality of structured light generators.

Optionally, the drive means causes the plurality of structured light generators to oscillate along a drive shaft of the drive means or to oscillate following toggling of a drive gear of the drive means.

Optionally, each set of structured light generators comprises the plurality of structured light generators arranged along or parallel to a base line.

Alternatively, the two infrared light image sensors are arranged on the left and right sides of a cross structure, two groups of structured light generators are arranged on the upper and lower sides of the cross structure, and the cross structure is fixed to the housing.

According to a second aspect of the present disclosure, there is provided a depth data measuring head characterized by comprising: the structured light projection device is used for projecting structured light to the measured space so as to form randomly distributed infrared textures on an object to be detected in the measured space; two sets of infrared light image sensors that are located structured light projection device both sides, every set of infrared light image sensor includes: the two infrared light image sensors are used for respectively imaging the measured space so as to form two infrared texture images, and a preset relative spatial position relationship exists between the two infrared light image sensors, so that the depth data of the infrared texture relative to the two infrared image sensors can be determined based on the position difference of texture fragment images correspondingly formed by the same texture fragment in the infrared texture in the two infrared texture images and the preset relative spatial position relationship; the controller is respectively connected with the structured light projection device and the two groups of infrared light image sensors and is used for controlling the two infrared light image sensors of the group of infrared light image sensors to synchronously shoot the infrared textures; and the shell is used for accommodating the structured light projection device, the two groups of infrared light image sensors and the controller and fixing the relative positions of the structured light projection device and the two groups of infrared light image sensors. The influence of the high light reflection phenomenon on the depth data measurement is thereby avoided by the staggered pairs of imaging devices.

Optionally, a connecting line of one group of infrared light image sensors is not overlapped with a connecting line of the other group of infrared light image sensors.

Optionally, a connection line of the one group of infrared light image sensors is perpendicular to a connection line of the other group of infrared light image sensors.

Alternatively, the one group of infrared light image sensors is arranged on both left and right sides of a cross structure, the other group of infrared light image sensors is arranged on both upper and lower sides of the cross structure, the structured light projecting device is arranged in the center of the cross structure, and the cross structure is fixed to the housing.

Alternatively, when the image captured by one group of infrared light image sensors is detected to be reflected by high brightness, the controller switches another group of infrared light image sensors to capture the image.

Optionally, the depth data measurement head further comprises: a processor to: determining depth data of the infrared texture relative to the two infrared image sensors based on position difference of texture fragment images correspondingly formed by the same texture fragment in the two infrared texture images and the preset relative spatial position relation; and detecting that structured light projected by one or more structured light generators is detected to transmit a high brightness reflection.

Optionally, the depth data measurement head further comprises: an interface device for: the two infrared light image sensors are sent to an external processor and an indication that a high brightness reflection is detected is obtained.

Optionally, the structured light projection device comprises: a plurality of structured light generators for emitting non-overlapping structured light; a fixing structure for fixing the plurality of structured light generators; and a driving device at least partially connected to the fixed structure and used for changing the emergent direction of the plurality of structured light generators.

Optionally, the drive means causes the plurality of structured light generators to oscillate along a drive shaft of the drive means or to oscillate following toggling of a drive gear of the drive means.

Optionally, the plurality of structured light generators are arranged along a baseline direction of a set of infrared light image sensors.

The utility model discloses a degree of depth data measuring head is through introducing the staggered structure light generator who arranges or staggered arrangement's two sets of image sensor, can avoid the high light reflection phenomenon to degree of depth data measuring's influence.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in greater detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.

Fig. 1 shows an example of a binocular head.

Fig. 2 shows a schematic diagram of a binocular head encountering total reflection.

Fig. 3 shows two specific ways of avoiding the effects of high light reflection.

Fig. 4 shows a schematic diagram of a depth data measuring head according to an embodiment of the present invention.

Fig. 5 shows a schematic diagram of a depth data measuring head according to an embodiment of the present invention.

Fig. 6 shows a perspective view of a structured light projection device according to an embodiment of the present invention.

Fig. 7 shows a schematic diagram of a depth data measuring head according to an embodiment of the present invention.

Detailed Description

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The conventional image photographing method can only obtain two-dimensional information of an object, and cannot obtain spatial depth information of the object, and a method for measuring depth by using structured light and an imaging device (e.g., an imaging lens) is very effective. The method has the advantages of small required calculation amount, high precision and capability of being used in places with small brightness. Therefore, more and more three-dimensional measurement schemes select structured light, especially textured infrared beams, in combination with binocular imaging devices to achieve measurement of target depth information.

In the depth measurement scheme using infrared texture binocular imaging, two image acquisition devices with fixed relative positions are used for continuously acquiring laser textures, a processing unit is used for sampling two images acquired by the two image acquisition devices simultaneously by using a sampling window, a laser texture pattern matched in the sampling window is determined, the depth distance of each laser texture sequence segment projected on the surface of a natural body is calculated according to the difference between the matched texture patterns, and the three-dimensional data of the surface of an object to be measured is further measured.

The structured light projection device is typically arranged on a base line of the two image acquisition devices (as shown in fig. 1) for reasons of compactness etc. At this time, when scanning a mirror surface perpendicular to the ground or other highly reflective surface, a total reflection phenomenon or a high light reflection phenomenon may occur as shown in fig. 2, thereby failing to perform the depth image acquisition and the three-dimensional data acquisition well.

In order to solve the problem that high light reflection affects depth imaging due to the projection source and the imaging device being located at the same height (for example, x direction), the inventors of the present disclosure have conceived that the high light reflection effect can be avoided by staggering S (projection source) and O (imaging position) located at the same height. Fig. 3 shows two specific ways of avoiding the effects of high light reflection. As shown on the upper right, one way is to avoid high light reflection effects by providing projection sources of different heights. As shown in the lower right side of the figure, another way can avoid high light reflection effects by providing imaging positions of different heights.

Fig. 4 shows a schematic diagram of a depth data measuring head according to an embodiment of the present invention. As shown, the depth data measuring head of the present invention includes a structured light projection device (including at least two structured light generators 410 and 440 arranged in a staggered manner in the baseline direction B), a first infrared image sensor 420, a second infrared image sensor 430, a controller (not shown), and a housing 450.

The direction of the line connecting the first infrared image sensor 420 and the second infrared image sensor 430 is referred to herein as a baseline direction, i.e., a baseline B shown by a dotted line in the figure. In order to avoid the influence of high light reflection, the structured light projection device of the present disclosure includes two structured light generators 410 and 440 arranged in a staggered manner in the baseline direction, and the at least two structured light generators are respectively used for projecting structured light to a measured space to form randomly distributed infrared textures on an object to be detected in the measured space. In other embodiments, more structured light generators may also be arranged. The texture carried by the infrared beam projected by the structured light projection device can be random speckle texture, stripe coding texture adopting De Bruijn (Debruuin sequence) sequence, and texture with other shapes.

The two infrared light image sensors 420 and 430 located at two sides of the structured light projection device are configured to respectively image the measured space, so as to form two infrared texture images, and the two infrared light image sensors have a predetermined relative spatial position relationship therebetween, so that depth data of the infrared texture relative to the two infrared image sensors can be determined based on a position difference of texture segment images correspondingly formed in the two infrared texture images by a same texture segment in the infrared texture and the predetermined relative spatial position relationship.

And the controller is respectively connected with the structured light projection device and the two infrared light image sensors and is used for controlling the two infrared light image sensors to synchronously shoot the infrared textures.

The housing 450 is configured to house the structured light projection device, the two infrared light image sensors, and the controller and to fix the relative positions of the structured light projection device and the two infrared light image sensors. For ease of illustration, only the internal fixation structure of the housing 450 is shown. It should be understood that the housing 450 may also include a housing for enclosing the above-described devices, and may also include other structures for fastening, heat dissipation, or indication, etc.

In one embodiment, for ease of calculation, at least two structured light generators are disposed at different positions on a perpendicular bisector of a connecting line of the two infrared light image sensors, for example, one is located on a base line (e.g., structured light generator 410) as shown in fig. 4, and the other is located at an upper or lower position thereof, for example, structured light generator 440. In other embodiments, two (or two sets of) structured light generators and two infrared light image sensors may be arranged in a cross in the xy-plane.

The two structured light generators 410 and 440 project structured light to the measured space at different times, and the first infrared image sensor 420 and the second infrared image sensor 430 can respectively shoot the structured light generated by the light generators with different structures under the control of the controller. In one embodiment, the projection photographing may be performed alternately by the two structured light generators 410 and 440, or after each N-times projection of the structured light generator 410, the structured light generator 440 projects the light, and the two infrared light image sensors 420 and 430 photograph the projections thereof, respectively. If the projection of the structured light generator 410 is found to be highly specular, the image may be deleted and the depth calculation may be performed directly using the image taken before or after the projection by the structured light generator 440.

In other embodiments, if the device has real-time computing and feedback capabilities, the controller may switch one or more structured light generators projecting structured light to project structured light at a different baseline height when structured light projected by the one or more structured light generators is detected as high intensity reflections.

In this case, it may be that the depth data measuring head itself has the above-described real-time calculation and feedback capabilities. Accordingly, the depth data measuring head may further include a processor. The processor may determine depth data of the infrared texture with respect to the two infrared image sensors based on a difference in position of texture fragment images formed in the two infrared texture images corresponding to the same texture fragment in the infrared texture and the predetermined relative spatial positional relationship; and detecting that structured light projected by one or more structured light generators is detected to transmit a high brightness reflection.

Alternatively or additionally, the depth data measuring head may be connected externally to an external computing device. The depth data measuring head may then further comprise interface means. The interface device may send the two infrared light image sensors to an external processor and obtain an indication that a high brightness reflection is detected.

At this time, the plurality of structural light generators 410 and 440 do not need to be alternately projected in a specific pattern, but when the computing device finds that an image currently photographed with respect to a certain structural light generator (e.g., 410) is overexposed (e.g., a slice of white) due to high light reflection, other structural light generators (e.g., 440) staggered in the x direction are immediately switched to be projected, thereby avoiding high light reflection.

Fig. 5 shows a schematic diagram of a depth data measuring head according to an embodiment of the present invention. As shown, two sets of structured light generators 510 and 540 and two infrared light image sensors 520 and 530 are arranged in a cross in the xy plane. FIG. 5 may be considered a preferred embodiment of a depth data measuring head of the present disclosure. It should be understood that in other embodiments, two (rather than two) structured light generators may be arranged at the upper and lower ends of the cross structure, respectively.

As described above, in the depth measurement scheme using the binocular imaging of the infrared texture, the two image acquisition devices with fixed relative positions continuously acquire the laser texture, the processing unit samples two images acquired by the two image acquisition devices simultaneously using the sampling window, determines the laser texture patterns matched in the sampling window, calculates the depth distance of each laser texture sequence segment projected on the surface of the natural body according to the difference between the matched texture patterns, and further measures to obtain the three-dimensional data of the surface of the object to be measured. In the matching process, the larger the sampling window is, the larger the amount of pattern information included in a single sampling is, and thus matching is easier to perform, but the larger the granularity of the obtained depth image is. Accordingly, the smaller the sampling window, the finer the granularity of the image, but the greater the mismatch rate.

In order to solve the contradiction between image granularity and the matching rate, the utility model discloses a provide more information quantity for the window of the same size and ensure the high matching confidence of little window sampling. The infrared textures projected from different positions or different angles can be repeatedly shot so as to include depth information of more positions on the object to be detected, and then the depth data of the object to be detected is obtained by fusing the repeatedly shot image information, so that the depth (three-dimensional) image of the object to be detected can be reproduced with finer granularity. Further, this scheme can be through introducing the drive structure of a plurality of structure light generator linkages, can realize that a plurality of structure light generators throw from different positions and throw with single structure light generator with different angles on the same position and combine together to in drive arrangement's driving capability scope, further deepening the integration of degree of depth information.

Therefore, the utility model discloses a degree of depth data measuring head can adopt one kind be applicable to the sampling of little window, can provide the structured light projection arrangement of multi-angle ability. As shown in fig. 5, it includes a perspective view of two sets of structured light projection devices disposed at both lower and upper sides of a vertical square line connecting the first and second image sensors 520 and 530, each set of structured light projection devices including two structured light generators (e.g., 511 and 512, and 541 and 542), and including driving devices (514 and 544) for driving the structured light generators via the fixing structures 513 and 543.

Fig. 6 shows a perspective view of a structured light projection device according to an embodiment of the present invention. The illustrated structured light projection devices correspond to the structured light projection devices 510, 540 and 710 and 740 used in fig. 5 and fig. 7 below, and have corresponding numbering.

As shown in fig. 6, the structured light projection device may comprise two structured light generators 611 and 612, a stationary structure 613 and a driving device 614 at least partly connected to the stationary structure.

As shown, the two structured light generators 611 and 612 are fixed to the same fixed structure 613, thus remaining relatively stationary and moving together under the drive of the drive means 614. The driving means 614 may be adapted to swing the two structured light generators along the driving shaft of the driving means or following the toggling of the driving gear of the driving means. In other embodiments, the structured light projection device may also comprise more structured light generators, which may likewise be linked under the drive of the drive device, for example by being connected to a fixed structure.

In the example shown in fig. 6, the driving device 614 may be a miniature Step Motor Driver (SMD) Motor. The specific structure of the stepping motor is included in its housing, not shown. If it is assumed that the z direction is the current structured light emitting direction of the structured light generators 611 and 612, the structured light generators 611 and 612 can be moved in the direction perpendicular to the y axis (the direction of the line connecting the two structured light generators) by the driving of the stepping motor. For example, in the xz plane, at varying angles, for example in the range of the swivel angle α.

The specific operating principle of the stepper motor is well known in the art and will not be described herein. In other embodiments, the driving device 614 can be other micro-motors, such as micro-mechanical (MEMS) motors, or voice coil motors.

Depending on the specific implementation, the change of the exit angle may be continuous or fixed, for example, in the case of a gear drive. The range of variation of the exit angle also depends on the specific setting of the drive means. The actual range of motion of the drive means may be directly transmitted to a structured light generator on the fixed structure or may be adjusted via a transmission. To this end, the structured light projection device may further include: a transmission for transmitting the movement of the drive to the fixed structure at a set transmission ratio.

In a preferred embodiment, the fixed structure 613 may be connected to the driving means 614 via conducting means 615, whereby the movement may be performed via a set transmission ratio (also called reduction ratio) compared to the movement of the driving motor. For example, in the case shown in fig. 6, the stepping motor 614 moves 1.8 ° in one pulse, and if the reduction ratio of the actuator 615 is set to 1:20, the swing angle of the structured light projecting device may be 0.09 °. If the drive motor can make a 10-grid swing, for example, the swing angle α of the structured light projector can be 0.45 ° upward and downward in each direction along the z-axis. That is, if the z-axis is set to 0 °, the structured light projection device can be swung in steps of 0.09 ° in the range of 0.45 ° to-0.45 ° in the xz plane.

As described above with reference to fig. 6, which shows a perspective view of a structured light projection device according to an embodiment of the present invention, it is apparent that various modifications within the scope of the present invention can be made to the above structure. For example, for the sake of completeness and ease of installation, the device may further comprise a housing for enclosing at least part of the structured light generator and the driving means and enabling structured light to exit, or a fixed structure may serve as part of the housing and be further integrated with a measuring head as described below.

As shown, the structured light generators 611 and 612 may be included as laser generators for emitting infrared laser light. In other embodiments, other laser generating devices may be included. Structured light generators 611 and 612 may also include diffractive optical elements disposed in the exit path of the laser light for generating the textured infrared beam. In the embodiments described below in connection with the depth data camera, the diffractive optical element described above may be arranged on the housing of the depth data camera, for example, inside the exit window, in addition to the exit ends of the structured light generators 611 and 612 shown in fig. 6.

The diffractive optical element may be a diffractive optical element surface, for example, which may be located directly outside the laser diode, or outside an open cover plate of the depth data measurement head housing. In one embodiment, the laser generator may be a laser diode. The single laser beam emitted by the laser diode can be collimated and then incident on the surface of the diffractive optical element, and the surface can be provided with a surface microstructure formed by an optical micromachining technology, so that the incident laser beam is diffracted and modulated into discrete spots with specific projection rules. Thereby realizing the precise control of the projected texture without additionally increasing the volume of the projection device and the design difficulty. In another embodiment, the laser generator may be a Vertical Cavity Surface Emitting Laser (VCSEL) for emitting a multi-spot laser array, and the diffractive optical element may have a surface structure that replicates the incident multi-spot laser array to produce periodic discrete spots.

A collimating means, for example a collimating lens, may optionally be arranged between the laser generator and the diffractive optical element for collimating the outgoing light of the laser generator. A direction changing structure, such as a right angle prism, may also optionally be disposed between the laser generator and the diffractive optical element for changing the direction of the laser light emitted by the laser generator to meet specific placement requirements.

In other embodiments, discrete spots can also be directly transmitted using a multi-spot laser array arrangement of VCSELs without the introduction of diffractive optical elements.

As above combine the structured light projection device that figure 6 described can alternate exit angle can be used to figure 5 and following figure 7 shown depth data measuring head, and it is through throwing the multi-angle to infrared texture, can acquire the depth data who waits to detect more position department on the object, thereby makes based on the utility model discloses the depth data who obtains can reflect the depth information who waits to detect the object more comprehensively.

Turning to fig. 5, fig. 5 may be considered as a preferred embodiment of a depth data measuring head according to the present invention. The measurement head may include a first infrared image sensor 520, a second infrared image sensor 530, a controller (not shown), two sets of structured light generators 510 and 540, and a housing including a cross-structure 550. The two infrared light image sensors are arranged on the left and right sides of the cross structure, the two groups of structured light generators are arranged on the upper and lower sides of the cross structure, and the cross structure is fixed to the housing, for example, while being covered by the housing, for example, a carbon fiber case. The figure shows that each group of structured light generators comprises a plurality of structured light generators arranged along a baseline parallel direction. In other embodiments, one of the sets of structured light generators may be arranged along a baseline and take the shape of a "T".

As described above, the structured light projection devices 510 and 540 can enable the plurality of structured light generators included therein to project the infrared light beams with textures at different angles to the space to be detected under the driving of the driving device, so as to form randomly distributed infrared light textures on the object to be detected in the space to be detected. The texture carried by the infrared beams projected by the structured light projection devices 510 and 540 may be random speckle texture, or stripe-coded texture using De Bruijn (De Bruijn sequence), or texture of other shapes.

The structured light projection devices 510 and 540 operate at different time periods. For example, in the case of projection based on feedback as described above, when the processor determines that highlight reflection occurs based on overexposure of an image captured for the structured light projection device currently being put into operation, the other structured light projection device may be switched into operation. For example, when the structured light projection device 510 is highly reflective due to the relative positions of the two image sensors, it may be switched to the structured light projection device 540 to be operated, and vice versa.

The structured light projection devices 510 and 540 may each have multiple modes of operation during their time periods of operation. Under different working modes, the infrared beams with textures can be projected to the detected space at different projection angles, so that texture patterns with different distributions can be formed on the object to be detected in the detected space under different working modes, namely under different working modes, a plurality of texture segments projected on the object to be detected are distributed at different positions on the object to be detected. Since the diffractive structure of the diffractive optical element and the relative position to the laser generator remain the same, or the laser array in a VCSEL implementation remains the same, the texture information carried by the ir beams projected by the structured light projection devices 510 and 540 in different modes of operation may be the same. However, due to the different projection positions and angles, the discrete spots projected onto the object to be measured have a small displacement, i.e., non-coincident projection points within the same or approximately the same projection range.

In particular, in each operating mode, the driving device may fine-tune the emission direction of the plurality of structured light generators, and the plurality of structured light generators may project structured light to the same imaging area at successive imaging frames.

For example, the depth data measuring head has 5 operating modes. In a first mode of operation, a first structured light generator and a second structured light generator project structured light to the same imaging area in a first and a second imaging frame, respectively. Subsequently, the driving device determines that the first and second structured light generators are rotated by an angle, e.g. 0.09 °, such that, in the second operation mode, the first and second structured light generators project structured light towards the same imaging area in the third and fourth imaging frames, respectively. Subsequently, the driving device determines that the first and second structured light generators continue to rotate by an angle, for example, also 0.09 °, whereupon, in a third operating mode, the first and second structured light generators project structured light to the same imaging area in a fifth and sixth imaging frame, respectively. Subsequently, the driving device determines that the first and second structured light generators continue to rotate by an angle, e.g. also 0.09 °, whereupon, in a fourth mode of operation, the first and second structured light generators project structured light to the same imaging area in a seventh and eighth imaging frame, respectively. Finally, the drive means determines that the first and second structured light generators continue to rotate by an angle, for example, also 0.09 °, whereupon, in a fifth operating mode, the first and second structured light generators project structured light towards the same imaging area in a ninth and tenth imaging frame, respectively.

Since the projection angle of the structured light generator between each operation mode is slightly changed (for example, only 0.09 degrees), each structured light generator still performs imaging on the same measured area when the projection angle is changed (although the measured area may have slight differences at each angle) under the condition that the depth camera is fixedly installed. However, the specific positions covered by the projected discrete light will be different due to the variation of the projection angle. In addition, the first structured light generator and the second structured light generator project discrete light spots that differ in successive frames due to their different positions. For this reason, with 10 imaging frames of 5 operation modes, (compared with the case where the driving device drives only one structured light generator), the present solution can acquire more projected speckle information shifted from each other by linkage and successive projection of a plurality of structured light generators with limited driving capability of the driving device.

The structured light projection devices 510 and 540 are located between the two infrared image sensors 520 and 530, for example, on the perpendicular square of the line connecting the two sensors. The first infrared image sensor 520 and the second infrared image sensor 530 have a predetermined relative spatial position relationship, and for each working mode of the structured light projection device (510 or 540), the first infrared image sensor 520 and the second infrared image sensor 530 can shoot the space to be detected for multiple times (the times are the same as the number of structured light generators participating in successive projection), so as to obtain images of the object to be detected in the space to be detected in different working modes. The images acquired by the first infrared image sensor 520 and the second infrared image sensor 530 are infrared texture images formed by projecting textured infrared light beams onto an object to be detected. Since the structured light projection device 510 or 540 projects the infrared light beams at different projection angles and from different projection positions in different working modes, so that the object to be detected in the detected space has texture patterns with different distributions, in different working modes, under the condition that different structured light generators with the same working mode participate in projection, the distributions of texture segments in the infrared texture images of the object to be detected, which are acquired by the first infrared image sensor 520 and the second infrared image sensor 530, are different.

The controller (not shown) is connected to the first infrared image sensor 520 and the second infrared image sensor 530, respectively, and can acquire the infrared texture images in the plurality of working modes captured by the first infrared image sensor 520 and the second infrared image sensor 530, and obtain the depth data of each texture segment in the infrared texture images in different working modes through processing.

For each projection of each working mode, the controller can determine the depth data of the texture segment according to the preset relative spatial position relationship between the two infrared light image sensors and based on the position difference of the texture segment images correspondingly formed by the same texture segment in the infrared texture on the object to be detected in the two infrared texture images. Thus, the controller may determine depth data of a plurality of texture segments on the object to be detected, i.e. determine depth data of the infrared texture on the object to be detected relative to the two infrared light image sensors. Therefore, the controller can obtain the depth data of each texture segment on the object to be detected under different projections of different working modes. Because the texture segments on the object to be detected are distributed differently under different projections of different working modes, the controller can fuse the depth data of each texture segment on the object to be detected determined under different projections of different working modes, and take the fused depth data as the depth data of the object to be detected.

In particular, since more abundant depth information can be obtained by a plurality of shots for a multi-angle multi-position projection than by a single shot for a single projection, the controller can increase the fineness of the depth image by reducing the size of the sampling window.

In the case of a single projection, the controller needs to use a window of, for example, 21x21 pixels in order to match the pattern in, for example, the left and right images taken simultaneously by the first infrared image sensor 2 and the second infrared image sensor 3 with a sufficiently high degree of confidence. The depth data for the texture segment is then determined again by the difference in position between the matched image pixels.

The projection device (510 or 540) can be used for multi-angle projection. For example, a structured light projection device (510 or 540) including two structured light generators may operate in five modes of operation corresponding to five angles, 0.09 ° and 0.18 ° offset left and right, respectively, from perpendicular to the exit face of the depth data measurement head. The controller may control the first infrared image sensor 520 and the second infrared image sensor 530 to continuously photograph the same object to be detected ten times, each time the projection of one structured light generator corresponding to one angle of the structured light projection device 210 is photographed, and perform fusion processing on twenty images obtained by the ten times photographing to obtain depth data more refined than that obtained by processing two images obtained by a single photographing.

In one embodiment, the controller may use a smaller sampling window (e.g., 2x2 pixels) to match the ten pairs of images taken ten times each, and by considering the matching results of the ten matches collectively, achieve finer small-window matches with sufficiently high confidence. In other words, the controller may select a sampling window with the same size to perform window matching on the two infrared texture images in each operating mode, and determine a final matching result according to the window matching result in each operating mode.

In a preferred embodiment, the high confidence small window matching described above can be achieved by peak data matching. Specifically, a difference value of image pixels (e.g., characterized by a gray value) of the two infrared texture images within the same sampling window may be obtained, and the difference value represents the similarity. Under each working mode, the similarity of each pixel of the window is summed, and the similarity under each working mode is summed to obtain a cost value. And finding a cost peak at each possible matching position, wherein the position with the maximum peak can be regarded as successful matching. The similarity difference may be an absolute difference, or a squared difference, or some other pixel similarity evaluation method. Thus, data fusion can be also achieved by simple peak matching, thereby achieving highly accurate depth data detection at extremely low computational cost.

In the process of processing data, a connecting line of optical imaging centers of the two image sensors can be used as a baseline direction, and at the moment, in each working mode, the position difference of texture segment images correspondingly formed by the same texture segment on the object to be detected in the two infrared texture images is in the baseline direction. As shown in fig. 5, the two infrared light image sensors may be disposed at both ends of the cross structure 550 to appropriately lengthen the distance between the two image sensors, facilitating the formation of parallax. The structured light projection devices 510 and 540 may be disposed at the other two ends of the cross-shaped structure 550, and also the distance between the two structured light projection devices is appropriately lengthened, so that when one projection device causes total reflection to switch to the other, the other device is not affected by the total reflection at all.

Preferably, the housing main body may be made of carbon fiber, thereby avoiding the change of the relative position between the devices caused by the linear expansion, and also improving the heat conductivity of the camera and achieving a certain electromagnetic shielding effect. Additionally, in other embodiments, the depth data measurement head may also include heat dissipation and indication devices. For example, a structured light projection device comprising a structured light generator may have disposed below it a processor (e.g., a processing circuit board or processing chip) that performs image fusion calculations or at least partial calculations. For this purpose, heat dissipating means, for example heat dissipating ribs, can be arranged under the control circuit, which extend beyond the housing. In addition, for convenience of operation, an indicating device for indicating the operating condition of the camera, for example, an indicator light, may be further provided. The indicator light may give an indication directly based on the signal of the control circuit and may also be equipped with a dust cover.

Preferably, the depth data measuring head of the present embodiment may further include a visible light image sensor, such as an RGB sensor, to further acquire color information in the measured object or space, so as to facilitate the relevant processing. The visible light image sensor 520 may be located between the two structured light generators included in 510 and 540 or may be located in the center of the cross structure.

The controller may be connected to the structured light projection devices 510 and 540 and the two infrared light image sensors (the first infrared light image sensor 520, the second infrared light image sensor 530), respectively, may control the two infrared light image sensors to photograph synchronously, and may switch the operation mode of the structured light projection device (510 or 540) so that the structured light projection device (510 or 540) may be switched to the next operation mode after exposure (photographing) of every two frames by the two infrared light image sensors is completed.

Specifically, the controller may trigger the two infrared image sensors to synchronously image by sending a trigger signal to the two infrared image sensors, and switch the operating mode of the structured light projection device (510 or 540) by sending a switching signal to the structured light projection device (510 or 540), wherein the trigger signal may be a frequency doubling signal of the switching signal.

In this way, the structured light projection device may switch to the next operation mode in response to the switching signal, and the first infrared light image sensor 520 and the second infrared light image sensor 530 may simultaneously image the measured space in the next operation mode in response to receiving the trigger signal from the controller (the trigger signal in each operation mode corresponds to the number of structured light generators participating in projection).

Further, the light emitting frequency (pulse frequency of emitting infrared light) of the laser generator in the structured light projection apparatus may be an integral multiple of the frame frequency of the infrared image sensor, thus providing a basis for the synchronous operation of the infrared generator and the infrared image sensor. Further, the light emitting frequency of the laser generator may be set to a high value (e.g., greater than 100HZ) such that the light emitted by the laser generator is substantially constant with respect to the image sensor.

Embodiments for avoiding the effects of high light reflection by providing projection sources of different heights are described above and in connection with fig. 4-6. Another way of avoiding the effects of high light reflection can be achieved by providing imaging positions of different heights, as described above in connection with fig. 3.

To this end, the present solution may also be realized as a depth data measuring head comprising two pairs of image sensors. The depth data measurement head may include a structured light projection device, two sets of infrared light image sensors located on either side of the structured light projection device, a controller, and a housing.

The structured light projection device is used for projecting structured light to the measured space to form randomly distributed infrared textures on an object to be detected in the measured space.

Each set of infrared light image sensors may include: the two infrared light image sensors are used for respectively imaging the measured space so as to form two infrared texture images, and a preset relative spatial position relationship exists between the two infrared light image sensors, so that the depth data of the infrared texture relative to the two infrared image sensors can be determined based on the position difference of texture fragment images correspondingly formed by the same texture fragment in the infrared texture in the two infrared texture images and the preset relative spatial position relationship;

the controller is respectively connected with the structured light projection device and the two groups of infrared light image sensors and is used for controlling the two infrared light image sensors of the group of infrared light image sensors to synchronously shoot the infrared textures.

The housing may be configured to house the structured light projection device, the two sets of infrared light image sensors, and the controller and to fix relative positions of the structured light projection device and the two sets of infrared light image sensors.

In order to provide different relative positions, it is necessary that the connecting line of one set of infrared light image sensors does not coincide with the connecting line of the other set of infrared light image sensors. To this end, the lines of the two sets of image sensors may be parallel to each other (but not coincident) or intersect at an angle.

Preferably, a connection line of the one group of infrared light image sensors and a connection line of the other group of infrared light image sensors are perpendicular to each other. Thereby maximally avoiding that when one set of image sensors is in the high light reflection position, the other set of image sensors is also affected by the high light reflection at the same time.

Fig. 7 shows a schematic diagram of a depth data measuring head according to an embodiment of the present invention. As shown, one set of infrared light image sensors 720 and 730 is disposed on both left and right sides of the cross structure, the other set of infrared light image sensors 720 'and 730' is disposed on both upper and lower sides of the cross structure, the structured light projecting device 710 is disposed in the center of the cross structure, and the cross structure 710 is fixed to the housing.

As shown, structured light projecting device 710 may have the structure shown in FIG. 6, including: two (or more) structured light generators 711 and 712 for emitting non-overlapping structured light; a fixing structure 713 for fixing the plurality of structured light generators; and a driving device 714 at least partially connected to the fixed structure for changing the emission direction of the plurality of structured light generators. Here, the specific functions of the structured light projection device 710 may be similar to those described above in conjunction with fig. 5 and 6, and will not be described again here. Similarly, the drive arrangement 714 may be such that the plurality of structured light generators oscillate along a drive shaft of the drive arrangement or following toggling of a drive gear of the drive arrangement. Alternatively, the drive 714 and the fixed structure 713 may be connected by a transmission to effect a ratio-based adjustment.

Further, a plurality of structured light generators (e.g., 711 and 712) are arranged along a baseline direction of a set of infrared light image sensors (720 and 730) to facilitate conventional depth data capture and calculation.

In different implementations, the two sets of image sensors may take alternately, or may preferably be switched based on feedback. At this time, when the image captured by one set of infrared light image sensors is detected to be reflected by high brightness, the controller switches the other set of infrared light image sensors to capture the image.

The feedback may be implemented via internal or external calculations. To this end, the depth data calculation apparatus including two sets of image sensors may further include a processor for: determining depth data of the infrared texture relative to the two infrared image sensors based on position difference of texture fragment images correspondingly formed by the same texture fragment in the two infrared texture images and the preset relative spatial position relation; and detecting that structured light projected by one or more structured light generators is detected to transmit a high brightness reflection.

Alternatively or additionally, the depth data calculation device comprising two sets of image sensors may further comprise interface means for: the two infrared light image sensors are sent to an external processor and an indication that a high brightness reflection is detected is obtained.

The depth data measuring head according to the present invention has been described in detail hereinabove with reference to the drawings, and avoids the influence of the high light reflection phenomenon on the depth data measurement by arranging the structured light projecting device and the binocular imaging device in a staggered manner from the angle of total reflection. Specifically, it is possible to avoid projection or imaging in the direction of total reflection by arranging pairs of structured light projection devices or imaging devices staggered up and down and switching when high light reflection is detected.

While various embodiments of the present invention have been described above, the above description is intended to be illustrative, not exhaustive, and not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (20)

1. A depth data measurement head, comprising:

the structured light projection device comprises at least two structured light generators which are arranged in a staggered manner in the baseline direction of the connection line of the two infrared light image sensors, wherein the at least two structured light generators are respectively used for projecting structured light to a measured space so as to form randomly distributed infrared textures on an object to be detected in the measured space;

the two infrared light image sensors are positioned on two sides of the structured light projection device and used for respectively imaging the measured space to form two infrared texture images, and a preset relative spatial position relationship exists between the two infrared light image sensors, so that the depth data of the infrared texture relative to the two infrared light image sensors can be determined based on the position difference of texture fragment images correspondingly formed in the two infrared texture images by the same texture fragment in the infrared texture and the preset relative spatial position relationship;

the controller is respectively connected with the structured light projection device and the two infrared light image sensors and is used for controlling the two infrared light image sensors to synchronously shoot the infrared textures; and

and the shell is used for accommodating the structured light projection device, the two infrared light image sensors and the controller and fixing the relative positions of the structured light projection device and the two infrared light image sensors.

2. The depth data measuring head of claim 1, wherein the at least two structured light generators are arranged at different positions on a perpendicular bisector of a line connecting the two infrared light image sensors.

3. The depth data measurement head of claim 1, wherein the at least two structured light generators project structured light at different times into the measured space.

4. The depth data measurement head of claim 3, wherein one or more structured light generators at different baseline heights are switched by the controller to project structured light when structured light projected by the one or more structured light generators is detected as high brightness reflections.

5. The depth data measurement head of claim 3, further comprising:

a processor to:

determining depth data of the infrared texture relative to the two infrared light image sensors based on position difference of texture fragment images correspondingly formed in the two infrared texture images by the same texture fragment in the infrared texture and the preset relative spatial position relation; and

and detecting that the structured light projected by one or more structured light generators generates high-brightness reflection.

6. The depth data measurement head of claim 3, further comprising:

an interface device for:

the two infrared light image sensors are sent to an external processor and an indication that a high brightness reflection is detected is obtained.

7. The depth data measurement head of claim 1, wherein the at least two structured light generators comprise:

lie in the multiunit structure light generator of different baseline heights, every group structure light generator includes:

a plurality of structured light generators for emitting non-overlapping structured light;

a fixing structure for fixing the plurality of structured light generators; and

and the driving device is at least partially connected to the fixed structure and used for changing the emergent direction of the plurality of structured light generators.

8. The depth data measuring head of claim 7, wherein the driving means causes the plurality of structured light generators to oscillate along a drive shaft of the driving means or to oscillate following toggling of a drive gear of the driving means.

9. The depth data measuring head of claim 7, wherein each set of structured light generators comprises the plurality of structured light generators arranged along or parallel to a baseline.

10. The depth data measuring head of claim 9, wherein the two infrared light image sensors are disposed on the left and right sides of a cross structure, two sets of structured light generators are disposed on the upper and lower sides of the cross structure, and the cross structure is fixed to the housing.

11. A depth data measurement head, comprising:

the structured light projection device is used for projecting structured light to the measured space so as to form randomly distributed infrared textures on an object to be detected in the measured space;

two sets of infrared light image sensors that are located structured light projection device both sides, every set of infrared light image sensor includes: the two infrared light image sensors are used for respectively imaging the measured space so as to form two infrared texture images, and a preset relative spatial position relationship exists between the two infrared light image sensors, so that the depth data of the infrared texture relative to the two infrared light image sensors can be determined based on the position difference of texture fragment images correspondingly formed by the same texture fragment in the infrared texture in the two infrared texture images and the preset relative spatial position relationship;

the controller is respectively connected with the structured light projection device and the two groups of infrared light image sensors and is used for controlling the two infrared light image sensors of the group of infrared light image sensors to synchronously shoot the infrared textures; and

and the shell is used for accommodating the structured light projection device, the two groups of infrared light image sensors and the controller and fixing the relative positions of the structured light projection device and the two groups of infrared light image sensors.

12. The depth data measurement head of claim 11 wherein the lines of one set of infrared light image sensors do not coincide with the lines of the other set of infrared light image sensors.

13. The depth data measuring head of claim 12 wherein the lines of the one set of infrared light image sensors are perpendicular to the lines of the other set of infrared light image sensors.

14. The depth data measuring head of claim 13, wherein the one set of infrared light image sensors is disposed on both left and right sides of a cross structure, the other set of infrared light image sensors is disposed on both upper and lower sides of the cross structure, the structured light projecting device is disposed in the center of the cross structure, and the cross structure is fixed to the housing.

15. The depth data measuring head of claim 11 wherein the controller switches one set of infrared light image sensors to capture images when a high brightness reflection is detected from the images captured by the other set of infrared light image sensors.

16. The depth data measurement head of claim 11, further comprising:

a processor to:

determining depth data of the infrared texture relative to the two infrared light image sensors based on position difference of texture fragment images correspondingly formed in the two infrared texture images by the same texture fragment in the infrared texture and the preset relative spatial position relation; and

detecting structured light projected by one or more structured light generators is detected to transmit high brightness reflections.

17. The depth data measurement head of claim 11, further comprising:

an interface device for:

the two infrared light image sensors are sent to an external processor and an indication that a high brightness reflection is detected is obtained.

18. The depth data measurement head of claim 11, wherein the structured light projecting means comprises:

a plurality of structured light generators for emitting non-overlapping structured light;

a fixing structure for fixing the plurality of structured light generators; and

and the driving device is at least partially connected to the fixed structure and used for changing the emergent direction of the plurality of structured light generators.

19. The depth data measuring head of claim 18, wherein the drive means causes the plurality of structured light generators to oscillate along a drive shaft of the drive means or to oscillate following toggling of a drive gear of the drive means.

20. The depth data measuring head of claim 18, wherein the plurality of structured light generators are arranged along a baseline direction of a set of infrared light image sensors.

Images