CN217883646U - Long-baseline depth camera module and electronic equipment - Google Patents

Abstract

The utility model provides a long baseline depth camera module, which is characterized in that the camera module comprises an RGB module, an IR module and a projection module; the RGB module is used for shooting an RGB image of a target object; the IR module is used for acquiring depth data; the projection module is used for emitting lattice structure light or floodlight to the target object according to the IR module signal; the IR module and the projection module are arranged on the base shell in a diagonal manner, so that a base line between the IR module and the projection module is maximized. The utility model discloses change basic shell shape, remove through the baseline that will throw module and IR module, no longer parallel with the limit of basic shell, but present certain contained angle for the space utilization of module is higher, and the device is arranged more closely, resists external force's level reinforcing simultaneously.

Description

Long baseline degree of depth module and electronic equipment of making a video recording

Technical Field

The utility model relates to a 3D field of making a video recording specifically relates to a module and electronic equipment are made a video recording to long baseline degree of depth.

Background

In 3D computer graphics, a Depth Map (Depth Map) is an image or image channel containing information about the distance of the surface of a scene object from a viewpoint. Where Depth Map is similar to a grayscale image except that each pixel value is the actual distance of the sensor from the object. Usually, the RGB image and the Depth image are registered, so that there is a one-to-one correspondence between the pixel points.

In the prior art, the projection module, the RGB module and the IR module are all on the same baseline, and the module is basically in a rectangular state, which is designed for a rectangular module scheme. This kind of design does benefit to manufacturing, but this kind of design can't make full use of the space of module, leads to space utilization not high, the integrated level is not high to make the module size can't be littleer.

SUMMERY OF THE UTILITY MODEL

Therefore, the utility model discloses change basic shell shape, remove through the baseline that will throw module and IR module, no longer parallel with the limit of basic shell, but present certain contained angle for the space utilization of module is higher, and the device is arranged inseparabler, resists the level reinforcing of external force simultaneously. The utility model provides a module and electronic equipment are made a video recording to long baseline degree of depth.

According to the utility model provides a long baseline depth camera module, which is characterized by comprising an RGB module, an IR module and a projection module;

the RGB module is used for shooting an RGB image of a target object;

the IR module is used for acquiring depth data;

the projection module is used for emitting lattice structure light or floodlight to the target object according to the IR module signal;

the IR module and the projection module are arranged on the base shell in a diagonal manner, so that a base line between the IR module and the projection module is maximized.

Optionally, the long-baseline depth camera module is characterized in that the projection module is provided with a zoom projection lens, and is used for projecting lattice structured light or floodlight to a target object through zooming;

the IR module can obtain depth data based on reflection of the projected lattice structured light and can obtain depth data according to the flight time of floodlight.

Optionally, the long baseline depth camera module is characterized in that the projection module comprises a discrete beam projector and a surface light source projector;

the discrete light beam projector is used for projecting lattice structured light to the target object;

the surface light source projector is used for projecting floodlight to the target object;

the IR module can obtain depth data based on reflection of the projected lattice structured light and depth data according to flight time of floodlight.

Optionally, the above-mentioned long baseline depth camera module is characterized in that the projection module includes a laser array, a collimating element, a beam splitter, and a diffuser;

the laser array is used for emitting lattice laser;

the collimating element is used for collimating the incident lattice laser to generate a collimated light beam;

the light splitting device is used for splitting the incident collimated light beam into a plurality of laser beams;

and the diffuser is used for diffusing the laser beams according to the signal of the IR module so as to enable the laser beams to be floodly emitted.

Optionally, the long baseline depth camera module is characterized in that the IR module is provided with a determining unit for determining whether the currently acquired depth data meets a preset requirement;

if yes, synthesizing the depth data with the RGB image;

if not, sending a signal to the projection module to switch the type of the projection light source.

Optionally, the long baseline depth camera module is characterized in that the RGB module is disposed adjacent to the IR module, and is configured to improve alignment quality between the depth map and the RGB image.

Optionally, the long baseline depth camera module is characterized in that the RGB module is located on a connection line between the projection module and the IR module, and is configured to improve an alignment quality between the depth map and the RGB image.

Optionally, the long baseline depth camera module is characterized in that a smaller included angle between a connection line between the projection module and the IR module and an edge of the long baseline depth camera module is 15 to 45 degrees.

The utility model also provides an electronic equipment, a serial communication port, including foretell any kind of long baseline degree of depth module of making a video recording.

Optionally, the electronic device is characterized in that the electronic device includes a mobile phone, an unmanned aerial vehicle, a robot, an automobile, and a ship.

Compared with the prior art, the utility model discloses following beneficial effect has:

the utility model discloses a structure is compacter, and space utilization is high, is adapted to the application scene in little space more. Under same transverse dimension, the utility model discloses a degree of depth camera module baseline is longer, can obtain the degree of depth data of higher accuracy to obtain more meticulous depth image data. The utility model discloses a length-width ratio of basic shell is littleer, and the structure is more stable to make the module more stable, be difficult for receiving external force to disturb. Because the utility model discloses a space utilization is higher, and the size of the module after the shaping is littleer, is favorable to being applied to more scenes.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the prior art descriptions will be briefly described below, it is obvious that the drawings in the following description are only embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts. Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

fig. 1 is a schematic block diagram of a long baseline depth camera module according to an embodiment of the present invention;

fig. 2 is a schematic structural diagram of a projection module according to an embodiment of the present invention;

fig. 3 is another schematic structural diagram of a projection module according to an embodiment of the present invention;

fig. 4 is a schematic structural diagram of a projection module according to an embodiment of the present invention;

FIG. 5 is a flowchart illustrating the operation of an IR module according to an embodiment of the present invention;

fig. 6 is a schematic structural diagram of a long baseline depth camera module according to an embodiment of the present invention;

fig. 7 is another schematic structural diagram of a long baseline depth camera module according to an embodiment of the present invention;

fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The present invention will be described in detail with reference to the following embodiments. The following examples will assist those skilled in the art in further understanding the present invention, but are not intended to limit the invention in any way. It should be noted that numerous variations and modifications could be made by those skilled in the art without departing from the spirit of the invention. These all belong to the protection scope of the present invention.

The terms "first," "second," "third," "fourth," and the like in the description and in the claims, as well as in the drawings, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the invention described herein may, for example, be implemented in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

The technical solution of the present invention will be described in detail with specific examples. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments.

The following describes the technical solution of the present invention and how to solve the above technical problems with specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.

Fig. 1 is the embodiment of the present invention provides a module diagram of a long baseline depth camera module. As shown in fig. 1, the utility model provides a pair of long baseline degree of depth camera module, including RGB module 101, IR module 102 and projection module 103.

The RGB module 101 is configured to capture an RGB image of the target object. The RGB module 101 may acquire RGB images using a CCD or CMOS. The principle of collection is to convert photons into electrons, where the number of photons is proportional to the number of electrons. Because the RGB images are obtained by means of bayer filters, their response to photons of different wavelengths is weaker than that of a grayscale camera, and their performance in many application scenarios is not sufficient to meet the requirements of the scenario, and they need to be enhanced from the dimension of depth.

The IR module 102 is configured to obtain depth data. The IR acquires infrared bands, and unlike the RGB module which acquires colors in a Bayer filter mode, the IR module acquires full-color images acquired over the infrared bands. The IR module 102 in this embodiment may obtain depth data based on the reflection of the projected lattice structured light, and may obtain depth data according to the time of flight of floodlight. The structure adopted by the IR module 102 is only required to satisfy the requirement of receiving the data required by the present invention, and the present invention is not limited to a specific structure.

And the projection module 103 is used for emitting lattice structure light or floodlight to the target object according to the IR module signal. The lattice structured light emits up to millions of projection rays from a camera head to a target object to form a three-dimensional image, can identify the object more accurately, and has a very obvious advantage particularly in short-distance identification. The pan light is a pulse wave of near infrared (-850 nm or 940 nm) emitted from a camera, the pulse wave is reflected back after encountering an object and is collected by a sensor, and depth data is obtained through frequency difference or time difference between the pulse waves, which is the working principle of the TOF (time of flight) technology. The projection module 103 can emit lattice structured light and flood light, so that the depth data can be acquired by adopting the lattice structured light in a short distance and the depth data can be acquired by adopting the flooding light in a long distance, and the application range of the module is wider.

The IR module 102 and the projection module 103 are arranged diagonally on the base housing to maximize the baseline therebetween. The base shell is a shell body for installing the camera module. The edge between the center point of the IR module 102 and the center point of the projection module 103 is called the baseline, and is an important parameter for acquiring depth data. The longer the baseline, the more accurate the acquired depth data. In the prior art, in order to ensure the maximization of the baseline, the IR module 102 and the projection module 103 are usually disposed at both ends of the base case in parallel with one side. However, since the base case is generally rectangular, the length of the diagonal line is greater than that of any side, so that the IR module 102 and the projecting module 103 are arranged diagonally on the base case in the present embodiment, so that the base line can be maximized within the range of the base case. The aspect ratio of the base shell is 3:1 to 1:1.

the utility model discloses a throw the module and can have multiple form. Fig. 2 is a schematic structural diagram of a projection module according to an embodiment of the present invention. And the projection module 103 is provided with a zoom projection lens and is used for projecting lattice structure light or floodlight to the target object through zooming. The projection module 103 includes a base 201, a light source 202, a micro lens unit 203, a main lens 204, and a diffractive optical element 205. The base 201 is used for fixing the light source 202. The shape of the base 201 is generally a regular shape such as a square, rectangle, hexagon, circle, etc. The base 201 may be a base that is separately disposed, or may be a base shell in the camera module, which is not limited in this embodiment. The light source 202 may be a large light source or a laser array composed of a plurality of sub-light sources. For the convenience of illustration, a laser array composed of 3 sub-light sources is shown in fig. 2. The light source type may be either vertical cavity surface laser (VCSEL) or edge-emitting laser. The direction of laser emission is perpendicular to the base 201. To arranging the direction of laser instrument, the utility model discloses do not do the restriction, as long as satisfy the requirement of laser emission direction can. In some embodiments, the laser array has a certain characteristic distribution, that is, the sub-light sources of the light source 202 are arranged non-uniformly.

The micro lens unit 203 primarily emits the light beam, has a small divergence degree, and can have a certain characteristic adding function, so that the pattern received by the IR module 102 has a higher recognition degree, and can be more accurately positioned. The main lens 204 is used to secondary condense the light beam for imaging, and may increase the diameter of the light beam. The position of the main lens 204 can be arbitrarily adjusted between the microlens unit 203 and the diffractive optical element 205, so that the light beam emitted from the projection module 103 can be switched between structured light and floodlight. The diffractive optical element 206 expands the incident light beam by several times to form a pattern with spots for projection. As the main lens 204 is farther away from the diffractive optical element 205, i.e. closer to the micro-lens unit 203, the larger the diameter of the light beam emitted from the main lens 204, the higher the degree of overlap between the different light beams, and thus closer to the floodlight, thus enabling switching between structured light and floodlight.

This embodiment adopts the mode of secondary imaging, once zooming to realize the switching between structured light and the floodlight, and the integrated level is high, is favorable to practicing thrift the space for the required space of throwing two kinds of laser is littleer, is favorable to the miniaturization and the integration of product, is applicable to the application scene that requires the height to the space. Meanwhile, as the related technology is mature, the product has high reliability and strong stability.

Fig. 3 is a schematic view of another structure of the projection module according to an embodiment of the present invention. The projection module 103 includes a discrete beam projector 301 and a surface light source projector 302.

A discrete beam projector 301 for projecting lattice structured light towards the target object. The shape of the lattice-structured light includes a line, a circle, a triangle, a hexagon, a quadrangle, a pentagon, a rectangle, a square, etc., although other identification techniques, such as a default beam, etc., may be used. The discrete beam projector 301 may be a single source projector or an array of multiple sub-source projectors. The lattice structure light projected by the discrete light beam projector 302 has pattern codes, so that the information amount of the IR module 102 after acquiring the signal can be increased, and the identification degree of the IR module can be improved.

A surface light source projector 302 for projecting floodlight towards the target object. The surface light source projector 302 projects laser pulses, so that the time delay of the IR module 102 receiving different pulses is calculated to obtain the laser flight time, thereby obtaining distance information, and obtaining depth data of the target object through the time difference or frequency difference of a plurality of different light beams. The light source of the area light source projector 302 may be various, including an LED lamp, an infrared light source, and the like. The signal that area source projector 302 sent is pulse signal, utilizes the time difference of repeated pulse signal to realize the range finding, and is more reliable in suitable scene.

In this embodiment, a scheme of respectively projecting lattice structured light and floodlight is adopted, and a signal of the IR receiving module 102 is received to control the discrete beam projector 301 or the surface light source projector 302 to operate, so as to control the laser projection type and achieve synchronization of light projection and reception. The light sources are respectively arranged in the embodiment, the structure of each submodule is more stable, the technology is more mature, and the submodule can be rapidly applied.

Fig. 4 is a schematic structural diagram of a projection module according to an embodiment of the present invention. The projection module 103 includes a laser array 401, a collimating element 402, a beam splitter 403, and a diffuser 404.

And a laser array 401 for emitting lattice laser. The laser array 401 is a laser array constituted by a plurality of sub light sources. The light source type may be either vertical cavity surface laser (VCSEL) or edge-emitting laser. The laser light emitted by the laser array 401 is directed perpendicularly towards the collimating element 402.

And a collimating element 402 for collimating the incident lattice laser light to generate a collimated light beam. The collimated light beams are closer to parallel, the diameters of the light beams are almost kept unchanged in the process of propagation, and light beam projection and information carrying identification can be better carried out.

A light splitting device 403 for splitting the incident collimated light beam into a plurality of laser beams. The order of magnitude of the split laser light is 1-2 times that of the laser light before splitting.

And the diffuser 404 is used for diffusing the laser beams according to the signal of the IR module so as to flood and emit the laser beams. When the signal from the IR module 102 indicates that the structured light is coming out, the diffuser 404 acts as a general lens and has no significant effect on the light beam. When the signal from the IR module 102 indicates flood light, the diffuser 404 acts as a convex lens, spreading the light beam and creating flood light at a distance from the diffuser 404. Preferably, the diffuser 404 is a liquid lens.

The embodiment integrates the structured light scheme and the TOF scheme, can realize the projection of two light sources in a smaller space, has a compact structure, and is favorable for the miniaturization arrangement of the structure. Simultaneously, this embodiment collimates the light beam for the structured light beam is more stable, and it is more reliable to carry information, promotes the stability of module.

Referring to fig. 5, a flowchart of the steps of the IR module according to an embodiment of the present invention is shown. The IR module 102 is provided with a determining unit for determining whether the currently acquired depth data meets a preset requirement. The working steps are as follows:

and S501, acquiring depth data.

In this step, the IR module 102 uses different data processing methods according to different light source types, so as to obtain corresponding depth data.

S502: and judging whether the depth data meet preset requirements or not.

In this step, since a better depth map can be obtained only by the depth data satisfying a certain requirement, the depth data needs to be filtered and identified. This step is used to determine whether the currently acquired depth data meets a preset requirement. If the current depth data meets the preset requirement, executing step S504; if the preset requirement is not satisfied, step S503 is performed.

S503, switching the projection light source type.

In this step, a signal is sent to the projection module 103 to switch the projection type. If the current projection light source type is lattice structured light, switching to flood light; and if the current projection light source type is floodlight, switching to form lattice structured light.

S504: and synthesizing the depth data with the RGB image.

In this step, the depth data meeting the preset requirements and the RGB image are synthesized into a depth map.

The embodiment determines whether the light source needs to be replaced or not through the identification of the depth data, and the judgment is more direct. Compared with a mode of switching the projection light sources depending on the distance, the method is more reliable, potential interference is avoided, data are more stable, meanwhile, the judgment on whether the light sources need to be switched or not can be achieved only through one projection light source, and the light source setting and the working time are saved.

Please refer to fig. 6, which illustrates a schematic structural diagram of a long baseline depth camera module according to an embodiment of the present invention. The RGB module 101 is disposed adjacent to the IR module 102 for improving the alignment quality of the depth map and the RGB image. The connecting line of the center points of the RGB module 101 and the IR module 102 is parallel to one side of the base housing. The diagonal arrangement of the projection module 103 and the IR module 102 maximizes the length of the baseline. And a small included angle between a connecting line of the projection module and the IR module and the edge of the long baseline depth camera module is 15-45 degrees.

This embodiment arranges IR module and RGB module along basic shell edge to arrange the IR module and throw the module along the diagonal, when making the baseline maximize, also make the main part of making a video recording be located the edge, thereby make the components and parts on the basic shell conveniently arrange more, be favorable to improving the integrated level of components and parts on the basic shell, be favorable to the miniaturization of module, integrate.

Please refer to fig. 7, which shows another schematic structural diagram of a long baseline depth camera module according to an embodiment of the present invention. The RGB module 101 is disposed adjacent to the IR module 102 for improving the alignment quality of the depth map and the RGB image. The center points of the RGB module 101, the IR module 102 and the projection module 103 are located on the same straight line. The diagonal arrangement of the projection module 103 and the IR module 102 maximizes the length of the baseline.

This embodiment arranges the RGB module and IR module adjacent to being located same straight line with projecting the module, make the RGB image that obtains of shooing and depth data's visual angle almost the same, during contract depth map, the alignment effect is better, guarantees to the at utmost and shoots the quality.

Fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. The electronic device 801 shooting end is provided with a depth camera module 802. The depth camera module 802 is any one of the above embodiments. For simplicity, the same arrangement as the depth camera module of fig. 7 is illustrated in fig. 8. According to different devices, the position of the depth camera module 802 may be different, and may be located at the edge or the middle. The electronic device in fig. 8 may be a cell phone, a scanner, an unmanned aerial vehicle, a robot, an automobile, a ship, etc.

Components of the electronic device 801 may include, but are not limited to: at least one processing unit, at least one memory unit, a bus connecting different platform components (including memory unit and processing unit), a display unit, etc.

The memory unit stores program code that can be executed by the processing unit.

The bus may be any of several types of bus structures including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The display unit may be configured to display the depth map acquired in any of the foregoing embodiments, thereby completing the display of the depth map. The display unit may be integrated on the electronic device 801 or may be externally connected, and the electronic device 801 provides a corresponding interface. The interface can be a plurality of interface types such as HDMI, USB and the like.

The electronic device may also communicate with one or more external devices (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device to communicate with one or more other computing devices. Such communication may be through an input/output (I/O) interface. Also, the electronic device may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet) via a network adapter. The network adapter may communicate with other modules of the electronic device over the bus.

In the present specification, the embodiments are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other. The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description of the specific embodiments of the invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by those skilled in the art within the scope of the appended claims without departing from the spirit of the invention.

Claims (9)

1. A long baseline depth camera module is characterized by comprising an RGB module, an IR module and a projection module;

the RGB module is used for shooting an RGB image of a target object;

the IR module is used for acquiring depth data;

the projection module is used for emitting lattice structure light or floodlight to the target object according to the IR module signal;

the IR module and the projection module are arranged in a diagonal line on the base shell so as to maximize a base line between the IR module and the projection module;

the projection module is provided with a zooming projection lens and is used for projecting lattice structure light or floodlight to a target object through zooming;

the IR module can obtain depth data based on reflection of the projected lattice structured light and can obtain depth data according to the flight time of floodlight.

2. A long baseline depth camera module is characterized by comprising an RGB module, an IR module and a projection module;

the RGB module is used for shooting an RGB image of a target object;

the IR module is used for acquiring depth data;

the projection module is used for emitting lattice structure light or floodlight to the target object according to the IR module signal;

the IR module and the projection module are arranged on the base shell in a diagonal manner, so that a base line between the IR module and the projection module is the maximum;

the projection module comprises a discrete light beam projector and a surface light source projector;

the discrete light beam projector is used for projecting lattice structured light to the target object;

the surface light source projector is used for projecting floodlight to the target object;

the IR module can obtain depth data based on reflection of the projected lattice structured light and depth data according to flight time of floodlight.

3. The long-baseline depth camera module according to claim 2, wherein the IR module is provided with a determining unit for determining whether currently acquired depth data meets a preset requirement;

if yes, synthesizing the depth data with the RGB image;

if not, sending a signal to the projection module to switch the type of the projection light source.

4. A long baseline depth camera module is characterized by comprising an RGB module, an IR module and a projection module;

the RGB module is used for shooting an RGB image of a target object;

the IR module is used for acquiring depth data;

the projection module is used for emitting lattice structure light or floodlight to the target object according to the IR module signal;

the IR module and the projection module are arranged in a diagonal line on the base shell so as to maximize a base line between the IR module and the projection module;

the projection module comprises a laser array, a collimation element, a light splitting device and a diffuser;

the laser array is used for emitting dot matrix laser;

the collimating element is used for collimating the incident dot matrix laser to generate a collimated light beam;

the beam splitting device is used for splitting the incident collimated light beam into a plurality of laser beams, and the order of magnitude of the split laser beams is 1-2 times that of the laser beams before beam splitting;

the diffuser is used for diffusing the laser beams according to the signal of the IR module so as to enable the laser beams to be subjected to floodlight emergence;

the IR module is provided with a judging unit for judging whether the currently acquired depth data meets the preset requirement or not;

if yes, synthesizing the depth data with the RGB image;

if not, sending a signal to the projection module to switch the type of the projection light source.

5. The long baseline depth camera module of claim 4, wherein the RGB module is disposed adjacent to the IR module for improving alignment quality of the depth map with the RGB image.

6. The long baseline depth camera module of claim 5, wherein the RGB module is located on a line connecting the projection module and the IR module, and is configured to improve the quality of alignment between the depth map and the RGB image.

7. The long baseline depth camera module of claim 6, wherein a smaller included angle between a connection line between the projection module and the IR module and an edge of the long baseline depth camera module is 15-45 degrees.

8. An electronic device comprising a long baseline depth camera module of any one of claims 1-7.

9. The electronic device of claim 8, wherein the electronic device comprises a mobile phone, a drone, a robot, an automobile, a ship.

Images