WO2021115296A1

WO2021115296A1 - Ultra-thin three-dimensional capturing module for mobile terminal

Info

Publication number: WO2021115296A1
Application number: PCT/CN2020/134753
Authority: WO
Inventors: 左忠斌; 左达宇
Original assignee: 左忠斌
Priority date: 2019-12-12
Filing date: 2020-12-09
Publication date: 2021-06-17
Also published as: CN115112017A; CN111060009A

Abstract

An ultra-thin three-dimensional capturing module for a mobile terminal comprising a data interface (1), a motion drive device (2), a motion device (3), and an image capturing device (4). The image capturing device (4) is disposed on the motion device (3), and is movable relative to the mobile terminal during an image capturing process. The motion drive device (2) is connected to the motion device (3). The motion drive device (2) is electrically connected to the mobile terminal via the data interface (1). The image capturing device (4) is electrically connected to the mobile terminal via the data interface (1). An optical axis of the image capturing device (4) and a motion plane of the image capturing device (4) have an included angle γ. Motion of the image capturing device (4) reduces the number of cameras used. The invention enables an external connection of the mobile terminal, and facilitates adding a new 3D capturing function to existing mobile phones. The entire apparatus is movable and easy to use outdoors. The invention uses an external connection, such that modifications of existing mobile phones are not required, thereby providing general applicability and lowering costs.

Description

Ultra-thin three-dimensional acquisition module for mobile terminal

Technical field

The present invention relates to the technical field of object collection, in particular to the technical field of using a camera to perform three-dimensional collection of a target in a mobile terminal.

Background technique

Currently, common 3D acquisition methods include structured light method and laser scanning method, but these methods all require a light source and a beam shaping system, which are costly, consume a lot of power, and take up a lot of space.

At present, mobile phones usually have 1-3 cameras to achieve some special shooting effects, such as background blur. However, there is no camera system that can be used for 3D acquisition on mobile phones. If only the current camera system is used, it is difficult to perform 3D stitching due to the limited shooting angle, and 3D images cannot be obtained. If you want to increase the shooting angle and increase the redundancy of the captured images, you need to set up multiple cameras. For example, the Digital Emily project of the University of Southern California uses a spherical bracket to fix hundreds of cameras at different positions and angles on the bracket. This conventional system that uses image acquisition equipment for 3D acquisition is difficult to use on small-sized mobile terminal devices such as mobile phones.

At the same time, there are also mobile phones that directly use the camera on the phone to take images of multiple angles of the target and then stitch them together. However, this kind of movement either requires the phone to be installed on an additional track, or it is free movement without tracks. The former limits the usage scenarios, while the latter leads to a decrease in the quality of the collection.

At present, there are also cameras that can be rotated on mobile phones, which are usually driven manually or electrically, but their purpose is to take pictures of corresponding angles, not to scan, and it is impossible to synthesize 3D models.

Moreover, the current use of mobile phones for 3D acquisition is usually limited to human faces. However, with the popularity of mobile phone applications, there is no corresponding solution to the 3D acquisition of other objects, especially distant objects. At present, the use of mobile phones for 3D acquisition usually requires the mobile phone to rotate around the target (with or without track), or the camera to rotate around the target. But obviously this does not apply to distant objects. The rotating parts arranged in the housing also increase the volume of the module, which is not conducive to the miniaturization of the equipment. For some applications, a 360° three-dimensional model of the target is not needed, but only a three-dimensional model within the line of sight, for example, only a front and partial side three-dimensional model of the park landscape is required. There is no suitable solution to this problem.

In the prior art, it has also been proposed to use an empirical formula including rotation angle, target size, and object distance to limit the camera position, so as to take into account the synthesis speed and effect. However, in practical applications, it is found that unless there is a precise angle measuring device, the user is not sensitive to the angle, and it is difficult to accurately determine the angle; the size of the target is difficult to accurately determine, especially in some applications where the target needs to be replaced frequently, and the tape is measured every time A lot of extra work is required, and professional equipment is required to accurately measure irregular targets. For example, for a building, it is sometimes not easy to know its length, and professional equipment is required. The measurement error leads to the camera position setting error, which will affect the acquisition and synthesis speed and effect.

In the prior art, mobile terminals usually have cameras, but these cameras do not move during the shooting process. The camera is usually hidden through movement before and after the opening. Since they do not move relative to the target during the shooting process, they can only shoot 2D images, and the captured images cannot be synthesized into 3D. Therefore, there is an urgent need in this field for high-quality, low-cost, and small-volume 3D acquisition devices that can be applied to mobile terminals.

Summary of the invention

In view of the above-mentioned problems, the present invention is proposed to provide an ultra-thin three-dimensional acquisition module for mobile terminals that overcomes the above-mentioned problems or at least partially solves the above-mentioned problems.

The invention provides an ultra-thin three-dimensional acquisition module for a mobile terminal, which includes a data interface, a motion drive device, a motion device and an image acquisition device;

The image acquisition device is arranged on the movement device, and the image acquisition device moves relative to the mobile terminal during the image acquisition process;

The motion drive device is connected to the motion device;

The motion drive device is electrically connected to the mobile terminal through the data interface;

The image acquisition device is electrically connected to the mobile terminal through a data interface;

The optical axis of the image acquisition device and the motion plane of the image acquisition device have an included angle γ.

Optionally, γ=90° or 0<γ<90° or 180°>γ>90°.

Optionally, when the image acquisition device is in different positions, the optical axis converges or diverges with respect to the vertical line of the motion plane of the image acquisition device.

Optionally, the module and the mobile terminal are independent of each other or embedded in the mobile terminal.

Optionally, there are multiple image acquisition devices.

Optionally, the image acquisition device includes a visible light image acquisition device and/or an infrared image acquisition device.

Optionally, the image acquisition device extends out of the module housing.

Optionally, the area where the image acquisition device moves further includes a light-transmitting shell part.

Optionally, the motion device is a turntable, a turntable, a curved guide rail, and a linear guide rail.

Optionally, the acquisition position of the image acquisition device is: two adjacent acquisition positions of the image acquisition device meet the following conditions:

μ<0.482

Where L is the linear distance between the optical centers of the image acquisition device at two adjacent acquisition positions; f is the focal length of the image acquisition device; d is the rectangular length of the photosensitive element (CCD) of the image acquisition device; M is the photosensitive element of the image acquisition device along the light The distance from the axis to the surface of the target; μ is the empirical coefficient.

The present invention also provides a three-dimensional acquisition method for a mobile terminal, and the image acquisition device is arranged in the mobile terminal,

The image acquisition device moves relative to the mobile terminal during the acquisition process, thereby capturing images of the target object at different positions;

The optical axis of the image acquisition device and the motion plane of the image acquisition device have an included angle γ, 0<γ<180°.

Invention points and technical effects

1. For the first time, a device structure that can apply the principle of image splicing for 3D acquisition in a mobile terminal is proposed.

2. Reduce the number of cameras used by moving the image acquisition device.

3. It can realize the external connection of the mobile terminal, which is convenient for adding new 3D collection function to the existing mobile phone.

4. The whole device can be moved, which is convenient for outdoor use.

5. The external connection mode is adopted, and there is no need to modify the existing mobile phones, which is more versatile and lower in cost.

6. It is collected in a way that the optical axis and the rotating plane are at a certain angle, which is smaller in size and more suitable for distant targets.

7. Optimize the position of the camera while improving the detection accuracy and speed. And when optimizing the position, there is no need to measure the angle, no need to measure the target size, and the applicability is stronger.

Description of the drawings

By reading the detailed description of the preferred embodiments below, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only used for the purpose of illustrating the preferred embodiments, and are not considered as a limitation to the present invention. Also, throughout the drawings, the same reference symbols are used to denote the same components. In the attached picture:

FIG. 1 is a schematic structural diagram of an implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

2 is a schematic structural diagram of another implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

3 is a schematic structural diagram of a third implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

4 is a schematic structural diagram of a fourth implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

5 is a schematic structural diagram of a fifth implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

6 is a schematic structural diagram of a sixth implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

FIG. 7 is a schematic structural diagram of a seventh implementation manner of a three-dimensional acquisition module in an embodiment of the present invention;

1 data interface, 2 movement drive device, 3 movement device, 4 image acquisition device, 5 angle adjustment device.

Detailed ways

Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the drawings show exemplary embodiments of the present disclosure, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

Mobile terminal module structure-translation and rotation type

In order to solve the above technical problem, an embodiment of the present invention provides a three-dimensional acquisition module for a mobile terminal. As shown in Figs. 1-7, it specifically includes: a data interface 1, a motion driving device 2, a motion device 3, and an image acquisition device 4.

The image acquisition device 4 is arranged on the movement device 3. The motion device 3 can be a sliding table and a circular track, the image acquisition device 4 is installed on the sliding table, or the housing of the image acquisition device 4 is directly installed on the guide rail as a sliding table, or the housing of the image acquisition device 4 and the module housing A sliding fit is formed with each other to realize the translation of the image acquisition device 4 on the guide rail. The motion driving device 2 is connected to the motion device 3, and can drive a sliding table or directly drive the housing of the image acquisition device 4 to move. For the lead screw or the tooth-engaged guide rail, the corresponding structure can also be driven to make the image acquisition device 4 translate. In other words, the image acquisition device 4 does not rely on manual movement, but is driven and moved according to the purpose of the acquisition, and has certain requirements for the acquisition position, and needs to meet the empirical formula setting (detailed below), so that it can Ensure the accuracy of 3D information collected. If you only rely on the customer's manual movement, it will cause uneven and incomplete image information collection, and it will even be difficult to match and stitch into a 3D image. At the same time, it does not rely on moving the entire mobile phone to realize image acquisition, because this kind of movement either requires the mobile phone to be installed on an additional track, or it is free movement without a track. The former limits the usage scenarios, while the latter leads to a decrease in the quality of the collection.

The guide rail is curved, such as a circular arc, as shown in Figs. 4 and 5, so that when the image acquisition device 4 moves on it, the movement track is arc-shaped. However, the direction of the optical axis of the image acquisition device 4 remains unchanged, that is, the guide rails are arranged parallel to the light emitting surface, so that the image acquisition rotating surface is approximately parallel to the target surface. For example, when using a mobile phone to photograph a 3D model of the opposite building, the rotating surface of the image acquisition device 4 is parallel to the height direction of the building, rather than parallel to the cross section of the building.

The guide rail is linear, as shown in FIG. 6, so that when the image acquisition device 4 moves on it, the movement track is a straight line, so as to realize the scanning of the target. At this time, the direction of the optical axis of the image acquisition device 4 is also unchanged, only the translation of the direction of the optical axis. In particular, there are two guide rails at this time, each of which is provided with an image acquisition device 4, which respectively translates along the corresponding guide rail.

There may be multiple image acquisition devices 4, and each image acquisition device 4 moves along a single guide rail, and the motion track is similar to the above. For example, two image acquisition devices 4 can be set to move along the upper and lower guide rails respectively, so that the acquisition range can be expanded, and at the same time, more pictures can be acquired per unit time, which is more efficient. Of course, for special needs, the two image acquisition devices 4 may be cameras of different wavelength bands, such as infrared wavelength band and visible light wavelength band. At the same time, it is also possible to run multiple image acquisition devices 4 on one rail. For example, two image acquisition devices running side by side on a single rail can also improve efficiency.

In one case, the image acquisition device 4 is exposed outside the housing of the acquisition module, that is, the housing of the acquisition module has a corresponding groove, and the image acquisition device 4 protrudes from the groove, as shown in FIGS. 2 and 3. Of course, it can be further designed. The image capture device 4 can be extended out of the groove when needed, and retracted into the housing when it is not working. In addition, the groove has a cover, which can close the groove when the image capture device 4 is retracted to avoid dust.

In one case, as shown in FIG. 1, on the movement track of the image acquisition device 4, the housing of the acquisition module opposite to the image acquisition device is made of a transparent material. In this way, the image acquisition device 4 can directly perform motion acquisition without extending the housing. This is beneficial to waterproof and dustproof.

Since the motion drive device 2 is connected to the motion device 3, the image acquisition device 4 is driven to translate according to the predetermined requirements of 3D acquisition. Therefore, the motion drive device 2 needs to have a data interface 1 to receive the corresponding motion instructions, that is, the motion drive device 2 passes the data The interface 1 is electrically connected to the mobile terminal.

It also includes a processor, also called a processing unit, which is used to synthesize a 3D model of the target object according to a 3D synthesis algorithm according to a plurality of images collected by the image acquisition device to obtain 3D information of the target object.

Mobile terminal module structure-fixed

The module includes a housing, a plurality of image acquisition devices 4 are distributed in the housing, and the separation distance of the image acquisition devices 4 is limited by the following empirical conditions. The lens of the image capture device is exposed outside the module housing or is located in the module housing.

When the lens of the image acquisition device 4 is exposed outside the module housing, a corresponding protection mechanism is provided to protect the lens. For example, a transparent cover is provided. When the lens of the image acquisition device 4 is located in the module housing, the housing of the acquisition module opposite to the image acquisition device 4 is made of transparent material.

The direction of the optical axis of the image capture device

The optical axis of the image acquisition device 4 and the housing of the mobile terminal have an included angle γ. Normally, γ=90°. In the above solution for the movement of the image acquisition device 4, the included angle between the optical axis and the moving plane is the included angle with the terminal housing, which is usually 90°. In a fixed solution, the angle between the optical axis of the image acquisition device 4 and the housing of the mobile terminal is usually 90°.

In some cases, γ<90°, that is, when the image capture device 4 is in different positions, the optical axis converges toward the vertical direction relative to the moving plane or the housing.

In some cases, γ>90°, that is, when the image acquisition device 4 is in different positions, the optical axis diverges in a direction perpendicular to the moving plane or the housing.

Either way, 0<γ<180°.

The above-mentioned situation that γ is not equal to 90° can be realized in the following ways: as shown in Figure 7, ① the image acquisition device 4 is fixed on the movement device 3 at an angle of γ; ② the image acquisition device 4 is set on the angle adjustment device 5, which can be The movement position changes, changing γ; ③The image acquisition device 4 is fixed on the module or the housing. The above-mentioned angle adjusting device 5 may be a turntable.

Module and mobile terminal connection

In one embodiment, the entire module is an external type. At this time, the data interface can be an interface that cooperates with Type-c interface, MicroUSB interface, USB interface, Lightning interface, wifi interface, Bluetooth interface, and cellular network interface. Wired or wirelessly connected to the mobile terminal.

In another embodiment, the entire module is built-in. In this case, the data interface 1 can be directly connected to the processor of the mobile terminal internally.

In another embodiment, the structure of the module is a part of the mobile phone, that is, although the present invention is described as a module, in fact, these structures already belong to a part of the mobile phone and are already completed when the mobile phone is produced and manufactured.

In order to reduce the volume and power consumption of the entire module, the image acquisition device 4 is electrically connected to the mobile terminal through the data interface 1 so as to transmit the collected images to the mobile terminal for storage and subsequent 3D processing.

Whether it is external or internal, there is a mechanical connection between the module and the mobile terminal. For example, in the external type, the module is inserted into the earphone jack of the mobile terminal through the earphone plug. Since the module and the mobile terminal must transmit control signals and image data to each other, in addition to the mechanical connection, there is also an electrical connection, especially a signal connection.

In the external type, the mechanical connection and the electrical connection are realized by the same structure. The mobile phone module is connected to the mobile phone through a mechanical connector/electrical connector, and the mobile phone module and the mobile phone are relatively rigidly connected, so that the two are integrated. For example, the earphone plug described above is inserted into the earphone jack of the mobile terminal, and the mechanical connection and the electrical connection are realized at the same time. Both the module and the mobile phone can be rigidly fixed to each other, and signals can be transmitted between each other. The mechanical connection can also utilize additional mechanical connections. For example, additional plugs and jacks, protrusions and card slots are provided between the module and the mobile phone to achieve a rigid and fixed connection between the module and the mobile phone. Of course, you can also use the existing sockets of the mobile phone. For example, the module has a headphone plug, a microUSB plug, a TepyC plug, and a Lightning plug, which are correspondingly inserted into the corresponding sockets of the mobile phone, but this kind of insertion is only used for mechanical connection, not For signal transmission, the signal is connected by other means. Through such a mechanical connection, the module and the mobile phone are integrated. When the user holds the mobile phone in a fixed position, the module can be fixed relative to the target object, and pictures from different angles can be taken by the movement of the image acquisition device 4.

In order to facilitate the translation or rotation of the image acquisition device 4, the movement device 3 may include a magnetic levitation device, so that the movement process is smoother and the user experience is improved.

The image capture device 4 moves in the housing of the module, and the housing part involved in the movement area is made of a transparent material, such as a transparent resin material.

The image acquisition device 4 may be a visible light camera/camera module or an infrared camera/camera module. When collecting at night, due to the limited light, the visible light camera will not be able to collect the image completely. At this time, the infrared camera can be used for collection, and in the subsequent processing, the images collected by the visible light camera and the infrared camera can be matched and fused with each other to realize 3D information collection. Of course, it is also possible to rely on only one of the visible light camera or the infrared camera. In addition, there may be more than one image acquisition device 4.

In a solution with an infrared camera, the infrared camera and the visible light camera can be side by side in the track. It is also possible to set up two tracks to install an infrared camera and a visible light camera respectively. And you can also use a single camera with a wider spectral sensing range, while taking into account visible light cameras and infrared cameras.

The shell of the module has a light source, and the light source is an LED lamp bead, but a smart light source can also be set, for example, different light source brightness, brightness, etc. can be selected according to needs. The light source is used to illuminate the target to prevent the target from being too dark to affect the collection effect and accuracy. But at the same time, it is necessary to prevent the light source from being too bright, causing the loss of target texture information. The light source can also use the mobile terminal's own light source to illuminate the part to be scanned.

In order to improve the user experience, the images collected by the module can be transmitted to the display module of the mobile terminal for display, so as to facilitate the user to observe their own collection process. Especially for the acquisition module that is too far or too close to the target, it can be displayed through the display module, and can be reminded through the voice module. It can be understood that the image collected by the module may not be displayed in the display module of the mobile terminal, but the information that it is too far or too close to the target object can be voiced through the mobile terminal to prompt the user to move. The connection between the module and the voice or display module of the mobile terminal is also realized through the data interface 1 of the module.

Image capture device capture location

When performing 3D acquisition, the optical axis direction of the image acquisition device does not change relative to the target at different acquisition positions, and is usually roughly perpendicular to the surface of the target (it can also be at a certain angle). At this time, the positions of two adjacent image acquisition devices , Or the two adjacent collection positions of the image collection device meet the following conditions:

μ<0.482

When the above two positions are along the length direction of the photosensitive element of the image capture device, d takes the length of the rectangle; when the above two positions are along the width direction of the photosensitive element of the image capture device, d is the width of the rectangle.

When the image capture device is in any one of the above two positions, the distance from the photosensitive element to the surface of the target along the optical axis is taken as M.

As mentioned above, L should be the linear distance between the optical centers of the two image capture devices, but because the position of the optical centers of the image capture devices is not easy to determine in some cases, the photosensitive of the image capture devices can also be used in some cases. The center of the component, the geometric center of the image capture device, the center of the axis connecting the image capture device and the pan/tilt (or platform, bracket), the center of the proximal or distal lens surface instead of Within the acceptable range, therefore, the above-mentioned range is also within the protection scope of the present invention.

Using a commercially available mobile phone camera module and using the device of the present invention, experiments were carried out, and the following experimental results were obtained.

From the above experimental results and a large amount of experimental experience, it can be concluded that the value of μ should satisfy μ<0.482. At this time, some 3D models can be synthesized. Although some of them cannot be synthesized automatically, it is acceptable if the requirements are not high. And you can make up for the parts that cannot be synthesized manually or by replacing the algorithm. In particular, when the value satisfies μ<0.403, the balance between synthesis effect and synthesis time can be optimally taken into account; in order to obtain a better synthesis effect, μ<0.326 can be selected, and the synthesis time will increase at this time, but the synthesis quality will be better. When μ>0.485, synthesis is no longer possible. However, it should be noted here that the above scope is only the best embodiment and does not constitute a limitation on the protection scope.

The above data is only obtained from experiments to verify the conditions of the formula, and does not limit the invention. Even without these data, it does not affect the objectivity of the formula. Those skilled in the art can adjust the equipment parameters and step details as needed to perform experiments, and obtain other data that also meets the conditions of the formula.

The adjacent acquisition positions in the present invention refer to two adjacent positions on the moving track where the acquisition action occurs when the image acquisition device moves relative to the target. This is usually easy to understand for the movement of the image capture device. However, when the target object moves to cause the two to move relative to each other, at this time, the movement of the target object should be converted into the target object's immobility according to the relativity of the movement, and the image acquisition device moves. At this time, measure the two adjacent positions of the image acquisition device where the acquisition action occurs in the transformed movement track.

3D synthesis method

When using the above-mentioned collected pictures for 3D synthesis, the existing algorithm can be used to realize it, or the optimized algorithm proposed by the present invention can be used, which mainly includes the following steps:

Step 1: Perform image enhancement processing on all input photos. The following filters are used to enhance the contrast of the original photo and suppress noise at the same time.

Where: g(x, y) is the gray value of the original image at (x, y), f(x, y) is the gray value of the original image _{after being enhanced by the Wallis filter, and m g is the local gray value of the original image} Degree mean, s _g is the local gray-scale standard deviation of the original image, m _f is the local gray-scale target value _{of the transformed image, and s f} is the local gray-scale standard deviation target value of the transformed image. c∈(0,1) is the expansion constant of the image variance, and b∈(0,1) is the image brightness coefficient constant.

The filter can greatly enhance the image texture patterns of different scales in the image, so the number and accuracy of feature points can be improved when extracting the point features of the image, and the reliability and accuracy of the matching result can be improved in the photo feature matching.

Step 2: Perform feature point extraction on all input photos, and perform feature point matching to obtain sparse feature points. The SURF operator is used to extract and match the feature points of the photos. The SURF feature matching method mainly includes three processes, feature point detection, feature point description and feature point matching. This method uses Hessian matrix to detect feature points, uses Box Filters to replace second-order Gaussian filtering, and uses integral images to accelerate convolution to increase the calculation speed and reduce the dimensionality of local image feature descriptors. To speed up the matching speed. The main steps include ① constructing the Hessian matrix to generate all points of interest for feature extraction. The purpose of constructing the Hessian matrix is to generate stable edge points (mutation points) of the image; ② constructing the scale space feature point positioning, which will be processed by the Hessian matrix Compare each pixel point with 26 points in the neighborhood of two-dimensional image space and scale space, and initially locate the key points, and then filter out the key points with weaker energy and the key points that are incorrectly positioned to filter out the final stable ③The main direction of the feature point is determined by using the Harr wavelet feature in the circular neighborhood of the statistical feature point. That is, in the circular neighborhood of the feature point, the sum of the horizontal and vertical harr wavelet features of all points in the 60-degree fan is counted, and then the fan is rotated at an interval of 0.2 radians and the harr wavelet eigenvalues in the area are counted again. The direction of the sector with the largest value is taken as the main direction of the feature point; ④ Generate a 64-dimensional feature point description vector, and take a 4*4 rectangular area block around the feature point, but the direction of the obtained rectangular area is along the main direction of the feature point. direction. Each sub-region counts 25 pixels of haar wavelet features in the horizontal and vertical directions, where the horizontal and vertical directions are relative to the main direction. The haar wavelet features are 4 directions after the horizontal direction value, after the vertical direction value, after the horizontal direction absolute value, and the vertical direction absolute value. These 4 values are used as the feature vector of each sub-block area, so there is a total of 4* 4*4=64-dimensional vector is used as the descriptor of Surf feature; ⑤Feature point matching, the degree of matching is determined by calculating the Euclidean distance between two feature points. The shorter the Euclidean distance, the better the matching degree of the two feature points. .

Step 3: Input the matching feature point coordinates, use the beam method to adjust, solve the sparse face 3D point cloud and the position and posture data of the camera, that is, obtain the sparse face model 3D point cloud and the position model coordinate value ; With sparse feature points as initial values, dense matching of multi-view photos is performed to obtain dense point cloud data. The process has four main steps: stereo pair selection, depth map calculation, depth map optimization, and depth map fusion. For each image in the input data set, we select a reference image to form a stereo pair for calculating the depth map. Therefore, we can get rough depth maps of all images. These depth maps may contain noise and errors. We use its neighborhood depth map to check consistency to optimize the depth map of each image. Finally, depth map fusion is performed to obtain a three-dimensional point cloud of the entire scene.

Step 4: Use the dense point cloud to reconstruct the face surface. Including the process of defining the octree, setting the function space, creating the vector field, solving the Poisson equation, and extracting the isosurface. The integral relationship between the sampling point and the indicator function is obtained from the gradient relationship, and the vector field of the point cloud is obtained according to the integral relationship, and the approximation of the gradient field of the indicator function is calculated to form the Poisson equation. According to the Poisson equation, the approximate solution is obtained by matrix iteration, the moving cube algorithm is used to extract the isosurface, and the model of the measured object is reconstructed from the measured point cloud.

Step 5: Fully automatic texture mapping of the face model. After the surface model is built, texture mapping is performed. The main process includes: ①The texture data is obtained through the image reconstruction target's surface triangle grid; ②The visibility analysis of the reconstructed model triangle. Use the image calibration information to calculate the visible image set of each triangle and the optimal reference image; ③The triangle surface clustering generates texture patches. According to the visible image set of the triangle surface, the optimal reference image and the neighborhood topological relationship of the triangle surface, the triangle surface cluster is generated into a number of reference image texture patches; ④The texture patches are automatically sorted to generate texture images. Sort the generated texture patches according to their size relationship, generate the texture image with the smallest enclosing area, and obtain the texture mapping coordinates of each triangle.

It should be noted that the above-mentioned algorithm is an optimized algorithm of the present invention, and this algorithm cooperates with the image acquisition conditions, and the use of this algorithm takes into account the time and quality of synthesis, which is one of the invention points of the present invention. Of course, the conventional 3D synthesis algorithm in the prior art can also be used, but the synthesis effect and speed will be affected to a certain extent.

The above-mentioned target object, target object, and object all represent objects for which three-dimensional information is pre-acquired. It can be a physical object, or it can be a combination of multiple objects. For example, it can be a vehicle, a large sculpture, etc. The three-dimensional information of the target includes a three-dimensional image, a three-dimensional point cloud, a three-dimensional grid, a local three-dimensional feature, a three-dimensional size, and all parameters with a three-dimensional feature of the target. The so-called three-dimensional in the present invention refers to three-dimensional information of XYZ, especially depth information, which is essentially different from only two-dimensional plane information. It is also essentially different from the definitions called three-dimensional, panoramic, holographic, and three-dimensional, but actually only include two-dimensional information, especially depth information.

The collection area mentioned in the present invention refers to the range that an image collection device (such as a camera) can shoot. The image acquisition device in the present invention can be CCD, CMOS, camera, video camera, industrial camera, monitor, camera, mobile phone, tablet, notebook, mobile terminal, wearable device, smart glasses, smart watch, smart bracelet and belt All devices with image capture function.

The 3D information of multiple regions of the target obtained in the above embodiment can be used for comparison, for example, for identity recognition. First, use the solution of the present invention to obtain 3D information of the human face and iris, and store it in the server as standard data. When in use, such as when identity authentication is required for payment, door opening, etc., the 3D acquisition device can be used to collect and obtain the 3D information of the human face and iris again, and compare it with the standard data. If the comparison is successful, the next step is allowed. One step. It is understandable that this kind of comparison can also be used for the identification of fixed assets such as antiques and artworks, that is, first obtain 3D information of multiple areas of antiques and artworks as standard data, and obtain 3D information of multiple areas again when authentication is required. Information, and compare with standard data to identify authenticity. The three-dimensional information of multiple regions of the target obtained in the above embodiments can be used to design, produce, and manufacture accessory items for the target. For example, by obtaining three-dimensional data of the oral cavity and teeth of the human body, more suitable dentures can be designed and manufactured for the human body. The three-dimensional information of the target obtained in the above embodiments can also be used to measure the geometric size and contour of the target.

In the instructions provided here, a lot of specific details are explained. However, it can be understood that the embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures, and technologies are not shown in detail, so as not to obscure the understanding of this specification.

Similarly, it should be understood that in order to simplify the present disclosure and help understand one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present invention, the various features of the present invention are sometimes grouped together into a single embodiment, Figure, or its description. However, the disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly stated in each claim. More precisely, as reflected in the following claims, the inventive aspect lies in less than all the features of a single embodiment disclosed previously. Therefore, the claims following the specific embodiment are thus explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present invention.

Those skilled in the art can understand that it is possible to adaptively change the modules in the device in the embodiment and set them in one or more devices different from the embodiment. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and/or processes or units are mutually exclusive, any combination can be used to compare all features disclosed in this specification (including the accompanying claims, abstract and drawings) and any method or methods disclosed in this manner or All the processes or units of the equipment are combined. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

In addition, those skilled in the art can understand that although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments means that they fall within the scope of the present invention. And form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

The various component embodiments of the present invention may be implemented by hardware, or by software modules running on one or more processors, or by a combination of them. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions based on some or all of the components in the device of the present invention according to the embodiments of the present invention. The present invention can also be implemented as a device or device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for realizing the present invention may be stored on a computer-readable medium, or may have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

It should be noted that the above-mentioned embodiments illustrate rather than limit the present invention, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of multiple such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices may be embodied in the same hardware item. The use of the words first, second, and third, etc. do not indicate any order. These words can be interpreted as names.

So far, those skilled in the art should realize that although multiple exemplary embodiments of the present invention have been illustrated and described in detail herein, they can still be disclosed according to the present invention without departing from the spirit and scope of the present invention. The content directly determines or derives many other variations or modifications that conform to the principles of the present invention. Therefore, the scope of the present invention should be understood and deemed to cover all these other variations or modifications.

Claims

An ultra-thin three-dimensional acquisition module for a mobile terminal, which is characterized in that it includes a data interface, a motion driving device, a motion device, and an image acquisition device;

The image acquisition device is arranged on the movement device, and the image acquisition device moves relative to the mobile terminal during the image acquisition process;

The motion drive device is connected to the motion device;

The motion drive device is electrically connected to the mobile terminal through the data interface;

The image acquisition device is electrically connected to the mobile terminal through a data interface;

The optical axis of the image acquisition device and the motion plane of the image acquisition device have an included angle γ.
The module according to claim 1, characterized in that: γ=90° or 0<γ<90° or 180°>γ>90°.
The module according to claim 1, wherein when the image acquisition device is in different positions, the optical axis converges or diverges with respect to the vertical line of the motion plane of the image acquisition device.
The module according to claim 1, wherein the module and the mobile terminal are independent of each other or embedded in the mobile terminal.
The module according to claim 1, wherein there are a plurality of image acquisition devices.
The module according to claim 1, wherein the image acquisition device comprises a visible light image acquisition device and/or an infrared image acquisition device.
The module according to claim 1, wherein the image acquisition device extends out of the module housing.
The module according to claim 1, wherein the moving area of the image acquisition device further comprises a light-transmitting shell part.
The module according to claim 1, wherein the moving device is a turntable, a turntable, a curved guide rail, and a linear guide rail.
The module according to claim 1, wherein the collection position of the image collection device is: two adjacent collection positions of the image collection device meet the following conditions:

μ<0.482 or μ<0.326

Where L is the linear distance between the optical centers of the image acquisition device at two adjacent acquisition positions; f is the focal length of the image acquisition device; d is the rectangular length of the photosensitive element of the image acquisition device; M is the photosensitive element of the image acquisition device to the target along the optical axis The distance between the surface of the object; μ is the empirical coefficient.
A three-dimensional acquisition method for a mobile terminal, characterized in that: the image acquisition device is arranged in the mobile terminal,

The image acquisition device moves relative to the mobile terminal during the acquisition process, thereby capturing images of the target object at different positions;

The optical axis of the image acquisition device and the motion plane of the image acquisition device have an included angle γ, 0<γ<180°.