WO2021012523A1

WO2021012523A1 - Three-dimensional image search method and system, computer device, and storage medium

Info

Publication number: WO2021012523A1
Application number: PCT/CN2019/118153
Authority: WO
Inventors: 陈恺; 赵付利
Original assignee: 平安科技（深圳）有限公司
Priority date: 2019-07-19
Filing date: 2019-11-13
Publication date: 2021-01-28
Also published as: CN110472085A; CN110472085B

Abstract

A three-dimensional image search method and system, relating to the technical field of image matching. The method comprises: obtaining a first two-dimensional image of a search object, and calculating a first corner point of the first two-dimensional image (S210); matching the first corner point with a second corner point of a second two-dimensional image corresponding to each three-dimensional image in an image database, and determining a target two-dimensional image from the second two-dimensional images (S220); calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image (S230); and determining the three-dimensional image corresponding to the target two-dimensional image having the viewing angle difference less than a first threshold value to be a search result (S240). By means of the foregoing three-dimensional image search method, according to the easily acquired first two-dimensional image of the search object, image matching is performed by means of an easy-to-recognize corner point in the two-dimensional image, thereby finding the three-dimensional image and using as the search result, and realizing the search of the three-dimensional image. Moreover, the two-dimensional image in the method is easily obtained, and the corner point serving as a feature is also easily recognized, thereby greatly improving the efficiency of search.

Description

Three-dimensional image search method, system, computer equipment and storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on July 19, 2019, the application number is 201910656211.3, and the invention title is "3D image search method, system, computer equipment and storage medium", the entire content of which is incorporated by reference Incorporated in this application.

Technical field

This application relates to the technical field of image matching. Specifically, this application relates to a three-dimensional image search method, a three-dimensional image search system, a computer device, and a storage medium.

Background technique

Three-dimensional images are also called stereo images, depth images, and three-dimensional digital models. Three-dimensional images contain the complete information of the three-dimensional coordinates and gray levels of the real object surface, and record the three-dimensional space coordinates of each sampling point on the surface of the object. Three-dimensional images are very different from traditional planar images. The traditional plane image can be regarded as the brightness distribution in the two-dimensional space, and the plane image is obtained by projecting the real three-dimensional world on the two-dimensional image plane.

Three-dimensional images carry more information and can be used to display the shape and structure of the product in a three-dimensional and omni-directional manner. With the advancement of science and technology and the development of commerce, three-dimensional images used to display products are widely used, and it is inevitable that the demand for three-dimensional image search to find corresponding products has increased.

The inventor realizes that three-dimensional images carry more information. Although the three-dimensional image can be perceived by observing the three-dimensional image, the image acquisition device such as a camera can only collect and record the two-dimensional image of the three-dimensional image at a certain angle of view. Therefore, it is generally impossible to directly obtain the three-dimensional space coordinates described by the three-dimensional image, and it is difficult to search the three-dimensional image by collecting the three-dimensional image.

Therefore, there is a problem that the three-dimensional image cannot be captured by the image capturing device for three-dimensional image search.

Summary of the invention

The purpose of this application is to at least solve one of the above-mentioned technical defects, especially the technical defect that the image acquisition device cannot collect 3D images for 3D image search.

This application provides a three-dimensional image search method, including the following steps: acquiring a first two-dimensional image of a search object, calculating a first corner point of the first two-dimensional image; comparing the first corner point with each of the image database Match the second corner points of the second two-dimensional image corresponding to the three-dimensional image, determine the target two-dimensional image from the second two-dimensional image; calculate the difference between the first two-dimensional image and the target two-dimensional image The viewing angle difference; the three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold is determined as the search result.

This application also provides a three-dimensional image search system, including: a corner point calculation module, configured to obtain a first two-dimensional image of the search object, and calculate the first corner point of the first two-dimensional image; a two-dimensional image matching module, Used to match the first corner point with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determine a target two-dimensional image from the second two-dimensional image; a viewing angle difference calculation module , Used to calculate the viewing angle difference between the first two-dimensional image and the target two-dimensional image; a three-dimensional image acquisition module, used to determine the three-dimensional image corresponding to the target two-dimensional image with the viewing angle difference less than the first threshold as search results.

The present application also provides a computer device, including a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of a three-dimensional image search method when the computer program is executed, Wherein, the steps of the three-dimensional image search method include: acquiring a first two-dimensional image of a search object, calculating a first corner point of the first two-dimensional image; comparing the first corner point with each three-dimensional image in the image database The second corner points of the corresponding second two-dimensional image are matched, and the target two-dimensional image is determined from the second two-dimensional image; the viewing angle difference between the first two-dimensional image and the target two-dimensional image is calculated ; Determine the three-dimensional image corresponding to the target two-dimensional image whose angle of view difference is less than the first threshold as the search result.

The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of a three-dimensional image search method are realized. The steps of the three-dimensional image search method include: obtaining search Calculate the first corner of the first two-dimensional image of the object; match the first corner with the second corner of the second two-dimensional image corresponding to each three-dimensional image in the image database , Determine the target two-dimensional image from the second two-dimensional image; calculate the angle of view difference between the first two-dimensional image and the target two-dimensional image; determine the target two-dimensional image whose angle of view difference is less than the first threshold The corresponding three-dimensional image is determined as the search result.

The above-mentioned three-dimensional image search method, system, computer equipment and storage medium, according to the easy-to-collect first two-dimensional image of the search object, use the easily recognizable corner points in the two-dimensional image to perform image matching, thereby searching for the three-dimensional image and As a result of the search, a three-dimensional image search is realized. Moreover, in this method, two-dimensional images are easy to obtain, and the corner points as features are also easy to identify, which greatly improves the search efficiency; at the same time, considering the three-dimensional characteristics of the search object, it also improves the accuracy of the three-dimensional image search based on the judgment of the viewing angle difference.

Description of the drawings

Fig. 1 is an implementation environment diagram of a three-dimensional image search method provided in an embodiment;

Figure 2 is a flowchart of a three-dimensional image search method in an embodiment;

Figure 3 is a flowchart of a three-dimensional image search method in another embodiment;

Fig. 4 is a schematic structural diagram of a three-dimensional image search system in an embodiment;

Figure 5 is a schematic diagram of the internal structure of a computer device in an embodiment.

Detailed ways

The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals indicate the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the drawings are exemplary, and are only used to explain the present application, and cannot be construed as a limitation to the present application.

Those skilled in the art can understand that, unless specifically stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of the described features, integers, steps, operations, elements, and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components, and/or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements may also be present. In addition, "connected" or "coupled" as used herein may include wireless connection or wireless coupling. The term "and/or" as used herein includes all or any unit and all combinations of one or more associated listed items.

As shown in FIG. 1, FIG. 1 is an implementation environment diagram of a three-dimensional image search method provided in an embodiment. In the implementation environment, a server 110 and a client 120 are included.

The server 110 may be provided with an image database, and the image database may store three-dimensional images and two-dimensional images displayed from the perspective of the three-dimensional images. The server 110 may be built on a computer device or a server device. A camera device may be configured on the client 120, and the client 120 may send a two-dimensional image to the server 110. The client 120 may run on a smart phone, a tablet computer, a notebook computer, or a desktop computer.

In an embodiment, as shown in FIG. 2, FIG. 2 is a flowchart of a three-dimensional image search method in an embodiment. In this embodiment, a three-dimensional image search method is proposed. The three-dimensional image search method can be applied to the above The server 110 may specifically include the following steps:

Step S210: Obtain a first two-dimensional image of the search object, and calculate a first corner point of the first two-dimensional image.

In this step, in the process of searching for the search object according to the first two-dimensional image, the first corner point of the first two-dimensional image may be calculated according to the corner detection.

Wherein, the search object may be a three-dimensional object with a three-dimensional structure, or a visible three-dimensional image with three-dimensional space coordinate information, and the search image can be collected from a perspective to obtain a two-dimensional image. A three-dimensional image can be viewed from an unlimited number of multiple angles, but a group of perspectives that can fully cover the three-dimensional image can be selected reasonably, and multiple two-dimensional images corresponding to the three-dimensional image can be collected according to the group of perspectives. In addition, for some symmetrical 3D images, the number of collected 2D images can be reduced appropriately.

Among them, under an intuitive overview, the corner points have the following characteristics: they can be corner points between contours; for the same scene, even if the viewing angle changes, they usually have the characteristics of stability; the pixels in the area near the point regardless of the gradient direction There is still a big change in its gradient amplitude. The corner point is an intuitive and easy-to-detect feature in the image, and it is also convenient for proper image matching.

Specifically, the step of acquiring the first two-dimensional image of the search object in step S210 and calculating the first corner point of the first two-dimensional image may include:

S211: Acquire two original images of the search target through the camera device within a preset time interval.

In this step, the two-dimensional image of the search object can be quickly collected by the camera device, wherein the two acquired original images belong to the two-dimensional image.

S212: Calculate the third corner point and the fourth corner point corresponding to the original image respectively.

S213: Perform corner point screening on the third corner point according to the fourth corner point to obtain the first corner point.

In the above method of calculating the first corner point, since the two original images have a certain similarity, the calculated third corner point and the fourth corner point will also have similarity. Therefore, it can be based on the relationship between the corner points. Perform corner point screening to eliminate noise caused by camera capture, or eliminate the interference of corner points in the background, and improve the accuracy of the first corner point.

Step S220: Match the first corner point with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determine the target two-dimensional image from the second two-dimensional image.

In this step, the first corner point is used for matching and searching in the image database to determine a target two-dimensional image similar to the first two-dimensional image.

Among them, the image database stores the second two-dimensional images corresponding to the three-dimensional images. These second two-dimensional images can come from the two-dimensional images used to display the three-dimensional images in a certain angle of view, or they can be derived from the calculation of the three-dimensional images. A two-dimensional image at a certain angle of view. Since 3D images have stereoscopic 3D characteristics, they carry more information. Generally, 2D images under a single view can only partially display 3D images. Generally, 3D images in an image database can be stored corresponding to the first view from multiple views. Two-dimensional image.

Step S230: Calculate the viewing angle difference between the first two-dimensional image and the target two-dimensional image.

In this step, the visual transformation feature can be used to calculate the angle of view difference between the first two-dimensional image and the target two-dimensional image based on the first corner point of the first two-dimensional image and the second corner point corresponding to the target two-dimensional image .

Step S240: Determine the three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold as the search result.

In this step, the accuracy of the corner point matching is further judged, and whether the viewing angle difference is within the error range. If the viewing angle difference is within the error range, that is, the viewing angle difference is less than the first threshold, the three-dimensional image corresponding to the target two-dimensional image is determined Is a search result.

The above-mentioned three-dimensional image search method, based on the easy-to-collect first two-dimensional image of the search object, performs image matching with the help of easily recognizable corner points in the two-dimensional image, so as to search for the three-dimensional image and use it as the search result to realize the search of the three-dimensional image . Moreover, in this method, two-dimensional images are easy to obtain, and the corner points as features are also easy to identify, which greatly improves the search efficiency; at the same time, considering the three-dimensional characteristics of the search object, it also improves the accuracy of the three-dimensional image search based on the judgment of the viewing angle difference.

In one of the embodiments, corner point matching can be divided into the following four steps: ① Extraction of detectors: Find the most easily recognizable pixels (corners) in the two images to be matched, such as the edges of objects with rich texture Wait. ②Extract descriptors: Use some mathematical features to describe the detected corner points, such as gradient histograms, local random binary features, etc. ③Matching: Judge their corresponding relationship in the two images through the descriptors of each corner point. ④Remove the outer point: remove the wrongly matched outer point and keep the correct inner point. Among them, matching strategy 1: Euclidean distance measurement for the descriptor vector, the simplest strategy is to first set a threshold (maximum distance), and then return all matches in another image within the threshold range. Or, matching strategy two: do nearest neighbor matching, that is, compare the ratio of the nearest neighbor distance to the next nearest neighbor distance, that is, the nearest neighbor ratio (NNDR).

In one embodiment, in step S220, the first corner point is matched with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and the step of determining the target two-dimensional image from the second two-dimensional image , Can include:

S221: Calculate the first similarity measure between each second two-dimensional image and the first two-dimensional image according to the second corner point and the first corner point of each second two-dimensional image in the image database.

In this step, the more matching corner points and the higher the similarity between the second corner point and the first corner point, the higher the first similarity measure value, and the corresponding second two-dimensional image and the first two-dimensional The higher the similarity between the images.

Specifically, the step of calculating the first similarity measure value between each second two-dimensional image and the first two-dimensional image in S221 may include:

A1: Obtain the second corner point corresponding to the second two-dimensional image of any three-dimensional image from the image database.

A2: Match the first corner point and the second corner point, and extract the corner point pair at the same position.

In this step, the corner pairs of each group in the first two-dimensional image and the second two-dimensional image can be matched one by one.

A3: Calculate the first similarity measure of the second two-dimensional image according to the corner point pair.

In this step, the first similarity metric value can be determined based on all corner point pairs.

S222: Select a second two-dimensional image whose first similarity measure value is within a second preset threshold as the target two-dimensional image.

In this step, the second preset threshold can be used to filter similarity metric values that meet the accuracy, so as to select a second two-dimensional image that meets the second preset threshold to improve the accuracy of the target two-dimensional image.

Specifically, the step of selecting the second two-dimensional image with the first similarity measure value within the second preset threshold as the target two-dimensional image in S222 may include:

B1: Obtain the viewing angle weight corresponding to each second two-dimensional image in the three-dimensional image.

In this step, the three-dimensional image can include second two-dimensional images with multiple viewing angles. Some of the viewing angles are commonly used viewing angles. It is more likely to collect the first two-dimensional image under common viewing angles. Therefore, the viewing angle can be further considered. Impact on image matching. In addition, the viewing angle weight of each second two-dimensional image can be set in advance according to the commonly used degree and importance of the viewing angle.

B2: Calculate the second similarity measure of the second two-dimensional image according to the viewing angle weight of the second two-dimensional image and the first similarity measure of the second two-dimensional image.

In this step, considering the influence of the viewing angle on the image matching, the second similarity measure value is calculated according to the viewing angle weight and the first similarity measure value. For example, the product of the viewing angle weight and the first similarity measure value may be used as the second similarity measure value.

B3: Select the second two-dimensional image whose second similarity measure value is within the second preset threshold as the target two-dimensional image.

In this step, the second preset threshold in this step may be a similarity measure value that satisfies the accuracy after considering the viewing angle weight, so as to select an accurate target two-dimensional image.

The above-mentioned three-dimensional image search method uses the corner pair between the second two-dimensional image and the first two-dimensional image to calculate the similarity measurement value, which facilitates the follow-up comparison of similarity measurement values and determines the target two-dimensional image with higher accuracy .

In an embodiment, the step of calculating the angle of view difference between the first two-dimensional image and the target two-dimensional image in step S230 may include:

S231: Establish a perspective transformation matrix according to the coordinates of the corner point pairs in the target two-dimensional image and the first two-dimensional image respectively.

In this step, the perspective difference between the target two-dimensional image and the first two-dimensional image, the perspective transformation matrix can be used to represent the same position mapping relationship between the target two-dimensional image and the first two-dimensional image, and the perspective transformation matrix can be used to represent The mapping relationship between perspectives.

S232: Calculate the viewing angle difference according to the perspective transformation matrix.

In the above-mentioned three-dimensional image search method, the perspective transformation matrix can contain the perspective relationship between the target two-dimensional image and the first two-dimensional image. The perspective transformation matrix can be converted to extract the parameters representing the perspective difference in the perspective transformation matrix, or according to the perspective The parameters in the transformation matrix calculate the difference in viewing angle.

In one embodiment, after the step S240 of determining the three-dimensional image corresponding to the target two-dimensional image whose angle of view difference is less than the first threshold as the search result, the method includes:

S251: Calculate the observation angle of the first two-dimensional image according to the angle of view difference and the angle of view of the target two-dimensional image in the three-dimensional image.

In this step, this observation angle may be the angle of view where the first two-dimensional image is actually displayed in the three-dimensional image; the target two-dimensional image is the second two-dimensional image derived from the three-dimensional image in the image database, so the target two-dimensional image The viewing angle of the image in the three-dimensional image is known, and the observation angle of the first two-dimensional image can be calculated based on the viewing angle difference.

S252: Display a three-dimensional image of the target at the observation perspective.

In this step, the three-dimensional image of the target is placed in the viewing angle for display, and the displayed effect is similar to the first two-dimensional image collected for the search object. The above-mentioned three-dimensional image search method is conducive to searching in the same or similar perspective. Contrast observation between the object and the target three-dimensional image.

In addition, when multiple target 3D images can be included, the target 3D images can also be sorted according to the perspective weight of each target 2D image, and the target 3D image with a higher perspective priority is given priority to display, that is, the target 3D image with a higher priority is displayed first. The downloaded 3D image is convenient for the user to quickly understand the shape of the displayed target 3D image according to the common viewing angle, and it is easy for the user to switch from the common viewing angle to the first two-dimensional image, improving readability.

In another embodiment, as shown in FIG. 3, FIG. 3 is a flowchart of a three-dimensional image search method in another embodiment. In this embodiment, a three-dimensional image search method is provided, which may specifically include the following steps:

S310: Acquire a first two-dimensional image of the search object, and calculate a first corner point of the first two-dimensional image.

Obtain two original images of the search object within a preset time interval through the camera device; calculate the third and fourth corner points corresponding to the original image respectively; perform corner filtering on the third corner point according to the fourth corner point to obtain the first corner.

S320: Match the first corner point with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determine the target two-dimensional image from the second two-dimensional image.

First, according to the second corner point and the first corner point of each second two-dimensional image in the image database, a first similarity measure value between each second two-dimensional image and the first two-dimensional image is calculated. Obtain the second corner point corresponding to the second two-dimensional image of any three-dimensional image from the image database; match the first corner point and the second corner point to extract the corner point pair at the same position; calculate the corner point pair according to the corner point pair The first similarity measure of the second two-dimensional image.

Then, a second two-dimensional image whose first similarity measure value is within a second preset threshold is selected as the target two-dimensional image. Acquiring the viewing angle weight of each second two-dimensional image corresponding to the three-dimensional image; calculating the second similarity measure value of the second two-dimensional image according to the viewing angle weight of the second two-dimensional image and the first similarity measure value of the second two-dimensional image; The second two-dimensional image whose second similarity measure value is within the second preset threshold is selected as the target two-dimensional image.

S330: Calculate the viewing angle difference between the first two-dimensional image and the target two-dimensional image.

The perspective transformation matrix is established according to the coordinates of the corner point pairs in the target two-dimensional image and the first two-dimensional image; the perspective difference is calculated according to the perspective transformation matrix.

S340: Determine a three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold as the search result.

S350: Calculate the observation angle of the first two-dimensional image according to the angle of view difference and the angle of view of the target two-dimensional image in the three-dimensional image, and display the target three-dimensional image at the observation angle.

The above-mentioned three-dimensional image search method, based on the easy-to-collect first two-dimensional image of the search object, uses the easily recognizable corner points in the two-dimensional image to match the image, so as to search for the three-dimensional image and use it as the search result to realize the search of the three-dimensional image . Moreover, in this method, two-dimensional images are easy to obtain, and the corner points as features are also easy to identify, which greatly improves the search efficiency; at the same time, considering the three-dimensional characteristics of the search object, it also improves the accuracy of the three-dimensional image search based on the judgment of the viewing angle difference.

In an embodiment, as shown in FIG. 4, FIG. 4 is a schematic structural diagram of a three-dimensional image search system in an embodiment. In this embodiment, a three-dimensional image search system is provided, which includes a corner point calculation module 410 and two-dimensional image matching. Module 420, angle difference calculation module 430 and 3D image acquisition module 440, wherein:

The corner point calculation module 410 is configured to obtain the first two-dimensional image of the search object, and calculate the first corner point of the first two-dimensional image.

The corner point calculation module 410 may calculate the first corner point of the first two-dimensional image according to corner point detection in the process of searching for the search object according to the first two-dimensional image.

Wherein, the search object may be a three-dimensional object with a three-dimensional structure, or a visible three-dimensional image with three-dimensional space coordinate information, and the search image can be collected from a perspective to obtain a two-dimensional image. Intuitively, the corner point has the following characteristics: it can be the corner point between the contours; for the same scene, even if the viewing angle changes, it usually has the characteristics of stability; the pixels in the area near the point are either in the gradient direction or There is a big change in the gradient amplitude. The corner point is an intuitive and easy-to-detect feature in the image, and it is also convenient for proper image matching.

The two-dimensional image matching module 420 is configured to match the first corner point with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determine the target two-dimensional image from the second two-dimensional image.

The two-dimensional image matching module 420 performs matching and searching in the image database based on the first corner point, and determines a target two-dimensional image similar to the first two-dimensional image.

The viewing angle difference calculation module 430 is configured to calculate the viewing angle difference between the first two-dimensional image and the target two-dimensional image.

The viewing angle difference calculation module 430 can use the characteristics of visual transformation to calculate the viewing angle between the first two-dimensional image and the target two-dimensional image according to the first corner point of the first two-dimensional image and the second corner point corresponding to the target two-dimensional image. difference.

The three-dimensional image acquisition module 440 is configured to determine the three-dimensional image corresponding to the target two-dimensional image whose angle of view difference is less than the first threshold value as the search result.

To further determine the accuracy of the corner point matching, the three-dimensional image acquisition module 440 detects whether the angle of view difference is within the error range. If the angle of view difference is within the error range, that is, the angle of view difference is less than the first threshold, the three-dimensional image corresponding to the target two-dimensional image Determined as a search result.

The above-mentioned three-dimensional image search system, based on the easy-to-collect first two-dimensional image of the search object, uses the easily recognizable corner points in the two-dimensional image to perform image matching, so as to search for the three-dimensional image and use it as the search result to realize the search of the three-dimensional image . Moreover, in this method, two-dimensional images are easy to obtain, and the corner points as features are also easy to identify, which greatly improves the search efficiency; at the same time, considering the three-dimensional characteristics of the search object, it also improves the accuracy of the three-dimensional image search based on the judgment of the viewing angle difference.

For the specific limitation of the 3D image search system, please refer to the above limitation of the 3D image search method, which will not be repeated here. Each module in the above-mentioned three-dimensional image search system can be implemented in whole or in part by software, hardware, and a combination thereof. The foregoing modules may be embedded in the form of hardware or independent of the processor in the computer device, or may be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the foregoing modules.

As shown in FIG. 5, FIG. 5 is a schematic diagram of the internal structure of a computer device in an embodiment. The computer device includes a processor, a nonvolatile storage medium, a memory, and a network interface connected through a system bus. Wherein, the non-volatile storage medium of the computer device stores an operating system, a database, and a computer program. When the computer program is executed by the processor, the processor can enable the processor to implement a three-dimensional image search method. The processor of the computer equipment is used to provide calculation and control capabilities, and supports the operation of the entire computer equipment. A computer program can be stored in the memory of the computer device, and when the computer program is executed by the processor, the processor can execute a three-dimensional image search method. The network interface of the computer device is used to connect and communicate with the terminal. Those skilled in the art can understand that the structure shown in FIG. 5 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may Including more or fewer parts than shown in the figure, or combining some parts, or having a different arrangement of parts.

In one embodiment, a computer device is proposed, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor implements the three-dimensional system in any of the above-mentioned embodiments when the computer program is executed. The steps of the image search method, wherein the steps of the three-dimensional image search method include: acquiring a first two-dimensional image of the search object, calculating a first corner point of the first two-dimensional image; and combining the first corner point with Match the second corner points of the second two-dimensional image corresponding to each three-dimensional image in the image database, determine the target two-dimensional image from the second two-dimensional image; calculate the first two-dimensional image and the target two-dimensional The viewing angle difference between the images; the three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold is determined as the search result.

In one embodiment, a storage medium storing computer-readable instructions is proposed. The storage medium of the computer-readable instructions may be non-volatile or volatile. A computer program is stored thereon. When the program is executed by the processor, the steps of the three-dimensional image search method in any of the above embodiments are implemented, wherein the steps of the three-dimensional image search method include: acquiring a first two-dimensional image of the search object, and calculating the first two-dimensional image Match the first corner point with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determine the target two-dimensional image from the second two-dimensional image; Calculating the viewing angle difference between the first two-dimensional image and the target two-dimensional image; determining the three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold value as the search result.

It should be understood that, although the various steps in the flowchart of the drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless explicitly stated in this article, the execution of these steps is not strictly limited in order, and they can be executed in other orders. Moreover, at least part of the steps in the flowchart of the drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the order of execution is also It is not necessarily performed sequentially, but may be performed alternately or alternately with other steps or at least a part of sub-steps or stages of other steps.

The above are only part of the implementation of this application. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and modifications can be made, and these improvements and modifications are also Should be regarded as the scope of protection of this application.

Claims

A three-dimensional image search method includes the following steps:

Acquiring a first two-dimensional image of the search object, and calculating a first corner point of the first two-dimensional image;

Matching the first corner point with a second corner point of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image;

Calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;

The three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold is determined as the search result.
The three-dimensional image search method according to claim 1, wherein the step of obtaining a first two-dimensional image of a search object and calculating a first corner point of the first two-dimensional image comprises:

Acquiring two original images of the search object within a preset time interval through a camera device;

Respectively calculating the third corner point and the fourth corner point corresponding to the original image;

Performing corner point screening on the third corner point according to the fourth corner point to obtain the first corner point.
The three-dimensional image search method according to claim 1, wherein the first corner point is matched with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and the The steps of determining the target two-dimensional image in the image include:

According to the second corner point and the first corner point of each second two-dimensional image in the image database, a first similarity measure value between each second two-dimensional image and the first two-dimensional image is calculated ；

A second two-dimensional image whose first similarity measure value is within a second preset threshold is selected as the target two-dimensional image.
3. The three-dimensional image search method according to claim 3, wherein the step of calculating the first similarity metric value between each of the second two-dimensional images and the first two-dimensional image comprises:

Acquiring, from the image database, the second corner point corresponding to the second two-dimensional image of any one of the three-dimensional images;

Matching the first corner point and the second corner point, and extracting a corner point pair at the same position;

The first similarity measure value of the second two-dimensional image is calculated according to the corner point pair.
The 3D image search method according to claim 3, wherein the step of selecting a second 2D image whose first similarity measure value is within a second preset threshold value as the target 2D image comprises:

Acquiring a perspective weight corresponding to each of the second two-dimensional images in the three-dimensional image;

Calculating a second similarity metric value of the second two-dimensional image according to the viewing angle weight of the second two-dimensional image and the first similarity metric value of the second two-dimensional image;

A second two-dimensional image whose second similarity measure value is within a second preset threshold is selected as the target two-dimensional image.
The 3D image search method according to claim 4, wherein the step of calculating the angle of view difference between the first 2D image and the target 2D image comprises:

Establishing a perspective transformation matrix according to the coordinates of the corner point pairs in the target two-dimensional image and the first two-dimensional image;

Calculate the viewing angle difference according to the perspective transformation matrix.
The three-dimensional image search method according to claim 1, after the step of determining the three-dimensional image corresponding to the target two-dimensional image whose angle of view difference is less than the first threshold as the search result, the method comprises:

Calculating the viewing angle of the first two-dimensional image according to the viewing angle difference and the viewing angle of the target two-dimensional image in the three-dimensional image;

The three-dimensional image of the target is displayed at the viewing angle of view.
A three-dimensional image search system, including:

A corner point calculation module, configured to obtain a first two-dimensional image of the search object, and calculate the first corner point of the first two-dimensional image;

The two-dimensional image matching module is used to match the first corner point with the second corner point of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determine the target two-dimensional image from the second two-dimensional image. image;

A viewing angle difference calculation module, configured to calculate a viewing angle difference between the first two-dimensional image and the target two-dimensional image;

The three-dimensional image acquisition module is used to determine the three-dimensional image corresponding to the target two-dimensional image whose angle of view difference is less than the first threshold as the search result.
A computer device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor. The processor implements the steps of a three-dimensional image search method when the computer program is executed, wherein the The steps of the 3D image search method include:

Acquiring a first two-dimensional image of the search object, and calculating a first corner point of the first two-dimensional image;

Matching the first corner point with a second corner point of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image;

Calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;

The three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold is determined as the search result.
The computer device according to claim 9, wherein the step of obtaining a first two-dimensional image of a search object and calculating a first corner point of the first two-dimensional image comprises:

Acquiring two original images of the search object within a preset time interval through a camera device;

Respectively calculating the third corner point and the fourth corner point corresponding to the original image;

Performing corner point screening on the third corner point according to the fourth corner point to obtain the first corner point.
The computer device according to claim 9, wherein the first corner point is matched with a second corner point of a second two-dimensional image corresponding to each three-dimensional image in the image database, and the second corner point is obtained from the second two-dimensional image The steps to determine the two-dimensional image of the target include:

According to the second corner point and the first corner point of each second two-dimensional image in the image database, a first similarity measure value between each second two-dimensional image and the first two-dimensional image is calculated ；

A second two-dimensional image whose first similarity measure value is within a second preset threshold is selected as the target two-dimensional image.
11. The computer device according to claim 11, wherein the step of calculating a first similarity metric value between each of the second two-dimensional images and the first two-dimensional image comprises:

Acquiring, from the image database, the second corner point corresponding to the second two-dimensional image of any one of the three-dimensional images;

Matching the first corner point and the second corner point, and extracting a corner point pair at the same position;

The first similarity measure value of the second two-dimensional image is calculated according to the corner point pair.
11. The computer device according to claim 11, wherein the step of selecting a second two-dimensional image whose first similarity measure value is within a second preset threshold as the target two-dimensional image comprises:

Acquiring a perspective weight corresponding to each of the second two-dimensional images in the three-dimensional image;

Calculating a second similarity metric value of the second two-dimensional image according to the viewing angle weight of the second two-dimensional image and the first similarity metric value of the second two-dimensional image;

A second two-dimensional image whose second similarity measure value is within a second preset threshold is selected as the target two-dimensional image.
The computer device according to claim 12, wherein the step of calculating the angle of view difference between the first two-dimensional image and the target two-dimensional image comprises:

Establishing a perspective transformation matrix according to the coordinates of the corner point pairs in the target two-dimensional image and the first two-dimensional image;

Calculate the viewing angle difference according to the perspective transformation matrix.
9. The computer device according to claim 9, after the step of determining the three-dimensional image corresponding to the target two-dimensional image whose angle of view difference is less than the first threshold as the search result, comprising:

Calculating the viewing angle of the first two-dimensional image according to the viewing angle difference and the viewing angle of the target two-dimensional image in the three-dimensional image;

The three-dimensional image of the target is displayed at the viewing angle of view.
A computer-readable storage medium having a computer program stored thereon, which when executed by a processor realizes the steps of a three-dimensional image search method, the steps of the three-dimensional image search method include:

Acquiring a first two-dimensional image of the search object, and calculating a first corner point of the first two-dimensional image;

Matching the first corner point with a second corner point of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image;

Calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;

The three-dimensional image corresponding to the target two-dimensional image whose viewing angle difference is less than the first threshold is determined as the search result.
15. The computer-readable storage medium according to claim 16, wherein the step of obtaining a first two-dimensional image of a search object and calculating a first corner point of the first two-dimensional image comprises:

Acquiring two original images of the search object within a preset time interval through a camera device;

Respectively calculating the third corner point and the fourth corner point corresponding to the original image;

Performing corner point screening on the third corner point according to the fourth corner point to obtain the first corner point.
The computer-readable storage medium according to claim 16, wherein the first corner point is matched with a second corner point of a second two-dimensional image corresponding to each three-dimensional image in the image database, and The steps of determining the target two-dimensional image in the three-dimensional image include:

According to the second corner point and the first corner point of each second two-dimensional image in the image database, a first similarity measure value between each second two-dimensional image and the first two-dimensional image is calculated ；

A second two-dimensional image whose first similarity measure value is within a second preset threshold is selected as the target two-dimensional image.
18. The computer-readable storage medium according to claim 18, wherein the step of calculating a first similarity metric value between each of the second two-dimensional image and the first two-dimensional image comprises:

Acquiring, from the image database, the second corner point corresponding to the second two-dimensional image of any one of the three-dimensional images;

Matching the first corner point and the second corner point, and extracting a corner point pair at the same position;

The first similarity measure value of the second two-dimensional image is calculated according to the corner point pair.
18. The computer-readable storage medium according to claim 18, wherein the step of selecting a second two-dimensional image whose first similarity metric value is within a second preset threshold as the target two-dimensional image comprises:

Acquiring a perspective weight corresponding to each of the second two-dimensional images in the three-dimensional image;

Calculating a second similarity metric value of the second two-dimensional image according to the viewing angle weight of the second two-dimensional image and the first similarity metric value of the second two-dimensional image;

A second two-dimensional image whose second similarity measure value is within a second preset threshold is selected as the target two-dimensional image.