CN110472085B - Three-dimensional image searching method, system, computer device and storage medium - Google Patents
Three-dimensional image searching method, system, computer device and storage medium Download PDFInfo
- Publication number
- CN110472085B CN110472085B CN201910656211.3A CN201910656211A CN110472085B CN 110472085 B CN110472085 B CN 110472085B CN 201910656211 A CN201910656211 A CN 201910656211A CN 110472085 B CN110472085 B CN 110472085B
- Authority
- CN
- China
- Prior art keywords
- dimensional image
- corner
- dimensional
- target
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/53—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention relates to the technical field of image matching, and provides a three-dimensional image searching method and system, wherein the method comprises the following steps: acquiring a first two-dimensional image of a search object, and calculating a first corner of the first two-dimensional image; matching the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image; calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image; and determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result. According to the three-dimensional image searching method, according to the first two-dimensional image of the easily-collected searching object, the matching of the images is carried out by means of the corner points which are easy to identify in the two-dimensional image, so that the three-dimensional image is searched out and used as a searching result, and the three-dimensional image searching is realized. In addition, the two-dimensional image is easy to obtain, and the corner serving as the characteristic is easy to identify, so that the searching efficiency is greatly improved.
Description
Technical Field
The present invention relates to the field of image matching technology, and in particular, to a three-dimensional image searching method, a three-dimensional image searching system, a computer device, and a storage medium.
Background
The three-dimensional image, which is also called a stereoscopic image, a depth image and a three-dimensional digital model, contains the three-dimensional coordinates and the complete gray level information of the surface of the real object, and records the three-dimensional space coordinates of each sampling point of the surface of the object. Three-dimensional images are quite different from traditional planar images. A conventional planar image can be regarded as a luminance distribution in a two-dimensional space, the planar image being obtained by projecting a real three-dimensional world on a two-dimensional image plane.
The three-dimensional image carries more information and can be used for displaying the shape and the structure of the product in a three-dimensional and all-dimensional way. With the advancement of technology and the development of commerce, three-dimensional images for displaying products are being widely used, and the need for searching three-dimensional images to find corresponding products is inevitably increased.
The three-dimensional image has more information, and although the stereoscopic impression of the three-dimensional image can be perceived by observing the three-dimensional image, the image acquisition device such as a camera only can acquire and record the two-dimensional image of the three-dimensional image under a certain view angle, so that the three-dimensional space coordinates described by the three-dimensional image cannot be directly obtained generally, and the three-dimensional image is difficult to search by acquiring the three-dimensional image.
Therefore, there is a problem in that three-dimensional image search cannot be performed by capturing three-dimensional images by the image capturing device.
Disclosure of Invention
The invention aims to at least solve one of the technical defects, in particular to the technical defect that three-dimensional image searching can not be performed by acquiring three-dimensional images through an image acquisition device.
The invention provides a three-dimensional image searching method, which comprises the following steps:
acquiring a first two-dimensional image of a search object, and calculating a first corner of the first two-dimensional image;
matching the first angular point with a second angular point of a second two-dimensional image corresponding to each three-dimensional image in an image database, and determining a target two-dimensional image from the second two-dimensional image;
calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;
and determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result.
In one embodiment, the step of acquiring a first two-dimensional image of the search object, and calculating a first corner of the first two-dimensional image includes:
acquiring two original images of the search object through the camera equipment within a preset time interval; respectively calculating a third corner point and a fourth corner point corresponding to the original image; and carrying out corner screening on the third corner according to the fourth corner to obtain the first corner.
In one embodiment, the step of matching the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image includes:
calculating a first similarity measurement value between each second two-dimensional image and each first two-dimensional image according to a second corner point and the first corner point of each second two-dimensional image in the image database; and selecting a second two-dimensional image of which the first similarity value is within a second preset threshold value as the target two-dimensional image.
In one embodiment, the step of calculating a first similarity measure between each of the second two-dimensional images and the first two-dimensional image includes:
acquiring a second corner point corresponding to a second two-dimensional image of any one three-dimensional image from the image database; matching the first corner point with the second corner point, and extracting corner point pairs at the same position; and calculating a first similarity value of the second two-dimensional image according to the corner pairs.
In one embodiment, the step of selecting the second two-dimensional image with the first similarity value within a second preset threshold as the target two-dimensional image includes:
acquiring view angle weights of the second two-dimensional images in the three-dimensional images; calculating a second similarity metric value of the second two-dimensional image according to the view angle weight of the second two-dimensional image and the first similarity metric value of the second two-dimensional image; and selecting a second two-dimensional image with the second similarity measurement value within a second preset threshold value as the target two-dimensional image.
In one embodiment, the step of calculating the viewing angle difference between the first two-dimensional image and the target two-dimensional image comprises:
establishing a perspective transformation matrix according to coordinates of the corner pairs in the target two-dimensional image and the first two-dimensional image respectively; and calculating the visual angle difference according to the perspective transformation matrix.
In one embodiment, after the step of determining, as the search result, a three-dimensional image corresponding to the target two-dimensional image having the view angle difference smaller than the first threshold, the method includes:
calculating an observation view angle of the first two-dimensional image according to the view angle difference and the view angle of the target two-dimensional image in the three-dimensional image; and displaying the target three-dimensional image at the observation view angle.
The invention also provides a three-dimensional image searching system, which comprises:
the corner calculation module is used for acquiring a first two-dimensional image of the search object and calculating a first corner of the first two-dimensional image;
the two-dimensional image matching module is used for matching the first angular point with a second angular point of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image;
a viewing angle difference calculation module for calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;
and the three-dimensional image acquisition module is used for determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result.
The invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the three-dimensional image searching method according to any of the embodiments are realized when the processor executes the computer program.
The present invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the three-dimensional image search method according to any of the above embodiments.
According to the three-dimensional image searching method, the system, the computer equipment and the storage medium, according to the first two-dimensional image of the easily-collected searching object, the matching of the images is carried out by means of the corner points which are easy to identify in the two-dimensional image, so that the three-dimensional image is searched out and used as a searching result, and the three-dimensional image searching is realized. In addition, the two-dimensional image is easy to obtain, and the corner serving as the characteristic is easy to identify, so that the searching efficiency is greatly improved; meanwhile, the three-dimensional characteristics of the searched object are considered, and the accuracy of three-dimensional image searching is improved according to the judgment of the visual angle difference. Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.
Drawings
The foregoing and/or additional aspects and advantages of the invention will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:
FIG. 1 is an environmental diagram of an implementation of a three-dimensional image search method provided in one embodiment;
FIG. 2 is a flow diagram of a three-dimensional image search method in one embodiment;
FIG. 3 is a flow chart of a three-dimensional image search method in another embodiment;
FIG. 4 is a schematic diagram of a three-dimensional image search system in one embodiment;
fig. 5 is a schematic diagram showing an internal structure of the computer device in one embodiment.
Detailed Description
Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The embodiments described below by referring to the drawings are illustrative only and are not to be construed as limiting the invention.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless expressly stated otherwise, as understood by those skilled in the art. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein includes all or any element and all combination of one or more of the associated listed items.
As shown in fig. 1, fig. 1 is a diagram of an implementation environment of a three-dimensional image searching method provided in one embodiment, where the implementation environment includes a server 110 and a client 120.
The server 110 may be provided with an image database, in which a three-dimensional image and a two-dimensional image displayed by the three-dimensional image under a viewing angle may be stored, and the server 110 may be built on a computer device or a server device. The client 120 may be configured with a camera device, and the client 120 may transmit a two-dimensional image to the server 110. The client 120 may run on a smart phone, tablet, notebook, desktop computer.
In one embodiment, as shown in fig. 2, fig. 2 is a flowchart of a three-dimensional image searching method in one embodiment, and in this embodiment, a three-dimensional image searching method is provided, and the three-dimensional image searching method may be applied to the server 110, and may specifically include the following steps:
step S210: a first two-dimensional image of the search object is acquired, and a first corner of the first two-dimensional image is calculated.
In this step, in searching the search object according to the first two-dimensional image, the first corner point of the first two-dimensional image may be calculated according to the corner point detection.
The search object may be a three-dimensional object with a three-dimensional structure, or a visual three-dimensional image with three-dimensional space coordinate information, and the search image may be acquired at a viewing angle to obtain a two-dimensional image. The three-dimensional image can be observed in a plurality of angles, but a group of view angles capable of fully covering the three-dimensional image can be reasonably selected, and a plurality of two-dimensional images corresponding to the three-dimensional image are acquired according to the group of view angles. In addition, for some three-dimensional images with symmetrical properties, the number of acquired two-dimensional images can be reduced by a proper amount.
In visual overview, the corner points have the following features: may be corner points between contours; for the same scene, even if the viewing angle changes, the scene is generally characterized by stable properties; the pixels in the region near the point have a large variation in both the gradient direction and the gradient magnitude. The corner points are features which are intuitive and easy to detect in the image and are also convenient for proper image matching.
Specifically, the step of acquiring the first two-dimensional image of the search object and calculating the first corner of the first two-dimensional image in step S210 may include:
s211: two original images of the search object are acquired by the image capturing apparatus within a preset time interval.
In this step, two-dimensional images of a search object can be quickly acquired by the image pickup apparatus, wherein the two acquired original images belong to the two-dimensional images.
S212: and respectively calculating a third corner point and a fourth corner point corresponding to the original image.
S213: and carrying out corner screening on the third corner according to the fourth corner to obtain a first corner.
In the above manner of calculating the first corner, since there is a certain similarity between the two original images, there is a similarity between the calculated third corner and the fourth corner, so that corner screening can be performed according to the relationship between the corners, noise caused by image capturing and acquisition can be eliminated, or interference of corners existing in the background can be eliminated, and accuracy of the first corner can be improved.
Step S220: and matching the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image.
In the step, matching and searching are carried out in an image database by means of the first corner point, and a target two-dimensional image similar to the first two-dimensional image is determined.
The second two-dimensional images corresponding to the three-dimensional images are stored in the image database, and the second two-dimensional images can be from two-dimensional images for displaying the three-dimensional images under a certain view angle or from two-dimensional images under a certain view angle which are calculated on the three-dimensional images. Because the three-dimensional image has three-dimensional characteristics, more information is carried, the three-dimensional image under a single view angle can only be displayed locally, and the three-dimensional image in the image database can be stored with the second two-dimensional image under a plurality of view angles correspondingly.
Step S230: a viewing angle difference between the first two-dimensional image and the target two-dimensional image is calculated.
In this step, the visual angle difference between the first two-dimensional image and the target two-dimensional image can be calculated according to the first angular point of the first two-dimensional image and the second angular point corresponding to the target two-dimensional image by means of the characteristics of visual transformation.
Step S240: and determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result.
In this step, the accuracy of corner matching is further determined, whether the angle of view difference is within the error range is detected, and if the angle of view difference is within the error range, that is, the angle of view difference is smaller than the first threshold, the three-dimensional image corresponding to the target two-dimensional image is determined as the search result.
According to the three-dimensional image searching method, according to the first two-dimensional image of the easily-collected searching object, the matching of the images is carried out by means of the corner points which are easy to identify in the two-dimensional image, so that the three-dimensional image is searched out and used as a searching result, and the three-dimensional image searching is realized. In addition, the two-dimensional image is easy to obtain, and the corner serving as the characteristic is easy to identify, so that the searching efficiency is greatly improved; meanwhile, the three-dimensional characteristics of the searched object are considered, and the accuracy of three-dimensional image searching is improved according to the judgment of the visual angle difference.
In one embodiment, corner matching can be divided into the following four steps: (1) extracting a detector: the most easily identified pixel points (corner points) are found in the two images to be matched, such as object edge points with rich textures. (2) Extracting descriptors: for the detected corner points, some mathematical features are used to describe them, such as gradient histograms, local random binary features, etc. (3) Matching: and judging the corresponding relation of the corner points in the two images through the descriptors of the corner points. (4) Removing the outer points: the mismatching outliers are removed and the correct inliers are retained. Wherein, matching policy one: the euclidean distance metric is made on the descriptive subvectors, and one of the simplest strategies is to set a threshold (maximum distance) first, and then return all matches in another image that are within this threshold. Or, matching policy two: nearest neighbor matching is performed, namely, the ratio of the nearest neighbor distance to the next nearest neighbor distance is compared, namely, nearest neighbor ratio (NNDR).
In one embodiment, in step S220, the step of matching the first corner with the second corner of the second two-dimensional image corresponding to each three-dimensional image in the image database, and determining the target two-dimensional image from the second two-dimensional image may include:
s221: and calculating a first similarity measurement value between each second two-dimensional image and the first two-dimensional image according to the second corner points and the first corner points of each second two-dimensional image in the image database.
In this step, the more corner points that can be matched between the second corner point and the first corner point and the higher the similarity, the higher the first similarity value, and the higher the similarity between the corresponding second two-dimensional image and the first two-dimensional image.
Specifically, the step of calculating the first similarity metric value between each of the second two-dimensional images and the first two-dimensional image in S221 may include:
a1: and acquiring a second corner point corresponding to a second two-dimensional image of any one three-dimensional image from the image database.
A2: and matching the first corner point with the second corner point, and extracting corner point pairs at the same position.
In this step, corner pairs of each group in the first two-dimensional image and the second two-dimensional image may be matched one by one.
A3: and calculating a first similarity value of the second two-dimensional image according to the corner pairs.
In this step, the first similarity measure may be determined according to all pairs of corner points.
S222: and selecting the second two-dimensional image with the first similarity value within a second preset threshold value as a target two-dimensional image.
In this step, the second preset threshold may be used to screen the similarity measure value satisfying the precision, so as to select the second two-dimensional image conforming to the second preset threshold, and improve the accuracy of the target two-dimensional image.
Specifically, the step of selecting the second two-dimensional image with the first similarity measurement value within the second preset threshold value as the target two-dimensional image in S222 may include:
b1: and obtaining the view angle weight of each second two-dimensional image in the three-dimensional image.
In this step, the three-dimensional image may include a second two-dimensional image with multiple viewing angles, where some viewing angles are common viewing angles, and the probability of capturing the first two-dimensional image at the common viewing angles is high, so that the influence of the viewing angle on the image matching may be further considered. In addition, the viewing angle weight of each second two-dimensional image may be set in advance according to the degree of commonality and importance of the viewing angle.
B2: and calculating a second similarity measurement value of the second two-dimensional image according to the view angle weight of the second two-dimensional image and the first similarity measurement value of the second two-dimensional image.
In this step, the influence of the view on the image matching is considered, and a second similarity measure is calculated according to the view weight and the first similarity measure. For example, the product of the view weight and the first similarity measure may be used as the second similarity measure.
B3: and selecting a second two-dimensional image with the second similarity measurement value within a second preset threshold value as a target two-dimensional image.
In this step, the second preset threshold may be a similarity value that satisfies accuracy by screening in consideration of the view angle weight, so as to select an accurate target two-dimensional image.
According to the three-dimensional image searching method, the similarity value is calculated through the corner pairs between the second two-dimensional image and the first two-dimensional image, so that the subsequent comparison of the similarity value is facilitated, and the target two-dimensional image with higher accuracy is determined.
In one embodiment, the step of calculating the viewing angle difference between the first two-dimensional image and the target two-dimensional image in step S230 may include:
s231: and establishing a perspective transformation matrix according to coordinates of the corner pairs in the target two-dimensional image and the first two-dimensional image respectively.
In this step, the perspective transformation matrix may be used to represent the same positional mapping relationship between the target two-dimensional image and the first two-dimensional image, and the perspective transformation matrix may be used to represent the mapping relationship of transformation between the perspectives.
S232: the perspective difference is calculated from the perspective transformation matrix.
According to the three-dimensional image searching method, the perspective transformation matrix can be used for accumulating the association relation of the visual angles between the target two-dimensional image and the first two-dimensional image, and the parameters representing the visual angle difference in the perspective transformation matrix can be extracted by converting the perspective transformation matrix, or the visual angle difference can be calculated according to the parameters in the perspective transformation matrix.
In one embodiment, after the step of determining the three-dimensional image corresponding to the target two-dimensional image having the view angle difference smaller than the first threshold as the search result in step S240, the method includes:
s251: and calculating the observation view angle of the first two-dimensional image according to the view angle difference and the view angle of the target two-dimensional image in the three-dimensional image.
In this step, the viewing angle may be the angle at which the first two-dimensional image is actually displayed in the three-dimensional image; the target two-dimensional image is a second two-dimensional image derived from the three-dimensional image in the image database, so that the viewing angle of the target two-dimensional image in the three-dimensional image is known, and the viewing angle of the first two-dimensional image can be estimated according to the viewing angle difference.
S252: and displaying the three-dimensional image of the target at the observation angle.
In the step, the target three-dimensional image is placed in the observation visual angle for display, the displayed effect is similar to that of the first two-dimensional image acquired by the search object, and the three-dimensional image search method is favorable for comparing and observing the search object and the target three-dimensional image at the same visual angle or similar visual angles.
In addition, when the target three-dimensional images can include a plurality of target three-dimensional images, the target three-dimensional images can be ordered according to the view angle weights of the target two-dimensional images, and the target three-dimensional images with higher corresponding view angle priority are preferentially displayed, namely the three-dimensional images under the condition of easy observation angle are preferentially displayed, so that a user can quickly read and understand the shape of the displayed target three-dimensional images according to the common view angle, and the common view angle of the user can be easily converted into the view angle of the first two-dimensional image, and the readability is improved.
In another embodiment, as shown in fig. 3, fig. 3 is a flowchart of a three-dimensional image searching method in another embodiment, where the three-dimensional image searching method may specifically include the following steps:
s310: a first two-dimensional image of the search object is acquired, and a first corner of the first two-dimensional image is calculated.
Acquiring two original images of a search object through the camera equipment within a preset time interval; respectively calculating a third corner point and a fourth corner point corresponding to the original image; and carrying out corner screening on the third corner according to the fourth corner to obtain a first corner.
S320: and matching the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determining a target two-dimensional image from the second two-dimensional image.
First, according to second angular points and first angular points of each second two-dimensional image in an image database, first similarity measurement values between each second two-dimensional image and the first two-dimensional image are calculated. Acquiring a second corner point corresponding to a second two-dimensional image of any three-dimensional image from an image database; matching the first corner point with the second corner point, and extracting corner point pairs at the same position; and calculating a first similarity value of the second two-dimensional image according to the corner pairs.
Then, selecting a second two-dimensional image with the first similarity value within a second preset threshold value as a target two-dimensional image. Acquiring view angle weights of the second two-dimensional images in the three-dimensional images; calculating a second similarity measurement value of the second two-dimensional image according to the view angle weight of the second two-dimensional image and the first similarity measurement value of the second two-dimensional image; and selecting a second two-dimensional image with the second similarity measurement value within a second preset threshold value as a target two-dimensional image.
S330: a viewing angle difference between the first two-dimensional image and the target two-dimensional image is calculated.
Establishing a perspective transformation matrix according to coordinates of the corner pairs in the target two-dimensional image and the first two-dimensional image respectively; the perspective difference is calculated from the perspective transformation matrix.
S340: and determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result.
S350: and according to the visual angle difference and the visual angle of the target two-dimensional image in the three-dimensional image, calculating the observation visual angle of the first two-dimensional image, and displaying the target three-dimensional image at the observation visual angle.
According to the three-dimensional image searching method, according to the first two-dimensional image of the easily-collected searching object, the matching of the images is carried out by means of the corner points which are easy to identify in the two-dimensional image, so that the three-dimensional image is searched out and used as a searching result, and the three-dimensional image searching is realized. In addition, the two-dimensional image is easy to obtain, and the corner serving as the characteristic is easy to identify, so that the searching efficiency is greatly improved; meanwhile, the three-dimensional characteristics of the searched object are considered, and the accuracy of three-dimensional image searching is improved according to the judgment of the visual angle difference.
In one embodiment, as shown in fig. 4, fig. 4 is a schematic structural diagram of a three-dimensional image searching system in one embodiment, and in this embodiment, a three-dimensional image searching system is provided, including a corner calculating module 410, a two-dimensional image matching module 420, a view angle difference calculating module 430, and a three-dimensional image obtaining module 440, where:
the corner calculation module 410 is configured to obtain a first two-dimensional image of the search object, and calculate a first corner of the first two-dimensional image.
The corner calculation module 410 may calculate a first corner of the first two-dimensional image according to the corner detection during searching of the search object according to the first two-dimensional image.
The search object may be a three-dimensional object with a three-dimensional structure, or a visual three-dimensional image with three-dimensional space coordinate information, and the search image may be acquired at a viewing angle to obtain a two-dimensional image. Intuitively, corner points have the following features: may be corner points between contours; for the same scene, even if the viewing angle changes, the scene is generally characterized by stable properties; the pixels in the region near the point have a large variation in both the gradient direction and the gradient magnitude. The corner points are features which are intuitive and easy to detect in the image and are also convenient for proper image matching.
The two-dimensional image matching module 420 is configured to match the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in the image database, and determine a target two-dimensional image from the second two-dimensional image.
The two-dimensional image matching module 420 performs matching and searching in the image database by means of the first corner point to determine a target two-dimensional image similar to the first two-dimensional image.
The second two-dimensional images corresponding to the three-dimensional images are stored in the image database, and the second two-dimensional images can be from two-dimensional images for displaying the three-dimensional images under a certain view angle or from two-dimensional images under a certain view angle which are calculated on the three-dimensional images. Because the three-dimensional image has three-dimensional characteristics, more information is carried, the three-dimensional image under a single view angle can only be displayed locally, and the three-dimensional image in the image database can be stored with the second two-dimensional image under a plurality of view angles correspondingly.
The viewing angle difference calculating module 430 is configured to calculate a viewing angle difference between the first two-dimensional image and the target two-dimensional image.
The view angle difference calculating module 430 may calculate the view angle difference between the first two-dimensional image and the target two-dimensional image according to the first corner of the first two-dimensional image and the second corner corresponding to the target two-dimensional image by using the characteristics of the visual transformation.
The three-dimensional image acquisition module 440 is configured to determine, as a search result, a three-dimensional image corresponding to the target two-dimensional image having a view angle difference smaller than the first threshold.
Further judging the accuracy of corner matching, the three-dimensional image acquisition module 440 detects whether the viewing angle difference is within the error range, and if the viewing angle difference is within the error range, that is, the viewing angle difference is smaller than the first threshold, determines the three-dimensional image corresponding to the target two-dimensional image as a search result.
According to the three-dimensional image searching system, according to the first two-dimensional image of the easily-collected searching object, the matching of the images is carried out by means of the corner points which are easy to identify in the two-dimensional image, so that the three-dimensional image is searched out and used as a searching result, and the three-dimensional image searching is realized. In addition, the two-dimensional image is easy to obtain, and the corner serving as the characteristic is easy to identify, so that the searching efficiency is greatly improved; meanwhile, the three-dimensional characteristics of the searched object are considered, and the accuracy of three-dimensional image searching is improved according to the judgment of the visual angle difference.
For specific limitations of the three-dimensional image search system, reference may be made to the above limitations of the three-dimensional image search method, and no further description is given here. The respective modules in the above three-dimensional image search system may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
As shown in fig. 5, fig. 5 is a schematic diagram of an internal structure of the computer device in one embodiment. The computer device includes a processor, a non-volatile storage medium, a memory, and a network interface connected by a system bus. The nonvolatile storage medium of the computer device stores an operating system, a database and a computer program, and when the computer program is executed by the processor, the processor can realize a three-dimensional image searching method. The processor of the computer device is used to provide computing and control capabilities, supporting the operation of the entire computer device. The memory of the computer device may store a computer program which, when executed by the processor, causes the processor to perform a three-dimensional image search method. The network interface of the computer device is for communicating with a terminal connection. It will be appreciated by those skilled in the art that the structure shown in fig. 5 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the three-dimensional image search method of any of the embodiments described above when the computer program is executed by the processor.
In one embodiment, a storage medium having stored thereon computer readable instructions is provided, which when executed by a processor, implement the steps of the three-dimensional image search method of any of the embodiments described above.
It should be understood that, although the steps in the flowcharts of the figures are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited in order and may be performed in other orders, unless explicitly stated herein. Moreover, at least some of the steps in the flowcharts of the figures may include a plurality of sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, the order of their execution not necessarily being sequential, but may be performed in turn or alternately with other steps or at least a portion of the other steps or stages.
The foregoing is only a partial embodiment of the present invention, and it should be noted that it will be apparent to those skilled in the art that modifications and adaptations can be made without departing from the principles of the present invention, and such modifications and adaptations are intended to be comprehended within the scope of the present invention.
Claims (8)
1. A three-dimensional image search method, comprising the steps of:
acquiring a first two-dimensional image of a search object, and calculating a first corner of the first two-dimensional image;
matching the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in an image database, and determining a target two-dimensional image from the second two-dimensional image, wherein the method comprises the following steps: calculating a first similarity measurement value between each second two-dimensional image and each first two-dimensional image according to a second corner point and the first corner point of each second two-dimensional image in the image database; acquiring view angle weights of the second two-dimensional images in the three-dimensional images; calculating a second similarity metric value of the second two-dimensional image according to the view angle weight of the second two-dimensional image and the first similarity metric value of the second two-dimensional image; selecting a second two-dimensional image with the second similarity measurement value within a second preset threshold value as the target two-dimensional image;
calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;
and determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result.
2. The three-dimensional image search method according to claim 1, wherein the step of acquiring a first two-dimensional image of a search object, calculating a first corner point of the first two-dimensional image, comprises:
acquiring two original images of the search object through the camera equipment within a preset time interval;
respectively calculating a third corner point and a fourth corner point corresponding to the original image;
and carrying out corner screening on the third corner according to the fourth corner to obtain the first corner.
3. The three-dimensional image search method according to claim 1, wherein the step of calculating a first similarity metric value between each of the second two-dimensional images and the first two-dimensional image comprises:
acquiring a second corner point corresponding to a second two-dimensional image of any one three-dimensional image from the image database;
matching the first corner point with the second corner point, and extracting corner point pairs at the same position;
and calculating a first similarity value of the second two-dimensional image according to the corner pairs.
4. The three-dimensional image search method according to claim 3, wherein the step of calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image comprises:
establishing a perspective transformation matrix according to coordinates of the corner pairs in the target two-dimensional image and the first two-dimensional image respectively;
and calculating the visual angle difference according to the perspective transformation matrix.
5. The three-dimensional image search method according to claim 1, characterized by comprising, after the step of determining as the search result a three-dimensional image corresponding to a target two-dimensional image having a view angle difference smaller than a first threshold value:
calculating an observation view angle of the first two-dimensional image according to the view angle difference and the view angle of the target two-dimensional image in the three-dimensional image;
and displaying the three-dimensional image of the target at the observation view angle.
6. A three-dimensional image search system, comprising:
the corner calculation module is used for acquiring a first two-dimensional image of the search object and calculating a first corner of the first two-dimensional image;
the two-dimensional image matching module is used for matching the first corner with a second corner of a second two-dimensional image corresponding to each three-dimensional image in an image database, and determining a target two-dimensional image from the second two-dimensional image, and comprises the following steps: calculating a first similarity measurement value between each second two-dimensional image and each first two-dimensional image according to a second corner point and the first corner point of each second two-dimensional image in the image database; acquiring view angle weights of the second two-dimensional images in the three-dimensional images; calculating a second similarity metric value of the second two-dimensional image according to the view angle weight of the second two-dimensional image and the first similarity metric value of the second two-dimensional image; selecting a second two-dimensional image with the second similarity measurement value within a second preset threshold value as the target two-dimensional image;
a viewing angle difference calculation module for calculating a viewing angle difference between the first two-dimensional image and the target two-dimensional image;
and the three-dimensional image acquisition module is used for determining a three-dimensional image corresponding to the target two-dimensional image with the visual angle difference smaller than the first threshold value as a search result.
7. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the three-dimensional image search method according to any one of claims 1 to 5 when the computer program is executed by the processor.
8. A computer-readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the three-dimensional image search method of any one of claims 1 to 5.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910656211.3A CN110472085B (en) | 2019-07-19 | 2019-07-19 | Three-dimensional image searching method, system, computer device and storage medium |
| PCT/CN2019/118153 WO2021012523A1 (en) | 2019-07-19 | 2019-11-13 | Three-dimensional image search method and system, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910656211.3A CN110472085B (en) | 2019-07-19 | 2019-07-19 | Three-dimensional image searching method, system, computer device and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110472085A CN110472085A (en) | 2019-11-19 |
| CN110472085B true CN110472085B (en) | 2023-06-20 |
Family
ID=68508326
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910656211.3A Active CN110472085B (en) | 2019-07-19 | 2019-07-19 | Three-dimensional image searching method, system, computer device and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN110472085B (en) |
| WO (1) | WO2021012523A1 (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113793422B (en) * | 2021-08-13 | 2024-02-23 | 深圳安泰创新科技股份有限公司 | Display control method for three-dimensional model, electronic device and readable storage medium |
| CN116274893B (en) * | 2023-02-24 | 2024-02-06 | 北京瓦特曼智能科技有限公司 | Method, processor, device and storage medium for detecting erosion amount of ladle lining |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104714986A (en) * | 2013-12-12 | 2015-06-17 | 三纬国际立体列印科技股份有限公司 | Three-dimensional picture searching method and three-dimensional picture searching system |
| EP3023909A1 (en) * | 2014-11-18 | 2016-05-25 | Thomson Licensing | Method and apparatus for ranking 2D candidate images |
| CN107481315A (en) * | 2017-06-29 | 2017-12-15 | 重庆邮电大学 | A kind of monocular vision three-dimensional environment method for reconstructing based on Harris SIFT BRIEF algorithms |
| CN109708649A (en) * | 2018-12-07 | 2019-05-03 | 中国空间技术研究院 | A kind of attitude determination method and system of remote sensing satellite |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2004164503A (en) * | 2002-11-15 | 2004-06-10 | Olympus Corp | Three-dimensional model retrieval method, three-dimensional model retrieval device, three-dimensional model retrieval program and three-dimensional model retrieval system |
| US9280597B2 (en) * | 2009-12-01 | 2016-03-08 | Apple Inc. | System and method for customizing search results from user's perspective |
| CN103116904A (en) * | 2012-10-23 | 2013-05-22 | 北京航空航天大学深圳研究院 | Two-dimensional feature extraction system and two-dimensional feature extraction method of three-dimensional model |
| CN103177098B (en) * | 2013-03-19 | 2015-08-26 | 浙江大学 | A kind of method of the three-dimensional model search based on Freehandhand-drawing figure |
| CN106484692B (en) * | 2015-08-25 | 2019-11-29 | 北京师范大学 | A kind of method for searching three-dimension model |
| CN107247752B (en) * | 2017-05-27 | 2020-05-29 | 西安电子科技大学 | Image retrieval method based on corner descriptor |
-
2019
- 2019-07-19 CN CN201910656211.3A patent/CN110472085B/en active Active
- 2019-11-13 WO PCT/CN2019/118153 patent/WO2021012523A1/en active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104714986A (en) * | 2013-12-12 | 2015-06-17 | 三纬国际立体列印科技股份有限公司 | Three-dimensional picture searching method and three-dimensional picture searching system |
| EP3023909A1 (en) * | 2014-11-18 | 2016-05-25 | Thomson Licensing | Method and apparatus for ranking 2D candidate images |
| CN107481315A (en) * | 2017-06-29 | 2017-12-15 | 重庆邮电大学 | A kind of monocular vision three-dimensional environment method for reconstructing based on Harris SIFT BRIEF algorithms |
| CN109708649A (en) * | 2018-12-07 | 2019-05-03 | 中国空间技术研究院 | A kind of attitude determination method and system of remote sensing satellite |
Non-Patent Citations (1)
| Title |
|---|
| 基于局部特征的大视角图像匹配;赵鹏图;达飞鹏;;光学学报(第05期);133-140 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110472085A (en) | 2019-11-19 |
| WO2021012523A1 (en) | 2021-01-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP5261501B2 (en) | Permanent visual scene and object recognition | |
| CN109544599B (en) | Three-dimensional point cloud registration method based on camera pose estimation | |
| CN104685513A (en) | Feature based high resolution motion estimation from low resolution images captured using an array source | |
| CN109934873B (en) | Method, device and equipment for acquiring marked image | |
| CN110472085B (en) | Three-dimensional image searching method, system, computer device and storage medium | |
| CN106295710B (en) | Image local feature matching process, device and terminal based on non-geometric constraint | |
| US20230394832A1 (en) | Method, system and computer readable media for object detection coverage estimation | |
| CN110793441B (en) | High-precision object geometric dimension measuring method and device | |
| CN110120012B (en) | Video stitching method for synchronous key frame extraction based on binocular camera | |
| Teixeira et al. | Epipolar based light field key-location detector | |
| CN116823816B (en) | Detection equipment and detection method based on security monitoring static memory | |
| WO2013164043A1 (en) | Method and system for determining a color mapping model able to transform colors of a first view into colors of at least one second view | |
| CN110930386B (en) | Image processing method, device, equipment and storage medium | |
| CN116091998A (en) | Image processing method, device, computer equipment and storage medium | |
| CN105608423A (en) | Video matching method and device | |
| CN113298871B (en) | Map generation method, positioning method, system thereof, and computer-readable storage medium | |
| Zhu et al. | Global and local geometric constrained feature matching for high resolution remote sensing images | |
| JP6055307B2 (en) | Corresponding point search device, camera posture estimation device, and programs thereof | |
| JP6717769B2 (en) | Information processing device and program | |
| JP6468642B2 (en) | Information terminal equipment | |
| CN112967398B (en) | Three-dimensional data reconstruction method and device and electronic equipment | |
| Urmanov et al. | Computer methods of image processing of volcanoes | |
| CN116363031B (en) | Imaging method, device, equipment and medium based on multidimensional optical information fusion | |
| CN114750147B (en) | Space pose determining method and device of robot and robot | |
| CN116863085B (en) | Three-dimensional reconstruction system, three-dimensional reconstruction method, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |