KR102223269B1 - Image search and matching method for scan region re-tracking and imaging processing apparatus therefor - Google Patents

Abstract

Disclosed are an image search and matching method for re-detection of a scan area, and an image processing apparatus performing the same. An image search and matching method for re-detection of a scan area according to an embodiment includes the steps of acquiring input image data partially scanned a target using an oral scanner and extracting feature information of each input image data, and feature information Generating a descriptor that can be distinguished from feature information of other input images at the extracted location, matching between the generated descriptors, calculating movement and rotation components around the matched descriptor, and the calculated And fusing the input image data using the movement and rotation components.

Description

[Image search and matching method for scan region re-tracking and imaging processing apparatus therefor}

The present invention relates to image analysis and processing technology.

Recently, dentistry has used a lot of dental oral scanners to generate data on the patient's oral cavity. In the case of the camera of the oral scanner, the field of view (FOV) is narrow, so the entire area cannot be scanned at once. . Accordingly, data on the whole jaw can be obtained by dividing the whole jaw into several zones and scanning, and matching images of several zones acquired through the scan. However, when matching images for multiple areas, the user must design the location and scan by re-detecting the scan location when the scan fails, so that the scan is possible only at the designated location and the speed decreases.

According to an embodiment, the user can freely scan, and even if the scan fails in the middle, a new scan is performed based on another area where the scan has already been performed, thereby increasing the user's convenience. And a matching method and an image processing apparatus performing the same.

Furthermore, since re-detection is performed based on the entire area, an image search and matching method and an image processing apparatus that perform the same are proposed, which can prevent false detections that are erroneously tracked when another area is captured.

An image search and matching method for re-detection of a scan area according to an embodiment includes the steps of acquiring input image data partially scanned a target using an oral scanner and extracting feature information of each input image data, and feature information Generating a descriptor that can be distinguished from feature information of other input images at the extracted location, matching between the generated descriptors, calculating movement and rotation components around the matched descriptor, and the calculated And fusing the input image data using the movement and rotation components.

In the step of extracting the feature information, the center coordinate (0,0,cz) and the camera coordinate (0,0,0) of the 3D model data as input image data may be extracted as feature points, where x and y 0 represents the center of the x-axis and y-axis, 0 of z represents the current position of the camera, and cz represents the distance between the camera and the center of the 3D model data.

In the step of generating the descriptor, the descriptor may be generated by calculating a relationship value representing the association between the central point and the surrounding points through comparison between the central point and the surrounding points based on the point group data. In this case, a relationship value may be calculated by using an angle between the normal vector of the central point of the point group data and the normal vector of the surrounding points.

In the step of generating the descriptor, the descriptor may be generated by calculating the relationship between the camera and the center point of the point group data based on the camera's view point. At this time, the angle (α) between the vector (Vp-Pi) between the center point (Vp) of the camera and one point (Pi) constituting the point group data, and the normal vector (ni) of the point (Pi) of the point group data By calculating, the relationship between the point Pi and the camera can be calculated.

In the step of generating the descriptor, when a plurality of relationship values are calculated, the relationship values calculated using the histogram method may be grouped into one descriptor.

The matching between descriptors includes calculating an average and variance for descriptors having the same value in the histogram when a scan fails, and calculating similarity between descriptors by comparing the descriptors in order from the highest to the lowest variance value. And, partially first matching descriptors having high similarity, and performing matching using a cascade method of entering a next matching step if the matching result is successful.

In the step of calculating the movement and rotation components, the movement and rotation components may be calculated using a normal vector and a distribution of data around the matched descriptor.

The image search and matching method further includes performing secondary fusion using ICP (Iterative Closest Point), and verifying the fusion result using the average distance calculated through ICP and the ratio of the matched vertices. can do.

An image processing apparatus according to another embodiment includes a location selector that obtains input image data partially scanned a target using an oral scanner and extracts feature information of each input image data, and a location selector that extracts feature information of each input image data; A descriptor generation unit that generates a descriptor that can be distinguished from the feature information of the input image, a matching unit that matches the generated descriptors, and calculates movement and rotation components around the matched descriptor, and uses the calculated movement and rotation components. And a fusion unit for fusing the input image data.

Based on the point group data, the descriptor generator calculates a relationship value representing the association between the central point and the surrounding points through comparison between the central point and its surrounding points, and the normal vector of the central point of the point group data and the normal vector of the surrounding points The relationship value can be calculated using the angle between.

The descriptor generator generates a descriptor by calculating the relationship between the camera and the center point of the point group data centering on the camera's view point, but the center point (Vp) of the camera and one point constituting the point group data ( The relationship between the corresponding point Pi and the camera may be calculated by calculating the angle α between the vector Vp-Pi between Pi) and the normal vector ni of the point Pi of the point group data.

The matching unit calculates the average and variance of descriptors having the same value in the histogram when a scan fails, and calculates the similarity between descriptors by comparing the descriptors in order from the highest to the lowest variance value, and partially extracts the descriptors with high similarity. Matching is performed first, and if the matching result is successful, matching may be performed using a cascade method of entering the next matching step.

The image processing apparatus may further include a verification unit for verifying the fusion result by using the average distance calculated through the ICP and the ratio of the matched vertices.

According to an image search and matching method for re-detection of a scan area according to an embodiment and an image processing apparatus performing the same, an environment in which a user can scan as desired is provided, and even if a scan fails in the middle, a scan has already been performed. Convenience for the user can be increased by performing a new scan based on a different area.

In addition, since re-detection is performed on the basis of the entire area, it is possible to prevent the occurrence of false detections that are erroneously tracked when another area that has already occurred is photographed.

Furthermore, it is possible to increase the accuracy of the scan by removing unnecessary parts through fusion verification.

In addition, it is possible to speed up the matching of descriptors by using the average variance and cascade method of the descriptors.

1 is a diagram for explaining a situation in which retrace is difficult due to lack of data after a scan tracking failure;
2 is a diagram showing a state in which a scan with increased degrees of freedom is provided through re-detection;
3 is a diagram illustrating a configuration of a scanning system capable of image search and image matching for re-detection of a scan area according to an embodiment of the present invention;
4 is a diagram showing a detailed configuration of the image processing apparatus of FIG. 3 according to an embodiment of the present invention;
5 is a diagram showing an example of input image data according to an embodiment of the present invention;
6 is a diagram showing an example of extracting feature information of input image data according to an embodiment of the present invention;
7 is a diagram showing an example of generating a descriptor centering on model data according to an embodiment of the present invention;
8 is a diagram illustrating an example of generating a view point descriptor according to another embodiment of the present invention;
9 is a diagram showing an example of a cascade method according to an embodiment of the present invention;
10 is a diagram illustrating a flow of an image search and matching method for re-detecting a scan area according to an embodiment of the present invention.

Advantages and features of the present invention, and a method of achieving them will become apparent with reference to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and the general knowledge in the technical field to which the present invention pertains. It is provided to completely inform the scope of the invention to the possessor, and the invention is only defined by the scope of the claims. The same reference numerals refer to the same elements throughout the specification.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted, and terms to be described later are in the embodiment of the present invention. These terms are defined in consideration of the functions of the user and may vary according to the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout the present specification.

Combinations of each block of the attached block diagram and each step of the flowchart may be executed by computer program instructions (execution engine), and these computer program instructions are used on a processor of a general-purpose computer, special purpose computer, or other programmable data processing device. As can be mounted, the instructions executed by the processor of a computer or other programmable data processing device generate means for performing the functions described in each block of the block diagram or each step of the flowchart.

These computer program instructions may also be stored in a computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing device to implement a function in a particular manner, so that the computer-usable or computer-readable memory It is also possible to produce an article of manufacture in which the instructions stored in the block diagram contain instruction means for performing the functions described in each block of the block diagram or each step of the flowchart.

In addition, since computer program instructions can be mounted on a computer or other programmable data processing device, a series of operation steps are performed on a computer or other programmable data processing device to create a computer-executable process. It is also possible that the instructions for performing the data processing apparatus provide steps for executing the functions described in each block in the block diagram and in each step in the flowchart.

In addition, each block or each step may represent a module, segment, or part of code containing one or more executable instructions for executing specified logical functions, and in some alternative embodiments mentioned in the blocks or steps. It should be noted that it is also possible for functions to occur out of order. For example, two blocks or steps shown in succession may in fact be performed substantially simultaneously, and the blocks or steps may be performed in the reverse order of a corresponding function as necessary.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, the embodiments of the present invention exemplified below may be modified in various other forms, and the scope of the present invention is not limited to the embodiments described below. Embodiments of the present invention are provided to more completely describe the present invention to those of ordinary skill in the art to which the present invention pertains.

1 is a diagram for describing a situation in which retrace is difficult due to lack of data after a scan tracking failure.

According to the scanning process using an oral scanner, 3D point cloud data (Point Cloud Data: PCD, hereinafter referred to as'PCD') is generated from various partial images captured by the oral scanner. After feature extraction and variance calculation of the current PCD generated from the currently photographed image and the previous PCD generated from the previously photographed image, the movement and rotation components are calculated by comparing them with each other. Then, using the calculation result, the current PCD and the previous PCD are fused to create a single model data, and the scan is performed.

In such a scan process, it is checked whether or not the scan trace is successful according to whether the previous PCD and the current PCD are accurately matched. If the scan tracking is correct, you can proceed as it is. For this reason, exception handling is necessary when scan tracking fails. As an exception processing method, there is a method of continuously comparing and re-detecting the PCD generated from the newly captured image based on the PCD determined as the last scan tracking success until the scan tracking is successful. In the case of such an exception handling method, as shown in FIG. 1, if there is less data, a similar area is also reduced, so re-detection is difficult. Even if it is re-detected, there are cases of extracting problematic movement and rotational components, and false detection may occur in which the incorrect part is erroneously tracked as the right part. When such a problem occurs, it cannot be solved with the existing exception handling, which causes the inconvenience of the user having to scan again from the beginning.

2 is a view showing a state in which a scan with increased degrees of freedom is provided through re-detection.

According to the above-described exception handling method with reference to FIG. 1, there is an inconvenience in that the user must scan the whole at once. This is because in this way, the user always has to scan another part based on the last scanned area. As shown in FIG. 2, after scanning the outer side of the tooth from left to right once, it is not possible to attempt to scan the upper side of the tooth from left to right. In other words, there is a difficulty in that the user has to try to scan only as given.

The technical problem of the present invention is to overcome the limitation of re-detection presented as a problem with reference to FIGS. 1 and 2 and to provide a function that allows a user to freely scan. For example, as shown in FIG. 2, it is intended to provide an environment in which a user can scan as desired. In addition, even if the scan fails in the middle, convenience for the user may be increased by performing a new scan based on another area in which the scan has already been performed. Furthermore, since detection is performed based on the entire area, it is possible to prevent the occurrence of erroneous detection that is erroneously tracked when another area that has already occurred is captured. Hereinafter, the configuration of the present invention for solving the above-described technical problem will be described in detail.

3 is a diagram illustrating a configuration of a scanning system capable of image search and image matching for re-detection of a scan area according to an embodiment of the present invention.

Referring to FIG. 3, the scanning system 1 includes an oral scanner 2 and an image processing device 3.

The oral scanner 2 scans the user's oral cavity through a camera to obtain image data inside the oral cavity. The image processing apparatus 3 receives image data from the oral scanner 2 and processes it using a processor. The image processing device 3 is an electronic device capable of executing an image processing program. Electronic devices include computers, notebook computers, laptop computers, tablet PCs, smartphones, mobile phones, personal media players (PMPs), personal digital assistants (PDAs), and the like. Image processing programs include scan programs, CAD programs, and guide design programs. In addition, it can be applied to programs for medical image processing, including dental implant surgery.

The oral scanner 2 may output light by driving a light source of internal illumination, and then illuminate the inside of the user's oral cavity through light irradiated to the outside along a light path. Then, the light reflected from the inside of the oral cavity again reaches the image sensor of the camera 20 along the optical path, thereby obtaining image data inside the oral cavity. The oral model data acquired through the oral scanner 2 is data having information on actual teeth including damaged teeth. The oral model data may be obtained by scanning a plaster model created after a patient's oral cavity with a 3D scanner. As another example, it may be obtained by scanning the inside of the patient's oral cavity using a 3D intra-oral scanner.

In FIG. 3, the image processing apparatus 3 has a structure separated from the oral scanner 2, but a structure in which the image processing apparatus 3 is located in the processor of the oral scanner 2 is also possible. In addition, in the present specification, the image processing apparatus 3 mainly describes image processing in which the image processing apparatus 3 searches and matches the oral image acquired through the oral scanner 2, but is not limited to dental use, and the same applies to other medical image processing. State that applicable.

4 is a diagram showing a detailed configuration of the image processing apparatus of FIG. 3 according to an embodiment of the present invention.

3 and 4, the image processing apparatus 3 includes an input unit 30, a processor 32, an output unit 34, and a storage unit 36.

The input unit 30 receives image data captured from the oral scanner 2. In this case, the captured image data is an image that partially scans a partial area of the object. The partially scanned image data is fused into one through the processor 32. The image data captured by the oral scanner 2 may be 2D image data converted into a digital form from analog 2D image data captured by the camera. The analog 2D image data is a signal sensed by the image sensor of the camera 110, and the digital image data refers to data recognized as an image by a device such as a PC according to the image data standard.

The processor 32 generates a control signal for controlling the scanning operation of the oral scanner 2 and controls each component of the image processing apparatus 3. The processor 32 according to an embodiment includes a data conversion unit 300, a location selection unit 310, a descriptor generation unit 320, a matching unit 340, and a fusion unit 350, and a verification unit 360 ) May be further included.

The processor 32 fuses the input image data scanned through the oral scanner 2 into one. The input image data may be data converted through the data conversion unit 300. For example, the data conversion unit 300 may convert 2D image data received from the oral scanner 2 into 3D image data. The 3D image data may be oral model data. The data conversion unit 300 may convert oral model data into 2D RGB data.

The location selector 310 selects a location for a portion to be feature information of each input image data in order to search for a portion to be fused from the partially scanned input image data. For example, the location selector 310 may extract a center coordinate (0,0,cz) and a camera coordinate (0,0,0) of 3D model data that is input image data as feature points. Here, 0 of x and y represents the center of the x-axis and y-axis, 0 of z represents the position of the current camera, and cz represents the distance between the camera and the center of the 3D model data.

The descriptor generation unit 320 generates a descriptor that can be distinguished from the feature information of other input images at the location from which the feature information is extracted through the location selector 310. Descriptor generation methods include a descriptor generation method centering on model data and a descriptor generation method centering on the camera's view point. A method of generating a descriptor centering on model data will be described later with reference to FIG. 7, and a method of generating a descriptor centering on a camera view point will be described later with reference to FIG. 8. When generating a descriptor, if a plurality of relationship values are calculated, the relationship values calculated using the histogram method may be grouped into one descriptor.

The matching unit 340 matches between descriptors generated through the descriptor generating unit 320. The matching unit 340 according to an embodiment accumulates the descriptor of the corresponding image data in the storage unit 36 when the scan is successful. Conversely, when a scan fails, the average and variance of the descriptors having the same value in the histogram are calculated, and the similarity between the descriptors is calculated by comparing the descriptors in the order from the highest to the lowest variance value. In this case, descriptor matching speed is improved by partially matching descriptors having high similarity first. If the matching result is successful, matching is performed using a cascade method that enters the next matching step. Descriptor matching can be accelerated by using the average variance of descriptors and the cascade method.

The fusion unit 350 calculates movement and rotation components based on the matched descriptor, and fuses the input image data using the calculated movement and rotation components. For example, after moving and rotating the current image data according to the calculated value of the movement and rotation component, it is fused with the previous image data. When calculating the movement and rotation components, the movement and rotation components may be calculated using a normal vector and a distribution of data around the matched descriptor.

The fusion unit 350 may additionally perform secondary fusion using an Iterative Closest Point (ICP). The verification unit 360 verifies the fusion result using the average distance calculated through the ICP and the ratio of the matched vertices. It is possible to provide a more accurate scan system by removing unnecessary parts by verifying whether the fusion has been correctly fused.

The storage unit 36 stores various data including the obtained oral model data. Descriptors generated by the processor 32 may be accumulated and stored in the storage unit 36. The output unit 34 displays the scan result on the screen.

5 is a diagram illustrating an example of input image data according to an embodiment of the present invention.

3 and 5, the image processing apparatus 3 acquires 3D oral model data 110 and 2D RGB image data 120 as input image data from image data photographed from the oral scanner 2 do.

6 is a diagram illustrating an example of extracting feature information of input image data according to an embodiment of the present invention.

Referring to FIG. 6, feature information of each input image data may be designated as the center of the input image data. For example, in the case of 3D model data, (0,0,0) and (0,0,cz) (600) are used as feature points. The meaning of 0 of x and y indicates the center of the x-axis and y-axis of the 3D model data, as shown in FIG. 6. Z of 0 represents the current camera position, and cz represents the distance between the camera and the center of the 3D model data. 7 to be described later, the current position of the camera 20 is (0,0,0). After determining the feature information of the model data using these two central coordinates, a descriptor capable of distinguishing the determined feature information from other feature information is generated. As a descriptor generation method, we propose a descriptor generation method centering on model data and a descriptor generation method centering on the camera's view point. An embodiment of this will be described later with reference to FIGS. 7 and 8.

7 is a diagram illustrating an example of generating a descriptor centering on model data according to an embodiment of the present invention.

3 and 7, the image processing apparatus 3 generates a descriptor based on model data. The model data center may be configured based on (0,0,cz) 600, which is the center coordinate of the model data, as described above with reference to FIG. 6. The descriptor centered on the model data can express the peripheral relationship between PCDs, and in particular, it is a method of generating a descriptor robust to rotation using a Darboux frame. Since it is robust to rotation, there is a characteristic that image matching is simplified because it is not necessary to consider rotation in image matching, and only movement is considered.

As one of the descriptor generation methods, we propose a method of calculating a relationship value representing the association between points through comparison between the central point and the peripheral points constituting the PCD. At this time, prior to the comparison, the feature information of the PCD is converted into a double frame coordinate axis that is robust to rotation, and the peripheral relationship value is calculated by using the difference between the center point and the normal vector of the surrounding point, which is the matching point. For example, an angle between a normal vector of a central point of a predetermined PCD and a normal vector of a peripheral point is calculated to calculate a peripheral relationship value. The degree of similarity with the periphery can be known according to the angle between each center point and the normal vector of the periphery point.

As shown in FIG. 7, there is one central point 600, but since there are several peripheral points, several of the above feature values are also output. The histogram representation method is used to group these multiple feature values into one descriptor. The histogram representation method is a method of sampling feature values at a certain size, and increasing the cumulative number of the corresponding section by one when the feature value enters a specific sample section. When accumulated one by one, several feature values are expressed as one histogram, which is used as a descriptor. The histogram representation method will be described later with reference to FIG. 9.

8 is a diagram illustrating an example of generating a view point descriptor according to another embodiment of the present invention.

3 and 8, the image processing apparatus 3 generates a descriptor configured based on the position of the camera. The position of the camera is (0,0,0) and is expressed in global center coordinates. As a feature of this descriptor, the relationship between the PCD and the viewpoint of the camera is expressed so that it is possible to distinguish where the image is taken. Since the descriptor generated around the model data described above with reference to FIG. 7 only sees the shape of the model, it is impossible to distinguish it if it has a similar shape. This limitation is characterized by the relationship between the camera and the model data, so that even if the camera has a similar shape, it can be distinguished when photographed from a different angle.

According to a method for generating a descriptor centered on a camera viewpoint according to an embodiment, a vector (Vp-Pi) between a center point (Vp) 610 of the camera 20 and one point (Pi) 600 of the PCD, and By calculating the angle α between the normal vector ni of the point Pi, the relationship between the point and the camera 20 is calculated. Since it is composed of several points in one model data, the relationship values between the camera 20 and the points are calculated as several. In order to group these values into one descriptor, a descriptor is generated using the histogram method described above.

When the scan is successful, the image processing apparatus 3 generates a descriptor of the corresponding image and accumulates it in the storage unit. Conversely, if the scan fails, the descriptor comparison process starts to find out where the corresponding model data was taken. In order to efficiently match when the number of descriptors is greater than a certain number, it is necessary to compare meaningful ones of each histogram value first. The average and variance of descriptors having the same value among the histograms are calculated to classify the significant ones. Among these variance values, the higher the probability of classification, the higher the probability of classification, and the smaller the higher the probability of poor classification. Accordingly, the comparison of those with a large variance value is performed first, and those with a low variance value are compared later in order to improve the descriptor matching speed.

9 is a diagram illustrating an example of a cascade method according to an embodiment of the present invention.

Referring to FIG. 9, in the matching method, a descriptor is partially first matched, and when the matching is greater than a predetermined threshold, it is determined as a matching failure, and when the matching is successful, a cascade method may be used. The reason for using the cascade method is to simplify the process because most of the model data can be easily identified due to some descriptors. Through this method, it is possible to speed up the descriptor search.

10 is a diagram illustrating a flow of an image search and matching method for re-detecting a scan area according to an embodiment of the present invention.

Referring to FIGS. 3 and 10, the image processing apparatus 3 acquires input image data that partially scans an object using the oral scanner 2 and extracts feature information of each input image data (S1010). At this time, the center coordinate (0,0,cz) and the camera coordinate (0,0,0) of the 3D model data as input image data may be extracted as feature points. Here, 0 of x and y represents the center of the x-axis and y-axis, 0 of z represents the position of the current camera, and cz represents the distance between the camera and the center of the 3D model data.

Subsequently, the image processing apparatus 3 generates a descriptor capable of distinguishing from the feature information of other input images at the location from which the feature information is extracted (S1020). According to the first embodiment, a descriptor is generated by calculating a relationship value representing the association between the central point and the surrounding points through comparison between the central point and the surrounding points based on point group data. For example, the relationship value is calculated by using the angle between the normal vector of the center point of the point group data and the normal vector of the surrounding points. According to the second embodiment, a descriptor is generated by calculating a relationship between a camera and a center point of point group data based on a view point of the camera. For example, the angle (α) between the center point (Vp) of the camera and the vector (Vp-Pi) between one point (Pi) constituting the point group data, and the normal vector (ni) of the point (Pi) of the point group data ) To calculate the relationship between the point Pi and the camera.

In the descriptor generation step S1020, when a plurality of relationship values are calculated, the relationship values calculated using the histogram method may be grouped into one descriptor.

Subsequently, the image processing apparatus 3 matches the generated descriptors (S1030). For example, when a scan fails, the average and variance of descriptors having the same value in the histogram are calculated, and the similarity between the descriptors is calculated by comparing the descriptors from the order of the highest to the lowest of the variance values. Subsequently, the descriptors having high similarity are partially first matched, and if the matching result is successful, matching is performed using a cascade method in which the next matching step is entered.

Subsequently, the image processing apparatus 3 calculates movement and rotation components based on the matched descriptor (S1040). For example, movement and rotation components are calculated using a normal vector and a distribution of data around the matched descriptor.

Subsequently, the image processing apparatus 3 fuses the input image data using the calculated movement and rotation components (S1050). Furthermore, secondary fusion may be performed using ICP (Iterative Closest Point), and the fusion result may be verified using the average distance calculated through ICP and the ratio of matched vertices.

So far, the present invention has been looked at around the embodiments. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention may be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from a descriptive point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the above description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (15)

Obtaining input image data obtained by partially scanning different regions of the object using the same camera of the oral scanner and extracting feature information of each input image data;
Generating a descriptor representing relationship information between features or between features in order to distinguish it from feature information of another input image at a location from which the feature information is extracted;
Matching the generated descriptors;
Calculating movement and rotation components based on the matched descriptors; And
Fusing the input image data using the calculated movement and rotation components;
Image search and matching method for re-detection of the scan area comprising a. The method of claim 1, wherein the extracting feature information comprises:
The center coordinate (0,0,cz) and camera coordinate (0,0,0) of the 3D model data, which is the input image data, are extracted as feature points,
Here, 0 of x and y represents the centers of the x and y axes, 0 of z represents the position of the current camera, and cz represents the distance between the camera and the center of the 3D model data. Image search and matching method for. Acquiring input image data and extracting feature information of each input image data;
Generating a descriptor capable of distinguishing from the feature information of other input images at the location from which the feature information is extracted;
Matching the generated descriptors;
Calculating movement and rotation components based on the matched descriptors; And
Fusing the input image data using the calculated movement and rotation components;
Including,
The step of creating the descriptor is
An image search and matching method for re-detection of a scan area, characterized in that a descriptor is generated by calculating a relationship value representing the relationship between the central point and the surrounding points through comparison between the central point and its surrounding points based on the point group data. . The method of claim 3, wherein generating the descriptor
An image retrieval and matching method for re-detection of a scan area, characterized in that a relationship value is calculated using an angle between a normal vector of a central point of point group data and a normal vector of a neighboring point. Acquiring input image data and extracting feature information of each input image data;
Generating a descriptor capable of distinguishing from the feature information of other input images at the location from which the feature information is extracted;
Matching the generated descriptors;
Calculating movement and rotation components based on the matched descriptors; And
Fusing the input image data using the calculated movement and rotation components;
Including,
The step of creating the descriptor is
An image search and matching method for re-detection of a scan area, comprising generating a descriptor by calculating a relationship between a camera and a center point of point group data based on a camera view point. The method of claim 5, wherein generating the descriptor
By calculating the angle (α) between the vector (Vp-Pi) between the camera's center point (Vp) and one point (Pi) constituting the point group data, and the normal vector (ni) of the point (Pi) of the point group data An image search and matching method for re-detecting a scan area, characterized in that calculating a relationship between a corresponding point Pi and a camera. The method of claim 1, wherein generating the descriptor comprises:
An image search and matching method for re-detection of a scan area, characterized in that when a plurality of relationship values are calculated, the relationship values calculated using a histogram method are grouped into one descriptor. Acquiring input image data and extracting feature information of each input image data;
Generating a descriptor capable of distinguishing from the feature information of other input images at the location from which the feature information is extracted;
Matching the generated descriptors;
Calculating movement and rotation components based on the matched descriptors; And
Fusing the input image data using the calculated movement and rotation components;
Including,
The step of matching between descriptors is
Calculating an average and variance of descriptors having the same value in the histogram when a scan fails;
Calculating a similarity between the descriptors while comparing the descriptors from the order of the variance value to the lowest order;
Partially first matching descriptors having a similarity higher than a preset value; And
If the matching result is successful, performing matching using a cascade method of entering a next matching step;
Image search and matching method for re-detection of the scan area comprising a. Acquiring input image data and extracting feature information of each input image data;
Generating a descriptor capable of distinguishing from the feature information of other input images at the location from which the feature information is extracted;
Matching the generated descriptors;
Calculating movement and rotation components based on the matched descriptors; And
Fusing the input image data using the calculated movement and rotation components;
Including,
The steps of calculating the movement and rotation components are
An image search and matching method for re-detection of a scan area, characterized in that the motion and rotation components are calculated using a normal vector and a distribution of data based on the matched descriptor. Acquiring input image data and extracting feature information of each input image data;
Generating a descriptor capable of distinguishing from the feature information of other input images at the location from which the feature information is extracted;
Matching the generated descriptors;
Calculating movement and rotation components based on the matched descriptors; And
Fusing the input image data using the calculated movement and rotation components;
Including,
How to search and match video
Performing secondary fusion using ICP (Iterative Closest Point); And
Verifying the fusion result using the average distance calculated through ICP and the ratio of the matched vertices;
An image search and matching method for re-detecting the scan area, further comprising. A location selector for acquiring input image data obtained by partially scanning different regions of the object using the same camera of the oral scanner and extracting feature information of each input image data;
A descriptor generator for generating a descriptor representing relationship information between features or between features in order to distinguish it from feature information of other input images at the location from which the feature information is extracted;
A matching unit matching the generated descriptors; And
A fusion unit that calculates a movement and rotation component based on the matched descriptor and fuses the input image data using the calculated movement and rotation component;
Image processing apparatus comprising a. A location selector for acquiring input image data that partially scans an object using an oral scanner and extracting feature information of each input image data;
A descriptor generating unit generating a descriptor capable of distinguishing from the feature information of other input images at the location where the feature information is extracted;
A matching unit matching the generated descriptors; And
A fusion unit that calculates a movement and rotation component based on the matched descriptor and fuses the input image data using the calculated movement and rotation component;
Including,
The descriptor generator
Based on the point group data, a relationship value representing the association between the central point and the surrounding points is calculated through comparison between the central point and its surrounding points,
An image processing apparatus, comprising: calculating a relationship value by using an angle between a normal vector of a central point of point group data and a normal vector of a neighboring point. A location selector for acquiring input image data that partially scans an object using an oral scanner and extracting feature information of each input image data;
A descriptor generating unit generating a descriptor capable of distinguishing from the feature information of other input images at the location where the feature information is extracted;
A matching unit matching the generated descriptors; And
A fusion unit that calculates a movement and rotation component based on the matched descriptor and fuses the input image data using the calculated movement and rotation component;
Including,
The descriptor generator
By calculating the relationship between the camera and the center point of the point group data centering on the camera's view point, a descriptor is generated,
By calculating the angle (α) between the vector (Vp-Pi) between the camera's center point (Vp) and one point (Pi) constituting the point group data, and the normal vector (ni) of the point (Pi) of the point group data An image processing apparatus, characterized in that calculating a relationship between the point Pi and the camera. A location selector for acquiring input image data that partially scans an object using an oral scanner and extracting feature information of each input image data;
A descriptor generating unit generating a descriptor capable of distinguishing from the feature information of other input images at the location where the feature information is extracted;
A matching unit matching the generated descriptors; And
A fusion unit that calculates a movement and rotation component based on the matched descriptor and fuses the input image data using the calculated movement and rotation component;
Including,
Matching Department
When a scan fails, the average and variance of descriptors having the same value in the histogram are calculated,
Computing the similarity between the descriptors by comparing the descriptors from the order of the variance value to the lowest,
Descriptors with a similarity higher than a preset value are partially matched first,
When the matching result is successful, matching is performed using a cascade method of entering a next matching step. A location selector for acquiring input image data that partially scans an object using an oral scanner and extracting feature information of each input image data;
A descriptor generating unit generating a descriptor capable of distinguishing from the feature information of other input images at the location where the feature information is extracted;
A matching unit matching the generated descriptors; And
A fusion unit that calculates a movement and rotation component based on the matched descriptor and fuses the input image data using the calculated movement and rotation component;
Including,
The image processing device
A verification unit for verifying the fusion result using the average distance calculated through the ICP and the ratio of the matched vertices;
An image processing apparatus further comprising a.

Images