KR102621270B1 - Data processing method and date processing apparatus using thereof - Google Patents

Abstract

The data processing method according to the present invention includes obtaining a scan shot representing an object, aligning the scan shot to at least one preformed inactive cluster, and aligning the scan shot to the inactive cluster. Accordingly, the method includes optionally merging the active cluster including the scan shot with the inactive cluster.

Description

Data processing method and date processing apparatus using the same}

The present invention relates to a data processing method and a data processing device using the same.

Recently, the development of 3D scanners used to measure and analyze narrow areas such as the inside of the oral cavity has been actively conducted. 3D scanners are designed in a handheld form so that a user (typically a dentist) can easily scan a small area.

When using a 3D scanner, the user continuously scans the patient's mouth and ultimately obtains a 3D model representing the object (the inside of the patient's mouth or a plaster cast modeled thereon). A 3D model that is modeled and created in real time during the scanning process can be formed by aligning continuously acquired image data. Exemplarily, when three image data (first image data, second image data, and third image data) are acquired sequentially, the second image data is aligned with the first image data, and the third image data is aligned with the first image data. Can be aligned to the second image data, and a three-dimensional model can be created in this way.

However, conventionally, if alignment between image data failed once, 3D modeling did not proceed. Users had to be careful to ensure that alignment was performed properly by carefully scanning again from the area where alignment failed. Although there is a difference depending on the user's scanning skill, alignment failure may occur dozens of times (for example, more than 10 times) while the user scans one arch, and the user must rescan from the part where alignment failed. Inconvenience occurs during the process, and scanning time also increases.

Additionally, the user is generally looking at the object, and even if alignment failure is fed back on the display screen, it is practically impossible to alternately check the object and the display screen. In addition, if the user alternately checks the object and the display screen, the user's concentration may be dispersed and the time required to complete the 3D model may increase. 3D scanners are used to quickly obtain a high-quality 3D model. A problem arises in which the purpose of development is not understood.

Republic of Korea Patent Publication No. 10-2020-0115580 (published on October 7, 2020)

In order to solve the above-described problem, the present invention aligns scan shots obtained in the process of scanning an object and preformed inactive clusters, and merges the clusters through the alignment to finally create one three-dimensional model. Provides a data processing method.

Additionally, the present invention provides a data processing device in which a data processing method is performed through the action of organically connected components to ultimately create a three-dimensional model.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the above-described problem, the data processing method according to the present invention includes obtaining a scan shot representing an object, aligning the scan shot to at least one preformed inactive cluster, and aligning the scan shot to the inactive cluster. The method includes selectively merging the active cluster including the scan shot and the inactive cluster depending on whether the cutting step is aligned.

Additionally, the step of aligning the scan shot to the immediately preceding scan shot included in the active cluster may be performed simultaneously or sequentially. You can.

Additionally, the step of aligning to the inactive cluster may align feature points of the scan shot with feature points of the inactive cluster.

In addition, the selective merging step may include merging the inactive cluster into the active cluster when the scan shot is aligned with the immediately preceding scan shot and the scan shot is aligned with the inactive cluster, and the inactive cluster is aligned with the inactive cluster. Scan shots included in can be included in the active cluster.

Additionally, if the scan shot is aligned to the previous scan shot and the scan shot is not aligned to the inactive cluster, the scan shot may be included in the active cluster.

In addition, if the scan shot is not aligned to the immediately preceding scan shot and the scan shot is not aligned to the inactive cluster, the active cluster is deactivated and a new active cluster is created, and the scan shot is the new active cluster. Can be included in a cluster.

Additionally, if the scan shot is not aligned with the previous scan shot and the scan shot is aligned with the inactive cluster, the scan shot may be included in the inactive cluster.

Additionally, when the scan shot is included in the inactive cluster, the inactive cluster may be activated, and the active cluster that does not include the scan shot may be deactivated.

In addition, the scan shot, the active cluster, and the inactive cluster are displayed on a user interface screen having a plurality of areas, including a main area occupying a predetermined area and a sub area occupying an area smaller than the main area. The scan shot and the active cluster are displayed in the main area, and if the scan shot is included in the inactive cluster, the inactive cluster is activated and displayed in the main area, and the scan shot is not included in the inactive cluster. Active clusters may be deactivated and displayed in the sub-area.

Meanwhile, a data processing device according to the present invention includes a scanner that scans an object, aligns a scan shot obtained by scanning the object to at least one preformed inactive cluster, and includes the scan shot depending on whether the alignment occurs. It includes a control unit that selectively merges the active cluster and the inactive cluster, and a display unit that displays at least one of the scan shot, the inactive cluster, and the active cluster.

In addition, the control unit may include a scan shot generator that generates the scan shot based on the object scanned by the scanner, aligns the scan shot with the previous scan shot included in the active cluster, or It may include an aligner that aligns an inactive cluster, and a cluster manager that creates and manages the inactive cluster and the active cluster according to an alignment result of the aligner.

Additionally, the aligner may simultaneously or sequentially perform a process of aligning the scan shot with the immediately preceding scan shot and a process of aligning the scan shot with the inactive cluster.

In addition, when the scan shot is aligned to the immediately preceding scan shot and the scan shot is aligned to the inactive cluster, the cluster manager merges the inactive cluster into the active cluster, and merges the inactive cluster into the inactive cluster. Scan shots can be included in the active cluster.

Additionally, the cluster manager may include the scan shot in the active cluster when the scan shot is aligned with the immediately preceding scan shot and the scan shot is not aligned with the inactive cluster.

In addition, if the scan shot is not aligned with the previous scan shot and the scan shot is not aligned with the inactive cluster, the cluster manager deactivates the active cluster and creates a new active cluster, and the scan shot is not aligned with the inactive cluster. Shots may be included in the new active cluster.

Additionally, the cluster manager may include the scan shot in the inactive cluster when the scan shot is not aligned with the previous scan shot and the scan shot is aligned with the inactive cluster.

Additionally, the cluster manager may convert an active cluster in which the scan shots are not aligned into the inactive cluster, and convert an inactive cluster in which the scan shots are aligned into the active cluster.

In addition, the display unit displays a user interface having a plurality of areas including a main area occupying a predetermined area and a sub area occupying an area smaller than the main area, and the scan shot and the active cluster are displayed in the main area. area, and if the scan shot is included in the inactive cluster, the inactive cluster can be activated to display in the main area, and the active cluster can be deactivated to display in the sub-area.

By using the data processing method and the data processing device using the same according to the present invention, the scanning process is performed while minimizing the number of clusters created, which has the advantage of simplifying the calculation process and maximizing the scanning speed.

In addition, even if an alignment failure occurs, the data can be divided into clusters and the scanning process can proceed, and the user can perform a scan regardless of whether alignment is successful or not, which has the advantage of improving the user's freedom of scanning.

1 is a flowchart of a data processing method according to the present invention.
Figure 2 is for explaining a process of acquiring a scan shot representing an object.
Figure 3 is for explaining the alignment process between scan shots and clusters.
Figure 4 is for explaining feature points used as a medium for aligning scan shots and clusters.
Figure 5 is for explaining the detailed process of S130.
6 to 10 are for explaining a process in which data is processed depending on the success or failure of scan shot alignment and cluster alignment in the data processing method according to the present invention.
Figure 11 is for explaining the undo process using the third cluster as an example.
Figure 12 is for explaining the process of switching the screen by aligning scan shots to inactive clusters in the data processing method according to the present invention.
Figures 13 and 14 are for explaining the screen switching process between the active cluster and the inactive cluster.
Figure 15 is a configuration diagram of a data processing device according to the present invention.

Hereinafter, some embodiments of the present invention will be described in detail through illustrative drawings. When adding reference numerals to components in each drawing, it should be noted that identical components are given the same reference numerals as much as possible even if they are shown in different drawings. Additionally, when describing embodiments of the present invention, if detailed descriptions of related known configurations or functions are judged to impede understanding of the embodiments of the present invention, the detailed descriptions will be omitted.

In describing the components of the embodiment of the present invention, terms such as first, second, A, B, (a), and (b) may be used. These terms are only used to distinguish the component from other components, and the nature, sequence, or order of the component is not limited by the term. Additionally, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present application. No.

1 is a flowchart of a data processing method according to the present invention.

Referring to FIG. 1, the data processing method according to the present invention includes obtaining a scan shot representing an object (S110). The step of acquiring a scan shot (S110) may mean that the scanner receives light reflected from the surface of the object and acquires image data representing the object. The camera built into the scanner receives light entering the scanner and obtains scan shots based on the received light. At this time, the scan shot may be two-dimensional image data or a three-dimensional piece of stereoscopic data. Scan shots acquired by the scanner can be stored in the order in which they were acquired. A known storage device may be used to store the scan shot, and the storage device may be a hard disk drive, solid state drive (SSD), or a removable storage device (USB Flash drive, etc.).

FIG. 2 is for explaining a process of acquiring a scan shot 100 representing an object O.

Referring to FIG. 2 , a plurality of scan shots 100 may be acquired according to the direction in which the scanner moves and the angle at which the scanner is aimed at the object O. Exemplarily, as shown in FIG. 2, the object O is scanned to produce a first scan shot 101, a second scan shot 102, a third scan shot 103, and a fourth scan shot 104. , the fifth scan shot 105, the sixth scan shot 106, the seventh scan shot 107, the eighth scan shot 108, and the ninth scan shot 109 can be obtained. At this time, the user does not necessarily need to scan the object O sequentially from one direction to the other, and the user can scan the object O in a convenient way.

Referring to Figures 1 and 2, scan shots 100 obtained in the data processing method according to the present invention are aligned to be connected to each other. The alignment performed between scan shots 100 is referred to as ‘scan shot alignment.’ At this time, scan shot alignment may mean aligning a scan shot acquired at a specific point in time with a scan shot acquired at the previous point in time (i.e., the previous scan shot). The process of performing alignment between scan shots 100 may use an Iteration Closest Point (ICP) method, but is not necessarily limited thereto.

When performing alignment between scan shots 100, if the overlap section does not exist or is insufficient (lower than the standard value), scan shot alignment may fail. As shown in FIG. 2, the third scan shot 103 and fourth scan shot 104, and the sixth scan shot 106 and seventh scan shot 107 are not scan shot aligned with each other. In this case, the first to third scan shots 101, 102, and 103 are one group, the fourth to sixth scan shots 104, 105, and 106 are another group, and the seventh The scan shots to the ninth scan shots 107, 108, and 109 may be classified and stored as another group. At this time, the group in which the scan shots 100 are each stored is referred to as a cluster (200), and the cluster 200 may be a set of scan shots 100 including at least one scan shot 100. . The cluster 200 may be newly created or different clusters 200 may be merged depending on whether the scan shot 100 obtained by scanning the object is aligned.

Below, inactive clusters and active clusters are described.

Figure 3 is for explaining the alignment process between scan shots and clusters.

Referring to Figures 1 and 3, the acquired scan shots 100 may be stored in at least one cluster 200. At this time, the scanning process displayed in real time through the display unit mainly displays the currently acquired scan shot 100, the cluster 200 including the scan shot 100, and the previously acquired scan shot 100. (100) and/or clusters (200) including them may be additionally displayed. As shown in FIG. 3, the currently acquired scan shot 100 and the third cluster 203 including it are mainly displayed in real time on the user interface screen 500, and the preformed first cluster 201 and The second cluster 202 is incidentally displayed on one side of the user interface screen 500. At this time, the primarily displayed cluster 200 may be referred to as an active cluster, and the incidentally displayed cluster 200 may be referred to as an inactive cluster. The active cluster includes the previous scan shot, and the previous scan shot, which is the scan shot 100 obtained during the scanning process, serves as a target for performing scan shot alignment. The inactive cluster serves as a target for which the scan shot 100 obtained during the scanning process performs cluster alignment.

Figure 4 is for explaining feature points used as a medium for aligning scan shots and clusters.

Referring to FIGS. 1, 3, and 4, the data processing method according to the present invention includes aligning the scan shot 100 to at least one preformed inactive cluster (S121). The step of aligning the scan shot 100 to a preformed inactive cluster is called cluster alignment.

When performing cluster alignment, a medium called a feature point can be used to align the inactive cluster and the scan shot 100. The scan shot 100 and the cluster 200 each have at least one feature point, and the feature point may have color information, curvature information, shape information, etc. that appear at a specific location of each data. Feature points can be obtained by sampling each data (clusters and scan shots formed in the form of a point cloud).

By way of example, the cluster 200 may have a plurality of cluster characteristic points (CP), and the scan shot 100 may have a plurality of scan shot characteristic points (SP). When aligning an inactive cluster and the scan shot 100, alignment can be performed by comparing the information contained in the cluster characteristic point (CP) and the information contained in the scan shot characteristic point (SP). In this way, the method of performing alignment using feature points may be included in the category of initial align. Additionally, the initial alignment may be performed using a method that aligns the center point of the scan shot 100 to a predetermined point of the cluster 200, somewhat differently from the method described above.

Meanwhile, the cluster alignment method may be somewhat different from the scan shot alignment method. As an example, the cluster alignment performs the above-described initial alignment, performs alignment using the characteristic points (SP, CP) of each of the scan shot 100 and the cluster 200, and then uses the ICP method. This allows precise alignment to be performed. In contrast, scan shot alignment can perform alignment between scan shots 100 using only the ICP method.

The cluster 200 formed on the left side of FIGS. 4(a), 4(b), and 4(c) is an inactive cluster, and the cluster 200 of FIGS. 4(a), 4(b), and 4(c) is an inactive cluster. The scan shot 100 formed on the right is obtained according to the scanning process. As the scan shot 100 is acquired, scan shot feature points (SP) can be acquired together by sampling the scan shot 100, and the scan shot feature points (SP) are compared with the cluster feature points (CP) to align the cluster. This is done. During the process of performing cluster alignment, the scan shot 100 that attempted to align clusters with preformed inactive clusters obtained during the scanning process does not perform cluster alignment again thereafter.

In the conventional technology, in order to attempt alignment between clusters 200, the overlapping portion (overlap area) between clusters 200 had to be continuously detected. More specifically, in the prior art, in order to compare one cluster with another cluster, the characteristics of one cluster as a whole and the other clusters as a whole were compared. In this process, as the number of scan shots accumulated in each cluster increases, the size of the cluster also increases, and a problem arises in which the computational load increases excessively when performing the inter-cluster comparison process. In contrast, in the present invention, after performing alignment between the scan shot 100 and the cluster 200, the scan shot 100 included in one cluster 200 is aligned with the other cluster 200. do not perform Accordingly, the time required to finally obtain one 3D model can be shortened, and the load required for calculation can also be reduced.

As shown in FIG. 4, the cluster 200 is shown as having 12 cluster feature points (CP), and the scan shot 100 is shown as having 4 scan shot feature points (SP), but the number is necessarily limited to this number. That is not the case. Cluster feature points (CP) and scan shot feature points (SP) can be set to an appropriate number to implement fast alignment speed and accurate alignment between data.

According to the foregoing, the data processing method according to the present invention may include two types of aligning steps (S120). One is a step of aligning the scan shot 100 acquired during the scanning process to a pre-formed inactive cluster (S121), and the other is a step of aligning the scan shot 100 acquired during the scanning process with the immediately preceding scan shot included in the active cluster. It may be an alignment step (S122). That is, step S121 may be a step of performing a cluster alignment process, and step S122 may be a step of performing a scan shot alignment process. The step of aligning to the inactive cluster (S121) and the step of aligning to the previous scan shot (S122) may be performed simultaneously or sequentially. For example, the acquired scan shot 100 may attempt to align clusters with inactive clusters while attempting scan shot alignment with the previous scan shot, as shown in FIG. 1 . Alternatively, the obtained scan shot 100 may first attempt scan shot alignment with the previous scan shot, and then attempt cluster alignment with inactive clusters. Conversely, it is also possible to first attempt to align the acquired scan shot 100 with inactive clusters, and then attempt to align the scan shot with the previous scan shot thereafter.

Below, the selective merging step (S130) of the data processing method according to the present invention will be described in detail.

Figure 5 is for explaining the detailed process of S130, and Figures 6 to 10 are for explaining the process of processing data depending on the success or failure of scan shot alignment and cluster alignment in the data processing method according to the present invention. will be.

Referring to FIG. 1, the data processing method according to the present invention includes the step of selectively merging the active cluster and the inactive cluster including the scan shot 100, depending on whether the aligning step (S121) to the inactive cluster is aligned. Includes (S130). Accordingly, the active cluster and the inactive cluster can form one merged cluster, and when the process of merging the clusters 200 is repeated, one complete three-dimensional model can be finally obtained. Selectively merging active clusters and inactive clusters may mean merging active clusters and inactive clusters under specific conditions depending on the success or failure of cluster alignment and scan shot alignment.

The selective merging step (S130) will be described in more detail. Referring to Figures 1 and 5, the selective merging step (S130) may include at least two determination steps. The selective merging step (S130) can determine whether cluster alignment and scan shot alignment were successful, respectively. As an example, the selective merging step (S130) may determine whether scan shot alignment was successful, and then determine whether cluster alignment was successful. However, the order is not necessarily limited, and the selective merging step (S130) may determine whether the scan shot alignment was successful after determining whether the cluster alignment was successful. Additionally, the selective merging step (S130) may determine the success or failure of scan shot alignment and cluster alignment in parallel.

Referring to FIG. 6, the determination step will be described in more detail through an example. As an example, eight scan shots 100 are first acquired, and each scan shot 100 is included in three clusters 200. At this time, the first cluster 201 includes the first scan shot 101, the second scan shot 102, and the third scan shot 103, and the second cluster 202 includes the fourth scan shot ( 104), the fifth scan shot 105, and the sixth scan shot 106, and the third cluster 203 may include the seventh scan shot 107 and the eighth scan shot 108. . The user may scan the object to obtain the eighth scan shot 108 followed by the ninth scan shot 109. At this time, the previous scan shot is the 8th scan shot 108, the third cluster 203 including the previous scan shot is the active cluster, and the first cluster 201 and the second cluster not including the previous scan shot ( 202) is an inactive cluster.

The 9th scan shot 109 attempts scan shot alignment with the 8th scan shot 108, which is the previous scan shot, and attempts cluster alignment with the first cluster 201 and the second cluster 202, which are inactive clusters. You can. At this time, the process of performing scan shot alignment and cluster alignment is the same as described above.

Referring to FIGS. 5 and 7, in the selective merging step (S130), the acquired scan shot (9th scan shot) is aligned with the previous scan shot (8th scan shot), and the scan shot (9th scan) When a shot) is aligned to an inactive cluster (either the first cluster or the second cluster), the scan shot and the cluster-aligned inactive cluster may be merged into the active cluster. As an example, it is assumed that the ninth scan shot 109 succeeds in cluster alignment to the first cluster 201. At this time, the first cluster 201, which is an inactive cluster, may be merged with the third cluster 203, which is an active cluster, to form a merged cluster 210. Additionally, the merge cluster 210 may include all scan shots 100 included in the first cluster 201 and the third cluster 203. That is, the scan shots 101, 102, and 103 included in the first cluster 201, which is an inactive cluster, may be included in the third cluster 203, which is an active cluster. The merge cluster 210 includes the first scan shot 101, the second scan shot 102, the third scan shot 103, the seventh scan shot 107, the eighth scan shot 108, and the ninth scan. It may include a shot 109, and the merge cluster 210 may function as an active cluster.

In addition, referring to FIGS. 5 and 8, the ninth scan shot 109 is scan shot aligned with the eighth scan shot 108, which is the previous scan shot, but the first cluster 201 and the second cluster ( If the cluster is not aligned in any of 202), the ninth scan shot 109 may be included in the third cluster 203, which is an active cluster. Thereafter, when the scanning process proceeds and a subsequent 10th scan shot (not shown) is acquired, the 9th scan shot 109 functions as the previous scan shot, and a third cluster including the 9th scan shot 109 ( 203) can function as an active cluster.

In addition, referring to FIGS. 5 and 9, the ninth scan shot 109 is not aligned with the eighth scan shot 108, which is the previous scan shot, and the ninth scan shot 109 is a first scan shot that is an inactive cluster. If the cluster is not aligned in any of the cluster 201 and the second cluster 202, it is determined that the ninth scan shot 109 is not aligned in any of the preformed inactive and active clusters 201, 202, and 203. It can be. In this case, the ninth scan shot 109 may be included in a newly created new active cluster that is not included in the preformed inactive and active clusters 201, 202, and 203. As an example, a fourth cluster 204, which is a new active cluster, is created, and the ninth scan shot 109 is included in the fourth cluster 204. Afterwards, when the scanning process proceeds and the subsequent 10th scan shot is acquired, the 9th scan shot 109 functions as the previous scan shot, and the 4th cluster ( 204) can function as an active cluster. At this time, the third cluster 203, which was an active cluster, may be converted to an inactive cluster.

In addition, referring to FIGS. 5 and 10, the ninth scan shot 109 is not aligned with the eighth scan shot 108, which is the previous scan shot, and the ninth scan shot 109 is a first scan shot that is an inactive cluster. When cluster aligned to one of the cluster 201 and the second cluster 202, the ninth scan shot 109 may be included in the inactive cluster to which the cluster is aligned. For example, if the ninth scan shot 109 is cluster aligned with the first cluster 201, which is an inactive cluster, the ninth scan shot 109 is included in the first cluster 201. In addition, when the subsequent scanning process proceeds and the subsequent 10th scan shot is acquired, the 9th scan shot 109 functions as the previous scan shot, so the first cluster 201 including the 9th scan shot 109 can function as an active cluster. That is, when the ninth scan shot 109 is included in the first cluster 201, which is an inactive cluster, the first cluster 201, which is an inactive cluster, is activated, and the third cluster 203, which is an active cluster, is activated to be inactive. Clusters and inactive clusters can be switched. The user can easily check the cluster 200 that includes the currently acquired scan shot 100, and thus there is an advantage in that the user can determine the location of the object being scanned. Additionally, there is the advantage of being able to freely scan regardless of alignment success or failure. In addition, even if a scan shot alignment failure occurs, if cluster alignment is successful, unnecessary new clusters are not created and inactive clusters are activated and used, which has the advantage of improving computation speed by minimizing the number of clusters considered in the alignment process. There is.

Figure 11 is for explaining the undo process using the third cluster as an example.

Referring to FIG. 11, the user may cancel acquisition of the scan shot 100. Exemplarily, referring to FIG. 11(a), the third cluster 203, which is an active cluster, includes the 7th scan shot 107, the 8th scan shot 108, and the 9th scan shot 109. It is assumed that The user can select the undo button (not shown) formed on the user interface screen to delete scan shots in reverse chronological order, starting from the most recently acquired scan shot 100. In FIG. 11(b), when the user selects the undo button, the most recently acquired ninth scan shot 109 may be deleted. Referring to FIG. 11(c), if the user selects the undo button again, the eighth scan shot 108 acquired before the deleted ninth scan shot 109 may be deleted. Referring to FIG. 11(d), if the user selects the undo button again, the seventh scan shot 107 acquired before the deleted eighth scan shot 108 may be deleted. At this time, since the third cluster 203 does not include any scan shots 100, the third cluster 203 may also be deleted. When the third cluster 203 is deleted, the cluster that functioned as the active cluster before the third cluster 203 functioned as the active cluster may be activated again and displayed on the user interface screen. Afterwards, if an additional undo process is performed, the most recently acquired scan shot 100 in the activated cluster can be deleted in reverse order of acquisition time.

Meanwhile, the process of deleting the acquired scan shot 100 may be performed in units of one shot as described above, but if necessary, a plurality of scan shots 100 may be deleted together.

Below, the process by which the user interface screen changes when the scan shot 100 is aligned to an inactive cluster and the inactive cluster and the active cluster are switched will be described in more detail.

Figure 12 is for explaining the process of switching the screen by aligning the scan shot 100 to the inactive cluster in the data processing method according to the present invention, and Figures 13 and 14 are the screen switching process between the active cluster and the inactive cluster. It is intended to explain.

Referring to FIGS. 12 and 13 , the object O is scanned to obtain a plurality of scan shots 100 representing the object O. Illustratively, the user scans the object O and uses the first scan shot 101, the second scan shot 102, the third scan shot 103, the fourth scan shot 104, and the fifth scan shot ( 105), the sixth scan shot 106, and the seventh scan shot 107 can be obtained. Based on what is shown in FIG. 12, the first scan shot 101, the second scan shot 102, and the third scan shot 103 are included in the first cluster 201, and the fourth scan shot 104 ), the fifth scan shot 105, and the sixth scan shot 106 may be included in the second cluster 202. Before the user acquires the seventh scan shot 107 through a scanning process, the second cluster 202 including the sixth scan shot 106, which is the previous scan shot, may function as an active cluster.

At this time, the scan shot 100, active cluster, and inactive cluster may be displayed on the user interface screen 500 having a plurality of areas. By way of example, the obtained scan shot 100 and the active cluster may be mainly displayed on the user interface screen 500, and the inactive cluster may be displayed incidentally on the user interface screen 500. More specifically, the user interface screen 500 may include a main area 510 that occupies a predetermined area and a sub-area 520 that occupies a smaller area than the main area 510. At this time, the main area 510 may be formed to include the center of the user interface screen 500 so that it is visually easily displayed to the user, and the sub area 520 may be formed on one side of the user interface screen 500. You can. In the user interface screen 500, in order to display the scanning process in real time along with the scan shot 100 that the user is currently acquiring, the acquired scan shot 100 and the second cluster 202, which is the active cluster, are displayed in the main area ( 510). The first cluster 201, which is an inactive cluster, may be displayed in the sub-area 520, and the user can determine the number of clusters 200 created through the main area 510 and the sub-area 520 and the object currently being scanned. Scan information such as the location of (O) can be obtained.

Meanwhile, referring to FIGS. 12 to 14 together, the seventh scan shot 107 is acquired, scan shot alignment is performed with the sixth scan shot 106, which is the immediately preceding scan shot, and the first cluster 201, which is an inactive cluster, is and cluster alignment can be performed. At this time, if scan shot alignment fails and cluster alignment succeeds, the seventh scan shot 107 may be included in the first cluster 201, which is an inactive cluster. Additionally, the first cluster 201 may be converted to an active cluster (activation of an inactive cluster), and the second cluster 202 may be converted to an inactive cluster (deactivation of an active cluster). Referring to FIG. 14, the first cluster 201, which was an inactive cluster and included a scan shot (more specifically, the seventh scan shot), is activated and displayed in the main area 510 of the user interface screen 500. . At the same time, the second cluster 202, which was an active cluster and did not include a scan shot, may be deactivated and displayed in the form of a thumbnail in the sub-area 520 of the user interface screen 500. In this way, by switching the displayed area according to the transition between the active cluster and the inactive cluster, the user can easily check the location of the object O currently being scanned, which has the advantage of increasing user convenience.

Meanwhile, in the above-described content, scan shot alignment and cluster alignment can be performed limited to a specific area. For example, when scan shot alignment and cluster alignment are performed, the area where alignment is performed may be limited to only the area representing teeth among the scan shot and cluster representing the object, and may include gums, tongue, and soft tissue. tissue, noise, etc. can be excluded from alignment. The process of distinguishing between the area representing teeth and the remaining areas can be performed using artificial intelligence (AI) through image learning, and the process of excluding noise, etc. can be performed using known noise filtering techniques. The data processing method according to the present invention has the advantage of being able to quickly and accurately perform scan shot alignment and/or cluster alignment by limiting only teeth to the alignment target. Additionally, since the merging speed between clusters can be improved, there is an advantage in shortening the time required to ultimately obtain a high-quality 3D model.

Below, a data processing device according to the present invention using the data processing method described above will be described. When describing the data processing device, content that overlaps with the above-described content will be briefly explained or omitted.

Figure 15 is a configuration diagram of a data processing device 900 according to the present invention.

Referring to FIG. 15, the data processing device 900 according to the present invention includes a scanner 910, a control unit 920, and a display unit 930. The scanner 910 can scan an object. The scanner 910 may be equipped with at least one camera (not shown) that receives light therein, and the light reflected from the surface of the object may be introduced into the interior of the scanner 910 and received in the lens of the camera. there is. Based on the light received by the camera, a scan shot representing the object may be created. The scanner 910 may be a table scanner that rotates or moves the object by placing it on a tray, or it may be a handheld scanner that a user holds and points toward the object to perform a scanning process.

The control unit 920 is electrically connected to the scanner 910 and can generate scan shots based on the light received in the scanner 910. In addition, scan shots can be aligned to preformed clusters (more specifically, inactive clusters), and active clusters and inactive clusters can be selectively merged depending on alignment.

The detailed configuration of the control unit 920 will be described in more detail. The control unit 920 includes a scan shot generator 921 that generates a scan shot based on the object scanned by the scanner 910. The scan shot generator 921 may generate a scan shot by receiving light received from the camera of the scanner 910. A scan shot may be two-dimensional image data or a three-dimensional data piece. The scan shot generator 921 is shown as a component of the control unit 920, but it is not necessarily limited thereto, and may be formed as a component of the scanner 910, if necessary.

Additionally, the control unit 920 includes an alignment unit 922. The aligner 922 aligns the scan shot generated by the scan shot generator 921 with the immediately preceding scan shot included in the active cluster (scan shot align), or aligns the scan shot with the inactive cluster (cluster align). In) it can be done. As an example, the process of aligning the scan shot generated and acquired by the scan shot generator 921 with the immediately preceding scan shot included in the active cluster may use the ICP align method, and the scan shot may be aligned with the inactive cluster. The initial alignment method can be used for this process. Scan shot alignment and cluster alignment can be performed by comparing scan shot feature points of a scan shot and cluster feature points of a cluster, and the process of performing alignment using feature points is the same as described above.

Additionally, the aligner 922 may simultaneously or sequentially perform the process of aligning the scan shot with the previous scan shot and the process of aligning the scan shot with the inactive cluster. That is, the aligner 922 can perform steps S121 and S122 described above. At this time, the execution order of steps S121 and S122 may be simultaneous or sequential, and step S121 may be performed first and step S122 may be performed later, or step S122 may be performed first and step S121 may be performed later. do.

Additionally, the control unit 920 includes a cluster management unit 923. The cluster management unit 923 is responsible for creating and managing a cluster. As an example, the cluster manager 923 may merge a plurality of clusters according to the alignment result of the aligner 922. Additionally, the cluster manager 923 may create a new active cluster according to the alignment result of the aligner 922 and manage the new active cluster to include scan shots. Additionally, the cluster management unit 923 can manage to activate an inactive cluster or deactivate an active cluster.

Below, detailed operations of the cluster management unit 923 will be described.

If the scan shot is aligned to the previous scan shot in the aligner 922 and the scan shot is aligned to the inactive cluster, the cluster manager 923 may merge the active cluster and the inactive cluster including the previous scan shot. . At this time, the cluster manager 923 may perform inter-cluster merging in such a way that an inactive cluster is merged into an active cluster, and the merged cluster includes scan shots included in the active cluster and scan shots included in the inactive cluster. Depending on the scan shot alignment and cluster alignment of the scan shot, the number of clusters constituting the 3D model can be reduced during the scanning process, and there is an advantage in obtaining a complete 3D model more quickly.

Additionally, if the scan shot is aligned to the previous scan shot in the aligner 922 and the scan shot is not aligned to the inactive cluster, the cluster management unit 923 includes the scan shot in the active cluster that includes the previous scan shot. You can do it. During subsequent scans, the active cluster containing the scan shot remains active.

Additionally, if the scan shot is not aligned to the previous scan shot in the aligner 922 and the scan shot is not aligned to the inactive cluster, the cluster manager 923 may control the creation of a new group containing the scan shot. You can. More specifically, the cluster manager 923 deactivates the active cluster, creates a new active cluster, and includes the scan shot in the new active cluster. Therefore, in the subsequent scan process, the new active cluster can function as an active cluster.

Additionally, if the scan shot in the aligner 922 is not aligned with the previous scan shot and the scan shot is aligned to an inactive cluster, the cluster manager 923 controls the scan shot to be included in the inactive cluster. At this time, the cluster management unit 923 includes the acquired scan shot in the inactive cluster, activates the inactive cluster including the scan shot (converts the inactive cluster to an active cluster), and deactivates the active cluster including the immediately preceding scan shot. (Convert an active cluster to an inactive cluster). The inactive cluster becomes the target of cluster alignment of the newly acquired scan shot, and the active cluster becomes the target of scan shot alignment, and the detailed description thereof is the same as described above. According to the above-described process, even if a scan shot alignment failure occurs, if cluster alignment is successful, the cluster management unit 923 can activate and use the inactive cluster without creating an unnecessary new cluster, and the cluster that is considered in the alignment process There is an advantage in improving calculation speed by minimizing the number of .

Meanwhile, the display unit 930 may visually display at least some of the scan shots and clusters among the above-described contents to the user. More specifically, the display unit 930 can display at least one of a scan shot, an inactive cluster, and an active cluster, and the user can check the display unit 930 to determine the location of the currently scanned object and the number of clusters created. etc. can be easily checked comprehensively. The display unit 930 may be a known visual display device. For example, the display unit 930 may be a monitor, tablet, touch screen, etc.

Meanwhile, the display unit 930 displays a user interface having a plurality of areas, including a main area occupying a predetermined area and a sub area occupying an area smaller than the main area. The display unit 930 can display active clusters and scan shots in the main area of the user interface, and display inactive clusters in the sub-area of the user interface.

In addition, when a scan shot is included in an inactive cluster under the control of the cluster management unit 923, the display unit 930 activates the inactive cluster to display it in the main area and deactivates the active cluster to display a thumbnail in the sub area. It can be displayed as . Accordingly, the user has the advantage of being able to easily check the location of the object currently being scanned through cluster switching.

The above description is merely an illustrative explanation of the technical idea of the present invention, and various modifications and variations will be possible to those skilled in the art without departing from the essential characteristics of the present invention.

Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for illustrative purposes, and the scope of the technical idea of the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted in accordance with the claims below, and all technical ideas within the equivalent scope should be construed as being included in the scope of rights of the present invention.

S110: Acquiring scan shots
S120: Aligning step
S130: Selective Merge Step
100: Scan Shot 200: Cluster
500: User interface screen 510: Main area
520: Sub area
900: data processing device

Claims (18)

Obtaining a scan shot representing an object;
Aligning the scan shot to the at least one pre-formed inactive cluster through cluster alignment between the scan shot and the at least one pre-formed inactive cluster; and
Depending on whether the step of aligning to the inactive cluster is aligned, the scan shot is aligned through scan shot alignment between the scan shot and the immediately preceding scan shot, and the active cluster including the scan shot and the scan shot are aligned through cluster alignment. selectively merging the aligned inactive clusters; A data processing method comprising: In claim 1,
Aligning the scan shot to the previous scan shot included in the active cluster; It further includes,
A data processing method, wherein the step of aligning to the inactive cluster and the step of aligning to the immediately preceding scan shot are performed simultaneously or sequentially. In claim 1,
The step of aligning to the inactive cluster aligns feature points of the scan shot with feature points of the inactive cluster. In claim 2,
The selective merging step includes:
If the scan shot is aligned to the previous scan shot and the scan shot is aligned to the inactive cluster, the inactive cluster is merged into the active cluster, and the scan shots included in the inactive cluster are added to the active cluster. A data processing method characterized by comprising: In claim 2,
The data processing method, wherein if the scan shot is aligned to the previous scan shot and the scan shot is not aligned to the inactive cluster, the scan shot is included in the active cluster. In claim 2,
If the scan shot is not aligned to the previous scan shot and the scan shot is not aligned to the inactive cluster, the active cluster is deactivated and a new active cluster is created, and the scan shot is aligned to the new active cluster. A data processing method comprising: In claim 2,
When the scan shot is not aligned with the immediately preceding scan shot and the scan shot is aligned with the inactive cluster, the scan shot is included in the inactive cluster. In claim 7,
When the scan shot is included in the inactive cluster, the inactive cluster is activated, and the active cluster that does not include the scan shot is deactivated. In claim 7,
The scan shot, the active cluster, and the inactive cluster are:
Displayed on a user interface screen having a plurality of areas, including a main area occupying a predetermined area and a sub area occupying an area smaller than the main area,
The scan shot and the active cluster are displayed in the main area,
When the scan shot is included in the inactive cluster, the inactive cluster is activated and displayed in the main area, and the active cluster that does not include the scan shot is deactivated and displayed in the sub area. method. A scanner that scans an object;
A scan shot obtained by scanning the object is aligned to the at least one preformed inactive cluster through cluster alignment between the scan shot and the at least one preformed inactive cluster, and the scan shot is aligned depending on whether the alignment occurs. a control unit for selectively merging the active cluster aligned through scan shot alignment between the last scan shot and including the scan shot with the inactive cluster in which the scan shot is aligned through cluster alignment; and
A display unit that displays at least one of the scan shot, the inactive cluster, and the active cluster. In claim 10,
The control unit,
a scan shot generator that generates the scan shot based on the object scanned by the scanner;
an aligner that aligns the scan shot with a previous scan shot included in the active cluster or aligns the scan shot with the inactive cluster; and
A data processing device comprising a cluster manager that creates and manages the inactive cluster and the active cluster according to the alignment result of the aligner. In claim 11,
The alignment unit,
A data processing device characterized in that the process of aligning the scan shot with the immediately preceding scan shot and the process of aligning the scan shot with the inactive cluster are performed simultaneously or sequentially. In claim 11,
The cluster management unit,
If the scan shot is aligned to the previous scan shot and the scan shot is aligned to the inactive cluster, the inactive cluster is merged into the active cluster, and the scan shots included in the inactive cluster are added to the active cluster. A data processing device comprising: In claim 11,
The cluster management unit,
If the scan shot is aligned to the previous scan shot and the scan shot is not aligned to the inactive cluster, the scan shot is included in the active cluster. In claim 11,
The cluster management unit,
If the scan shot is not aligned to the immediately preceding scan shot and the scan shot is not aligned to the inactive cluster, deactivate the active cluster, create a new active cluster, and add the scan shot to the new active cluster. A data processing device comprising: In claim 11,
The cluster management unit,
When the scan shot is not aligned with the previous scan shot and the scan shot is aligned with the inactive cluster, the scan shot is included in the inactive cluster. In claim 16,
The cluster management unit,
Data processing device, characterized in that converting an active cluster in which the scan shot is not aligned to the inactive cluster, and converting an inactive cluster in which the scan shot is aligned to the active cluster. In claim 16,
The display unit,
Displaying a user interface having a plurality of areas, including a main area occupying a predetermined area and a sub area occupying an area smaller than the main area,
The scan shot and the active cluster are displayed in the main area,
When the scan shot is included in the inactive cluster, the inactive cluster is activated and displayed in the main area, and the active cluster is deactivated and displayed in the sub area.

Images