JP7429430B2 - 3D assembly model search system, method and program - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Published on the website of the proceedings of the 11th Forum on Data Engineering and Information Management (DEIM2019) on February 25, 2019. Applicable address https://db-event. jpn. org/deim2019/post/program. html March 6, 2019, The 11th Data Engineering and Information Management Conference, sponsored by the Institute of Electronics, Information and Communication Engineers, Data Engineering Research Specialist Committee, Database Society of Japan, Information Processing Society of Japan, Database Systems Research Group Announced at the forum (DEIM2019).

The present invention relates to a search system for three-dimensional assembly models with different component arrangements.

In recent years, due to significant improvements in GPU performance and the spread of low-priced 3D printers and materials, manufacturing using 3D models has become widespread in the manufacturing industry. In fields such as automobiles, mobile phones, and buildings, 3D models using digital modeling software such as CAD are used for design and strength analysis, and 3D models are also used not only in the industrial field but also in various fields such as education and medicine. are being used, and the amount of data for 3D models is increasing year by year. The 3D model data incorporates the know-how of engineers, and by referring to the existing 3D model data, in addition to the model data itself, information on model strength analysis, production processes, and tools for machining can be obtained. All information related to engineering, including cost information, can be used. Therefore, by reusing existing 3D models, 3D model designers can efficiently design new products. Therefore, much research has been conducted on searching for three-dimensional models, and many methods have been studied for searching for shape-similar models, such as those used to identify airplanes and cars (for example, in the non-patent literature 1).

However, in industrial fields such as mechanical design, it has become common to design using a three-dimensional CAD assembly model (hereinafter referred to as an assembly model) in which one model is constructed by combining parts. As an example, Figure 1 shows the shape and internal structure of an assembly model called Gear. The Gear shown in Figure 1 consists of six types of parts: Cover, Bolt, Cap, Planet, Output Shaft, and Spacer Case. In addition, it is assumed that where parts have different colors, such as Bolt and Planet, different labels are given to distinguish parts that have the same shape but different materials, for example. If a method could be established that takes into account subtle differences in internal structure and component information and searches for models that have the same shape and internal structure, assembly models could be handled more efficiently. can. In addition, if the labels given to each type of part are unified in advance in the target database, the corresponding part can be determined based on the given label value, but for example, when creating a model, There may be cases where the assigned label values are unreliable, such as when the labels are not assigned using the same criteria due to different people. Even if the labels are unified in advance, the more models in the database, the less realistic the label unification becomes. If the labels are not unified, it is necessary to consider the search method on the premise that the correspondence of the parts to be compared is unknown. Therefore, it is necessary to take into consideration the fact that the correspondence relationships between these parts are unknown, and to consider the subtle differences in internal structure for identification, and to come up with a method that can search for models that have the same shape and internal structure. be.

Regarding this point, for example, Non-Patent Document 2 proposes a shape model search method called LightField Discriptor (LFD) based on multi-view rendering. In LFD, rendering is performed from the vertices of a regular dodecahedron centered on the model, and the frequency spectrum and Zernike moment of the obtained silhouette image are searched as features. This allows for searching even if the 3D model is rotated. is possible. Parallel movement and size are handled by posture normalization performed before rendering. Furthermore, Non-Patent Document 3 proposes a method for identifying a three-dimensional shape by directly extracting and utilizing SIFT and SURF features from a three-dimensional voxel grid. The method in Non-Patent Document 4 shell-samples a voxelized three-dimensional model (concentric spherical sampling), performs spherical harmonic transformation on each spherical surface, and calculates the norm for each order, thereby obtaining feature quantities that are invariant to the rotation of the model. It consists of However, this method requires translation normalization in advance because the centers of the models must be aligned. In addition, there is a problem in that continuity between each spherical surface due to shell sampling is not taken into account.

The method proposed by Non-Patent Document 5 uses two-dimensional features obtained by performing discrete Fourier transform on depth images from six axes, and the three-dimensional feature using shell sampling and spherical harmonic transformation proposed by Non-Patent Document 4. A method is used in which this is combined with the dimensional feature and used as the feature of the entire model. By combining two-dimensional feature amounts based on depth images, the problem that the continuity between each spherical surface due to shell sampling, which was in the method of Non-Patent Document 4, is not taken into account is solved. However, rotation and translation of the model are left to prior normalization. These methods using spherical harmonics are methods for searching for models with similar shapes for 3D models, and research on searching for identical models that takes shape and internal structure into consideration for assembly models has not yet been conducted. Not yet. In addition, as a model search method targeting assembly models, Non-Patent Document 6 proposes a model search method in which the internal structure of the model is identified by converting the component parts of the model into a graph with vertices and performing a graph search. There is. Furthermore, Non-Patent Document 7 proposes a model search in which the assembly model is divided into parts and then the LFD of Non-Patent Document 2 is used as a feature quantity. However, although these methods are search methods that take into account the internal structure of the assembly model, they do not take into account the internal structure such as the arrangement and types of constituent parts.

In addition, research on reducing the calculation cost of multi-view rendered feature sets includes a method described in Non-Patent Document 8. This method uses machine learning to select a small number of highly discriminative viewpoints, reducing the effort involved in calculating similarity. In another method, Furuya et al. [10] reduced local features from multiple depth images extracted from a multi-view image group to one SIFT feature per 3D model using the Bag-Of-Feature (BOF) method. Integration reduces the computational cost of similarity calculation. However, these studies are methods for searching for models with similar shapes for three-dimensional models, and do not assume the same model search for assembly models.

Regarding this point, in a previous study, the present inventor proposed a search method using transmission projection images rendered from multiple viewpoints as a same model search method that takes into account the shape and internal structure of the assembly model (Non-Patent Document 9). . This search method is a method of calculating feature amounts of each of a plurality of projected images and using a set of them as feature amounts of a three-dimensional model. In this method, features are constructed using the following steps. As preprocessing, each type of part is labeled based on the reciprocal of its volume, and a large number of transmission projection images are taken from around the assembly model. Then, a projection image containing information about the internal structure of the assembly model and the arrangement of parts corresponding to the projection viewpoint is obtained. By performing two-dimensional Radon transform and radial discrete Fourier transform on the projected image, feature quantities that are invariant with respect to rotation and translation within the plane of the projected image can be obtained.

A method that combines multi-view rendering and rotation-invariant global features of an image (within a plane) allows rotation-invariant searches of the original assembly model if a sufficient number of viewpoints are taken. However, since one model is described by a set of features equal to the number of viewpoints, comparing a pair of models involves comparing feature sets and feature sets, which has the problem of high computational cost. In a three-dimensional model search, as the number of database models increases, the number of times the similarity between the models is calculated increases accordingly, so the computational cost of calculating the similarity greatly affects the overall processing time of the three-dimensional model search.

J.H. Tangelder and R.C. Veltkamp. A survey of content 3d shape retrieval methods. Multimedia Tools and Applications,Vol. 39, No. 3, pp. 441-471, 2008.

Chen Ding-Yun and Tian Xiao-Pei and Shen Yu-Te and Ouhyoung Ming, On Visual Similarity Based 3D Model Retrieval,Computer Graphics Forum, 2003

Knopp Jan and Prasad Mukta and Willems Geert and Timofte Radu and Van Gool Luc, Hough Transform and 3D SURF for Robust Three Dimensional Classification, Proceedings of the 11th European Conference on Computer Vision: Part VI, 2010

Michael Kazhdan and Thomas Funkhouser and Szymon Rusinkiewicz, Rotation Invariant Spherical Harmonic Representation of 3D Shape Descriptors, Symposium on Geometry Processing, 2003

Papadakis, Panagiotis and Pratikakis, Ioannis and Theoharis, Theoharis and Passalis, Georgios and Perantonis, Stavros, 3D Object Retrieval using an Efficient and Compact Hybrid Shape Descriptor, The Eurographics Association,2008

A.S. Deshmukh, A.G. Banerjee, S.K. Gupta, and R.D. Sriram.Content-based assembly search: A step towards assembly reuse. Computer-Aided Design, Vol. 40, No. 2, pp.244-261, 2008.

Hu, Kai-Mo and Wang, Bin and Yong, Jun-Hai and Paul, Jean-Claude, Relaxed lightweight assembly retrieval using vector space model., Computer-Aided Design, 2013

A Bayesian 3-D Search Engine Using Adaptive Views Clustering,Filali Ansary, Tarik and Daoudi, Mohamed and Vandeborre,Jean-Philippe, IEEE Transactions on Multimedia,Institute of Electrical and Electronics Engineers, 9, 1, pp.78-88, 2007 , https://hal.archives-ouvertes.fr/hal-00666134

Kaoru Katayama and Takumi Sato. A Matching Method for 3D CAD Models with Different Assembly Structures Using Projections of Weighted Components. Journal of Information Processing, Vol. 25, pp. 376-385, 2017.

The present invention has been made in view of the above, and an object of the present invention is to reduce the calculation cost in searching for a three-dimensional assembly model.

The present invention proposes a method for searching for models with the same shape and internal structure by using multi-view rendered projection images and spherical harmonic transformation for assembly models in which the correspondence of parts is unknown. At this time, a more efficient same model search method is realized by integrating the feature set from the multi-view rendered projected image into a single feature using spherical harmonic transformation. The present invention proposes two methods for multi-view rendered projection images: a method that uses a one-dimensional projection of an assembly model, and a method that uses a two-dimensional projection of an assembly model.

According to the present invention, compared to previous studies, the processing time for similarity calculation is greatly reduced, and the processing time for the entire same model search is also reduced. Additionally, compared to previous studies, it has become possible to search for the same model using smaller feature quantities.

FIG. 2 is a perspective view showing the shape and internal structure of a model of a search target. FIG. 2 is a schematic diagram showing the procedure of the present invention. FIG. 2 is a schematic diagram showing an overview of a feature amount analysis method. FIG. 3 is a schematic diagram showing an overview of feature quantity comparison during search. FIG. 2 is a schematic diagram showing an overview of a method for calculating distance during a search. FIG. 2 is a schematic diagram illustrating an overview of a method for obtaining feature amounts for each part of a search target object. FIG. 1 is a functional block diagram of a three-dimensional assembly model search system according to the present embodiment. It is a flowchart of the process performed by the three-dimensional assembly model search system of this embodiment. It is a flowchart of feature value generation processing A of this embodiment. FIG. 2 is a conceptual diagram for explaining feature amount generation processing A of the present embodiment. FIG. 2 is a conceptual diagram for explaining feature amount generation processing A of the present embodiment. It is a flowchart of feature value generation processing B of this embodiment. FIG. 7 is a conceptual diagram for explaining feature amount generation processing B of the present embodiment. FIG. 7 is a conceptual diagram for explaining feature amount generation processing B of the present embodiment. FIG. 7 is a conceptual diagram for explaining feature amount generation processing B of the present embodiment. FIG. 3 is a perspective view showing an assembly model used in the experiment. FIG. 2 is a perspective view showing details of the assembly model used in the experiment. It is a chart showing similarity at the time of search. It is a chart showing processing time. It is a chart showing the characteristic amount of parts. It is a chart showing the correct answer rate as an experimental result.

The present invention will be described below with reference to embodiments, but the present invention is not limited to the embodiments described below. In the following, a three-dimensional assembly model that is made up of a combination of multiple three-dimensional subassemblies is referred to as an "assembly model." A subassembly is a collection of mechanical elements that have the same shape and characteristics such as materials. say. Furthermore, hereinafter, a three-dimensional subassembly model may be referred to as a "subassembly." By the way, parts are the smallest mechanical elements that cannot be further divided. Each part is labeled according to its characteristics, and a collection of parts given the same label is a subassembly.

This embodiment proposes a method for searching for models with the same shape and internal structure using multi-view rendered projection images and spherical harmonic transformation, targeting assembly models in which the correspondence relationships between parts are unknown. At this time, a more efficient same model search method is realized by integrating the feature set from the multi-view rendered projected image into a single feature using spherical harmonic transformation. For multi-view rendered projection images, there are two methods: a method that uses one-dimensional projection of the assembly model (hereinafter referred to as the first proposed method (1D projection)) and a method that uses two-dimensional projection of the assembly model (hereinafter referred to as the second proposed method). We propose two methods (referred to as 2D projection).

In the first proposed method (1D projection), the same model search is performed by constructing features of an assembly model that are invariant to rotation and translation using three-dimensional Radon transform, discrete Fourier transform, and spherical harmonic transform. Calculate the feature values for each part with the same label that makes up the assembly model using the following steps. First, a one-dimensional projection from the surroundings is created by three-dimensional Radon transformation. Projection is performed for each subassembly, which is a collection of parts with the same label, and is created by compositing the subassembly and other parts. This is to indicate the location of the subassembly in the overall model. Note that the projections of the subassembly and others are normalized by their respective volumes. By performing discrete Fourier transform in the radial direction for each three-dimensional Radon-transformed projection and obtaining an amplitude spectrum, it becomes a feature (portion corresponding to a low frequency component) that is invariant to the translation of the original model. Then, a set of amplitude spectra corresponding to each projection angle is regarded as a two-dimensional matrix of angle×frequency. By dividing this matrix into frequencies, treating each as a spherical field, performing spherical harmonic transformation, and finding the norm for each order, we obtain features that are invariant to the rotation of the original model. Finally, by arranging the norms of each spherical field in order, the features of the final subassembly can be obtained. Further, the distance between assembly models is set as a value when the sum of Euclidean distances of feature quantities between subassemblies is the minimum, and the degree of similarity between assembly models is set as its reciprocal.

The second proposed method (2D projection) uses two-dimensional projection calculation, two-dimensional Radon transform, discrete Fourier transform, and spherical harmonic transform to construct features of an assembly model that are invariant to rotation and translation. Perform model search. Calculate the feature values for each part with the same label that makes up the assembly model using the following steps. First, a two-dimensional projection from the surroundings is created based on spherical coordinates. Similar to the first proposed method (1D projection), projection is performed in subassembly units, and is created by combining the subassembly and other parts. The projections of the subassembly and others are then normalized by their respective volumes. After creating a projection for each subassembly, a two-dimensional Radon transform is performed on each projection. By performing discrete Fourier transform in the radial direction on the two-dimensional array obtained by the two-dimensional Radon transform to obtain an amplitude spectrum, features that are invariant with respect to translation of the original model are obtained. Subsequently, a discrete Fourier transform is performed in the angular direction to obtain an amplitude spectrum, thereby obtaining features that are invariant with respect to rotation within the plane of the two-dimensional projection. As a result, we have obtained a two-dimensional array feature that is invariant with respect to the rotation and translation of the original model for each projection angle.If we arrange the two-dimensional array features in a row to make a one-dimensional array, we can create a one-dimensional array for each projection angle × It becomes a two-dimensional matrix of frequencies. After that, the final feature amount of the subassembly is obtained using the same procedure as after the spherical harmonic transformation of the first proposed method (1D projection). Furthermore, the method of calculating the distance and similarity between assembly models is also the same as the first proposed method (1D projection).

In this embodiment, we propose an identical model search method using spherical harmonic transformation that takes into account the shape and internal structure of a three-dimensional assembly model in which the correspondence between parts is unknown. In this embodiment, two methods are proposed. In other words, when extracting feature quantities, there are two methods: one that uses a one-dimensional projection of a three-dimensional model, and one that uses a two-dimensional projection of a three-dimensional model. This is expressed as the proposed method (2D projection). Note that in this embodiment, the assembly model used for search is a collection of files for each part. A file indicating the correspondence between subassemblies and component files specifies which subassembly each component belongs to.

<First proposed method (1D projection)>
In the first proposed method (1D projection), a feature quantity is constructed from a one-dimensional projection of a three-dimensional model, and an identical model search is performed based on the feature quantity. A three-dimensional Radon transform is used as a method to obtain a one-dimensional projection from a three-dimensional model. After that, features of the assembly model that are invariant to rotation and translation are constructed using discrete Fourier transform and spherical harmonic transform.

(Extraction of features)
FIG. 2 shows the procedure for obtaining features from an assembly model using the first proposed method (1D projection). Further, Table 1 shows the feature extraction procedure (Algorithm 1), and Table 2 shows the three-dimensional Radon transformation procedure (Algorithm 2).

In this embodiment, an assembly model is decomposed into subassemblies with different labels, and feature amounts are calculated for each subassembly. First, a one-dimensional projection from the surroundings is created by three-dimensional Radon transformation. This projection is the sum of the normalized projection of the subassembly itself divided by its volume, and the normalized projection of the entire model excluding that subassembly divided by its volume. Normalizing by volume and calculating the sum in this way is important to obtain high search accuracy. Then, by performing discrete Fourier transform on each one-dimensional projection in the radial direction, an amplitude spectrum that is invariant to translation of the original model corresponding to each projection viewpoint can be obtained. FIG. 3 shows a flow of integrating the amplitude spectra for each projection angle obtained in this way into one feature quantity. First, a set of amplitude spectra corresponding to each projection angle is regarded as a two-dimensional matrix of angle×frequency. Next, this matrix is divided into frequencies, each of which is treated as a spherical field, and spherical harmonic transformation is performed. Then, by finding the norm for each order, we obtain features that are invariant to the rotation of the original model. Finally, by arranging the norms of each spherical field in order, the features of the final subassembly can be obtained.

(similarity calculation)
The procedure for calculating the distance between two assembly models from the feature amounts is shown in FIGS. 4 and 5. Further, Table 3 shows the procedure (Algorithm 3) for calculating the distance between subassemblies of two assembly models.

The feature amount of the assembly model is a three-dimensional array, and is represented by a collection of two-dimensional arrays that are feature amounts for each subassembly. The distance between subassemblies is expressed by the Euclidean distance of each feature. To find the distance between two assembly models, first, as shown in FIG. 4, all Euclidean distances between each subassembly are examined. Then, from the obtained matrix of distances between subassemblies, a combination that minimizes the total Euclidean distance is determined. In this embodiment, Munkres' allocation algorithm (Hungarian method) can be used to find the optimal combination. Finally, as shown in FIG. 5, the total value of the combination is taken as the distance between the two assembly models. Furthermore, the degree of similarity is the reciprocal of the distance.

<Second proposed method (2D projection)>
In the second proposed method (2D projection), a feature quantity is constructed from a two-dimensional projection of a three-dimensional model, and an identical model search is performed based on the feature quantity. The overall feature extraction procedure is similar to the first proposed method (1D projection), but the process for obtaining features that are invariant with respect to changes in the model's posture from the projection is different. In the second proposed method (2D projection), features of an assembly model that are invariant to rotation and translation are constructed using two-dimensional projection calculation, two-dimensional Radon transform, discrete Fourier transform, and spherical harmonic transform.

(Extraction of features)
FIG. 6 shows the procedure for obtaining features from an assembly model using the second proposed method (2D projection). Further, Table 4 shows the procedure for extracting feature quantities (Algorithm 4), and Table 5 shows the procedure for creating a weighted two-dimensional projection (Algorithm 5).

From FIG. 6, the feature amount is calculated for each subassembly that constitutes the assembly model using the following procedure. First, a weighted two-dimensional projection is created from the surroundings of the model. This projection is also the sum of the normalized projection of the subassembly itself divided by its volume, and the normalized projection of the entire model excluding that subassembly divided by its volume. Next, by performing two-dimensional Radon transformation on each two-dimensional projection, it is converted into a two-dimensional array (sinogram) in the radial direction x angular direction. When the two-dimensional projection is translated or rotated, the values of the elements in the two-dimensional array are translated in the radial direction and the angular direction. Therefore, by performing discrete Fourier transform in the radial direction on the obtained two-dimensional array to obtain an amplitude spectrum, it is possible to obtain features that are invariant to the translation of the original model. Furthermore, by subsequently performing discrete Fourier transform in the angular direction to obtain an amplitude spectrum, it is possible to obtain features that are invariant with respect to rotation within the plane of the two-dimensional projection. Through the steps up to this point, a two-dimensional array feature that is invariant with respect to the rotation and translation of the original model has been obtained for each projection angle. Here, if the two-dimensional array feature values for each projection angle are arranged in one column to form a one-dimensional array, it becomes a two-dimensional matrix of each projection angle x frequency. This is the same as the form immediately before the processing using spherical harmonic transformation in the first proposed method (1D projection), and feature amounts can be integrated by spherical harmonic transformation similarly to the first proposed method (1D projection). Therefore, from now on, the final feature amount of the part is obtained using the same procedure as in FIG. 3 of the first proposed method (1D projection).

(Similarity calculation)
The method for calculating the distance and similarity between assembly models in the second proposed method (2D projection) is the same as that in the first proposed method (1D projection), so a description thereof will be omitted.

The proposed method of the present invention has been described above.Next, a three-dimensional assembly model retrieval system implementing the above-mentioned proposed method will be described.

FIG. 7 shows a functional block diagram of a three-dimensional assembly model search system 100 that is an embodiment of the present invention. As shown in FIG. 7, the 3D assembly model search system 100 of this embodiment includes an assembly model decomposition unit 10 that decomposes a 3D assembly model to be searched into subassemblies, and a feature value that generates feature quantities of the subassemblies. The quantity generation unit 20, the similarity calculation unit 30 that calculates the similarity between the 3D assembly model to be searched and the 3D assembly model registered in the database 50 (hereinafter referred to as registered assembly model), The detection result output unit 40 is configured to include a detection result output unit 40 that outputs the detection results.

The contents of the process executed by the three-dimensional assembly model search system 100 will be explained based on the flowchart shown in FIG.

First, in step 101, the assembly model decomposition unit 10 decomposes a three-dimensional assembly model to be searched into a plurality of subassemblies.

In the subsequent step 102, the feature amount generation unit 20 executes feature amount generation processing. The feature amount generation process here is performed by either the first proposed method (1D) or the second proposed method (2D) described above.

First, feature amount generation processing A based on the first proposed method (1D) described above will be explained based on the flowchart shown in FIG.

First, in step 201, a subassembly that generates a feature amount is selected from among a plurality of subassemblies that constitute a three-dimensional assembly model to be searched.

In the following step 202, for the selected subassembly, three-dimensional Radon transformation is performed from M projection angles (θ, φ) based on spherical coordinates (M is an integer of 2 or more; the same applies hereinafter) to obtain M 1 Obtain dimensional projection data. The one-dimensional projection data is weighted projection data normalized by volume as described above.

In the following step 203, each of the acquired M one-dimensional projection data is subjected to discrete Fourier transform in the radial direction to obtain N discrete Fourier transforms for N frequencies (N is an integer of 2 or more; the same applies hereinafter). By obtaining an amplitude spectrum, (M×N) amplitude spectrum values are obtained. FIG. 10 conceptually shows the processing of steps 202-203.

In the following step 204, the relationship between the projection angle and the amplitude spectrum value for each frequency is determined when the obtained (M×N) amplitude spectrum values are arranged into a two-dimensional matrix of M projection angles and N frequencies. Assuming that is a scalar function on a spherical surface, N scalar functions are defined.

In the following step 205, for each of the N scalar functions defined, spherical harmonic transformation is performed using the maximum expansion order L of spherical harmonic coefficients (L is an integer of 2 or more. The same applies hereinafter) to obtain the norm for each order. By doing so, (N×L) norms are obtained.

In the following step 206, a vector having (N×L) norms as elements, which is obtained by arranging the obtained (N×L) norms according to a predetermined rule, is generated as a feature amount of the subassembly selected in step 201. Then, the generated (N×L) dimensional vector is stored in a predetermined storage area. FIG. 11 conceptually shows the processing of steps 202 to 206.

Thereafter, the series of processes described above (steps 201 to 206) are repeated until the feature quantities of all subassemblies constituting the three-dimensional assembly model to be searched are generated. At that point (step 207, Yes), the process ends.

The feature amount generation process A based on the first proposed method (1D) has been explained above, so next, the feature amount generation process B based on the second proposed method (2D) will be explained based on the flowchart shown in FIG. do.

First, in step 301, a subassembly that generates a feature amount is selected from among a plurality of subassemblies that constitute a three-dimensional assembly model to be searched.

In the following step 302, perspective projection is performed on the selected subassembly from M projection angles (θ, φ) based on spherical coordinates to obtain M two-dimensional projections. This projection is also a weighted projection normalized by volume as described above. FIG. 13 conceptually shows the process of step 302.

In the following step 303, two-dimensional Radon transform is performed on each of the M acquired two-dimensional projections to acquire M sinograms.

In the following step 304, one-dimensional discrete Fourier transform is performed on each of the M acquired sinograms in the radial direction to obtain N discrete amplitude spectra for N frequencies. Next, one-dimensional discrete Fourier transform is performed in the angular direction to obtain N discrete amplitude spectra for N frequencies. As a result, (M×N×N) amplitude spectrum values are obtained. FIG. 14 conceptually shows the processing of steps 303 to 304.

In the following step 305, the projection angle for each frequency is determined when the obtained (M×N×N) amplitude spectrum values are arranged into a two-dimensional matrix of M projection angles and (N×N) frequencies. The relationship between the amplitude spectrum value and the amplitude spectrum value is regarded as a scalar function on a spherical surface, and (N×N) scalar functions are defined. Here, it is assumed that all amplitude spectrum values are used (N x N), but it is not limited to this, and it is also possible to use only N out of (N x N). be. In this case, the Fourier transform described above can be reduced to just that number. Therefore, for example, two arbitrary integers n1 and n2 can be determined, and an arbitrary size such as a size of (n1×n2) can be set.

In the following step 306, spherical harmonic transformation is performed on each of the defined (N×N) scalar functions using the maximum expansion order L of spherical harmonic coefficients (L is an integer of 2 or more; the same applies hereinafter), and each order is By finding the norm of , (N×N×L) norms are obtained.

In the following step 307, the acquired (N×N×L) norms are arranged according to a predetermined rule, and a vector having (N×N×L) norms as elements is created by arranging the obtained (N×N×L) norms according to a predetermined rule. It is generated as a feature quantity, and the generated (N×N×L)-dimensional vector is stored in a predetermined storage area. FIG. 15 conceptually shows the processing of steps 302 to 307.

Thereafter, the series of processes described above (steps 301 to 307) are repeated until the feature quantities of all subassemblies constituting the three-dimensional assembly model to be searched are generated. At that point (step 308, Yes), the process ends.

Since the two types of feature amount generation processing (A, B) have been described above, the description will be continued by returning to FIG. 8 again.

When the generation of features of the subassemblies constituting the three-dimensional assembly model to be searched is completed, the process proceeds to step 103.

In the subsequent step 103, the similarity calculation unit 30 selects a registered assembly model to be compared from among the registered assembly models registered in the database 50.

In the following step 104, the similarity calculation unit 30 selects a one-to-one correspondence between the subassembly of the 3D assembly model to be searched and the subassembly of the selected registered 3D assembly model. Search for a combination that minimizes the sum of feature distances between subassemblies. The search here can be regarded as a problem of assigning a distance matrix of features between subassemblies, and can be solved using the Hungarian method, for example.

In the following step 105, the similarity calculation unit 30 calculates the reciprocal of the sum of the distances of the feature amounts in the searched combination as the similarity of the registered three-dimensional assembly model selected in step 103, and uses the calculated similarity as the similarity of the registered three-dimensional assembly model selected in step 103. Save it in the specified storage area.

Thereafter, the series of processes described above (steps 103 to 105) are repeated until the similarity of all registered 3D assembly models registered in the database 50 is calculated, and the similarity of all registered 3D assembly models is calculated. is generated (step 106, Yes), the process proceeds to step 107.

In the following step 107, the detection result output unit 40 compares the degrees of similarity stored in a predetermined storage area, outputs the registered three-dimensional assembly model having the greatest degree of similarity as a detection result, and ends the process. .

Although the present invention has been described above using embodiments, the present invention is not limited to the above-described embodiments, and various changes, modifications, improvements, etc. can be made without departing from the spirit of the present invention. Needless to say.

Note that each function of the embodiments described above can be written as a computer-executable program using an appropriate programming language, and the program can be stored and distributed in any recording medium, and can be distributed over a network. can be transmitted via.

Evaluation experiments were conducted on the proposed method of the present invention described above using three-dimensional CAD assembly models with different arrangement of parts. In this evaluation experiment, 15 types of three-dimensional assembly models shown in FIG. 16 were used. These are models selected from GrabCAD (http://www.grabcad.com/library) and models that combine multiple of them. However, some parts were excluded for simplicity. For each of the 15 types of assembly models, types 1 to 5 with different parts placements were prepared.

FIG. 17 shows an example of the pump model used in the experiment. The different types of parts used are expressed by different colors, and as shown in Figures 17(a) to (e), the overall shape and the shape and number of parts used are the same. However, the parts are arranged differently from each other. These 75 models of 15 types x 5 types were used as database models. However, rotations and translations were randomly added to the database model. In addition, type 1 of each assembly model was used as a query model. During the experiment, rotations and translations were added randomly, similar to the database model. The number of projection images in the proposed method of the present invention is determined by the spherical orthogonal coordinates, so the number of projection images is 12 (= 3 × 4), 45 (= 5 × 9), 91 (= 7 × 13), and 162 (= 9×18) sheets, 253(=11×23) sheets, 364(=13×28) sheets, or 495(=15×33) sheets. |R| in Algorithm 1, which is the projection size, was set to 96.

On the other hand, using a model similar to the one described above, the present inventor's previous research (Non-Patent Document 9: Kaoru Katayama and Takumi Sato. A Matching Method for 3D CAD Models with Different Assembly Structures Using Projections of Weighted Components. Journal of Information Processing, Vol. 25, pp. 376-385, 2017.) (hereinafter referred to as the previous study), we conducted a control experiment. In previous research, the number of projection images was determined by the number of vertices of the GeodesicDome, so the number of projection images was set to 12, 42, 92, 162, 252, 362, or 492.

The PC environment used in this evaluation experiment is as follows.
・CPU:Intel Core i7-7700K(4.20GHz)
・OS: Windows (registered trademark) 10 Education (64bit)
・Memory (RAM): 32.0 GB
・Software: MATLAB 2018b

<Identification confirmation experiment of similar models>
An identical model search was performed using the first proposed method (1D projection), and the degree of similarity between each query model and each database model was evaluated. A search for the same model was performed with the number of projections set to 364 ((|Θ|, |Φ|)=(13, 28)).

The experimental results are shown in FIG. FIG. 18 shows the degree of similarity between 15 types of assembly models that are query models and 75 types that are databases. The correct model is type 1 of each model, and is indicated by a red diamond on each polygonal line. As shown in FIG. 18, each query model had the highest degree of similarity with its respective correct model. Furthermore, there were no cases where the similarity between the query model and other types of models was higher than the similarity between the query model and models of the same type. The results show that the proposed method of the present invention can identify differences in shape and can also identify models with different internal structures.

<Experiment comparing processing time with previous research>
We compared the processing time required to search for the same model per query model when changing the number of projections for the previous research, the first proposed method (1D projection), and the second proposed method (2D projection). Figure 19(a), (b), () shows the changes in processing time of the previous study, the first proposed method (1D projection), and the second proposed method (2D projection) when the number of projection images is changed. Shown in c). The problem with previous studies is that when calculating the similarity between assembly models, it is necessary to match viewpoints by brute force calculation, which increases the processing time. The results shown in FIG. 19 show that when the number of projected images is 91 or more, the proposed methods (first and second) of the present invention have a shorter overall processing time. According to the proposed method of the present invention, compared to previous studies, the time required for similarity calculation was reduced by up to 50%, and the overall processing time was reduced by up to 31%.

<Experiment comparing feature size with previous research>
Regarding the previous research, the first proposed method (1D projection), and the second proposed method (2D projection), the feature amount per part in the parameter settings where the correct answer rate becomes 100% for the first time when the number of projections is increased. The size was evaluated. In previous research, the correct answer rate reached 100% only when the number of projected images was 362, and in the proposed method of the present invention (first and second), the number of projected images was 364 ((|Θ|, |Φ|)=(13, 28)), the correct answer rate reached 100% for the first time.

FIG. 20 shows the size of the feature amount per component at this time. The vertical axis is a logarithmic axis. From FIG. 20, the sizes of the feature quantities of the first proposed method (1D projection) and the second proposed method (2D projection) are the same. This is because the maximum expansion order is determined according to the number of projections in the process of spherical harmonic transformation in feature quantity extraction. Therefore, if the number of projections is the same and the size of the projections is also the same, the size of the feature amount will also be the same. As shown in FIG. 20, it was found that according to the proposed method of the present invention, it is possible to search for the same model using a feature amount that is about 1/100th the size of previous research.

<Comparison experiment of correct answer rate for each method>
Regarding the previous research, the first proposed method (1D projection), and the second proposed method (2D projection), we evaluated the correct answer rate of the same model search when changing the number of projections. FIG. 21 shows changes in the correct answer rate for each method when the number of projection images is changed. As shown in Figure 21, the correct answer rate approaches 100% for each method as the number of projection images increases; in previous research, the correct answer rate reached 100% for the first time when the number of projection images was 362, and the first proposed method (1D Projection) and the second proposed method (2D projection), the correct answer rate reached 100% for the first time when the number of projected images was 364 ((|Θ|, |Φ|)=(13, 28)). It was found that the first proposed method (1D projection) had a higher correct answer rate than the second proposed method (2D projection). Note that when the number of projections is small, the correct answer rate for the methods proposed by the present invention (first and second) is lower than in previous studies, but this is because the spatial resolution in the angular direction decreases due to the reduction in the number of projections. This seems to be due to angular aliasing occurring during spherical harmonic transformation using the proposed method. To prevent aliasing in the angular direction, it is necessary to increase the number of projections according to the angular complexity (frequency) of the original model, and as shown in Figure 21, as the number of projections increases, the correct answer rate approaches 100%. I understand that.

10... Assembly model decomposition unit 20... Feature value generation unit 30... Similarity calculation unit 40... Detection result output unit 50... Database 100... Three-dimensional assembly model search system

Claims (13)

A computer-implemented method for generating features of a three-dimensional model, the method comprising:
performing three-dimensional Radon transformation from M projection angles (M is an integer of 2 or more) on the target three-dimensional model to obtain M one-dimensional projection data;
By performing discrete Fourier transform in the radial direction on each of the M one-dimensional projection data to obtain N amplitude spectra for N frequencies (N is an integer of 2 or more) , (M×N) obtaining an amplitude spectrum of
When the (M×N) amplitude spectra are arranged into a two-dimensional matrix of the projection angle and the frequency, the relationship between the projection angle and the amplitude spectrum for each frequency is regarded as a scalar function on a spherical surface. , defining N scalar functions;
a step of obtaining (N×L) norms by performing spherical harmonic transformation on each of the N scalar functions at a maximum expansion order L (L is an integer of 2 or more) and obtaining a norm for each order; ,
A method including the step of generating a vector having the (N×L) norms as elements as a feature of the three-dimensional model. In the step of acquiring the one-dimensional projection data, the target three-dimensional model is a three-dimensional model of a subassembly (three-dimensional subassembly model), which is a collection of parts that are given the same label according to the characteristics of the parts. , the projection of the three-dimensional subassembly model and the projection of the portion of the three-dimensional assembly model excluding the three-dimensional subassembly model are normalized, and the projection data obtained by combining them is used as the one-dimensional projection data. 2. The method of claim 1, wherein: A computer-implemented method for generating features of a three-dimensional model, the method comprising:
performing projections on the target three-dimensional model from M projection angles (M is an integer of 2 or more) to obtain M two-dimensional projections;
performing a two-dimensional Radon transform on each of the M two-dimensional projections to obtain M two-dimensional projection data;
Discrete Fourier transform is performed on each of the M two-dimensional projection data in the radial direction, and then discrete Fourier transform is performed in the angular direction to obtain N amplitude spectra for N frequencies (N is an integer of 2 or more). obtaining (M×N×N) amplitude spectra by obtaining;
When the (M×N×N) amplitude spectra are arranged into a two-dimensional matrix of the projection angle and the frequency, the relationship between the projection angle and the amplitude spectrum for each frequency is expressed as a scalar function on a spherical surface. a step of defining (N×N) scalar functions;
By performing spherical harmonic transformation on each of the (N×N) scalar functions at the maximum expansion order L (L is an integer of 2 or more) and finding the norm for each order, we can obtain the (N×N×L) scalar functions. a step of obtaining the norm;
generating a vector having the (N×N×L) norms as elements as a feature of the three-dimensional model;
method including. In the step of acquiring the two-dimensional projection, the target three-dimensional model is a three-dimensional model of a subassembly (three-dimensional subassembly model) that is a collection of parts that are given the same label according to the characteristics of the parts, 4. The method according to claim 3, wherein a projection of the three-dimensional subassembly model and a projection of a portion of the three-dimensional assembly model excluding the three-dimensional subassembly model are respectively normalized and combined to obtain the two-dimensional projection. A computer-executed method for calculating the similarity of a three-dimensional assembly model formed by combining a plurality of three-dimensional part models, the method comprising:
For each subassembly 3D model (3D subassembly model), which is a collection of parts given the same label according to the characteristics of the parts, which constitutes the 3D assembly model to be searched, M (M is performing a three-dimensional Radon transformation from the projection angle (an integer of 2 or more) to obtain M one-dimensional projection data;
The three-dimensional subassembly model is obtained by performing discrete Fourier transform on each of the M one-dimensional projection data in the radial direction to obtain N amplitude spectra for N frequencies (N is an integer of 2 or more). obtaining (M×N) amplitude spectra for each
When the (M×N) amplitude spectra are arranged into a two-dimensional matrix of the projection angle and the frequency, the relationship between the projection angle and the amplitude spectrum for each frequency is regarded as a scalar function on a spherical surface. , defining N scalar functions for each of the three-dimensional subassembly models;
By performing spherical harmonic transformation on each of the N scalar functions at the maximum expansion order L (L is an integer of 2 or more) and finding the norm for each order, (N×L) is calculated for each three-dimensional subassembly model. a step of obtaining the norm of
generating a vector having the (N×L) norms as elements as a feature quantity of each of the three-dimensional subassembly models;
Among the combinations of matching the 3D subassembly models that constitute the 3D assembly model to be searched and the 3D subassembly models of the registered 3D assembly models, select between the matched 3D subassembly models. searching for a combination that minimizes the sum of feature distances;
A method including the step of calculating the reciprocal of the total sum in the searched combinations as the similarity. In the step of acquiring the one-dimensional projection data, the projection of the three-dimensional subassembly model and the projection of the portion of the three-dimensional assembly model excluding the three-dimensional subassembly model are normalized and synthesized. 6. The method according to claim 5, wherein data of the one-dimensional projection is acquired as the one-dimensional projection data. A computer-executed method for calculating the similarity of a three-dimensional assembly model formed by combining a plurality of three-dimensional part models, the method comprising:
For each subassembly 3D model (3D subassembly model), which is a collection of parts given the same label according to the characteristics of the parts, which constitutes the 3D assembly model to be searched, M (M is obtaining M two-dimensional projections by performing projection from a projection angle (an integer greater than or equal to 2);
performing a two-dimensional Radon transformation on each of the M two-dimensional projections to obtain M two-dimensional projection data for each of the three-dimensional subassembly models;
Discrete Fourier transform is performed on each of the M two-dimensional projection data in the radial direction, and then discrete Fourier transform is performed in the angular direction to obtain N amplitude spectra for N frequencies (N is an integer of 2 or more). obtaining (M×N×N) amplitude spectra for each of the three-dimensional subassembly models;
When the (M×N×N) amplitude spectra are arranged into a two-dimensional matrix of the projection angle and the frequency, the relationship between the projection angle and the amplitude spectrum for each frequency is expressed as a scalar function on a spherical surface. and defining (N×N) scalar functions for each of the three-dimensional subassembly models;
By performing spherical harmonic transformation on each of the (N×N) scalar functions at the maximum expansion order L (L is an integer of 2 or more) and finding the norm for each order, ( obtaining N×N×L) norms;
generating a vector having the (N×N×L) norms as elements as a feature quantity of each of the three-dimensional subassembly models;
Among the combinations of matching the 3D subassembly models that constitute the 3D assembly model to be searched and the 3D subassembly models of the registered 3D assembly models, select between the matched 3D subassembly models. searching for a combination that minimizes the sum of feature distances;
A method including the step of calculating the reciprocal of the total sum in the searched combinations as the similarity. In the step of obtaining the two-dimensional projection, the projection of the three-dimensional subassembly model and the projection of the portion of the three-dimensional assembly model excluding the three-dimensional subassembly model are respectively normalized and combined to obtain the two-dimensional projection. 8. A method according to claim 7, wherein a projection is obtained. A program for causing a computer to execute each step of the method according to any one of claims 1 to 8. A system for searching a three-dimensional assembly model formed by combining a plurality of three-dimensional part models,
Regarding the 3D model of the subassembly (3D subassembly model), which is a collection of parts given the same label according to the characteristics of the parts, which constitutes the 3D assembly model to be searched, for each 3D subassembly model. a feature quantity generation means for generating a feature quantity;
similarity calculation means for calculating the similarity between the 3D assembly model to be searched and the registered 3D assembly model based on the feature amount of each 3D subassembly model;
and means for outputting the registered three-dimensional assembly model having the maximum similarity as a detection result,
The feature amount generation means includes:
means for performing three-dimensional Radon transformation from M projection angles (M is an integer of 2 or more) for each of the three-dimensional subassembly models to obtain M one-dimensional projection data;
By performing discrete Fourier transform in the radial direction on each of the M one-dimensional projection data to obtain N (N is an integer of 2 or more) amplitude spectra for N frequencies , (M×N) amplitude spectra are obtained. means for obtaining an amplitude spectrum of
When the (M×N) amplitude spectra are arranged into a two-dimensional matrix of the projection angle and the frequency, the relationship between the projection angle and the amplitude spectrum for each frequency is regarded as a scalar function on a spherical surface. , means for defining N scalar functions;
means for obtaining (N×L) norms by performing spherical harmonic transformation on each of the N scalar functions at a maximum expansion order L (L is an integer of 2 or more) and obtaining a norm for each order; ,
generating a vector having the (N×L) norms as elements as a feature quantity of each of the three-dimensional subassembly models;
The similarity calculation means includes:
Among the combinations in which the 3D subassembly models constituting the 3D assembly model to be searched are matched with the 3D subassembly models of the registered 3D assembly model, the matched 3D subassembly models are selected. means for searching for a combination that minimizes the sum of the distances of the features;
and means for calculating the reciprocal of the sum of the searched combinations as the similarity.
Search system. The means for acquiring the one-dimensional projection data is obtained by normalizing and synthesizing the projection of the three-dimensional subassembly model and the projection of a portion of the three-dimensional assembly model excluding the three-dimensional subassembly model. The search system according to claim 10, wherein data of a projected projection is acquired as the one-dimensional projection data. A system for searching a three-dimensional assembly model formed by combining a plurality of three-dimensional part models,
Regarding the 3D model of the subassembly (3D subassembly model), which is a collection of parts given the same label according to the characteristics of the parts, which constitutes the 3D assembly model to be searched, for each 3D subassembly model. a feature quantity generation means for generating a feature quantity;
similarity calculation means for calculating the similarity between the 3D assembly model to be searched and the registered 3D assembly model based on the feature amount of each 3D subassembly model;
and means for outputting the registered three-dimensional assembly model having the maximum similarity as a detection result,
The feature amount generation means includes:
means for obtaining M two-dimensional projections by projecting each of the three-dimensional subassembly models from M projection angles (M is an integer of 2 or more) ;
means for performing two-dimensional Radon transformation on each of the M two-dimensional projections to obtain M two-dimensional projection data;
Discrete Fourier transform is performed on each of the M two-dimensional projection data in the radial direction, and then discrete Fourier transform is performed in the angular direction to obtain N amplitude spectra for N frequencies (N is an integer of 2 or more). means for obtaining (M×N×N) amplitude spectra by obtaining;
When the (M×N×N) amplitude spectra are arranged into a two-dimensional matrix of the projection angle and the frequency, the relationship between the projection angle and the amplitude spectrum for each frequency is expressed as a scalar function on a spherical surface. means for defining (N×N) scalar functions,
By performing spherical harmonic transformation on each of the (N×N) scalar functions at the maximum expansion order L (L is an integer of 2 or more) and finding the norm for each order, we can obtain the (N×N×L) scalar functions. A means to obtain the norm,
generating a vector having the (N×N×L) norms as elements as a feature quantity of each of the three-dimensional subassembly models;
The similarity calculation means includes:
Among the combinations in which the 3D subassembly model of the 3D assembly model to be searched and the 3D subassembly model of the registered 3D assembly model are matched, the features between the matched 3D subassembly models are selected. means for searching for a combination that minimizes the sum of distances of quantities;
and means for calculating the reciprocal of the sum of the searched combinations as the similarity.
Search system. The means for acquiring the two-dimensional projection data normalizes the projection of the three-dimensional subassembly model and the projection of the portion of the three-dimensional assembly model excluding the three-dimensional subassembly model, and synthesizes the two. 13. The search system of claim 12, wherein a dimensional projection is obtained.