JP2021039647A - Image data classification device and image data classification method - Google Patents

Abstract

To provide an image data classification technique that allows a user to efficiently and easily use appropriate image data used for restoring a three-dimensional shape of an object among a large amount of image data.SOLUTION: An image data classification device 1 includes: an image data acquisition unit 10 that acquires identification information indicating ordering of a plurality of image data and each of the plurality of image data; a feature amount calculation unit 11 that calculates a feature amount of each of the plurality of acquired image data; a similarity calculation unit 12 that calculates, based on the feature amount calculated by the feature amount calculation unit 11, degree of similarity between the plurality of image data; and a classification unit 15 that classifies, based on the similarity calculated by the similarity calculation unit 12 and the identification information attached to each of the plurality of image data, the plurality of image data into one or more groups.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image data classification device and an image data classification method, and more particularly to an image data classification technique used for restoring a three-dimensional shape.

With the improvement of computer processing power and the progress of video technology, it has become possible to draw three-dimensional information on general-purpose computers, smartphones, tablet terminals, and the like. Therefore, various applications and services using real-world 3D models have appeared. For large spaces such as buildings, plants, and factories, efforts such as measurement of 3D spatial information in real space and facility management applying it are being promoted.

Conventionally, as a technique for restoring a three-dimensional shape in a real space, a high-precision laser scanner, Light Detection and Ranger, Laser Imaging Detection and Ranking (LiDAR), and the like have been used. However, the laser scanner has problems that the equipment is expensive, and LiDAR has a problem that the spatial decomposition is low in order to make the space three-dimensional.

Therefore, in recent years, with the development of computer vision, technological developments such as visual SLAM and Structure from Motion (SfM), which can restore a three-dimensional shape from a plurality of still images, have been progressing. VisualSLAM and SfM are technologies for extracting feature points from a plurality of still images, calculating parameters for restoring the three-dimensional shape from matching of the feature points, and constructing the three-dimensional shape.

In the conventional technique using visualSLAM or SfM, it is generally possible to construct a precise three-dimensional shape when shooting so as to surround the shooting target or when there are many overlapping regions of continuous still images. However, in the conventional technology using visualSLAM or SfM, when there are few textures or when trying to restore a three-dimensional shape in a large room, depending on the still image taken, the three-dimensional shape desired by the user can be obtained. It may not be possible to restore.

Specifically, even if the still image group is suitable for restoring 3D information, there are images including blur, blur, etc. in the images before and after the image, which may reduce the accuracy of the restored 3D information. In some cases. If the moving distance of the camera is large and does not include many common parts, it may not be regarded as a continuous series of still images in the calculation. In addition, when the number of images taken by the camera is insufficient for the object, it is excluded from the calculation target in the three-dimensional shape restoration process as a group of still images that are unsuitable for calculation or unnecessary. There is also.

In order to determine whether or not the three-dimensional shape can be obtained as expected by the user, it is conceivable to restore the three-dimensional shape at the site where the image is taken. However, when the number of still images to be captured is large, it is not realistic because the calculation time of the restoration process takes a certain amount.

Therefore, a method is conceivable in which the user confirms the image used for restoring the three-dimensional shape on the display screen in advance. However, it is practically difficult for a user who does not have specialized knowledge about image processing to individually select a more effective still image for restoring a desired object to a three-dimensional shape.

For example, Patent Document 1 discloses a technique for selecting an image that can calculate more accurate camera position and feature point position information in order to improve the accuracy of restoration of a three-dimensional shape. However, even if the still image to be used is selected by the logic and parameters in the computer, it is not always possible to make the restoration target desired by the user three-dimensional.

Further, for example, Patent Document 2 and Patent Document 3 disclose a technique for detecting and classifying scenes from an image. In the techniques described in Patent Documents 2 and 3, on the premise of moving images, feature quantities that require high continuity with respect to an imaged object are used. When such a conventional technique is applied to a still image taken by a user with a digital camera or the like, continuity may not be maintained and the restoration accuracy of the three-dimensional shape of the object may be lowered. Further, the techniques described in Patent Documents 2 and 3 focus on searching for a specific photographing target, and the tendency does not necessarily match the classification of scenes for restoring a three-dimensional shape.

Japanese Patent No. 501625 JP-A-2017-112448 Japanese Patent No. 4177689

In the conventional technique, there is a problem that the user cannot efficiently and easily use the appropriate image data used for restoring the three-dimensional shape of the object among a large amount of image data.

The present invention has been made to solve the above-mentioned problems, and the user can efficiently and easily obtain appropriate image data used when restoring the three-dimensional shape of an object among a large amount of image data. It is an object of the present invention to provide an image data classification technique that can be used for.

In order to solve the above-mentioned problems, the image data classification device according to the present invention is configured to acquire a plurality of image data and identification information indicating the ordering attached to each of the plurality of image data. Based on the acquisition unit, the feature amount calculation unit configured to calculate the feature amount of each of the acquired plurality of image data, and the feature amount calculated by the feature amount calculation unit, the plurality of A similarity calculation unit configured to calculate the similarity between image data, the similarity calculated by the similarity calculation unit, and the identification information attached to each of the plurality of image data. It is provided with a classification unit configured to classify the plurality of image data into one or a plurality of groups based on the above.

Further, in the image data classification device according to the present invention, the identification information may be information indicating the order in which the plurality of image data are captured.

Further, in the image data classification device according to the present invention, the identification information may be information indicating the time when each of the plurality of image data was photographed.

Further, in the image data classification device according to the present invention, the plurality of image data are still images, and the feature amount calculation unit detects feature points included in each of the plurality of image data and calculates the similarity. The unit may calculate the geometric distance between the feature points in the plurality of image data and compare the calculated distances to calculate the similarity.

Further, in the image data classification device according to the present invention, the classification unit may classify the plurality of image data into a group for each representative image data selected from the plurality of image data.

Further, in the image data classification device according to the present invention, the similarity is determined based on the similarity calculated by the similarity calculation unit and the identification information attached to the image data corresponding to the similarity. A representative image selection unit configured to select the maximum image data as the representative image data may be further provided.

Further, the image data classification device according to the present invention may further include a display unit configured to display the classification result by the classification unit on the display device.

Further, the image data classification device according to the present invention may further include a smoothing processing unit configured to filter and smooth the similarity calculated by the similarity calculation unit.

Further, the image data classification device according to the present invention further includes an adjustment unit configured to change and adjust the number of the groups in which the plurality of image data are classified, and the classification unit includes the plurality of adjustment units. The image data may be classified into the group in the number changed by the adjusting unit.

In order to solve the above-mentioned problems, the image data classification method according to the present invention includes a first step of acquiring a plurality of image data and identification information indicating the ordering attached to each of the plurality of image data, and the first step. Based on the second step of calculating the feature amount of each of the plurality of image data acquired in one step and the feature amount calculated in the second step, the degree of similarity between the plurality of image data is calculated. One or a plurality of groups of the plurality of image data based on the third step to be calculated, the similarity calculated in the third step, and the identification information attached to each of the plurality of image data. It is provided with a fourth step of classifying into.

According to the present invention, a plurality of image data are classified into one or a plurality of groups based on the similarity between the plurality of image data and the identification information indicating the ordering attached to each of the plurality of image data. , The user can efficiently and easily use the appropriate image data used when restoring the three-dimensional shape of the object from the plurality of image data.

FIG. 1 is a block diagram showing a configuration of an image data classification device according to the first embodiment of the present invention. FIG. 2 is a diagram for explaining image data according to the first embodiment. FIG. 3 is a diagram for explaining a classification unit according to the first embodiment. FIG. 4 is a block diagram showing an example of the hardware configuration of the image data classification device according to the first embodiment. FIG. 5 is a flowchart for explaining the operation of the image data classification device according to the first embodiment. FIG. 6 is a diagram for explaining selection of representative image data according to the first embodiment. FIG. 7 is a diagram showing a display example of the classification result according to the first embodiment. FIG. 8 is a block diagram showing a configuration of an image data classification device according to a second embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to FIGS. 1 to 8.

[Outline of Invention]
First, an outline of the image data classification device 1 according to the embodiment of the present invention will be described.
The image data classification device 1 classifies a plurality of image data into groups for each of a plurality of similar image data and presents them to the user. Since the representative image data representing each group is displayed on the display screen, the user does not need to check a plurality of image data one by one.

For example, consider a case where a user restores a three-dimensional shape of an object from a plurality of still images and analyzes the object. In such a case, the user can confirm the representative image data representing the group and select only the image data group to be analyzed.

For example, using well-known hierarchical clustering, it is possible to divide a plurality of image data into hierarchical clusters and present them to the user. However, in the conventional example using hierarchical clustering, a plurality of image data are classified into groups having high similarity, but the image data may not be classified according to the order in which they were taken. Image data divided into hierarchical clusters may not be easy for the user to check.

The image data classification device 1 calculates the degree of similarity with each other in a large amount of image data taken by the user, and identifies the calculated degree of similarity and the ordering such as time information in which the image data was taken and the order in which the image data was taken. Based on the information, the image data is classified into a plurality of groups and the classification result is presented to the user.

As described above, the image data classification device 1 according to the present embodiment is characterized in that it classifies the image data by reflecting the time series information such as the time when the image data was taken and the order in which the image data was taken. It is one.

[First Embodiment]
Next, the image data classification device 1 according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 7.

[Functional block of image data classification device]
As shown in FIG. 1, the image data classification device 1 includes an image data acquisition unit 10, a feature amount calculation unit 11, a similarity calculation unit 12, a smoothing processing unit 13, a representative image selection unit 14, a classification unit 15, and a storage unit. A display unit 17 and a display unit 17 are provided.

The image data acquisition unit 10 acquires the plurality of image data and the identification information attached to each of the plurality of image data. The image data is, for example, image data such as a plurality of still images taken continuously or discontinuously at regular time intervals by a user using a digital camera (not shown) or the like. The number of image data may be, for example, hundreds to thousands.

Identification information indicating the ordering of the image data is attached to each of the image data. The identification information is information indicating the order in which the image data was taken or the order in which the image data acquisition unit 10 acquired the image data. For example, as the identification information, a sequential number of image data, a sequential image name, or the like can be used. Instead of the order in which the image data is taken, information indicating the time when the plurality of image data are taken or the time acquired by the image data acquisition unit 10 can be used as the identification information.

The image data acquisition unit 10 can acquire a moving image and generate a plurality of still images, in addition to the case of acquiring continuously captured still images.

Further, the image data acquisition unit 10 may perform preprocessing such as adjusting the brightness and removing noise on the acquired image data. The identification information corresponding to each image data acquired by the image data acquisition unit 10 is stored in the storage unit 16.

The feature amount calculation unit 11 calculates the feature amount of each of the plurality of image data acquired by the image data acquisition unit 10. More specifically, the feature amount calculation unit 11 detects feature points such as edges and corners included in each of the plurality of image data, and calculates the feature amount of the detected feature points. The feature amount referred to here is a feature vector obtained by converting a region around a feature point into a feature vector based on a pixel value or a differential value. The feature amount calculation unit 11 calculates the feature amount of the image data by using the feature amount extraction algorithm of the image data such as the KAZE algorithm, the AKAZE algorithm, the SIFT algorithm, or the SURF algorithm.

The similarity calculation unit 12 calculates the degree of similarity between the plurality of image data calculated by the feature amount calculation unit 11. For example, the similarity calculation unit 12 extracts image data captured immediately before or after the arbitrary image data based on the order in which the image data indicated by the identification information attached to the image data is captured. In addition, the similarity calculation unit 12 calculates the similarity between the corresponding feature points in the image data.

The similarity calculation unit 12 detects the feature points having the highest degree of similarity among the image data as feature points corresponding to each other. As the degree of similarity between the feature points, the Euclidean distance, the urban distance, the correlation coefficient, etc. of the feature quantity (feature vector) at the feature points may be used. The similarity calculation unit 12 integrates the similarity or averages the similarity by the number of corresponding points to obtain the similarity between the image data. Further, the similarity calculation unit 12 may set the number of corresponding feature points as the similarity between the image data.

The smoothing processing unit 13 filters and smoothes the similarity calculated by the similarity calculation unit 12. For example, the smoothing processing unit 13 uses a moving average filter to perform a moving average for the similarity between the corresponding feature points of the two image data to perform smoothing. More specifically, the smoothing processing unit 13 performs a smoothing process of similarity using the following equation (1).

In the above equation (1), the smoothed similarity is SIM_Ave (i), and i is the identification information indicating the reference image data (i = N + 1, N + 2, ..., Nk) (however, N ≧ 1) and (2N + 1) represent the size of the filter.

The representative image selection unit 14 uses the similarity calculated by the similarity calculation unit 12 and the identification information attached to the image data corresponding to the similarity to generate representative image data representing a plurality of image data. Select. Specifically, the representative image selection unit 14 calculates the differential SIM_Ave'(i) for the similarity SIM_Ave (i) smoothed by the smoothing processing unit 13. The representative image selection unit 14 selects the image data in which the smoothed differential SIM_Ave'(i-1)> 0 and the SIM_Ave'(i) <0 are the image data in which the similarity is maximized, that is, Select as representative image data I.

The classification unit 15 sets one or a plurality of image data based on the degree of similarity between the plurality of image data calculated by the similarity calculation unit 12 and the identification information attached to each of the plurality of image data. Classify into the group of.

More specifically, the classification unit 15 uses the representative image data I selected by the representative image selection unit 14 as a group for classifying the image data. The classification unit 15 calculates the degree of similarity between each image data and two representative image data that are close in time series to the image data, based on the order in which the image data was taken. The classification unit 15 compares the two calculated similarities and classifies the image data into the group of the representative image data having the higher similarity.

Specifically, the classification unit 15 is closer to the order of the image data i in chronological order based on the order in which the image data i was photographed attached to the image data i for the arbitrary image data i. Two representative image data I and I + 1 are extracted. The classification unit 15 calculates the degree of similarity (distance) between the image data i and the extracted representative image data I and I + 1. The classification unit 15 determines the classification boundary line of the image data at the position of the image data i such that [similarity between the image data i and the representative image data I] <[similarity between the image data i and the representative image data I + 1]. To do.

Based on the determined boundary line, the classification unit 15 sets the image data existing between the representative image data I and the representative image data I + 1 to have a higher degree of similarity among the representative image data I or the representative image data I + 1 (distance). Classify into the group (closer to).

The storage unit 16 stores a plurality of image data and identification information acquired by the image data acquisition unit 10. Further, the storage unit 16 stores information indicating the representative image data selected by the representative image selection unit 14 and the classification result by the classification unit 15.

The display unit 17 causes the display device 108 to display the classification result by the classification unit 15. For example, as shown in the example of FIG. 2, the display unit 17 displays a plurality of image data (IM1, IM2, ..., IMN) on the display screen of the display device 108 according to the order of shooting 1 to N. Display side by side from left to right on the screen.

Further, as shown in the example of FIG. 3, the display unit 17 displays the image data classified into the classification groups GR1, GR2, ..., GRN corresponding to the representative image data as the classification result by the classification unit 15. Display in. For example, the groups GR1, GR2, ... Can be displayed side by side from the top to the bottom of the screen in the order in which the representative image data is captured. Further, the image data classified into each group can be displayed side by side from the left to the right of the screen according to the order in which the image data are taken.

[Hardware configuration of image data classification device]
Next, an example of the hardware configuration of the image data classification device 1 having the above-mentioned functions will be described with reference to FIG.

As shown in FIG. 4, the image data classification device 1 includes, for example, a processor 102, a main storage device 103, a communication interface 104, an auxiliary storage device 105, an input / output I / O 106, and an input device 107 connected via a bus 101. , And a computer equipped with a display device 108, and a program that controls these hardware resources.

The processor 102 is composed of a CPU, a GPU, and the like. When the processor 102 is composed of a plurality of GPUs, the information related to the image signal is appropriately divided and parallel processing is performed by the plurality of GPUs.

A program for the processor 102 to perform various controls and calculations is stored in the main storage device 103 in advance. The image data classification device 1 includes the feature amount calculation unit 11, the similarity calculation unit 12, the smoothing processing unit 13, the representative image selection unit 14, and the classification unit 15 shown in FIG. 1 by the processor 102 and the main storage device 103. Each function is realized.

The communication interface 104 is an interface circuit for connecting the image data classification device 1 and various external electronic devices to a network. For example, the classification result by the classification unit 15 can be transmitted from the communication interface 104 to an external terminal device (not shown) via the communication network NW.

The auxiliary storage device 105 is composed of a readable and writable storage medium and a drive device for reading and writing various information such as programs and data to and from the storage medium. A semiconductor memory such as a hard disk or a flash memory can be used as the storage medium in the auxiliary storage device 105.

The auxiliary storage device 105 has a program storage area for storing a program for the image data classification device 1 to execute the image data classification process. Further, the auxiliary storage device 105 has an area for storing the feature amount extraction algorithm of the image data used by the feature amount calculation unit 11. Further, the auxiliary storage device 105 has an area for storing the similarity calculation function used by the similarity calculation unit 12. Further, the auxiliary storage device 105 has an area for storing the filter information used by the smoothing processing unit 13.

The auxiliary storage device 105 realizes the storage unit 16 described with reference to FIG. Further, for example, it may have a backup area for backing up the above-mentioned data, programs, and the like.

The input / output I / O 106 is composed of an I / O terminal that inputs a signal from an external device and outputs a signal to the external device.

The input device 107 is composed of a keyboard, a touch panel, or the like, and outputs a signal corresponding to an operation input from the outside.

The display device 108 is composed of a liquid crystal display or the like. The display device 108 realizes the display unit 17 described with reference to FIG.

The camera 109 can convert an optical signal into an image signal to generate a moving image or a still image. More specifically, the camera 109 has an image pickup device such as a CCD (charge-coupled device) image sensor or a CMOS image sensor, and forms an image of light incident from the image pickup region on the light receiving surface. Convert to an electrical signal. The frame rate, exposure value, magnification, focus, etc. when the camera 109 shoots a moving image are automatically set in advance by a control unit (not shown) included in the camera 109. Communication between the camera 109 and the image data classification device 1 may be performed wirelessly.

[Image data classification method]
Next, the operation of the image data classification device 1 having the above-described configuration will be described with reference to the flowchart of FIG. In the following, it is assumed that the feature amount extraction algorithm of the image data is stored in the storage unit 16 in advance. In addition, the user continuously captures still images of the object to be analyzed and the periphery of the target area by the camera 109 in advance by restoring the shape to a three-dimensional shape. Further, it is assumed that sequence information indicating the order in which the images are taken is attached as identification information to the image data taken by the camera 109.

First, the image data acquisition unit 10 acquires a plurality of image data captured by the camera 109 and the corresponding identification information from the camera 109 (step S1). More specifically, the image data acquisition unit 10 acquires image data continuously photographed by the camera 109 and information indicating the order in which the image data is photographed, and stores the image data in the storage unit 16. The image data acquisition unit 10 can preprocess the acquired image data by known signal processing such as adjusting the brightness and removing noise.

Next, the feature amount calculation unit 11 detects feature points such as edges and corners for each image data acquired by the image data acquisition unit 10, and calculates the feature amount of the detected feature points (step S2). More specifically, the feature amount calculation unit 11 calculates the feature amount of the image data by using a known feature amount extraction algorithm such as the KAZE algorithm, the AKAZE algorithm, the SIFT algorithm, or the SURF algorithm. The calculated feature points and feature amounts of the image data are stored in the storage unit 16 for each image data.

Next, the similarity calculation unit 12 calculates the distance between the image data based on the feature points and the feature amount of each image data using the two image data, and from the calculated distance, the image data of each other The degree of similarity with each other is calculated (step S3). For example, the similarity calculation unit 12 can calculate the similarity based on the Euclidean distance between the feature points of the two image data. The calculated similarity is stored in the storage unit 16 in association with each image data.

Next, the smoothing processing unit 13 filters and smoothes the similarity calculated in step S3 (step S4). Specifically, the smoothing processing unit 13 uses the above equation (1) to apply a moving average filter to smooth the similarity between the corresponding feature points of the two image data. .. The smoothed similarity is stored in the storage unit 16.

Next, the representative image selection unit 14 selects a plurality of representative image data representing the plurality of image data (step S5). More specifically, the representative image selection unit 14 calculates the derivative of the similarity between the corresponding feature points of the two image data smoothed in step S4, and the similarity is determined based on the calculated value. The maximum image data is selected as the representative image data.

FIG. 6 is a diagram for explaining representative image data selected by the representative image selection unit 14. The horizontal axis of FIG. 6 indicates the order in which the image data was taken. Each vertical bar indicates one image data. The vertical axis shows the number of feature points.

In FIG. 6, b1 to b9 show representative image data selected by the representative image selection unit 14. That is, the representative image data b1 to b9 indicate nine groups used by the classification unit 15 for classifying the image data.

Returning to FIG. 5, the classification unit 15 classifies each of the plurality of image data into one of the groups corresponding to the representative image data b1 to b9 (step S6). More specifically, the classification unit 15 extracts two representative image data I, I + 1, which are close to the order in which arbitrary image data i of the plurality of image data was taken, and each representative image data I, The degree of similarity with I + 1 is calculated. At this time, the classification unit 15 sets the classification boundary line at the position of the image data i such that [similarity between the image data i and the representative image data I] <[similarity between the image data i and the representative image data I + 1]. decide. The classification unit 15 calculates and groups the similarity of all of the plurality of image data.

After that, the display unit 17 causes the display device 108 to display the classification result by the classification unit 15 (step S7). FIG. 7 is a diagram showing a display example by the display unit 17. The horizontal axis of FIG. 7 indicates the order in which the image data was taken. Each vertical bar indicates one image data. The vertical axis shows the number of feature points. In the example of FIG. 7, the classification result of classifying 60 image data into 9 groups is shown. The display unit 17 displays all the image data in chronological order in the order in which they were photographed. In addition, the display unit 17 can display the classification result by color-coding each group.

The image data (orders 1 to 5) shown in the example of FIG. 7 are classified into the group of representative image data a1. In the example of FIG. 7, the representative image data a1 is image data obtained by photographing an area in which a wall, a compressor, and an air tank are installed. The image data (orders 6 to 8) are classified into the group of representative image data a2. The representative image data a2 is image data obtained by photographing the compressor from the front. The image data (orders 9 to 14) are classified into the group of representative image data a3. The representative image data a3 is image data obtained by photographing an area moved clockwise from the front of the compressor.

The image data (orders 15 to 30) are classified into the group of representative image data a4. The representative image data a4 is image data obtained by photographing an area on the side surface of the air source. The image data (orders 31 to 34) are classified into the group of representative image data a5. The representative image data a5 is image data obtained by photographing the periphery of the dehumidifier. The image data (orders 35 to 40) are classified into the group of representative image data a6. The representative image data a6 is image data obtained by photographing the periphery of the dehumidifier and the header.

The image data (orders 41 to 46) are classified into the group of representative image data a7. The representative image data a7 is image data obtained by photographing the periphery of the control panel. The image data (orders 47 to 53) are classified into the group of representative image data a8. The representative image data a8 is image data obtained by photographing the periphery of the air tank. The image data (orders 54 to 60) are classified into the group of representative image data a7. The representative image data a7 is image data obtained by photographing the periphery of the control panel.

The user confirms only the representative image data a1 to a9 on the display screen out of the 60 image data, and selects a group of image data including the object or area to be analyzed by restoring the three-dimensional shape. can do. By using the image data group that surely includes the object that the user wants to analyze in the calculation of the restoration of the three-dimensional shape, the calculation load of the restoration processing of the three-dimensional shape is reduced and the processing time is shortened. be able to.

In addition, the user can confirm the number of image data classified into each group on the display screen of the display device 108. For example, when the image data of a certain group is sparse, the image data classified into the group can be added. Based on the display of the classification result, the user can take measures to add the image data, so that the continuity of the image data can be improved. As a result, it is possible to prevent the image data from being excluded from the processing target as an image data group that is not suitable for calculation in the three-dimensional shape restoration process because the image data is sparse.

As described above, according to the image data classification device 1 according to the first embodiment, a plurality of image data are classified into a plurality of groups using the order in which the image data was taken and the time information. Therefore, among a large amount of image data, the user can efficiently and easily use the appropriate image data used when restoring the three-dimensional shape of the object.

Further, according to the image data classification device 1 according to the first embodiment, since the classification results of the image data are displayed on the display screen in the order in which the images are taken, a bird's-eye view of the situation of a large amount of image data is obtained. Information to be used can be presented to the user.

Further, according to the image data classification device 1 according to the first embodiment, a large amount of image data is classified by a simpler calculation procedure, so that in the field where the user is photographing the object with the camera 109, the image data is classified. The classification result can be presented to the user. Therefore, the user can take appropriate measures such as adding image data based on the classification result presented at the shooting site.

Further, according to the image data classification device 1 according to the first embodiment, time-series information such as the order in which the image data is taken is reflected in the classification result, so that the user is specialized in the restoration of the three-dimensional shape. Even if you do not have sufficient knowledge, you can select appropriate image data to be used in the restoration of the three-dimensional shape of the object. Further, as a result, the three-dimensional shape of the object or area desired by the user is more reliably restored.

In the embodiment described, the case where the representative image selection unit 14 selects the representative image data representing the plurality of image data has been described. However, the representative image data may be arbitrarily selected from a plurality of image data by the user, and the representative image data may be selected by the input device 107 receiving the operation input from the user.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In the following description, the same components as those in the first embodiment described above are designated by the same reference numerals, and the description thereof will be omitted.

In the first embodiment, the case where the display unit 17 causes the display device 108 to display the classification result of the image data by the classification unit 15 has been described. On the other hand, the image data classification device 1A according to the second embodiment further includes an adjustment unit 18 for adjusting the number of representative image data, that is, the number of classification groups.

FIG. 8 is a block diagram showing a configuration of the image data classification device 1A according to the present embodiment. The image data classification device 1A includes an image data acquisition unit 10, a feature amount calculation unit 11, a similarity calculation unit 12, a smoothing processing unit 13, a representative image selection unit 14, a classification unit 15, a storage unit 16, a display unit 17, and a display unit 17. The adjusting unit 18 is provided. The configurations other than the adjusting unit 18 are the same as those of the first embodiment.

The adjusting unit 18 changes and adjusts the number of groups in which a plurality of image data are classified. More specifically, the adjusting unit 18 changes the parameters of the moving average filter used by the smoothing processing unit 13. Based on the parameters of the moving average filter changed by the adjusting unit 18, the smoothing processing unit 13 smoothes the similarity. In addition, the representative image selection unit 14 selects a corresponding number of representative image data using the smoothed similarity. For example, the adjusting unit 18 can change the parameters of the moving average filter based on the operation input from the user received by the input device 107.

Specifically, first, based on the parameters of the initially set moving average filter, the smoothing process 13 performs the smoothing process, the representative image selection unit 14 selects the representative image data, and the classification unit 15 classifies the image data. Is executed. After that, the input device 107 receives an operation input indicating the change in the number of classification groups by the user according to the classification result displayed on the display screen by the display unit 17, and the adjustment unit 18 changes the parameters of the moving average filter. be able to.

As described above, according to the image data classification device 1A according to the second embodiment, the number of groups in which a plurality of image data are classified can be adjusted, so that the user can more efficiently and easily classify the classification results. It can be used.

In addition, in the above-described embodiment, the case where the image data classification devices 1 and 1A acquire and classify the image data captured by the camera 109 has been described. However, the image data may be prepared separately and stored in the storage unit 16 in advance.

The image data classification devices 1 and 1A described in the first and second embodiments are not limited to the case where they are composed of a single device, and each functional unit is a form in which each functional unit is distributed to a plurality of terminal devices. You may.

Although the embodiments of the image data classification apparatus and the image data classification method of the present invention have been described above, the present invention is not limited to the described embodiments, and those skilled in the art are within the scope of the invention described in the claims. It is possible to make various transformations that can be imagined.

1 ... Image data classification device, 10 ... Image data acquisition unit, 11 ... Feature amount calculation unit, 12 ... Similarity calculation unit, 13 ... Smoothing processing unit, 14 ... Representative image selection unit, 15 ... Classification unit, 16 ... Memory Unit, 17 ... Display unit, 101 ... Bus, 102 ... Processor, 103 ... Main storage device, 104 ... Communication interface, 105 ... Auxiliary storage device, 106 ... Input / output I / O, 107 ... Input device, 108 ... Display device, 109 ... Camera, NW ... Communication network.

Claims (10)

An image data acquisition unit configured to acquire identification information indicating a plurality of image data and the ordering attached to each of the plurality of image data, and an image data acquisition unit.
A feature amount calculation unit configured to calculate the feature amount of each of the acquired plurality of image data, and a feature amount calculation unit.
A similarity calculation unit configured to calculate the similarity between the plurality of image data based on the feature amount calculated by the feature amount calculation unit, and a similarity calculation unit.
The plurality of image data are classified into one or a plurality of groups based on the similarity calculated by the similarity calculation unit and the identification information attached to each of the plurality of image data. An image data classification device including a classification unit. In the image data classification apparatus according to claim 1,
The identification information is an image data classification device characterized in that the identification information is information indicating the order in which the plurality of image data are taken. In the image data classification apparatus according to claim 1 or 2.
The identification information is an image data classification device, characterized in that each of the plurality of image data is information indicating a time when the image data was taken. In the image data classification apparatus according to any one of claims 1 to 3,
The plurality of image data are still images, and the plurality of image data are still images.
The feature amount calculation unit detects feature points included in each of the plurality of image data, and detects the feature points.
The image data classification is characterized in that the similarity calculation unit calculates a geometric distance between the feature points in the plurality of image data and compares the calculated distances to calculate the similarity. apparatus. In the image data classification apparatus according to any one of claims 1 to 4.
The image data classification device is characterized in that the classification unit classifies the plurality of image data into a group for each representative image data selected from the plurality of image data. In the image data classification apparatus according to claim 5,
Based on the similarity calculated by the similarity calculation unit and the identification information attached to the image data corresponding to the similarity, the image data having the maximum similarity is selected as the representative image data. An image data classification device characterized by further including a representative image selection unit configured to perform the above. In the image data classification apparatus according to any one of claims 1 to 6,
An image data classification device further comprising a display unit configured to display the classification result by the classification unit on the display device. In the image data classification apparatus according to any one of claims 1 to 7.
An image data classification device further comprising a smoothing processing unit configured to filter and smooth the similarity calculated by the similarity calculation unit. In the image data classification apparatus according to any one of claims 1 to 8.
Further, an adjustment unit configured to change and adjust the number of the groups in which the plurality of image data are classified is provided.
The image data classification device is characterized in that the classification unit classifies the plurality of image data into the group in a number changed by the adjustment unit. The first step of acquiring the identification information indicating the ordering of the plurality of image data and the plurality of image data, and
The second step of calculating the feature amount of each of the plurality of image data acquired in the first step, and
Based on the feature amount calculated in the second step, the third step of calculating the similarity between the plurality of image data to each other, and
A fourth step of classifying the plurality of image data into one or a plurality of groups based on the similarity calculated in the third step and the identification information attached to each of the plurality of image data. Image data classification method comprising.