JP6641396B2 - Article classification apparatus, article classification system, article classification method and program - Google Patents

Description

  The present disclosure relates to an article classification device, an article classification system, an article classification method, and a program for classifying a plurality of different types of articles for each type.

  For example, in a process of producing an article such as a pipe used as a pipe in a factory or the like, a plurality of articles having different types such as shapes and specifications may be mixed. In particular, after a post-processing step, a finishing step, or the like after welding, the type number, symbol, or the like imprinted for each article is erased, and classification by type may become difficult.

  In this case, for example, articles having completely different shapes, such as U-shaped articles, S-shaped articles, and L-shaped articles, can be easily classified by visual confirmation. However, for example, in a plurality of L-shaped articles each having substantially the same size, a difference in shape such as an article bent at 90 °, an article bent at 88 °, and an article bent at 86 ° is obtained. In the case of a minute article, visual confirmation becomes difficult, and there is a possibility that the classification operation becomes complicated. Therefore, it is desired to develop a technique for easily and accurately classifying a plurality of types of mixed articles as described above.

  By the way, in recent years, when recognizing a subject, two-dimensional data based on three-dimensional shape data and texture data of an object, which is a subject, are used as feature amount data to be used for matching photographed subject data with learning data stored in advance. 2. Description of the Related Art There is known a technique for recognizing a subject at high speed by converting the image into a developed image and performing collation using feature amount data of the two-dimensional developed image (for example, see Patent Document 1). In addition, a technique is known that can classify a three-dimensional image into a plurality of classes with a small amount of computation using a neural network with high accuracy (for example, see Patent Document 2).

JP 2012-178133 A JP 2017-174039 A

  However, in Patent Literatures 1 and 2 described above, a subject is recognized at high speed by performing a predetermined collation process (Patent Literature 1), or classified into a plurality of classes with a small amount of computation (Patent Literature 2). However, there is no concept of using the similarity of the shape based on these data in the comparison between the photographed subject data and the learning data stored in advance. Further, in the above-mentioned Patent Documents 1 and 2, information processing for converting a three-dimensional image into a two-dimensional image is complicated.

  Therefore, the present disclosure has been made in view of the circumstances described above, and has as its object to provide an article classification device, an article classification system, an article classification method, and a program that can easily and accurately classify a plurality of types of mixed articles. I do.

  In order to achieve the above object, an article classification device according to a first aspect of the present disclosure is an article classification device that classifies a plurality of types of articles by type, and acquires captured image data of an article to be classified. A captured image acquisition unit, a storage unit that stores two-dimensional image data from a plurality of different viewpoints for each article, a similarity calculation unit that calculates a similarity between the captured image data and the two-dimensional image data, And a classification unit that classifies the articles based on the classification.

  Further, in order to achieve the above object, an article classification system according to a second aspect of the present disclosure is an article classification system that classifies a plurality of types of articles by type, and captures image data of an article to be classified. An image capturing unit that stores two-dimensional image data from a plurality of different viewpoints for each type of article; a similarity calculating unit that calculates a similarity between the captured image data and the two-dimensional image data; It is characterized by comprising a classification unit that classifies articles by type based on the degree, and an output unit that outputs a result of classification by the classification unit.

  Further, in order to achieve the above object, an article classification method according to a third aspect of the present disclosure is an article classification method for classifying a plurality of types of articles by type, and acquires captured image data of an article to be classified. A captured image acquiring step, two-dimensional image data from a plurality of different viewpoints for each type of article stored in the storage unit, a similarity calculating step of calculating a similarity between the captured image data, and a similarity calculating step. And a classification step of classifying the articles by type.

  In order to achieve the above object, a program according to a fourth aspect of the present disclosure provides a computer with: a captured image acquisition unit configured to acquire captured image data of an article to be classified; Function as storage means for storing two-dimensional image data from different viewpoints, similarity calculating means for calculating similarity between captured image data and two-dimensional image data, classification means for classifying articles by type based on the similarity Let it.

  According to the present disclosure, a plurality of types of mixed articles can be easily and accurately classified.

It is a block diagram showing roughly composition of an article classification system. FIG. 9 is a diagram provided for describing pre-processing of captured image data. FIG. 4 is a diagram provided for description of data stored in a storage unit. FIG. 4 is a diagram illustrating a part of data stored in a storage unit. FIG. 7 is a diagram provided for describing calculation of a similarity in the first embodiment. It is a flowchart which shows the procedure of a classification process. FIG. 14 is a diagram provided for describing calculation of a similarity in the second embodiment. It is a figure offered for explanation of the classification of the goods in a third embodiment.

  Hereinafter, an embodiment of an article classification device, an article classification system, an article classification method, and a program according to the present disclosure will be described with reference to the drawings. The embodiments described below do not unduly limit the contents of the present disclosure described in the claims. In addition, all of the configurations described in the following embodiments are not necessarily essential components of the present disclosure.

<First embodiment>
(Constitution)
As illustrated in FIG. 1, an article classification system 1 according to a first embodiment of the present disclosure is for classifying a plurality of different types of mixed articles by type, and includes an imaging unit 10 and an article classification device. 20 and an output unit 30, and are connected to each other via a network NW such as a LAN (Local Area Network). At this time, the LAN may be wireless or wired. Further, the imaging unit 10, the article classification device 20, and the output unit 30 may be integrally configured.

  The imaging unit 10 captures captured image data of an article to be classified (hereinafter, appropriately referred to as an object), and includes, for example, a digital camera, a camera in a mobile terminal such as a smartphone, or the like. Preferably, the imaging unit 10 has a memory (not shown) for storing the captured image data. The image data captured by the image capturing unit 10 may be transmitted together with the image capturing to the article classification device 20 or the output unit 30 via a network, or may be selectively or temporarily recorded in a memory. All may be sent. The memory may be a recording medium such as various removable memory cards, or may be a storage medium such as a built-in HDD. In addition, it is preferable that the image data captured by the imaging unit 10 is imaged from a plurality of different viewpoints such as up, down, left, and right in the reference direction with each article centering on a predetermined reference direction.

  The imaging unit 10 includes a table (not shown) on which an object to be imaged is placed. A plurality of cameras are arranged on the table, and the object is viewed from a plurality of different viewpoints such as up, down, left, and right in the above-described reference direction. An object may be imaged.

  The article classification device 20 is roughly configured to include a captured image acquisition unit 21, a storage unit 22, a similarity calculation unit 23, and a classification unit 24.

  The captured image acquisition unit 21 acquires captured image data of an object captured by the imaging unit 10 via the network NW. The acquired captured image data is subjected to, for example, a control unit (not shown) or the like, as shown in FIG. If the value is higher than the value, the image is converted to white, and if the value is lower than the value, a binary conversion process is performed. Then, the captured image data after the binarization processing is transmitted to the similarity calculating unit 23, and is used for calculating the similarity with the two-dimensional image data described later.

  The storage unit 22 is configured by an HDD, an SSD, or the like, and functions as a database in which two-dimensional image data from a plurality of different viewpoints is stored in advance for each type of article to be classified. Specifically, as shown in FIG. 3, for each type of article (for example, A type, B type, C type...), Three-dimensional image data and a plurality of different types corresponding to the three-dimensional image data are provided. It has a table composed of two-dimensional image data projected from a viewpoint and specification data including information on specifications such as dimensions, weight and aspect ratio data of the article. In this table, two-dimensional image data and specification data corresponding to the three-dimensional image data are associated with each other. Here, the articles are described by type of A to C, but may be classified by the model number of the article, the product name, or the like, and the present invention is not limited thereto. Further, the two-dimensional image data is not limited to being stored in advance, and for example, may be appropriately developed from the three-dimensional image data as needed.

  The two-dimensional image data includes, for each type of article, a file (a.iges, b.iges, c.iges...) Based on three-dimensional image data composed of compatible 3D CAD data of the iges method. It is composed of data files (a.001.bmp..., B.010.bmp..., C.100.bmp...) Generated by conversion. At this time, the two-dimensional image data is generated as image data from a plurality of different viewpoints by rotating by a predetermined rotation angle with respect to two axes orthogonal to the three-dimensional image data and projecting the two-dimensional image onto a two-dimensional plane. You. These two-dimensional image data are preferably binarized as shown in FIG. Note that the two-dimensional image data is not limited to the bitmap method, but may be CAD data having a vector format compatibility such as the DXF method.

  The similarity calculation unit 23 calculates the similarity between the captured image data acquired by the captured image acquisition unit 21 and the two-dimensional image data from a plurality of different viewpoints stored in the storage unit 22. Details of the calculation of the similarity will be described later.

  The classification unit 24 calculates an evaluation value for each type of article based on the similarity calculated by the similarity calculation unit 23, and classifies the type having the higher evaluation value as the type to which the target object should be classified. I do. The details of the calculation of the evaluation value and the classification method will be described later.

  The output unit 30 outputs the result of the classification performed by the article classification device 20 (more specifically, the classification unit 24). For example, the output unit 30 is preferably configured by a display device such as a monitor that displays the classification result by the classification unit 24, or a portable terminal such as a tablet terminal or a smartphone. Thereby, the worker can visually confirm the classification result of the target object by the article classification device 20 as appropriate. The output unit 30 is not limited to the display device, and may be, for example, an interface connected to a drive mechanism for physically sorting articles. With the article classification device 20 provided with the mechanism, it is possible to automatically sort the articles based on the classification result of the classification unit 24 without bothering the operator.

(Classification method)
Next, a method of classifying an object in the article classification device 20 will be described.
Specifically, as shown in FIG. 5, the similarity calculation unit 23 compares all the two-dimensional image data stored in the storage unit 22 with the captured image data obtained by the captured image acquisition unit 21 on a brute force basis. In FIG. 5, the captured image data file Img1 is shown as the captured image data. The two-dimensional image data includes two-dimensional image data files a.001.bmp, a.010.bmp, and a.010.bmp for each type of article (here, type A, type B, type C,...). 100.bmp ..., b.001.bmp, b.010.bmp, b.100.bmp ..., c.001.bmp, c.010.bmp, c.100.bmp ... Notation. Here, in order to calculate an evaluation value described later more accurately, for example, for a data file having the same identifier after “.”, Such as “* .010” (* is one of the alphabets a to c), It is preferable to include two-dimensional data projected from the same angle for each type.

  Here, the similarity calculation unit 23 calculates the similarity of each of the two-dimensional image data to the captured image data. That is, the shape of the binarized captured image data and the shape of the two-dimensional image data are respectively determined as to whether the shapes are similar, and whether characteristic portions such as protrusions, bent portions, concave portions, and openings match. (So-called shape matching), and the result is calculated as a similarity. At this time, since the captured image data and the two-dimensional image data are binarized, comparison between these image data is facilitated. Here, when calculating the similarity, the similarity of the two-dimensional image data to the captured image data is calculated (that is, the similarity to each of the two-dimensional image data is calculated based on the captured image data). However, the present disclosure is not limited to this. When calculating the similarity, the similarity of the captured image data to the two-dimensional image data is calculated (that is, based on the two-dimensional image data, (Similarity is calculated).

  More specifically, the similarity calculation unit 23 uses the shape matching technology to store the captured image data file Img1 of the object acquired by the captured image acquisition unit 21 and the two types of each type stored in the storage unit 22. Dimensional image data files a.001.bmp, a.010.bmp, a.100.bmp ..., b.001.bmp, b.010.bmp, b.100.bmp ..., c.001. The feature points of bmp, c.010.bmp, c.100.bmp... are each vectorized. Then, the cosine distance (cosine similarity) between all these vectors is calculated, and the similarity (0 to 1) is calculated based on the calculated cosine distance (−1 to +1). At this time, the similarity is calculated, for example, by adding 1 to the cosine distance and dividing by 2. Further, as the similarity, the closer to “1”, the higher the similarity (higher relevance), and the closer to “0”, the lower the similarity (lower relevance).

  In FIG. 5, the similarity between the captured image data file Img1 and the A-type two-dimensional image data file a.001.bmp is “0.3” in the calculation result by the similarity calculator 23. Similarly, the similarity with the two-dimensional image data file a.010.bmp is “0.3”, and the similarity with the two-dimensional image data file a.100.bmp is “0.1”. The similarity with the B-type two-dimensional image data file b.001.bmp is “0.9”, the similarity with the two-dimensional image data file b.010.bmp is “0.8”, and the two-dimensional image The similarity with the data file b.100.bmp is “0.5”. Further, the similarity with the C type two-dimensional image data file c.001.bmp is “0.6”, the similarity with the two-dimensional image data file c.010.bmp is “0.6”, and the two-dimensional image The similarity with the data file c.100.bmp is “0.3”.

  The classification unit 24 calculates an evaluation value for each type of article based on the similarity calculated by the similarity calculation unit 23, and classifies a type having a high evaluation value as a type to which the target object should be classified. I do. For example, the classification unit 24 may give points according to a ranking based on the degree of similarity calculated by the similarity calculation unit 23. In the case of the similarity as shown in FIG. 5, the B type two-dimensional image data file b.001.bmp (0.9) has the highest similarity, and then the B type two-dimensional image data file b.010. bmp similarity (0.8), C type two-dimensional image data file c.001.bmp and C type two-dimensional image data file c.010.bmp similarity (0.6) . In accordance with this ranking, for example, points such as “10”, “7”, “5”,... Are assigned in descending order, and the points are totaled for each type of article to calculate an evaluation value. ing. The value obtained by summing the points thus given for each type (type) (in this case, “8” for the A type, “30” for the B type, and “18” for the C type) is used as the evaluation value. It will be calculated.

  Further, the present invention is not limited to the above method, and the classification unit 24 may calculate the evaluation value from the total of the scores calculated based on the similarity for each type of article by the similarity calculation unit 23. At this time, the classification unit 24 may assign points according to the similarity calculated by the similarity calculation unit 23, and calculate the evaluation value by summing the points for each type of article. For example, in the case of the similarity as shown in FIG. 5, a value obtained by multiplying the similarity by 10 is defined as a point, the point of the A-type two-dimensional image data file a.001.bmp is set to “3”, and the two-dimensional image The point of the data file a.010.bmp is “3”, and the point of the two-dimensional image data file a.100.bmp is “1”. The point of the B type two-dimensional image data file b.001.bmp is “9”, the point of the two-dimensional image data file b.010.bmp is “8”, and the point of the two-dimensional image data file b.100.bmp is The point is “5”. Further, the point of the C type two-dimensional image data file c.001.bmp is “6”, the point of the two-dimensional image data file c.010.bmp is “6”, and the point of the two-dimensional image data file c.100.bmp is The point is “3”. Then, the points thus given may be totaled for each type (type), and the total value may be calculated as an evaluation value. Furthermore, as described above, the same score (for example, “1”) is uniformly assigned within a predetermined rank (for example, within the top 10) in the ranking based on the degree of similarity calculated by the similarity calculation unit 23. )). Then, the points thus given may be totaled for each type (type), and the total value may be calculated as an evaluation value. In addition, for example, an average value of similarities for each type within a predetermined order of ranking (for example, within the top 10) based on the degree of similarity calculated by the similarity calculating unit 23 is calculated as an evaluation value. You may do so.

  In this way, the classification unit 24 classifies a type having a high evaluation value as a type of the target object based on the calculated evaluation value. Therefore, in the case shown in FIG. 5, the classification unit 24 classifies the type B as the type of the target object, which is the subject of the captured image data file Img1, as the type having the highest similarity and the highest relevance.

(Processing method)
Next, a classification procedure of the target object in the article classification device 20 will be described with reference to a flowchart shown in FIG.

  In step S10, the imaging unit 10 captures an image of an article (target) to be classified. In step S11, the captured image acquisition unit 21 of the article classification device 20 acquires captured image data of the target imaged by the imaging unit 10 (captured image acquisition step). Then, the acquired captured image data is binarized by the control unit or the like of the article classification device 20 and sent to the similarity calculation unit 23.

  In step S <b> 12, the similarity calculation unit 23 compares the two-dimensional image data from a plurality of different viewpoints for each type of article stored in the storage unit 22 in advance with the captured image data acquired by the captured image acquisition unit 21. The similarity is calculated (similarity calculation step).

  Here, as illustrated in FIG. 5, the similarity calculation unit 23 compares all the two-dimensional image data stored in the storage unit 22 with the captured image data obtained by the captured image acquisition unit 21 on a brute force basis. At this time, the captured image data and the two-dimensional image data are each binarized, and the result of shape matching of these is calculated as the similarity. Specifically, the similarity calculation unit 23 vectorizes the feature points of the captured image data of the object acquired by the captured image acquisition unit 21 and all the two-dimensional image data stored in the storage unit 22, respectively. Compute the cosine distance between all vectors. Then, the similarity (0 to 1) is calculated based on the calculated cosine distance (−1 to +1). As the similarity, the closer to “1”, the higher the similarity (higher relevance), and the closer to “0”, the lower the similarity (lower relevance).

  In step S13, the classification unit 24 ranks each two-dimensional image data based on the similarity calculated by the similarity calculation unit 23. That is, the closer the similarity is to "1", the higher the ranking, and the closer the similarity is to "0", the lower the ranking.

  In step S14, the classification unit 24 gives points according to the ranking in step S13. That is, points such as “10”, “7”, and “5” are given in order from the top of the ranking.

  In step S15, the classification unit 24 calculates the evaluation value by summing the points given in step S14 for each type of article. Then, in step S16, the classification unit 24 classifies, based on the evaluation value calculated in step S15, the type having the higher evaluation value as the type of the target object. For example, in the case of the captured image data file Img1 as shown in FIG. 5, the classification unit 24 classifies the type B as the type having the highest similarity and the high relevance.

  The classification processing method of the present embodiment is not limited to this. For example, the ranking is not performed in step S13, and in step S14, points are given according to the similarity calculated by the similarity calculator 23. You may do so. In addition, the process directly proceeds from step S12 to step S15 without giving the points in step S14, and the similarity calculating unit 23 calculates the evaluation value from the sum of the scores calculated based on the similarity for each type of article. You may make it.

(effect)
As described above, in the article classification system 1 of the present embodiment, the similarity between the captured image data of the target object and all the two-dimensional image data in the storage unit 22 is calculated, and based on the calculated similarity. , And lineking the similarity. Then, an object having a high total of points to be awarded according to the ranking is classified as a type to which the object is to be classified. Thus, a plurality of types of mixed articles can be easily and accurately classified.

  As shown in FIG. 3, the storage unit 22 stores a specification data file a as specification data including at least one of weight data, length data, and aspect ratio data based on the specification of the article. .001.spc, b.001.spc, c.001.spc... Are stored.

  Here, for example, a weighing unit may be provided on a base provided in the imaging unit 10 and the weight may be measured by placing an article on the weighing unit. Further, a scale for measuring the dimensions may be provided on the table so that the dimensions can be grasped when the articles are placed. In this case, in addition to the similarity calculated by the similarity calculation unit 23, the classification unit 24 performs comparison using weight data, length data, aspect ratio data, or the like based on the specification data in the storage unit 22. By performing the classification by the classification, the type of the target object can be classified more accurately, and the accuracy and reliability of the classification can be further improved. Further, in the present embodiment, the similarity is calculated based on the cosine distance, but another method may be used. In addition, as the weight data, length data, and aspect ratio data stored in the storage unit 22, data based on the specification may be used, or a measurement result measured separately may be used.

<Second embodiment>
In the first embodiment described above, the case where the captured image data file Img1 is applied as the captured image data and the similarity of the two-dimensional image data to the captured image data file Img1 is calculated has been described, but the present disclosure is not limited thereto. . For example, as shown in FIG. 7 in which parts corresponding to those in FIG. 5 are denoted by the same reference numerals, in addition to a captured image data file Img1 as captured image data of an object, captured image data projected from a different viewpoint of the object. The file Img2 may also be applied, and the similarity of the two-dimensional image data to these may be calculated.

  That is, the captured image acquisition unit 21 acquires captured image data (captured image data file Img1 and captured image data file Img2) from a plurality of different viewpoints for each object, and the similarity calculation unit 23 Is calculated for each of the captured image data from a plurality of different viewpoints. Then, the classification unit 24 may classify the target object by type based on each similarity calculated by the similarity calculation unit 23.

  In this case, for example, as shown in FIG. 7, the similarity calculation unit 23 stores the two captured image data (the captured image data file Img1 and the captured image data file Img2) by the captured image acquisition unit 21 in the storage unit 22 respectively. All the two-dimensional image data stored in are compared on a brute force basis. Here, for convenience, a case will be described where captured image data from two different viewpoints in one object is used. However, the present disclosure is not limited to this, and one object is projected from three or more different viewpoints. The captured image data obtained may be used.

  Here, the similarity calculation unit 23 calculates the similarity of each of the two-dimensional image data with respect to the two pieces of captured image data. For example, in FIG. 7, the calculation result by the similarity calculation unit 23 is that the similarity between the captured image data file Img1 and the A-type two-dimensional image data file a.001.bmp is “0.2”, and the captured image data file The similarity between Img2 and the A-type two-dimensional image data file a.001.bmp is “0.3”. Similarly, the similarity between the captured image data file Img1 and the two-dimensional image data file a.010.bmp is “0.3”, and the similarity between the captured image data file Img2 and the two-dimensional image data file a.010.bmp Is "0.1". Furthermore, the similarity between the captured image data file Img1 and the two-dimensional image data file a.100.bmp is “0.1”, and the similarity between the captured image data file Img2 and the two-dimensional image data file a.100.bmp is It is "0.2".

  The similarity between the captured image data file Img1 and the B type two-dimensional image data file b.001.bmp is “0.9”, and the similarity between the captured image data file Img2 and the two-dimensional image data file b.001.bmp is The similarity is “0.9”. Similarly, the similarity between the captured image data file Img1 and the two-dimensional image data file b.010.bmp is “0.8”, and the similarity between the captured image data file Img2 and the two-dimensional image data file b.010.bmp Is "0.8". Further, the similarity between the captured image data file Img1 and the two-dimensional image data file b.100.bmp is “0.6”, and the similarity between the captured image data file Img2 and the two-dimensional image data file b.100.bmp is It is "0.8".

  Further, the similarity between the captured image data file Img1 and the C type two-dimensional image data file c.001.bmp is “0.5”, and the captured image data file Img2 and the C type two-dimensional image data file c.001.b. The similarity with bmp is “0.6”. Similarly, the similarity between the captured image data file Img1 and the two-dimensional image data file c.010.bmp is “0.4”, and the similarity between the captured image data file Img2 and the two-dimensional image data file c.010.bmp. Is "0.2". Furthermore, the similarity between the captured image data file Img1 and the two-dimensional image data file c.100.bmp is “0.3”, and the similarity between the captured image data file Img2 and the two-dimensional image data file c.100.bmp is It is "0.3".

  Then, the classification unit 24 assigns points according to the value obtained by adding the similarities of the two pieces of captured image data calculated by the similarity calculation unit 23, and evaluates the points by summing the points for each type of article. Calculate the value. That is, the points are higher as the similarity is closer to “2”, which is the sum of “1” and “1”, and the points are lower as the similarity is closer to “0”. The method for calculating the evaluation value is not limited to this, and can be calculated by the various methods described in the first embodiment.

  For example, in FIG. 7, the point of the A type two-dimensional image data file a.001.bmp is “3”, the point of the two-dimensional image data file a.010.bmp is “2”, and the two-dimensional image data file a. The point of .100.bmp is "1". The point of the B type two-dimensional image data file b.001.bmp is “9”, the point of the two-dimensional image data file b.010.bmp is “8”, and the point of the two-dimensional image data file b.100.bmp is “9”. The point is “7”. Furthermore, the point of the C type two-dimensional image data file c.001.bmp is “5”, the point of the two-dimensional image data file c.010.bmp is “4”, and the point of the two-dimensional image data file c.100.bmp is The point is “4”. Then, the value obtained by summing the points thus given for each type (type) (in this case, “7” for the A type, “35” for the B type, and “17” for the C type) is used as the evaluation value. It will be calculated.

  In this way, the classification unit 24 classifies a type having a high evaluation value as a type of the target object based on the calculated evaluation value. Therefore, in the case illustrated in FIG. 7, the classification unit 24 determines that the type B has the highest similarity as the type of the object as the subject of the two captured image data files Img1 and Img2 projected from different viewpoints, Is classified as a high type.

  For example, as shown in FIG. 8, in an E-type article having a triangular pyramid shape and an F-type article having a cylindrical shape, captured image data projected from a certain viewpoint (for example, the bottom side) has the same shape. (In this case, a circular shape). However, in the E-type article and the F-type article, in the captured image data projected from a viewpoint different from the above-mentioned certain viewpoint, the former has a triangular shape, the latter has a rectangular shape, and is completely different. It turns out that it is an article.

  As described above, in the article classification system 1 of the second embodiment, the similarity is calculated for one target object using the captured image data projected from a plurality of (here, two) different viewpoints. As compared with the case where the number of the captured image data is one, more accurate similarity can be calculated, and points and evaluation values can be calculated based on the similarity. Therefore, the accuracy and reliability of classification can be further improved. it can.

<Third embodiment>
Note that, in the first and second embodiments described above, the case where the imaging unit 10 includes a camera or the like in a portable terminal such as a digital camera or a smartphone has been described, but the present disclosure is not limited thereto.

  For example, a wearable device worn by an operator may be used as the camera of the imaging unit 10. In this case, the imaging unit 10 can sequentially image the object to be imaged from a plurality of different arbitrary viewpoints by enabling the worker to appropriately image while observing the object from an arbitrary position.

  Therefore, the similarity calculation unit 23 calculates the similarity every time the captured image acquisition unit 21 acquires the captured image data projected from a plurality of different arbitrary viewpoints. Each time the similarity is calculated by 23, an evaluation value can be calculated based on the calculated similarity and the classification result can be updated. That is, in this case, the classification result by the classification unit 24 can be updated by repeating the classification processing routine illustrated in FIG. 6 each time the same target is imaged by the imaging unit 10 from a plurality of different arbitrary viewpoints. . Therefore, the degree of similarity to the target object, and further, the accuracy and reliability of the classification can be further improved.

<Fourth embodiment>
Further, in the above-described first to third embodiments, the case where the shape matching method is used in the calculation of the similarity has been described, but the present disclosure is not limited to this. For example, information on the weight of the target object may be added to the shape matching method in the above-described similarity calculation. Specifically, the weight of the object to be placed can be measured on the base of the imaging unit 10, and it is determined whether or not the measured weight matches the weight information based on the specification data in the storage unit 22. , The similarity may be calculated, and the object may be classified. Thus, even if the articles appear to have the same shape in appearance, articles having different internal shapes and structures (for example, hollow articles and articles having different thicknesses) based on the information on the weight of the target object. ) Can be accurately classified. In addition, as the weight data stored in the storage unit 22, data based on a specification may be used, or a measurement result measured separately may be used.

  Further, for example, information on the dimensions of the target object may be added to the shape matching method in the above-described similarity calculation. Specifically, a scale for measuring the size of the object to be mounted is provided on the table of the imaging unit 10 so that the absolute size of the object mounted on the table can be measured. Then, the similarity may be calculated including the determination as to whether or not the measured dimension matches the length information based on the specification data in the storage unit 22, and the target object may be classified.

  Further, for example, information on the aspect ratio of the target object may be added to the shape matching method in the above-described similarity calculation. Specifically, a scale for measuring the size of the object to be mounted is provided on the table of the imaging unit 10, and the aspect ratio (ratio, aspect ratio, etc.) of the object mounted on the table can be calculated. And Then, the similarity may be calculated including determination of whether or not the calculated aspect ratio matches information on the aspect ratio based on the specification data in the storage unit 22, and the target object may be classified. . As described above, the scale for measuring the size of the object is provided on the base of the imaging unit 10, and the length information based on the specification data and the information of the aspect ratio are used to obtain the same appearance. Even if the article looks like a shape, articles having different dimensions can be correctly classified based on information on the length of the target object. In addition, by using the information of the aspect ratio based on the specification data, the absolute size of the object (sintered body such as ceramic or powder metallurgy or molded part) with the object having isotropic shrinkage degree can be obtained. Can be accurately classified even if the value slightly changes. In addition, as the length data and the aspect ratio data stored in the storage unit 22, data based on a specification may be used, or a measurement result measured separately may be used.

<Program>
Note that, in the present embodiment, mainly based on the similarity, in consideration of the relevance between the article to be classified and the type, an article that classifies a type having a high similarity to the object is classified. Although the configuration and operation of the classification system 1 have been described, the present invention is not limited to this, but includes two-dimensional image data from a plurality of different viewpoints for each type of article stored in the storage unit 11 in advance. Based on the similarity with the image data captured by the image processing apparatus 10, in consideration of the relationship between the object and the type of the article, a program for calculating which type has a high similarity to the object is It may be configured.

  Further, a program for realizing each function constituting the article classification system 1 is recorded on a computer-readable recording medium, and the program recorded on this recording medium is read by a computer system and executed. May be.

  Specifically, the program includes a captured image acquisition step of acquiring captured image data of an article to be classified, and two-dimensional image data from a plurality of different viewpoints for each type of article stored in the storage unit 22 in advance. And a similarity calculating step of calculating a similarity with the captured image data, and a classifying step of classifying the articles by type based on the similarity by a computer.

  Further, the “computer system” here includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system.

  Further, a “computer-readable recording medium” is a dynamic medium in a short period of time, such as a communication line for transmitting a program via a network such as a wired or wireless LAN or the Internet, or a communication line such as a telephone line. And a program that holds a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client in that case. Further, the above-mentioned program may be for realizing a part of the functions described above, or may be for realizing the above-mentioned functions in combination with a program already recorded in a computer system. .

DESCRIPTION OF SYMBOLS 1 ... Article classification system 10 ... Imaging part 20 ... Article classification apparatus 21 ... Captured image acquisition part 22 ... Storage part 23 ... Similarity calculation part 24 ... Classification part NW ... Network

Claims (13)

An article classification device for classifying a plurality of types of articles by type,
A captured image acquisition unit that acquires captured image data of an article to be classified;
A storage unit in which two-dimensional image data from a plurality of different viewpoints are stored for each type of the article,
A similarity calculation unit that calculates respective similarities between the captured image data and the two-dimensional image data,
Said based-out similarity, and calculates an evaluation value for each type of the article, the classification unit that classifies with the evaluation value is high type is the type of article to be the classification target,
An article classification device comprising: The two-dimensional image data stored in the storage unit includes image data projected from a plurality of different viewpoints converted from the three-dimensional image data,
The article classification device according to claim 1, wherein: The classification unit is configured to add a point according to the similarity calculated by the similarity calculation unit, and calculate the evaluation value by summing the points for each type of the article.
The article classification device according to claim 1 or 2 , wherein: The classifying unit assigns the points according to a ranking based on the height of the similarity calculated by the similarity calculating unit,
The article classification device according to claim 3 , wherein: The captured image acquiring unit acquires captured image data from a plurality of different viewpoints for each article to be classified,
The similarity calculation unit calculates the similarity between the captured image data obtained by the plurality of different viewpoints acquired by the captured image acquisition unit and the two-dimensional image data,
The classification unit classifies the articles by type based on the similarity calculated by the similarity calculation unit,
The article classification device according to any one of claims 1 to 4 , wherein: The classification unit calculates an evaluation value for each type of the article based on the similarity calculated by the similarity calculation unit, and classifies the type having the higher evaluation value as the type of the article to be classified. Do
The article classification device according to claim 5 , wherein: An article classification device for classifying a plurality of types of articles by type,
A captured image acquisition unit that acquires captured image data of an article to be classified;
A storage unit in which two-dimensional image data from a plurality of different viewpoints are stored for each type of the article,
A similarity calculation unit that calculates the similarity between the captured image data and the two-dimensional image data each time the captured image acquisition unit acquires the captured image data,
A classification unit that classifies the articles by type based on the similarity and updates a classification result based on the similarity calculated by the similarity calculation unit ;
An article classification device comprising: An article classification device for classifying a plurality of types of articles by type,
A captured image acquisition unit that acquires captured image data of an article to be classified;
A storage unit in which at least one piece of information of two-dimensional image data and the weight data of the article, length data, and aspect ratio data is stored by a plurality of different viewpoints for each type of the article,
A similarity calculating unit that calculates a similarity between the captured image data and the two-dimensional image data,
A classification unit that classifies the articles by type in consideration of the similarity and the weight data, the length data, or the aspect ratio data of the articles,
An article classification device comprising: The similarity calculation unit, when calculating the similarity, calculates the similarity of the two-dimensional image data to the captured image data,
The article classification device according to any one of claims 1 to 8 , wherein: The similarity calculation unit calculates the similarity of the captured image data with respect to the two-dimensional image data when calculating the similarity,
The article classification device according to any one of claims 1 to 8 , wherein: An article classification system for classifying a plurality of types of articles by type,
An imaging unit that captures captured image data of an article to be classified;
A storage unit in which two-dimensional image data from a plurality of different viewpoints are stored for each type of the article,
A similarity calculator that calculates respective similarities between the captured image data and the two-dimensional image data,
Said based-out similarity, and calculates an evaluation value for each type of the article, the classification unit that classifies with the evaluation value is high type is the type of article to be the classification target,
An output unit that outputs a result of the classification by the classification unit;
An article classification system comprising: An article classification method for classifying a plurality of types of articles by type,
A captured image obtaining step of obtaining captured image data of an article to be classified;
Two-dimensional image data with a plurality of different viewpoints for each type of the article stored in the storage unit, and a similarity calculation step of calculating each similarity with the captured image data,
-Out based on the similarity, and calculates an evaluation value for each type of the article, a classification step of classifying with the evaluation value is high type is the type of article to be the classification target,
An article classification method comprising: Computer
A captured image acquisition unit that acquires captured image data of an article to be classified;
Storage means for storing two-dimensional image data from a plurality of different viewpoints for each type of the article,
Similarity calculating means for calculating respective similarities between the captured image data and the two-dimensional image data,
Wherein based-out similarity, and calculates an evaluation value for each type of the article, classifying means for classifying with the evaluation value is high type is the type of article to be the classification target,
Program to function as