JP2018120418A - Threshold setting support program, threshold setting support device and threshold setting support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To facilitate setting of a threshold for distinguishing an object.SOLUTION: A threshold setting support program makes a computer execute processing of extracting a plurality of types of feature quantities indicating a feature of an object from a plurality of images including an object, classifying the plurality of images into a plurality of clusters based on a first feature quantity to which the threshold for distinguishing the object is set out of the plurality of types of feature quantities, extracting images having a common feature between the plurality of clusters out of the images classified into the plurality of clusters based on a second feature amount out of the plurality of feature quantities, selecting at least one of groups of images having the common feature as a display group, and displaying images belonging to the display group side by side based on a value indicated by the first feature amount.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a threshold setting support program, a threshold setting support device, and a threshold setting support method that support setting of a threshold when sorting an object.

  An inspection method for automating the appearance inspection of an object to be inspected has been proposed (for example, see Patent Document 1). In this type of inspection method, a feature amount extracted from an image obtained by imaging an object to be inspected is compared with a preset threshold value, and it is determined whether the product is good or defective based on the comparison result.

  The threshold value is set based on, for example, a distribution of feature amounts extracted from a plurality of images that are identified in advance as a non-defective image in which a non-defective product is imaged or a defective product image in which a defective product is imaged (for example, Patent Documents). 2). Alternatively, the threshold is set based on the distribution of feature amounts extracted from a plurality of non-defective images (see, for example, Patent Document 3).

JP 2011-232302 A JP 2006-220648 A JP2015-94589A

  An operator such as an inspector who sets a threshold for dividing an object to be inspected into a non-defective product and a defective product, for example, visually observes a plurality of images in which the object to be inspected is imaged and divides the image into a good product image and a defective product image . Then, the operator labels the non-defective image with identification information indicating the non-defective product, and labels the defective image with identification information indicating the defective product. Increasing the number of images used for setting the threshold improves the reliability of the set threshold, but also increases the number of images to be labeled, which increases the burden on the operator. Even when only non-defective images are used for setting the threshold, work such as excluding defective products from a plurality of images obtained by imaging the object to be inspected occurs, so the number of images used for setting the threshold As the number of operators increases, the burden on the operator increases.

  In one aspect, an object of the present invention is to facilitate setting of a threshold value for sorting an object.

  According to one aspect of the program, a plurality of types of feature quantities representing features of the target object are extracted from a plurality of images including the target object, and a threshold value for setting the target object among the plurality of types of feature quantities is set. Based on the first feature amount, a plurality of images are classified into a plurality of clusters, and among the images classified into the plurality of clusters, an image having a common feature among the plurality of clusters is classified into a plurality of types of feature amounts. Are extracted based on the second feature amount, and at least one of the groups of images having the common feature is selected as a display group, and an image belonging to the display group is set to a value indicated by the first feature amount. There is provided a threshold setting support program for causing a computer to execute a process of displaying images side by side.

  In one embodiment, the threshold setting support apparatus extracts an object from among a plurality of types of feature amounts, an extraction unit that extracts a plurality of types of feature amounts representing features of the target object from a plurality of images including the target object. A classification unit that classifies a plurality of images into a plurality of clusters based on a first feature amount for which a threshold for classification is set, and a feature that is common among the plurality of clusters among the images classified into the plurality of clusters And a selection unit that selects at least one of the groups of images having a common feature as a display group, and a display group. And a display unit that displays the images to which the images belong, side by side based on the value indicated by the first feature amount.

  According to one aspect of the method, a computer extracts a plurality of types of feature quantities representing features of a target object from a plurality of images including the target object, and separates the target object from the plurality of types of feature quantities. Based on the first feature amount for which a threshold is set, a plurality of images are classified into a plurality of clusters, and among images classified into a plurality of clusters, a plurality of types of images having features common to the plurality of clusters are selected. Are extracted based on the second feature amount of the feature amounts, and at least one of the groups of images having the common feature is selected as a display group, and images belonging to the display group are selected as the first feature amount. There is provided a threshold setting support method for displaying side by side based on the indicated value.

  In one aspect, the present invention can facilitate the setting of a threshold for sorting objects.

It is a figure which shows one Embodiment of a threshold value setting assistance apparatus, a threshold value setting assistance method, and a threshold value setting assistance program. It is a figure which shows another embodiment of a threshold value setting assistance apparatus. It is a figure which shows an example of the feature-value database shown in FIG. It is a figure which shows an example of operation | movement of the classification | category part shown in FIG. It is a figure which shows an example of operation | movement of the selection part shown in FIG. It is a figure which shows an example of operation | movement of the display part and threshold value setting part which were shown in FIG. It is a figure which shows another example of operation | movement of the display part shown in FIG. It is a figure which shows an example of operation | movement of the threshold value setting assistance apparatus shown in FIG. It is a figure which shows an example of the threshold value setting assistance process shown in FIG. It is a figure which shows another embodiment of a threshold value setting assistance apparatus. It is a figure which shows an example of operation | movement of the selection part shown in FIG. It is a figure which shows an example of the hardware which performs a threshold value setting assistance program.

  Hereinafter, embodiments will be described with reference to the drawings. Hereinafter, the image itself and the image data for displaying the image are referred to as an image without any particular distinction.

  FIG. 1 shows an embodiment of a threshold setting support device, a threshold setting support method, and a threshold setting support program. The threshold setting support apparatus 10 illustrated in FIG. 1 is included in the sorting system 1 that, for example, analyzes an image IMG obtained by capturing an object to be inspected by the image capturing apparatus 80 and classifies the object. The sorting system 1 includes an imaging device 80 and a display device 90 that are communicably connected to the threshold setting support device 10. For example, in the sorting system 1 that separates objects such as food into good and defective products, it is determined whether the value indicated by the feature value indicating the characteristics of the target object extracted from the image IMG indicates good products or defective products. A threshold for sorting is set in advance. The imaging device 80 is a digital camera or the like, for example, and the display device 90 is a display or the like that displays an image IMG that is referred to when a threshold is set in the sorting system 1.

  For example, when an operator such as an inspector sets a threshold value in the sorting system 1, the threshold value setting support device 10 executes a threshold value setting support process for facilitating the setting of the threshold value. For example, the threshold setting support apparatus 10 includes a processor such as a CPU (Central Processing Unit), and the processor stores a threshold setting support program for realizing the functions of the extraction unit 20, the classification unit 30, the selection unit 40, and the display unit 50. By executing this, threshold setting support processing is executed. Note that the threshold setting support device 10 may be realized only by hardware. For example, the extraction unit 20, the classification unit 30, the selection unit 40, and the display unit 50 may be realized only by hardware. An example of the outline of the threshold setting support method is shown in parentheses in FIG.

  The threshold setting support device 10 includes an extraction unit 20, a classification unit 30, a selection unit 40, and a display unit 50. The extraction unit 20 receives, for example, a plurality of images IMG (IMG1-IMG12) obtained by imaging the target object from the imaging device 80, and a plurality of types of feature amounts representing the characteristics of the target object from the plurality of images IMG including the target object. To extract. In the example illustrated in FIG. 1, the extraction unit 20 extracts, for each of a plurality of images IMG, shape characteristics such as circularity and area, luminance, and the like as the plurality of types of feature amounts representing the characteristics of the object. Then, the extraction unit 20 stores information indicating a plurality of types of feature amounts extracted from the image IMG, the image IMG, and the like in a storage device such as a memory (not shown).

  The classification unit 30 converts the plurality of images IMG into a plurality of clusters based on a first feature amount in which a threshold value for classifying an object is set among a plurality of types of feature amounts extracted by the extraction unit 20. Classify. For example, when the first feature amount is circularity, the classification unit 30 classifies the plurality of images IMG into a plurality of clusters based on the circularity among the circularity, area, and luminance of the object. In the example illustrated in FIG. 1, the classification unit 30 divides the images IMG1 to IMG12 into a cluster with a circularity of “0.85”, a cluster with a circularity of “0.90”, and a circularity of “0.95”. And four clusters, a cluster having a circularity of “1”.

  For example, the classification unit 30 classifies the images IMG1, IMG2, and IMG3 into clusters with a circularity “0.85”, and classifies the images IMG4 and IMG5 into clusters with a circularity “0.90”. The classifying unit 30 classifies the images IMG6, IMG7, and IMG8 into clusters with a circularity of “0.95”, and classifies the images IMG9, IMG10, IMG11, and IMG12 into clusters with a circularity of “1”.

  For example, the selection unit 40 divides the image IMG into six groups G (G1-G6) based on the area other than the circularity and the luminance among the circularity, area, and luminance of the object. In this case, the area and the luminance are examples of the second feature amount among the plurality of types of feature amounts. In the example shown in FIG. 1, when the area of the object extracted from the image IMG is less than the predetermined first area, the area is divided into small areas, and the second predetermined value is equal to or larger than the first area. When it is less than the area, it is classified into a medium area, and when it is greater than or equal to the second area, it is classified into a large area. Further, the luminance of the object extracted from the image IMG is classified as high luminance when it is equal to or higher than a predetermined luminance value, and is classified as low luminance when it is lower than the predetermined luminance value. The group G1-G6 corresponds to one of a combination of three sections of area and two sections of luminance. The number of area divisions and the number of luminance divisions are not limited to the example shown in FIG.

  The image IMG belongs to one of six groups G1-G6 classified according to area and luminance. The groups G1 and G3 include one cluster including the image IMG, the groups G2, G4, and G6 include two clusters including the image IMG, and the group G5 includes four clusters including the image IMG.

  For example, the group G2 is a group G of images IMG having features in which the area and luminance of the object extracted from the image IMG are a small area and a low luminance, respectively, and include images IMG3 and IMG11. That is, the images IMG3 and IMG11 are images IMG having a common feature (small area and low luminance) among a plurality of clusters. In other words, the feature exhibiting low luminance with a small area is a feature common to the images IMG3 and IMG11 classified into different clusters.

  Further, for example, the group G5 is a group G of images IMG having a feature that the area and luminance of the object extracted from the image IMG are large and high, respectively, and include images IMG1, IMG4, IMG8, and IMG9. . That is, the images IMG1, IMG4, IMG8, and IMG9 belonging to different clusters are images IMG having a common characteristic (large area and high brightness) among a plurality of clusters. In other words, the feature exhibiting high luminance in a large area is a feature common to the images IMG1, IMG4, IMG8, and IMG9 classified into different clusters.

  The selection unit 40 extracts groups G2, G4, G5, and G6 as a group G of images IMG having features common to a plurality of clusters. Then, the selection unit 40 selects the group G5 having the largest number of clusters including the image IMG among the groups G2, G4, G5, and G6 as the display group G5.

  As described above, the selection unit 40 selects, based on the second feature amount among the plurality of types of feature amounts, the image IMG having features common to the plurality of clusters among the image IMGs classified into the plurality of clusters. Extract. Then, the selection unit 40 selects at least one of the groups I of the image IMG having features common to the plurality of clusters as the display group G.

  For example, the display unit 50 displays the images IMG1, IMG4, IMG8, and IMG9 belonging to the display group G5 selected by the selection unit 40 on the display device 90 in the order of the circularity values. As described above, the display unit 50 displays the images IMG belonging to the display group G side by side based on the value indicating the first feature amount.

  The images IMG1, IMG4, IMG8, and IMG9 displayed on the display device 90 are divided into, for example, a non-defective image in which a non-defective product is imaged and a defective product image in which a non-defective product is imaged according to an instruction from an operator. Hereinafter, the same reference numerals as the images are used for the non-defective images and the defective images.

  For example, the operator visually checks the images IMG1, IMG4, IMG8, and IMG9 displayed on the display device 90 among the images IMG1 to IMG12, and the non-defective product is obtained from the images IMG1, IMG4, IMG8, and IMG9 displayed on the display device 90. Select the image IMG. Then, the threshold setting support device 10 or the like sets the threshold value of the circularity based on the circularity range of the non-defective image IMG among the images IMG1, IMG4, IMG8, and IMG9. For example, an image IMG4 included in a cluster with a circularity “0.90”, an image IMG8 included in a cluster with a circularity “0.95”, and an image IMG9 included in a cluster with a circularity “1” are used as non-defective images IMG. When selected, the circularity threshold is set to “0.90”. In this case, an object having a circularity of less than “0.90” is classified as a defective product, and an object having a circularity of “0.90” or more and “1” or less is classified as a non-defective product.

  As described above, the threshold setting support apparatus 10 can reduce the number of images IMG visually checked by the operator and separate the objects as compared with the case where all the images IMG of the images IMG1 to IMG12 are visually checked. The threshold value can be easily set. Further, the plurality of images IMG are displayed side by side in the order of the first feature value (for example, circularity) for which a threshold is set. For this reason, compared with the case where a plurality of images IMG are arranged and displayed in random order, the comparison of the first feature amount can be facilitated.

  When a threshold value is set for each of a plurality of types of feature values, the threshold value setting support apparatus 10 changes the first feature value and repeats the classification of the image IMG, the selection of the display group G, and the like. In this case, the threshold setting support device 10 determines whether the image IMG that has already been determined to be good or defective in the threshold setting support processing for other types of feature values among the images IMG belonging to each display group G is good. Information indicating whether the product is defective may be displayed together with the image IMG.

  The configuration and operation of the threshold setting support apparatus 10 are not limited to the example shown in FIG. For example, the number of types of feature amounts representing the characteristics of the object may be two, or four or more. Also, the type of feature amount may be other than circularity, area, and luminance. Alternatively, the selection unit 40 sets, as the second feature, the image IMG classified based on the first feature amount by using one type other than the first feature amount among the plurality of types of feature amounts as the second feature amount. You may group based on quantity. For example, the selection unit 40 may divide the images IMG classified into a plurality of clusters into three groups G according to only the area, regardless of the luminance of the object. In this case, the images IMG1, IMG4, IMG5, IMG8, IMG9, and IMG12 belong to the large area group G, and the large area group G is selected as the display group G. Further, the threshold setting support device 10 may be communicably connected to a storage device that stores the image IMG captured by the image capturing device 80 instead of being communicably connected to the image capturing device 80.

  As described above, in the embodiment illustrated in FIG. 1, the threshold value setting support apparatus 10 extracts a plurality of types of feature amounts representing features of a target object from a plurality of images IMG including the target object. Then, the threshold setting support device 10 classifies the plurality of images IMG into a plurality of clusters based on a first feature amount for which a threshold value for classifying an object is set among a plurality of types of feature amounts. Further, the threshold setting support apparatus 10 selects an image IMG having a feature common to the plurality of clusters among the images IMG classified into the plurality of clusters based on the second feature amount among the plurality of types of feature amounts. Extract. Then, the threshold value setting support apparatus 10 selects at least one of the groups I of the image IMG having a common feature among a plurality of clusters as the display group G. Further, the threshold setting support device 10 displays the images IMG belonging to the display group G side by side based on the value indicated by the first feature amount. Thereby, the setting of the threshold value for classifying a target object can be made easy.

  FIG. 2 shows another embodiment of the threshold setting support apparatus. The same or similar elements as those described in FIG. 1 are denoted by the same or similar reference numerals, and detailed description thereof will be omitted. The threshold setting support apparatus 100 illustrated in FIG. 2 is included in, for example, the selection system 1 that analyzes an image IMG obtained by imaging an object with the imaging device 80 and classifies the object. The sorting system 1 illustrated in FIG. 2 includes an imaging device 80, a display device 90, and an input device 92 that are communicably connected to the threshold setting support device 100. The input device 92 is, for example, a mouse or a keyboard.

  For example, the threshold setting support device 100 executes threshold setting support processing for facilitating setting of the threshold when the operator sets the threshold in the sorting system 1. Note that the threshold setting support apparatus 100 may be realized only by hardware, or may be realized by controlling hardware by software. For example, a computer controlled by software such as a threshold setting support program may execute the threshold setting support processing of the threshold setting support device 100.

  The threshold setting support apparatus 100 includes an extraction unit 200, a storage unit 210, a classification unit 300, a selection unit 400, a display unit 500, and a threshold setting unit 600. The storage unit 210 is a storage device such as a memory. For example, the storage unit 210 stores a feature amount database 212 in which feature amounts representing features of an object are registered, an image database 214 in which images IMG are registered, and the like.

  The extraction unit 200 is the same as or similar to the extraction unit 20 shown in FIG. For example, the extraction unit 200 receives a plurality of images IMG obtained by imaging the target object from the imaging device 80, and extracts a plurality of types of feature amounts representing the characteristics of the target object from the plurality of images IMG including the target object. Then, the extracting unit 200 registers a plurality of types of feature amounts extracted from the image IMG in the feature amount database 212 in association with the image IMG. Further, the extraction unit 200 registers the image IMG received from the imaging device 80 in the image database 214.

  A plurality of types of feature amounts extracted from the object by the extraction unit 200 are, for example, shape features, texture features, and the like described in “Mikio Takagi, Yoshihisa Shimoda: New Image Analysis Handbook, University of Tokyo Press, 2004.” Color characteristics and the like, which are determined in advance. The shape features are area, perimeter, circularity, etc., the texture features are luminance histogram average and variance, edge density, etc., and the color features are color histogram average and variance, etc. Note that the type of feature amount extracted from the object is not limited to the above example.

  The classification unit 300 is the same as or similar to the classification unit 30 shown in FIG. For example, the classification unit 300 generates a plurality of images IMG based on a first feature amount for which a threshold for classifying an object is set among a plurality of types of feature amounts registered in the feature amount database 212. Class. Details of the operation of the classification unit 300 will be described with reference to FIG.

  The selection unit 400 is the same as or similar to the selection unit 40 shown in FIG. For example, the selection unit 400 extracts an image IMG having a feature common to the plurality of clusters from the images IMG classified into the plurality of clusters based on the second feature amount among the plurality of types of feature amounts. . Then, the selection unit 400 selects at least one of the groups I of the image IMG having features common to the plurality of clusters as the display group G. Details of the operation of the selection unit 400 will be described with reference to FIG.

  The display unit 500 is the same as or similar to the display unit 50 shown in FIG. For example, the display unit 500 acquires the image IMG belonging to the display group G selected by the selection unit 400 from the image database 214. The display unit 500 displays the image IMG acquired from the image database 214 on the display device 90 side by side based on the value indicating the first feature amount. Details of the operation of the display unit 500 will be described with reference to FIGS.

  The threshold value setting unit 600 receives, for example, information SINF indicating the non-defective image IMG among the plurality of images IMG displayed on the display device 90 from the input device 92. Then, the threshold setting unit 600, for example, based on a value indicating the first feature amount extracted from the non-defective image IMG among the plurality of images IMG displayed on the display device 90, and the like. Set the threshold. For example, the operator visually confirms the plurality of images IMG displayed on the display device 90, and among the plurality of images IMG displayed on the display device 90, the operator uses the input device 92 such as a mouse. select. Then, the input device 92 outputs information SINF indicating the non-defective image IMG selected by the operator to the threshold setting unit 600 or the like. The threshold setting unit 600 sets the first feature amount threshold based on the information SINF received from the input device 92. For example, the threshold setting unit 600 stores the first feature amount threshold in the storage unit 210 or the like.

  The configuration of the threshold setting support apparatus 100 is not limited to the example illustrated in FIG. For example, the image IMG captured by the imaging device 80 may be registered in the image database 214 before a plurality of types of feature amounts representing the characteristics of the object are extracted. In this case, the extraction unit 200 acquires a plurality of images IMG obtained by imaging the target object from the image database 214, and extracts a plurality of types of feature amounts representing the characteristics of the target object from the plurality of images IMG acquired from the image database 214. It may be extracted. Further, the feature amount database 212 may be included in the image database 214. That is, the image database 214 may have items registered in the feature amount database 212 in addition to the image IMG. Further, the threshold setting support device 100 may be communicably connected to an external storage device that stores the image IMG captured by the image capturing device 80 instead of being communicably connected to the image capturing device 80.

  FIG. 3 shows an example of the feature amount database 212 shown in FIG. In the feature amount database 212, a data group DG including data indicating the object identification number, the image file name, and the feature amount is registered for each image IMG from which the feature amount representing the feature of the object is extracted.

  The object identification number is an identification number for identifying the object. For example, the same identification number is assigned to an object of the same product type such as an object for which the same threshold is set. The image file name is the file name of the image IMG used for extracting the feature amount. The feature amount is a feature amount representing the feature of the object extracted from the image IMG indicated by the image file name. In the example illustrated in FIG. 3, the area, perimeter, circularity, edge density, and the like of the object are registered in the feature amount database 212 as feature amounts.

  The items registered in the feature amount database 212 are not limited to the example shown in FIG. For example, when the threshold setting support apparatus 100 has the feature amount database 212 for each target of the same product type, the target object identification number may be omitted from items registered in the feature amount database 212.

  FIG. 4 shows an example of the operation of the classification unit 300 shown in FIG. In FIG. 4, in order to make the characteristics of the object easy to understand, an image IMG corresponding to the data group DG is shown in parentheses shown above the data group DG. As described with reference to FIG. 3, the data group DG includes data indicating image file names, feature amounts, and the like. Further, the image IMG corresponding to the data group DG is an image IMG designated by an image file name included in the data group DG.

  The classification unit 300 acquires a predetermined data group DG (DG20-DG34) from the feature amount database 212. For example, the classification unit 300 acquires, from the feature amount database 212, a data group DG corresponding to a target for which a threshold is to be set among the data groups DG registered in the feature amount database 212. When the feature amount database 212 is provided for each target of the same product type, the classification unit 300 may acquire all data groups DG from the feature amount database 212 corresponding to the target for which a threshold is set. .

  In addition, the classification unit 300 selects a first feature amount for setting a threshold from a plurality of types of feature amounts representing features of the object. The classification unit 300 may select the first feature amount from a plurality of types of feature amounts based on an instruction from the operator, or may select the first feature amount based on a predetermined order. Then, the classification unit 300 clusters the data group DG20 to DG34 acquired from the feature amount database 212 with a value indicating the first feature amount.

  For example, the classification unit 300 selects the circularity as the first feature value for setting the threshold value. Then, the classification unit 300 clusters the data groups DG20 to DG34 acquired from the feature amount database 212 with circularity values. Thus, the data groups DG21, DG22, and DG23 are classified into clusters having a circularity of “0.85”, and the data groups DG27, DG29, and DG33 are classified into clusters having a circularity of “0.90”. The data groups DG24, DG26, DG31, and DG34 are classified into clusters with a circularity of “0.95”, and the data groups DG20, DG25, DG28, DG30, and DG32 are classified into clusters with a circularity of “1”. .

  In this manner, the classification unit 300 selects a plurality of data groups DG based on the first feature amount in which a threshold value for sorting the object is set among the plurality of types of feature amounts representing the feature of the object. Classify into multiple clusters. Thereby, the plurality of images IMG respectively corresponding to the plurality of data groups DG are also classified into a plurality of clusters based on the first feature amount. In other words, the classification unit 300 converts a plurality of images IMG into a plurality of images based on a first feature amount in which a threshold for classifying an object is set among a plurality of types of feature amounts extracted by the extraction unit 200. Classify into clusters. The operation of the classification unit 300 is not limited to the example shown in FIG.

  FIG. 5 shows an example of the operation of the selection unit 400 shown in FIG. In FIG. 5, as in FIG. 4, the image IMG corresponding to the data group DG is illustrated in parentheses shown above the data group DG in order to make the features of the object easy to understand. In FIG. 5, in order to make the drawing easy to see, the reference numerals of the images IMG other than the image IMG20 are omitted. In FIG. 5, four types of circularity, area, luminance, and edge density are set as a plurality of types of feature amounts extracted by the extraction unit 200, the first feature amount is set as circularity, and the second feature amount is set as area and luminance. It will be described as three types of feature amounts of edge density.

  First, the selection unit 400 groups the data group DG (DG20-DG34) classified into a plurality of clusters by the classification unit 300 with common features. For example, the selection unit 400 compares each data indicating the area, brightness, and edge density, which are the second feature amounts, between the clusters, and groups the data groups DG having the same features. Note that by grouping the data groups DG classified into a plurality of clusters, a plurality of images IMG respectively corresponding to the data groups DG classified into the plurality of clusters are also grouped.

  In the example shown in FIG. 5, the area of the object extracted from the image IMG is divided into one of a small area, a medium area, and a large area, as in FIG. 1. In addition, when the luminance of the object extracted from the image IMG is equal to or higher than a predetermined first luminance value, the luminance is classified as high luminance and is equal to or higher than a predetermined second luminance value less than the first luminance value. In the case of, it is classified into medium luminance, and when it is less than the second luminance value, it is classified into low luminance. When the edge density of the object extracted from the image IMG is equal to or higher than the predetermined edge density, the edge density is classified as a high edge density. When the edge density is lower than the predetermined edge density, the edge density is classified as a low edge density. For example, the feature indicated by the second feature amount is represented by a combination of three sections of area, three sections of luminance, and two sections of edge density.

  The groups G10 to G15 correspond to any of combinations of three sections of area, three sections of luminance, and two sections of edge density. Data group DG20-DG34 belongs to one of six groups G10-G15 classified according to area, brightness, and edge density. In FIG. 5, the description of the group G that does not include any of the data groups DG20 to DG34 (for example, the group G of the data group DG having a small area, high brightness, and high edge density) is omitted. . The number of area sections, the number of brightness sections, and the number of edge density sections are not limited to the example shown in FIG.

  The groups G10 and G11 include three clusters including the data group DG, the group G12 includes one cluster including the data group DG, the group G13 includes four clusters including the data group DG, the group G14, G15 includes two clusters including the data group DG.

  For example, the group G10 is a group G of the data group DG having a feature amount in which the area, luminance, and edge density of the object extracted from the image IMG are a small area, high luminance, and low edge density, respectively, and the data group DG23 , DG24, and DG28. That is, the data groups DG23, DG24, and DG28 are data groups DG having the same features (features that exhibit a small area, high luminance, and low edge density). In other words, the feature indicating a small area, high luminance, and low edge density is a feature common to the data groups DG23, DG24, and DG28 classified into different clusters.

  Further, for example, the group G13 is a group G of the data group DG having a feature amount in which the area, luminance, and edge density of the object extracted from the image IMG are large area, medium luminance, and low edge density, respectively. Includes groups DG22, DG27, DG26, DG30. That is, the data groups DG22, DG27, DG26, and DG30 are data groups DG having the same features (features that exhibit large area, medium luminance, and low edge density). In other words, the feature indicating large area, medium luminance, and low edge density is a feature common to the data groups DG22, DG27, DG26, and DG30 classified into different clusters.

  Further, for example, the group G15 is a group G of the data group DG having a feature amount in which the area, luminance, and edge density of the object extracted from the image IMG are medium area, high luminance, and high edge density, respectively. Groups DG34 and DG32 are included. That is, the data groups DG34 and DG32 are data groups DG having the same features (features showing medium area, high luminance, and high edge density). In other words, the features showing medium area, high brightness, and high edge density are features common to the data groups DG34 and DG32 classified into different clusters.

  As described above, the selection unit 400 groups the data group DG for each feature (combination of three sections of area, three sections of luminance, and two sections of edge density) indicated by the second feature amount. Then, the selection unit 400 extracts groups G10, G11, G13, G14, and G15 as the group G of the data group DG that has features common to a plurality of clusters.

  Next, the selection unit 400 selects a display image feature amount. The display image feature amount is a feature amount of the image IMG to be displayed. That is, the display image feature amount is a feature amount representing a feature common to the data group DG belonging to the display group G. For example, the selection unit 400 selects the display group G that includes the image IMG to be displayed on the display device 90. Thereby, the display image feature amount is selected.

  For example, the selection unit 400 selects, as the display group G13, the group G13 having the largest number of clusters including the data group DG having common characteristics among the groups G10, G11, G13, G14, and G15. Then, the selection unit 400 stores features common to the data group DG belonging to the display group G13 (features indicating large area, medium luminance, and low edge density) in the storage unit 210 and the like as display image feature amounts. That is, the selection unit 400 stores, as a display image feature quantity, a combination feature quantity having an area divided into a large area, a luminance value classified into a medium brightness, and an edge density classified into a high edge density, as the storage section 210 and the like. To remember.

  As described above, the selection unit 400 selects, from the data group DG classified into a plurality of clusters, the data group DG having a common feature among the plurality of clusters as the second feature amount (a plurality of feature amounts). For example, extraction is performed based on area, luminance, and edge density. Then, the selection unit 400 selects at least one of the groups G of the data group DG having a common feature among a plurality of clusters as the display group G. That is, the selection unit 400 extracts an image IMG having a feature common to the plurality of clusters from the images IMG classified into the plurality of clusters based on the second feature amount of the plurality of types. Then, the selection unit 400 selects at least one of the groups I of the image IMG having features common to the plurality of clusters as the display group G. Note that the operation of the selection unit 400 is not limited to the example shown in FIG.

  FIG. 6 shows an example of operations of the display unit 500 and the threshold setting unit 600 shown in FIG. The operation of the display unit 500 illustrated in FIG. 6 is the operation of the display unit 500 when the number of images IMG belonging to the display group G is equal to or less than a predetermined number. In the example shown in FIG. 6, “5” is preset as the predetermined number. The predetermined number is not limited to “5”. In FIG. 6, the operations of the display unit 500 and the threshold setting unit 600 will be described by taking as an example the case where the group G13 illustrated in FIG. 5 is selected as the display group G.

  First, the display unit 500 corresponds to the data group DG having the display image feature amount selected by the selection unit 400 (in the display group G13 shown in FIG. 5, the feature having a large area, medium luminance, and low edge density). An image IMG is acquired from the image database 214. That is, the display unit 500 acquires the images IMG22, IMG27, IMG26, and IMG30 belonging to the display group G13 selected by the selection unit 400 from the image database 214. Accordingly, the image IMG having the display image feature amount selected by the selection unit 400 is extracted as the display candidate image IMG. When there are a plurality of data having the same display image feature amount in the same cluster, one data may be extracted from the data.

  Next, since the number of display candidate images IMG is “5” or less, the display unit 500 displays all the display candidate images IMG22, IMG27, IMG26, and IMG30 acquired from the image database 214 in the order of the circularity values. They are displayed side by side on the display device 90. That is, the display unit 500 displays the images IMG belonging to the display group G selected by the selection unit 400 on the display device 90 in the order of the values indicated by the first feature amount. As described above, since the plurality of images IMG are displayed side by side based on the value (for example, the value of circularity) indicated by the first feature amount, compared to the case where the plurality of images IMG are displayed side by side in a random order. The comparison of the first feature amount can be facilitated.

  Next, the threshold setting unit 600 acquires information indicating the non-defective range. For example, the operator visually checks the images IMG22, IMG27, IMG26, and IMG30 displayed on the display device 90. Then, the operator selects the non-defective images IMG26 and IMG30 among the images IMG22, IMG27, IMG26, and IMG30 displayed on the display device 90 by using the input device 92 such as a mouse. Thereby, the input device 92 outputs information SINF indicating the non-defective images IMG26 and IMG30 selected by the operator to the threshold setting unit 600 and the like. The threshold setting unit 600 receives information SINF indicating the non-defective IMG 26 and IMG 30 among the plurality of images IMG displayed on the display device 90 from the input device 92. The information SINF indicating the non-defective images IMG26 and IMG30 is information indicating the range of non-defective products. Note that the information SINF indicating the non-defective image IMG is an example of information indicating a result of sorting a plurality of images IMG.

  Next, the threshold setting unit 600 sets the threshold of the first feature amount based on information SINF (non-defective range) received from the input device 92. For example, the threshold value setting unit 600 includes an image IMG26 included in a cluster having a circularity “0.95” and an image IMG30 included in a cluster having a circularity “1” as indicated by the information SINF received from the input device 92. Therefore, the circularity threshold is set to “0.95”. In this case, an object having a circularity of less than “0.95” is classified as a defective product, and an object having a circularity of “0.95” or more and “1” or less is classified as a non-defective product.

  Note that, for example, in the case of the image IMG27 included in the cluster having the circularity “0.90” and the image IMG26 included in the cluster having the circularity “0.95”, the threshold setting unit 600 sets the lower limit of the circularity. And the upper threshold are set to “0.90” and “0.95”, respectively. In this case, an object having a circularity of less than “0.90” and an object having a circularity of greater than “0.95” are classified into defective products, and the circularity is “0.90” or more and “0.95”. The following objects are classified into non-defective products.

  As described above, the threshold setting unit 600 sets the first feature amount threshold based on the information SINF received from the input device 92. In the threshold setting support device 100, the number of images IMG visually confirmed by the operator can be reduced as compared with the case where all the images IMG20 to IMG34 are visually confirmed. Setting can be facilitated. The operations of display unit 500 and threshold setting unit 600 are not limited to the example shown in FIG.

  FIG. 7 shows another example of the operation of the display unit 500 shown in FIG. 7 is an operation of the display unit 500 when the number of images IMG belonging to the display group G is greater than a predetermined number. In the example shown in FIG. 7, “5” is preset as the predetermined number. The predetermined number is not limited to “5”. In FIG. 7, the operation of the display unit 500 will be described by taking as an example the case where nine images IMG40, IMG41, IMG42, IMG43, IMG44, IMG45, IMG46, IMG47, and IMG48 belong to the display group G.

  The circularity of the object extracted from each of the images IMG40, IMG41, IMG42, and IMG43 is “0.85”, “0.90”, “0.93”, and “0.95”, respectively. The circularity of the object extracted from each of the images IMG44, IMG45, IMG46, IMG47, and IMG48 is “0.96”, “0.97”, “0.98”, “0.99”, and “1”. The images IMG40 to IMG48 are classified into clusters according to their circularity.

  First, the display unit 500 acquires the image IMG corresponding to the data group DG having the display image feature amount selected by the selection unit 400, that is, the images IMG40 to IMG48 belonging to the display group G from the image database 214. Accordingly, nine images IMG40-IMG48 having the display image feature amount selected by the selection unit 400 are extracted as display candidate images IMG.

  Next, since the number of display candidate images IMG is greater than “5”, the display unit 500 includes five (predetermined number) images IMG40 and IMG41 from the display candidate images IMG40 to IMG48 acquired from the image database 214. , IMG43, IMG45, and IMG48 are selected. The display unit 500 displays the five images IMG40, IMG41, IMG43, IMG45, and IMG48 on the display device 90 in the order of the circularity values.

  For example, when displaying the image IMG for the first time, the display unit 500 does not know the range of the non-defective product. Therefore, the image IMG 40 having the smallest circularity and the image IMG48 having the largest circularity are selected from among the five images IMG. Select as two images IMG. Then, the display unit 500 makes the increments of the circularity when the circularity range (the range of “0.85” to “1”) corresponding to the range of the display candidate image IMG is divided into four as evenly as possible. Thus, the remaining three images IMG are selected from the five images IMG. Accordingly, five images IMG40, IMG41, IMG43, IMG45, and IMG48 among the display candidate images IMG40 to IMG48 are displayed on the display device 90 in the order of the circularity values.

  For example, the operator visually confirms the images IMG40, IMG41, IMG43, IMG45, and IMG48 displayed on the display device 90. Then, the operator uses the input device 92 to select the non-defective image IMG48 among the images IMG40, IMG41, IMG43, IMG45, and IMG48 displayed on the display device 90. As a result, the input device 92 outputs information SINF indicating the non-defective image IMG 48 selected by the operator to the display unit 500 or the like. Then, the display unit 500 receives information SINF (information SINF indicating a non-defective product range) indicating the non-defective image IMG48 among the plurality of images IMG displayed on the display device 90 from the input device 92.

  Upon receiving the information SINF indicating the non-defective image IMG 48, the display unit 500 receives the images IMG 45, IMG 48 located on both sides of the boundary between the non-defective product and the defective product among the images IMG 40, IMG 41, IMG 43, IMG 45, and IMG 48 displayed side by side on the display device 90. Is identified. That is, the images IMG45 and IMG48 are examples of the first image IMG and the second image IMG located on both sides of the separation boundary.

  The circularity of each of the images IMG45 and IMG48 specified as the images IMG located on both sides of the boundary corresponds to the start point and the end point of the circularity range corresponding to the range of the display candidate image IMG to be re-extracted. For example, the display unit 500 displays four images IMG45, IMG46, IMG47, and IMG48 positioned between one of the images IMG45 and IMG48 when the images IMG are arranged in the order of the circularity values. Re-extract as. Then, since the number of display candidate images IMG is “5” or less, the display unit 500 arranges all of the re-extracted display candidate images IMG45, IMG46, IMG47, and IMG48 in the order of the circularity values. To display.

  For example, the operator visually checks the images IMG45, IMG46, IMG47, and IMG48 displayed on the display device 90, and selects the non-defective images IMG47 and IMG48 using the input device 92. Thereby, the display unit 500 receives information SINF indicating the non-defective images IMG47 and IMG48 among the plurality of images IMG displayed on the display device 90 from the input device 92. In this case, among the images IMG45, IMG46, IMG47, and IMG48 displayed side by side on the display device 90, the images IMG located on both sides of the boundary between the non-defective product and the defective product are the images IMG46 and IMG47. When the images IMG are arranged in the order of the circularity values, there is no image IMG other than the images IMG46 and IMG47 between one of the images IMG46 and IMG47, and thus the display processing of the image IMG by the display unit 500 ends. To do. In this case, for example, the threshold value setting unit 600 sets the circularity threshold value to “0.99” based on the information SINF indicating the non-defective images IMG47 and IMG48.

  When images IMG are arranged in the order of the circularity values, if there is an image IMG other than images IMG46 and IMG47 between one of images IMG46 and IMG47, display unit 500 has a predetermined number or less. The process of selecting and displaying the image IMG is executed again. For example, the display unit 500 re-extracts the image IMG existing between the image IMG46 and the image IMG47 as the display candidate image IMG, and the image IMG having a predetermined number or less including the images IMG46 and IMG47 from the display candidate image IMG. Select and display them in order of circularity value.

  That is, the display unit 500 selects and displays a predetermined number or less of the images IMG from the images IMG belonging to the display group G every time the information SINF is received until the separation boundary in the images belonging to the display group G is specified. . Accordingly, the display unit 500 narrows down the first feature amount range corresponding to the range of the image IMG of a predetermined number or less to be displayed in a stepwise manner.

  Here, for example, when all the display candidates IMG40 to IMG48 are displayed on the display device 90 at a time, the display of each image IMG is displayed as compared with the case where a predetermined number of images IMG are displayed on the display device 90. The area becomes smaller. In this case, it may be difficult for the operator to visually check the image IMG to determine whether the object is good or bad. On the other hand, the display unit 500 can suppress a reduction in the display area of each image IMG by setting the number of images IMG to be displayed on the display device 90 to a predetermined number or less. Thus, it can be suppressed that it is difficult to confirm. Note that the operation of the display unit 500 is not limited to the example shown in FIG.

  FIG. 8 shows an example of the operation of the threshold value setting support apparatus 100 shown in FIG. Note that the operation shown in FIG. 8 is an aspect of a threshold setting support method executed by a computer. For example, the operation illustrated in FIG. 8 may be realized only by hardware, or may be realized by controlling the hardware by software such as a threshold setting support program. That is, a computer controlled by the threshold setting support program may execute the operation shown in FIG. The program that causes the computer to execute the operation of FIG. 8 is an aspect of the threshold setting support program.

  In the operation shown in FIG. 8, a series of processing from step S100 to step S130 is executed as feature amount extraction processing for extracting feature amounts representing features of the object from the image IMG. In the operation shown in FIG. 8, a series of processing from step S200 to step S700 is executed as threshold setting processing for setting the threshold value of the feature amount. That is, in the operation shown in FIG. 8, the feature amount extraction process (steps S100 to S130) and the threshold value setting process (steps S200 to S700) are executed in parallel. First, the feature amount extraction process (steps S100 to S130) will be described.

  In step S100, the extraction unit 200 acquires the image IMG from the imaging device 80, and detects an object from the image IMG. For example, when the imaging device 80 images the same object at a predetermined frame rate, the extraction unit 200 may acquire a plurality of frame images IMG obtained by imaging the same object from the imaging device 80. In this case, the extraction unit 200 tracks the object over a plurality of frames, and extracts the object from the image IMG of the frame in which the object is most imaged at the center of the image IMG. That is, the extraction unit 200 may extract a target object from any one of the plurality of images IMG even when acquiring a plurality of images IMG obtained by capturing the same target object.

  Next, in step S110, the extraction unit 200 extracts a plurality of types of feature amounts representing the features of the target object extracted in step S100. The plurality of types of feature amounts are, for example, shape features, texture features, color features, and the like, as described with reference to FIG.

  Next, in step S <b> 120, the extraction unit 200 registers the feature amount extracted in step S <b> 110 (a plurality of types of feature amounts representing the feature of the object) in the feature amount database 212. For example, as described with reference to FIG. 3, the extraction unit 200 registers a data group DG including data indicating a feature amount or the like in the feature amount database 212 in association with the image IMG from which the feature amount has been extracted.

  Next, in step S130, the extraction unit 200 registers the image IMG obtained by extracting the object in step S100 in the image database 214. For example, the extraction unit 200 registers the image IMG from which the feature amount is extracted in the image database 214 so as to correspond to the feature amount registered in the feature amount database 212. Note that the image IMG registered in the image database 214 may be an image IMG obtained by cutting out only the region of the object from which the feature amount is extracted.

  When the process in step S130 ends, the feature amount extraction process for the object extracted in step S100 ends. For this reason, the extraction unit 200 starts a feature amount extraction process for another object from step S100. Next, the threshold setting process (steps S200 to S700) will be described.

  In step S200, the classification unit 300 acquires a predetermined data group DG from the feature amount database 212 as described with reference to FIG.

  Next, in step S300, the classification unit 300 selects a first feature amount for which a threshold is set, as described with reference to FIG. For example, the classification unit 300 may select the first feature amount from a plurality of types of feature amounts to be set as threshold values based on an instruction from the operator, or may select based on a predetermined order. Good. The plurality of types of feature amounts to be set as threshold values may be, for example, the plurality of types of feature amounts extracted in step S110, or may be some types of feature amounts determined in advance among the plurality of types of feature amounts. Alternatively, the threshold setting target feature value may be one type of feature value determined in advance among the plurality of types of feature values extracted in step S110.

  Next, in step S400, the threshold setting support apparatus 100 executes a threshold setting support process that supports setting of the threshold. As a result, the image IMG viewed when setting the threshold value of the first feature amount selected in step S300 is displayed on the display device 90 as described with reference to FIG. Details of the threshold setting support processing will be described with reference to FIG. After the threshold setting support process ends, the operation of the threshold setting support device 100 proceeds to step S500.

  In step S500, the threshold value setting unit 600 acquires information SINF indicating the non-defective range from the input device 92, as described with reference to FIG.

  Next, in step S600, as described with reference to FIG. 6, the threshold setting unit 600 sets the first feature amount threshold based on the information SINF indicating the non-defective range acquired in step S500. For example, the threshold value setting unit 600 determines a threshold value of the first feature value based on the information SINF, and stores the determined threshold value of the first feature value in the storage unit 210 or the like.

  Next, in step S700, the threshold value setting support apparatus 100 determines whether or not the threshold values of all feature quantities to be set as threshold values have been set. When the threshold values of all feature amounts to be set are already set, the threshold setting process by the threshold setting support device 100 ends. On the other hand, when there is a feature value for which a threshold value is not set among feature values to be set as threshold values, the operation of the threshold value setting support apparatus 100 returns to step S300.

  Note that the operation of the threshold setting support apparatus 100 is not limited to the example shown in FIG. For example, the process of step S130 may be executed before the process of step S120, or may be executed in parallel with the process of step S120.

  FIG. 9 shows an example of the threshold setting support process shown in FIG. That is, the series of processing in steps S410 to S490 is an example of the threshold setting support processing in step S400 illustrated in FIG. For example, the process of step S410 is executed after the process of step S300 shown in FIG. 8 is executed.

  In step S410, the classification unit 300 classifies the plurality of data groups DG acquired from the feature amount database 212 by the first feature amount as described with reference to FIG. The plurality of data groups DG is the data group DG acquired from the feature amount database 212 in step S200 shown in FIG. 8, and the first feature amount is the first selected in step S300 shown in FIG. It is a feature quantity.

  Next, in step S420, as described with reference to FIG. 5, the selection unit 400 groups the data group DG classified by the first feature amount by the second feature amount. The second feature amount is, for example, a feature amount other than the first feature amount among a plurality of types of feature amounts representing the object extracted from the image IMG in step S110 illustrated in FIG. When the second feature amount is one type of feature amount, the selection unit 400 indicates a value indicated by the second feature amount (a feature indicated by the second feature amount) of the data group DG classified by the first feature amount. Group according to. In addition, when the second feature amount is two or more types of feature amounts, the selection unit 400 selects two or more types of feature amounts included in the second feature amount from the data group DG classified by the first feature amount. Grouping is performed according to combinations of values indicated (features indicated by the second feature amount).

  Next, in step S430, the selection unit 400 selects a display image feature amount that is a feature amount of the image IMG to be displayed, as described with reference to FIG. The display image feature amount is selected from the features indicated by the second feature amount used when grouping the plurality of data groups DG in step S420. For example, the selection unit 400 selects the display group G from the plurality of groups G of the data group DG grouped for each feature indicated by the second feature amount. Thereby, the display image feature amount is selected.

  Next, in step S440, as described with reference to FIGS. 6 and 7, the display unit 500 extracts the image IMG having the display image feature amount selected in step S430 as the display candidate image IMG. For example, the display unit 500 acquires the image IMG belonging to the display group G from the image database 214 as the display candidate image IMG.

  Next, in step S450, the display unit 500 determines whether or not the number of display candidate images IMG is equal to or less than a predetermined number. When the number of display candidate images IMG is equal to or smaller than the predetermined number, the operation of the display unit 500 proceeds to step S490. On the other hand, when the number of display candidate images IMG is larger than the predetermined number, the operation of the display unit 500 proceeds to step S460.

  In step S460, the display unit 500 selects a predetermined number or less of the images IMG from the display candidate images IMG and displays them on the display device 90, as described with reference to FIG.

  Next, in step S470, the display unit 500 determines whether or not information SINF indicating the non-defective range (information SINF indicating the non-defective image IMG) is received from the input device 92. The information SINF indicating the non-defective range is information indicating, for example, a result obtained by sorting the image IMG of a predetermined number or less displayed on the display device 90 in step S460. When the information SINF indicating the non-defective range is received from the input device 92, the operation of the display unit 500 moves to step S480. On the other hand, when the information SINF indicating the non-defective range is not received from the input device 92, the operation of the display unit 500 returns to step S470. That is, the display unit 500 waits for execution of the process of step S480 until information SINF indicating the range of non-defective items selected by the operator is received from the input device 92.

  In step S480, as described with reference to FIG. 7, the display unit 500 displays the image IMG from one (first image IMG) to the other (second image IMG) of the images IMG on both sides of the boundary between the non-defective product and the defective product. Are extracted as display candidate images IMG. For example, the display unit 500 identifies two images IMG located on both sides of the boundary between the non-defective product and the defective product among a predetermined number or less of the images IMG displayed side by side on the display device 90. When the image IMG is arranged in the order of the value of the first feature amount, the display unit 500 displays the two images IMG (images IMG on both sides of the boundary of the images IMG displayed on the display device 90) from one to the other. The image IMG located until the time is re-extracted as the display candidate image IMG.

  After the process of step S480 is executed, the operation of the display unit 500 returns to step S450. Accordingly, since a predetermined number or less of the images IMG are selected again and displayed, the range of the first feature amount corresponding to the range of the image IMG of the predetermined number or less to be displayed is near the boundary between the non-defective product and the defective product. Squeezed in stages.

  As described above, when the display unit 500 receives the information SINF indicating the non-defective range, the first image IMG located on both sides of the separation boundary from the predetermined number or less of the images IMG displayed side by side on the display device 90. And a second image IMG is identified. Then, when the image IMG belonging to the display group G is arranged based on the value of the first feature amount, the display unit 500 includes the image IMG between the first image IMG and the second image IMG. Then, a predetermined number of images IMG or less are selected again and displayed. For example, when the images IMG belonging to the display group G are arranged on the basis of the value of the first feature amount, the display unit 500 determines from the image IMG positioned between the first image IMG and the second image IMG. Images IMG equal to or smaller than the number of images are selected and displayed. As described above, when the number of images IMG belonging to the display group G is larger than the predetermined number, the display unit 500 determines the first feature amount range corresponding to the range of the image IMG equal to or less than the predetermined number to be displayed stepwise. Narrow down to. The threshold setting support process is not limited to the example shown in FIG.

  As described above, also in the embodiment shown in FIG. 2 to FIG. 9, the same effect as that of the embodiment shown in FIG. 1 can be obtained. For example, the threshold setting support device 100 clusters the images IMG with the first feature amount of the threshold setting target among the plurality of types of feature amounts representing the features of the object extracted from the plurality of images IMG. Then, the threshold setting support apparatus 100 groups the images IMG having the common features common to the plurality of clusters based on the second feature amount among the plurality of types of feature amounts, and grouped images IMG. At least one of the groups is selected as the display group G. For example, the threshold value setting support apparatus 100 selects the group G having the largest number of clusters including the image IMG having features common to a plurality of clusters as the display group G. This makes it possible to increase the number of image IMG candidates arranged in the order of the first feature value, and to compare the first feature quantity as compared with the case where the number of images IMG arranged in the order of the first feature value is small. Can be made easier. Even in this case, the number of images IMG displayed on the display device 90 is smaller than the number of all images IMG acquired from the imaging device 80.

  Therefore, the threshold setting support apparatus 100 can reduce the number of images IMG that the operator visually confirms compared with the case where all the images IMG acquired from the imaging apparatus 80 are visually confirmed, and can classify the objects. Setting of the threshold can be facilitated. Furthermore, since the plurality of images IMG are displayed side by side based on the value indicated by the first feature amount, it is easier to compare the first feature amount than when the plurality of images IMG are displayed side by side in a random order. Can be.

  FIG. 10 shows another embodiment of the threshold setting support apparatus. Elements that are the same as or similar to those described in FIGS. 1 to 9 are given the same or similar reference numerals, and detailed descriptions thereof are omitted. The threshold setting support apparatus 102 illustrated in FIG. 10 is included in, for example, the selection system 1 that analyzes an image IMG obtained by imaging an object with the imaging apparatus 80 and classifies the object. The sorting system 1 illustrated in FIG. 10 includes an imaging device 80, a display device 90, and an input device 92 that are communicably connected to the threshold setting support device 102.

  For example, the threshold setting support device 102 executes a threshold setting support process for facilitating the setting of the threshold when the operator sets the threshold in the sorting system 1. Note that the threshold setting support apparatus 102 may be realized only by hardware, or may be realized by controlling hardware by software. For example, a computer controlled by software such as a threshold setting support program may execute the threshold setting support processing of the threshold setting support device 102.

  The threshold setting support apparatus 102 is the same as or similar to the threshold setting support apparatus 100 shown in FIG. 2 except that it includes a selection unit 402 instead of the selection unit 400 shown in FIG. For example, the threshold setting support apparatus 102 includes an extraction unit 200, a storage unit 210, a classification unit 300, a selection unit 402, a display unit 500, and a threshold setting unit 600. The extraction unit 200, storage unit 210, classification unit 300, display unit 500, and threshold setting unit 600 are the same as the extraction unit 200, storage unit 210, classification unit 300, display unit 500, and threshold setting unit 600 shown in FIG. It is the same.

  Similar to the selection unit 400 shown in FIG. 2, the selection unit 402 selects an image IMG having a common feature among a plurality of clusters from among a plurality of types of feature amounts. Extraction is performed based on the second feature amount. Then, the selection unit 402 selects at least one of the groups I of the image IMG having features common to the plurality of clusters as the display group G. Note that the selection unit 402 preferentially selects, as the display group G, the group G that includes the image IMG in a plurality of clusters adjacent to each other by the value indicated by the first feature amount among the plurality of groups G. Details of the operation of the selection unit 402 will be described with reference to FIG. Note that the configuration of the threshold setting support apparatus 102 is not limited to the example shown in FIG.

  Further, in the threshold setting support program for causing the computer to execute the threshold setting support processing of the threshold setting support apparatus 102, the group G including the image IMG is displayed in the plurality of clusters adjacent to each other by the value indicated by the first feature amount. Select as priority. Other processes of the threshold setting support program for causing the computer to execute the threshold setting support process of the threshold setting support apparatus 102 are the same as or similar to the threshold setting support program in the embodiment shown in FIGS. Similarly, in the threshold setting support method, the group G including the image IMG is preferentially selected as the display group G in a plurality of clusters adjacent to each other by the value indicated by the first feature value, as shown in FIGS. This is the same as or similar to the threshold setting support method in the embodiment shown in FIG.

  FIG. 11 shows an example of the operation of the selection unit 402 shown in FIG. In FIG. 10, as in FIG. 5, the image IMG corresponding to the data group DG is illustrated in parentheses shown above the data group DG in order to make the features of the object easier to understand. In FIG. 11, as in FIG. 5, four types of circularity, area, luminance, and edge density are set as a plurality of types of feature amounts extracted by the extraction unit 200, the first feature amount is set as a circularity, The feature amount will be described as three types of feature amounts: area, luminance, and edge density. Detailed descriptions of operations similar to those described in FIG. 5 are omitted.

  First, the selection unit 402 groups the data groups DG (DG50 to DG60) classified into a plurality of clusters by the classification unit 300 with common features. The groups G10 to G13 are the same as or similar to the groups G10 to G13 described in FIG. 5 except for the data group DG belonging to each group G.

  The groups G10, G12, and G13 include three clusters that include the data group DG, and the group G11 includes two clusters that include the data group DG.

  For example, the group G10 includes data groups DG58 and DG59 included in a cluster having a circularity “0.85” and a cluster having a circularity “0.90” adjacent to each other, and a cluster having a circularity “1”. And a data group DG60 included. Note that a cluster with a circularity of “0.90” and a cluster with a circularity of “1” have a circularity of “0.95” between a cluster with a circularity of “0.90” and a cluster with a circularity of “1”. Since clusters exist, the circularity value is not a plurality of adjacent clusters.

  The group G11 includes a data group DG56 included in a cluster having a circularity “0.85” and a data group DG57 included in a cluster having a circularity “1”. The group G12 includes data groups DG53, DG54, which are included in a cluster of circularity “0.85”, a cluster of circularity “0.90”, and a cluster of circularity “0.95” that are adjacent to each other. It has DG55. The group G13 includes data groups DG50, DG51, and DG52 included in a cluster having a circularity of “0.90”, a cluster having a circularity of “0.95”, and a cluster having a circularity of “1”, respectively. Have. Therefore, the selection unit 402 extracts groups G10, G12, and G13 as a group G that includes the data group DG in a plurality of clusters adjacent to each other in the circularity value (value indicated by the first feature value).

  Next, the selection unit 402 selects a display image feature amount. For example, the selection unit 402 selects the two groups G12 and G13 that include the image IMG and have the largest number of adjacent clusters with the value indicated by the circularity value from among the groups G10, G12, and G13. Select as (G12, G13). As a result, the data group DG belonging to the display group G12 has a large area, high luminance and low edge density, and the data group DG belonging to the display group G13 has a large area, medium luminance and low edge density. Is selected as the display image feature amount. Then, the selection unit 402 stores the selected display image feature amount in the storage unit 210 or the like.

  In this case, the display unit 500 displays the image group in which the images IMG53, IMG54, and IMG55 belonging to the display group G12 are arranged and the image group in which the images IMG50, IMG51, and IMG52 that belong to the display group G13 are arranged on the top and bottom of the screen of the display device 90. May be displayed respectively. Alternatively, the display unit 500 sequentially displays an image group in which the images IMG53, IMG54, and IMG55 belonging to the display group G12 are arranged and an image group in which the images IMG50, IMG51, and IMG52 belonging to the display group G13 are arranged in order for each group G. It may be displayed on the device 90.

  Here, the cluster range including the image IMG belonging to the group G12 and the cluster range including the image IMG belonging to the group G13 are different from each other. Therefore, by selecting the two groups G12 and G13 as the display group G, the circularity range corresponding to the range of the image IMG displayed side by side on the display device 90 is displayed, and only one of the groups G12 and G13 is displayed. Compared to the case of selecting as, it can be widened. That is, when two groups G12 and G13 are selected as the display group G, the range of circularity that can be visually confirmed can be widened as compared with the case where only one of the groups G12 and G13 is selected as the display group G. The threshold value can be easily set.

  For example, even when two groups G11 and G12 are selected as the display group G, the range of circularity corresponding to the range of the image IMG displayed in the order of the value of the first feature amount is the two groups G12 and G13. This is the same as when the display group G is selected. However, the image IMG55 included in the cluster with the circularity “0.95” and the image IMG57 included in the cluster with the circularity “1” belong to different groups G12 and G11 and have different display image feature amounts. . For this reason, in the comparison between the image IMG55 and the image IMG57, it may be difficult to visually confirm the image IMG51 and the image IMG52 having the same display image feature amount. In other words, by selecting the group G13 including the images IMG51 and IMG52 having the same display image feature amount as the display group G together with the group G12, the circularity can be compared as compared with the case of selecting the group G11. Can be easily.

  As described above, the selection unit 402 preferentially selects, as the display group G, the group G that includes the image IMG in the plurality of clusters adjacent to each other by the value indicated by the first feature amount. Then, the selection unit 402 extracts a plurality of groups G including the image IMG in a plurality of clusters adjacent to each other by the value indicated by the first feature amount, and selects the extracted plurality of groups G as the display group G.

  Note that the operation of the selection unit 402 is not limited to the example shown in FIG. For example, the selection unit 402 may select two groups G10 and G13 as the display group G. Even when the two groups G10 and G13 are selected as the display group G, the range of the circularity corresponding to the range of the image IMG to be displayed in the order of the value of the first feature amount is the two groups G12 and G13. It is the same as when selecting as. In this case, since the images IMG58 and IMG59 have the same display image feature amount, it is possible to suppress difficulty in visually confirming. The number of groups G selected as the display group G may be three or more, or one. For example, when the range of the cluster including the image IMG belonging to the group G12 is the same as the range of the cluster including the image IMG belonging to the group G13, the selection unit 402 selects only one of the groups G12 and G13 as the display group G. May be.

  As described above, also in the embodiment shown in FIGS. 10 to 11, the same effect as that of the embodiment shown in FIGS. 2 to 9 can be obtained. For example, the threshold setting support apparatus 102 clusters the image IMG with the first feature amount of the threshold setting target among a plurality of types of feature amounts representing the features of the object extracted from the plurality of images IMG. Then, the threshold setting support apparatus 102 groups the images IMG having common features common to the plurality of clusters based on the second feature amount among the plurality of types of feature amounts, and the grouped image IMG. At least one of the groups is selected as the display group G. Thereby, the threshold value setting support apparatus 102 can reduce the number of images IMG that the operator visually confirms compared with the case where all the images IMG acquired from the imaging apparatus 80 are visually confirmed, so that the object is sorted. The threshold value can be easily set.

  Further, the threshold setting support apparatus 102 preferentially selects, as the display group G, a group G that includes the image IMG in a plurality of clusters adjacent to each other by the value indicated by the first feature amount. Thus, when the images IMG belonging to the display group G are displayed on the display device 90 in the order of the values indicated by the first feature amount, the first features corresponding to the ranges of the plurality of images IMG displayed on the display device 90 are displayed. The value indicated by the quantity is continuous. As a result, the comparison of the first feature value can be facilitated compared to the case where the value indicated by the first feature value corresponding to the range of the plurality of images IMG displayed side by side on the display device 90 is discontinuous. .

  Further, the threshold value setting support apparatus 102 has a plurality of groups when the first feature value ranges corresponding to the plurality of groups G each including the image IMG in a plurality of adjacent clusters having a value indicated by the first feature value are different. G is selected as the display group G. In this case, the range of the first feature amount corresponding to the range of the image IMG displayed side by side on the display device 90 can be made wider than when only one of the plurality of groups G is selected as the display group G. That is, when the plurality of groups G are selected as the display group G, the range of the first feature amount that can be visually confirmed is made wider than when only one of the plurality of groups G is selected as the display group G. The threshold value can be easily set. Even in this case, the number of images IMG displayed on the display device 90 is smaller than the number of all images IMG acquired from the imaging device 80.

  FIG. 12 shows an example of hardware that executes the threshold setting support program. The same elements as those described in FIGS. 1 to 11 are denoted by the same reference numerals, and detailed description thereof will be omitted. For example, the threshold setting support program realizes functions of threshold setting support devices such as the threshold setting support devices 10, 100, and 102 shown in FIGS.

  For example, the computer 1000 includes a processor 1100, a memory 1200, a hard disk device 1300, an input interface 1400, an output interface 1500, and an optical drive device 1600. The processor 1100, the memory 1200, the hard disk device 1300, the input interface 1400, the output interface 1500, and the optical drive device 1600 are connected to the bus 1700. The functions of the threshold setting support devices such as the threshold setting support devices 10, 100, and 102 are implemented by, for example, the processor 1100, the memory 1200, the hard disk device 1300, the input interface 1400, and the output interface 1500.

  The optical drive device 1600 can be mounted with a removable disk 1620 such as an optical disk, and reads and records information recorded on the mounted removable disk 1620. The computer 1000 communicates with the outside of the computer 1000 via the input interface 1400 and the output interface 1500. For example, the input interface 1400 receives an image IMG or the like from the imaging device 80 and transfers it to the processor 1100 or the like. The input interface 1400 receives information SINF indicating the non-defective range from the input device 92 and transfers it to the processor 1100 or the like. The output interface 1500 outputs, for example, an image IMG to be displayed on the display device 90 to the display device 90.

  The memory 1200 stores, for example, the operating system of the computer 1000. The memory 1200 also stores application programs such as a threshold setting support program for the processor 1100 to execute operations of threshold setting support devices such as the threshold setting support devices 10, 100, and 102. The memory 1200 may store an image IMG, a feature amount database 212, an image database 214, and the like. Note that the image IMG, the feature database 212, the image database 214, and the like may be stored in the hard disk device 1300.

  An application program such as a threshold setting support program can be recorded and distributed on a removable disk 1620 such as an optical disk. For example, the computer 1000 may read an application program such as a threshold setting support program from the removable disk 1620 via the optical drive device 1600 and store it in the memory 1200 or the hard disk device 1300. In addition, the computer 1000 may download an application program such as a threshold setting support program via a communication device (not shown) connected to a network such as the Internet and store it in the memory 1200 or the hard disk device 1300. Further, the computer 1000 may acquire the image IMG from the imaging device 80 via a communication device connected to the network.

  Note that the hardware for executing the threshold setting support program is not limited to the example shown in FIG. For example, in the computer 1000, the optical drive device 1600 may be omitted. Further, for example, the computer 1000 may include an SSD (Solid State Drive) or the like instead of the hard disk device 1300.

The inventions described in the above embodiments are organized and disclosed as additional notes as follows.
(Appendix 1)
Extracting a plurality of types of feature quantities representing features of the object from a plurality of images including the object,
Classifying the plurality of images into a plurality of clusters based on a first feature amount set with a threshold for classifying the object among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. Select at least one of the groups of images as the display group,
A threshold setting support program for causing a computer to execute a process of displaying images belonging to the display group side by side based on a value indicated by the first feature amount.
(Appendix 2)
In the threshold setting support program described in Appendix 1,
A threshold setting support program for causing a computer to execute processing for preferentially selecting, as the display group, the group having a large number of clusters including images from among the plurality of groups.
(Appendix 3)
In the threshold setting support program described in Appendix 1,
Threshold setting support for causing a computer to execute processing for preferentially selecting the group including an image in a plurality of clusters adjacent to each other by a value indicated by the first feature value among the plurality of groups. program.
(Appendix 4)
In the threshold setting support program described in Appendix 3,
Threshold setting for causing a computer to execute a process of extracting a plurality of groups including images in a plurality of clusters adjacent to each other by a value indicated by the first feature value, and selecting the plurality of extracted groups as the display group Support program.
(Appendix 5)
In the threshold value setting support program according to any one of appendix 1 to appendix 4,
A threshold setting support program for causing a computer to execute a process in which at least two types of feature amounts other than the first feature amount among the plurality of types of feature amounts are set as the second feature amount.
(Appendix 6)
In the threshold value setting support program according to any one of appendix 1 to appendix 5,
When the number of images belonging to the display group is a predetermined number or less, all the images belonging to the display group are displayed side by side based on the value indicated by the first feature amount,
When the number of images belonging to the display group is greater than the predetermined number, the image of the predetermined number or less is selected from the images belonging to the display group, and the selected image of the predetermined number or less is the first feature. A threshold setting support program for causing a computer to execute a process of displaying side by side based on a value indicated by a quantity.
(Appendix 7)
In the threshold setting support program described in Appendix 6,
When information indicating the result of sorting the predetermined number or less of the images displayed side by side is received, the first image and the second image located on both sides of the separation boundary are received from the predetermined number or less of the images displayed side by side. When an image is specified and images belonging to the display group are arranged based on the value of the first feature amount, and there is an image between the first image and the second image, the first The first feature corresponding to the range of the predetermined number of images or less to be displayed by selecting and displaying the predetermined number of images or less from images positioned between one image and the second image. A threshold setting support program that causes a computer to execute processing that narrows the amount range.
(Appendix 8)
An extraction unit that extracts a plurality of types of feature amounts representing features of the object from a plurality of images including the object;
A classifying unit that classifies the plurality of images into a plurality of clusters based on a first feature amount in which a threshold for classifying the object is set among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. A selection unit for selecting at least one of the groups of images as a display group;
A threshold setting support device, comprising: a display unit configured to display images belonging to the display group side by side based on a value indicated by the first feature amount.
(Appendix 9)
In the threshold value setting support device according to attachment 8,
The threshold value setting support apparatus, wherein the selection unit preferentially selects the group having a large number of clusters including images from the plurality of groups as the display group.
(Appendix 10)
In the threshold value setting support device according to attachment 8,
The selection unit preferentially selects, as the display group, the group including an image in a plurality of clusters adjacent to each other by a value indicated by the first feature amount among the plurality of groups. Setting support device.
(Appendix 11)
In the threshold setting support device according to attachment 10,
The selection unit extracts a plurality of the groups including images in a plurality of clusters adjacent to each other by a value indicated by the first feature value, and selects the extracted plurality of groups as the display group. Threshold setting support device.
(Appendix 12)
In the threshold setting support device according to any one of appendix 8 to appendix 11,
The threshold value setting support apparatus, wherein the selection unit uses at least two types of feature values other than the first feature value among the plurality of types of feature values as the second feature value.
(Appendix 13)
In the threshold setting support device according to any one of appendix 8 to appendix 12,
The display unit
When the number of images belonging to the display group is a predetermined number or less, all the images belonging to the display group are displayed side by side based on the value indicated by the first feature amount,
When the number of images belonging to the display group is greater than the predetermined number, the image of the predetermined number or less is selected from the images belonging to the display group, and the selected image of the predetermined number or less is the first feature. A threshold value setting support device, characterized by displaying side by side based on a value indicated by a quantity.
(Appendix 14)
In the threshold setting support device according to attachment 13,
When the display unit receives information indicating the result of sorting the predetermined number or less of the images displayed side by side, the first display unit is located on both sides of the separation boundary from the predetermined number or less of the images displayed side by side. When an image and a second image are identified and images belonging to the display group are arranged based on the value of the first feature amount, an image exists between the first image and the second image In the case of selecting, the predetermined number or less images are selected and displayed from the images positioned between the first image and the second image, and correspond to the range of the predetermined number or less images to be displayed. A range of the first feature amount is narrowed down.
(Appendix 15)
A threshold setting support method executed by a computer,
The computer is
Extracting a plurality of types of feature quantities representing features of the object from a plurality of images including the object,
Classifying the plurality of images into a plurality of clusters based on a first feature amount set with a threshold for classifying the object among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. Select at least one of the groups of images as the display group,
The threshold value setting support method, wherein images belonging to the display group are displayed side by side based on a value indicated by the first feature amount.
(Appendix 16)
Extracting a plurality of types of feature quantities representing features of the object from a plurality of images including the object,
Classifying the plurality of images into a plurality of clusters based on a first feature amount set with a threshold for classifying the object among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. Select at least one of the groups of images as the display group,
A computer-readable recording medium having recorded thereon a threshold setting support program for causing a computer to execute a process of displaying images belonging to the display group side by side based on a value indicated by the first feature amount .

  From the above detailed description, features and advantages of the embodiments will become apparent. This is intended to cover the features and advantages of the embodiments described above without departing from the spirit and scope of the claims. Also, any improvement and modification should be readily conceivable by those having ordinary knowledge in the art. Therefore, there is no intention to limit the scope of the inventive embodiments to those described above, and appropriate modifications and equivalents included in the scope disclosed in the embodiments can be used.

  DESCRIPTION OF SYMBOLS 1 ... Sorting system; 10 ... Threshold setting support device; 20 ... Extraction part; 30 ... Classification part; 40 ... Selection part; 50 ... Display part; 80 ... Imaging device; 90 ... Display device; ...... Threshold setting support apparatus; 200, extraction unit, 210, storage unit, 212, feature database, 214, image database, 300, classification unit, 400, 402, selection unit, 500, display unit, 600, threshold setting unit; 1000 ... Computer; 1100 ... Processor; 1200 ... Memory; 1300 ... Hard disk drive; 1400 ... Input interface; 1500 ... Output interface; 1600 ... Optical drive device; 1620 ... Removable disk;

Claims (9)

Extracting a plurality of types of feature quantities representing features of the object from a plurality of images including the object,
Classifying the plurality of images into a plurality of clusters based on a first feature amount set with a threshold for classifying the object among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. Select at least one of the groups of images as the display group,
A threshold setting support program for causing a computer to execute a process of displaying images belonging to the display group side by side based on a value indicated by the first feature amount. In the threshold value setting support program according to claim 1,
A threshold setting support program for causing a computer to execute processing for preferentially selecting, as the display group, the group having a large number of clusters including images from among the plurality of groups. In the threshold value setting support program according to claim 1,
Threshold setting support for causing a computer to execute processing for preferentially selecting the group including an image in a plurality of clusters adjacent to each other by a value indicated by the first feature value among the plurality of groups. program. In the threshold setting support program according to claim 3,
Threshold setting for causing a computer to execute a process of extracting a plurality of groups including images in a plurality of clusters adjacent to each other by a value indicated by the first feature value, and selecting the plurality of extracted groups as the display group Support program. In the threshold setting support program according to any one of claims 1 to 4,
A threshold setting support program for causing a computer to execute a process in which at least two types of feature amounts other than the first feature amount among the plurality of types of feature amounts are set as the second feature amount. In the threshold value setting support program according to any one of claims 1 to 5,
When the number of images belonging to the display group is a predetermined number or less, all the images belonging to the display group are displayed side by side based on the value indicated by the first feature amount,
When the number of images belonging to the display group is greater than the predetermined number, the image of the predetermined number or less is selected from the images belonging to the display group, and the selected image of the predetermined number or less is the first feature. A threshold setting support program for causing a computer to execute a process of displaying side by side based on a value indicated by a quantity. In the threshold value setting support program according to claim 6,
When information indicating the result of sorting the predetermined number or less of the images displayed side by side is received, the first image and the second image located on both sides of the separation boundary are received from the predetermined number or less of the images displayed side by side. When an image is specified and images belonging to the display group are arranged based on the value of the first feature amount, and there is an image between the first image and the second image, the first The first feature corresponding to the range of the predetermined number of images or less to be displayed by selecting and displaying the predetermined number of images or less from images positioned between one image and the second image. A threshold setting support program that causes a computer to execute processing that narrows the amount range. An extraction unit that extracts a plurality of types of feature amounts representing features of the object from a plurality of images including the object;
A classifying unit that classifies the plurality of images into a plurality of clusters based on a first feature amount in which a threshold for classifying the object is set among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. A selection unit for selecting at least one of the groups of images as a display group;
A threshold setting support device, comprising: a display unit configured to display images belonging to the display group side by side based on a value indicated by the first feature amount. A threshold setting support method executed by a computer,
The computer is
Extracting a plurality of types of feature quantities representing features of the object from a plurality of images including the object,
Classifying the plurality of images into a plurality of clusters based on a first feature amount set with a threshold for classifying the object among the plurality of types of feature amounts;
Of the images classified into the plurality of clusters, an image having a common feature among a plurality of clusters is extracted based on a second feature amount among the plurality of types of feature amounts, and has the common feature. Select at least one of the groups of images as the display group,
The threshold value setting support method, wherein images belonging to the display group are displayed side by side based on a value indicated by the first feature amount.

Images