JP2019082853A - Information processing device, information processing method, and information processing program - Google Patents

Abstract

To improve determination accuracy, and to expand determinable objects.SOLUTION: An information processing device (1) comprises: a complexity information acquisition unit (104) which acquires complexity information of an imaging object; an image information acquisition unit (103) which acquires image information of the imaging object; a polarization information acquisition unit (105) which acquires polarization information of the imaging object; and a fuzzy inference unit (107) which determines prescribed determination items related to the imaging object on the basis of each information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an information processing apparatus or the like that determines predetermined determination items by image analysis.

  2. Description of the Related Art Conventionally, development of techniques for determining predetermined determination items by image analysis has been advanced. For example, Patent Document 1 below discloses a technique for capturing an image of food, performing binarization processing or the like on the captured image, and detecting foreign matter mixed in the food.

JP 2008-541007 gazette (announced on November 20, 2008)

  However, in the prior art as described above, there is room for improvement in the improvement of the determination accuracy and the expansion of the target that can be determined. For example, food may be contaminated with transparent foreign matter such as plastic, glass or vinyl. However, since the presence of such foreign matter is difficult to recognize in a normal captured image, the technique of Patent Document 1 can not or can not detect such foreign matter. The detection accuracy may be lowered. Moreover, in the technique of Patent Document 1, it is difficult to distinguish foreign particles having minute differences in shape from each other.

  An aspect of the present invention aims to realize improvement in determination accuracy and expansion of a target that can be determined in an information processing apparatus that determines predetermined determination items by image analysis.

  In order to solve the above-mentioned subject, information processor concerning one mode of the present invention acquires complexity information which shows a degree of complexity of shape of the above-mentioned imaging object from a picture which picturized imaging object An acquisition unit, an image information acquisition unit for acquiring image information indicating at least one of the shape and the color of the imaging object from the image of the imaging object, and polarization information from the polarization image of the imaging object And a determination unit that determines a predetermined determination item regarding the imaging target based on the polarization information acquisition unit, the complexity information, the image information, and the polarization information.

  Further, in order to solve the above problems, an information processing method according to an aspect of the present invention is an information processing method by an information processing apparatus, and the complexity of the shape of the imaging target from an image obtained by imaging the imaging target A complexity information acquisition step of acquiring complexity information indicating the degree of image; an image information acquisition step of acquiring image information indicating at least one of the shape and the color of the imaging object from the image obtained by imaging the imaging object; A predetermined determination item regarding the imaging target is determined using a polarization information acquisition step of acquiring polarization information from a polarization image obtained by imaging the imaging target, the complexity information, the image information, and the polarization information. And a determination step.

  According to one aspect of the present invention, the determination accuracy can be improved and the target that can be determined can be expanded compared to the prior art.

It is a block diagram showing an example of the principal part composition of the information processor concerning Embodiment 1 of the present invention. It is a figure which shows an example of the imaging target of Embodiment 1 of this invention, and its imaging method. It is a flowchart which shows an example of the process which the said information processing apparatus performs. It is a flowchart which shows an example of the determination processing performed by S6 of FIG. It is a figure which shows an example of the fuzzy inference in Embodiment 1 of this invention. It is a flowchart which shows an example of the determination processing determined by fuzzy inference using image information, polarization information, and complexity information. It is a figure which shows the example of the membership function corresponding to image information. It is a figure which shows the pattern of the determination processing using image information, polarization information, and complexity information. It is a figure which shows an example of the imaging object in Embodiment 2 of this invention, and its imaging method. It is a figure which shows an example of the imaging object in Embodiment 3 of this invention, and its imaging method. It is a figure which shows the example of the captured image and polarization image which imaged the inside of the refuse pit. It is a figure which shows an example of the imaging object in Embodiment 4 of this invention, and its imaging method. It is a figure which shows an example of the imaging object in Embodiment 5 of this invention, and its imaging method. It is a figure which shows an example of the imaging object in Embodiment 6 of this invention, and its imaging method. It is a figure which shows the example of the image imaged with the camera of FIG. It is a figure which shows an example of the imaging target in Embodiment 7 of this invention, and its imaging method. It is a figure which shows the example which marked the area | region where polarization information is characteristic in the polarization image which imaged the road surface. It is a figure which shows an example of the imaging object in Embodiment 8 of this invention, and its imaging method.

Embodiment 1
Hereinafter, an embodiment of the present invention will be described in detail based on FIGS. 1 to 6.

〔Device configuration〕
FIG. 1 is a block diagram showing an example of the main configuration of the information processing apparatus 1 according to the first embodiment of the present invention. The information processing apparatus 1 is an apparatus that determines a predetermined determination item based on an image obtained by imaging an imaging target. The information processing device 1 may be, for example, a computing device such as a personal computer. The information processing device 1 may be a stationary device or a portable device. In addition, the information processing apparatus 1 may have a function other than the above determination. In this case, the above determination may be a main function, or another function may be a main function.

  As illustrated, the information processing apparatus 1 includes a control unit 10 that integrally controls each part of the information processing apparatus 1 and a storage unit 11 that stores various data used by the information processing apparatus 1. The control unit 10 includes a captured image acquisition unit 101, a polarization image acquisition unit 102, an image information acquisition unit 103, a complexity information acquisition unit 104, a polarization information acquisition unit 105, an image information determination unit 106, a fuzzy inference unit 107, and learning. Section 108 is included. The storage unit 11 stores a fuzzy rule 111 and a membership function 112.

  The information processing apparatus 1 further includes an input unit 12 that receives an input to the information processing apparatus 1 and an output unit 13 for the data processing apparatus 1 to output data. For example, the input unit 12 may receive data from an external device in a wired or wireless manner, or may be an input device such as a mouse or a keyboard. The output unit 13 may be a display device or the like that performs display output, a speaker or the like that performs audio output, or a printer or the like that performs print output. Besides this, for example, the output unit 13 may output data to an external device in a wired or wireless manner. Further, for example, a device such as a touch panel in which the input unit 12 and the output unit 13 are integrally configured may be employed. The input unit 12 and the output unit 13 may be devices external to the information processing device 1 and external to the information processing device 1.

  The captured image acquisition unit 101 acquires an image obtained by capturing an imaging target. This image may be input through the input unit 12. In addition, when the image is stored in the storage unit 11, the captured image acquisition unit 101 may acquire the image from the storage unit 11. Hereinafter, an image acquired by the captured image acquisition unit 101 will be referred to as a captured image. The captured image may be any image that indicates the shape (external shape or internal shape), color, etc. of the imaging target. For example, the captured image may be an image captured without a polarizing filter or the like by a normal optical camera that is not a polarizing camera. Further, as long as it indicates the shape of an imaging target, it may be an image or the like captured by a polarization camera or the like. In addition to this, an image obtained by image processing each of the above-described images may be used as a captured image.

  The polarization image acquisition unit 102 acquires a polarization image obtained by imaging an imaging target. The polarized light image is an image including information related to polarized light contained in reflected light reflected by the imaging target or transmitted light transmitted through the imaging target, and is, for example, an image captured by a polarized camera. Since the s-polarized light and the p-polarized light have different reflectances, a difference occurs in polarization information regarding the s-polarized light and the p-polarized light before and after entering the imaging target. In addition, when light passes through the imaging target, a change occurs in the phase difference (polarization information) of the light before and after the transmission. The polarization image acquisition unit 102 acquires a polarization image including such polarization information. Even in the case of an imaging target that is transparent to visible light, the above difference or change may occur in the polarization information. Therefore, by making a determination using a polarization image, the determination on a transparent imaging target can be made highly accurate. It will be possible to

  The image information acquisition unit 103 generates image information from the captured image acquired by the captured image acquisition unit 101. The image information is information that is included in the captured image and indicates at least one of the shape and the color of the imaging target.

  For example, when the captured image represents the shape and color of the imaging target that can be perceived visually from the outside of the imaging target, that is, the appearance of the imaging target, information indicating at least a part of the elements of the appearance is used as the image information . For example, information indicating an outline of an imaging target may be used as image information, information indicating a color of an imaging target may be used as image information, or both of them may be used as image information.

  Further, for example, when the captured image represents a shape inside the imaging target that can not be viewed from the outside of the imaging target, information indicating the inside shape is used as the image information. For example, in the case of using an X-ray imaging image captured using X-rays, the image information indicates the internal shape of the imaging target (such as the shape of a bone if the imaging target is a living body).

  Further, the image information indicating the appearance of the imaging target and the image information indicating the internal shape of the imaging target may be used in combination. Furthermore, for example, information indicating the property (for example, material and the like) of the imaging target obtained by analysis of the captured image or the like may be used in combination with the image information. In the present embodiment, an example in which the image information acquisition unit 103 acquires the image information generated by itself is described. However, the image information may be generated by an external device of the information processing apparatus 1. In this case, the image information may be generated. The information acquisition unit 103 acquires image information from the device.

  The complexity information acquisition unit 104 generates complexity information indicating the degree of complexity of the shape of the imaging target from the captured image acquired by the captured image acquisition unit 101. The complexity information can be generated from a captured image captured by natural light, and can also be generated from a captured image captured using light rays other than natural light such as an X-ray captured image. It can also be generated from polarized images. The generation of the complexity information will be described later. In the present embodiment, an example in which the complexity information acquisition unit 104 acquires the complexity information generated by itself is described, but the complexity information may be generated by an apparatus external to the information processing apparatus 1, and in this case The complexity information acquisition unit 104 acquires the complexity information from the device.

  The polarization information acquisition unit 105 generates polarization information from the polarization image acquired by the polarization image acquisition unit 102. The polarization information may be information relating to the polarization of the reflected light reflected by the imaging target or the transmitted light transmitted through the imaging target, and, for example, at least one of polarization orientation, phase difference, polarization degree and ellipticity may be used as polarization information. Good. When the ellipticity is used as polarization information, the polarization information acquisition unit 105 calculates the ellipticity from the pixel value (light intensity) in each pixel of the polarization image. In the present embodiment, an example in which the polarization information acquisition unit 105 acquires polarization information generated by itself is described, but polarization information may be generated by an apparatus outside the information processing apparatus 1, and in this case, polarization information may be generated. The information acquisition unit 105 acquires polarization information from the device.

  The image information determination unit 106 uses the image information generated by the image information acquisition unit 103 to make a determination regarding an imaging target. Specifically, the image information determination unit 106 determines, using the image information, what the imaging target shown in the captured image is. For this determination, for example, an FCM (Fuzzy-C-Means) identifier may be used. In this case, those that may be imaging targets are set as classes, and the correspondence between input data and classes is learned in advance. Thereby, the corresponding class can be determined from the new input data (image information generated by the image information acquisition unit 103).

  The fuzzy inference unit 107 determines predetermined determination items regarding the imaging target based on the complexity information, the image information, and the polarization information. Although the details will be described later, this determination includes a fuzzy rule including at least two of image information, complexity information, and polarization information in an antecedent part and a determination result of a predetermined determination item being a consequent part Do by inference.

  The learning unit 108 performs machine learning on the correlation between at least two combinations of image information, complexity information, and polarization information, and at least one of a correct determination result of the predetermined determination item and an incorrect determination result.

  The fuzzy rule 111 and the membership function 112 are data used by the fuzzy inference unit 107 for fuzzy inference. The details of these will be described later. The membership function 112 reflects the result of machine learning by the learning unit 108. This will be described later in the section of [Learning].

[Image and judgment items used for judgment]
The imaging target of the present embodiment is a small fish, and the determination item of the present embodiment is a foreign substance mixed in the small fish. This will be described based on FIG. FIG. 2: is a figure which shows an example of the imaging target of Embodiment 1 of this invention, and its imaging method.

  In the example of FIG. 2, a large number of small fish F are placed on a table, and a camera (imaging device) C is disposed above the small fish F. That is, the imaging target of the camera C is the small fish F. When the small fish F is mixed with a foreign matter, the information processing apparatus 1 specifies what the foreign matter is. In FIG. 2, the foreign substances X1 to X5 are mixed with the small fish F. X1 is a piece of glass, X2 is a shell, X3 is a piece of vinyl, X4 is a wire, and X5 is a hair.

  The information processing apparatus 1 analyzes the captured image obtained by capturing the image capturing target by the camera C, and determines what the foreign substance mixed in the small fish is. Further, in the same manner as in FIG. 2, the image pickup target is imaged by a polarization camera to obtain a polarization image. This polarized image is also used for the above determination.

[Flow of processing]
The flow of processing executed by the information processing apparatus 1 will be described based on FIG. FIG. 3 is a flowchart showing an example of processing executed by the information processing apparatus 1.

  In S1, the captured image acquisition unit 101 acquires a captured image obtained by capturing an imaging target. Then, in S2, the image information acquisition unit 103 generates image information using the captured image acquired in S1.

  In S3, the complexity information acquisition unit 104 generates complexity information using the captured image acquired in S1. If the imaging target is the same, a captured image different from the captured image acquired in S1 may be used. In the generation of the complexity information, for example, the complexity information acquisition unit 104 may first extract the edge of the captured image, that is, the outline of the outer shape of the imaging target. Then, the complexity information acquisition unit 104 may calculate the fractal dimension of the contour line as the complexity information. For calculation of the fractal dimension, for example, a box counting method (box counting method), a modified pixel dilation method, or the like can be used. If the image information generated in S2 indicates the edge extracted in the captured image, the fractal dimension may be calculated using the information.

  In S4, the polarization image acquisition unit 102 acquires a polarization image obtained by imaging an imaging target. Then, in S5, the polarization information acquisition unit 105 generates polarization information using the polarization image acquired in S4. For example, when ellipticity is used as polarization information, the polarization information acquisition unit 105 calculates the ellipticity from the pixel value (light intensity) in each pixel of the polarization image.

  In S6, the image information determination unit 106 and the fuzzy inference unit 107 perform the determination process using the image information generated in S2, the complexity information generated in S3, and the polarization information generated in S5. Thus, the illustrated process ends.

[Flow of determination processing]
The flow of the determination process will be described based on FIG. FIG. 4 is a flowchart showing an example of the determination process performed in S6 of FIG. Also, processing related to fuzzy inference will be described with reference to FIG. FIG. 5 is a diagram showing an example of fuzzy inference in the present embodiment.

  In S61, the image information determination unit 106 determines what the imaging target is from the image information. As in the present embodiment, when a plurality of imaging targets are included in one captured image, the plurality of imaging targets are extracted from the captured image using the image information, and the determination is performed for each of the extracted imaging targets. You can do it. For this determination, it is possible to use, for example, an FCM discriminator in which image information is used as input data and the class is learned as “small fish”, “hair”, “wire”, “glass, shell, or vinyl” or the like. it can. As a result of this determination, for example, the probability that the object to be imaged is “wire” is within the range of 50 ± α [%] (α is an arbitrary value).

  The processing of S62 and subsequent steps is performed to enhance the determination accuracy of S61. For example, if the determination result in S61 is that the probability that the imaging target is “wire” is 50 ± α (α = 20) [%], that is, 30% to 70%, the imaging target is “ It can not be said that it can be distinguished whether it is a wire or a hair. For this reason, in the processing of S62 and thereafter, it is determined by fuzzy inference whether the imaging target is "wire" or "hair". Therefore, the membership function 112 and the fuzzy rule 111 used in the process of S62 and subsequent steps can determine at least whether the imaging target is "wire" or "hair". Note that different membership functions 112 and fuzzy rules 111 may be used according to the determination result of S61. In this case, a plurality of sets of membership functions 112 and fuzzy rules 111 may be prepared in advance, and membership functions 112 and fuzzy rules 111 may be selected and used according to the determination result of S61.

  In S62, the fuzzy inference unit 107 calculates the degree of adaptation of each piece of polarization information to each fuzzy set. For example, a membership function 112 as shown in FIG. 5A can be used to calculate the degree of fitness in S62. In the illustrated membership function 112, the horizontal axis is the ellipticity, and the vertical axis is the degree of matching (0 to 1). In this membership function, three fuzzy sets of ellipticity "small", "medium" and "large" are defined. For example, in the example of the same figure, when the value of the ellipticity is xa, the fuzzy inference unit 107 substitutes xa into the membership function 112, and the fitness of the "small" ellipticity is a2, "medium" The degree of matching of is calculated as a1. In this case, the matching degree of the ellipticity “large” is 0.

  In S63, the fuzzy inference unit 107 calculates the degree of adaptation to each fuzzy set for the complexity information. For example, a membership function 112 as shown in (b) of FIG. 5 can be used to calculate the degree of fitness in S63. In the illustrated membership function 112, the horizontal axis is the complexity, and the vertical axis is the fitness (0-1). In this membership function 112, three fuzzy sets of complexity "small", "medium" and "large" are defined. For example, in the example of the same figure, when the value of the complexity is xb, the fuzzy inference unit 107 substitutes xb for the membership function 112, and the fitness of the "medium" complexity is b1, "large" The matching degree of is calculated as b2, and the matching degree of "small" is 0.

  In S64, the fuzzy inference unit 107 calculates an inference value for each fuzzy rule 111 using the degree of matching calculated in S62 and S63. In the fuzzy rule 111 to be used, if the antecedent part (IF) includes the conditions related to the ellipticity and the complexity, and the consequent part (THEN) indicates what the imaging target is Good. For example, fuzzy rules 1 to 3 as shown in FIG. 5C can be used. Thus, in the fuzzy rule 111, since the correlation is expressed in language, there is an advantage that the intention of the rule can be clearly understood. The process of S64 in the case of using the fuzzy rules 1 to 3 of (c) of FIG. 5 is as described in the next paragraph.

  Since the fitness of ellipticity "large" in the antecedent part of fuzzy rule 1 is 0, and the fitness of low complexity "0" is also 0, fuzzy inference unit 107 sets the inferred value of fuzzy rule 1 to 0. Do. On the other hand, the matching degree of the ellipticity "medium" in the antecedent part of the fuzzy rule 2 is a1, and the matching degree of the complexity "medium" is b1. The fuzzy inference unit 107 sets the lower one of the numbers as the value of the antecedent. That is, the fuzzy inference unit 107 sets the inference value of the fuzzy rule 2 to a1. Further, since the fitness of ellipticity “small” in the antecedent part of fuzzy rule 3 is a 2 and the fitness of complexity “large” is b 2, fuzzy inference unit 107 determines the inferred value of fuzzy rule 3 Assume a2.

  In S65, the fuzzy inference unit 107 calculates an output value corresponding to each fuzzy rule, using each inference value calculated in S64. Since this output value indicates the determination result of what the imaging target is, in S65, the predetermined determination item is determined.

  For example, a membership function 112 as shown in (d) of FIG. 5 can be used to calculate the output value of S65. The vertical axis of this membership function 112 is the fitness, and the horizontal axis is the probability (%) that the object to be imaged is "hair," "wires," or "glass, shell, or vinyl." For example, the left end of the fuzzy set of "hair" has 100% probability that the imaging object is "hair", and the right end of the fuzzy set of "hair" has 0% probability of the imaging object being "hair". The probability of being "wire" is 100%.

  If the inferred value of the fuzzy rule 2 is a1 as in the above example, the trapezoidal portion surrounded by the portion corresponding to the “wire” of the membership function 112 and the straight line of the matching degree = a1 is the fuzzy rule 2 It becomes the value of the consequent part. Similarly, if the inferred value of fuzzy rule 3 is a2, a trapezoidal portion surrounded by a portion corresponding to “hair” of membership function 112 and a straight line of goodness of fit = a2 is a consequent portion of fuzzy rule 3 It becomes the value of. The fuzzy inference unit 107 integrates these trapezoidal portions to calculate a final output value. The fuzzy inference unit 107 may calculate, for example, the position of the center of gravity in a figure in which two trapezoids are superimposed as a final output value.

  In S66, the fuzzy inference unit 107 causes the output unit 13 to output the determination result based on the output value calculated in S65. For example, if the output value indicates that the probability that the object to be imaged is “hair” is 80% and the probability that it is “wire” is 20%, an image indicating that is output unit 13 May be displayed and output.

  In addition, the fuzzy inference unit 107 may reflect the determination result of S61 on the final output value. For example, in the determination result in S61, the probability that the imaging target is “wire” is in the range of 50 ± α (α = 20) [%], and in the determination result in S65, the probability that the imaging target is “wire” Consider the case where Y is Y%. In this case, for example, the final determination result may be calculated by the following equation, assuming that the determination result in S65 is a ratio to the range of 50 ± α. According to this formula, for example, when α = 20 [%] and Y = 30 [%], the final determination result is 42%.

(Final judgment result (%)) = (50−α) + 2α × Y / 100
Also, for example, when the determination result of S61 is calculated not as a numerical value range but as a numerical value, the average value (for example, arithmetic average value) of the determination result X [%] of S61 and the determination result Y [%] of S65 is finally determined It is good also as a judgment result.

[Modification of Judgment Processing]
In the example of FIG. 4, after the image information is used to determine what the imaging target is at S61, the final determination result is output by fuzzy inference using polarization information and complexity information. . However, the final determination result is not limited to the above example as long as it is calculated using at least three types of information of image information, polarization information, and complexity information. For example, the final determination result may be output by fuzzy inference using image information, polarization information, and complexity information.

  FIG. 6 is a flow chart showing an example of the determination processing by fuzzy inference using image information, polarization information, and complexity information. In the example of FIG. 6, in order to perform fuzzy inference using image information, in addition to the membership function 112 corresponding to polarization information and the membership function 112 corresponding to complexity information, the membership corresponding to image information The function 112 is prepared in advance. In addition, fuzzy rules 111 in which the degree of matching calculated by the membership function 112 is an antecedent part are prepared in advance.

  In S161, the fuzzy inference unit 107 uses the membership function 112 corresponding to the image information to calculate the adaptability to each fuzzy set for the image information generated in S2 of FIG. The membership function 112 corresponding to the image information may be, for example, as shown in FIG. FIG. 7 is a diagram showing an example of the membership function 112 corresponding to image information. In the membership function 112 in (a) of the figure, three fuzzy sets of “short”, “medium”, and “long” are defined for the length of the imaging target indicated by the image information. Further, in the membership function 112 of (b) of the figure, three fuzzy sets of “small”, “medium”, and “large” are defined for the size of the imaging target indicated by the image information. In the membership function 112 of (c), three fuzzy sets of “black”, “gray”, and “white” are defined for the color of the imaging target indicated by the image information. By using these membership functions 112, it is possible to determine what an imaging target is from three factors of length, size, and color. Of course, these membership functions 112 are illustrative, and membership functions 112 focusing on other characteristics of the imaging target may be applied.

  The subsequent S162 and S163 are the same as S62 and S63 of FIG. That is, in S162, the fuzzy inference unit 107 calculates the degree of conformity to each fuzzy set for the polarization information generated in S4 of FIG. 3 using the membership function 112 corresponding to the polarization information. Further, in S163, the fuzzy inference unit 107 calculates the fitness to each fuzzy set for the complexity information generated in S3 of FIG. 3 using the membership function 112 corresponding to the complexity information.

  In S164, the fuzzy inference unit 107 calculates an inference value for each fuzzy rule, using the degree of fitness calculated in S161 to S163. In the fuzzy rules used, the antecedent part (IF) includes conditions related to polarization information, complexity information, and image information, and the consequent part (THEN) indicates what the imaging target is What is necessary.

  In S165, the fuzzy inference unit 107 calculates an output value corresponding to each fuzzy rule, using each inference value calculated in S64. Since this output value indicates the determination result of what the imaging target is, in S165, the predetermined determination item is determined. The membership function 112 prepared beforehand can be used for calculation of the output value of S165 similarly to S65 of FIG. S166 is the same as S66 of FIG.

  The main pattern of the determination process is represented, for example, as shown in FIG. FIG. 8 is a diagram showing a pattern of determination processing using image information, polarization information, and complexity information. In the pattern of (a) of the figure, an output value is calculated by integrating the determination result based on image information, the determination result based on polarization information, and the determination result based on complexity information. The calculation of the output value is performed by the fuzzy inference unit 107 as described above. The example of FIG. 6 described above corresponds to this pattern.

  In the pattern of (b) of the figure, after making determination based on image information, the determination result based on polarization information and the determination result based on complexity information are integrated to calculate an output value. The example of FIG. 4 described above corresponds to this pattern. In the determination based on the polarization information, image information may be used in addition to the polarization information. The determination based on the complexity information is also the same.

  In the pattern of (c) of the figure, after making a determination based on polarization information, an output value is calculated by integrating a determination result based on image information and a determination result based on complexity information. In this case, instead of the image information determination unit 106, a polarization information determination unit (third determination unit) may be provided to determine a predetermined determination item based on the polarization information. Then, the fuzzy inference unit (fourth determination unit) 107 determines the predetermined determination item based on the image information and the complexity information and the determination result of the polarization information determination unit.

  Although the pattern of (c) in the same figure is difficult to determine with image information and complexity information, it can be suitably applied when determination is possible using polarization information. For example, when an object such as a black object or a sphere is detected from an imaging target, it may be difficult to distinguish whether the object is a plane or a solid in two-dimensional information of the object acquired from a captured image. On the other hand, since three-dimensional shape information is reflected in the polarization information, by using the polarization information, it is possible to detect an object having high possibility of being three-dimensional from the image. By using this information, first, the detection target is roughly determined by the polarization information, and then the determination regarding the detection target can be efficiently performed by making detailed determination using other information.

[Learning]
Since the information processing apparatus 1 includes the learning unit 108, the determination accuracy of the fuzzy inference unit 107 can be improved. The machine learning by the learning unit 108 may be a prior learning performed in advance before the determination by the fuzzy inference unit 107, or may be a posterior learning performed based on the determination result by the fuzzy inference unit 107. Both of these may be used.

  When performing prior learning, a large number of learning data sets in which captured images, polarization images, and correct determination results are associated are prepared. The fuzzy inference unit 107 makes a determination using the captured image and the polarization image, and the learning unit 108 makes an evaluation value according to the difference between the determination result and the correct determination result (the smaller the difference, the higher the evaluation). Calculate Then, the learning unit 108 optimizes the parameters used for the determination by the fuzzy inference unit 107 based on the evaluation value.

  For example, the learning unit 108 may calculate the evaluation value using a predetermined evaluation function. The parameter to be optimized may be, for example, the position of the membership function 112 (input membership function) used to calculate the degree of fitness (for example, a numerical value indicating the center of each fuzzy set). Further, the parameter to be optimized may be the position of the membership function 112 (output membership function) used to calculate the determination result, or both of them.

  When the post learning is performed, the user may input the evaluation of the determination result by the fuzzy inference unit 107. The subsequent processing is the same as prior learning. The determination accuracy of the information processing device 1 can be improved by performing the post learning.

  The machine learning method may be any method that can improve the determination accuracy, and is not limited to the above example. For example, learning using a neural network is also possible. Besides this, for example, learning using genetic algorithm, deep learning, reinforcement learning, or support vector machine can be applied.

Second Embodiment
Other embodiments of the present invention are described below. In addition, about the member which has the same function as the member demonstrated in the said embodiment for convenience of explanation, the same code | symbol is appended, and the description is not repeated. The same applies to the third and subsequent embodiments.

  The imaging target of the present embodiment is a product conveyed on the conveyance line, and the determination item of the present embodiment is whether or not foreign matter is present in the conveyance line. This will be described based on FIG. FIG. 9 is a diagram illustrating an example of an imaging target and an imaging method thereof according to a second embodiment of the present invention.

  In the example of FIG. 9, a plurality of products P are transported on the transport line L in the direction indicated by the outlined arrow in the same drawing, and the camera (imaging device) C is disposed above it. An imaging target of the camera C is a product P. The transport line L is, for example, a belt conveyor, and the product is, for example, a food. The information processing apparatus 1 continuously determines whether or not the foreign matter X is present on the transport line L while the product P is being transported. Further, in the same manner as in FIG. 9, the image pickup target is imaged by a polarization camera to obtain a polarization image. This polarized image is also used for the above determination. As a result, even if the foreign matter X has translucency, such as a transparent film, its presence can be detected without missing.

  Then, when it is determined that the foreign matter is present, the information processing apparatus 1 outputs information indicating that. This makes it possible to prevent the product P mixed with the foreign matter X from being shipped. When the information processing apparatus 1 determines that a foreign object is present, the information processing apparatus 1 may perform processing such as stopping conveyance or turning on a warning lamp to further notify the user of the presence of the foreign object. .

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. In the case where the foreign matter is a translucent film or the like having translucency, the influence of the foreign matter is not easily reflected in the complexity information obtained from the normal captured image. For this reason, S3 may be performed after S4, and in S3, complexity information may be generated from the polarization image.

  In the determination process, as in the example of FIG. 4, first, it may be determined whether or not a foreign object is present from image information, and then final determination may be performed by fuzzy inference using polarization information and complexity information . In this case, the image information determination unit 106 may determine whether or not a foreign object is present from the image information using, for example, an FCM discriminator. Here, when the image information determination unit 106 determines that there is a foreign object, the fuzzy inference unit 107 sets the determination result as the final determination result. On the other hand, when the image information determination unit 106 determines that no foreign object is present, the fuzzy inference unit 107 may make the final determination by fuzzy inference using polarization information and complexity information. In this case, a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, as shown in FIG. 5B) corresponding to complexity information are used. Use. Further, for the final determination, for example, membership functions as shown in (d) of FIG. 5 (however, there are two fuzzy sets: existence and nonexistence of foreign matter) are used.

  Also, as shown in the example of FIG. 6, the final determination may be made by fuzzy inference using image information, polarization information, and complexity information. In addition to this, for example, it is determined by fuzzy inference using complexity information obtained from a captured image and image information whether a foreign object is present or not, and then final determination is performed using polarization information. May be

  In addition, when imaging a polarized light image, you may irradiate the light of wavelengths other than visible light, such as infrared light and an ultraviolet light. The same applies to capturing a normal captured image. Thus, it is possible to detect with high accuracy an imaging target capable of capturing a characteristic image when the light of the specific wavelength is irradiated.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. For example, when the fuzzy inference unit 107 determines the presence or absence of a foreign object, the learning unit 108 may obtain a captured image and a polarization image of the presence of a foreign object (product and foreign object), and Machine learning using captured images and polarization images without foreign matter (product only). When there are many variations in the appearance etc. of the foreign matter that may be mixed, there is a possibility that the machine learning can not be performed efficiently by the captured image and the polarization image with the foreign matter (the product and the foreign matter). In such a case, machine learning may be performed using only the captured image with no foreign substance (product only) and the polarization image, and it may be determined whether there is no foreign substance or not. As described above, the semi-supervised learning in which machine learning is performed using either one of the correct and incorrect learning data can be appropriately used in the third and subsequent embodiments.

Third Embodiment
Further embodiments of the present invention will be described below. The imaging target of the present embodiment is the waste accumulated in the waste pit, and the determination item of the present embodiment is how much the agitation state of the waste is. This will be described based on FIG. FIG. 10 is a diagram illustrating an example of an imaging target and an imaging method thereof according to a third embodiment of the present invention.

  In the example of FIG. 10, the camera (imaging device) C is disposed above the dust pit P where dust is accumulated. The imaging target of the camera C is the dust accumulated in the dust pit P. The waste carried by the waste collection vehicle W is thrown into the waste pit P as illustrated. The waste that has just been thrown in includes those in the waste bag B, and such waste may contain a large amount of water and may be difficult to burn. Therefore, it is necessary to break the waste bag B by carrying out an operation of stirring the waste with a crane (not shown) (specifically, an operation of holding the waste with the crane and lifting it up and releasing it).

  The information processing apparatus 1 continuously determines the stirring degree of each region set in the dust pit P during the operation period of the dust pit P. Further, as in the case of FIG. 10, a polarized image obtained by capturing an image of the object to be captured by a polarized camera is also used for the determination. In this way, it is possible to accurately determine the degree of agitation by recognizing a translucent or transparent garbage bag. Therefore, by using this determination result, it is also possible to streamline or automate the stirring operation.

  FIG. 11 is a view showing an example of a captured image obtained by imaging the inside of the dust pit P and a polarization image. In these images, the left end side is the loading gate side where the waste collection vehicle W throws in the waste, and the right end side is the waste hopper side where the waste is fed into the incinerator. From the captured image shown in FIG. 11 (a), there is a lot of waste that the garbage bag is not broken to the left than the center of the figure, and there is a lot of rubbish that the garbage bag is broken to the right than the center You can see that. This is because the waste bag which was not torn at the time of loading is broken by repetition of the stirring operation of being lifted and separated by a crane, whereby the waste which has come out of the waste bag is transferred to the hopper side. That is, from the captured image shown in (a) of FIG. 11, it can be seen that the degree of agitation on the loading gate side is low and the degree of agitation on the waste hopper side is high.

  In addition, in the area where the waste bag is broken, the waste is separated, so the outline of the waste is more complicated than the area where the waste bag is not broken. That is, the area with high degree of agitation has high complexity, and the area with low degree of agitation has low complexity.

  And (b) of the same figure has shown the polarization direction calculated from the polarization picture which picturized the inside of dust pit P with the same angle of view as (a) of the same figure. In comparison with (a) of the same figure, it is understood that the distribution of the polarization direction is correlated with the distribution of the region where the dust bag is not broken. Because the garbage bag is transparent or translucent, it is difficult to identify whether the garbage bag is broken or not when using a normal captured image, but by using the polarization orientation, the area and the tearing of the garbage bag are broken It is possible to clearly distinguish it from the area that is not

  As described above, since all of the captured image, the complexity, and the polarization direction reflect the agitation state of the dust, the agitation degree calculated based on the image information, the complexity information, and the polarization information is It is a highly reliable value that accurately reflects the agitation of waste.

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. In the determination process, as in the example of FIG. 6, the degree of agitation may be determined by fuzzy inference using image information, polarization information, and complexity information. In this case, the image information determination unit 106 is unnecessary. Also, in this case, the fuzzy inference unit 107 may use a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, FIG. And the membership function (for example, as shown in FIG. 7) corresponding to the image information. The degree of agitation may be calculated by dividing the inside of the waste pit P into a plurality of regions and calculating each of the regions. Further, for the final determination, for example, a membership function as shown in (d) of FIG. 5 is used. However, the fuzzy set in this membership function relates to the degree of agitation. For example, the degree of agitation is low (the waste in that area needs additional agitation before being dumped into the hopper) and the degree of agitation is high (the agitation is sufficient, and the waste in that area can be dumped into the hopper) It is also good.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. For example, the learning unit 108 may perform machine learning using a captured image obtained by capturing the dust pit P and a polarization image. The correct answer in learning, that is, the correct degree of agitation may be set by the user.

Embodiment 4
Further embodiments of the present invention will be described below. The imaging target of the present embodiment is a somatic cell of a human or an animal, and the determination item of the present embodiment is the detection of abnormal cells. This will be described based on FIG. FIG. 12 is a diagram illustrating an example of an imaging target and an imaging method thereof according to a fourth embodiment of the present invention.

  In the example of FIG. 12, the endoscope E captures an image of a cell to be imaged. Cells may include abnormal cell CEX in addition to normal cell CE. The abnormal cell CEX is, for example, a cancer cell. Cancer cells are known to have different polarization characteristics from normal cells, and it is possible to detect cancer cells by using polarization information. Specifically, the cells are irradiated with light having an arbitrary polarization state, and the reflected light is imaged. From the polarization image imaged in this manner, it is possible to specify the rotation direction (left or right) when light having an arbitrary polarization state is reflected by cells. Then, from this rotation direction, it can be determined whether or not cancer cells are contained in the imaged cells (for each of the imaged cells, whether or not the cells are cancer cells). In addition, arbitrary polarization states indicate polarization states, such as linear polarization, circular polarization, elliptical polarization, non-polarization, etc., for example.

  Therefore, in the present embodiment, the polarization image acquisition unit 102 acquires the polarization image as described above, and the polarization information acquisition unit 105 generates polarization information indicating the rotation direction of light having an arbitrary polarization state from the polarization image. Do. In addition, when imaging in a living body, a light source such as an optical fiber is introduced into the living body together with the endoscope. In addition, when cells can be sampled from a living body, the sampled cells may be imaged.

  Further, the information processing apparatus 1 captures an image of the imaging target with an endoscope and obtains a captured image as in FIG. This captured image is also used for the above determination. Thereby, the abnormal cell CEX can be detected with high accuracy. In addition, it becomes possible to detect abnormal cells which are conventionally difficult to detect, such as early cancer cells.

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. In the determination processing, as in the example of FIG. 6, the presence or absence of abnormal cells may be determined by fuzzy inference using image information, polarization information, and complexity information. In this case, the image information determination unit 106 is unnecessary. Also, in this case, the fuzzy inference unit 107 may use a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, FIG. And the membership function (for example, as shown in FIG. 7) corresponding to the image information. Further, for the final determination, for example, a membership function as shown in (d) of FIG. 5 (fuzzy sets are, for example, two abnormal cells and no abnormal cells) is used.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. For example, when the fuzzy inference unit 107 determines the presence or absence of an abnormal cell, the learning data may be data corresponding to the determination item. Machine learning is performed using the captured image and the polarization image of

Fifth Embodiment
Further embodiments of the present invention will be described below. The imaging target of the present embodiment is an inspection object that transmits light, and the determination item of the present embodiment is whether or not there is a defect on at least one of the surface and the inside of the inspection object. This will be described based on FIG. FIG. 13 is a diagram illustrating an example of an imaging target and an imaging method thereof according to a fifth embodiment of the present invention.

  FIG. 13 shows an example in which the inspection object is a silicon wafer S. In this example, the light source L emits light of a wavelength that transmits the silicon wafer S, and of the light, the transmitted light transmitted through the silicon wafer S is imaged by the polarization camera PC. If the polarization information of the light to be irradiated is known, it can be assumed what the polarization information of the transmitted light will be. Then, in the polarization information obtained by actually imaging the silicon wafer S, if there is a portion different from the assumed polarization information, it is possible to identify that there is a defect in the portion. For example, a crack CR is generated in the silicon wafer S of FIG. In the portion of the crack CR, the aspect of transmission and refraction of light is different from that of the normal portion in the silicon wafer S, so that the information processing apparatus 1 detects the portion of the crack CR from the polarization image captured by the polarization camera PC. Can.

  Further, the information processing apparatus 1 captures a silicon wafer S to obtain a captured image. This captured image is also used for the above detection. If the defect in the silicon wafer S can be recognized in appearance, it can be detected from the captured image. Further, even if it is a defect which is difficult to recognize in appearance, such as a crack CR generated in the silicon wafer S or an internal cavity, it can be detected by using a polarization image. Therefore, highly accurate defect detection becomes possible by using a captured image and a polarization image together.

  The inspection object is not limited to the silicon wafer S as long as it has a light transmitting property. Also, the determination items are not limited to the presence or absence of defects. For example, it is also possible to determine the type of defect (crack, chip, cavity, etc.). Furthermore, if a defect is found on the substrate on which the circuit is printed, the extent of the defect (whether the circuit can be recovered excluding the defective portion, a defect that affects product performance, etc.) It is also possible to determine Further, for example, it is also possible to determine the presence or absence of a defect in a wafer on which a pattern is formed (processed wafer). In this case, the presence or absence of a defect can be determined from the difference between the information obtained by imaging the processed wafer (inspection target) and the information based on the designed pattern. Also, if there is a defect, it is also possible to determine the site.

  As a prior art document regarding the detection method of the defect of a silicon wafer, patent 5007979 is mentioned, for example. In this document, light of a wavelength that can be transmitted inside the inspection object is applied to the inspection object to which stress is applied and the inspection object to which stress is not applied, and p polarized light and s of scattered light due to the presence or absence of stress Defects are determined by the difference in polarization. The term "stress" as used herein refers to, for example, stress due to thermal stress or ultrasonic waves.

  However, in the method of the above document, it is necessary to switch the presence or absence of stress on the inspection object, and it takes time for detection. In addition, this method is not suitable for wide-area and early defect detection because it is necessary to change the irradiation position.

  On the other hand, according to the information processing apparatus 1 of the present embodiment, defects in a wider range of the inspection object can be detected in a shorter time as compared with the technique of the above-mentioned document.

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. In the determination process, as in the example of FIG. 6, the degree of agitation may be determined by fuzzy inference using image information, polarization information, and complexity information. In this case, the image information determination unit 106 is unnecessary. Also, in this case, the fuzzy inference unit 107 may use a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, FIG. And the membership function (for example, as shown in FIG. 7) corresponding to the image information. Further, for the final determination, for example, a membership function as shown in (d) of FIG. 5 (a fuzzy set is, for example, two with and without a defect) is used.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. For example, when the fuzzy inference unit 107 determines the presence or absence of a defect, the learning data may be a captured image and a polarization image with a defect, and a captured image without a defect. And machine learning using polarized images.

Sixth Embodiment
Further embodiments of the present invention will be described below. The imaging target of the present embodiment is a line, and the determination item of the present embodiment is the presence or absence of an obstacle on the line. This will be described based on FIG. 14 and FIG. FIG. 14 is a diagram illustrating an example of an imaging target and an imaging method thereof according to a sixth embodiment of the present invention. FIG. 15 is a diagram showing an example of an image captured by the camera C of FIG.

  In FIG. 14, the track R ahead of the traveling direction of the train T, that is, the track R to be passed from now on is imaged by a camera (imaging device) C installed at the head of the train T. The captured image is, for example, as shown in FIG. In this example, cobbles, sleepers, and two tracks R are shown. In addition, a polarization image is captured in the same manner as in FIG. The camera C may be installed at a position at which an image of the track R ahead of the traveling direction of the train T can be captured, and may be installed, for example, in the car of the train T or the like.

  Here, since the upper surface of the line (portion in contact with the wheel) is flat and the material does not change, it is assumed that the polarization information is uniform and the polarization information is changed in the upper surface of the line in the polarization image. Too loose. The control unit 10 of the information processing device 1 includes a line detection unit (area detection unit) that detects a line area (planar area) using this feature. For example, in the polarization image, the line detection unit may detect, as the line area, a continuous area including pixels whose polarization orientation is within a predetermined range and the degree of polarization is equal to or more than a predetermined value. In (b) of FIG. 15, the detected line area is shown in gray. The line detection unit also detects the line area in the captured image by using the detection result of the line area in the polarization image.

  Then, the fuzzy inference unit 107 determines whether there is an obstacle in this line area. As described above, by narrowing down the area for determining the presence or absence of an obstacle on the line, it is possible to perform high-speed and accurate determination. For example, the presence or absence of an obstacle is determined based on an image captured from the train T while traveling, and when it is determined that there is an obstacle, the train T is urgently stopped or the driver is present. It also becomes possible to notify.

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. However, before the determination process, the line detection unit detects an area in which the line is shown in the captured image and the polarization image. In the determination processing, as in the example of FIG. 6, the presence or absence of an obstacle in the line region may be determined by fuzzy inference using image information, polarization information, and complexity information. In this case, the image information determination unit 106 is unnecessary. Also, in this case, the fuzzy inference unit 107 may use a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, FIG. And the membership function (for example, as shown in FIG. 7) corresponding to the image information. Further, for the final determination, for example, a membership function as shown in (d) of FIG. 5 (however, fuzzy sets are two with and without an obstacle) is used.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. Since the learning data may be determined according to the determination item, the learning unit 108 performs machine learning using, for example, the captured image and the polarization image with an obstacle, and the captured image and the polarization image without an obstacle.

Seventh Embodiment
Further embodiments of the present invention will be described below. The imaging target of the present embodiment is a road surface of a road, and the determination items of the present embodiment are the presence or absence of abnormality on the road surface. This will be described based on FIG. 16 and FIG. FIG. 16 is a diagram illustrating an example of an imaging target and an imaging method thereof according to a seventh embodiment of the present invention. FIG. 17 is a diagram showing an example of marking a region where polarization information is characteristic in a polarization image obtained by imaging a road surface.

  In FIG. 16, the camera (imaging device) C installed in the vehicle V images a road surface ahead of the traveling direction of the vehicle V, that is, a road surface on which the vehicle V will pass. In addition, a polarization image is captured in the same manner as in FIG. The camera C may be installed at a position at which an image of the road surface ahead of the traveling direction of the vehicle V can be captured. For example, the camera C may be installed inside the vehicle V or the like.

  In a polarization image obtained by imaging a flat road surface, the change in polarization information is gradual as with the upper surface of the track. The control unit 10 of the information processing device 1 includes a road surface detection unit (region detection unit) that detects a road surface region (planar region) using this feature. The road surface detection unit may detect, as the road surface region, a continuous region including pixels whose polarization orientation is within a predetermined range and whose polarization degree is equal to or more than a predetermined value in the polarization image, as in the line detection unit described above. In (a) of FIG. 17, the detected road surface area R is shown by marking.

  By narrowing down the area for determining the presence or absence of abnormality to the road surface area thus detected, it is possible to perform high-speed and accurate determination. For example, the presence or absence of abnormality is determined based on the image captured from the vehicle V while traveling, and when it is determined that there is an abnormality, the vehicle V is urgently stopped or the driver is notified of an abnormality. It also becomes possible to

  Moreover, in the polarization image which imaged the road surface which abnormality, such as a crack and a depression, has produced, the polarization information of the location which the said abnormality has largely changed from the polarization information in the road surface without the abnormality around that. In (b) of FIG. 17, in the road surface area R, the area where the change in polarization information is equal to or more than the threshold is marked as a crack area CR. As described above, in the polarization image, an abnormal area and an area without abnormality can be clearly identified in the road surface, so that the abnormality in the road surface can be detected by using the polarization information generated from the polarization image.

  And since the road surface where abnormality such as a crack or depression has occurred is different from the road surface without abnormality and the appearance is complicated, the image information and the complexity generated from the captured image obtained by imaging the road surface Road surface abnormalities can also be detected using information. Further, the polarization information is characterized in that it is hard to be affected by disturbances such as how light hits the road surface, and it is easy to detect a transparent body such as glass or ice. Therefore, the presence or absence of an abnormality in the road surface can be determined with high accuracy by determining the presence or absence of an abnormality in the road surface using the polarization information, the image information, and the complexity information in combination. The type of abnormality to be detected is arbitrary, and an abnormality other than a crack or a depression, for example, presence or absence of a falling object or the like may be detected.

  In addition, it is possible to detect a crack which is covered with paint or the like and which is difficult to recognize from the appearance from a polarized image taken from a traveling vehicle. This is because the road surface vibrates under pressure as the vehicle travels, but the mode of vibration differs depending on the presence or absence of a crack, and therefore polarization information of light reflected from the cracked road surface and a crack-free road surface This is because there is also a difference between the polarization information of the reflected light. Therefore, the information processing apparatus 1 uses a plurality of polarized images obtained by continuously imaging the same portion of the road surface vibrating due to the traveling of the vehicle or the like, thereby making the presence or absence of a crack under coating high without peeling off the coating. Accuracy can be determined. The plurality of such polarization images may be different in imaging timing to such an extent that the time change of the polarization state can be grasped. For example, the road surface may be continuously imaged from a traveling vehicle with a polarization camera capable of continuous imaging at high speed, or a plurality of frames may be time-series from moving images captured by the polarization camera from a traveling vehicle It may be extracted.

  Specifically, the polarization information acquisition unit 105 obtains the surface direction (normal vector) of the road surface area detected by the road surface detection unit from each of the plurality of polarization images. The normal vector determined from each of the plurality of polarization images can be said to be polarization information indicating the time-dependent change of the polarization state in the road surface area. On a non-cracked road surface, the change in polarization state due to vibration becomes uniform, but if there is a crack, the vibration causes the change in polarization state to be uneven around the crack. Therefore, the presence or absence of a crack can be determined based on whether or not the change of the normal vector is uniform.

  Therefore, the fuzzy inference unit 107 can determine whether or not there is an abnormality in the road surface area based on the polarization information and at least one of the image information and the complexity information. The fuzzy inference unit 107 may input polarization information indicating time change of the polarization state in the road surface area, or may input a determination result of the presence or absence of a crack using the polarization information. The fuzzy inference unit 107 can cope with various inputs if the fuzzy rules and the membership function according to the inputs are defined.

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. However, before the determination process, the road surface detection unit detects an area where the road surface is shown in the captured image and the polarization image. In the determination process, as in the example of FIG. 6, the presence or absence of an abnormality in the road surface area may be determined by fuzzy inference using image information, polarization information, and complexity information. In this case, the image information determination unit 106 is unnecessary. Also, in this case, the fuzzy inference unit 107 may use a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, FIG. And the membership function (for example, as shown in FIG. 7) corresponding to the image information. Further, for the final determination, for example, a membership function as shown in (d) of FIG. 5 (however, two fuzzy sets with and without abnormality) are used.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. Since the learning data may be determined according to the determination item, the learning unit 108 performs machine learning using, for example, the captured image and the polarization image with abnormality, and the captured image and the polarization image without abnormality.

[Embodiment 8]
Further embodiments of the present invention will be described below. The imaging target of the present embodiment is a bridge, and the determination item of the present embodiment is the presence or absence of an abnormality in the bridge. This will be described based on FIG. FIG. 18 is a diagram showing an example of an imaging target and an imaging method thereof according to an eighth embodiment of the present invention.

  In FIG. 18, the bridge BR is imaged by a camera (imaging device) C. In addition, a polarization image is captured as in FIG.

  In the polarization image obtained by imaging a flat surface in the bridge BR, the change in polarization information is gradual as in the case of the top surface of the line. The control unit 10 of the information processing device 1 includes a bridge detection unit (region detection unit) that detects a bridge region (planar region) using this feature. The bridge detection unit may detect, as the bridge region, a continuous region including pixels whose polarization orientation is within a predetermined range and whose polarization degree is equal to or more than a predetermined value in the polarization image, as in the line detection unit described above. By narrowing down the area for determining the presence or absence of an abnormality to the bridge area thus detected, it is possible to perform high-speed and accurate determination.

  Further, in a polarization image obtained by imaging a bridge in which an abnormality such as a crack or a depression occurs, polarization information of a portion where the abnormality occurs is largely changed from polarization information in a portion where there is no abnormality around it. And the non-abnormal area can be clearly identified. Therefore, by using the polarization information generated from the polarization image, it is possible to detect an abnormality of the bridge. In addition, it is arbitrary what kind of abnormality is to be detected, and an abnormality other than a crack or a depression may be detected.

  And since the place where abnormality, such as a crack and a depression, has occurred is different from the place without abnormality and the appearance is different, and the shape is complicated, the image information and complexity generated from the captured image which imaged the bridge It is also possible to detect bridge anomalies using information. Further, the polarization information is characterized in that it is hard to be affected by disturbances such as how light hits the road surface, and it is easy to detect a transparent body such as glass or ice. Therefore, the presence or absence of abnormality of the bridge can be determined with high accuracy by determining the presence or absence of abnormality of the bridge using polarization information, image information, and complexity information in combination.

  Further, as described above, by using a plurality of polarized images obtained by continuously imaging a vibrating imaging target, it is possible to detect a crack which is covered with painting or the like and which is difficult to recognize from the appearance. Therefore, the information processing apparatus 1 can determine the presence or absence of a crack under coating with high accuracy without removing the coating of the bridge by using such a polarized image.

  Moreover, the presence or absence of the same abnormality can be determined also about arbitrary structures other than a bridge. When it is determined whether or not there is an abnormality in the building, the site is vibrated by applying an external force to the site to be inspected (imaging target) and / or the periphery thereof, and the site is obtained by imaging. A polarized image may be used.

  Besides this, it is also possible to conduct a defect inspection on a metal molded article such as a steel material after processing by the same method, for example. In a vibrating metal molded product, the change in polarization state due to vibration is uniform in the part without cracks and flaws, while the change in polarization state is nonuniform due to vibration in the parts with cracks and flaws. It is for. In addition, this method can also be applied to defect inspection of structural materials such as concrete and stone.

〔Determination process〕
Also in the present embodiment, as in the first embodiment, acquisition of an image and generation of various information are performed according to the flow shown in FIG. 3, and determination processing is performed based on the generated various information. However, before the determination process, the bridge detection unit detects an area in which the bridge is shown in the captured image and the polarization image. In the determination processing, as in the example of FIG. 6, the presence or absence of an abnormality in the bridge area may be determined by fuzzy inference using image information, polarization information, and complexity information. In this case, the image information determination unit 106 is unnecessary. Also, in this case, the fuzzy inference unit 107 may use a membership function (for example, as shown in FIG. 5A) corresponding to polarization information and a membership function (for example, FIG. And the membership function (for example, as shown in FIG. 7) corresponding to the image information. Further, for the final determination, for example, a membership function as shown in (d) of FIG. 5 (however, two fuzzy sets with and without abnormality) are used.

[Learning]
Machine learning in the present embodiment can also be performed in the same manner as in the first embodiment. Since the learning data may be determined according to the determination item, the learning unit 108 performs machine learning using, for example, the captured image and the polarization image with abnormality, and the captured image and the polarization image without abnormality.

[Modification]
In each of the above embodiments, an example has been described in which a predetermined determination item is determined using polarization information, image information, and complexity information that are analysis results of a polarization image and a captured image. However, polarization information, image information, and It is good also as composition of performing predetermined processing according to complexity information. In this case, in the fuzzy inference, a fuzzy rule in which the processing content is defined in the consequent part instead of the determination result may be used.

  For example, in the first embodiment, instead of determining the foreign matter mixed in the small fish, predetermined processing according to the foreign matter mixed in the small fish (for example, notifying the user of the name of the foreign matter to prompt removal) Etc.). Further, for example, in the second embodiment, instead of determining whether or not foreign matter is present in the transport line, processing such as stopping the belt conveyor may be performed when foreign matter is present. The same applies to the third and subsequent embodiments.

  In each of the above-described embodiments, an example has been described in which predetermined determination items are determined using at least two of polarization information, image information, and complexity information by fuzzy inference. Then, in these predetermined determinations, a first determination result based on at least one of the above information and a second determination result based on at least one of the above information are integrated by fuzzy inference to be finally obtained. The result of the judgment is output. However, methods other than fuzzy inference can be applied to the above integration. For example, the first determination result and the second determination result may be weighted and integrated, and the final determination result may be output. Besides this, it is also possible to output a final determination result by, for example, Bayesian estimation.

[Example of software implementation]
The control block of the information processing apparatus 1 (particularly, each unit included in the control unit 10) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software. Good.

  In the latter case, the information processing apparatus 1 includes a computer that executes instructions of a program that is software that realizes each function. The computer includes, for example, one or more processors, and a computer readable recording medium storing the program. Then, in the computer, the processor reads the program from the recording medium and executes the program to achieve the object of the present invention. For example, a CPU (Central Processing Unit) can be used as the processor. As the above-mentioned recording medium, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit or the like can be used besides “a non-temporary tangible medium”, for example, a ROM (Read Only Memory). In addition, a RAM (Random Access Memory) or the like for developing the program may be further provided. The program may be supplied to the computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention.

[Items to be added]
An information processing apparatus according to an aspect of the present invention includes: a complexity information acquisition unit that acquires complexity information indicating a degree of complexity of the shape of the imaging target from an image obtained by imaging the imaging target; An image information acquisition unit that acquires image information indicating at least one of the shape and color of the imaging target from the captured image; a polarization information acquisition unit that acquires polarization information from a polarization image of the imaging object; A determination unit that determines a predetermined determination item on the imaging target based on the degree information, the image information, and the polarization information.

  According to the above configuration, since the determination is performed in consideration of the degree of complexity of the shape of the imaging target, it is possible to improve the determination accuracy compared to the determination based only on the shape of the imaging target. Further, according to the above configuration, since the determination is performed in consideration of the polarization information, it is possible to appropriately determine even determination items that are difficult to determine only from the shape of the imaging target.

  In addition, the determination unit may correlate the correlation between at least two combinations of the image information, the complexity information, and the polarization information, and at least one of a correct determination result and an incorrect determination result of the predetermined determination item. The predetermined determination items may be determined based on the result of machine learning.

  According to the above configuration, since the determination is performed based on the machine-learned result, the determination can be performed with high accuracy. In addition, it is possible to further improve the determination accuracy by repeatedly using the information processing apparatus.

  In addition, the determination unit uses a fuzzy rule in which at least two of the image information, the complexity information, and the polarization information are included in the antecedent, and the determination result of the predetermined determination item is the consequent. The above-described predetermined determination items may be determined by fuzzy inference.

  According to the above configuration, since the determination is performed using fuzzy inference, it is possible to output a determination result appropriately taking into consideration both information having different properties such as image information, complexity information, and polarization information. Further, in the determination using fuzzy inference, the determination result falls within the category of the defined fuzzy set, so stable determination results can be expected. Furthermore, since the correlation uses the rules of fuzzy inference expressed in language, there is an advantage that the intention can be clearly understood.

  Further, the determination unit determines the predetermined determination item based on the image information, the first determination unit, the complexity information and the polarization information, and the determination result of the first determination unit. A third determination unit that includes a second determination unit that determines a predetermined determination item, or determines the predetermined determination item based on the polarization information; the image information and the complexity information; And a fourth determination unit that determines the predetermined determination item based on the determination result of the third determination unit.

  According to the above configuration, after the determination based on the image information is performed, the determination is performed again based on the determination result and the complexity information and the polarization information, or the determination based on the polarization information is performed. Then, the determination is performed again based on the determination result and the complexity information and the image information. Therefore, the accuracy of the determination can be further enhanced than the determination based on the image information.

  Further, the determination unit may determine whether a foreign object is present together with the imaging target. This configuration can be suitably applied to, for example, product inspection of food and the like.

  Further, the imaging target may be dust accumulated in the dust pit, and the determination unit may determine the agitation degree of the dust. According to this configuration, the degree of agitation of the waste can be determined with high accuracy. This determination result can also be used, for example, for automatic operation of a crane that stirs the waste in the waste pit.

  Further, the imaging target may be a cell of a living body, and the polarization information acquisition unit acquires polarization information indicating a rotation direction when light having an arbitrary polarization state is reflected by the cell, and the determination is performed. The unit may determine whether the cell is an abnormal cell. According to this configuration, abnormal cells such as cancer cells can be detected with high accuracy.

  In addition, at least a part of the imaging target may have translucency, and the polarization information acquisition unit acquires polarization information of transmitted light transmitted through the imaging target, and the determination unit is configured to Whether or not there is a defect on at least one of the surface and the inside of the imaging target may be determined. According to this configuration, even if there is a defect inside the imaging target, it can be detected.

  The polarization information acquisition unit acquires the polarization information indicating time change of the polarization state of the imaging target from a plurality of polarization images at different imaging timings, and the determination unit determines the polarization information and the image. Based on at least one of the information and the complexity information, it may be determined whether the imaging target has an abnormality.

  According to the above configuration, it is possible to determine the presence or absence of an abnormality (crack or the like) which is covered with paint or the like and which is difficult to recognize from the appearance. The information processing apparatus may further include an area detection unit that detects a planar area in the polarization image and the image obtained by imaging the imaging target using the polarization information. In this case, the determination unit may determine whether there is an abnormality in the area detected by the area detection unit. As a result, since it is possible to narrow down the area for which it is determined whether or not there is an abnormality in the imaging target to the area detected by the area detection unit, it is possible to quickly determine the presence or absence of an abnormality.

  An information processing method by an information processing apparatus, comprising: complexity information acquiring step of acquiring complexity information indicating a degree of complexity of a shape of the imaging object from an image obtained by imaging the imaging object; and imaging the imaging object An image information acquiring step of acquiring image information indicating at least one of a shape and a color of the imaging target from an image; a polarization information acquiring step of acquiring polarization information from a polarization image obtained by imaging the imaging target; If it is an information processing method including information, the image information, and a determination step of determining a predetermined determination item on the imaging target using the polarization information, the same operation and effect as the information processing apparatus can be obtained.

  The information processing apparatus according to each aspect of the present invention may be realized by a computer. In this case, the computer is operated as each unit (software element) included in the information processing apparatus, and the information processing apparatus is operated as a computer An information processing program to be realized by the present invention and a computer readable recording medium recording the same also fall within the scope of the present invention.

1 Information processing apparatus 103 Image information acquisition unit 104 Complexity information acquisition unit 105 Polarization information acquisition unit 106 Image information determination unit (determination unit / first determination unit)
107 Fuzzy inference unit (judgment unit / second judgment unit, fourth judgment unit)
111 Fuzzy rules

Claims (10)

A complexity information acquisition unit that acquires complexity information indicating the degree of complexity of the shape of the imaging target from an image obtained by imaging the imaging target;
An image information acquisition unit that acquires image information indicating at least one of a shape and a color of the imaging target from an image obtained by imaging the imaging target;
A polarization information acquisition unit that acquires polarization information from a polarization image obtained by imaging the imaging target;
What is claimed is: 1. An information processing apparatus comprising: a determination unit that determines a predetermined determination item on the imaging target based on the complexity information, the image information, and the polarization information.   The determination unit performs machine learning of a correlation between at least two combinations of the image information, the complexity information, and the polarization information and at least one of a correct determination result and an incorrect determination result of the predetermined determination item. The information processing apparatus according to claim 1, wherein the predetermined determination item is determined based on a result of the determination.   The determination unit includes a fuzzy rule that includes at least two of the image information, the complexity information, and the polarization information in the antecedent, and the determination result of the predetermined determination item is the consequent The information processing apparatus according to claim 1, wherein the predetermined determination item is determined by inference. The judgment unit
A first determination unit that determines the predetermined determination item based on the image information;
A second determination unit that determines the predetermined determination item based on the complexity information, the polarization information, and the determination result of the first determination unit, or
A third determination unit that determines the predetermined determination item based on the polarization information;
And a fourth determination unit that determines the predetermined determination item based on the image information, the complexity information, and the determination result of the third determination unit.
The information processing apparatus according to any one of claims 1 to 3, characterized in that: The above imaging target is the waste accumulated in the waste pit,
The information processing apparatus according to any one of claims 1 to 4, wherein the determination unit determines the degree of agitation of the dust. The imaging target is a cell of a living body,
The polarization information acquisition unit acquires polarization information indicating a rotation direction when light having an arbitrary polarization state is reflected by the cell,
The information processing apparatus according to any one of claims 1 to 4, wherein the determination unit determines whether the cell is an abnormal cell. At least a part of the imaging target is translucent.
The polarization information acquisition unit acquires polarization information of transmitted light transmitted through the imaging target,
The information processing apparatus according to any one of claims 1 to 4, wherein the determination unit determines whether or not there is a defect on at least one of the surface and the inside of the imaging target. The polarization information acquisition unit acquires the polarization information indicating time change of the polarization state of the imaging target from the plurality of polarization images different in imaging timing.
The determination unit determines whether or not there is an abnormality in the imaging target based on the polarization information, and at least one of the image information and the complexity information. The information processing apparatus according to any one of the above. An information processing method by an information processing apparatus
A complexity information acquisition step of acquiring complexity information indicating a degree of complexity of the shape of the imaging target from an image obtained by imaging the imaging target;
An image information acquisition step of acquiring image information indicating at least one of a shape and a color of the imaging target from an image obtained by imaging the imaging target;
A polarization information acquisition step of acquiring polarization information from a polarization image obtained by imaging the imaging target;
An information processing method comprising: determining using the complexity information, the image information, and the polarization information to determine predetermined determination items regarding the imaging target.   An information processing program for causing a computer to function as the information processing apparatus according to claim 1, wherein the complexity information acquisition unit, the image information acquisition unit, the polarization information acquisition unit, and the computer function as the determination unit. Information processing program to make it happen.