JP5326881B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  A method for generating a filter or a converter using evolutionary computation such as a genetic algorithm or genetic programming is known (see Non-Patent Document 1). According to the method of generating a filter or a converter using such evolutionary computation, fewer filters or converters having a complicated structure that are optimal for each case and difficult to obtain analytically are obtained. It can be designed with effort and time.

Masayoshi Maebuchi and two others, “Study on Image Filter Design Using Genetic Algorithm” [online], Computer Utilization Education Council, [March 20, 2008 search], Internet <URL: http: //www.ciec. or.jp/event/2003/papers/pdf/E00086.pdf>

  In learning of an image filter, it is common to calculate the fitness of an image filter for the entire area of the learning input image and the target image. However, there may be a case where it is desired to generate an image filter adapted for a part of a region in a captured image in order to adapt to inspection of an object having various shapes.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus, an image processing method, and an image processing program capable of solving the above-described problems. This object is achieved by a combination of features described in the independent claims. The dependent claims define further advantageous specific examples of the present invention.

  In order to solve the above-mentioned problem, in the first aspect of the present invention, an image processing apparatus that selects an adapted image filter adapted for conversion from a learning input image to a target image from a plurality of image filters. A model storage unit that stores a three-dimensional model of an object that is an object of image conversion by the adapted image filter, and a non-inspection region in a captured image obtained by imaging the object is specified based on the three-dimensional model. For each of a plurality of image filters, a weight image generation unit that generates a weight image in which the weight of the non-inspection area is smaller than the weight of other areas in the captured image, and the output image of the image filter and the target Based on the similarity of each of a plurality of image filters, a calculation unit that calculates the similarity by weighting the difference from the image for each region using a weighted image, and a plurality of image frames. The image processing apparatus comprising: a filter selection unit, the selecting the adaptive-image-filter from the filter, as well as an image processing method, and an image processing program concerning the image processing apparatus.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 shows a configuration of an image processing apparatus 10 according to the present embodiment. An example of the image filter 20 which combined the filter component 22 which concerns on this embodiment in series is shown. An example of the image filter 20 in which the filter component 22 according to the present embodiment is combined in a tree structure is shown. An example of a genetic operation performed on the image filter 20 in which the filter components 22 according to the present embodiment are combined in series is shown. An example of the crossover operation performed on the image filter 20 in which the filter component 22 according to the present embodiment is combined in a tree structure is shown. An example of a mutation operation performed on the image filter 20 in which the filter component 22 according to the present embodiment is combined in a tree structure is shown. An example of a method for calculating the similarity of the image filter 20 according to the present embodiment is shown. An example of the processing flow of the image processing apparatus 10 which concerns on this embodiment is shown. An example of a processing flow until the image filter 20 is selected in step S14 of FIG. An example of a weight image update processing flow by the weight image generation unit 43 in step S21 of FIG. 9 is shown. An example of inspection object area | region specification using the designation | designated range of the three-dimensional model which concerns on this embodiment is shown. 2 shows an exemplary hardware configuration of a computer 1900 according to the present embodiment.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows a configuration of an image processing apparatus 10 according to the present embodiment. The image processing apparatus 10 evolves an image filter group including at least one image filter 20 over a plurality of generations based on evolutionary computation. Here, the image processing apparatus 10 identifies a region that is not to be inspected in the captured image using the three-dimensional model, and evolves the image filter 20 based on the image of the region to be inspected. In the present embodiment, each image filter 20 may have a structure in which inputs and outputs of a plurality of filter components that process a learning input image and output a processing result as an output image are combined. Then, the image processing apparatus 10 generates an image filter 20 suitable for converting the learning input image into the target image. The image processing apparatus 10 is realized by a computer as an example.

  The image processing apparatus 10 includes a filter storage unit 34, a learning input image storage unit 36, a model storage unit 56, a specifying unit 54, a filter processing unit 38, an output image storage unit 40, and a target image storage unit 42. A weight image generation unit 43, a calculation unit 44, a filter selection unit 46, a filter update unit 48, and a filter generation unit 50. The filter storage unit 34 stores a plurality of image filters 20 having different configurations.

  The learning input image storage unit 36 stores one or a plurality of learning input images to be converted by the image filter 20. As an example, the learning input image is an image captured in advance by a user and an image generated in advance.

  The model storage unit 56 stores at least one three-dimensional model of an inspection object such as a product or a part that is an object of image conversion by the adapted image filter 20. Here, the three-dimensional model may represent the outer shape of a target object by arranging a large number of two-dimensional polygons such as triangles or quadrilaterals. It may represent the structure of an object. Further, the expression format of the three-dimensional model is not limited to these examples, and may be arbitrary. As an example, the model storage unit 56 stores a three-dimensional model indicating a structure of an inspection target or the like that is preset in the image processing apparatus 10 or newly prepared by the user.

  The specifying unit 54 specifies a non-inspection target region in the captured image obtained by imaging the inspection target based on the three-dimensional model supplied from the model storage unit 56. For example, based on the three-dimensional model, the specifying unit 54 specifies a region in which the target object is not captured in the captured image, that is, a background portion of the captured image, for example, as a non-test target region. Or as an example, the specific | specification part 54 specifies a one part area | region among the target objects in a captured image as a non-inspection area | region based on a three-dimensional model.

  The filter processing unit 38 sequentially acquires the plurality of image filters 20 stored in the filter storage unit 34. The filter processing unit 38 converts each learning input image stored in the learning input image storage unit 36 by each acquired image filter 20 to generate each output image.

  The output image storage unit 40 stores the output image generated by the filter processing unit 38. For example, the output image storage unit 40 stores each output image in association with the converted image filter 20.

  The target image storage unit 42 stores a target image that is a target of an output image generated by converting the learning input image. For example, the target image may be an image generated in advance by the user or prepared by the image processing apparatus 10. Here, the target image may be generated by the user by specifying and extracting a defective portion to be extracted from an image obtained by capturing a sample of the object.

  The weight image generation unit 43 generates a weight image in which the weight of the non-inspection area specified by the specification unit 54 is smaller than the weight of other areas in the captured image. For example, the weight image generation unit 43 generates, as a weight image, an image in which the ratio of increasing the weight in the non-inspection area is smaller than that in the inspection area. In this case, as an example, the weighted image generation unit 43 generates a weighted image that represents a weight based on a value associated with each data area (for example, each pixel) in the captured image.

  Further, the weight image generation unit 43 may update the weight image indicating the weight for each region of the image based on the output image and the target image in at least one generation.

  The weight image generation unit 43 includes a weight storage unit 60 and a weight update unit 62. The weight storage unit 60 stores a weight image. The weight update unit 62 calculates, for example, a pixel-by-pixel difference between at least one output image and the target image, and increases the weight of the region where the difference for each pixel is larger than the weight of the region where the difference is smaller. The weight image stored in the weight storage unit 60 is updated.

  The weight update unit 62 may update the weight image for each generation. Here, for example, the weight updating unit 62 adds a predetermined value to the weight corresponding to data whose difference from the target image in the current generation weight image is larger than the threshold, and the difference from the target image in the current generation weight image. A value smaller than a predetermined value is added to the weight corresponding to data that is less than or equal to the threshold value.

  In this way, the weight update unit 62 generates the next generation weight image based on the current generation weight image. Instead, the weight updating unit 62 adds a predetermined value to the weight corresponding to data in which the difference from the target image is larger than the threshold in the current generation weight image, and the data in which the difference from the target image is equal to or less than the threshold. A predetermined value may be subtracted from the weight corresponding to.

  Note that the weight storage unit 60 may store the weight image before being updated separately from the updated weight image every time the weight image is updated by the weight update unit 62, for example. In this way, the image processing apparatus 10 evolves the image filter 20 by an evaluation in which a region to be inspected is more weighted.

  For each of the plurality of image filters 20 stored in the filter storage unit 34, the calculation unit 44 calculates the difference between the output image of the image filter 20 and the target image for each region using the weight image generated by the weight image generation unit 43. The similarity is calculated by weighting. Here, the similarity is an index value indicating whether or not the output image obtained by converting the learning input image to the target image is similar, and the higher the value, the more similar the learning input image can be converted to the target image. Represents.

  In addition, for each of the plurality of image filters 20, the calculation unit 44 resembles the output image obtained by converting the learning input image corresponding to each of the plurality of imaging positions by the image filter 20 and the target image of the output image. Each degree may be calculated. In this case, the imaging position is obtained by changing the imaging angle with respect to the object, and may be a combination of one or a plurality of camera orientations, directions, heights, imaging distances, and the like.

  The filter selection unit 46 selects at least one adapted image filter 20 from among the plurality of image filters 20 stored in the filter storage unit 34 based on the respective similarities of the plurality of image filters 20. In this case, the filter selection unit 46 preferentially selects the image filter 20 having a higher degree of similarity among the plurality of image filters 20. More specifically, the filter selection unit 46 selects at least one image filter 20 to be left by a method of modeling a natural selection of a living thing.

  For example, the filter selection unit 46 selects at least one image filter 20 by genetic calculation such as elite selection and roulette selection based on the similarity of each of the plurality of image filters 20 stored in the filter storage unit 34. In addition, the filter selection unit 46 selects the content of the plurality of image filters 20 stored in the filter storage unit 34 based on the similarity calculated for each of the plurality of imaging positions for each of the plurality of image filters 20. At least one adapted image filter 20 may be selected. In this case, for example, the filter selection unit 46 preferentially selects the image filter 20 having a larger sum of similarities or weighted sums of the plurality of imaging positions.

  The filter update unit 48 leaves the image filter 20 selected by the filter selection unit 46 among the plurality of image filters 20 stored in the filter storage unit 34, and sets the image filter 20 that has not been selected by the filter selection unit 46. Hesitate. For example, the filter update unit 48 hesitates by deleting the image filter 20 from the filter storage unit 34.

  The filter generation unit 50 generates one or a plurality of image filters 20 that process the learning input image and output the processing result as an output image by genetic processing. The filter generation unit 50 generates a new image filter 20 by genetic operations such as crossover and mutation for at least one image filter 20 remaining in the update processing by the filter update unit 48.

  Such an image processing apparatus 10 includes a conversion process by the filter processing unit 38, a similarity calculation process by the calculation unit 44, a selection process of the image filter 20 by the filter selection unit 46, an update process by the filter update unit 48, and a filter generation unit. The process of generating a new image filter 20 by 50 is repeated a plurality of times (for example, a plurality of generations). Thereby, the image processing apparatus 10 can generate the image filter 20 suitable for converting the learning input image into the target image by using evolutionary calculation.

  FIG. 2 shows an example of the image filter 20 in which the filter components 22 according to the present embodiment are combined in series. FIG. 3 shows an example of the image filter 20 in which the filter component 22 according to the present embodiment is combined in a tree structure.

  The image filter 20 may have a configuration in which a plurality of filter components 22 are combined in series as shown in FIG. The image filter 20 may have a configuration in which a plurality of filter components 22 are combined in a tree structure as shown in FIG. Further, the image filter 20 may have a plurality of output ends with respect to one input end. The image filter 20 may have a configuration having a plurality of input ends and a plurality of output ends.

  The image filter 20 performs a filter calculation process on the received learning input image and outputs an output image. As an example, the image filter 20 is a program that performs an operation on image data. In addition, the image filter 20 may be an arithmetic expression that represents the content of calculation to be performed on the image data.

  Note that the image filter 20 having the structure in which the filter parts 22 are combined in a tree structure is given the same learning input image to each of the end (lowest) filter parts 22 of the tree structure, and the highest level of the tree structure. The output image is output from the filter component 22. Instead of this, such an image filter 20 may be provided with different learning input images for each of the plurality of terminal filter components 22.

  In this embodiment, the filter generation unit 50 generates a new image filter 20 by genetic processing from a plurality of image filters 20 including at least one of a plurality of filter components 22 that respectively convert an input image into an output image. In addition to the plurality of image filters 20, a plurality of image filters 20 are generated. Here, the image filter 20 may have a configuration in which a filter component 22 that is a program for performing an operation on image data is combined. Further, the image filter 20 may have a configuration in which a filter component 22 that is an arithmetic expression representing the content of calculation to be performed on image data is combined.

  Each of the plurality of filter components 22 receives the image data output from the filter component 22 arranged in the previous stage, performs an operation on the received image data, and gives it to the filter component 22 arranged in the subsequent stage. Each of the plurality of filter components 22 may be a program module, an arithmetic expression, or the like, and performs binarization operation, histogram operation, smoothing operation, edge detection operation, morphology operation, and / or frequency on received image data. Unary operations such as space operations (for example, low-pass filtering operations and high-pass filtering operations) may be performed.

  In addition, each of the plurality of filter components 22 performs an average operation, a difference operation, and / or a fuzzy operation (for example, logical sum operation, logical product operation, algebraic sum, algebraic product, limit sum, limit product, Binomial operations such as intense sum and intense product) may be performed.

  FIG. 4 shows an example of a genetic operation performed on the image filter 20 in which the filter components 22 according to this embodiment are combined in series. FIG. 5 shows an example of a crossover operation performed on the image filter 20 in which the filter component 22 according to this embodiment is combined in a tree structure. FIG. 6 shows an example of a mutation operation performed on the image filter 20 in which the filter component 22 according to this embodiment is combined in a tree structure.

  For example, the filter generation unit 50 performs a crossover operation, which is an example of a genetic operation, on two or more image filters 20 to generate two or more new image filters 20. As an example, as illustrated in FIG. 4 and FIG. 5, the filter generation unit 50 converts a part of the filter component group 24 </ b> A of at least one image filter 20 </ b> A that has already been generated into another image filter that has already been generated. New image filters 20E and 20F are generated by replacing at least a part of the filter component group 24B of 20B. The filter component group 24 may be a member in which one or a plurality of filter components 22 are combined.

  For example, the filter generation unit 50 performs a mutation operation, which is an example of a genetic operation, on one image filter 20 to generate a new image filter 20. As an example, as illustrated in FIG. 4 and FIG. 6, the filter generation unit 50 selects a part of the filter component group 24 </ b> C of the already generated one image filter 20 </ b> C as another filter component group 24 </ b> G selected at random. To generate a new image filter 20G.

  Further, the filter generation unit 50 may leave the current generation image filter 20 as the next generation image filter 20 as it is. As an example, the filter generation unit 50 generates a next-generation image filter 20H that includes the configuration of the filter component 22 of the image filter 20D as it is, as shown in FIG.

  The filter selection unit 46 preferentially selects one having a high degree of fitness from one or a plurality of image filters 20 generated by the filter generation unit 50, and makes the appropriate person survive. The filter update unit 48 saves the selected image filter 20 in the filter storage unit 34 and deletes the unselected image filter 20 from the filter storage unit 34.

  FIG. 7 shows an example of a method for calculating the similarity of the image filter 20 according to the present embodiment. In the present embodiment, the calculation unit 44 calculates a similarity indicating how similar the output image generated by converting the learning input image by the image filter 20 and the target image are. This similarity is used as an evaluation value or index indicating that the larger the value is, the more appropriate the image filter 20 that generated the output image is.

  The calculation unit 44 calculates a comparison value by comparing the value of each region of the output image with the value of each region corresponding to the target image, and multiplies the comparison value for each region by the weight specified by the weight image. Then, the calculation unit 44 calculates a value obtained by averaging or summing the comparison values for each region multiplied by the weight for all regions, and outputs the calculated value as the similarity of the output image.

  For example, the calculation unit 44 calculates the difference or ratio between the luminance value for each pixel of the output image and the luminance value of the corresponding pixel of the target image. Then, as an example, the calculation unit 44 multiplies each calculated difference or ratio for each pixel by the corresponding weight specified by the weight image, and sums or averages the difference or ratio for each pixel multiplied by the weight. Thus, the similarity is calculated.

  For example, the calculation unit 44 calculates the similarity by performing an operation represented by the following formula (1). In this case, the sizes of the output image, the target image, and the weight image are the same.

In the formula (1), f represents the similarity. Y _max represents the maximum luminance value. X is a variable indicating the pixel position in the horizontal direction of the image. y is a variable indicating the pixel position in the vertical direction of the image. xmax is a constant indicating the number of pixels in the horizontal direction of the image. ymax is a constant indicating the number of pixels in the vertical direction of the image.

I _weight (x, y) represents the luminance value of the pixel at the (x, y) position in the weighted image. I _target (x, y) represents the luminance value of the pixel at the (x, y) position in the target image. I _filter (x, y) represents the luminance value of the pixel at the (x, y) position in the output image.

  That is, as indicated by the numerator part in the curly braces of Expression (1), the calculation unit 44 calculates the luminance value of the weighted image with respect to the absolute value of the difference between the luminance value of the target image and the luminance value of the output image. The multiplied weighted difference value is calculated for every pixel in the screen, and a total weighted difference value is calculated by adding the calculated weighted difference values for all pixels. Further, as indicated by the denominator part in the curly braces of Expression (1), the calculation unit 44 calculates a total weight obtained by adding the luminance values of the weighted image for all pixels.

  Further, the calculation unit 44 multiplies the division value (in the curly braces of the equation (1)) obtained by dividing the total weighted difference value by the total weight by the reciprocal (1 / Ymax) of the maximum value of the luminance value (normalized value). The second term of the formula (1) is calculated. Then, the calculation unit 44 calculates a value obtained by subtracting the normalized value from 1 as the similarity f. In this way, the calculation unit 44 can calculate the similarity by weighting the difference between the output image and the target image for each region.

  FIG. 8 shows an example of the processing flow of the image processing apparatus 10 according to the present embodiment. First, the image processing apparatus 10 acquires a three-dimensional model that is set in advance in the model storage unit 56 or newly prepared and stored by the user, and is projected onto the learning input image by the specifying unit 54 to be inspected. An outer region is specified (S18).

  Here, as an example, the specifying unit 54 inspects a region (for example, a background region) in which the target object is not captured in the captured image based on a three-dimensional model supplied from the model storage unit 56 or directly designated by user input. Identified as a non-target area. The specifying unit 54 may store the three-dimensional model edited by the user in the model storage unit 56.

  Then, the weight update unit 62 generates a weight image in which the ratio of increasing the weight of the non-inspection area is smaller than the inspection area from the captured image in which the non-inspection area is specified by the specifying unit 54. Further, the weight update unit 62 may generate a weight image for each of a plurality of imaging positions captured by the user. As an example, the weighted image generating unit 43 sets a value corresponding to each pixel (for example, a luminance value) according to each imaging position in the captured image to a ratio that increases the weight of the non-inspection area. A weighted image smaller than the area is generated. Then, the image processing apparatus 10 repeatedly executes each process of step S12 to step S15 a plurality of times (for example, a plurality of generations) (S11, S16).

  In each generation process, the filter generation unit 50 acquires at least one image filter 20 remaining in the update process by the filter update unit 48 remaining from the previous generation. Subsequently, the filter generation unit 50 performs genetic operations such as a crossover operation and a mutation operation on the acquired at least one image filter 20 to replace at least some filter components 22 with other filter components 22. At least one new image filter 20 replaced with is generated (S12). Then, the filter generation unit 50 writes the generated at least one new image filter 20 in the filter storage unit 34. Note that, as an example, the filter generation unit 50 randomly generates or has already generated the image filter 20 as the initial image filter 20 for generating the image filter 20 that converts the learning input image into the output image. Stored in the filter storage unit 34 in advance.

  Subsequently, the filter processing unit 38 generates an output image by filtering the learning input image with each of the plurality of image filters 20 remaining from the previous generation and the plurality of image filters 20 newly generated in step S12. (S13). Thereby, the filter processing unit 38 can generate a plurality of output images corresponding to the plurality of image filters 20.

  Subsequently, the calculation unit 44 weights and compares the output image obtained by converting the learning input image by the image filter 20 and the target image with the weight image generated by the weight image generation unit 43. Here, the calculation unit 44 calculates the degree of similarity between each of the plurality of output images and the target image. Note that the calculation unit 44 calculates the degree of fitness with higher evaluation in accordance with the number of filter components 22, the low processing load, the degree of parallelism, and the like in addition to the similarity or approximation. Also good.

  Then, the filter selection unit 46 selects a plurality of image filters 20 corresponding to an output image having a higher similarity (S14). Note that, in the last generation, the filter selection unit 46 may select one image filter 20 corresponding to one output image having the highest similarity.

  Further, the filter update unit 48 causes the one or more image filters 20 selected in step S14 to remain in the next generation, and the image filter 20 that has generated the output image not selected in step S14 is stored in the filter storage unit 34. By removing from the image filter group at (S15). The filter generation unit 50 may return the newly generated plurality of image filters 20 to the filter storage unit 34 and store them in addition to the already generated plurality of image filters 20.

  The image processing apparatus 10 repeatedly executes the above processing for a plurality of generations (for example, tens of generations or hundreds of generations or more), and executes the processing up to the last generation (for example, the Nth generation, where N is a natural number of 2 or more). After that, the flow is exited (S16). In this way, the image processing apparatus 10 evolves an image filter 20 suitable for converting a captured image obtained by capturing an object having a three-dimensional structure or a learning input image into a target image in an appropriate examination range. It can be generated using calculations.

  FIG. 9 shows an example of a processing flow until the image filter 20 is selected in step S14 of FIG. The image processing apparatus 10 performs the following processing in step S14 shown in FIG.

  First, the weight image generation unit 43 updates the weight image so as to generate an appropriate weight image for each generation (step S21). The weight image update processing by the weight image generation unit 43 will be described in detail later with reference to FIG.

  Subsequently, the calculation unit 44 performs the following step S23 for each of the plurality of output images generated by each of the plurality of image filters 20 stored in the filter storage unit 34 (S22, S24). . In step S23, the calculation unit 44 compares the output image and the target image with weighting according to the weight image generated by the weight image generation unit 43, and calculates the similarity between the output image and the target image. To do. When the calculation unit 44 finishes the process for all of the plurality of output images, the process proceeds to step S25 (S24).

  Subsequently, the filter selection unit 46 selects an output image closer to the target image from the plurality of output images based on the respective similarities of the plurality of output images (S25). As an example, the filter selection unit 46 preferentially selects an output image having a higher degree of similarity among a plurality of output images converted by the plurality of image filters 20. As an example, the filter selection unit 46 selects an output image whose similarity is higher than the reference similarity. In addition, the filter selection unit 46 may select a selection target image within a predetermined range from the top of the similarity. Further, the filter selection unit 46 may randomly select an output image under a condition that is set so that an output image with a higher degree of similarity is selected with a higher probability.

  Then, the filter selection unit 46 selects the image filter 20 that generates the selected output image as the image filter 20 that converts the learning input image into an image similar to the target image (S26). When the process of step S26 is completed, the image processing apparatus 10 ends the flow.

  FIG. 10 shows an example of a weight image update processing flow by the weight image generation unit 43 in step S21 of FIG. The weight image generation unit 43 performs the following steps S31 to S33 in the weight image update process in step S21 shown in FIG.

  For example, the weight update unit 62 includes at least one of a plurality of output images generated by each of the plurality of image filters 20 remaining from the previous generation and the plurality of image filters 20 newly generated in step S12 of FIG. One output image is extracted as an output image for updating the weighted image (S31). Here, as an example, the weight updating unit 62 extracts an output image generated by the image filter 20 having the highest similarity calculated in the previous generation among the plurality of image filters 20. Further, the weight update unit 62 may extract one or a plurality of output images generated by the image filter 20 whose similarity calculated in the previous generation is equal to or greater than a predetermined value. For example, the weight update unit 62 extracts an output image generated by any one of the plurality of image filters 20.

  Subsequently, the weight updating unit 62 calculates a difference for each region between at least one output image selected in step S31 and the target image (S32). As an example, the weight update unit 62 calculates a difference for each pixel between at least one output image selected in step S31 and the target image. When a plurality of output images are selected in step S31, the weight update unit 62, for example, is the largest of a plurality of differences between each of the selected plurality of output images and one target image. One difference or an average value of the plurality of differences is selected for each region (for example, for each pixel).

  Subsequently, the weight update unit 62 changes the weight image stored in the weight storage unit 60, that is, the weight image used in the previous generation process, based on the difference for each region calculated in step S32 (S33). ). More specifically, the weight update unit 62 updates the weight data stored in the weight storage unit 60 so that the weight of the region with the larger difference is larger than the weight of the region with the smaller difference. Note that the weight updating unit 62 may update the weight in advance within a range identified as outside the examination area by the identifying unit 54 without setting the weight to 0 or increasing it.

  In the above, the weight image generation unit 43 can generate a weight image that converts the learning input image into a similar image by the learning target image.

  FIG. 11 shows an example of inspection area specification using a specified range of a three-dimensional model. In this figure, the specifying unit 54 projects a three-dimensional model on the captured image and designates a non-inspection area.

  Based on the three-dimensional model, the specifying unit 54 specifies a region (for example, a background region) in which the target object is not captured in the captured image as a non-inspection region. As an example, the specifying unit 54 projects a three-dimensional model onto a screen according to the field of view of the imaging apparatus, sets an area on which the three-dimensional model is projected as an inspection target area, and an area on which the three-dimensional model is not projected as an inspection target. It may be an outside region. In addition, when the three-dimensional model is projected onto the captured image, the specifying unit 54 specifies, as the non-inspection area, an area in the captured image on which a portion to be excluded from the inspection target on the three-dimensional model is projected. Good. Here, the specifying unit 54 performs inspection such as a part having a surface area less than or equal to a reference of a predetermined part in the three-dimensional model and / or a part where the angle between the normal of the surface and the imaging direction is a predetermined value or more. As such, a location that is not appropriate may be specified as a non-inspection area.

  Further, the specifying unit 54 may specify a non-inspection region in each of a plurality of captured images obtained when an object is imaged from each of a plurality of imaging positions based on a three-dimensional model. In this case, the specifying unit 54 specifies a non-inspection area of a plurality of captured images having different imaging positions by preparing a plurality of screens for projecting the three-dimensional model according to the imaging positions.

  FIG. 12 shows an example of a hardware configuration of a computer 1900 according to this embodiment. A computer 1900 according to this embodiment is connected to a CPU peripheral unit having a CPU 2000, a RAM 2020, a graphic controller 2075, and a display device 2080 that are connected to each other by a host controller 2082, and to the host controller 2082 by an input / output controller 2084. An input / output unit having a communication interface 2030, a hard disk drive 2040, and a DVD drive 2060, and a legacy input / output unit having a ROM 2010, a flexible disk drive 2050, and an input / output chip 2070 connected to the input / output controller 2084 Is provided.

  The host controller 2082 connects the RAM 2020 to the CPU 2000 and the graphic controller 2075 that access the RAM 2020 at a high transfer rate. The CPU 2000 operates based on programs stored in the ROM 2010 and the RAM 2020 and controls each unit. The graphic controller 2075 acquires image data generated by the CPU 2000 or the like on a frame buffer provided in the RAM 2020 and displays it on the display device 2080. Instead of this, the graphic controller 2075 may include a frame buffer for storing image data generated by the CPU 2000 or the like.

  The input / output controller 2084 connects the host controller 2082 to the communication interface 2030, the hard disk drive 2040, and the DVD drive 2060, which are relatively high-speed input / output devices. The communication interface 2030 communicates with other devices via a network. The hard disk drive 2040 stores programs and data used by the CPU 2000 in the computer 1900. The DVD drive 2060 reads a program or data from the DVD 2095 and provides it to the hard disk drive 2040 via the RAM 2020.

  The input / output controller 2084 is connected to the ROM 2010, the flexible disk drive 2050, and the relatively low-speed input / output device of the input / output chip 2070. The ROM 2010 stores a boot program that the computer 1900 executes at startup and / or a program that depends on the hardware of the computer 1900. The flexible disk drive 2050 reads a program or data from the flexible disk 2090 and provides it to the hard disk drive 2040 via the RAM 2020. The input / output chip 2070 connects the flexible disk drive 2050 to the input / output controller 2084 and inputs / outputs various input / output devices via, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like. Connect to controller 2084.

  A program provided to the hard disk drive 2040 via the RAM 2020 is stored in a recording medium such as the flexible disk 2090, the DVD 2095, or an IC card and provided by the user. The program is read from the recording medium, installed in the hard disk drive 2040 in the computer 1900 via the RAM 2020, and executed by the CPU 2000.

  A program installed in the computer 1900 and causing the computer 1900 to function as the image processing apparatus 10 includes a model storage module, a specific module, a filter processing module, a weight image generation module, a calculation module, a filter selection module, and a filter update. A module and a filter generation module. These programs or modules work on the CPU 2000 or the like to make the computer 1900, the model storage unit 56, the specifying unit 54, the filter processing unit 38, the weight image generation unit 43, the calculation unit 44, and the filter selection unit 46. And function as a filter update unit 48 and a filter generation unit 50, respectively.

  The information processing described in these programs is read by the computer 1900, whereby the model storage unit 56, which is a specific means in which the software and the various hardware resources described above cooperate, the specifying unit 54, The filter processing unit 38, the weight image generation unit 43, the calculation unit 44, the filter selection unit 46, the filter update unit 48, and the filter generation unit 50 function. And the specific image processing apparatus 10 according to the intended use is constructed | assembled by implement | achieving the calculation or processing of the information according to the intended use of the computer 1900 in this embodiment by these concrete means.

  As an example, when communication is performed between the computer 1900 and an external device or the like, the CPU 2000 executes a communication program loaded on the RAM 2020 and executes a communication interface based on the processing content described in the communication program. A communication process is instructed to 2030. Under the control of the CPU 2000, the communication interface 2030 reads transmission data stored in a transmission buffer area or the like provided on a storage device such as the RAM 2020, the hard disk drive 2040, the flexible disk 2090, or the DVD 2095, and transmits it to the network. Alternatively, the reception data received from the network is written into a reception buffer area or the like provided on the storage device. As described above, the communication interface 2030 may transfer transmission / reception data to / from the storage device by a DMA (direct memory access) method. Instead, the CPU 2000 transfers the storage device or the communication interface 2030 as a transfer source. The transmission / reception data may be transferred by reading the data from the data and writing the data to the transfer destination 2030 or the storage device.

  In addition, the CPU 2000 reads all or necessary portions from files or databases stored in an external storage device such as the hard disk drive 2040, the DVD drive 2060 (DVD 2095), and the flexible disk drive 2050 (flexible disk 2090). The data is read into the RAM 2020 by DMA transfer or the like, and various processes are performed on the data on the RAM 2020. Then, CPU 2000 writes the processed data back to the external storage device by DMA transfer or the like. In such processing, since the RAM 2020 can be regarded as temporarily holding the contents of the external storage device, in the present embodiment, the RAM 2020 and the external storage device are collectively referred to as a memory, a storage unit, or a storage device. Various types of information such as various programs, data, tables, and databases in the present embodiment are stored on such a storage device and are subjected to information processing. Note that the CPU 2000 can also store a part of the RAM 2020 in the cache memory and perform reading and writing on the cache memory. Even in such a form, the cache memory bears a part of the function of the RAM 2020. Therefore, in the present embodiment, the cache memory is also included in the RAM 2020, the memory, and / or the storage device unless otherwise indicated. To do.

  In addition, the CPU 2000 performs various operations, such as various operations, information processing, condition determination, information search / replacement, etc., described in the present embodiment, specified for the data read from the RAM 2020 by the instruction sequence of the program. Is written back to the RAM 2020. For example, when performing the condition determination, the CPU 2000 determines whether the various variables shown in the present embodiment satisfy the conditions such as large, small, above, below, equal, etc., compared to other variables or constants. When the condition is satisfied (or not satisfied), the program branches to a different instruction sequence or calls a subroutine.

  Further, the CPU 2000 can search for information stored in a file or database in the storage device. For example, in the case where a plurality of entries in which the attribute value of the second attribute is associated with the attribute value of the first attribute are stored in the storage device, the CPU 2000 displays the plurality of entries stored in the storage device. The entry that matches the condition in which the attribute value of the first attribute is specified is retrieved, and the attribute value of the second attribute that is stored in the entry is read, thereby associating with the first attribute that satisfies the predetermined condition The attribute value of the specified second attribute can be obtained.

  The program or module shown above may be stored in an external recording medium. As the recording medium, in addition to the flexible disk 2090 and the DVD 2095, an optical recording medium such as DVD or CD, a magneto-optical recording medium such as MO, a tape medium, a semiconductor memory such as an IC card, and the like can be used. Further, a storage device such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the computer 1900 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

DESCRIPTION OF SYMBOLS 10 Image processing apparatus, 20 Image filter, 22 Filter components, 24 Filter component group, 34 Filter storage part, 36 Learning input image storage part, 38 Filter processing part, 40 Output image storage part, 42 Target image storage part, 43 Weight Image generation unit, 44 calculation unit, 46 filter selection unit, 48 filter update unit, 50 filter generation unit, 54 identification unit, 56 model storage unit, 60 weight storage unit, 62 weight update unit, 1900 computer, 2000 CPU, 2010 ROM 2020 RAM, 2030 communication interface, 2040 hard disk drive, 2050 flexible disk drive, 2060 DVD drive, 2070 input / output chip, 2075 graphic controller, 2080 display device, 2082 host co Controller, 2084 input-output controller, 2090 flexible disk, 2095 DVD

Claims (8)

An image processing apparatus for selecting an adapted image filter adapted for conversion from a learning input image to a target image from a plurality of image filters,
A model storage unit for storing a three-dimensional model of an object to be subjected to image conversion by the adapted image filter;
A specifying unit that specifies a non-inspection region in a captured image obtained by capturing the object based on the three-dimensional model;
A weight image generation unit that generates a weight image in which the weight of the non-inspection area is smaller than the weight of other areas in the captured image;
For each of the plurality of image filters, a calculation unit that calculates the degree of similarity by weighting the difference between the output image of the image filter and the target image for each region using the weighted image;
A filter selection unit that selects the adapted image filter from the plurality of image filters based on the similarity of each of the plurality of image filters;
Equipped with a,
The specifying unit specifies the non-inspection region in each of the plurality of captured images obtained when the target is imaged from each of a plurality of imaging positions based on the three-dimensional model,
The weight image generation unit is an image processing device that generates the weight image for each of the plurality of imaging positions . The image processing device according to claim 1, wherein the specifying unit specifies, as the non-inspection region, a region where the target object in the captured image is not captured based on the three-dimensional model. When the three-dimensional model is projected onto the captured image, the specifying unit defines, as the non-inspection area, an area in the captured image on which a portion to be excluded from the inspection target on the three-dimensional model is projected The image processing apparatus according to claim 1 or 2. The calculation unit calculates, for each of the plurality of image filters, a similarity between an output image obtained by converting the learning input image corresponding to each of the plurality of imaging positions by the image filter and a target image of the output image. Calculate
The filter selection unit selects the adapted image filter from the plurality of image filters based on the similarity calculated corresponding to each of the plurality of imaging positions for each of the plurality of image filters. The image processing apparatus according to any one of claims 1 to 3 . The image processing apparatus according to any one of 4 from claim 1, further comprising a filter generator for generating a plurality of image filter. The filter generation unit generates a new image filter by genetic processing from the plurality of image filters each including a plurality of filter components that respectively convert an input image into an output image, and adds the image filter to the plurality of image filters. 5. The image processing apparatus according to 5 . An image processing method for selecting an adapted image filter adapted for conversion from a learning input image to a target image from a plurality of image filters,
A model storing step for storing a three-dimensional model of an object to be subjected to image conversion by the adapted image filter;
A specifying step of specifying a non-inspection region in a captured image obtained by imaging the object based on the three-dimensional model;
A weight image generation step of generating a weight image in which the weight of the non-inspection area is smaller than the weight of other areas in the captured image;
For each of the plurality of image filters, a calculation step of calculating the similarity by weighting the difference between the output image of the image filter and the target image for each region by the weighted image;
A filter selection step of selecting the adapted image filter from the plurality of image filters based on the similarity of each of the plurality of image filters;
Equipped with a,
The specifying step specifies, based on the three-dimensional model, the non-inspection region in each of the plurality of captured images obtained when the object is imaged from each of a plurality of imaging positions;
The weight image generation step is an image processing method for generating the weight image for each of the plurality of imaging positions . An image processing program for causing a computer to function as an image processing device that selects an adapted image filter adapted for conversion from a learning input image to a target image from a plurality of image filters,
A model storage unit for storing a three-dimensional model of an object to be subjected to image conversion by the adapted image filter;
A specifying unit that specifies a non-inspection region in a captured image obtained by capturing the object based on the three-dimensional model;
A weight image generation unit that generates a weight image in which the weight of the non-inspection area is smaller than the weight of other areas in the captured image;
For each of the plurality of image filters, a calculation unit that calculates the degree of similarity by weighting the difference between the output image of the image filter and the target image for each region using the weighted image;
A filter selection unit that selects the adapted image filter from the plurality of image filters based on the similarity of each of the plurality of image filters;
Equipped with a,
The specifying unit specifies the non-inspection region in each of the plurality of captured images obtained when the target is imaged from each of a plurality of imaging positions based on the three-dimensional model,
The weight image generation unit is an image processing program that functions as an image processing device that generates the weight image for each of the plurality of imaging positions .

Images