JP2007102458A - Method for automatically drawing notice portion by image processing, device therefor and recording medium recording program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To dispense with work for removing an image element except a notice portion from an image including a noise element except the notice portion photographed so as to inspect an abnormal state such as a crack of a concrete structure to visually confirm the abnormal state of the notice portion. <P>SOLUTION: In this method, an objective image wherein the image element except the notice portion is deleted from a known original image is created. Data on an individual as an optimized image processing flow imparting an image most approximate to the objective image obtained by applying evolutionary algorithm to the original image, and image characteristic data showing a characteristic of the known original image are stored as data corresponding to the known image about each of a plurality of original images to form a database. The original image most approximate to an unknown original image in the database is selected by performing comparison with image characteristic data of the unknown original image, and image processing to the unknown image is performed by the image processing flow shown by the optimized individual about the selected original image to obtain an image wherein the notice portion is drawn. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a method and apparatus for automatically drawing a target portion in an image by image processing, and more particularly, a target portion by a digital image processing technique to which an evolutionary algorithm (genetic algorithm, immune algorithm) is applied. The present invention relates to a method and an apparatus for drawing and detecting the image.

  As a practical example of applying a technique for drawing a portion of interest in an image, for example, detecting a deformation such as a crack in a concrete structure can be cited. Concrete structures are the foundation of social and economic development for buildings, transportation and transportation facilities, etc., and are said to be corrosion resistant and highly durable. The performance may be lost due to damage caused by the above. Therefore, the concrete structure is regularly inspected and detailed surveys are performed, and deterioration diagnosis and countermeasures are performed based on the results. At the time of inspection, a method of checking and recording the deformation by visual inspection and percussion is used, but such inspection operation by visual inspection and percussion requires specialized knowledge and skill, and such an expert engineer. As the number of structures to be processed increases and the scale increases, the amount of inspection work and the amount of processing data increase.

  For such inspection work by visual inspection and percussion, proposals have been made to detect cracks in concrete structures by image processing techniques.

In order to inspect cracks by image processing, a method is used in which image data is processed to identify deformations such as cracks in concrete, dirt and scratches, and the deformation is extracted. Some use a function to trace cracks on an image. This is because the operator can measure the length and area of the traced part by emphasizing the deformed part and tracing the sketched part of the deformed part that appears in the image. This is to save labor. The trace work for the operator to reveal the deformation in the image and the work for measuring the length and area of the deformation still remain. Although there is an example using a pattern recognition method, the work until pattern recognition still remains, and it cannot be said that sufficient labor saving is achieved. Such background art is disclosed in the following documents.
"PC cross-linking construction method 2002 version" Vol. 47, no. 1, Jan. 2005 (issued by the Prestressed Concrete Technology Association, pages 81-85) JP 2000-155011 A In JP-A-2004-239870, non-patent document 1 describes the feature of a damaged part such as a crack on an image of a concrete structure using a digital camera, and measures the crack width. However, there is no specific indication of how it can actually be implemented and how much results can be achieved.

  In Patent Document 1, in measuring the position of a target marking such as an alignment mark provided on a printed board for positioning, using a set template and a search area, the type and order of filter processing, and a thinning rate are disclosed. It has been shown that the parameters are adapted to the sample image and the parameters are optimized using a genetic algorithm. However, in the inspection of concrete structures, it is difficult to draw irregularities such as cracks. It is not intended to be applied to image processing.

  In Patent Document 2, in the inspection of a screen defect such as a stain on a liquid crystal display screen, a spatial filter is created and used by automatically calculating a filter configuration value using a genetic algorithm in accordance with the state of the screen defect. It is shown. In this case, a spatial filter that performs image processing of an image including a screen defect is created using a genetic algorithm, but it is intended to evaluate the position, size, number, etc. of scattered image defects. Such a technique cannot be applied to image processing when rendering deformations showing irregular shapes such as cracks in the inspection of concrete structures.

  In the conventional technology, when inspecting deformation such as cracks in a concrete structure, it is inevitable to visually check the captured image for deformation. In order to perform such work on a large number of images including noise elements such as dirt other than the shape, sufficient experience, a lot of labor and time are required. Therefore, when a noise element other than the target portion is included in the image as described above, it has been desired to draw the target portion by removing the image elements other than the target portion.

The present invention has been made to solve the above-mentioned problems, and initializing a plurality of individuals each representing an image processing flow;
Obtaining a fitness value indicating superiority or inferiority when image processing is performed on an image given by an image processing flow represented by each individual for the plurality of individuals using a preset evaluation function;
An optimization process is performed so as to obtain a plurality of individuals having the highest fitness by repeating a process of generating a plurality of new individuals so that the fitness becomes higher based on the evaluation function for the plurality of individuals. To do and
An image processing flow using a plurality of individuals obtained by applying a genetic algorithm consisting of and automatically deleting a target part by deleting image elements other than the target part in the image,
Create an objective image with the image elements other than the part of interest removed from the known original image, and give an image that most closely approximates the target image obtained by applying a genetic algorithm to the original image And generating a plurality of individuals representing the converted image processing flow, generating image feature data indicating features of the known original image, and using the plurality of known data and the image feature data of the plurality of individuals To store data as data corresponding to a plurality of original images to form a database,
Comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Obtaining an image depicting a portion of interest by performing image processing on the unknown image according to an image processing flow representing the optimized individual for the selected original image;
It is made up of.

The present invention also includes initializing a plurality of antibodies each representing an image processing flow;
Obtaining an affinity indicating superiority when image processing is performed on an image given by an image processing flow represented by each individual for the plurality of antibodies using a preset evaluation function;
An optimization process is performed so as to obtain a plurality of antibodies having the highest affinity by repeating a process of generating a plurality of new antibodies so that the affinity becomes higher based on the evaluation function for the plurality of antibodies. To do and
An image processing flow using a plurality of antibodies obtained by applying an immunity algorithm consisting of:
Optimization that creates a target image by deleting image elements other than the part of interest from a known original image, and gives an image that most closely approximates the target image obtained by applying an immunization algorithm to the original image Generating a plurality of antibodies representing the processed image processing flow, generating image feature data indicating characteristics of the known original image, and converting the plurality of antibody data and the image feature data into the plurality of known images Performing a plurality of original images to be stored as corresponding data to form a database;
Comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Obtaining an image depicting a target portion by performing image processing on the unknown image according to an image processing flow represented by the optimized antibody for the selected original image;
You may make it consist of.

  The present invention may be a recording medium on which a program for executing such automatic drawing of the attention portion on a computer is recorded.

The present invention also includes means for capturing an image and extracting image feature data;
Obtaining a plurality of individuals each representing an image processing flow by using an evaluation function set in advance to indicate fitness indicating superiority or inferiority when the image given by the image processing flow represented by each individual is processed; Genetics for performing optimization processing to obtain a plurality of individuals having the highest fitness by repeating a process of generating a plurality of new individuals so that the fitness of the individual is higher based on the evaluation function A means of executing a genetic algorithm;
Create an objective image with the image elements other than the part of interest removed from the known original image, and give an image that most closely approximates the target image obtained by applying a genetic algorithm to the original image And generating a plurality of individuals representing the converted image processing flow, generating image feature data indicating features of the known original image, and using the plurality of known data and the image feature data of the plurality of individuals A database formed by performing a plurality of original images to be stored as data corresponding to
Means for comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Means for performing image processing on the unknown image by an image processing flow representing the optimized individual for the selected original image to obtain an image depicting a target portion;
It is good also as an apparatus which draws the attention part which consists of automatically.

The present invention also includes means for capturing an image and extracting image feature data;
A plurality of antibodies each representing an image processing flow are obtained by using an evaluation function set in advance for an affinity indicating superiority or inferiority when the image given by the image processing flow represented by each antibody is processed, Immunization is performed so that a plurality of antibodies having the highest affinity are obtained by repeating the process of generating a plurality of new antibodies so that the affinity becomes higher based on the evaluation function. Means for executing the algorithm;
Optimization that creates a target image by deleting image elements other than the part of interest from a known original image, and gives an image that most closely approximates the target image obtained by applying an immunization algorithm to the original image Generating a plurality of antibodies representing the processed image processing flow, generating image feature data indicating characteristics of the known original image, and converting the plurality of antibody data and the image feature data into the plurality of known images A database formed by performing a plurality of original images to be stored as corresponding data;
Means for comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Means for performing image processing on the unknown image by an image processing flow represented by the optimized antibody for the selected original image to obtain an image depicting a target portion;
It is good also as an apparatus which draws the attention part which consists of automatically.

  In the method and apparatus of the present invention, the portion of interest in the image may be a crack as a deformation of the concrete structure.

  According to the present invention, an image processing flow obtained by applying an evolutionary algorithm such as a genetic algorithm or an immune algorithm to a plurality of preselected original images is accumulated as a database, and an unknown image is Then, the most similar original image is selected from a plurality of original images, and an unknown image is processed by an image processing flow for the original image to automatically draw a noticed portion in the image such as the deformation of the object. It is possible to obtain a processed image, and it is possible to omit work that requires experience, labor, and time, such as checking a deformation such as a crack of a concrete structure by visual observation, and the work efficiency is remarkably improved.

  The present invention, for example, performs image processing on a digital image obtained by photographing a concrete structure, removes elements that are not cracked, and forms an image in which the crack itself as a target portion is drawn. Of the evolutionary algorithm indispensable for realizing this, the genetic algorithm will be outlined.

[Genetic algorithm] The genetic algorithm (GA) is an algorithm that imitates the adaptation process and evolution mechanism of inheritance and natural selection of natural biological systems, and uses characteristic operations such as natural selection, crossover, and mutation. This is a technique for generating a new individual and finding a practical solution or an optimum solution at high speed.

In GA, biological and genetic concepts are used in correspondence with each other as follows, for example.
Individual: Individual search point environment: Given problem generation: Individual group constituting individual population at a certain time Chromosome: Describes genetic information of individual gene: Basic component of chromosome, whether chromosome is bit string 0 or 1
Reproduction: Generation of offspring (later generation) individuals 淘汰: Process of selecting and inferior inferior individuals Fitness: Evaluation of individuals showing high suitability to the environment Crossover: Method for generating progeny chromosomes in reproduction Mutation: Random changes that occur in genes during reproduction A simple GA (SGA) processing procedure shown in FIG. 1 will be described as a GA processing procedure.
(1) Generation of initial individuals An initial individual population generated from N individuals I _i (1 ≦ i ≦ N) is generated. Also called the initial population. In the normal case where there is no prior knowledge about the existence range of the optimal solution in the target search space, the individual chromosome is set at random.
(2) Calculation of fitness A fitness f (I _i ) is obtained for N individuals I _i (1 ≦ i ≦ N). The fitness indicates how good the individual is. The higher the value, the better the individual. In general, it is determined for each individual using a predetermined evaluation function.
(3) Selection In SGA, a natural selection operation that compensates for a new individual by deceiving an inferior individual based on fitness is expressed by this processing. Duplicate is allowed from N individuals I _i (1 ≦ i ≦ N), and new N individuals are selected. At this time, the probability P (I _i ) that the individual I _i having the fitness f (I _i ) is selected in one trial for selecting the individual is expressed by the following equation.

このようにその個体の適応度が高いほど、個体が選択される確率は適応度が高く、適応度に基づく自然淘汰であると言える。
（４）交叉
適応度が高い個体を次世代に残すとともに、それらをもとにして新しい個体（探索点）を発生させる必要がある。これはＧＡの本質的部分であり、生物が進化するように最適解を表す個体の一部分を持った個体同士が交叉すれば、より最適解に近い個体が得られるという考えに基づいている。ＳＧＡでは選択後のＮ個の個体をランダムに２つずつ選んでペアを作る。そして交叉率（生起確率）に基づいて、そのペアを交叉させるか否かを決定する。交叉させることが決まったペアについては、それらの染色体を交叉させて新しい染色体を作り、それらによってもとの染色体を置き換える処理を行う。交叉法としては、２つの個体の遺伝子型のランダムに選んだ位置に交差点が設定され、その点で遺伝子型を２つに切り後半部分を入れ換えることによって子孫の遺伝子型が生成される１点交叉が典型的であるが、ほかにも交叉位置を２点にした２点交叉、交叉位置を３点以上にした多点交叉、遺伝子ごとにいずれかの親の対応遺伝子をそれぞれ確率ｐ，１−ｐで選択する一様交叉などがある。図２に、（ａ）１点交叉、（ｂ）２点交叉、（ｃ）一様交叉の例を示す。
（５）突然変異
染色体の交叉の後、全個体の染色体の全遺伝子に対して非常に低い生起確率（突然変異率）に基づいて、その遺伝子をランダムに対立遺伝子に置き換える処理が突然変異であり、その例を図２（ｄ）に示す。例えば、ビット列で表現される染色体の場合、そのビットを反転させる。突然変異の目的は
ａ．探索空間内の探索範囲を限定してしまうことの回避
ｂ．局所解からの脱出
初期個体の生成方法によっては、交叉だけでは絶対に生じない遺伝子型が生じることがある。例えば、初期個体群のある遺伝子座の遺伝子が全て０であった場合、通常の交叉ではその遺伝子座の遺伝子を１にすることは不可能である。また、進化過程の早い時期において個体集団が局所解に収束してしまうような場合に、突然変異によって生成された染色体がもとになって局所解から抜け出し、再び最適解を目指すようにすることが可能になる。
（６）終了判定
選択、交叉、突然変異といった操作によって、次世代のＮ個の個体が確定された。これらの適応度を求め、個体集団があらかじめ定められた終了基準を満たしていれば全ての処理を終了し、満たしていなければ選択の処理に戻る。終了のための評価基準としては
ａ．世代数＞しきい値
ｂ．適応度の上昇率＜しきい値
ｃ．個体集団中の最大適応度＞しきい値
ｄ．個体集団の平均適応度＞しきい値
などがある。
〔画像処理の手法〕 本発明では画像処理によりコンクリート構造物のひび割れのような変状の描出を行うのであるが、その場合の画像処理フローについて説明する。なお、コンクリート構造物のひび割れを画像中の注目部分の例として説明するが、本発明による手法は、他に遊離石灰、コンクリートの剥離、コンクリートの剥落、錆汁、鉄筋の露出のような、種々の変状を注目部分としても同様に適用されるものである。

Thus, it can be said that the higher the fitness level of the individual, the higher the fitness of selecting the individual, and natural selection based on the fitness level.
(4) Crossover It is necessary to leave individuals with high fitness in the next generation and generate new individuals (search points) based on them. This is an essential part of GA, and is based on the idea that an individual closer to the optimal solution can be obtained if individuals having a part of the individual representing the optimal solution cross over so that the organism evolves. In SGA, N individuals after selection are randomly selected two by two to make a pair. Based on the crossover rate (occurrence probability), it is determined whether or not to cross the pair. For a pair that has been determined to be crossed, the chromosomes are crossed to create a new chromosome, and the original chromosome is replaced by them. As a crossover method, an intersection is set at a randomly selected position of the genotypes of two individuals, and the genotype of the offspring is generated by switching the genotype into two at that point and replacing the latter half. In other cases, two-point crossover with two crossover positions, multipoint crossover with three or more crossover positions, and the probability of p, 1- There is a uniform crossover selected by p. FIG. 2 shows examples of (a) one-point crossover, (b) two-point crossover, and (c) uniform crossover.
(5) Mutation After the crossover of chromosomes, the process of randomly replacing alleles with alleles based on a very low probability of occurrence (mutation rate) for all genes on the chromosomes of all individuals is mutation. An example thereof is shown in FIG. For example, in the case of a chromosome represented by a bit string, the bit is inverted. The purpose of the mutation is a. Avoiding limiting the search range in the search space b. Escape from local solutions Depending on how the initial individuals are generated, genotypes that never occur by crossover alone may occur. For example, if all genes at a certain locus in the initial population are 0, it is impossible to set the gene at that locus to 1 by normal crossover. In addition, when an individual population converges to a local solution at an early stage of the evolution process, it is necessary to escape from the local solution based on the chromosomes generated by the mutation and aim for the optimal solution again. Is possible.
(6) Termination determination Next generation N individuals were determined by operations such as selection, crossover, and mutation. These fitness values are obtained, and if the individual group satisfies a predetermined termination criterion, all the processes are terminated, and if not, the process returns to the selection process. Evaluation criteria for termination are: a. Number of generations> Threshold value b. Increase rate of fitness <threshold c. Maximum fitness in population> threshold d. Average fitness of individual population> threshold.
[Image Processing Method] In the present invention, the deformation such as a crack of a concrete structure is drawn by image processing. The image processing flow in that case will be described. Note that cracks in the concrete structure will be described as an example of an attention part in the image. However, the method according to the present invention can be applied to various other methods such as free lime, concrete peeling, concrete peeling, rust juice, and rebar exposure. The above-mentioned deformation is also applied to the attention part.

  An image to be subjected to image processing is captured by a digital camera or the like to capture an object such as a concrete structure. The captured image is corrected for the optical characteristics and shooting conditions of the digital camera (lens aberration correction, tilt correction, etc.), and when the images are divided and taken, the image is synthesized. Make corrections.

  Image processing for rendering deformations such as cracks is performed on the geometrically corrected image. It is considered that the image processing procedure for drawing a crack is preferably performed at the stage of preprocessing → binarization → binary image processing. The pre-processing is processing for sharpening cracks that are the target portion in the image, and performs filter processing, adjustment of brightness and contrast, and the like. Filter processing includes a smoothing filter (weighted average filter, median filter, edge preserving filter, etc.) that removes noise contained in the image by blurring the image, and an edge as a boundary between one region and another region in the image There are feature drawing filters (differential filter, sharpening filter, moving difference filter, shading correction, etc.) for detecting the image and representing the feature of the image, and these are used in combination selectively.

  A binarization process is performed in order to separate an attention part such as a crack from the pre-processed image and other parts. Binarization is performed by threshold processing for specific parameters such as luminance. The binary image processing is performed as an improvement process for the fact that the binarized image does not completely separate a target portion such as a crack and other image elements due to the binarization. Are divided into several connected parts, and a label is assigned to each of the parts, isolated point removal, thinning, noise removal by saturation, seam removal, and the like.

[Creation of image processing flow using genetic algorithm]
Consider applying GA to the creation of an image processing flow. This image processing is performed to remove the elements that are not cracked by performing image processing on the digital image obtained by photographing the concrete structure, and to form an image in which the crack itself is depicted. It is necessary to optimize the image processing procedure and parameters for obtaining a target image in which cracks can be drawn from the image by applying GA. Therefore, image processing procedures and parameters are expressed as genes, each individual (image processing flow) is generated by GA, image processing is performed according to the image processing flow of each individual, and cracks can be drawn in the obtained image. If so, the fitness level is set higher, and if the crack is not drawn, the fitness level is set low. Processing flow with high fitness survives to the next generation, and flow with low fitness cannot be adapted to the environment. By crossing the surviving processing flows with high fitness, individuals that can depict more cracks are generated. By repeating this, it is considered that an individual (image processing flow) that can finally draw the most cracks can be obtained. In order to apply GA to the creation of an image processing flow in this way, it is necessary to determine how to code the chromosome and how to evaluate the created image processing flow, which will be described. .

A. Chromosome coding As an image processing procedure for drawing a crack from an image of a concrete structure, a form of “preprocessing” → “binarization processing” → “binary image processing” is widely used. This procedure is used as a processing flow in a broad sense, and a processing method is selected so as not to break the shape of the broad processing flow. In creating an image processing flow using GA, the chromosome is defined as gene information (0, 1 bit string) for the image processing flow in the form of “preprocessing” → “binarization processing” → “binary image processing”. Is done. In FIG. 3, (a) shows an example of an image processing flow, and (b) shows an example of coding for the image processing flow. The shape of a bit string indicating a chromosome is any image processing flow. It depends on whether the shape of is set. In the image processing flow shown in FIG. 3A, the median filter, the sharpening filter, the brightness adjustment, and the contrast adjustment are performed as preprocessing, and the threshold is specified by the direct threshold specification method as binarization processing. Labeling processing is performed as value image processing. Regarding the threshold setting when binarization is performed, an ordinary binary number (binary code) may be used, but if a numerical value having a different value is converted to a genotype, a gray code having a Hamming distance of 1 is used. It is considered advantageous in terms of search ability. For each of these processes, 1 to several bits are assigned in a bit string representing a chromosome. When coding a chromosome, it is important to select a combination of treatments in consideration of what kind of treatments are effective for drawing cracks. In addition, as the gene length (bit length) increases, the time required for optimization increases, so it is also necessary to set the number and combination of practically effective processes.

B. Evaluation of the created image processing flow The process of optimizing the image processing flow for drawing the attention part such as cracks in the image by GA is the chromosome (image processing flow) when considered with the simple GA (SGA) shown in FIG. N individuals with a bit string representing the image) are randomly set, and how well the image processed according to the image processing flow of each individual for the given image is able to depict the attention part such as cracks. It is defined as the fitness level, and it is repeated to select individuals according to the GA so that the fitness level is high, and an optimized image processing flow is obtained.

In GA, the initial generation (0th generation) individuals are randomly created, genetic manipulation (crossover and mutation) is performed on these early generation individuals to create the next generation, and further genetic manipulation is performed on one generation individuals. The optimum individual (optimum image processing flow) is searched by repeating (generation change). here,
a) In GA, the same number of individuals is created for every generation. That is, the same number of individuals exist in each generation until the calculation is completed.
b) In GA, there is a method of elite preservation, and a high fitness (evaluation value) is always left to the next generation.
c) The calculation in GA ends when the generation change (maximum generation number) of the determined number of generations is repeated. The individual of the last generation converges to almost one individual (the individual with the highest fitness), and the elite individual (the one with the highest fitness in the search) is always preserved.

  In this way, an optimal solution can be obtained, but under the given conditions, it may not necessarily reach the optimal solution as a result. In that case, the one with the highest fitness obtained under such conditions is set as a sub-optimal solution.

  In order to optimize an individual (image processing flow) and draw a cracked image satisfactorily, an image processing flow or an evaluation method of an image generated by image processing based on the image processing flow is important. Therefore, in the case of a cracked image as an attention part in the image, considering the evaluation of this image processing flow, FIG. 4 (a) is obtained by performing image processing with an optimal image processing flow for an image obtained by photographing a concrete structure. It is the obtained image which has drawn the crack well. On the other hand, FIG. 4B is an image obtained by performing some other image processing on the same image. In reality, this image includes noise other than cracks in the target image, or the target image. In some cases, the image has a defect, but it can be said that an image processing flow in which such noise and defect are reduced is more preferable. From this, in order to quantify the degree of coincidence with the target image, the noise rate (Noise / Back) and the loss rate (Loss / Cack) in the image are taken, and the evaluation function is calculated.

として、画像処理フローの評価に用いる。好適な画像処理フローの場合、ノイズ率、欠損率がともに０に近く、評価関数は１に近い値となる。
（１）対話型ＧＡによる手法
ここでひび割れ等の注目部分がどの程度良好に描出し得たかについての判断は、人が目視により行い、それ以外の処理をコンピュータ上で行うという最適化処理の形が対話型ＧＡの手法である。どの程度良好に注目部分が描出されているかについての評価、判断は高度な識別力を要することであるのことからすれば、対話型ＧＡは便宜的なものであるが、コンクリート構造物のような多量の画像を処理する場合には、多大の時間を要することになる。
（２）画像中の注目部分の自動抽出
画像中のひび割れ等の注目部分がどの程度良好に描出されているかについての人の判断自体をコンピュータ上で行うことは困難であるが、このような人の判断結果を累計的にデータベースとして蓄積しておき、これを利用して描出の評価、判断をコンピュータ上で行うことにより、注目部分の自動描出がなされる。

Used to evaluate the image processing flow. In the case of a suitable image processing flow, both the noise rate and the loss rate are close to 0, and the evaluation function is close to 1.
(1) Method by interactive GA Here, the judgment of how well the attention part such as cracks could be drawn is a form of optimization processing in which a person makes a visual observation and performs other processing on a computer. Is the interactive GA technique. Interactive GA is convenient because it requires a high level of discriminating power to evaluate and judge how well the target part is drawn, but it does not look like a concrete structure. When processing a large amount of images, it takes a lot of time.
(2) Automatic extraction of the attention part in the image It is difficult for a person to judge how well the attention part such as a crack in the image is depicted on the computer. These judgment results are accumulated as a database in a cumulative manner, and the evaluation and judgment of rendering are performed on a computer using this database, so that the target portion is automatically rendered.

  FIG. 5 shows a flow of storing an image processing flow as a database as a component of a system that automatically draws a target portion in such an image. The equipment for accumulating the image processing flow includes an image capturing function, a function for generating image feature data representing the characteristics of the captured image, and an individual that has been optimized by performing SGA or GA processing shown in FIG. A computer system having a function for obtaining data, image feature data, and storage means for storing individual data is used. The image feature data is data serving as an index for determining the closeness of a plurality of images, and the feature amount of each image is obtained using a statistical texture feature calculation method. This calculation method includes a method known as a density histogram method, a simultaneous normal matrix method, a difference statistic method, and the like. The feature amount calculated for each image by these multiple methods is used to determine the similarity of images. The image similarity data is determined by comparing the data.

  Using a computer system with such functions, GA is applied to a known original image from which a target image that has been fully drawn in advance is obtained, and an individual (image processing flow) is applied to the original image The image obtained in this way is optimized so as to approximate the target image most closely. The solution for the individual obtained as a result is stored in the database as the optimum solution for the specific original image. At the same time, the image feature quantity of the original image is stored in correspondence with the original image as data for specifying the original image. In other words, image feature amount data and data of an optimal solution individual (image processing flow) are stored corresponding to a specific known original image. This operation is repeated for the original images from which a plurality of target images are obtained, and the database is enriched. Here, the enhancement of the database is important, and the more data that is stored, the more images that can be used for automatic rendering.

  FIG. 6 shows a flow of automatically rendering a crack for an unknown image using the image processing flow database formed by the flow shown in FIG. For this purpose, equipment having the same function as that at the time of database formation may be used. The optimum solution of the image processing flow for the stored known image can be extracted using the feature amount data for the image as a key. In order to automatically draw a crack for an unknown image, first, the unknown image is read, image data is taken in, and feature amount data of the image is acquired. Compare this feature quantity data with the feature quantity data of the known image stored in the database, select the feature quantity data that is the most approximate from their similarity, and unknown the corresponding individual (image processing flow) Image processing is performed as an optimal image processing flow for rendering the cracks in the image, and the cracks are rendered. This automatic rendering of cracks is based on the fact that the optimal image processing flow for rendering cracks from crack images is also effective for similar cracked images, and it is possible to extract unknown crack images that are not stored in the database. On the other hand, a crack image can be satisfactorily rendered by applying an image processing flow for an image having the closest image feature amount in the database. When creating a database, it is necessary for a person to set a target image and determine similarity to the target image. However, after the database is formed, a crack can be drawn automatically by the function of the computer.

[Creation of image processing flow using immune algorithm]
When GA is applied in the optimization of the image processing flow, there may be difficulties such as initial convergence in which the diversity of the individual population is lost at the initial stage of solution search. It is conceivable to apply an immune algorithm as an optimization method based on the immune system. Therefore, an outline of the immune system and immune algorithm will be described.

A. Immune System The immune system is a biological defense mechanism that eliminates an antigen by producing an antibody corresponding to the antigen by reconstructing the antibody in order to deal with an unknown antigen that enters the living body. A series of treatments for this antigen is called an immune response and is considered to be realized by the following steps.
Step 1: Recognition of antigen The lymphatic system recognizes an antigen that has entered the body and initiates an immune response.
Step 2: Reconstruction of antibody In order to construct an antibody capable of eliminating the antigen by an antibody-producing cell, the antibody is reconstructed by selection, crossover, mutation, proliferation, etc. to produce an appropriate antibody.
Step 3: Exclusion of antigen by antibody A biological defense is performed by degrading and neutralizing the invading antigen.
Step 4: Memory of Antibody Used for Exclusion Antigen that has been excluded once is quickly excluded by using the memory of the antibody stored in the memory cell.
Step 5: Suppression of antibodies As a result of antibody production in large quantities due to antigen exclusion, the diversity of antibody populations in the body has been lost. To maintain diversity.

The immune system is roughly classified into the following three mechanisms.
1) Primary immune response Antigen exclusion performed on an unknown antigen. Effective antibodies used for antigen exclusion are stored in memory cells.
2) Secondary immune response A faster antigen elimination is realized by the antigen elimination performed on the antigen once eliminated and the use of memory cells acquired at the time of the primary immune response.
3) Inhibition mechanism Diversity is maintained by suppressing the production of too many antibodies, and preparations are made for the invasion of unknown enormous antigens.

  The immune system is in the form as shown in FIG. When the immune system is applied to the automatic depiction of cracks, the primary immune response is equivalent to obtaining an image processing procedure (antibody) to obtain an image most similar to the target image from the original image (antigen). The response is performed by searching the database (memory cells) based on the similarity of image features (similarity of antigen), and using the image processing procedure processed in the past (effective individuals processed in the past) Equivalent to eliminating.

B. Immune Algorithm Immune Algorithm (IA) is a search algorithm based on the production mechanism of diverse antibodies of the immune system and its self-regulatory mechanism. There is a difference in terms between GA and IA as shown in Table 1 regarding the corresponding concept.

ＩＡは、免疫システムの持つ多様性のある抗体の産生機構と、その自己調節機構を模した探索アルゴリズムであり、複数の最適解を探索することが可能である。ＩＡの基本的なフローを図８に示す。それぞれの内容は次のようになる。
（１）初期抗体群の生成
抗原を入力情報として認識させた後、ＧＡと同様に抗体を生成する。このとき、過去に有効な抗体を記憶細胞（データベース）から読み込み、初期抗体群を生成する。記憶細胞が存在しない、もしくは記憶細胞が抗体群の総数に満たないときには、抗体の各遺伝子をランダムに発生することによって初期抗体群を生成する。
（２）抗体間の親和度の計算
全ての抗体について他の全ての抗体との親和度を計算する。抗体間の親和度ａｙ_ｉ，ｙは

IA is a search algorithm that imitates the production mechanism of various antibodies of the immune system and its self-regulation mechanism, and can search for a plurality of optimal solutions. The basic flow of IA is shown in FIG. The contents of each are as follows.
(1) Generation of initial antibody group After an antigen is recognized as input information, an antibody is generated in the same manner as GA. At this time, an antibody effective in the past is read from a memory cell (database) to generate an initial antibody group. When there are no memory cells or the memory cells are less than the total number of antibody groups, an initial antibody group is generated by randomly generating each gene of the antibody.
(2) Calculation of affinity between antibodies Calculate the affinity of all antibodies with all other antibodies. The affinity ay _{i, y} between antibodies is

で与えられる。ここで、Ｈ_ｉ，ｊは抗体ｉと抗体ｊとの距離を表す。抗体間の距離はハミング距離を用いている。
（３）抗体と抗原との親和度の計算
全ての抗体について抗原との親和度（解の評価値）を計算する。
（４）終了判定
ＧＡと同様に、あらかじめ設定した終了条件を満たせば計算を終了する。
（５）記憶細胞、サプレッサー細胞への分化
探索過程で得られた有効な抗体を記憶細胞とサプレッサー細胞に分化させる。この分化された記憶細胞が最適解の候補となる。まず、抗体ｉの濃度ｃ_ｉを

Given in. Here, H _{i, j} represents the distance between the antibody i and the antibody j. Hamming distance is used as the distance between antibodies.
(3) Calculation of affinity between antibody and antigen The affinity (evaluation value of the solution) with the antigen is calculated for all antibodies.
(4) End determination As with the GA, the calculation ends when a preset end condition is satisfied.
(5) Differentiation into memory cells and suppressor cells The effective antibody obtained in the search process is differentiated into memory cells and suppressor cells. These differentiated memory cells are candidates for the optimal solution. First, the concentration c _{i of} antibody i is

によって計算する。ここで、Ｔ_ａｃ１は抗体間の類似度の閾値であり、Ｎは抗体の総数である。つまり、抗体ｉと抗体ｊがＴ_ａｃ１以上であれば、同一種類の抗体とみなす。そして、抗体の濃度ｃ_ｉが閾値Ｔ_ａｃ２を超えた抗体を記憶細胞に分化させる。ただし、同世代に分化する記憶細胞の数は１つ以下とする。なお、記憶細胞の数には上限があり、記憶細胞の総数が上限に達した場合、それまでに保存してきた記憶細胞と分化した記憶細胞との親和度を計算し、その中で最も親和度の高い記憶細胞と入れ換えを行う。ただし、入れ換えは抗原との親和度が選択された記憶細胞よりも新たに分化した記憶細胞の方が高い場合にのみ行う。また、新たに分化した記憶細胞と同じ遺伝子をもつ抗体をサプレッサー細胞にも分化させる。
（６）抗体産生の促進と抑制
抗体産生の促進と抑制は以下の手順によって行われる。
１）抗体の中で抗原との親和度が低いものＮ／２個を自然消滅させる。
２）１）の操作で生き残った各抗体についてサプレッサー細胞との親和度を比較し、親和度の閾値を超えた抗体を消滅させる。
３）２）の操作で生き残った各抗体について次世代に残る期待値を

Calculate by Here, _Tac1 is a threshold of similarity between antibodies, and N is the total number of antibodies. That is, if antibody i and antibody j are equal to or higher than _Tac1 , they are regarded as the same type of antibody. Then, the antibody having the antibody concentration c _i exceeding the threshold T _ac2 is differentiated into a memory cell. However, the number of memory cells that differentiate into the same generation is one or less. There is an upper limit on the number of memory cells, and when the total number of memory cells reaches the upper limit, the affinity between the memory cells that have been stored so far and the differentiated memory cells is calculated, and the highest affinity among them. Replace with high memory cells. However, the replacement is performed only when the newly differentiated memory cell has a higher affinity with the antigen than the selected memory cell. In addition, antibodies having the same gene as newly differentiated memory cells are also differentiated into suppressor cells.
(6) Promotion and suppression of antibody production Promotion and suppression of antibody production are performed by the following procedure.
1) N / 2 antibodies having a low affinity with an antigen are naturally extinguished.
2) Compare the affinity of each antibody surviving in the operation of 1) with suppressor cells, and eliminate the antibody exceeding the affinity threshold.
3) Expectation value remaining in the next generation for each antibody that survived the operation of 2)

で与える。ここで、Ｔ_ａｃ３はサプレッサー細胞との類似度の閾値であり、Ｎ′は生き残っている抗体数、Ｓはサプレッサー細胞の総数、ｋは産生抑制力である。式（６）によって期待値を与えることにより、各抗体において抗原との親和度が高いほど次世代に残る確率が高く、サプレッサー細胞との親和度が高いほど次世代に残る確率が低くなるように設定される。サプレッサー細胞に反応する抗体の産生を抑制することによって同種類の記憶細胞の産生を防ぎ、複数解の探索効率を向上させる。
（７）抗体の産生
（６）における２）のステップによって消滅した抗体に代わる新しい抗体をランダムに発生させた遺伝子によって産生する。次に、現段階で生存しているＮ／２個の抗体から重複を許してＮ／４個のペアを作り、交叉させることによって新たにＮ／２個の抗体を産生する。なお、ペアを作るための抗体の選択は、期待値が高い抗体ほど高確率で選択されたものとする。また、産生したＮ／２個の抗体に対して突然変異を行う。ここで交叉と突然変異はＧＡと同様に行う。

Give in. Here, _Tac3 is a threshold value of similarity to suppressor cells, N ′ is the number of surviving antibodies, S is the total number of suppressor cells, and k is a production inhibitory power. By giving the expected value by equation (6), the higher the affinity with the antigen in each antibody, the higher the probability of remaining in the next generation, and the higher the affinity with the suppressor cell, the lower the probability of remaining in the next generation. Is set. By suppressing the production of antibodies that react with suppressor cells, the production of the same type of memory cells is prevented, and the search efficiency of multiple solutions is improved.
(7) Production of antibody A new antibody is produced by a gene generated at random in place of the antibody that disappeared in step 2) in (6). Next, N / 2 antibodies are produced by allowing N / 4 pairs to be duplicated from the N / 2 antibodies surviving at the present stage and crossing them. In addition, as for the selection of the antibody for making a pair, it is assumed that an antibody having a higher expected value is selected with higher probability. In addition, mutation is performed on the produced N / 2 antibodies. Here, crossover and mutation are performed in the same manner as GA.

The IA flow as described above is applied to a cracked image in the same manner as when GA is used, the image processing flow is optimized, and the crack is automatically drawn. In IA, as in the case of GA, an initial generation (0 generation) individual is randomly created, and genetic manipulation (crossover and mutation) is performed on this initial generation individual to create the next generation. The optimum individual (optimum image processing flow) is searched by repeating the operation (generation change). here,
a) In IA, the same number of individuals is created for every generation. That is, the same number of individuals exist in each generation until the calculation is completed.
b) In the IA, the solution having the highest fitness (evaluation value) among the solutions searched for in the memory cell is always stored (more specifically, a higher number of solutions are stored).
c) The calculation in IA ends when the generation change (maximum generation number) of the determined number of generations is repeated. The individual of the last generation does not converge to one individual (the individual with the highest fitness), but since the individual with the highest fitness is stored in the memory cell, this is the optimal solution (image processing flow) )

  In this way, an optimal solution can be obtained, but under the given conditions, it may not necessarily reach the optimal solution as a result. In that case, the one with the highest fitness obtained under such conditions is set as a sub-optimal solution.

  The IA realizes escape from the local solution, which is a problem of GA, and leaves room for selecting a solution to be executed by preparing a plurality of alternatives in automatic image processing by similar image search. it can.

  In the above, an example of a crack in a concrete structure has been described as a target portion in an image to be drawn. However, when applying the GA or IA algorithm according to the present invention, the target portion is limited to a crack in a concrete structure. The method using the GA or IA algorithm according to the present invention is not limited to the drawing of other noticeable parts such as free lime, concrete peeling, concrete peeling, rust juice, rebar exposure, etc. Can be applied similarly.

It is a figure which shows a simple genetic algorithm. (A) It is a figure which shows the example of one point crossing. (B) It is a figure which shows the example of 2 point crossing. (C) It is a figure which shows the example of uniform crossing. (D) It is a figure which shows the example of a mutation. (A) It is a figure which shows the example of the image processing flow which an individual represents. (B) It is a figure which shows the example of the coding of an image processing flow. (A) It is an example of the image which has drawn the crack favorably. (B) It is an example of the image containing elements other than a crack. It is a figure which shows the flow which produces the database about a known image. It is a figure which shows the flow which draws a crack automatically about an unknown image. It is a figure which shows the form of an immune system. It is a figure which shows the basic flow of an immunity algorithm.

Claims (7)

Initializing multiple individuals, each representing an image processing flow;
Obtaining a fitness value indicating the superiority or inferiority when performing image processing of an image given by the image processing flow represented by each individual for the plurality of individuals using a preset evaluation function;
An optimization process is performed so as to obtain a plurality of individuals having the highest fitness by repeating a process of generating a plurality of new individuals so that the fitness becomes higher based on the evaluation function for the plurality of individuals. To do and
An image processing flow using a plurality of individuals obtained by applying a genetic algorithm consisting of an image element other than the attention part in the image and automatically drawing the attention part,
Create an objective image with the image elements other than the part of interest removed from the known original image, and give an image that most closely approximates the target image obtained by applying a genetic algorithm to the original image And generating a plurality of individuals representing the converted image processing flow, generating image feature data indicating features of the known original image, and using the plurality of known data and the image feature data of the plurality of individuals To store data as data corresponding to a plurality of original images to form a database,
Comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Obtaining an image depicting a portion of interest by performing image processing on the unknown image according to an image processing flow representing the optimized individual for the selected original image;
A method for automatically drawing a feature of interest characterized by comprising Initializing a plurality of antibodies each representing an image processing flow;
Obtaining an affinity indicating superiority when image processing is performed on an image given by an image processing flow represented by each individual for the plurality of antibodies using a preset evaluation function;
An optimization process is performed so as to obtain a plurality of antibodies having the highest affinity by repeating a process of generating a plurality of new antibodies so that the affinity becomes higher based on the evaluation function for the plurality of antibodies. To do and
An image processing flow using a plurality of antibodies obtained by applying an immunity algorithm consisting of a method for automatically drawing a target portion by deleting image elements other than the target portion in the image,
Optimization that creates a target image by deleting image elements other than the part of interest from a known original image, and gives an image that most closely approximates the target image obtained by applying an immunization algorithm to the original image Generating a plurality of antibodies representing the processed image processing flow, generating image feature data indicating characteristics of the known original image, and converting the plurality of antibody data and the image feature data into the plurality of known images Performing a plurality of original images to be stored as corresponding data to form a database;
Comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Obtaining an image depicting a target portion by performing image processing on the unknown image according to an image processing flow represented by the optimized antibody for the selected original image;
A method for automatically drawing a feature of interest characterized by comprising   The method for automatically rendering deformation according to claim 1, wherein the portion of interest in the image is a crack as deformation of a concrete structure.   A recording medium recording a program for executing on a computer a method for automatically rendering a portion of interest according to claim 1. Means for capturing an image and extracting image feature data;
Obtaining a plurality of individuals each representing an image processing flow by using an evaluation function set in advance to indicate fitness indicating superiority or inferiority when the image given by the image processing flow represented by each individual is processed; Genetics for performing optimization processing to obtain a plurality of individuals having the highest fitness by repeating a process of generating a plurality of new individuals so that the fitness of the individual is higher based on the evaluation function A means of executing a genetic algorithm;
Create an objective image with the image elements other than the part of interest removed from the known original image, and give an image that most closely approximates the target image obtained by applying a genetic algorithm to the original image And generating a plurality of individuals representing the converted image processing flow, generating image feature data indicating features of the known original image, and using the plurality of known data and the image feature data of the plurality of individuals A database formed by performing a plurality of original images to be stored as data corresponding to
Means for comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Means for performing image processing on the unknown image by an image processing flow representing the optimized individual for the selected original image to obtain an image depicting a target portion;
A device that automatically draws a target area characterized by comprising Means for capturing an image and extracting image feature data;
A plurality of antibodies each representing an image processing flow are obtained by using an evaluation function set in advance for an affinity indicating superiority or inferiority when the image given by the image processing flow represented by each antibody is processed, Immunization is performed so that a plurality of antibodies having the highest affinity are obtained by repeating the process of generating a plurality of new antibodies so that the affinity becomes higher based on the evaluation function. Means for executing the algorithm;
Optimization that creates a target image by deleting image elements other than the part of interest from a known original image, and gives an image that most closely approximates the target image obtained by applying an immunization algorithm to the original image Generating a plurality of antibodies representing the processed image processing flow, generating image feature data indicating characteristics of the known original image, and converting the plurality of antibody data and the image feature data into the plurality of known images A database formed by performing a plurality of original images to be stored as corresponding data;
Means for comparing the image feature data of an unknown original image to be drawn of interest with the feature data of a plurality of original images stored in the database, and selecting the closest original image in the database;
Means for performing image processing on the unknown image by an image processing flow represented by the optimized antibody for the selected original image to obtain an image depicting a target portion;
A device that automatically draws a target area characterized by comprising   7. The apparatus for automatically rendering a focused portion according to claim 5, wherein the focused portion in the image is a crack as a deformation of the concrete structure.

Images