JP2021165909A - Information processing apparatus, information processing method for information processing apparatus, and program - Google Patents

Abstract

To select learning data effective for learning in good balance to determine a learning subset.SOLUTION: An information processing apparatus has: first evaluation means (202) that receives input of a plurality of pieces of learning data each including input data and training data, and acquires evaluation values of the input data based on the training data by using a model; learning subset determination means (205) that, based on the evaluation values and the order of priority of the attributes of the learning data, selects some pieces of learning data of the plurality of pieces of learning data, and determines a learning subset; and learning means (206) that performs learning of the model by using the determined leaning subset.SELECTED DRAWING: Figure 2

Description

The present invention relates to an information processing device, an information processing method and a program of the information processing device.

In machine learning, it is important to collect data effective for learning, create learning data, and perform learning in order to improve the performance of the model. As a method for creating effective learning data, it is known how to perform recognition processing on learning data using a model in the middle of learning and create learning data by extracting some misrecognized data for learning. Has been done. In this learning method, it is possible to intensively learn the data that the model is not good at erroneously recognizing, so that the learning can be performed more efficiently than learning all the data. However, when there are data of a plurality of attributes in the training data, if this method is applied, only the data of a specific attribute may be collected and the data may be biased. Therefore, it is desirable to study in a well-balanced manner in consideration of attributes. Patent Document 1 describes a method of collecting and learning data of all attributes in a well-balanced manner in consideration of the attributes of the data.

Japanese Unexamined Patent Publication No. 2012-208710

When learning by mixing data of multiple attributes, create training data in a well-balanced manner by considering both the result of examining the data that the model is not good at and the information of which attribute data is to be learned preferentially. This is very important. At present, there is no disclosure of a method of adjusting the data balance and creating learning data in consideration of both of them. Patent Document 1 describes a method of creating training data in consideration of attributes, but does not select training data based on the result of examining data that the model is not good at.

An object of the present invention is to be able to determine a learning subset by selecting learning data effective for learning in a well-balanced manner.

The information processing apparatus of the present invention is a first method in which a plurality of learning data including input data and teacher data are input, and an evaluation value of each of the input data is acquired based on the teacher data by using a model. An evaluation means, a learning subset determining means for selecting a part of the learning data from the plurality of learning data based on the evaluation value and the priority for each attribute of the learning data, and determining the learning subset. It has a learning means for learning the model using the determined learning subset.

According to the present invention, learning data effective for learning can be selected in a well-balanced manner to determine a learning subset.

It is a block diagram which shows the hardware configuration example of an information processing apparatus. It is a block diagram which shows the functional structure example of an information processing apparatus. It is a flowchart which shows the information processing method. It is a flowchart which shows the process of a balance adjustment part. It is a figure for demonstrating data. It is a figure for demonstrating the process of a priority setting part. It is a figure for demonstrating the processing of the weight determination part. It is a figure for demonstrating the processing of a learning subset determination part. It is a figure for demonstrating the processing of the balance adjustment part. It is a block diagram which shows the functional structure example of an information processing apparatus. It is a flowchart which shows the information processing method. It is a figure for demonstrating the process of a priority setting part.

<Embodiment 1>
The first embodiment relates to a method for creating learning data by collecting data effective for learning in a well-balanced manner when learning by mixing data of a plurality of attributes. First, the premise of the first embodiment will be described.

First, the status of the assumed data will be described. The purpose of the first embodiment is to create a model that recognizes data of a specific recognition target with high performance. Then, in the first embodiment, it is assumed that teacher data is added to a part of the data to be recognized, learning is performed using the data, and the remaining data to which the teacher data is not added is recognized. Hereinafter, for ease of distinction, some of the recognition target data used for learning will be referred to as learning recognition target data, and the remaining data will be referred to as test recognition target data.

For the purpose of recognizing the test recognition target data with high performance, the learning recognition target data is important because it is data acquired from the same recognition target data as the test recognition target data. However, the amount of recognition target data for training is very small, and it does not include all the patterns existing in the recognition target data for testing. Therefore, even if the model is created by learning using only the recognition target data for learning, it is difficult to recognize the entire recognition target data for testing with high performance.

In such a situation, in general, the recognition target data for learning is learned by mixing the existing data for the purpose of supplementing the amount of data. The existing data is enormous, and there are many useless data for the above-mentioned purpose, but some of them include data that is effective for learning, specifically, data that exists regardless of the difference in the data to be recognized and that is easily mistaken. There is. A pattern that is easy to make a mistake may also exist in the test recognition target data. Therefore, by learning by mixing the existing data with the learning recognition target data, it may be possible to suppress erroneous recognition as compared with learning using only the learning recognition target data. However, the existing data is data for supplementing the learning recognition target data, and the data to be preferentially learned is the learning recognition target data.

Further, in machine learning, a framework is used in which learning data is created and learned by extracting some learning data of data that the model in the middle of learning is not good at erroneously recognizing. The first embodiment is premised on learning based on this framework. Specifically, a large amount of data is recognized using a model in the middle of learning, an image containing erroneously recognized data is collected to create a subset, and the next learning is performed using the subset. By this method, it is possible to efficiently learn the data that the model in the middle of learning is not good at. In this learning, the result recognized for each learning data is compared with the teacher data, and the data having a large error is used in the next learning. In the first embodiment, based on such learning, instead of learning using all the learning data, the data used for the next learning is extracted and the learning is repeated. Hereinafter, the data set extracted from the training data for the next learning shall be described as a learning subset in order to distinguish it from the population of the training data. As in the above premise, especially when learning using a large amount of data, by extracting data effective for learning from the learning data by learning of the above-mentioned framework and learning, learning proceeds quickly and Recognition performance is improved.

However, in the above-mentioned assumed situation, that is, when the data to be preferentially learned is small in the whole, the learning of the above-mentioned framework does not always succeed. This is because if the data to be preferentially trained is relatively small in the population of training data, the proportion of the data in the learning subset becomes extremely small. In the first embodiment, on the premise of the above, a method of appropriately adjusting the data balance to create a learning subset will be shown.

In addition, the element that determines whether or not to preferentially learn, such as the recognition target data for learning or the existing data, will be described as an attribute hereafter. In the first embodiment, the attribute is information associated with the data, specifically, the content of the data set, and there are two types of data to be recognized for learning and existing data. Further, the balance in the present embodiment refers to the ratio of the number of data for each attribute in the learning subset used for learning. In the following, the process of examining data that the model in the middle of learning is not good at will be described as evaluation.

Next, the outline of the first embodiment will be described. In the first embodiment, the learning subset is created and learned by adjusting the balance of the data in consideration of the evaluation result of the error and the priority of the data determined based on the attributes. Hereinafter, for the sake of simplicity, the priority of data determined based on the attributes will be simply referred to as priority. Considering only the evaluation results, the proportion of high-priority data in the learning subset becomes small. On the other hand, if only the priority is considered, the learning of the data of the pattern that is easy to make a mistake becomes sparse. Therefore, in the first embodiment, a learning subset having an appropriate data balance is created in consideration of both the evaluation result and the priority, and learning is performed. In the first embodiment, the amount of data for each attribute in the learning subset is determined using the evaluation result and the priority, and a part of the data is extracted from all the learning data based on the result to create the learning subset. As a procedure for this, first, a temporary data amount for each attribute in the learning subset is determined based on the evaluation result. Then, based on the priority, the provisional amount of data for each attribute in the learning subset is determined. Next, the amount of data for each attribute in the learning subset is determined by manipulating whether the weight is placed on the provisional data amount based on the evaluation value or the provisional data amount based on the priority. The parameters used for this operation will be referred to as weights hereafter. The weight is specified as a percentage, and the weight is used to weight the temporary data amount based on the evaluation value and the temporary data amount based on the priority, thereby balancing the number of data for each attribute in the learning subset. decide. In the first embodiment, a method of adjusting the balance of data by appropriately weighting according to the progress of learning while repeating learning is shown.

In the first embodiment, a method of adjusting the balance of data based on the evaluation result and the priority to create the learning data will be described by taking the learning of the deformation recognition model in the inspection of the infrastructure structure as an example.

First, the inspection of the infrastructure structure will be described. In the inspection of the wall surface of the infrastructure structure, the inspector records the deformation such as cracks on the concrete wall surface. In the inspection using images, the inspector photographs the wall surface of the structure and creates the inspection result by recording the position and range of the deformation from the captured images. The created inspection results are managed in association with the drawings of the structure together with the images. At this time, the work of discovering and recording all the deformations in the image becomes a difficult work. Therefore, in recent years, machine learning has been used to learn a recognition model for recognizing deformation of a concrete wall surface image to improve the efficiency of image inspection. Hereinafter, the model for recognizing deformation is assumed to be an image recognition model for recognizing cracks from a concrete wall surface image.

In the first embodiment, the wall surface image of a certain structure is used as the recognition target data, and the teacher data is added to a part of the areas. Further, as existing data, it is assumed that wall images of various structures and teacher data are prepared for those wall images, and some data to which teacher data is added among the images of the structures to be recognized. And create training data from existing data. The model trained using the training data is used to perform recognition processing on the test recognition target data, that is, the wall surface image to which the teacher data is not added.

Further, although details will be described later, in the first embodiment, in order to learn a model for recognizing a crack, data composed of a concrete wall surface image including the crack and teacher data indicating the correct position of the crack in the image are provided. deal. Hereinafter, the data to which the teacher data is given will be described as the input data. In the case of the first embodiment, the input data refers to a wall surface image. In addition, data consisting of a pair of input data and teacher data is described as learning data.

FIG. 1 is a block diagram showing a hardware configuration example of the information processing apparatus 100 according to the first embodiment. The information processing device 100 includes a CPU 101, a ROM 102, a RAM 103, an HDD 104, a display unit 105, an operation unit 106, and a communication unit 107.

The CPU 101 is a central processing unit that performs arithmetic operations, logical determinations, and the like for various processes, and controls each component connected to the system bus 108. The ROM (Read-Only Memory) 102 is a program memory, and stores a program for control by the CPU 101 including various processing procedures described later. The RAM (Random Access Memory) 103 is used as a temporary storage area such as a main memory and a work area of the CPU 101. The program memory may be realized by loading the program into the RAM 103 from an external storage device or the like connected to the information processing device 100.

The HDD 104 is a hard disk for storing electronic data and programs. An external storage device may be used to play the same role as the HDD 104. Here, the external storage device can be realized by, for example, a medium (recording medium) and an external storage drive for realizing access to the medium. Examples of such media include a flexible disk (FD), a CD-ROM, a DVD, a USB memory, an MO, a flash memory, and the like. Further, the external storage device may be a server device or the like connected by a network.

The display unit 105 is, for example, a CRT display, a liquid crystal display, or the like, and is a device for displaying an image on a display screen. The display unit 105 may be an external device connected to the information processing device 100 by wire or wirelessly. The operation unit 106 has a keyboard and a mouse, and accepts various operations by the user. The communication unit 107 performs two-way communication by wire or wireless with another information processing device, a communication device, an external storage device, or the like by using a known communication technique.

FIG. 2 is a block diagram showing a functional configuration example of the information processing apparatus 100. The information processing device 100 includes a data storage unit 201, an evaluation unit 202, a priority setting unit 203, a balance adjustment unit 204, a learning subset determination unit 205, a learning unit 206, a model storage unit 207, and weight determination. It has a part 208 and. The model storage unit 207 and the weight determination unit 208 may not be provided.

Each of these functional units is realized by the CPU 101 expanding the program stored in the ROM 102 into the RAM 103, executing the program, and executing processing according to each flowchart described later. Then, the CPU 101 holds the execution result of each process in the RAM 103 or the HDD 104. Further, for example, when hardware is configured as an alternative to software processing using the CPU 101, a calculation unit or a circuit corresponding to the processing of each functional unit described here may be configured.

FIG. 5 is a diagram showing data stored in the data storage unit 201. The data storage unit 201 stores all the learning data 504. The total learning data 504 includes the learning recognition target data 502 created from a part of the recognition target data 501 and the existing data 503. Further, the total learning data 504 is data for learning a model for recognizing a crack in an image, and is composed of a pair of input data which is a concrete wall surface image and teacher data indicating a crack position in the image. .. The teacher data is an image having the same size as the input data, and different values are set for the cracked portion and the background portion, and 1 is stored in the pixel at the cracked position and 0 is stored in the pixel in the background portion. The pair of all training data 504 has an image 506 of the input data of the balloon 505 and an image 507 of the teacher data. The pair of the image 506 of the input data and the image 507 of the teacher data is a set of training data. The information processing device 100 trains a model using a data group obtained by collecting a large number of these pairs. Any machine learning algorithm may be used for learning the model, and for example, an algorithm such as a neural network can be used. There are two types of data attributes, the recognition target data 502 for learning and the existing data 503. The learning recognition target data 502 is a very small amount of data. The existing data 503 is a huge amount of data. Further, as shown in Table 508, the data storage unit 201 associates the attribute information such as the learning recognition target data 502 or the existing data 503 with the input data d ₁ , d ₂ , ... d _n. Store. Input data d ₁ , d ₂ , ... d _n are input data of each learning data of all training data 504.

FIG. 3 is a flowchart showing an information processing method of the information processing apparatus 100. The information processing device 100 extracts a predetermined number of data as a learning subset from all the learning data, and repeats learning a predetermined number of times. In the following description, the description of the process (step) is omitted by adding S at the beginning of each process (step). Since the process of the balance adjusting unit 204 shown in S304 is a process of adjusting the balance of the data used for learning and is an important part, it will be described in more detail with reference to FIG.

First, in S301, the evaluation unit 202 inputs a plurality of learning data from the data storage unit 201. Each learning data is composed of a pair of input data and teacher data. Next, in S302, the evaluation unit 202 evaluates each of the acquired learning data. Here, the evaluation process of the evaluation unit 202 will be described. First, the evaluation unit 202 executes detection processing on the input data (image) using the model in the middle of learning. By the detection process, the evaluation unit 202 outputs an image in which a high score is stored in a pixel having a high possibility of cracking and a low score is stored in a pixel having a low possibility of cracking. The image output here will be referred to as a detection image below. The evaluation unit 202 acquires each evaluation value of the input data based on the teacher data by using the model in the middle of learning. For example, the evaluation value is the number of falsely detected pixels, that is, the number of pixels that does not match the pixel value on the teacher data among all the pixels on the detected image. Specifically, the evaluation unit 202 acquires the difference image by subtracting the teacher data from the binarized image of the detected image. Then, the evaluation unit 202 counts the number of non-zero pixels in the difference image, and sets the count value as the number of false detection pixels. The evaluation unit 202 uses this number of false positive pixels as an evaluation value of certain input data. Data with a high evaluation value indicates that the number of wrong pixels is large, so it can be judged that the data is not good for the model.

The evaluation value is not limited to the number of falsely detected pixels, but may be the number of undetected pixels or the sum of squared errors. The number of undetected pixels is obtained by subtracting a binary image obtained by binarizing the detected image from the teacher data to obtain a difference image, and counting the number of non-zero pixels in the difference image. The error squared sum is obtained by calculating the square of the difference between the pixel value and the corresponding pixel on the teacher data for each pixel of the detected image and calculating the sum of the entire image. The evaluation unit 202 performs recognition processing of the input data, and acquires the sum of squared errors, the number of false detection pixels, or the number of undetected pixels based on the result of the recognition processing and the teacher data as evaluation values.

Specifically, the evaluation unit 202 calculates the evaluation value E by the equation (1). However, n is the total number of pixels in the detected image. y _k is the score of the pixel k of the detected image. t _k is the value of the teacher data of a certain pixel k.

Next, in S303, the priority setting unit 203 sets the priority for each attribute of the learning data. Here, the processing of the priority setting unit 203 will be described. The priority setting unit 203 sets the priority of the input data for each input data based on the attribute. For example, there are two types of attributes, the existing data 503 and the learning recognition target data 502. The recognition target data 502 for learning has a higher priority than the existing data 503. Based on this, the priority setting unit 203 sets the priority of the learning recognition target data 502 to a value higher than the priority of the existing data 503. For example, the priority setting unit 203 sets the priority of the learning recognition target data 502 to 0.9 and sets the priority of the existing data 503 to 0.1. The priority when the attribute is the recognition target data 502 for learning is higher than the priority when the attribute is the existing data 503.

Next, in S304, the balance adjusting unit 204 acquires the evaluation value from the evaluation unit 202, acquires the priority for each attribute from the priority setting unit 203, and based on these evaluation values and priorities, the learning subset. Adjust the balance of the data in. As mentioned above, the balance here is the ratio of the data used for the learning subset for each attribute. Adjusting the balance is equivalent to determining the proportion. The information on the ratio for each attribute is required when the learning subset determination unit 205 later selects the data to be used for learning. As a rough procedure of the balance adjustment unit 204, first, the balance adjustment unit 204 determines the ratio of the provisional data amount for each attribute in the learning subset based on the evaluation value. Subsequently, the balance adjustment unit 204 determines the ratio of the provisional data amount for each attribute in the learning subset based on the priority. Next, the balance adjustment unit 204 weights the ratio of the provisional data amount based on the evaluation value and the ratio of the provisional data amount based on the priority, which is determined by the weight determination unit 208 described later. , Determine the percentage of the amount of data for each attribute in the learning subset.

FIG. 4 is a flowchart showing the details of the process of S304 of FIG. First, in S401, the balance adjustment unit 204 determines the ratio of the provisional learning data amount for each attribute of the learning subset based on the evaluation value. As a procedure, first, the balance adjustment unit 204 sorts the input data in descending order of evaluation value, that is, in descending order of the number of false positive pixels. Then, the balance adjusting unit 204 selects a predetermined number of input data from the one with the highest evaluation value. Next, the balance adjustment unit 204 counts the number of input data for each attribute of the selected input data. For example, suppose that the predetermined number is 10,000, the number of recognition target data 502 for learning is 1000, and the number of existing data 503 is 9000. Next, the balance adjusting unit 204 calculates the ratio between the number of learning recognition target data 502 and the number of existing data 503 from the number of learning recognition target data 502 and the number of existing data 503 counted here. Then, the balance adjustment unit 204 determines the ratio of the provisional data amount for each attribute of the learning subset based on the calculated ratio. Specifically, the balance adjustment unit 204 sets the ratio of the provisional data amount of the recognition target data 502 for learning to 0.1 as the ratio of the provisional data amount determined from the evaluation value based on the ratio of the number of counts described above. Then, the ratio of the provisional data amount of the existing data 503 is set to 0.9. For ease of explanation, the ratios are distributed so that the total is 1.

Next, in S402, the balance adjustment unit 204 determines the ratio of the provisional data amount for each attribute in the learning subset based on the priority. The priority is set by the process of S303 in FIG. 3, the priority of the recognition target data 502 for learning is 0.9, and the priority of the existing data 503 is 0.1. For the sake of simplicity, the balance adjusting unit 204 uses the priority value of the learning recognition target data 502 and the priority value of the existing data 503 as they are, and temporarily uses the learning recognition target data 502 and the existing data 503 in the learning subset. Determined as a percentage of the amount of data in. The processing order of S401 and S402 may be reversed.

Next, the balance adjusting unit 204 weights the ratio of the provisional data amount determined based on the evaluation value in S401 and the ratio of the provisional data amount determined based on the priority in S402. Performs the process of determining the ratio of the amount of data for each attribute in the learning subset. This process is performed by the processes of S403 and S404, but the weighting will be described before the description of these processes. The weighting is an operation of balancing the ratio of the provisional data amount determined by the evaluation value and the priority, and is specifically performed by the equation (2).

i is a subscript that differs for each attribute. Here, the subscript i of the recognition target data 502 for learning is set to 1, and the subscript i of the existing data 503 is set to 2. _Di is the ratio of the amount of data of the attribute i in the learning subset. D _ai is the ratio of the provisional data amount of the attribute i determined from the evaluation value. D _bi is the ratio of the provisional data amount of the attribute i determined by the priority. w _a and w _b correspond to the above weights. w _{a is the weight for the ratio D ai} of the provisional data amount based on the evaluation value. w _b is the weight for the ratio D _bi of the provisional data amount based on the priority.

A method of determining _{the ratio D 1} of the amount of data of the recognition target data 502 for learning of the learning subset will be described. _{In that case, the balance adjusting unit 204 substitutes 0.1 for the ratio D a1} of the provisional data amount based on the evaluation value in the equation (2) from the result of the processing of S401 and S402, and provisionally based on the priority. Substitute 0.9 for the data volume ratio D _b1.

Similarly, a method of determining _{the quantity D 2} of the existing data 503 in the learning subset will be described. _{In that case, the balance adjusting unit 204 substitutes 0.9 for the ratio D a2} of the provisional data amount based on the evaluation value in the equation (2) from the result of the processing of S401 and S402, and provisionally based on the priority. Substitute 0.1 for the data volume ratio D _b2.

The weights w _a and w _b are set so that the sum is 1. The balance adjusting unit 204 balances the provisional data amount determined from the evaluation value and the priority. Based on the above, the description of the process of weighting the flowchart of FIG. 4 will be returned.

In S403, the weight determination unit 208 determines the weights w _a and w _b in the equation (2). A method of determining the weights w _a and w _b based on the result of the evaluation value by the weight determination unit 208 will be described. Specifically, the weight determination unit 208 calculates the sum of the evaluation values of all the input data, and determines the _{weights w a} and w _{b based on the sum.} Since the evaluation value is the total number of false positive pixels of the input data, the total evaluation value is the total number of false positive pixels of the entire input data. More specifically, as shown in FIG. 7, the weight determination unit 208 determines a rule so that the _{weights w a} and w _{b change according to the sum of the evaluation values.} For example, when the sum of the evaluation values is larger than the predetermined value, it can be interpreted that there are many data that are mistaken as a whole. Therefore, the weight determination unit 208 determines the weights w _a and w _b so that the wrong data can be intensively learned. Specifically, the weight determination unit 208 sets the weight so that the weight w _a of the ratio of the provisional data amount based on the evaluation value is larger than _{the weight w b} of the ratio of the provisional data amount based on the priority. In the example of FIG. 7, when the sum of the evaluation values is 50,000 or more, the weight determination unit 208 sets the weight w _a to 0.8 and the weight w _b to 0.2.

On the other hand, when the sum of the evaluation values is smaller than the predetermined value, the weight determination unit 208 does not give a weight to the result of the evaluation value because there is little data to be mistaken as a whole, but puts a weight to the priority. _{The weights w a} and w _b are determined so that a large amount of recognition target data 502 for learning can be collected and learned. Specifically, the weight determination unit 208 sets the weight w _b of the provisional data amount based on the priority to be larger than _{the weight w a} of the provisional data amount based on the evaluation value. In the example of FIG. 7, when the sum of the evaluation values is smaller than the predetermined value, for example, when the sum of the evaluation values is less than 10,000, the weight determination unit 208 sets the weight w _a to 0.2 and _{sets the weight w b} to 0. It is set to 8.

Further, the weight determining unit 208, the sum of the evaluation value may not satisfy any of the above, for example, in the case of less than 50,000 total 10,000 or more evaluation values, to equalize the weights w _a and w _b, the weights w _a and w _{Both of b} are set to 0.5. In this way, the weight determination unit 208 can appropriately determine _{the weights w a} and w _b according to the model in the middle of learning by determining the _{weights w a} and w _{b based on the sum of the evaluation values.}

Subsequently, in S404, the balance adjusting unit 204 acquires the _{weights w a} and w _{b from the weight determining unit 208, and uses the weights w a} and w _b to determine the ratio of the amount of data for each attribute of the learning subset. .. First, the balance adjusting unit 204 weights and adds the ratios D _a1 , D _a2 , D _b1 , and D _{b2 based on the weights w a} and w _{b according} to the equation (2) for learning in the learning subset. calculating a ratio D ₁ of the recognition target data 502 the ratio D ₂ of the existing data 503. For example, in S403, it is determined _{that the weight w a} is 0.2 and the weight w _{b is 0.8. From the ratios D a1} , D _a2 , D _b1 , and D _b2 determined by S401 and S402, the ratio D ₁ is 0.74 and the ratio D ₂ is 0.26. Next, the balance adjusting unit 204 determines the number of data for each attribute from _{the ratios D 1} and D _{2 for each attribute.} The balance adjustment unit 204 _{calculates the number of data for each attribute in the learning subset by multiplying the ratios D 1} and D ₂ of the amount of data by a predetermined number of data in the learning subset. As mentioned above, the number of data in the learning subset is 10,000. In that case, the number of data to be recognized for learning 502 in the learning subset is determined to be 7400, and the number of existing data 503 in the learning subset is determined to be 2600.

The above is the processing content of the balance adjustment of S304. In this way, the balance adjustment unit 204 appropriately balances the data in the learning subset by manipulating the weights of the data collection based on the evaluation value and the data collection based on the priority by using the _{weights w a} and w _b. Adjust to.

Next, in S305, the learning subset determination unit 205 acquires the number of data for each attribute in the learning subset from the balance adjustment unit 204 and determines the learning subset. The processing of the learning subset determination unit 205 will be described below. The learning subset determination unit 205 determines the learning subset by selecting a part of the learning data of all the learning data 504 based on the number of data for each attribute in the learning subset determined by the balance adjustment unit 204. The balance adjusting unit 204 has determined that the number of data to be recognized for learning 502 in the learning subset is 7400, and the number of existing data 503 in the learning subset is 2600. The learning subset determination unit 205 selects learning data so as to match the number of these data, and determines the learning subset.

The learning subset determination unit 205 selects learning data for creating the learning subset based on the evaluation value. The learning subset determination unit 205 determines the learning subset by selecting the learning data in order from the one with the highest evaluation value by the number of data for each attribute determined by the balance adjustment unit 204. The procedure of the process will be described with reference to FIG. In the total learning data 810, the recognition target data 502 for learning is arranged in descending order of evaluation value, and the existing data 503 is arranged in descending order of evaluation value. The learning subset determination unit 205 selects 7400 learning recognition target data 811 from the one with the highest evaluation value among the learning recognition target data 502, and 2600 from the one with the highest evaluation value among the existing data 503. The existing data 812 is selected and the learning subset 820 is determined.

Next, in S306, the learning unit 206 learns the model in the middle of learning by using the learning subset determined by the learning subset determination unit 205. The learning unit 206 stores the trained model in the model storage unit 207 in order to use it as an evaluation value for the next learning.

The above is the flow of one learning. Since the learning unit 206 repeatedly performs learning, in S307, the learning unit 206 determines whether or not the number of times of learning has reached a predetermined number of times. When the number of times of learning reaches a predetermined number of times, the learning unit 206 ends the process. Further, when the number of learnings has not reached the predetermined number, the learning unit 206 returns to S302, and the evaluation unit 202, the priority setting unit 203, the balance adjustment unit 204, the learning subset determination unit 205, and the learning unit 206 , S302 to S306 are repeated a predetermined number of times. The evaluation of S302 is performed using the trained model of S306. As described above, when the information processing apparatus 100 repeats a series of processes while updating the learning model, the sum of the evaluation values changes according to the progress of learning. Therefore, as shown in FIG. 7, the weights are weighted. _{The weights w a} and w _b determined by the determination unit 208 change. Therefore, the information processing apparatus 100 can adjust the data balance while appropriately changing _{the weights w a} and w _b according to the learning of the model at that time.

According to the above method, the information processing apparatus 100 can adjust the balance of data to determine the learning subset based on the result and priority of the evaluation value, and can collect effective learning data in a well-balanced manner and perform learning.

In the above description, the evaluation unit 202 evaluates the entire input data, but may evaluate the data set obtained by extracting a part from the entire input data. As described above, the input data refers to the data to which the teacher data is added among the learning data. In the following, the data set extracted here will be referred to as a learning candidate subset in order to distinguish it from the learning subset. A predetermined number of learning data are randomly extracted from all the learning data, a learning candidate subset is created, and the evaluation unit 202 evaluates the learning candidate subset. In the above description, the learning subset determination unit 205 determines the learning subset from all the learning data, but in this case, the learning subset is determined from the learning candidate subset. This learning candidate subset shall be reselected each time learning is repeated. By performing such learning, it is not necessary to evaluate all the input data each time the learning is performed, so that the processing time can be shortened. Also, if the data to be evaluated is the same each time, the same data may be collected at the top of the evaluation value each time learning is performed, but by creating a learning subset from the learning candidate subset, only the same data can be collected. Can be prevented.

The learning unit 206 may end the learning based _{on the weights w a} and w _b determined by the weight determining unit 208 during the iterative learning. Specifically, the learning unit 206 _{determines that the learning has converged when the change of the weights w a} and w _b becomes smaller than the threshold value even during the learning, and ends the learning. The evaluation unit 202, the priority setting unit 203, the balance adjustment unit 204, the learning subset determination unit 205, and the learning unit 206 repeat the process until the changes in the _{weights w a} and w _{b become smaller than the threshold value.} As a case where this process is effective, the sum of the evaluation values may not exceed a predetermined value as the learning progresses. If learning is continued in this state, when the weight determination unit 208 determines the weight based on the rule shown in FIG. 9, the learning is repeated many times in a state where the weight of data collection based on the priority is large. As a result, the learning of the existing data 503 learned in the early stage is forgotten, and over-learning occurs by selecting and learning only a small amount of the recognition target data 502 for learning. In such a case, the learning unit 206 can prevent these problems by ending the learning when the changes in the _{weights w a} and w _{b have disappeared.}

In the description so far, the balance adjusting unit 204 adjusts the balance _{by using the weights w a} and w _b determined by the weight determining unit 208, but the balance adjusting unit 204 does not use the weight determining unit 208. May perform the process using a predetermined weight. Specifically, the balance adjustment unit 204 determines the weight for each number of learnings, and the weight of data collection based on the evaluation result becomes large in the early stage of learning, and conversely, the weight of data collection based on the priority is increased in the final stage of learning. The weight is gradually changed with each learning so that the weight becomes large. More specifically, the balance adjusting unit 204 sets the weights w _a and w _b of the equation (2) to 0.9 and 0.1, respectively, in the early stage of learning, and sets the weights w _a and w _b of 0, respectively, in the middle stage of learning. It is set to 5 and 0.5, and the weights w _a and w _b are set to 0.1 and 0.9, respectively, at the end of learning. By changing the weight in this way while repeating learning, the balance adjustment unit 204 learns data that is easily mistaken at an early stage, has less false recognition, and is characterized by the characteristics of the recognition target data 502 for learning gradually. It is possible to carry out specialized learning. In addition to the above example, the weight may be set in advance by referring to the weight information when learning was successful in the past.

In the above, the learning subset determination unit 205 selects learning data from all the learning data to create a learning subset, but there are cases where this method cannot be applied. Specifically, the number of learning data of a specific attribute may be small, and the number of learning data determined by the balance adjusting unit 204 may not be prepared. In this situation, the learning subset determination unit 205 increases the number of data by using the data obtained by performing image processing conversion on the input data based on the number of training data for each attribute determined by the balance adjustment unit 204. , It is advisable to use the increased data to determine the learning subset. Image processing transformations include, for example, geometric transformations such as rotation and inversion, color tone transformations, and gradation transformations. By adjusting the number of data by such processing, a learning subset can be created in a well-balanced manner even when the learning data of the attribute to be learned is particularly small.

In the above description, the weight determination unit 208 determines the weights w _a and w _b based on the sum of the evaluation values, but counts the number of erroneous input data and the weights w _a and w based on the result. _b may be determined. In this method, the weight determination unit 208 does not total the amount of false positives on a pixel-by-pixel basis, but counts the number of input data in which the number of false positive pixels is larger than a predetermined value as the falsely identified input data. _{The weight determination unit 208 increases the weight w a of} data collection based on the evaluation value of the equation (2) when the number of erroneous input data is larger than the predetermined value, and increases the weight w a when the number of erroneous input data is less than the predetermined value. _{, Increase the weight w b of} data collection based on the priority of equation (2). When the weight determination unit 208 determines the weights w _a and w _b based on the number of erroneous input data, the calculation cost can be reduced as compared with the determination of the _{weights w a} and w _b based on the sum of the evaluation values.

<Variations of priority setting method>
In the above description, there are two types of attributes, the recognition target data 502 for learning and the existing data 503, and the input data of the recognition target data 502 for learning has a high priority. The scope of application is not limited to this. Here, three types of methods for determining the priority based on the learning recognition target data 502 will be described with reference to examples other than the above as attributes.

The first is a method of determining the priority based on the user operation. In this method, the priority setting unit 203 sets the priority so that the image of the existing data 503 similar to the image to be recognized can be preferentially learned according to the user operation. As a processing flow, first, the priority setting unit 203 displays information on the learning recognition target data 502 on the screen via the display unit 105, and then accepts a selection operation from the user via the operation unit 106. Set the priority based on the user operation. Specifically, as existing data 503, there is a data set that collects only images with lines drawn with chalk on the wall surface, a data set that collects only images of the mold, and a data set that collects only blurred images. do. The attributes are the types of data sets, and are four types: recognition target data 502 for learning, existing data 503 for chokes, existing data 503 for formwork, and existing data 503 for blurring. In the above explanation, the learning recognition target data 502 has a high priority, but here, not only the learning recognition target data 502 but also the existing data 503 includes data to be learned with relatively high priority. Imagine a situation. For example, suppose that a person looks at a part of the recognition target data 502 for learning and recognizes that there is a line drawn with chalk in the image. Since the part drawn with the chalk is linear, it is easy to mistakenly recognize it as a crack when applying the model for crack detection, so the data showing the line drawn with the chalk is the data that should be learned with priority. People judge. Based on this determination, the priority setting unit 203 sets the priority of the learning recognition target data 502 and the existing data 503 of the choke to be high, and sets the priority of the remaining formwork and the existing data 503 of blur to be low. For example, the priorities of the recognition target data 502 for learning, the existing data 503 of the choke, the existing data 503 of the formwork, and the existing data 503 of the blur are 0.5, 0.3, 0.1, and 0.1, respectively. Set as.

A specific example of the user interface for realizing this method is shown in window 610 of FIG. The priority setting unit 203 displays the window 610 of FIG. 6 via the display unit 105. In the window 610, the recognition target data 611 for learning and the information 612 regarding the attributes of the existing data 503 are displayed. By displaying the recognition target data 611 for learning in the window 610, it becomes easy to determine which attribute data is useful for learning. For simplicity, information 612 describes the attributes of choke, formwork, and blur as A, B, and C. Information 612 has a check column for each attribute of the existing data 503. Then, the operation unit 106 causes the user to operate the mouse pointer 613. When the user puts the mouse pointer 613 on the check field and clicks it, the display unit 105 displays the check in the check field. Input data having a high priority is determined by such processing of the display unit 105 and the operation unit 106. The example of window 610 shows an image when the user checks the chalk check box. Although FIG. 6 shows an example in which only the attribute name is described as the attribute information of the existing data 503, an image sample of each attribute may be displayed. Displaying a sample image also makes it easier for the user to intuitively determine the input data that should be given higher priority.

Priority setting method The second variation is a method of determining the priority based on the commonality of attributes. Here, it is assumed that there is a data set of the recognition target data 502 for learning, the existing data 503 of the structure S1, the existing data 503 of the structure S2, and the existing data 503 of the structure S3. In addition, the data storage unit 201 stores information on a plurality of attributes for each data set. In the examples so far, the type of data set is used as an attribute, but here, a plurality of more detailed data information is used as an attribute. Specifically, as in the example shown in FIG. 12, the type of structure, the wall surface state, and the shooting conditions are stored as attributes for each data set. The priority setting unit 203 collates the attributes of the learning recognition target data 502 with the attributes of other data sets, and sets the priority of the data set having the common attributes to be high. In the example of FIG. 12, for each data set, if it corresponds to each attribute, it is described as ◯, and if it does not correspond, it is described as ×. Here, the image of the target structure for which cracks are to be detected is used as the learning recognition target data 502. For example, focusing on the attribute of the type of the structure, the recognition target data 502 for learning, the existing data 503 of the structure S1, and the existing data 503 of the structure S2 are the learning data of the image of the bridge, and the structure S3. The existing data 503 of the above is the training data of the image of the tunnel. When determining high-priority data, select the data with the highest number of attribute matches. At this time, attributes that must always match may be set in advance. For example, assuming that the structure type is an important attribute, first select a dataset that matches at least the structure type attribute. Then, from the datasets selected based on the type of structure, the one with the most matching attributes is selected. In the example of FIG. 12, the type of the structure and the shooting conditions are described as attributes, but the resolution and the coordinate information of the input data in the entire structure may be used.

Priority setting method The third variation is a method of determining the priority based on the statistic of the input data. In this method, the priority setting unit 203 acquires a statistic representing the characteristics of the input data from the learning recognition target data 502 and the existing data 503 as an attribute, and determines the priority of the existing data 503 based on the attribute. For example, suppose that the existing data 503 includes data sets S1, S2, and S3 of wall surface images of three types of structures. The statistic is obtained by acquiring the feature amount such as the texture information of the image of the input data and calculating the center of gravity, the average, the median value, etc. of the feature amount for each data set. Texture information can be extracted, for example, by applying a Fourier transform, which is one of the methods for acquiring frequency components, to input data. After obtaining the statistic, the priority setting unit 203 calculates the center of gravity of the statistic for each data set. The priority setting unit 203 calculates the distance between the learning recognition target data 502 and the center of gravity of the statistic of each data set, and sets this as an attribute. It is determined that the input data of the data set close to the position of the center of gravity of the learning recognition target data 502 can be learned according to the learning recognition target data 502, and the priority is set high. In this way, the priority setting unit 203 can preferentially select a data set having a background atmosphere similar to that of the learning recognition target data 502 by setting the priority based on the statistic. It is difficult for a person to judge what the image atmosphere is similar because it is subjective, but in the method using such statistics, the priority can be automatically set for each data set.

<Supplement to learning subset determination unit 205>
In the above description, the learning subset determination unit 205 creates the learning subset by selecting the learning data having a high evaluation value for each attribute described in FIG. 8, but when the input data is associated with a plurality of attributes, , This method cannot be applied as it is. Specifically, in the case of learning by mixing the learning recognition target data 502 and the existing data 503, the attribute is the content of the input data (image), and here, the attribute is the learning recognition target data 502 and the existing data 503. , Data with cracks and data without cracks. Of these, it is assumed that the recognition target data 502 for learning and the data showing cracks have high priority. Whether or not a crack is reflected is determined by, for example, whether or not there is a pixel indicating a crack in the teacher data paired with the input data. The data in which the cracks are reflected exists in both the learning recognition target data 502 and the existing data 503. At this time, the data to be recognized for learning 502 and the data in which the cracks are reflected are associated with the two attributes. In such a case, as described in the explanation of the learning subset determination unit 205 above, if a list is created for each attribute and data having a high evaluation value is collected as data used for learning, the same input data can be obtained for a plurality of attributes. It is selected from the list and the training data is duplicated. When the number of learning data used for one learning is a fixed value, if there is duplicate data in the learning subset, the variation of the learning data used for one learning is reduced. In this case, after removing the duplicated data from the learning subset, it is necessary to supplement the learning subset by selecting other data from the list of each attribute so that the learning subset reaches a predetermined number. Specifically, the learning subset determination unit 205 creates a list in which the recognition target data 502 for learning and the data in which the cracks are reflected are arranged in the order of evaluation values. For example, the learning subset determination unit 205 selects 5000 data from the list of the data to be recognized for learning 502 with the highest evaluation value, and selects 5000 data from the list of the data in which the cracks are reflected. At this time, it is assumed that 1000 data are duplicated in the data selected from the list of the recognition target data 502 for learning and the data selected from the list of the cracked data. At this time, the learning subset determination unit 205 removes 1000 duplicate data from the learning subset. Subsequently, the learning subset determination unit 205 selects from each list the data having the highest evaluation value of 5001 to the data having the highest evaluation value of 5500, and adds the data to the learning subset for supplementation. Then, the learning subset determination unit 205 creates a learning subset by repeating such a process of removal and replenishment until there is no duplication. When replenishing, the data selection method is not limited to this, and may be randomly selected. As described above, the learning subset determination unit 205 can prevent a decrease in the variation of the learning data used for one learning by considering the duplication of data when determining the learning subset.

<Embodiment 2>
In the first embodiment, the balance adjustment unit 204 determines the amount of data based on the ratio of the provisional data amount determined based on the evaluation value and the ratio of the provisional data amount determined based on the priority. Not limited to. In the second embodiment, the balance adjusting unit 204 calculates an index called a sample weight for each input data based on the evaluation value and the priority. The sample weight is an index used when creating a learning subset. Specifically, the sample weight is increased when both the evaluation value and the priority are high, and decreased when both are low. By doing so, when creating the learning subset, the input data having a large sample weight is easily selected, and the input data having a small sample weight is difficult to be selected. That is, the balance adjusting unit 204 adjusts the balance of the data by calculating the sample weight and controlling the data selected at the time of creating the learning subset.

In the second embodiment, as in the first embodiment, an example in which learning for crack recognition is performed for the purpose of inspecting the infrastructure structure will be described. In the second embodiment, as in the first embodiment, teacher data is added to a part of the wall surface image of the structure to be recognized to obtain the recognition target data 502 for learning.

The processing of each functional unit will be described below. The data storage unit 201 stores the learning recognition target data 502 and the existing data 503. Similar to the first embodiment, there are two types of attributes, the recognition target data 502 for learning and the existing data 503.

Since the processing of the evaluation unit 202 and the priority setting unit 203 is the same as that of the first embodiment, detailed description thereof will be omitted. The evaluation unit 202 acquires the number of erroneous detection pixels by using the teacher data for each input data as the evaluation value. The priority setting unit 203 assumes that the learning recognition target data 502 has a higher priority than the existing data 503, sets the priority of the learning recognition target data 502 to 0.9, and sets the priority of the existing data 503 to 0. Let it be 1.1.

Next, the processing of the balance adjusting unit 204 will be described. The balance adjustment unit 204 calculates the sample weight based on the evaluation value and the priority. The balance adjustment unit 204 adjusts the balance of the data selected when determining the learning subset by calculating the sample weight. The sample weight is a value calculated for each input data from the evaluation value and the priority for each attribute. For example, the balance adjustment unit 204 calculates the product of the evaluation value and the priority for each attribute as a sample weight. When the learning subset determination unit 205 creates the learning subset in a later step, it can select data having a high evaluation value and a high priority by collecting data having a large sample weight. A specific example is shown in FIG. FIG. 9 shows the evaluation value, the priority, and the sample weight for each input data. Since the input data d1 has a high evaluation value and a high priority, the sample weight value also becomes large, so that it is easy to be selected when creating the learning subset. The input data d2 and the input data d3 have the same evaluation value, but have different priorities. When the product of the evaluation value and the priority is calculated, the input data d2 has a high priority, so that the sample weight becomes relatively large and it is easy to be selected when creating the learning subset. On the other hand, since the input data d3 has a low priority, the sample weight becomes relatively small, and it becomes difficult to select the input data d3 when creating the learning subset. In this way, the balance adjusting unit 204 can adjust the balance of the data selected at the time of creating the learning subset by calculating the sample weight.

Next, the processing of the learning subset determination unit 205 will be described. The learning subset determination unit 205 determines the learning subset by selecting a part of the learning data of all the learning data 504 based on the sample weight calculated by the balance adjustment unit 204. First, the learning subset determination unit 205 sorts the input data in descending order of sample weight. FIG. 9 shows the sorted input data 910. Next, the learning subset determination unit 205 selects a predetermined number of input data 911 from the one with the highest sample weight value, and determines as the learning subset.

When the processing up to this point is completed, the learning unit 206 trains the model in the middle of learning using the learning subset. The information processing apparatus 100 controls the data selected when determining the learning subset by calculating the sample weight based on the evaluation value and the priority by performing the above processing, and adjusts the balance of the data. can.

In the above description, the sample weight calculation method is the product of the evaluation value and the priority, but the calculation method is not limited to this. For example, the balance adjusting unit 204 calculates the sample weight sw based on the linear sum of the evaluation value x and the priority y according to the equation (3).

Here, α and β are coefficients. α and β are determined in advance. The balance adjusting unit 204 calculates the sample weight sw by substituting the evaluation value x and the priority y into the equation (3) for each input data. As shown in the equation (3), the balance adjusting unit 204 can set the balance between the data collection based on the evaluation value x and the data collection based on the priority y by using the coefficients α and β.

<Embodiment 3>
In the third embodiment, as in the first embodiment, the balance adjusting unit 204 weights the ratio of the provisional data amount determined based on the evaluation value and the ratio of the provisional data amount determined based on the priority. Adjust the data balance. However, the methods for determining the _{weights w a} and w _b are different between the first and third embodiments. In the first embodiment, the weight determination unit 208 determines the weights w _a and w _b based on the sum of the evaluation values. In the third embodiment, the weight determination unit 208 constructs an equation for estimating an appropriate weight using the information accumulated in the past learning, assuming that there should be an appropriate weight for each training data set. Estimate the appropriate weights for the training data set to be learned. The weight determination unit 208 uses the weight estimation formula to set more appropriate weights according to the training data set.

The general flow of the third embodiment will be described. First, the information processing apparatus 100 learns the training data set using various weights, and after learning, determines the optimum weight in the training data set based on the performance of the model. The information processing apparatus 100 executes such processing on various training data sets, and checks an appropriate weight for each training data set. Next, the information processing apparatus 100 acquires the feature amount of the entire training data set for each training data set, and creates a pair of the feature amount and the optimum weight for each training data set. When these processes are completed, the information processing apparatus 100 constructs an equation expressing the relationship between the feature quantity and the optimum weight by using a large number of pairs of the feature quantity and the optimum weight, and uses this as a weight estimation formula. By constructing the weight estimation formula, the information processing apparatus 100 can estimate the weight suitable for the training data from the feature amount and the model of the entire data set of the training data set for which the model is to be trained.

In order to realize such processing, the information processing apparatus 100 requires advance preparation unlike the conventional embodiments. The advance preparation includes a process of accumulating past information and a process of constructing a weight estimation formula from the accumulated information. When the preparation is completed, the information processing apparatus 100 estimates an appropriate weight by the weight estimation formula for the new recognition target data 502 for learning, and performs learning.

The data handled in the third embodiment is an image of the wall surface of the structure for infrastructure inspection, as in the previous embodiments. Further, as in the other embodiment, in the recognition target data 501, the part used for learning is the learning recognition target data 502, and the part used for the test is the test recognition target data. Then, the learning unit 206 performs learning for the purpose of detecting the recognition target data for the test with high performance. The data storage unit 201 stores the learning recognition target data 502 and the existing data 503. Similar to the first embodiment, there are two types of attributes, the recognition target data 502 for learning and the existing data 503.

FIG. 10 is a block diagram showing a functional configuration example of the information processing apparatus 100 according to the third embodiment. The information processing device 100 of FIG. 10 is obtained by adding an extraction unit 209, a storage unit 210, a weight learning unit 211, and an analysis unit 212 to the information processing device 100 of FIG. The added components are also used for advance preparation.

The processing contents of the information processing apparatus 100 will be described below. First, as the first preparation, the process of accumulating information will be described. Here, the information processing apparatus 100 accumulates weight information used when a high-performance model could be generated in the past. Specifically, the information processing apparatus 100 performs learning a plurality of times while changing the combination of the set values of _{the weights w a} and w _b of the equation (2) described in the first embodiment, and the model generated as a result of the learning. Compare the performance of and determine the optimum weight based on the performance. In addition, the analysis unit 212 acquires a feature amount representing the characteristics of the data from the data used for learning. Then, the analysis unit 212 stores the optimum weight and feature amount in the storage unit 210.

FIG. 11 is a flowchart showing an information processing method of the information processing apparatus 100. Since the processes of S301 to S303 and S304 to S306 described in FIG. 11 are the same as the processes of the same number shown in FIG. 3, detailed description thereof will be omitted.

First, the information processing apparatus 100 processes S301 to S303 in the same manner as in FIG. When the processing of the evaluation unit 202 and the priority setting unit 203 is completed in S301 to S303, the weight determination unit 208 sets a plurality of temporary weights in S1101. For example, the weight determination unit 208 randomly prepares five types of combinations of the set values of _{the weights w a} and w _{b in the equation (2).}

Next, in S1102, the weight determination unit 208 selects one temporary weight. Next, in S304, the balance adjusting unit 204 determines the balance of the data using the tentative weight, and calculates the number of data for each attribute in the learning subset. Next, in S305, the learning subset determination unit 205 determines the learning subset. Next, in S306, the learning unit 206 trains the model using the learning subset.

The processing of S1102 to S306 is repeated while changing the temporary weight. In S1103, the learning unit 206 determines whether or not the process is completed by using a set of temporary weights. The learning unit 206 returns to S1102 if it is not completed, and proceeds to S1104 if it is completed. Since five types of tentative weights are prepared, five models are generated for each tentative weight at this point. The learning unit 206 stores these models in the model storage unit 207.

Next, in S1104, the analysis unit 212 evaluates the performance of the model stored in the model storage unit 207 and determines the optimum weight. The analysis unit 212 evaluates the model stored in the model storage unit 207 and analyzes the model having the highest performance. Performance evaluation is performed by, for example, adding teacher data to a part of the test recognition target data, performing recognition processing on the area of the test recognition target data, and examining the difference between the recognition result and the teacher data. Then, the analysis unit 212 determines the tentative weight used when the high-performance model is generated as the optimum weight.

As described above, the balance adjustment unit 204, the learning subset determination unit 205, and the learning unit 206 each perform processing on each of the temporary weights while changing the temporary weights. The analysis unit 212 is an evaluation unit, evaluates the performance of the model with respect to each temporary weight learned by the learning unit 206, and determines the temporary weight corresponding to the model with the highest performance as the optimum weight.

Next, in S1105, the extraction unit 209 extracts the feature amount of the entire data set of the learning data, which is the combination of the learning recognition target data 502 and the existing data 503. As a feature amount extraction method, the extraction unit 209 calculates a histogram generally used in the image processing field for each input data (image) of a data set, and calculates an average value and a variance from the histogram of all the input data. Then, this result is used as the feature quantity of the entire data set. The histogram is information obtained by counting the number of pixels having the brightness value for each brightness value of the image. In the above, the target for acquiring the feature amount is the entire learning data including the recognition target data 502 for learning and the existing data 503, but the target is not limited to this. In addition, the extraction unit 209 may set only the learning recognition target data 502 as the target of feature quantity extraction without including the existing data 503 in the entire learning data. Further, the extraction unit 209 can also set a data set in which the learning recognition target data 502 and the test recognition target data are combined as the target of feature quantity extraction.

Finally, in S1106, the storage unit 210 stores the optimum weight and the feature amount together. The above is the first preparation. The information processing device 100 performs such processing on various learning recognition target data 502, and accumulates information on the optimum weight and feature amount for each learning data set. In FIG. 11, the information processing apparatus 100 determines the optimum weight in S1104, extracts the feature amount in S1105, and then executes the process of storing the information in S1106, but the order of the processes is not necessarily the same. It's okay. The S1106 for storing the information may be located both after the optimum weight is determined in S1104 and after the feature amount is extracted in S1105. Further, since S1105 for extracting the feature amount is independent of the process of determining the optimum weight through S301 to S1104, it may be executed at any stage.

Next, as a second preparation, the weight learning unit 211 is a construction unit, receives information on the optimum weight and feature amount from the storage unit 210, and uses the optimum weight and feature amount as the feature amount. Build an estimation formula to estimate the weight based on it. By constructing the weight estimation formula by the weight learning unit 211, when the new learning recognition target data 502 is learned, the weight estimation formula can be used to determine the weight suitable for the learning of the learning recognition target data 502. .. Equations (4) and (5) show examples of estimation equations for the _{weights w a} and w _b.

Here, n is the number of dimensions of the feature quantity. k is a subscript that differs depending on the dimension of the feature quantity. x _k is a feature quantity. a _k is a coefficient. The weight learning unit 211 can construct an estimation formula for estimating the weights w _a and w _b from the feature quantity x _{k by determining the coefficient a k} of the formula (4). The coefficient a _k can be determined by the least squares method using a plurality of optimal weight / feature pairs obtained from the storage unit 210. The least squares method is a method of determining the coefficient so that the difference between the input value, the teacher value, and the actual output value is minimized. Here, the input value corresponds to the feature amount x _k , and the teacher value corresponds to the optimum weight acquired from the storage unit 210.

This is the preparation. When the preparation is completed, the weight determination unit 208 calculates the _{weights w a} and w _b using the weight estimation equations (4) and (5). Then, the information processing apparatus 100 uses the weights w _a and w _b to perform learning for generating a model for crack detection, as in the first embodiment. Subsequent processing is the same as that of the first embodiment except that the processing of the extraction unit 209 is added and the processing of the weight determination unit 208 is changed. Therefore, the extraction unit 209 and the weight determination unit 208 will be described, and the details of other processes will be omitted.

The extraction unit 209 extracts the entire feature amount of the learning data including the new recognition target data 502 for learning and the existing data 503. Here, the extraction unit 209 extracts the feature amount of the entire data set of the training data by the same method as the feature amount extraction method used in S1105 prepared in advance. In the above example, since the extraction unit 209 uses the histogram as the feature amount, the histogram is also extracted as the feature amount here.

Next, the weight determination unit 208 receives the weight estimation formula from the weight learning unit 211, receives the feature amount from the extraction unit 209, and determines the _{weights w a} and w _{b based on the feature amount using the weight estimation formula.} .. The weight determination unit 208 calculates _{the weights w a} and w _b that match the learning recognition target data 502 by substituting the feature amount into the weight estimation formula (4). The balance adjusting unit 204 _{receives the weights w a} and w _b calculated here, and adjusts the data balance. Other processing can be carried out by the same processing as in the first embodiment.

As described above, the information processing apparatus 100 can construct a weight estimation formula, estimate appropriate weights w _a and w _b according to the learning data set, adjust the data balance, and perform learning. ..

<Embodiment 4>
In the above embodiment, as an example of the recognition target, crack recognition for inspection of the infrastructure structure has been described as an example. The applicable target is not limited to this, and may be used for other purposes. In the fourth embodiment, learning of a recognition model used in a fixed camera such as a surveillance camera will be described as another adaptation destination. Since the hardware configuration and the functional configuration of the fourth embodiment are the same as those of the first to third embodiments, the description thereof will be omitted.

In a fixed camera such as a surveillance camera, an object of the fourth embodiment is to perform learning for detecting a suspicious person in an environment in which the fixed camera is installed. In order to apply the information processing device 100 for this purpose, the data stored in the data storage unit 201 may be changed to the data related to recognition by the fixed camera. Learning can be performed by the same processing as in other embodiments, except for the data stored in the data storage unit 201 and the processing of the priority setting unit 203. Therefore, in the following, the data stored in the data storage unit 201 when the fixed camera is targeted will be described.

In the present embodiment, a data set of a moving image taken by a fixed camera is created as recognition target data 501, and learning recognition target data 502 to which teacher data is added to a part of the data set is prepared and stored in the data storage unit 201. The data storage unit 201 stores the existing data 503 in addition to the learning recognition target data 502. The existing data 503 is data taken in an environment different from the recognition target data 501, and includes a data set in which a suspicious person is shown, a data set in which a mask wearer is shown, and a data set taken at night. The learning data is composed of a pair of input data and teacher data as in the conventional embodiments, but the difference is that the teacher data is set for each image. Specifically, 1 teacher data is given to the input data in which the suspicious person is shown, and 0 teacher data is given to the other input data. The information processing device 100 learns a model using such learning data, and sets a value close to 1 when there is a high possibility that a suspicious person is shown as a detection result for each image, and 0 otherwise. Generate a model that outputs close values. Further, the attributes are four types of data set, that is, the recognition target data 502 for learning, the existing data 503 of the suspicious person data, the existing data 503 of the mask wearer data, and the existing data 503 of the night photography data. ..

Since the processing of each of the other functional parts is the same as that of the embodiments described so far, detailed description thereof will be omitted. The evaluation unit 202 receives the data of the data storage unit 201, executes a process of detecting a suspicious person using the model in the middle of learning, compares the detection result with the teacher data, and acquires an evaluation value for each input data. The evaluation value is, for example, the result of calculating the sum of the error squares of the detection result and the teacher data for each input data. The priority setting unit 203 sets the priority of the recognition target data 502 for learning and the existing data 503 of the suspicious person data to be high for learning for the purpose of detecting a suspicious person in the environment where the fixed camera is installed. After that, the balance adjustment unit 204 adjusts the amount of data for each attribute in the learning subset based on the evaluation result and priority. Finally, the learning subset determination unit 205 creates a learning subset based on the amount of data, and the learning unit 206 performs learning.

As described above, the information processing apparatus 100 can be used for model selection of an arbitrary recognition target by changing the attributes.

In the above description, the input data is an image, and the teacher data is given in pixel units or image units, but the present invention is not limited to this. The input data is not limited to images, but may be voice data or text data. Further, the teacher data can be given not in pixel units but in data units. The information processing device 100 can perform the same processing as the contents described in each embodiment.

According to the first to fourth embodiments, the information processing apparatus 100 can create a learning subset by collecting learning data effective for learning in a well-balanced manner when learning by mixing learning data of various attributes. .. The information processing device 100 can create a learning subset by adjusting the balance of the learning data in consideration of the result of examining the learning data that the model is not good at and the attributes.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

It should be noted that all of the above embodiments merely show specific examples for carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from the technical idea or its main features.

100 Information processing device, 201 Data storage unit, 202 Evaluation unit, 203 Priority setting unit, 204 Balance adjustment unit, 205 Learning subset determination unit, 206 Learning unit, 207 Model storage unit, 208 Weight determination unit

Claims (18)

A first evaluation means, each of which inputs a plurality of training data including input data and teacher data, and obtains an evaluation value of each of the input data based on the teacher data by using a model.
A learning subset determination means for selecting a part of the learning data from the plurality of learning data and determining the learning subset based on the evaluation value and the priority for each attribute of the learning data.
An information processing apparatus comprising a learning means for learning the model using the determined learning subset. The ratio of the amount of data for each attribute in the training subset based on the ratio of the amount of data for each attribute of the training data based on the evaluation value and the ratio of the amount of data for each attribute of the training data based on the priority. Further have adjustment means to adjust
The learning subset determining means selects a part of the learning data from the plurality of learning data based on the ratio of the amount of data for each attribute in the adjusted learning subset, and determines the learning subset. The information processing apparatus according to claim 1. The adjusting means weights and adds the ratio of the amount of data for each attribute of the training data based on the evaluation value and the ratio of the amount of data for each attribute of the learning data based on the priority. The information processing apparatus according to claim 2, wherein the ratio of the amount of data for each attribute in the subset is calculated. It further has a weight determining means for determining the weight based on each evaluation value of the input data.
The information processing apparatus according to claim 3, wherein the adjusting means is weighted and added based on the weights. The information processing according to any one of claims 1 to 4, wherein the learning subset determining means selects a part of the learning data from the plurality of learning data based on the evaluation value. Device. The learning subset determining means is characterized in that a part of the learning data among the plurality of learning data is selected based on the product or linear sum of the evaluation value and the priority of each attribute of the learning data. The information processing apparatus according to claim 1. While changing the weights, the adjusting means, the learning subset determining means, and the learning means each perform processing on their respective weights.
The claim is characterized by further having a second evaluation means for evaluating the performance of the model with respect to each weight learned by the learning means and determining the weight corresponding to the model with the highest performance as the first weight. The information processing apparatus according to 3. An extraction means for extracting the first feature amount of the entire plurality of training data, and an extraction means.
It further has a construction means for constructing an equation for estimating the weight based on the feature amount based on the first weight and the first feature amount.
The extraction means extracts the second feature amount of the whole of the new plurality of training data.
It further has a weight determining means for determining the weight based on the second feature amount by using the above formula.
The information processing apparatus according to claim 7, wherein the adjusting means is weighted and added based on the determined weight. The information processing apparatus according to any one of claims 1 to 8, wherein the attribute is stored in association with the input data. The information processing apparatus according to any one of claims 1 to 8, wherein the attribute is a statistic of the input data. The first evaluation means performs recognition processing of the input data, and acquires the sum of squares of errors, the number of false detection pixels, or the number of undetected pixels based on the result of the recognition processing and the teacher data as the evaluation value. The information processing apparatus according to any one of claims 1 to 10, wherein the information processing apparatus is used. The information processing apparatus according to any one of claims 1 to 11, further comprising a priority setting means for setting a priority for each attribute of the learning data according to a user operation. The attributes include recognition target data and existing data.
The information processing apparatus according to any one of claims 1 to 11, wherein the priority when the attribute is recognition target data is higher than the priority when the attribute is existing data. The information processing apparatus according to any one of claims 1 to 13, wherein the first evaluation means, the learning subset determining means, and the learning means repeat processing a predetermined number of times. The fourth aspect of claim 4, wherein the first evaluation means, the weight determination means, the adjustment means, the learning subset determination means, and the learning means repeat processing until the change in the weight becomes smaller than the threshold value. Information processing device. The information processing according to any one of claims 1 to 15, wherein the learning subset determining means determines the learning subset using data obtained by performing image processing conversion on the input data. Device. A first evaluation step in which a plurality of training data including input data and teacher data are input, and an evaluation value of each of the input data is acquired based on the teacher data using a model.
A learning subset determination step of selecting a part of the learning data from the plurality of learning data and determining the learning subset based on the evaluation value and the priority of each attribute of the learning data.
An information processing method for an information processing apparatus, which comprises a learning step for learning the model using the determined learning subset. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 16.

Images