JP7045103B1 - Data Expansion Devices, Data Expansion Systems, and Data Expansion Programs - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data expansion device, a data expansion system, and a data expansion method for supporting data expansion for machine learning. SOLUTION: This is a data expansion device 1 that supports data expansion for machine learning, and is an acquisition of acquiring expansion target information including target data to be data expansion and annotation information given to the target data. Data extension that generates new target data based on the part, the generation part that generates index information based on the annotation information, and the index information generated by the generation part, and generates extended data including the generated new target data. It has a part. [Selection diagram] Fig. 1

Description

The present invention relates to a data expansion device, a data expansion system, and a data expansion program that support data expansion for machine learning. Regarding data expansion devices, data expansion systems, and data expansion programs .

Conventionally, as a data expansion system for machine learning, for example, the X-ray image object recognition system of Patent Document 1 has been proposed.

The X-ray image object recognition system disclosed in Patent Document 1 is based on a learning network that performs machine learning using a learning set including an X-ray image of an object and a correct label corresponding to the object, and a learning set of objects. A shooting information calculation unit that calculates shooting information for deriving shooting conditions during X-ray shooting, and a data expansion parameter used when performing data expansion to create a new X-ray image from an X-ray image based on the shooting information. It is characterized by having a data expansion parameter determination unit that determines And.

Japanese Unexamined Patent Publication No. 2020-14799

Here, when the number of learning images used for machine learning is insufficient, at present, data expansion is performed by generating a plurality of new images from one image by using rotation, enlargement / reduction, and the like. Is going. Therefore, if there are many patterns to be detected, it is possible to perform data expansion at random, but if only a specific pattern whose shape or color does not change is detected, it is more limited than learning at random. The accuracy of data expansion is improved by applying the above-mentioned data expansion method. This can prevent the learning from expanding the pattern that is not possible in reality, but it is necessary to check all the detection patterns, which takes a lot of man-hours and time, and is appropriate and appropriate. There is a concern that data expansion cannot be performed quickly.

In this regard, in the disclosure technique of Patent Document 1, the data expansion determined based on the calculated imaging information by calculating the imaging information for deriving the imaging conditions at the time of X-ray imaging of the object from the learning set prepared in advance. Data expansion is performed based on the parameters. Therefore, if there is no learning set for calculating shooting information, or if the target data that is the source of data expansion is small or small, the data expansion parameters for deriving the shooting conditions at the time of shooting are insufficient. , Proper and proper data expansion cannot be performed quickly. Under these circumstances, it is desired to improve the accuracy of data expansion in machine learning.

Therefore, the present invention has been devised in view of the above-mentioned problems, and an object thereof is a data expansion device, a data expansion system, and a data expansion system capable of improving the accuracy of data expansion in machine learning. It is to provide a data extension program .

The data expansion device according to the first invention is a data expansion device that supports data expansion for machine learning, and is included in a target image in which an object is captured, and the data expansion is combined with the image of the object. The target data of the target to be performed, the identification information given to the target data for identifying the target data, the feature information indicating the characteristics of the target data, the coordinate information indicating the position in the target image, and the generation of the target data. An acquisition unit that acquires extended target information including at least annotation information that includes process information indicating the process that has been performed, and a generation that generates a plurality of index information that exhibits different characteristics based on the annotation information acquired by the acquisition unit. An extension that generates new target data based on the unit, an output unit that outputs the index information generated by the generation unit, and the index information output by the output unit , and includes the generated new target data. It is characterized by including a data extension unit for generating data.

The data expansion device according to the second invention is characterized in that, in the first invention, the target data acquired by the acquisition unit is at least one of image data, video data, and text data. ..

In the data expansion device according to the third invention, in the first invention or the second invention, the index information generated by the generation unit indicates identification information associated with the target data and vertical and horizontal characteristics of the expansion target information. At least one of length information, rotation information, tilt information, hue information, saturation information, brightness information, brightness information, linearity information, coordinate information, distribution information, range information, deviation information, data intensity information, or pattern information. It is characterized by including.

The data expansion device according to the fourth invention further includes an adjustment unit for adjusting a plurality of the index information output by the output unit in any one of the first invention to the third invention, and the data expansion unit is the said. It is characterized in that the target data is expanded based on the index information adjusted by the adjusting unit to generate extended data.

The data expansion device according to the fifth invention is the plurality of data included in the past expansion target information acquired in advance in any one of the first invention to the fourth invention, and reference information associated with the past expansion target information. A reference database that stores a plurality of data contained in the data and a reference database that stores a relationship having a plurality of degrees of association corresponding to a weight variable between the data, and the reference database based on the extended target information acquired by the acquisition unit. Further, an evaluation unit that selects the optimum reference information associated with the solution calculated based on the association and generates an evaluation result including a plurality of the index information based on the optimum reference information. The output unit is characterized in that it outputs the evaluation result generated by the evaluation unit.

The data expansion system according to the sixth invention is a data expansion system that supports data expansion for machine learning, and the object is included in a captured target image, and the data expansion is performed together with the object. Target data, identification information that identifies the target data given to the target data, feature information that indicates the characteristics of the target data, coordinate information that indicates the position in the target image, and a process in which the target data is generated. An acquisition means for acquiring extended target information including at least annotation information including process information indicating the above, and a generation means for generating a plurality of index information showing different characteristics based on the annotation information acquired by the acquisition means. , An output means that outputs the index information generated by the generation means, and an extended data including the new target data generated by generating new target data based on the index information output by the output means. It is characterized by comprising a data expanding means for generating.

The data expansion program according to the seventh invention is a data expansion program that supports data expansion for machine learning, is included in an object image in which an object is captured, and performs the data expansion together with the object. Target data, identification information that identifies the target data given to the target data, feature information that indicates the characteristics of the target data, coordinate information that indicates the position in the target image, and a process in which the target data is generated. An acquisition step for acquiring extended target information including at least annotation information including process information indicating the above, and a generation step for generating a plurality of index information showing different characteristics based on the annotation information acquired by the acquisition step. , An output step that outputs the index information generated by the generation step, and an extended data including the generated new target data by generating new target data based on the index information output by the output step. It is characterized by having a computer execute a data expansion step to be generated.

According to the first to fifth inventions, the acquisition unit acquires the expansion target information including the target data of the object for which the data is expanded and the annotation information given to the target data. Therefore, it is possible to generate a plurality of index information for generating extended data based on the target data and the annotation information. This makes it possible to improve the accuracy of data expansion in machine learning.

Further, according to the first to fifth inventions, the generation unit generates index information showing different characteristics based on the annotation information. Therefore, it is possible to generate extended data with various data extensions based on the index information showing different characteristics. This makes it possible to improve the accuracy of data expansion in machine learning.

In particular, according to the second invention, the target data is at least one of image data, video data, and text data. Therefore, it is possible to generate index information according to the type of target data. This makes it possible to improve the accuracy of data expansion in machine learning.

In particular, according to the third invention, the index information includes identification information associated with the target data, vertical / horizontal information indicating the characteristics of the extended target information, vertical / horizontal information, rotation information, inclination information, hue information, saturation information, and the like. It includes at least one of brightness information, brightness information, linearity information, coordinate information, distribution information, range information, deviation information, data intensity information, or pattern information. Therefore, a plurality of index information can be output. This makes it possible to improve the accuracy of data expansion in machine learning.

In particular, according to the fourth invention, the adjusting unit adjusts the index information. Therefore, the data expansion unit can generate extended data including the new target data based on the index information adjusted by the adjustment unit. This makes it possible to improve the accuracy of data expansion in machine learning.

In particular, according to the fifth invention, the reference database corresponds to a weight variable between the past extended target information acquired in advance and a plurality of data included in the reference information associated with the past extended target information. Memorize the relationships that have multiple degrees of connection. Therefore, even when evaluating unknown extended target information different from past extended target information, quantitative evaluation can be performed. This makes it possible to further improve the accuracy of data expansion in machine learning.

Further, according to the sixth invention, the acquisition means acquires the expansion target information including the target data of the object for which the data is expanded and the annotation information given to the target data. Therefore, it is possible to generate a plurality of index information for generating extended data based on the target data and the annotation information. This makes it possible to improve the accuracy of data expansion in machine learning.

Further, according to the seventh invention, the acquisition step acquires the expansion target information including the target data of the object for which the data is expanded and the annotation information given to the target data. Therefore, it is possible to generate a plurality of index information for generating extended data based on the target data and the annotation information. This makes it possible to improve the accuracy of data expansion in machine learning.

FIG. 1 is a schematic diagram showing an example of a data expansion system according to the present embodiment. FIG. 2 is a schematic diagram showing an example of the operation of the data expansion system in the present embodiment. FIG. 3A is a schematic diagram showing an example of the configuration of the data expansion system in the present embodiment, and FIG. 3B is a schematic diagram showing an example of the function of the data expansion system in the present embodiment. FIG. 4 is a schematic diagram showing an example of a reference database in this embodiment. FIG. 5 is a schematic diagram showing a first modification of the reference database in the present embodiment. FIG. 6 is a flowchart showing an example of the operation of the data expansion system in the present embodiment.

Hereinafter, the data expansion device, the data expansion system, and the data expansion system according to the embodiment to which the present invention is applied, and
An example of the data expansion method will be described with reference to the drawings.

(Data Expansion System 100)
An example of the data expansion system 100 and the data expansion device 1 in the present embodiment will be described with reference to FIG. 1.

The data expansion system 100 in this embodiment has a data expansion device 1 as shown in FIG. 1, for example. The data expansion device 1 may be connected to another terminal 2 or a server 3 via, for example, a public communication network 4. The data expansion device 1 acquires the target image 20 via, for example, another terminal 2 connected by the public communication network 4.

In the data expansion system 100, for example, as shown in FIG. 2, the target image 20 is acquired by using the data expansion device 1. The target image 20 includes, for example, an object 20a such as a subject or a workpiece, and target data 21 whose data is to be expanded together with the object 20a. When expanding the data of the target image 20, for example, by increasing the patterns, variations, etc. of the "rotation", "hue", and "luminance" of the target data 21 while keeping the image of the object 20a in the target image 20 as it is. The target image 20 is inflated as extended data.

After acquiring the target image 20, the data expansion system 100 acquires expansion target information including, for example, the target data 21 which is a target portion for data expansion and the annotation information 21a given to the target data 21. The data expansion system 100 generates a plurality of index information (index information 31a, index information 31b, index information 31c, etc.) based on a plurality of information included in the annotation information 21a given to the target data 21, for example. The data expansion system 100 outputs, for example, a plurality of generated index information 31a to 31c to the display unit 109 or the like. The plurality of index information 31a to 31c are analyzed based on, for example, a known technique such as image analysis, quantified as a plurality of index information constituting the target data 21, and output to the display unit 109 or the like as, for example, a distribution map. You may.

The target data 21 is included in the target image 20 acquired via the data expansion device 1 by, for example, the data expansion system 100. In addition to the target image 20, the target data 21 may be, for example, video data or at least one of text data. When the data acquired through the data expansion device 1 is, for example, video data or music data, the characteristics of the annotation information added to each of them are analyzed, and a plurality of index information is obtained based on the analyzed results. Generate. For the analysis of the target data 21, for example, a known image analysis technique, music analysis technique, or the like is used, and a plurality of different index information may be generated, for example, by a numerical value or a distribution map.

The index information 30 is generated, for example, by the data expansion system 100 based on a plurality of information included in the annotation information 21a given to the target data 21 acquired via the data expansion device 1, for example, rotation of the target data 21. Is generated as index information 31a indicating the degree of hue, index information 31b indicating the degree of hue, and index information 31c indicating the degree of brightness. The data expansion system 100 outputs, for example, a plurality of generated index information 30 to a display unit 109 or the like, and a plurality of index information 30 based on a plurality of designated index information 30 according to an instruction input via the data expansion device 1. The extended data 40 of is generated.

The index information 30 includes, for example, identification information associated with the target data 21, vertical / horizontal information indicating the characteristics of the extended target information, rotation information, tilt information, hue information, saturation information, brightness information, brightness information, linearity information, and the like. It may include at least one of coordinate information, distribution information, range information, deviation information, data intensity information, or pattern information.

The extended data 40 is generated by, for example, the data expanding device 1 based on a plurality of index information 30. For example, when the generated index information is the index information 31a, the data expansion device 1 rotates the target data 21 based on the plurality of index information 31a to generate a plurality of extended data 41a. Further, for example, when the generated index information is the index information 31b, the data expansion device 1 changes the hue of the target data 21 based on the plurality of index information 31b and generates a plurality of expansion data 41b. Further, for example, when the generated index information is the index information 31c, the data expansion device 1 changes the brightness of the target data 21 based on the plurality of index information 31c to generate a plurality of expansion data 41c.

The extended data 40 may be, for example, a combination of a plurality of index information 30s. When the plurality of index information 30 designated via the data expansion device 1 is, for example, index information 31a (rotation), index information 31b (hue), and index information 31c (brightness), the plurality of index information 31a to Based on the combination of 31c, a plurality of extended data in which the rotation, hue, and brightness of the target data 21 are changed may be generated. As a result, it becomes possible to quickly generate appropriate and appropriate extended data based on the target data 21 and the annotation information 21a, and it is possible to improve the accuracy of data expansion in machine learning using the expanded data. It will be possible.

As the data expansion device 1, for example, an electronic device such as a personal computer (PC) is used, and for example, an electronic device such as a smartphone, a tablet terminal, a wearable terminal, an IoT (Internet of Things) device, a Raspbury Pi (registered trademark), and the like. A single board computer may be used, and an input device (not shown) such as a microphone or a camera may be built in.

The target image 20 is directly input to the data expansion device 1 by an operator or the like so as to be linked to the target data 21 and the annotation information 21a acquired by the data expansion device 1, or is connected to, for example, a public communication network 4. It may be transmitted from another terminal 2 or the like.

(Data expansion device 1)
Next, an example of the data expansion device 1 in the present embodiment will be described with reference to FIG. FIG. 3A is a schematic diagram showing an example of the configuration of the data expansion device 1 in the present embodiment, and FIG. 3B is a schematic diagram showing an example of the function of the data expansion device 1 in the present embodiment. be.

As shown in FIG. 3A, for example, the data expansion device 1 stores a housing 10, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and the storage. A unit 104 and I / F 105 to 107 are provided. Each configuration 101 to 107 is connected by an internal bus 110.

The CPU 101 controls the entire data expansion device 1. The ROM 102 stores the operation code of the CPU 101. The RAM 103 is a work area used when the CPU 101 operates.

The storage unit 104 stores various information such as expansion target information and a reference database. As the storage unit 104, for example, in addition to an HDD (Hard Disk Drive), a data storage device such as an SSD (Solid State Drive), a USB memory, or a cloud is used. For example, the data expansion device 1 may have a GPU (Graphics Processing Unit) (not shown). Having a GPU enables higher-speed arithmetic processing than usual.

The I / F 105 is an interface for transmitting and receiving various information with the data expansion device 1, and also for transmitting and receiving various information with other terminals 2 and servers 3 via, for example, a public communication network 4. Interface may be used.

The I / F 106 is an interface for transmitting / receiving information to / from the input unit 108. For example, a keyboard is used as the input unit 108, and an operator or the like who uses the data expansion device 1 inputs various information or a control command of the data expansion device 1 via the input unit 108.

The I / F 107 is an interface for transmitting and receiving various information to and from the display unit 109. The display unit 109 outputs various information such as evaluation results stored in the storage unit 104, or the processing status of the data expansion device 1. A display is used as the display unit 109, and a touch panel type may be used, for example.

<Reference database>
The reference database stored in the storage unit 104 stores the relationship between the past extended target information acquired in advance and the reference information associated with the past extended target information, for example, a learning model having a relationship. Is remembered.

For example, past extended target information and reference information may be stored in the reference database. The association is constructed by machine learning using a plurality of learning data, for example, using the past extended target information and the reference information as a set of learning data. As a learning method, deep learning such as a convolutional neural network is used.

In this case, for example, the association indicates the degree of connection between many-to-many information (a plurality of data included in the past extended target information, a pair, and a plurality of data included in the reference information).

Relationships are updated as appropriate during the process of machine learning. That is, the association indicates, for example, a function optimized based on the past target data 21, annotation information 21a, and reference information. Therefore, the evaluation result for the extended target information is generated by using the association constructed based on all the results of evaluating the extended target information in the past. Thereby, even when the extended target information has a configuration state in which the extended target information is combined with the other target data 21, the annotation information 21a, and the like, the optimum evaluation result can be generated.

Further, the reference database can quantitatively generate the optimum evaluation result even when the extended target information is the same as or similar to the past extended target information or when it is dissimilar. By increasing the generalization ability when performing machine learning, it is possible to improve the evaluation accuracy for unknown extended target information.

The association may have, for example, a plurality of association degrees indicating the degree of connection between the plurality of data included in the past extended target information and the plurality of data included in the reference information. The degree of association can correspond to a weight variable, for example, when the learning model is constructed by a neural network.

The past expansion target information indicates the same type of information as the expansion target information described above. The past expansion target information includes, for example, a plurality of expansion target information acquired when the target data 21 was evaluated in the past.

The reference information is linked to the past extended target information, and indicates information related to the target data 21 and the annotation information 21a. The reference information indicates an evaluation (for example, "rotation", "hue", "luminance", etc.) based on the content, state, or influence of the target data 21, and also relates to the priority, combination, numerical value, distribution, etc. of the index information. Information may be included. The specific contents included in the reference information can be arbitrarily set.

The association may indicate the degree of connection between the past extended target information and the reference information, as shown in FIG. 4, for example. In this case, by using the association, a plurality of data included in the reference information (FIG. 4) for each of the plurality of data included in the past extended target information (“Data A” to “Data C” in FIG. 4). In 4, the degree of relationship between "reference A" to "reference C") can be associated and stored. Therefore, for example, it is possible to link a plurality of data included in the reference information to one data included in the past extended target information through association, and it is possible to generate a multifaceted evaluation result. be able to.

The association has, for example, a plurality of association degrees in which a plurality of data included in the past extended target information and a plurality of data included in the reference information are associated with each other. The degree of association is shown in three or more steps such as percentage, 10 steps, or 5 steps, and is shown by, for example, the characteristics of the line (for example, thickness). For example, the "data A" included in the past expansion target information indicates the degree of association AA "80%" with the "reference A" included in the reference information, and the "reference B" included in the reference information. The degree of association between them is AB "65%". That is, the "degree of association" indicates the degree of connection between each data, and for example, the higher the degree of association, the stronger the connection of each data. When constructing the association by the above-mentioned machine learning, the association may be set to have three or more levels of association.

As shown in FIG. 5, for example, the past extended target information may be stored in the reference database by dividing the past target data and the past annotation information. In this case, the degree of association is calculated based on the relationship between the combination of the past target data and the past annotation information and the reference information. In addition to the above, the past expansion target information may be stored in the reference database by dividing the past countermeasure information.

For example, the combination of the "target data a" included in the past target data and the "annotation information a" included in the past annotation information indicates a degree of association AAA "56%" with the "reference A". The degree of association ABA "23%" with "reference B" is shown. In this case, the past target data and the past annotation information can be stored independently. Therefore, when generating the evaluation result, it is possible to improve the accuracy and expand the range of options.

The past extended target information may include, for example, synthetic data and similarity. The synthetic data is indicated by three or more levels of similarity with past target data or past annotation information. The composite data may be stored in the reference database in the form of a numerical value, a matrix, a histogram, or the like, or may be stored in the form of, for example, an image or a character string.

FIG. 3B is a schematic diagram showing an example of the function of the data expansion device 1. The data expansion device 1 includes an acquisition unit 11, a generation unit 12, an output unit 13, a data expansion unit 14, an adjustment unit 15, a storage unit 16, an update unit 17, and an evaluation unit 18. Each function shown in FIG. 3B is realized by the CPU 101 executing a program stored in the storage unit 104 or the like using the RAM 103 as a work area, and may be controlled by, for example, artificial intelligence.

<Acquisition unit 11>
The acquisition unit 11 acquires the target image 20. The target image 20 includes, for example, an object 20a and target data 21. For example, when the target image 20 includes a plurality of objects 20a, the acquisition unit 11 uses the target data 21 included in the range selected by the operator via the input unit 108 as the target data 21 for data expansion. You may try to get it.

The acquisition unit 11 acquires the expansion target information including the target data 21 for data expansion and the annotation information 21a given to the target data 21. In addition to acquiring the target image 20 from the data expansion device 1, the acquisition unit 11 may acquire the target image 20 and various data in other formats from, for example, another connected terminal 2.

For example, when the target image 20 is acquired, the acquisition unit 11 acquires the target data 21 and the annotation information 21a together with the target image 20 as extended target information. The range, number of sheets, frequency, period, and the like for the acquisition unit 11 to acquire the target image 20, the target data 21, and the extended target information are arbitrary.

The acquisition unit 11 acquires at least one of image data, video data, and text data as the target data 21 to be acquired by the acquisition unit 11.

Further, the acquisition unit 11 acquires the target image 20 first via the data expansion device 1, for example, and then obtains the target data 21 and the annotation information 21a given to the target data 21 via the other terminal 2. You may try to get it.

For example, when the target data 21 is an image, the acquisition unit 11 identifies the target data 21 as, for example, the feature "◆ (black diamond-shaped image)" of the image of the target data 21 and the added annotation information 21a. Information "target data 001", feature "diamond" of target data 21, color information "RGB (0,0,0)" of target data 21, position "coordinates (X, Y)" in target image 20, tag information " "Shards", the process "production process" in which the target data 21 is generated, and the like are acquired as various annotation information 21a.

The acquisition unit 11 receives various information transmitted to the data expansion device 1. The acquisition unit 11 receives various information such as designations for the target image 20, annotation information 21a, and index information 30 transmitted from an external terminal such as another terminal 2 via, for example, the public communication network 4 and the I / F 105. You may.

For example, the acquisition unit 11 may refer to the learning model stored in the storage unit 104, select the annotation information 21a corresponding to the target data 21, and acquire it as the extended target information.

The acquisition unit 11 may specify a range of the target data 21 input via, for example, the input unit 108 of the data expansion device 1, and acquire the target data 21 included in the range. When the range of data expansion input via the input unit 108 or the like includes the object 20a, the acquisition unit 11 may acquire the object 20a and the target data 21 as the range of data expansion.

<Generator 12>
The generation unit 12 provides a plurality of index information 30 showing different characteristics that enable a plurality of data expansion based on a plurality of information included in the annotation information 21a given to the target data 21 acquired by the acquisition unit 11. Generate.

For example, when there are a plurality of objects 20a and target data 21 included in the target image 20, the generation unit 12 may recognize each of them as a partial image of the target image 20 and generate index information 30 for each partial image. The generation unit 12 subdivides the object 20a and the target data 21 shown in the target image 20 into each, and discriminates each of them, for example, by a known image discrimination technique or the like.

The generation unit 12 generates index information 31a related to "rotation" in numerical information, distribution, etc., based on evaluation results such as position information, size, shape, etc. of the target data 21 in the target image 20, for example.

The generation unit 12 generates index information 30 for, for example, "hue" and "luminance" based on, for example, the target data 21 in the target image 20 and the annotation information 21a. Similar to the above, the generation unit 12 evaluates based on, for example, the target data 21 and the annotation information 21a given to the target data 21, and based on the evaluation result, the index information 31b regarding “hue” and the index regarding “luminance”. Generate information 31c.

The generation unit 12 generates appropriate index information 30 for each of the discriminated partial images, for example, based on various information such as the arrangement position, size, color, shooting conditions, and annotation information 21a in the target image 20. You may do so. Thereby, even if a plurality of objects 20a and target data 21 are included in the target image 20, they can be distinguished based on the images, the characteristics of the data, the positional relationship of each, and the like. , It becomes possible to generate index information 30 for improving the accuracy of data expansion in machine learning.

<Output unit 13>
The output unit 13 outputs a plurality of index information 30 generated by the generation unit 12. The output unit 13 transmits the evaluation result to the display unit 109 via the I / F 107, and also transmits the index information 30 to another terminal 2 or the like via, for example, the I / F 105. The output unit 13 outputs, for example, a plurality of index information 30, extended data 40, and the like shown in FIG. 2 to the display unit 109 and the like.

<Data expansion unit 14>
The data expansion unit 14 generates expansion data 40 including new target data based on the index information 30 output by the output unit 13. When the index information 31a indicating "rotation" is selected by the operator via, for example, the data expansion device 1, the data expansion unit 14 rotates the target data 21 based on the numerical information indicating "rotation". Generate new target data. The data expansion unit 14 generates expansion data 41a, which is a combination of the newly generated target data and the object 20a included in the target image 20.

For example, the data expansion unit 14 modifies or moves the original target data 21 based on the index information 30 output by the output unit 13 to generate a plurality of the target data 21 of the target image 20 as new target data. The data expansion unit 14 replaces a plurality of new target data with the target data of the target image 20 for data expansion, and generates the expansion data 40.

Based on the index information 31a, the data expansion unit 14 generates new target data with a further increased angle (rotation) of the target data 21, and generates it as extended data 40 including the new target data. When the index information 30 is, for example, "rotation" and index information related to the position and size, the data expansion unit 14 has the arrangement position and size of the target data 21 based on the respective index information 30. A plurality of extended data 40 including new target data with different values are generated.

The data expansion unit 14 generates new target data in which the "hue" is changed based on, for example, the index information 31b regarding the "hue", and generates extended data 41b including the new target data. Further, the data expansion unit 14 may generate the expansion data 41c including the new target data in which the “brightness” is changed, based on the index information 31c regarding the “brightness”, for example, and is generated by the generation unit 12. Based on the plurality of index information 30, different new target data is generated, and a plurality of extended data including the new target data are generated.

The data expansion unit 14 is a new model in which "hue" and "luminance" are changed according to, for example, rotation, position, and size, based on a combination of a plurality of index information 30 of a plurality of index information 30 output by the output unit 13. A plurality of extended data 40 including the target data may be generated. The data expansion unit 14 generates new expansion data 40 including new target data based on a plurality of index information 30 adjusted by the adjustment unit 15 described later.

For example, the data expansion unit 14 may generate expansion data 40 including a plurality of the same new target data and changing only the arrangement position based on the same index information 30. As a result, for example, even if the target image 20 contains only one target data 21, the extended data 40 in which the number of new target data (for example, “reduced”) is changed, and the new target data (for example, “position”) are changed. It becomes possible to generate the extended data 40 in which the arrangement of ") is changed.

<Adjustment unit 15>
The adjusting unit 15 adjusts each index information 30 (for example, index information 31a to 31c, etc.) indicated by a plurality of index information 30 output by the output unit 13, for example. The adjusting unit 15 can adjust the numerical value, range, position, etc. of the index information 30 displayed on the display unit 109, for example, via the input unit 108 of the data expansion device 1. Based on the numerical value and the range of the index information 30 adjusted by the adjusting unit 15, the extended data 40 including the corresponding new target data is generated in the data expanding unit 14.

<Memory unit 16>
The storage unit 16 retrieves various information such as a reference database stored in the storage unit 104 as needed. The storage unit 16 stores various information acquired or generated by the respective configurations 11, 13 to 16 in the storage unit 104.

<Display unit 109>
The display unit 109 displays the target image 20, a plurality of index information 30, a plurality of extended data 40, and an evaluation result. As shown in FIG. 2, for example, the display unit 109 displays a plurality of index information 30 generated by the generation unit 12. The index information 31a shows the index information about "rotation", the index information 31b shows the index information about "hue", and the index information 31c shows the index information about "luminance".

The display unit 109 displays a plurality of extended data 40. The extended data 41a shows extended data including new target data generated based on the index information 31a, the extended data 41b shows extended data generated based on the index information 31b, and the extended data 41c is based on the index information 31c. Shows extended data, including the new target data generated. The display unit 109 may display the evaluation result using, for example, only a list or a character string. A known technique can be used for the above display method.

<Update part 17>
The update unit 17 updates, for example, the reference database. When the update unit 17 newly acquires the relationship between the past expansion target information and the reference information, the update unit 17 reflects the relationship in the association. For example, when the evaluator or the like determines the accuracy of the content of the evaluation result based on the evaluation result generated by the evaluation unit 18 and the data expansion device 1 acquires the determination result, the update unit 17 determines the reference database based on the determination result. Update the associations contained in.

<Evaluation unit 18>
The evaluation unit 18 refers to the reference database and generates an evaluation result for the extended target information. For example, the evaluation unit 18 uses the extended target information as input data, selects the optimum reference information associated with the solution calculated based on the association, and generates an evaluation result based on the optimum reference information.

When referring to the reference database shown in FIG. 4, for example, the evaluation unit 18 selects data that is the same as or similar to the data included in the extended target information (for example, “data A”: first data). As the first data, data that partially or completely matches the extended target information is selected, and for example, similar data is selected. When the expansion target information is represented by a numerical value such as a matrix, the numerical range included in the selected first data may be set in advance.

The evaluation unit 18 selects the reference information associated with the selected first data and the degree of association (first degree of association) between the selected first data and the reference information, and selects the selected reference information and the first degree of association. The evaluation result is generated based on. The first degree of association is selected from the associations constructed in advance, and may be calculated by the evaluation unit 18.

For example, the evaluation unit 18 has the data "reference A" included in the reference information associated with the first data "data A", and the first degree of association (linkage degree AA) between "data A" and "reference A". Select "80%". The reference information and the first degree of association may include a plurality of data. In this case, in addition to the above-mentioned "reference A" and "80%", the reference information "reference B" associated with the first data "data A" and the second between "data A" and "reference B". 1 Linkage degree (linkage degree AB) "65%" may be selected and evaluation results may be generated based on "reference A" and "85%", as well as "reference B" and "33%".

The evaluation result may include extended target information. The evaluation result may show the recommendation (extensibility) of the index information expressed by the probability by using, for example, the reference information and the degree of association.

The evaluation unit 18 uses format data such as an output format stored in advance in the storage unit 104 or the like to allow the evaluator or the like to understand the above-selected reference information and the first association degree (for example, a character string). ) Is generated. For example, a known technique may be used for setting the format when generating the evaluation result.

The evaluation unit 18 determines the content of the evaluation result based on, for example, the selected first degree of association. For example, the evaluation unit 18 generates an evaluation result based on the reference information associated with the first association degree of "50%" or more, and uses the reference information associated with the first association degree of less than "50%" as the evaluation result. It may be set not to reflect. As for the determination criteria based on the first degree of association, for example, an evaluator or the like may set a threshold value or the like in advance, and the threshold range or the like can be arbitrarily set. Further, the evaluation unit 18 may determine the content of the evaluation result based on, for example, the result of calculating two or more first association degrees and the comparison of two or more first association degrees.

<Public communication network 4>
The public communication network 4 is, for example, an Internet network or the like to which a data expansion device 1 or the like is connected via a communication circuit. The public communication network 4 may be configured by a so-called optical fiber communication network. Further, the public communication network 4 may be realized by a known communication network such as a wireless communication network in addition to the wired communication network.

<Other terminal 2>
As the other terminal 2, for example, a terminal embodied in an electronic device similar to the data expansion device 1 is used. The other terminal 2 indicates, for example, a central control device capable of communicating with the data expansion device 1. The other terminal 2 can be connected to, for example, the data expansion device 1, and can acquire the index information 30, the expansion data 40, or the evaluation result generated by each data expansion device 1. Thereby, for example, the data expansion of the target image 20 and the evaluation result can be analyzed at a plurality of places, and the efficiency of the data expansion of the target image 20 can be improved.

<Server 3>
For example, the server 3 stores the above-mentioned various information. Various information transmitted via, for example, the public communication network 4 is stored in the server 3. For example, information similar to that of the storage unit 104 may be stored in the server 3, and various information may be transmitted / received to / from one or more data expansion devices 1 via the public communication network 4. That is, the data expansion device 1 may use the server 3 instead of the storage unit 104.

(Example of operation of data expansion system 100)
Next, an example of the operation of the data expansion system 100 in this embodiment will be described. FIG. 6 is a flowchart showing an example of the operation of the data expansion system 100 in the present embodiment.

<Acquisition means S110>
The acquisition unit 11 acquires the expansion target information including the target data 21 for data expansion and the annotation information 21a given to the target data 21 (acquisition means S110). The acquisition unit 11 acquires, for example, image data, video data, or text data as the target data 21 to be acquired.

Further, the acquisition unit 11 acquires adjustments to the plurality of index information 30 output by the output unit 13 via the input unit 108. The acquisition unit 11 stores the extended target information in the storage unit 104, for example, via the storage unit 16.

The acquisition unit 11 acquires, for example, the target data 21 and the extended target information including the annotation information 21a given to the target data 21. The annotation information 21a is input to the data expansion device 1 by an evaluator or the like so as to be linked to the target data 21 in advance, and the acquisition unit 11 obtains each information suitable for the target data 21 based on the target data 21, for example. You may choose. In this case, the acquisition unit 11 selects each information suitable for the target data 21 from the plurality of annotation information 21a stored in the storage unit 104 in advance.

<Generation means S120>
Next, the generation unit 12 generates a plurality of index information 30 (generation means S120). The generation unit 12 exhibits different characteristics based on, for example, the target data 21 included in the target extension information acquired by the acquisition unit 11 and a plurality of information included in the annotation information 21a associated with the target data 21. Index information 30 is generated.

The generation unit 12 has, for example, based on the expansion target information acquired by the acquisition unit 11, identification information associated with the target data 21, vertical and horizontal information indicating the characteristics of the expansion target information, rotation information, tilt information, hue information, and the like. The index information 30 indicated by saturation information, lightness information, luminance information, linearity information, coordinate information, distribution information, range information, deviation information, data intensity information, pattern information, etc. is generated. Each index information 30 may be subdivided into each component and generated as a distribution map or the like by using, for example, a known image analysis technique or voice analysis technique.

<Output means S130>
Next, the output unit 13 outputs a plurality of index information 30 (output means S130). The output unit 13 outputs a plurality of index information 30 to the display unit 109 and the like. The output unit 13 may output to another terminal 2 or a server 3 via, for example, a public communication network 4.

<Data expansion means S140>
Next, the data expansion means S140 generates a plurality of expansion data 40 including new target data (data expansion means S140). The data expansion unit 14 generates a plurality of expansion data 40 including the corresponding new target data based on the plurality of index information 30 output by the output unit 13.

The data expansion unit 14 repeatedly generates extended data 40 including the new target data by superimposing the newly generated target data on the image obtained by removing the target data 21 from the target image 20, and thus machine learning. Perform data expansion for.

<Adjustment means S150>
Further, the adjusting unit 15 adjusts a plurality of index information 30 (adjusting means S150). The adjusting unit 15 adjusts, for example, a plurality of index information 30 output by the output unit 13. The adjusting unit 15 is vertically and horizontally long to show the characteristics of the identification information associated with the target data 21 and the extended target information via the input unit 108 for the individual index information 31a to the index information 31c output to the display unit 109. Select and select information, rotation information, tilt information, hue information, saturation information, brightness information, brightness information, linearity information, coordinate information, distribution information, range information, deviation information, data intensity information, pattern information, etc. The index information 30 is adjusted. As a result, the operator or the evaluator can modify or edit the index information 30, and more accurate extended data 40 can be generated.

<Evaluation means S160>
Next, the reference database is referred to, and the evaluation result for the extended target information is generated (evaluation means S160). The evaluation unit 18 acquires the extended target information acquired by the acquisition unit 11, and acquires, for example, the reference database stored in the storage unit 104. For example, the evaluation unit 18 uses the extended target information as input data, selects the optimum reference information associated with the solution calculated based on the association shown by the function or the like, and generates an evaluation result based on the optimum reference information. .. At this time, for example, the evaluation unit 18 may select a plurality of reference information for one extended target information.

The evaluation unit 18 may generate one evaluation result for one extended target information, or may generate one evaluation result for a plurality of extended target information, for example. The evaluation unit 18 generates an evaluation result using, for example, format data such as an output format stored in the storage unit 104. The evaluation unit 18 stores the evaluation result in the storage unit 104, for example, via the storage unit 16.

<Update means S170>
In addition, for example, when the relationship between the past expansion target information and the reference information is newly acquired, the relationship may be reflected in the association (update means S170). For example, when the data expansion device 1 acquires a determination result in which the evaluator or the like determines the accuracy of the evaluation result based on the evaluation result generated by the evaluation unit 18, the update unit 17 stores the determination result in the reference database based on the determination result. Update the association.

As a result, the operation of the data expansion system 100 in the present embodiment is completed. The timing when the update means S170 is carried out is arbitrary.

According to the present embodiment, the evaluation unit 18 refers to the reference database and generates an evaluation result for the extended target information. The reference information includes the index information 30. Therefore, it is possible to generate an evaluation result based on the result of evaluating the past expansion target information. This makes it possible to improve the accuracy of data expansion in machine learning.

Further, according to the present embodiment, the association is constructed by machine learning using the past extended target information and the reference information as learning data. Therefore, even when evaluating unknown extended target information different from past extended target information, quantitative evaluation can be performed. This makes it possible to further improve the evaluation accuracy.

Further, according to the present embodiment, when the update unit 17 newly acquires the relationship between the past expansion target information and the reference information, the update unit 17 reflects the relationship in the association. Therefore, the association can be easily updated, and the evaluation accuracy can be continuously improved.

Further, according to the present embodiment, the acquisition step for performing the acquisition means S110, the generation step for performing the generation means S120, the output step for performing the output means S130, and the data expansion step for performing the data expansion means S140 are provided as data expansion methods. May be. This makes it possible to improve the accuracy of data expansion in machine learning.

Although embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1: Data expansion device 2: Terminal 3: Server 4: Public communication network 10: Housing 11: Acquisition unit 12: Generation unit 13: Output unit 14: Data expansion unit 15: Adjustment unit 16: Storage unit 17: Update unit 18 : Evaluation unit 20: Target image 20a: Target object 21: Target data 21a: Annotation information 30: Index information 31a: Index information 31b: Index information 31c: Index information 40: Extended data 41a: Extended data 41b: Extended data 41c: Extended Data 100: Data expansion system 101: CPU
102: ROM
103: RAM
104: Storage unit 105: I / F
106: I / F
107: I / F
108: Input unit 109: Display unit 110: Internal bus S110: Acquisition means S120: Generation means S130: Output means S140: Data expansion means S150: Adjustment means S160: Evaluation means S170: Update means

Claims (7)

A data expansion device that supports data expansion for machine learning.
The target data that is included in the target image in which the target is captured and whose data is expanded together with the image of the target, the identification information that identifies the target data given to the target data, and the target data. An acquisition unit that acquires extended target information including feature information indicating the characteristics of the target image, coordinate information indicating a position in the target image, and annotation information including at least process information indicating the process in which the target data is generated.
A generation unit that generates a plurality of index information showing different characteristics based on the annotation information acquired by the acquisition unit, and a generation unit.
An output unit that outputs the index information generated by the generation unit, and
A data expansion unit that generates new target data based on the index information output by the output unit and generates extended data including the generated new target data.
A data expansion device characterized by comprising. The target data acquired by the acquisition unit is at least one of image data, video data, and text data.
The data expansion device according to claim 1. The index information generated by the generation unit includes identification information associated with the target data, vertical / horizontal information indicating the characteristics of the extended target information, rotation information, tilt information, hue information, saturation information, and brightness information. Includes at least one of brightness information, chromaticity information, coordinate information, distribution information, range information, deviation information, data intensity information, or pattern information.
The data expansion apparatus according to claim 1 or 2. Further, an adjusting unit for adjusting the index information generated by the generating unit is provided.
The data expansion unit is
To generate new target data based on the index information adjusted by the adjusting unit, and to generate extended data including the generated new target data.
The data expansion device according to any one of claims 1 to 3. It has a plurality of association degrees corresponding to weight variables between a plurality of data included in the past expansion target information acquired in advance and a plurality of data included in the reference information associated with the past expansion target information. A reference database that stores relationships and
Based on the extended target information acquired by the acquisition unit, the reference database is referred to, the optimum reference information associated with the solution calculated based on the association is selected, and the optimum reference information is selected. Further equipped with an evaluation unit that generates an evaluation result including a plurality of the index information.
The output unit outputs the evaluation result generated by the evaluation unit.
The data expansion device according to any one of claims 1 to 4. A data expansion system that supports data expansion for machine learning.
Target data that is included in the target image in which the target is captured and whose data is expanded together with the target, identification information that identifies the target data given to the target data, and features of the target data. An acquisition means for acquiring extended target information including feature information indicating the above, coordinate information indicating a position in the target image, and annotation information including at least process information indicating the process in which the target data is generated.
A generation means that generates a plurality of index information showing different characteristics based on the annotation information acquired by the acquisition means, and a generation means.
An output means for outputting the index information generated by the generation unit, and
A data expansion means that generates new target data based on the index information output by the output means and generates extended data including the generated new target data.
A data expansion system characterized by being equipped with. A data expansion program that supports data expansion for machine learning.
Target data that is included in the target image in which the target is captured and whose data is expanded together with the target, identification information that identifies the target data given to the target data, and features of the target data. The acquisition step of acquiring the extended target information including the feature information indicating the feature information, the coordinate information indicating the position in the target image, and the annotation information including at least the process information indicating the process in which the target data is generated.
A generation step that generates a plurality of index information showing different characteristics based on the annotation information acquired by the acquisition step, and a generation step.
An output step that outputs the index information generated by the generation step, and
A data expansion step that generates new target data based on the index information output by the output step and generates extended data including the generated new target data.
To let the computer run
A data extension program featuring.

Images