JP2020027182A - Learning data generation method, learning method, and evaluation device - Google Patents

Abstract

To provide a learning data generation method, a learning method, and an evaluation device capable of easily acquiring learning data.SOLUTION: The learning data generation method for generating in a pseudo manner sound data to be used as learning data for machine learning, includes: an acquiring step of acquiring a reference image obtained by extracting a part of spectrogram converted from the sound data, and a training image obtained by deleting a part of the reference image; a first database generating step of generating a first database by machine learning using the reference image and the training image as a pair of input data; a generating step of generating a pseudo image based on the new reference image or the new training image by referring to the first database; and a converting step of converting the pseudo image into pseudo sound data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a learning data generation method, a learning method, and an evaluation device.

  Conventionally, as a technique for evaluating sounds such as utterances, for example, an evaluation device disclosed in Patent Literature 1 has been proposed.

  In Patent Literature 1, for example, an input unit to which a speech signal of a free utterance of a speaker is input, a feature extraction unit to extract a feature used for evaluation from the input speech signal, and a feature extracted by the feature extraction unit A feature evaluation unit that compares a reference feature stored in advance, and an output unit that outputs a comparison result, wherein the features used for the evaluation include at least clarity of speech, and clarity of the speech. Is represented by an inhibition sound-to-resonance ratio in the input audio signal, and the feature extraction unit divides the input audio signal into a plurality of segments, and converts the obtained segment into an inhibition sound and a resonance sound. An utterance evaluation device is disclosed, which obtains the disturbing sound-to-resonant sound ratio using a classifying unit.

JP-A-2015-68897

  Here, for example, in a technique for evaluating a sound as disclosed in Patent Literature 1, a database (a classifier or the like) generated using machine learning may be referred to. When machine learning is used, an enormous number of learning data is required to improve accuracy. In particular, when acquiring learning data for sounds, since parameters such as sound frequency and observation time are increased, the number of learning data files and memory occupancy tend to be required compared to characters and images. There is. For this reason, acquiring the learning data requires enormous time and expense, and it is an issue to easily acquire the learning data. In this regard, it is difficult to solve the above-described problem with the technology disclosed in Patent Document 1.

  The present invention has been devised in view of the above-described problems, and has as its object to provide a learning data generation method, a learning method, and an evaluation device capable of easily acquiring learning data. Is to do.

  A learning data generating method according to a first aspect of the present invention is a learning data generating method for artificially generating sound data used as learning data for machine learning, and extracts a part of a spectrogram converted from the sound data for learning. An acquisition step of acquiring the obtained reference image and a training image from which a part of the reference image has been deleted, and generating a first database by machine learning using the reference image and the training image as a pair of input data. A first database generating step, a generating step of generating a pseudo image based on the new reference image or the new training image by referring to the first database, and a converting step of converting the pseudo image into pseudo sound data And the following.

  In the learning data generation method according to a second invention, in the first invention, the generation step generates a plurality of the pseudo images for one new reference image or one new training image, Are converted into the different pseudo sound data, respectively.

  A learning data generation method according to a third invention is characterized in that, in the first invention or the second invention, the first database generation step generates the first database based on machine learning.

  A learning data generation method according to a fourth invention is characterized in that, in the first invention or the second invention, the first database generation step generates the first database based on machine learning of a generation system.

  According to a fifth aspect of the present invention, in the learning data generation method according to any one of the first to fourth aspects, the sound data includes a connector sound and a surrounding environment sound.

  The learning data generation method according to a sixth aspect is the learning data generation method according to any one of the first to fifth aspects, wherein the conversion step is performed on the pseudo sound data converted from the pseudo image using an inverse short-time Fourier transform. And adding noise generated by random numbers.

  A learning method according to a seventh aspect of the present invention is a learning method for performing machine learning using the pseudo sound data generated by the learning data generating method according to the first to sixth aspects as learning data, wherein the pseudo sound data and the pseudo sound data A second database generating step of generating a second database by machine learning using the evaluation data linked to the sound data as a pair of input data.

  An evaluation device according to an eighth invention is an evaluation device that evaluates sound data for an evaluation target using the second database generated by the learning method according to the seventh invention, and acquires the sound data for an evaluation target. It is characterized by comprising an acquisition unit and an evaluation unit that generates an evaluation result based on the evaluation target sound data with reference to the second database.

  An evaluation device according to a ninth aspect is an evaluation device that evaluates sound data for evaluation purposes, and includes a reference image that extracts a part of a spectrogram converted from sound data for learning, and a part of the reference image. A first acquisition unit that acquires the deleted training image; a first database generation unit that generates a first database by machine learning using the reference image and the training image as a pair of input data; With reference to a database, a pseudo image generation unit that generates a pseudo image based on the new reference image or the new training image, a conversion unit that converts the pseudo image into the pseudo sound data, and the pseudo sound data. A second database generating unit that generates a second database by machine learning using the evaluation data linked to the pseudo sound data as a pair of input data, A second obtaining unit that Tokusuru, with reference to the second database, characterized in that it comprises an evaluation unit for generating an evaluation result based on the sound data for the evaluation.

  A learning data generation method according to a tenth aspect is a learning data generation method for generating sound data to be used as learning data for machine learning in a pseudo manner, wherein reference data based on the sound data for learning, and reference data of the reference data. An acquisition step of acquiring training data with a part deleted, a first database generation step of generating a first database by machine learning using the reference data and the training data as a pair of input data, And generating a pseudo data based on the new reference data or the new training data with reference to one database.

  In a learning data generation method according to an eleventh aspect, in the tenth aspect, the generation step includes generating a plurality of the pseudo data with respect to one new reference data or one new training data. Features.

  A learning method according to a twelfth invention is a learning method for performing machine learning using the pseudo data generated by the learning data generation method according to the tenth invention or the eleventh invention as learning data, wherein the pseudo data and the pseudo data are A second database generating step of generating a second database by machine learning using the linked evaluation data as a pair of input data.

  An evaluation device according to a thirteenth invention is an evaluation device that evaluates sound data for an evaluation target using the second database generated by the learning method according to the twelfth invention, and acquires the sound data for an evaluation target. It is characterized by comprising an acquisition unit and an evaluation unit that generates an evaluation result based on the evaluation target sound data with reference to the second database.

  According to the first to eighth inventions, the generation step generates a pseudo image based on a new reference image or a new training image, and the conversion step converts the pseudo image into pseudo sound data. That is, even when the sound data used as the learning data is small, the pseudo sound data can be used as the learning data. For this reason, learning data used for machine learning can be easily acquired. This makes it possible to reduce the time and cost for acquiring the learning data.

  In particular, according to the eighth aspect, the evaluation unit refers to the second database and generates an evaluation result based on the sound data for evaluation. Therefore, even when the sound data used as the learning data is small, the accuracy of the evaluation result can be improved by referring to the second database generated by the machine learning using the pseudo sound data. .

  According to the ninth aspect, the pseudo image generation unit generates a pseudo image based on a new reference image or a new training image, and the conversion unit converts the pseudo image into pseudo sound data. That is, even when the sound data used as the learning data is small, the pseudo sound data can be used as the learning data. For this reason, learning data used for machine learning can be easily acquired. This makes it possible to reduce the time and cost for acquiring the learning data.

  According to the ninth aspect, the evaluation unit refers to the second database and generates an evaluation result based on the evaluation target sound data. Therefore, even when the sound data used as the learning data is small, the accuracy of the evaluation result can be improved by referring to the second database generated by the machine learning using the pseudo sound data. .

  According to the tenth to thirteenth aspects, the generation step generates pseudo data based on new reference data or new training data. That is, even when the sound data used as the learning data is small, the pseudo data can be used as the learning data. For this reason, learning data used for machine learning can be easily acquired. This makes it possible to reduce the time and cost for acquiring the learning data.

  In particular, according to the thirteenth aspect, the evaluation unit refers to the second database and generates an evaluation result based on the sound data for evaluation. Therefore, even when the sound data used as the learning data is small, the accuracy of the evaluation result can be improved by referring to the second database generated by the machine learning using the pseudo sound data. .

FIG. 1A is a schematic diagram illustrating an example of an application of the evaluation device according to the present embodiment, and FIG. 1B is a schematic diagram illustrating an outline of a learning method according to the present embodiment. 2) is a schematic diagram illustrating an outline of a learning data generation method according to the present embodiment. FIG. 2A is a schematic diagram illustrating an example of sound data, FIG. 2B is a schematic diagram illustrating an example of a spectrogram, and FIG. 2C is a schematic diagram illustrating an example of a reference image. FIG. 2D is a schematic diagram illustrating an example of a training image. FIG. 3 is a flowchart illustrating an example of the learning data generation method according to the present embodiment. FIGS. 4A to 4C are schematic diagrams showing a relationship among sound data, a spectrogram, a reference image, and a training image. FIG. 5 is a flowchart illustrating an example of the learning method according to the present embodiment. FIG. 6A is a schematic diagram illustrating an example of a configuration of the evaluation device according to the present embodiment, and FIG. 6B is a schematic diagram illustrating an example of functions of the evaluation device according to the present embodiment. FIG. 7 is a flowchart illustrating an example of the operation of the evaluation device according to the present embodiment. FIG. 8 is a schematic diagram illustrating an example of sound data and training data according to the present embodiment.

  Hereinafter, an example of a learning data generation method, a learning method, and an evaluation device in an embodiment to which the present invention is applied will be described with reference to the drawings.

  An example of a learning data generation method, a learning method, and an evaluation device 1 according to the present embodiment will be described with reference to FIG. FIG. 1A is a schematic diagram illustrating an example of a use of the evaluation device 1 according to the present embodiment, and FIG. 1B is a schematic diagram illustrating an outline of a learning method according to the present embodiment. (c) is a schematic diagram showing an outline of the learning data generation method in the present embodiment.

  The evaluation device 1 according to the present embodiment acquires sound data (evaluation target sound data) and outputs an evaluation result for the sound data, for example, as shown in FIG. The evaluation device 1 is installed, for example, in a factory or the like, and is used for evaluating whether or not a specific sound is included, for shipping inspection of a product, and for evaluating a space that requires control of environmental sound and noise sound. Used. As the evaluation device 1, for example, an electronic device such as a personal computer (PC) is used.

  The evaluation target sound data acquired by the evaluation device 1 is generated based on sounds collected by a sound collection device such as a microphone (not shown). The evaluation target sound data includes, for example, a machine sound such as an operation sound of an apparatus in a factory, an air-conditioning sound, a connector sound and the like, a surrounding environment sound, and a voice such as a human utterance. The evaluation target sound data is represented by an amplitude with respect to a time axis, for example, as shown in FIG.

  The evaluation device 1 refers to an evaluation database (second database) and outputs an evaluation result for the sound data. The evaluation device 1 outputs, for example, a result of evaluating the sound data for evaluation within (OK) or out of range (NG) such as a standard as an evaluation result. The content of the evaluation result can be arbitrarily set when the evaluation database is generated.

  In the learning method according to the present embodiment, as shown in FIG. 1B, for example, an evaluation database is generated by machine learning using artificially generated pseudo sound data as learning data. The learning data includes a plurality of pairs of pseudo-sound data and evaluation data associated with the pseudo-sound data.In addition, the learning data includes sound data generated from sound collected by a sound collection device such as a microphone and sound data. A plurality of pairs of data with the associated evaluation data may be included. The ratio of the pseudo sound data included in the learning data is arbitrary.

  The pseudo-sound data and the sound data are represented by an amplitude with respect to a time axis, for example, as shown in FIG. The evaluation data indicates, for example, an evaluation result represented by a binary value such as “OK” or “NG”, or may indicate a result of evaluating the characteristic of the linked sound data. The contents of the evaluation data can be arbitrarily set by a user or the like.

  In the learning method, for example, an evaluation database is generated using machine learning using a neural network as a model. The evaluation database is generated using, for example, machine learning using a CNN (Convolution Neural Network) or an auto encoder as a model, or an arbitrary model may be used. In the evaluation database, for example, the degree of association between the pseudo sound data and the evaluation data is stored. The degree of association indicates the degree of connection between the pseudo sound data and the evaluation data. For example, it can be determined that the higher the degree of association, the stronger the connection between the data. The association degree is indicated by, for example, three values or more (three steps or more) such as a percentage, or may be indicated by two values (two steps).

  The learning data generation method according to the present embodiment is to acquire a sample image, generate a pseudo image for the sample image, and convert the pseudo image into pseudo sound data, as shown in FIG. 1C, for example. In the learning data generation method, a pseudo image for a sample image is generated with reference to a generation database. The learning data generation method can generate at least one pseudo image for one sample image.

  In the learning data generation method, a generation database (first database) is generated using, for example, machine learning using a neural network as a model. In the learning data generation method, a generation database is generated using machine learning using a GAN (Generative Adversary Network) or an auto encoder as a model. In particular, using a machine learning using a model of pix2pix, which is a kind of conditional GAN, is used. , A generation database may be generated.

  In the learning data generation method, when generating a generation database, a plurality of pairs of data of a reference image and a training image are used as learning data. In the learning data generation method, a reference image obtained by extracting a part of a spectrogram obtained by converting sound data for learning and a training image obtained by deleting a part of the reference image are used.

  As the learning sound data used in the learning data generation method, for example, as shown in FIG. 2A, the same kind of data as the above-described evaluation target sound data is used. The spectrogram is represented by an intensity (amplitude) with respect to a time axis and a frequency axis as shown in FIG. 2B, for example, and is converted from sound data using, for example, a Fourier transform (for example, a short-time Fourier transform). In the learning data generation method, the spectrogram is used as image data. For example, 1 pixel × 1 pixel corresponds to a range of 0.064 sec × 15.624 Hz.

  As the reference image, for example, as shown in FIG. 2C, an image obtained by extracting a part (a broken line frame in FIG. 2B) of the spectrogram is used. As the reference image, for example, an image obtained by extracting a range of 0.512 sec × 1280 Hz in the spectrogram is used.

  As the training image, for example, as shown in FIG. 2D, an image in which a part of the reference image is deleted (a broken line portion in FIG. 2C) is used. As the training image, for example, an image from which the range of 0.512 sec × 125 Hz is deleted is used.

  The generation database is generated by machine learning using a reference image and a training image as a pair of input data, and can be used to generate pseudo data for input data (for example, a sample image). The generation of the pseudo data can be realized by using a technique of image complementation performed using a model having two networks of a generator and a discriminator such as GAN. That is, by learning the reference image as the correct image of the training image, the accuracy of generating the pseudo image can be increased.

  As a sample image used in the learning data generation method, for example, an image of the same type as the reference image or the training image is used. That is, as a sample image, a reference image (new reference image) in which a part of a spectrogram obtained by converting sound data obtained as a sample is extracted, or a training image in which a part of the reference image is deleted (new training image) Is used. Whichever image is used, a plurality of pseudo images can be generated from one sample image.

  The pseudo sound data generated by the learning data generation method is converted from the pseudo image using, for example, an inverse Fourier transform (for example, an inverse short-time Fourier transform). Thereby, pseudo sound data can be obtained for the sound data obtained as a sample.

In the learning data generation method, for example, data in which sound data for learning is not converted to an image is used as reference data, data in which a part of the reference data is deleted is used as training data, and reference data and training data are used as a pair of data. The generation database may be generated by machine learning. In this case, for example, the data obtained by extracting the amplitude of a specific period in the sound data is used as reference data, and the data obtained by removing the amplitude of the specific period in the reference data is used as training data. Here, the “amplitude” may be any of an analog value, a digital value, or an image display value. In this case, the same sample data as the reference data or the training data is used instead of the above-described sample image.
For example, FIG. 8 is a schematic diagram illustrating an example of sound data and training data according to the present embodiment. FIG. 8A shows an example of sound data for learning. FIG. 8B shows training data from which a part of the sound data has been deleted.

  Note that the evaluation device 1 has a learning data generation method and a function of executing the learning method described above, and acquires at least one of a generation database, pseudo sound data, and an evaluation database generated by another terminal or the like. May be.

(Learning data generation method)
Next, an example of a learning data generation method according to the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart illustrating an example of the learning data generation method according to the present embodiment. In the following, the operation using the reference image and the training image will be described. However, the same operation is performed when the reference data and the training data are used, and the description is omitted.

<Acquisition step S110>
First, reference information and training information are acquired (acquisition step S110). In the acquiring step S110, a reference image in which a part of a spectrogram converted from learning sound data serving as a reference for evaluation is extracted, and a training image in which a part of the reference image is deleted are acquired. In the acquisition step S110, for example, in addition to acquiring reference information and training information generated in advance, for example, sound data generated based on sounds collected by a sound collection device is obtained, and the sound data is converted into a spectrogram (see FIG. 4 (a)), the reference image is extracted from the spectrogram (FIG. 4 (b)), and the evaluation device 1 that executes the operation of acquiring the training image from which a part of the reference image is deleted (FIG. 4 (c)), etc. May be used. In this case, the range for acquiring the reference image from the spectrogram and the range for deleting a part of the reference image may be set in advance.

<First database generation step S120>
Next, a generation database (first database) is generated (first database generation step S120). In the first database generation step S120, a generation database is generated by machine learning using a reference image and a training image as a pair of input data. The input data includes a plurality (for example, about 1000) of a pair of data of a reference image and a training image. For example, for one reference image, a plurality of training images having different deletion locations are input data as a pair of data. May be included. In the first database generation step S120, a generation database is generated using, for example, machine learning using pix2pix as a model.

<Generation step S130>
Next, a pseudo image is generated (generation step S130). In the generation step S130, a pseudo image based on the sample image (new reference image or new training image) is generated with reference to the generation database. In the generation step S130, for example, the reference image or the training image used in the first database generation step S120 may be used as a sample image, and for example, a reference image or a training image not used in the first database generation step S120 may be sampled. It may be used as an image. In any case, the sample image can be obtained by the same method as the reference information or the training image obtained in the obtaining step S110.

  The generation step S130 may generate a plurality of pseudo images for one sample image, for example. In this case, the plurality of pseudo images are respectively converted into different pseudo sound data. Therefore, a plurality of pseudo sound data can be generated from one sound data. This makes it possible to easily secure learning data necessary for machine learning even when the learning data is small.

<Conversion step S140>
Next, the pseudo image is converted into pseudo sound data (conversion step S140). In the conversion step S140, the pseudo image is converted into pseudo sound data. In the conversion step S140, for example, noise generated by random numbers may be added to the pseudo sound data. For this reason, the pseudo sound data can be made closer to the sound actually acquired. This makes it possible to improve the quality of the learning data.

  By performing each step described above, the learning data generation method in the present embodiment is completed. When the reference data and the training data are used, the conversion step S140 may be omitted.

(Learning method)
Next, an example of a learning method according to the present embodiment will be described with reference to FIG. FIG. 5 is a flowchart illustrating an example of the learning method according to the present embodiment.

<Second database generation step S210>
In the learning method according to the present embodiment, an evaluation database (second database) is generated (second database generation step S210). In the second database generation step S210, an evaluation database is generated by machine learning using pseudo sound data and evaluation data associated with the pseudo sound data as a pair of input data. The input data includes a plurality of pairs of pseudo sound data and evaluation data. For example, the input data may include a plurality of pairs of sound data generated based on sounds collected from the sound collection device and evaluation data. In the second database generation step S210, an evaluation database is generated using, for example, machine learning using the CNN as a model.

  By performing the above steps, the learning method according to the present embodiment is completed.

(Evaluation device 1)
Next, an example of the evaluation device 1 in the present embodiment will be described with reference to FIG. FIG. 6A is a schematic diagram illustrating an example of a configuration of the evaluation device 1 in the present embodiment, and FIG. 6B is a schematic diagram illustrating an example of functions of the evaluation device 1 in the present embodiment.

  As shown in FIG. 6A, for example, the evaluation device 1 includes a housing 10, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and a storage unit. 104 and I / Fs 105 to 107. Each of the components 101 to 107 is connected by an internal bus 110.

  The CPU 101 controls the entire evaluation device 1. The ROM 102 stores the operation code of the CPU 101. The RAM 103 is a work area used when the CPU 101 operates. The storage unit 104 stores various information such as sound data. As the storage unit 104, for example, a data storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a floppy disk is used. Note that, for example, the evaluation device 1 may include a GPU (Graphics Processing Unit) not shown. By having a GPU, higher-speed arithmetic processing can be performed than usual.

  The I / F 105 is an interface for transmitting and receiving various types of information to and from other terminals and dedicated servers via the public communication network 2 such as the Internet. The I / F 106 is an interface for transmitting and receiving information to and from the input unit 108. For example, a keyboard is used as the input unit 108, and a user of the evaluation device 1 inputs various information or control commands of the evaluation device 1 via the input unit 108. The I / F 107 is an interface for transmitting and receiving various information to and from the output unit 109. The output unit 109 outputs various information stored in the storage unit 104 or the processing status of the evaluation device 1 or the like. A display is used as the output unit 109, and may be, for example, a touch panel type.

  FIG. 6B is a schematic diagram illustrating an example of a function of the evaluation device 1. The evaluation device 1 includes an acquisition unit 11, a database generation unit 12, a pseudo image generation unit 13, a conversion unit 14, an evaluation unit 15, an output unit 16, and a storage unit 17. The function illustrated in FIG. 6B is realized by the CPU 101 executing a program stored in the storage unit 104 or the like using the RAM 103 as a work area, and may be controlled by, for example, artificial intelligence.

<Acquisition unit 11>
The obtaining unit 11 obtains evaluation target sound data. The acquisition unit 11 may acquire, for example, sound data (pseudo-sound data) for generating an evaluation database, evaluation data, and the like. The acquisition unit 11 may acquire, for example, sound data, reference information, training information, a sample image, and the like for generating pseudo sound data.

  The acquisition unit 11 may include, for example, a first acquisition unit and a second acquisition unit, and acquire different information. In this case, the first acquisition unit acquires the reference image and the training image, and the second acquisition unit acquires the sound data for evaluation.

<Database generation unit 12>
The database generation unit 12 generates at least one of a generation database and an evaluation database. The method of generating at least one of the generation database and the evaluation database is the same as the above-described content. When the generation database and the evaluation database are generated by another terminal or the like, the evaluation device 1 may not include the database generation unit 12.

  The database generation unit 12 may include, for example, a first database generation unit and a second database generation unit, and generate different databases. In this case, the first database generation unit generates a generation database, and the second database generation unit generates an evaluation database.

<Pseudo image generation unit 13>
The pseudo image generation unit 13 generates a pseudo image based on the sample image (new reference image or new training image) with reference to the generation database. The method of generating the pseudo image is the same as that described above. When the evaluation database is generated by another terminal or the like, the evaluation device 1 may not include the pseudo image generation unit 13.

<Conversion unit 14>
The conversion unit 14 converts the pseudo image into pseudo sound data. The converter 14 adds, for example, noise generated by random numbers to the pseudo sound data. The method of converting the pseudo sound data is the same as described above. When the evaluation database is generated by another terminal or the like, the evaluation device 1 may not include the conversion unit 14.

<Evaluation unit 15>
The evaluation unit 15 generates an evaluation result based on the evaluation target sound data with reference to the evaluation database. The method of generating the evaluation result is the same as that described above.

<Output unit 16>
The output unit 16 outputs the evaluation result and the like to the output unit 109 and the like. The output unit 16 transmits the evaluation result to another terminal or the like via the public communication network 2, for example.

<Storage unit 17>
The storage unit 17 stores the various information acquired by the acquisition unit 11, the evaluation result generated by the evaluation unit 15, and the like in the storage unit 104, and extracts the various information stored in the storage unit 104 as needed.

(Operation of the evaluation device 1)
Next, an example of the operation of the evaluation device 1 according to the present embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating an example of the operation of the evaluation device 1 according to the present embodiment.

<Evaluation data acquisition step S310>
First, evaluation target sound data is obtained (evaluation data obtaining step S310). The acquisition unit 11 acquires, for example, sound data generated based on the sound collected by the sound collection device as sound data for evaluation. The obtaining unit 11 obtains evaluation target sound data in the same format as the above-described learning sound data. The obtaining unit 11 stores the obtained evaluation target sound data in the storage unit 104 via, for example, the storage unit 17.

<Evaluation result generation step S320>
Next, an evaluation result based on the evaluation target sound data is generated (evaluation result generation step S320). The evaluation unit 15 refers to the evaluation database and generates an evaluation result based on the evaluation target sound data. For example, the evaluation unit 15 may generate one evaluation result for one evaluation target sound data, or may generate one evaluation result for a plurality of evaluation target sound data.

  By performing the above-described steps, the operation of the evaluation device 1 in the present embodiment is completed. When the learning data generation method or the learning method is performed using the evaluation device 1, the learning data generation method or the learning method can be performed using the database generation unit 12 described above.

  According to the present embodiment, the generation step S130 generates a pseudo image based on a sample image (new reference image or new training image), and the conversion step S140 converts the pseudo image into pseudo sound data. That is, even when the sound data used as the learning data is small, the pseudo sound data can be used as the learning data. For this reason, learning data used for machine learning can be easily acquired. This makes it possible to reduce the time and cost for acquiring the learning data.

  Further, according to the present embodiment, the evaluation unit 15 generates an evaluation result based on the evaluation target sound data with reference to the evaluation database (second database). Therefore, even when the sound data used as the learning data is small, the accuracy of the evaluation result can be improved by referring to the evaluation database generated by the machine learning using the pseudo sound data. .

  Further, according to the present embodiment, the pseudo image generation unit 13 generates a pseudo image based on a new reference image or a new training image, and the conversion unit 14 converts the pseudo image into pseudo sound data. That is, even when the sound data used as the learning data is small, the pseudo sound data can be used as the learning data. For this reason, learning data used for machine learning can be easily acquired. This makes it possible to reduce the time and cost for acquiring the learning data.

  According to the present embodiment, the evaluation unit 15 generates an evaluation result based on the evaluation target sound data with reference to the evaluation database. Therefore, even when the sound data used as the learning data is small, the accuracy of the evaluation result can be improved by referring to the evaluation database generated by the machine learning using the pseudo sound data. .

  Further, according to the present embodiment, the plurality of pseudo images are respectively converted into different pseudo sound data. Therefore, a plurality of pseudo sound data can be generated from one sound data. This makes it possible to easily secure learning data necessary for machine learning even when the learning data is small.

  According to the present embodiment, the first database generation step S120 generates a generation database based on the GAN. For this reason, pseudo data can be easily generated as compared with the case where another learning model is used.

  Further, according to the present embodiment, the conversion step S140 may add noise generated by random numbers to the pseudo sound data. For this reason, the pseudo sound data can be made closer to the sound actually acquired. This makes it possible to improve the quality of the learning data.

  Further, according to the present embodiment, the generation step S130 generates pseudo data based on sample data (new reference data or new training data). That is, even when the sound data used as the learning data is small, the pseudo data can be used as the learning data. For this reason, learning data used for machine learning can be easily acquired. This makes it possible to reduce the time and cost for acquiring the learning data.

  Although an embodiment of the present invention has been described, this embodiment is provided by way of example and is not intended to limit the scope of the invention. These new embodiments can be implemented in other various forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and their equivalents.

1: evaluation device 2: public communication network 10: housing 11: acquisition unit 12: database generation unit 13: pseudo image generation unit 14: conversion unit 15: evaluation unit 16: output unit 17: storage unit 101: CPU
102: ROM
103: RAM
104: storage unit 105: I / F
106: I / F
107: I / F
108: input part 109: output part 110: internal bus S110: acquisition step S120: first database generation step S130: generation step S140: conversion step S210: second database generation step S310: evaluation data acquisition step S320: evaluation result generation step

Claims (13)

A learning data generation method for artificially generating sound data used as learning data for machine learning,
An acquisition step of acquiring a reference image in which a part of a spectrogram converted from the sound data for learning is extracted, and a training image in which a part of the reference image is deleted,
A first database generation step of generating a first database by machine learning using the reference image and the training image as a pair of input data;
A generation step of generating a pseudo image based on the new reference image or the new training image by referring to the first database;
A conversion step of converting the pseudo image into pseudo sound data;
A learning data generation method, comprising:
The generating step generates a plurality of the pseudo images for one new reference image or one new training image,
The learning data generation method according to claim 1, wherein the plurality of pseudo images are respectively converted into different pseudo sound data.
The learning data generation method according to claim 1, wherein the first database generation step generates the first database based on machine learning.
The learning data generation method according to claim 1 or 2, wherein the first database generation step generates the first database based on machine learning of a generation system.
The learning data generation method according to any one of claims 1 to 4, wherein the sound data includes a connector sound and a surrounding environment sound.
The said conversion step adds the noise which generate | occur | produced by the random number with respect to the said pseudo sound data converted from the said pseudo image using the inverse short-time Fourier-transform, The noise according to any one of Claims 1-5. The learning data generation method described in the section.
A learning method for performing machine learning on the pseudo sound data generated by the learning data generation method according to any one of claims 1 to 6 as learning data,
A second database generating step of generating a second database by machine learning using the pseudo sound data and the evaluation data associated with the pseudo sound data as a pair of input data;
A learning method, comprising:
An evaluation device that evaluates evaluation target sound data using the second database generated by the learning method according to claim 7,
An acquiring unit for acquiring the evaluation target sound data,
An evaluation unit that generates an evaluation result based on the evaluation target sound data with reference to the second database;
An evaluation device comprising:
An evaluation device for evaluating sound data for an evaluation object,
A first acquisition unit configured to acquire a reference image obtained by extracting a part of the spectrogram converted from the sound data for learning, and a training image obtained by deleting a part of the reference image;
A first database generation unit that generates a first database by machine learning using the reference image and the training image as a pair of input data;
A pseudo image generation unit that generates a pseudo image based on the new reference image or the new training image by referring to the first database;
A conversion unit that converts the pseudo image into pseudo sound data,
A second database generating unit that generates a second database by machine learning using the pseudo sound data and the evaluation data associated with the pseudo sound data as a pair of input data;
A second acquisition unit that acquires the evaluation target sound data;
An evaluation unit that generates an evaluation result based on the evaluation target sound data with reference to the second database;
An evaluation device comprising:
A learning data generation method for artificially generating sound data used as learning data for machine learning,
An acquisition step of acquiring reference data based on the sound data for learning, and training data in which a part of the reference data has been deleted,
A first database generation step of generating a first database by machine learning using the reference data and the training data as a pair of input data;
A generating step of referring to the first database and generating pseudo data based on the new reference data or the new training data;
A learning data generation method, comprising:
The learning data generating method according to claim 10, wherein the generating step generates a plurality of the pseudo data for one new reference data or one new training data.
A learning method for performing machine learning on the pseudo data generated by the learning data generation method according to claim 10 or 11, as learning data,
A second database generation step of generating a second database by machine learning using the pseudo data and the evaluation data associated with the pseudo data as a pair of input data;
A learning method, comprising:
An evaluation device that evaluates evaluation target sound data using the second database generated by the learning method according to claim 12,
An acquiring unit for acquiring the evaluation target sound data,
An evaluation unit that generates an evaluation result based on the evaluation target sound data with reference to the second database;
An evaluation device comprising: