WO2023090193A1 - Information processing device, control system, information processing method, and program - Google Patents

Information processing device, control system, information processing method, and program Download PDF

Info

Publication number
WO2023090193A1
WO2023090193A1 PCT/JP2022/041433 JP2022041433W WO2023090193A1 WO 2023090193 A1 WO2023090193 A1 WO 2023090193A1 JP 2022041433 W JP2022041433 W JP 2022041433W WO 2023090193 A1 WO2023090193 A1 WO 2023090193A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
abnormality
model
information processing
transforming
Prior art date
Application number
PCT/JP2022/041433
Other languages
French (fr)
Japanese (ja)
Inventor
利次 土井
篤淑 籔内
慎司 伊藤
Original Assignee
株式会社エヌ・ティ・ティ・データCcs
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社エヌ・ティ・ティ・データCcs filed Critical 株式会社エヌ・ティ・ティ・データCcs
Publication of WO2023090193A1 publication Critical patent/WO2023090193A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01H—MEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES
    • G01H17/00—Measuring mechanical vibrations or ultrasonic, sonic or infrasonic waves, not provided for in the preceding groups
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2131—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on a transform domain processing, e.g. wavelet transform
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2123/00—Data types
    • G06F2123/02—Data types in the time domain, e.g. time-series data

Definitions

  • Embodiments relate to an information processing device, a control system, an information processing method, and a program.
  • ISO International Organization for Standard 532 defines loudness as an index in sound analysis. Loudness is a psychological quantity indicating the loudness of sound based on human hearing. In the sensory test, the loudness is used as an aid to judgment by the inspectors.
  • the judgment criteria based on the sensory test described above may depend on the subjectivity of the inspector. Therefore, different inspectors may obtain different determination results. That is, it is not possible to objectively determine the presence or absence of an abnormality in the mechanism by the sensory inspection described above.
  • the present invention has been made with a focus on the above circumstances, and its purpose is to provide means for automatically and objectively determining the presence or absence of an abnormality in the mechanism.
  • An information processing apparatus calculates a plurality of values each associated with a set of a frequency and a fluctuation frequency based on time-series data; and calculating a probability that the data contains an anomaly.
  • FIG. 1 is a block diagram showing an example of the configuration of a mechanism control system according to an embodiment;
  • 1 is a block diagram showing an example of a hardware configuration of a model learning device according to an embodiment;
  • FIG. 1 is a block diagram showing an example of a functional configuration of a model learning device according to an embodiment
  • FIG. FIG. 4 is a diagram showing an example of the relationship between a plurality of teacher feature quantity candidate maps and a teacher feature quantity map generated by the model learning device according to the embodiment
  • 4 is a flowchart showing an example of mechanism control processing in the mechanism control system according to the embodiment
  • 4 is a flowchart showing an example of abnormality determination processing in the abnormality determination device according to the embodiment
  • 4 is a flowchart showing an example of feature quantity map generation processing in the abnormality determination device according to the embodiment
  • 4 is a flowchart showing an example of model learning processing in the model learning device according to the embodiment
  • 4 is a flowchart showing an example of teacher feature quantity map generation processing in the model learning device according to the embodiment.
  • FIG. 1 is a block diagram showing an example of the configuration of the mechanism control system according to the embodiment.
  • the mechanism control system 1 includes a mechanism 2, a sensor 3, an abnormality determination device 4, a model learning device 5, and a control device 6.
  • Mechanism 2 is the sound source for abnormality determination.
  • the mechanism 2 is, for example, an electric seat of an automobile or a wind power generation facility in which a motor or the like is assembled.
  • Mechanism 2 may be assembled with a plurality of motors each having equivalent specifications.
  • Mechanism 2 generates sound and vibration when operating in a predetermined mode of operation.
  • the operation mode is, for example, a reclining operation, a forward/backward moving operation, a left/right moving operation, and the like. Sounds and vibrations from the mechanism 2 may vary depending on the operating mode of the mechanism and the presence or absence of anomalies within the mechanism.
  • the operating mode of mechanism 2 is controlled by a control signal CNT from controller 6 .
  • the control signal CNT contains, for example, information instructing the mechanism 2 to start, suspend and end the operating mode.
  • the sensor 3 is, for example, a microphone or a vibrometer.
  • a sensor 3 is arranged in the vicinity of or in contact with the mechanism 2 .
  • Sensor 3 measures sound or vibration generated by operation of mechanism 2 .
  • the sound or vibration measured by the sensor 3 is transmitted to the abnormality determination device 4 as time-series measurement data S.
  • the measurement data S may be either an analog signal or a digital signal.
  • the abnormality determination device 4 is an information processing device such as a personal computer.
  • the abnormality determination device 4 executes abnormality determination processing regarding the measurement data S based on the operation mode signal MODE from the control device 6 .
  • the operation mode signal MODE is a signal for identifying the operation mode of mechanism 2 .
  • the abnormality determination device 4 generates a determination history R and an abnormality notification signal ANOM as a result of the abnormality determination process.
  • the abnormality notification signal ANOM is a signal for notifying the control device 6 that the measurement data S is abnormal.
  • the determination history R is information indicating whether or not the measurement data S is abnormal.
  • the abnormality determination device 4 transmits an abnormality notification signal ANOM to the control device 6 and outputs the determination history R to the user. Details of the abnormality determination process will be described later.
  • the model learning device 5 is an information processing device such as a personal computer.
  • the model learning device 5 executes model learning processing.
  • the model learning device 5 generates a judgment model M as a result of model learning processing.
  • the model learning device 5 transmits the generated determination model M to the abnormality determination device 4 . Details of the model learning process will be described later.
  • the judgment model M is, for example, a mathematical model including a neural network.
  • the determination model M is used for abnormality determination processing in the abnormality determination device 4 .
  • a neural network in the decision model M includes multiple parameters.
  • a plurality of parameters in the judgment model M are optimized for abnormality judgment processing by model learning processing.
  • the judgment model M has, for example, different models for each operation mode of the mechanism 2 .
  • the control device 6 is a control terminal that mainly controls the operation of the mechanism 2.
  • the control device 6 is a test device for checking the operation of the electric seat in the pre-shipment inspection.
  • the control device 6 is a control device for real-time operation control and failure monitoring of the installed facility.
  • Control device 6 transmits an operation mode signal MODE to abnormality determination device 4 .
  • Control device 6 receives an abnormality notification signal ANOM from abnormality determination device 4 .
  • Control device 6 sends control signal CNT to mechanism 2 .
  • FIG. 2 is a block diagram showing an example of the hardware configuration of the abnormality determination device according to the embodiment.
  • the abnormality determination device 4 includes a control circuit 11, a storage 12, a communication module 13, a user interface 14, a drive 15, and a storage medium 16.
  • the control circuit 11 is a circuit that controls each component of the abnormality determination device 4 as a whole.
  • the control circuit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and the like.
  • the storage 12 is an auxiliary storage device for the abnormality determination device 4.
  • the storage 12 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a memory card.
  • the storage 12 stores the measurement data S, the determination model M, and the like used for the abnormality determination process.
  • the storage 12 may store a program for executing the abnormality determination process.
  • the communication module 13 is a circuit used for transmitting and receiving data with the sensor 3 and with the model learning device 5.
  • the user interface 14 is a circuit for communicating information between the user and the control circuit 11 .
  • User interface 14 includes input and output devices.
  • the input device includes, for example, a touch panel and operation buttons.
  • Output devices include, for example, displays and printers. Also, the output device may include a lamp, a buzzer, or the like.
  • the drive 15 is a device for reading software stored in the storage medium 16.
  • the drive 15 includes, for example, a CD (Compact Disk) drive, a DVD (Digital Versatile Disk) drive, and the like.
  • the storage medium 16 is a medium that stores software by electrical, magnetic, optical, mechanical or chemical action.
  • the storage medium 16 may store a program for executing the abnormality determination process.
  • FIG. 3 is a block diagram showing an example of the hardware configuration of the model learning apparatus according to the embodiment.
  • model learning device 5 includes control circuit 21 , storage 22 , communication module 23 , user interface 24 , drive 25 and storage medium 26 .
  • the control circuit 21 is a circuit that controls each component of the model learning device 5 as a whole.
  • the control circuit 21 includes a CPU, RAM, ROM, and the like.
  • the storage 22 is an auxiliary storage device for the model learning device 5.
  • the storage 22 is, for example, an HDD, SSD, memory card, or the like.
  • the storage 22 stores data used for model learning processing.
  • the storage 22 may also store a program for executing model learning processing.
  • the communication module 23 is a circuit used for transmitting and receiving data to and from the abnormality determination device 4.
  • the user interface 24 is a circuit for communicating information between the user and the control circuit 21 .
  • User interface 24 includes input and output devices.
  • the input device includes, for example, a touch panel and operation buttons.
  • Output devices include, for example, displays and printers.
  • the drive 25 is a device for reading software stored in the storage medium 26.
  • the drive 25 includes, for example, a CD drive, a DVD drive, and the like.
  • the storage medium 26 is a medium that stores software by electrical, magnetic, optical, mechanical or chemical action.
  • the storage medium 26 may store a program for executing model learning processing.
  • FIG. 4 is a block diagram showing an example of the functional configuration of the abnormality determination device according to the embodiment.
  • the CPU of the control circuit 11 expands a program related to abnormality determination processing stored in the storage 12 or the storage medium 16 to the RAM. Then, the CPU of the control circuit 11 interprets and executes the program developed in the RAM.
  • the abnormality determination device 4 functions as a computer including a first conversion unit 31 , a spectrogram generation unit 32 , a second conversion unit 33 , a feature map generation unit 34 , a probability calculation unit 35 and a determination unit 36 .
  • the first conversion unit 31 is a functional block that converts the time-series measurement data S into the frequency domain.
  • the first converter 31 converts the measurement data S from analog signals to digital signals.
  • the first converter 31 divides the measurement data S converted into digital signals into data units each having a length of period T1.
  • the period T1 is, for example, several milliseconds to several tens of milliseconds.
  • Two data units adjacent in time series may have overlapping periods. Two data units adjacent in time series may not have periods that overlap each other.
  • the first transformation unit 31 can perform, for example, fast Fourier transformation (FFT) processing on the measurement data S for each data unit.
  • FFT fast Fourier transformation
  • the first conversion unit 31 may convert the measurement data S into the frequency domain by using a plurality of bandpass filters that transmit different frequency bands, such as octave band analysis.
  • the first conversion unit 31 transmits the result of conversion processing for each data unit to the spectrogram generation unit 32 .
  • the conversion process to the frequency domain is also called "first conversion process”.
  • the sampling width (frequency band) of the frequency domain in the first conversion processing of the first conversion unit 31 does not have to be divided into constant widths like the FFT processing.
  • the frequency band in the first conversion process of the first conversion unit 31 may be divided into 1/3 octave bands or bands based on the Mel scale.
  • application of the Bark scale to frequency bands is not preferable from the viewpoint of computer load.
  • the spectrogram generation unit 32 is a functional block that generates a spectrogram using the result of the first conversion processing by the first conversion unit 31.
  • the spectrogram generation unit 32 accumulates the results of the first conversion processing for each data unit by the first conversion unit 31 in time series over the period T2.
  • the period T2 is longer than the period T1.
  • the period T2 is, for example, several hundred milliseconds.
  • the spectrogram generator 32 generates a spectrogram based on the results of the multiple first conversion processes accumulated over the period T2.
  • the spectrogram generator 32 transmits the generated spectrogram to the second converter 33 .
  • FIG. 5 is a diagram showing an example of a spectrogram generated by the abnormality determination device according to the embodiment.
  • a spectrogram is a collection of three-dimensional data. The three components of each three-dimensional data in the spectrogram correspond to time, frequency, and signal strength (amplitude), respectively.
  • the X and Y axes correspond to time and frequency, respectively.
  • the magnitude of amplitude corresponding to a certain time width and frequency band is indicated by the shade of color for each grid.
  • a grid in a spectrogram is a rectangular area in the spectrogram that is delimited by one time span and one frequency band.
  • a plurality of grids (aligned in the Y direction) corresponding to a certain time width correspond to the results of the first conversion processing for one data unit.
  • a spectrogram is formed by arranging the results of the first conversion processing for such data units in time series (in the X direction) in a number corresponding to the period T2.
  • a spectrogram is a set of multiple data groups arranged in time series in corresponding frequency bands.
  • the second conversion unit 33 is a functional block that converts the spectrogram generated by the spectrogram generation unit 32 into the fluctuating frequency domain. Fluctuation frequency is a frequency representation of the time variation of the peak value in a signal that oscillates at a certain frequency.
  • the second conversion unit 33 divides the spectrogram into multiple data groups. A plurality of data groups correspond to frequency bands different from each other. That is, each of the multiple data groups includes multiple grids (aligned in the X direction) corresponding to a certain frequency band in the spectrogram.
  • the second conversion unit 33 can perform FFT processing for each data group, for example.
  • the second transforming unit 33 may transform the spectrogram into the variable frequency domain by using a plurality of bandpass filters that transmit different frequency bands, such as octave band analysis.
  • the second conversion unit 33 transmits the result of conversion processing for each data group to the feature map generation unit 34 .
  • the conversion processing to the variable frequency domain is also called "second conversion processing".
  • the sampling width (fluctuation frequency band) of the fluctuation frequency band in the second conversion processing of the second conversion unit 33 does not have to be divided into constant widths unlike the FFT processing.
  • the variable frequency band in the second conversion process of the second conversion unit 33 may be divided into 1/3 octave bands or bands based on the Mel scale.
  • application of the Bark scale to fluctuating frequency bands is not preferable from the viewpoint of computer load.
  • the second conversion unit 33 may perform each of the level correction process, the time masking process, the frequency masking process, and the time weighting process that are performed when calculating the loudness.
  • the level correction process, time masking process, frequency masking process, and time weighting process are unnecessary. Therefore, the second conversion unit 33 does not need to perform level correction processing, time masking processing, frequency masking processing, and time weighting processing.
  • the feature map generation unit 34 is a functional block that uses the result of the second conversion processing by the second conversion unit 33 to generate a feature map.
  • the feature quantity map generation unit 34 accumulates the results of the second conversion processing for all data groups generated from one spectrogram as feature quantities.
  • the feature quantity map generation unit 34 generates a feature quantity map by mapping the accumulated feature quantity.
  • the feature map generation unit 34 transmits the generated feature map to the probability calculation unit 35 .
  • FIG. 6 is a diagram showing an example of a feature quantity map generated by the abnormality determination device according to the embodiment.
  • a feature map is a set of three-dimensional data. Three components of each three-dimensional data in the feature quantity map respectively correspond to frequency, fluctuation frequency, and signal strength (amplitude).
  • the X and Y axes correspond to frequency and variation frequency, respectively.
  • the magnitude of the amplitude corresponding to a certain frequency band and the variable frequency band is indicated by the shade of color for each grid.
  • a grid in the feature map is a rectangular area in the feature map that is partitioned by one frequency band and one variable frequency band.
  • a plurality of grids (aligned in the Y direction) corresponding to a certain frequency band correspond to the result of the second transformation processing for one data group.
  • a feature quantity map is formed by arranging a plurality of results of the second conversion processing of such a data group in the frequency direction (in the X direction).
  • the probability calculation unit 35 is a functional block that calculates the probability that the measurement data S is abnormal.
  • the probability calculator 35 selects the determination model M corresponding to the operation mode of the mechanism 2 based on the operation mode signal MODE.
  • the probability calculation unit 35 inputs the feature quantity map to the selected determination model M.
  • the probability calculation unit 35 calculates the abnormality probability of the measurement data S based on the output result from the determination model M to which the feature quantity map is input.
  • the abnormality probability is, for example, a real number between 0 and 1 inclusive. For example, the higher the abnormality probability, the higher the probability that the measurement data S is abnormal.
  • the probability calculation unit 35 transmits the calculated abnormality probability to the determination unit 36 .
  • the judgment model M includes a neural network. More specifically, the neural network included in the judgment model M is, for example, a convolutional neural network (CNN).
  • CNN convolutional neural network
  • the feature quantity map is input to the determination model M as if it were image information.
  • the amplitude values in each grid of the feature map are handled by the determination model M in the same way as the pixel values in the image.
  • the determination unit 36 is a functional block that determines the presence or absence of an abnormality in the measurement data S based on the abnormality probability.
  • the determination unit 36 executes determination processing each time it receives an abnormality probability from the probability calculation unit 35 .
  • the determination unit 36 determines that the measurement data S is abnormal.
  • the condition may be, for example, that the abnormality probability is greater than or equal to a first threshold (eg, 0.5). Further, the condition may be, for example, that the number of times the abnormality probability becomes equal to or greater than the first threshold is equal to or greater than the second threshold. In this way, arbitrary conditions are applied according to the operation mode of the mechanism 2 and the characteristics of the mechanism 2 .
  • the determination unit 36 generates a determination history R in which the results of determination processing are arranged in chronological order, and outputs it to the user. Further, when it is determined that the measurement data S has an abnormality, the determination unit 36 generates an abnormality notification signal ANOM and transmits it to the control device 6 . Note that, when it is determined that the measurement data S has an abnormality, the determination unit 36 may notify the user of the abnormality by sounding a buzzer, lighting a lamp, or the like.
  • FIG. 7 is a diagram showing an example of determination history generated by the abnormality determination device according to the embodiment.
  • the example of FIG. 7 shows a case where the determination history R is output to the user as a graph in which the abnormality probabilities are arranged in chronological order.
  • the determination history R is updated in the time direction each time a new abnormality probability is calculated.
  • the determination unit 36 controls the abnormality notification signal ANOM at each timing from time t1 to t8. Send to device 6 .
  • the abnormality determination device 4 determines whether or not there is an abnormality in the mechanism 2 based on the measurement data S using the determination model M, and notifies the user and the control device 6 of the result. be able to. Then, the control device 6 can interrupt the operation mode of the mechanism 2 by receiving the abnormality notification signal ANOM from the abnormality determination device 4 .
  • Model Learning Device A functional configuration of the model learning device according to the embodiment will be described.
  • FIG. 8 is a block diagram showing an example of the functional configuration of the model learning device according to the embodiment.
  • the CPU of the control circuit 21 loads a program related to model learning processing stored in the storage 22 or the storage medium 26 into the RAM. Then, the CPU of the control circuit 21 interprets and executes the program developed in the RAM.
  • the model learning device 5 functions as a computer including a first conversion unit 41 , a spectrogram generation unit 42 , a second conversion unit 43 , a teacher feature map generation unit 44 , a probability calculation unit 45 and an update unit 46 .
  • the model learning device 5 also stores the teacher data set T and the pre-learning model M0.
  • the pre-learning model M0 is a mathematical model including a neural network.
  • the configuration of the pre-learning model M0 is the same as that of the judgment model M, except that the parameter values are different.
  • the pre-learning model M0 is a determination model in an initial state, that is, before the parameters are optimized for the abnormality determination process.
  • a teacher data set T includes multiple teacher data.
  • Each of the plurality of teacher data includes data and labels associated with each other.
  • the data in each teacher data is, for example, data obtained by measuring the sound or vibration from the mechanism 2 in advance.
  • the length of data in each teacher data is, for example, period T3 or longer. Period T3 is longer than period T2. Therefore, a plurality of feature quantity maps can be generated from the data in each teacher data.
  • the label in each training data is information indicating the true judgment result for the corresponding data.
  • the labels within each training data include the anomaly probability of the corresponding data. That is, a label corresponding to data including abnormal sound or vibration indicates an abnormality probability of "1". On the other hand, a label corresponding to data that does not contain abnormal sound or vibration indicates an abnormality probability of "0".
  • the teacher data set T is processed by the model learning device 5 using teacher data as a processing unit.
  • the functional configuration of the model learning device 5 for each teacher data will be described below.
  • the functional configurations of the first conversion unit 41, the spectrogram generation unit 42, and the second conversion unit 43 are the first conversion unit 31, the spectrogram generation unit 32, and the It is equivalent to the second converter 33 .
  • the first conversion unit 41 is a functional block that converts time-series teacher data into the frequency domain.
  • the first conversion unit 41 divides the teacher data into data units having a length of period T1.
  • the first conversion unit 41 performs, for example, a first conversion process (for example, an FFT process) for each data unit on teacher data.
  • the first conversion unit 41 transmits the result of the first conversion process for each data unit to the spectrogram generation unit 42 . From the viewpoint of consistency between the judgment model M and the pre-learning model M0, it is desirable that the same frequency band as that of the first conversion unit 31 is applied as the frequency band in the first conversion processing of the first conversion unit 41 .
  • the spectrogram generation unit 42 is a functional block that generates a spectrogram using the result of the first conversion processing by the first conversion unit 41.
  • the spectrogram generation unit 42 accumulates the results of the first conversion processing for each data unit by the first conversion unit 41 in time series over the period T2.
  • the spectrogram generator 42 generates a spectrogram based on the results of a plurality of first conversion processes accumulated over the period T2.
  • the spectrogram generator 42 transmits the generated spectrogram to the second converter 43 .
  • the second conversion unit 43 is a functional block that converts the spectrogram generated by the spectrogram generation unit 42 into the fluctuating frequency domain.
  • the second conversion unit 43 divides the spectrogram into multiple data groups.
  • the second conversion unit 43 performs, for example, second conversion processing (for example, FFT processing) for each data group.
  • the second conversion unit 43 transmits the result of the second conversion processing for each data group to the teacher feature quantity map generation unit 44 . It should be noted that, from the viewpoint of consistency between the determination model M and the pre-learning model M0, the same fluctuation frequency band as that of the second conversion unit 33 may be applied to the fluctuation frequency band in the second conversion processing of the second conversion unit 43. desirable.
  • the teacher feature quantity map generation unit 44 is a functional block that uses the result of the second conversion processing by the second conversion unit 43 to generate a teacher feature quantity map.
  • the teacher feature quantity map generation unit 44 accumulates the results of the second conversion processing of all data groups generated from one spectrogram as teacher feature quantity candidates.
  • the teacher feature quantity map generating unit 44 generates a teacher feature quantity candidate map for each spectrogram by mapping the accumulated teacher feature quantity candidates. Then, the teacher feature quantity map generating unit 44 generates a teacher feature quantity map based on a plurality of teacher feature quantity candidate maps corresponding to a plurality of spectrograms.
  • the teacher feature quantity map generation unit 44 transmits the generated teacher feature quantity map to the probability calculation unit 45 .
  • FIG. 9 is a diagram showing an example of the relationship between a plurality of teacher feature quantity candidate maps generated by the model learning device according to the embodiment and the teacher feature quantity map. As shown in FIG. 9, for the amplitude value of each grid in the teacher feature quantity map, for example, the maximum value of the amplitude values in the corresponding grids of the plurality of teacher feature quantity candidate maps is applied.
  • model learning device 5 The functional configuration of the model learning device 5 will be described with reference to FIG. 8 again.
  • the functional configuration of the probability calculation unit 45 is the same as that of the probability calculation unit 35, except that the teacher feature map is input instead of the feature map, and that the pre-learning model M0 is used instead of the judgment model M. be.
  • the probability calculation unit 45 is a functional block that calculates the probability that teacher data is abnormal.
  • the probability calculation unit 45 inputs the teacher feature quantity map to the pre-learning model M0.
  • the probability calculation unit 45 calculates the abnormality probability of teacher data based on the output result from the pre-learning model M0 to which the teacher feature quantity map is input.
  • the probability calculator 45 transmits the calculated abnormality probability to the updater 46 .
  • the update unit 46 is a functional block that updates the parameters of the pre-learning model M0 based on the abnormality probability calculated by the probability calculation unit 45. Specifically, for example, the updating unit 46 calculates an evaluation function based on the calculated abnormality probability and label. For the evaluation function, for example, a function that minimizes when the calculated abnormality probability matches the label is applied. When the calculated evaluation function is equal to or greater than the threshold, the updating unit 46 updates the parameters of the pre-learning model M0. When updating the parameters, the updating unit 46 uses, for example, the error backpropagation method. The pre-learning model M ⁇ b>0 whose parameters have been updated by the update unit 46 is fed back to the probability calculation unit 45 . When all the evaluation functions calculated for the teacher data set T are less than the threshold, the update unit 46 transmits the pre-learning model M0 as the determination model M to the abnormality determination device 4.
  • the model learning device 5 can provide the abnormality determination device 4 with the determination model M optimized for the abnormality determination process.
  • FIG. 10 is a flowchart showing an example of mechanism control processing in the mechanism control system according to the embodiment.
  • the control device 6 transmits an operation mode signal MODE corresponding to the input operation mode to the abnormality determination device 4 (S1).
  • the operation mode signal MODE is used for selecting the determination model M in the abnormality determination device 4 .
  • control device 6 After the processing of S1, the control device 6 starts the operation of the mechanism 2 according to the input operation mode (S2).
  • the mechanism 2 After the processing of S2, the mechanism 2 starts the operation mode indicated by the control signal CNT.
  • the abnormality determination device 4 executes abnormality determination processing for the measurement data S based on the operation mode signal MODE transmitted in the process of S1.
  • the control device 6 waits until the input operation mode ends or until it receives the abnormality notification signal ANOM (S3).
  • the control device 6 determines whether or not the abnormality notification signal ANOM has been received (S4).
  • the processing of S4 is synonymous with the control device 6 determining whether or not the operation mode of the mechanism 2 has ended.
  • the control device 6 stops the operation of the mechanism 2 by interrupting the operation mode being executed ( S5).
  • the operation mode of the mechanism 2 can be automatically interrupted.
  • FIG. 11 is a flowchart showing an example of abnormality determination processing in the abnormality determination device according to the embodiment. In the example of FIG. 11, it is assumed that the abnormality determination device 4 has received the determination model M from the model learning device 5 in advance.
  • the probability calculation unit 35 selects the determination model M based on the received operation mode signal MODE (S11).
  • the abnormality determination device 4 waits until it starts receiving the measurement data S (S12).
  • the first conversion unit 31, the spectrogram generation unit 32, the second conversion unit 33, and the feature map generation unit 34 execute feature map generation processing (S13). Details of the feature map generation process will be described later.
  • the probability calculation unit 35 calculates an abnormality probability by inputting the feature quantity map generated in the process of S13 to the judgment model M (S14).
  • the determination unit 36 outputs the determination history R to the user based on the abnormality probability calculated by the process of S14 (S15).
  • the determination unit 36 determines whether or not there is an abnormality in the measurement data S based on the abnormality probability calculated by the process of S14 (S16).
  • the determination unit 36 transmits an abnormality notification signal ANOM to the control device 6 (S17).
  • the abnormality determination device 4 determines whether or not the reception of the measurement data S has ended (S18).
  • the first conversion unit 31, the spectrogram generation unit 32, the second conversion unit 33, and the feature map generation unit 34 receive the newly received measurement data S (S13). Then, following the process of S13, the processes of S14 to S18 are executed. In this way, the processes of S13 to S18 are repeated until the reception of the measurement data S is completed.
  • FIG. 12 is a flowchart showing an example of feature map generation processing in the abnormality determination device according to the embodiment.
  • the processing of S21 to S28 in FIG. 12 corresponds to the details of the processing of S13 in FIG.
  • the first conversion unit 31 waits until the measurement data S for the period T1 (that is, data unit) is accumulated (S21).
  • the first conversion unit 31 converts the accumulated measurement data S for the period T1 into the frequency domain (S22).
  • the first conversion unit 31 determines whether or not the measurement data S for the period T2 has been converted by the process of S22 (S23).
  • the first conversion unit 31 waits until new measurement data S for the period T1 is accumulated (S21). Then, when the measurement data S for the period T1 is accumulated, the first converter 31 executes the process of S22. In this manner, the processes of S21 and S22 are repeated until the measurement data S for the period T2 are converted by the process of S22.
  • the spectrogram generator 32 When the measurement data S for the period T2 has been converted (S23; yes), the spectrogram generator 32 generates a spectrogram based on the results of the process of S22 executed multiple times over the period T2 in chronological order (S24). .
  • the second conversion unit 33 selects a frequency band from the frequency domain of the spectrogram generated in the process of S24 (S25).
  • the second transforming unit 33 transforms the time-series data (that is, data group) corresponding to the frequency band selected in the process of S25 from the spectrogram into the fluctuating frequency domain (S26).
  • the second conversion unit 33 determines whether or not all frequency bands have been selected from the spectrogram (S27).
  • the second conversion unit 33 selects an unselected frequency band from the frequency domain of the spectrogram generated in S24 (S25). Then, following the process of S25, the processes of S26 and S27 are executed. In this manner, the processes of S25 to S27 are repeated until the data groups corresponding to all frequency bands in the spectrogram are converted by the process of S26.
  • the feature map generation unit 34 performs a plurality of S26 processes on all data groups within the same spectrogram. Based on the results, A feature quantity map is generated (S28).
  • model learning processing in the model learning device according to the embodiment will be described.
  • FIG. 13 is a flowchart showing an example of model learning processing in the model learning device according to the embodiment.
  • the model learning device 5 pre-stores the teacher data set T and the pre-learning model M0.
  • the control circuit 21 selects teacher data from the teacher data set T (S31).
  • the first conversion unit 41, the spectrogram generation unit 42, the second conversion unit 43, and the teacher feature amount map generation unit 44 execute the teacher feature amount map generation process based on the teacher data selected in the process of S31 (S32 ). Details of the teacher feature quantity map generation processing will be described later.
  • the probability calculation unit 45 calculates an abnormality probability by inputting the teacher feature quantity map generated in the process of S32 to the pre-learning model M0 (S33).
  • the update unit 46 determines whether the abnormality probability calculated by the process of S33 satisfies the conditions for the label in the teacher data selected by the process of S31 (S34). For example, the updating unit 46 calculates an evaluation function based on the abnormality probability and the label. The updating unit 46 then compares the calculated evaluation function with a threshold value to determine the abnormality probability and label matching.
  • the updating unit 46 updates the pre-learning model M0 (S35).
  • the probability calculation unit 45 inputs the teacher feature map generated in the process of S32 to the pre-learning model M0 updated in the process of S35 (S33). Then, following the process of S33, the processes of S34 and S35 are executed. In this manner, the processes of S33 to S35 are repeated until it is determined that the abnormality probability satisfies the condition for the label.
  • the control circuit 21 determines that all the teacher data in the teacher data set T have been selected. (S36).
  • the control circuit 21 selects unselected teacher data from the teacher data set T (S31). Then, following the process of S31, the processes of S32 to S36 are executed. In this manner, the processes of S31 to S36 are repeated until all teaching data are selected.
  • the update unit 46 transmits the pre-learning model M0 as the determination model M to the abnormality determination device 4 (S37).
  • FIG. 14 is a flowchart showing an example of teacher feature map generation processing in the model learning device according to the embodiment.
  • the processing of S41 to S50 in FIG. 14 corresponds to the details of the processing of S32 in FIG.
  • the first conversion unit 41 selects a period T1 (ie, data unit) from the selected teacher data. (S41).
  • the first conversion unit 41 transforms the selected teacher data for the period T1 into the frequency domain (S42).
  • the first conversion unit 41 determines whether or not the teacher data for period T2 has been converted by the process of S42 (S43).
  • the first conversion unit 41 further selects unselected teacher data for period T1 (S41). Then, when the teacher data for the period T1 is selected, the first conversion unit 41 performs the process of S42 on the newly selected teacher data for the period T1. In this way, the processes of S41 and S42 are repeated until the teacher data for period T2 is converted by the process of S42.
  • the spectrogram generation unit 42 When the teacher data for period T2 has been converted (S43; yes), the spectrogram generation unit 42 generates a spectrogram based on the results of the process of S42 executed multiple times over period T2 in chronological order (S44).
  • the second conversion unit 43 selects a frequency band from the frequency domain of the spectrogram generated in the process of S44 (S45).
  • the second conversion unit 43 converts the time-series data (that is, the data group) corresponding to the frequency band selected in the process of S45 from the spectrogram into the fluctuation frequency domain (S46).
  • the second conversion unit 43 determines whether or not all frequency bands have been selected from the spectrogram (S47).
  • the second conversion unit 43 selects an unselected frequency band from the frequency domain of the spectrogram generated in S44 (S45). Then, following the process of S45, the processes of S46 and S47 are executed. In this way, the processes of S45 to S47 are repeated until the data groups corresponding to all frequency bands in the spectrogram are converted by the process of S46.
  • the teacher feature map generation unit 44 performs multiple processing of S46 on all data groups within the same spectrogram. , a teacher feature value candidate map is generated (S48).
  • the teacher feature quantity map generation unit 44 determines whether or not the teacher feature quantity candidate map has been generated over the entire period of the teacher data (S49).
  • the first conversion unit 41 converts the training data of the period not used for generating the teacher feature quantity candidate map into the period T1 minute (ie, data unit) is selected (S41). Then, following the process of S41, the processes of S42 to S49 are executed. In this way, the processing of S41 to S49 is repeated until the teacher feature quantity candidate map is generated over all the periods in the teacher data.
  • the teacher feature quantity map generation unit 44 If the teacher feature quantity candidate map is generated over all periods in the teacher data (S49; yes), the teacher feature quantity map generation unit 44 generates a teacher feature quantity map based on a plurality of teacher feature quantity candidate maps. (S50). Specifically, for example, the teacher feature quantity map generation unit 44 applies the maximum value of the amplitude values in the grids of the plurality of teacher feature quantity candidate maps to the corresponding grid of the teacher feature quantity map.
  • the first conversion unit 31, the spectrogram generation unit 32, the second conversion unit 33, and the feature amount map generation unit 34 of the abnormality determination device 4 are based on the measurement data S, and each of the frequency band and fluctuation A plurality of feature quantities associated with the set of frequency bands are calculated.
  • the probability calculation unit 35 calculates the abnormality probability of the measurement data S by inputting the plurality of calculated feature amounts to the determination model M.
  • the judgment model M a convolutional neural network is used.
  • the abnormality determination device 4 can automatically calculate the abnormality probability as an index for objective determination processing without the intervention of an inspector.
  • the first conversion unit 31 converts each of the plurality of data units in the measurement data S into the frequency domain.
  • the spectrogram generation unit 32 generates a plurality of data groups arranged in time series in corresponding frequency bands based on the result of the first conversion processing.
  • the second transforming unit 33 transforms each of the plurality of data groups into the fluctuating frequency domain.
  • the feature quantity map generator 34 maps the result of the second conversion process to the corresponding frequency band and fluctuation frequency band.
  • each of the plurality of data units is divided into frequency bands based on a scale other than the Bark scale.
  • each of the plurality of data groups is divided into varying frequency bands based on the scale other than the Bark scale.
  • a feature quantity map can be generated while omitting calculation of a psychological quantity such as loudness.
  • a psychological quantity such as loudness.
  • more rational and objective determination processing can be performed.
  • by omitting the processing requiring a large amount of calculation required for loudness calculation even when a general-purpose computer is applied to the abnormality determination device 4, it is possible to calculate a plurality of feature amounts in real time based on the measurement data S. can.
  • the determination unit 36 determines whether or not there is an abnormality in the measurement data S based on the calculated abnormality probability.
  • the abnormality determination device 4 can automatically output the determination history R and the notification of the abnormality to the user and transmit the abnormality notification signal ANOM to the control device 6 without the intervention of an inspector.
  • control device 6 suspends the operation mode of the mechanism 2 based on the abnormality notification signal ANOM. Thereby, when it is determined that the sound or vibration generated from the mechanism 2 is abnormal, the mechanism 2 can be automatically interrupted. In this way, the mechanism control system 1 can automatically perform everything from determining whether there is an abnormality to controlling the operation of the mechanism 2 .
  • the teacher feature quantity map generation unit 44 of the model learning device 5 generates a plurality of teacher feature quantity candidate maps corresponding to the plurality of feature quantity maps arranged in time series.
  • the teacher feature quantity map generation unit 44 generates a teacher feature quantity map by mapping the maximum value of a plurality of teacher feature quantity candidate maps to the corresponding grid.
  • labels can be associated one-to-one with all the data in the teacher data, the workload of labeling the data can be reduced.
  • the updating unit 46 updates the parameters of the pre-learning model M0 based on the anomaly probability calculated from the pre-learning model M0 to which the teacher feature quantity map is input, and the label. As a result, the abnormality probability calculated from the pre-learning model M0 can be brought closer to the label abnormality probability. Therefore, the model learning device 5 can provide the abnormality determination device 4 with the determination model M optimized for the abnormality determination process.
  • labels may be assigned on a one-to-one basis to portions of data in the training data for period T2.
  • the processing of S49 and S50 shown in FIG. 14 is omitted.
  • the teacher feature quantity candidate map generated in the process of S48 is used as the teacher feature quantity map.
  • control device 6 suspends the operation mode of the mechanism 2 when the abnormality notification signal ANOM is received has been described, but the present invention is not limited to this.
  • the control device 6 may perform various control operations such as shifting the mechanism 2 to the safe mode. Further, the control device 6 may notify the user of the abnormality in response to the abnormality notification signal ANOM.
  • the program for executing the abnormality determination operation is executed by the abnormality determination device 4, and the program for executing the model learning operation is executed by the model learning device 5.
  • the program for executing the abnormality determination operation and the program for executing the model learning operation may be executed by the same information processing device.
  • the program for executing the abnormality determination operation and the program for executing the model learning operation may be executed by computing resources on the cloud.
  • the present invention is not limited to the above-described embodiments, and can be variously modified in the implementation stage without departing from the gist of the present invention. Further, each embodiment may be implemented in combination as appropriate, in which case the combined effect can be obtained. Furthermore, various inventions are included in the above embodiments, and various inventions can be extracted by combinations selected from a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiments, if the problem can be solved and effects can be obtained, the configuration with the constituent elements deleted can be extracted as an invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Testing And Monitoring For Control Systems (AREA)

Abstract

Provided is a means for determining automatically and objectively presence or absence of an anomaly of a mechanism. An information processing device according to an embodiment comprises a processor configured to execute: calculation of a plurality of values each associated with a set of a frequency and a fluctuation frequency on the basis of time-series data S; and inputting of the plurality of values to a model M to calculate a probability of an anomaly being included in the time-series data.

Description

情報処理装置、制御システム、情報処理方法、及びプログラムInformation processing device, control system, information processing method, and program
　実施形態は、情報処理装置、制御システム、情報処理方法、及びプログラムに関する。 Embodiments relate to an information processing device, a control system, an information processing method, and a program.
　機構から発生する音を解析する技術が知られている。例えば、官能検査では、機構から発生する音を検査員が聞くことによって、異常の有無が判定される。 Technology for analyzing the sound generated from the mechanism is known. For example, in a sensory test, the presence or absence of an abnormality is determined by an inspector listening to the sound generated from the mechanism.
　また、ＩＳＯ（International Organization for Standard）５３２には、音の解析における指標として、ラウドネス（Loudness）が定義される。ラウドネスは、人の聴感に基づく音の大小を示す心理量である。官能検査において、ラウドネスは、検査員による判定に補助的に用いられる。 In addition, ISO (International Organization for Standard) 532 defines loudness as an index in sound analysis. Loudness is a psychological quantity indicating the loudness of sound based on human hearing. In the sensory test, the loudness is used as an aid to judgment by the inspectors.
　上述の官能検査では、異常の有無の判定は、検査員によってなされる。このため、機構から発生する音の異常の有無を自動的に判定することができない。 In the above sensory inspection, the presence or absence of abnormalities is determined by the inspector. Therefore, it is not possible to automatically determine whether or not there is an abnormality in the sound generated from the mechanism.
　また、上述の官能検査による判定基準は、検査員の主観に依存し得る。このため、検査員によって異なる判定結果が得られる可能性がある。すなわち、上述の官能検査では、機構の異常の有無を客観的に判定することができない。 In addition, the judgment criteria based on the sensory test described above may depend on the subjectivity of the inspector. Therefore, different inspectors may obtain different determination results. That is, it is not possible to objectively determine the presence or absence of an abnormality in the mechanism by the sensory inspection described above.
　本発明は、上記事情に着目してなされたもので、その目的とするところは、機構の異常の有無を自動的かつ客観的に判定する手段を提供することにある。 The present invention has been made with a focus on the above circumstances, and its purpose is to provide means for automatically and objectively determining the presence or absence of an abnormality in the mechanism.
　一態様の情報処理装置は、時系列データに基づき、各々が周波数及び変動周波数の組に対応づけられた複数の値を算出することと、上記複数の値をモデルに入力して、上記時系列データに異常が含まれる確率を算出することと、を実行するように構成されたプロセッサを備える。 An information processing apparatus according to one aspect calculates a plurality of values each associated with a set of a frequency and a fluctuation frequency based on time-series data; and calculating a probability that the data contains an anomaly.
　実施形態によれば、機構の異常の有無を自動的かつ客観的に判定する手段を提供することができる。 According to the embodiment, it is possible to provide means for automatically and objectively determining whether there is an abnormality in the mechanism.
実施形態に係る機構制御システムの構成の一例を示すブロック図。1 is a block diagram showing an example of the configuration of a mechanism control system according to an embodiment; FIG. 実施形態に係る異常判定装置のハードウェア構成の一例を示すブロック図。The block diagram which shows an example of the hardware constitutions of the abnormality determination apparatus which concerns on embodiment. 実施形態に係るモデル学習装置のハードウェア構成の一例を示すブロック図。1 is a block diagram showing an example of a hardware configuration of a model learning device according to an embodiment; FIG. 実施形態に係る異常判定装置の機能構成の一例を示すブロック図。The block diagram which shows an example of the functional structure of the abnormality determination apparatus which concerns on embodiment. 実施形態に係る異常判定装置で生成されるスペクトログラムの一例を示す図。The figure which shows an example of the spectrogram produced|generated by the abnormality determination apparatus which concerns on embodiment. 実施形態に係る異常判定装置で生成される特徴量マップの一例を示す図。The figure which shows an example of the feature-value map produced|generated by the abnormality determination apparatus which concerns on embodiment. 実施形態に係る異常判定装置で生成される判定履歴の一例を示す図。The figure which shows an example of the determination log|history produced|generated by the abnormality determination apparatus which concerns on embodiment. 実施形態に係るモデル学習装置の機能構成の一例を示すブロック図。1 is a block diagram showing an example of a functional configuration of a model learning device according to an embodiment; FIG. 実施形態に係るモデル学習装置で生成される複数の教師特徴量候補マップと教師特徴量マップとの関係の一例を示す図。FIG. 4 is a diagram showing an example of the relationship between a plurality of teacher feature quantity candidate maps and a teacher feature quantity map generated by the model learning device according to the embodiment; 実施形態に係る機構制御システムにおける機構制御処理の一例を示すフローチャート。4 is a flowchart showing an example of mechanism control processing in the mechanism control system according to the embodiment; 実施形態に係る異常判定装置における異常判定処理の一例を示すフローチャート。4 is a flowchart showing an example of abnormality determination processing in the abnormality determination device according to the embodiment; 実施形態に係る異常判定装置における特徴量マップ生成処理の一例を示すフローチャート。4 is a flowchart showing an example of feature quantity map generation processing in the abnormality determination device according to the embodiment; 実施形態に係るモデル学習装置におけるモデル学習処理の一例を示すフローチャート。4 is a flowchart showing an example of model learning processing in the model learning device according to the embodiment; 実施形態に係るモデル学習装置における教師特徴量マップ生成処理の一例を示すフローチャート。4 is a flowchart showing an example of teacher feature quantity map generation processing in the model learning device according to the embodiment.
　以下、図面を参照して実施形態について説明する。 Embodiments will be described below with reference to the drawings.
　１．　構成
　１．１　機構制御システム
　まず、実施形態に係る機構制御システムの構成について説明する。 1. Configuration 1.1 Mechanism Control System First, the configuration of the mechanism control system according to the embodiment will be described.
　図１は、実施形態に係る機構制御システムの構成の一例を示すブロック図である。図１に示すように、機構制御システム１は、機構２と、センサ３と、異常判定装置４と、モデル学習装置５と、制御装置６と、を含む。 FIG. 1 is a block diagram showing an example of the configuration of the mechanism control system according to the embodiment. As shown in FIG. 1, the mechanism control system 1 includes a mechanism 2, a sensor 3, an abnormality determination device 4, a model learning device 5, and a control device 6.
　機構２は、異常判定対象の音源である。機構２は、例えば、モータ等が組み付けられた自動車の電動シートや風力発電設備である。機構２には、各々が同等の仕様を有する複数のモータが組み付けられていてもよい。機構２は、所定の動作モードで動作する際に音及び振動を発生させる。機構２が電動シートの場合、動作モードは、例えば、リクライニング動作、前後移動動作、左右移動動作等である。機構２からの音及び振動は、機構の動作モード、及び機構内の異常の有無によって変化し得る。機構２の動作モードは、制御装置６からの制御信号ＣＮＴによって制御される。制御信号ＣＮＴは、例えば、動作モードの開始、中断、及び終了を機構２に指示する情報を含む。 Mechanism 2 is the sound source for abnormality determination. The mechanism 2 is, for example, an electric seat of an automobile or a wind power generation facility in which a motor or the like is assembled. Mechanism 2 may be assembled with a plurality of motors each having equivalent specifications. Mechanism 2 generates sound and vibration when operating in a predetermined mode of operation. When the mechanism 2 is an electric seat, the operation mode is, for example, a reclining operation, a forward/backward moving operation, a left/right moving operation, and the like. Sounds and vibrations from the mechanism 2 may vary depending on the operating mode of the mechanism and the presence or absence of anomalies within the mechanism. The operating mode of mechanism 2 is controlled by a control signal CNT from controller 6 . The control signal CNT contains, for example, information instructing the mechanism 2 to start, suspend and end the operating mode.
　センサ３は、例えば、マイク又は振動計である。センサ３は、機構２の近傍に、又は機構２に接触するように配置される。センサ３は、機構２が動作することによって発生する音又は振動を計測する。センサ３によって計測された音又は振動は、時系列の計測データＳとして異常判定装置４に送信される。計測データＳは、アナログ信号及びデジタル信号のいずれであってもよい。 The sensor 3 is, for example, a microphone or a vibrometer. A sensor 3 is arranged in the vicinity of or in contact with the mechanism 2 . Sensor 3 measures sound or vibration generated by operation of mechanism 2 . The sound or vibration measured by the sensor 3 is transmitted to the abnormality determination device 4 as time-series measurement data S. The measurement data S may be either an analog signal or a digital signal.
　異常判定装置４は、パーソナルコンピュータ等の情報処理装置である。異常判定装置４は、制御装置６からの動作モード信号ＭＯＤＥに基づき、計測データＳに関する異常判定処理を実行する。動作モード信号ＭＯＤＥは、機構２の動作モードを識別するための信号である。異常判定装置４は、異常判定処理の結果として、判定履歴Ｒ及び異常通知信号ＡＮＯＭを生成する。異常通知信号ＡＮＯＭは、計測データＳに異常があることを制御装置６に通知するための信号である。判定履歴Ｒは、計測データＳに異常があるか否かを示す情報である。異常判定装置４は、異常通知信号ＡＮＯＭを制御装置６に送信し、判定履歴Ｒをユーザに出力する。異常判定処理の詳細については、後述する。 The abnormality determination device 4 is an information processing device such as a personal computer. The abnormality determination device 4 executes abnormality determination processing regarding the measurement data S based on the operation mode signal MODE from the control device 6 . The operation mode signal MODE is a signal for identifying the operation mode of mechanism 2 . The abnormality determination device 4 generates a determination history R and an abnormality notification signal ANOM as a result of the abnormality determination process. The abnormality notification signal ANOM is a signal for notifying the control device 6 that the measurement data S is abnormal. The determination history R is information indicating whether or not the measurement data S is abnormal. The abnormality determination device 4 transmits an abnormality notification signal ANOM to the control device 6 and outputs the determination history R to the user. Details of the abnormality determination process will be described later.
　モデル学習装置５は、パーソナルコンピュータ等の情報処理装置である。モデル学習装置５は、モデル学習処理を実行する。モデル学習装置５は、モデル学習処理の結果として、判定モデルＭを生成する。モデル学習装置５は、生成された判定モデルＭを異常判定装置４に送信する。モデル学習処理の詳細については、後述する。 The model learning device 5 is an information processing device such as a personal computer. The model learning device 5 executes model learning processing. The model learning device 5 generates a judgment model M as a result of model learning processing. The model learning device 5 transmits the generated determination model M to the abnormality determination device 4 . Details of the model learning process will be described later.
　判定モデルＭは、例えば、ニューラルネットワークを含む数学モデルである。判定モデルＭは、異常判定装置４における異常判定処理に用いられる。判定モデルＭ内のニューラルネットワークは、複数のパラメタを含む。判定モデルＭ内の複数のパラメタは、モデル学習処理によって異常判定処理に対して最適化される。判定モデルＭは、例えば、機構２の動作モード毎に異なるモデルを有する。 The judgment model M is, for example, a mathematical model including a neural network. The determination model M is used for abnormality determination processing in the abnormality determination device 4 . A neural network in the decision model M includes multiple parameters. A plurality of parameters in the judgment model M are optimized for abnormality judgment processing by model learning processing. The judgment model M has, for example, different models for each operation mode of the mechanism 2 .
　制御装置６は、主に機構２の動作を制御する制御端末である。機構２が電動シートの場合、制御装置６は、出荷前検査において電動シートの動作を確認するための試験装置である。機構２が風力発電設備である場合、制御装置６は、設置された設備に対する動作制御及び不具合モニタをリアルタイムで行うための管制装置である。制御装置６は、異常判定装置４に動作モード信号ＭＯＤＥを送信する。制御装置６は、異常判定装置４から異常通知信号ＡＮＯＭを受信する。制御装置６は、機構２に制御信号ＣＮＴを送信する。 The control device 6 is a control terminal that mainly controls the operation of the mechanism 2. When the mechanism 2 is an electric seat, the control device 6 is a test device for checking the operation of the electric seat in the pre-shipment inspection. When the mechanism 2 is a wind power generation facility, the control device 6 is a control device for real-time operation control and failure monitoring of the installed facility. Control device 6 transmits an operation mode signal MODE to abnormality determination device 4 . Control device 6 receives an abnormality notification signal ANOM from abnormality determination device 4 . Control device 6 sends control signal CNT to mechanism 2 .
　１．２　ハードウェア構成
　次に、実施形態に係る機構制御システムのハードウェア構成について説明する。 1.2 Hardware Configuration Next, the hardware configuration of the mechanism control system according to the embodiment will be described.
　１．２．１　異常判定装置
　図２は、実施形態に係る異常判定装置のハードウェア構成の一例を示すブロック図である。図２に示すように、異常判定装置４は、制御回路１１、ストレージ１２、通信モジュール１３、ユーザインタフェース１４、ドライブ１５、及び記憶媒体１６を含む。 1.2.1 Abnormality Determination Device FIG. 2 is a block diagram showing an example of the hardware configuration of the abnormality determination device according to the embodiment. As shown in FIG. 2, the abnormality determination device 4 includes a control circuit 11, a storage 12, a communication module 13, a user interface 14, a drive 15, and a storage medium 16.
　制御回路１１は、異常判定装置４の各構成要素を全体的に制御する回路である。制御回路１１は、ＣＰＵ（Central Processing Unit）、ＲＡＭ（Random Access Memory）、及びＲＯＭ（Read Only Memory）等を含む。 The control circuit 11 is a circuit that controls each component of the abnormality determination device 4 as a whole. The control circuit 11 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), and the like.
　ストレージ１２は、異常判定装置４の補助記憶装置である。ストレージ１２は、例えば、ＨＤＤ（Hard Disk Drive）、ＳＳＤ（Solid State Drive）、又はメモリカード等である。ストレージ１２は、異常判定処理に用いられる計測データＳ及び判定モデルＭ等を記憶する。また、ストレージ１２は、異常判定処理を実行するためのプログラムを記憶してもよい。 The storage 12 is an auxiliary storage device for the abnormality determination device 4. The storage 12 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a memory card. The storage 12 stores the measurement data S, the determination model M, and the like used for the abnormality determination process. Moreover, the storage 12 may store a program for executing the abnormality determination process.
　通信モジュール１３は、センサ３との間、及びモデル学習装置５との間のデータの送受信に用いられる回路である。 The communication module 13 is a circuit used for transmitting and receiving data with the sensor 3 and with the model learning device 5.
　ユーザインタフェース１４は、ユーザと制御回路１１との間で情報を通信するための回路である。ユーザインタフェース１４は、入力機器及び出力機器を含む。入力機器は、例えば、タッチパネル及び操作ボタン等を含む。出力機器は、例えば、ディスプレイ及びプリンタ等を含む。また、出力機器は、ランプやブザー等を含んでいてもよい。 The user interface 14 is a circuit for communicating information between the user and the control circuit 11 . User interface 14 includes input and output devices. The input device includes, for example, a touch panel and operation buttons. Output devices include, for example, displays and printers. Also, the output device may include a lamp, a buzzer, or the like.
　ドライブ１５は、記憶媒体１６に記憶されたソフトウェアを読み込むための機器である。ドライブ１５は、例えば、ＣＤ（Compact Disk）ドライブ、及びＤＶＤ（Digital Versatile Disk）ドライブ等を含む。 The drive 15 is a device for reading software stored in the storage medium 16. The drive 15 includes, for example, a CD (Compact Disk) drive, a DVD (Digital Versatile Disk) drive, and the like.
　記憶媒体１６は、ソフトウェアを、電気的、磁気的、光学的、機械的又は化学的作用によって記憶する媒体である。記憶媒体１６は、異常判定処理を実行するためのプログラムを記憶してもよい。 The storage medium 16 is a medium that stores software by electrical, magnetic, optical, mechanical or chemical action. The storage medium 16 may store a program for executing the abnormality determination process.
　１．２．２　モデル学習装置
　図３は、実施形態に係るモデル学習装置のハードウェア構成の一例を示すブロック図である。図３に示すように、モデル学習装置５は、制御回路２１、ストレージ２２、通信モジュール２３、ユーザインタフェース２４、ドライブ２５、及び記憶媒体２６を含む。 1.2.2 Model Learning Apparatus FIG. 3 is a block diagram showing an example of the hardware configuration of the model learning apparatus according to the embodiment. As shown in FIG. 3 , model learning device 5 includes control circuit 21 , storage 22 , communication module 23 , user interface 24 , drive 25 and storage medium 26 .
　制御回路２１は、モデル学習装置５の各構成要素を全体的に制御する回路である。制御回路２１は、ＣＰＵ、ＲＡＭ、及びＲＯＭ等を含む。 The control circuit 21 is a circuit that controls each component of the model learning device 5 as a whole. The control circuit 21 includes a CPU, RAM, ROM, and the like.
　ストレージ２２は、モデル学習装置５の補助記憶装置である。ストレージ２２は、例えば、ＨＤＤ、ＳＳＤ、又はメモリカード等である。ストレージ２２は、モデル学習処理に使用されるデータを記憶する。また、ストレージ２２は、モデル学習処理を実行するためのプログラムを記憶してもよい。 The storage 22 is an auxiliary storage device for the model learning device 5. The storage 22 is, for example, an HDD, SSD, memory card, or the like. The storage 22 stores data used for model learning processing. The storage 22 may also store a program for executing model learning processing.
　通信モジュール２３は、異常判定装置４との間のデータの送受信に使用される回路である。 The communication module 23 is a circuit used for transmitting and receiving data to and from the abnormality determination device 4.
　ユーザインタフェース２４は、ユーザと制御回路２１との間で情報を通信するための回路である。ユーザインタフェース２４は、入力機器及び出力機器を含む。入力機器は、例えば、タッチパネル及び操作ボタン等を含む。出力機器は、例えば、ディスプレイ及びプリンタ等を含む。 The user interface 24 is a circuit for communicating information between the user and the control circuit 21 . User interface 24 includes input and output devices. The input device includes, for example, a touch panel and operation buttons. Output devices include, for example, displays and printers.
　ドライブ２５は、記憶媒体２６に記憶されたソフトウェアを読み込むための機器である。ドライブ２５は、例えば、ＣＤドライブ、及びＤＶＤドライブ等を含む。 The drive 25 is a device for reading software stored in the storage medium 26. The drive 25 includes, for example, a CD drive, a DVD drive, and the like.
　記憶媒体２６は、ソフトウェアを、電気的、磁気的、光学的、機械的又は化学的作用によって記憶する媒体である。記憶媒体２６は、モデル学習処理を実行するためのプログラムを記憶してもよい。 The storage medium 26 is a medium that stores software by electrical, magnetic, optical, mechanical or chemical action. The storage medium 26 may store a program for executing model learning processing.
　１．３　機能構成
　次に、実施形態に係る機構制御システムの機能構成について説明する。 1.3 Functional Configuration Next, the functional configuration of the mechanism control system according to the embodiment will be described.
　１．３．１　異常判定装置
　図４は、実施形態に係る異常判定装置の機能構成の一例を示すブロック図である。制御回路１１のＣＰＵは、ストレージ１２又は記憶媒体１６に記憶された異常判定処理に関するプログラムをＲＡＭに展開する。そして、制御回路１１のＣＰＵは、ＲＡＭに展開されたプログラムを解釈及び実行する。これにより、異常判定装置４は、第１変換部３１、スペクトログラム生成部３２、第２変換部３３、特徴量マップ生成部３４、確率算出部３５、及び判定部３６を備えるコンピュータとして機能する。 1.3.1 Abnormality Determination Device FIG. 4 is a block diagram showing an example of the functional configuration of the abnormality determination device according to the embodiment. The CPU of the control circuit 11 expands a program related to abnormality determination processing stored in the storage 12 or the storage medium 16 to the RAM. Then, the CPU of the control circuit 11 interprets and executes the program developed in the RAM. Thereby, the abnormality determination device 4 functions as a computer including a first conversion unit 31 , a spectrogram generation unit 32 , a second conversion unit 33 , a feature map generation unit 34 , a probability calculation unit 35 and a determination unit 36 .
　第１変換部３１は、時系列の計測データＳを周波数領域に変換する機能ブロックである。計測データＳがアナログ信号の場合、第１変換部３１は、計測データＳをアナログ信号からデジタル信号に変換する。第１変換部３１は、デジタル信号に変換後の計測データＳを期間Ｔ１の長さのデータ単位に分割する。期間Ｔ１は、例えば、数ミリ秒～数十ミリ秒である。時系列に隣り合う２個のデータ単位は、互いに重複する期間を有していてもよい。時系列に隣り合う２個のデータ単位は、互いに重複する期間を有していなくもよい。第１変換部３１は、例えば、計測データＳに対して、データ単位毎に高速フーリエ変換（ＦＦＴ：Fast Fourier Transformation）処理を実行し得る。また、第１変換部３１は、例えば、オクターブバンド分析のように、互いに異なる周波数帯を透過させる複数のバンドパスフィルタを用いることによって、計測データＳを周波数領域に変換してもよい。第１変換部３１は、データ単位毎の変換処理の結果をスペクトログラム生成部３２に送信する。以下の説明では、周波数領域への変換処理は、“第１変換処理”とも呼ぶ。 The first conversion unit 31 is a functional block that converts the time-series measurement data S into the frequency domain. When the measurement data S are analog signals, the first converter 31 converts the measurement data S from analog signals to digital signals. The first converter 31 divides the measurement data S converted into digital signals into data units each having a length of period T1. The period T1 is, for example, several milliseconds to several tens of milliseconds. Two data units adjacent in time series may have overlapping periods. Two data units adjacent in time series may not have periods that overlap each other. The first transformation unit 31 can perform, for example, fast Fourier transformation (FFT) processing on the measurement data S for each data unit. Further, the first conversion unit 31 may convert the measurement data S into the frequency domain by using a plurality of bandpass filters that transmit different frequency bands, such as octave band analysis. The first conversion unit 31 transmits the result of conversion processing for each data unit to the spectrogram generation unit 32 . In the following description, the conversion process to the frequency domain is also called "first conversion process".
　なお、第１変換部３１の第１変換処理における周波数領域のサンプリング幅（周波数帯域）は、ＦＦＴ処理のように定幅に分割されなくてもよい。例えば、第１変換部３１の第１変換処理における周波数帯域は、１／３オクターブバンドやメル尺度に基づく帯域で分割されてもよい。ただし、周波数帯域へのバーク尺度の適用は、計算機負荷の観点から好ましくない。 Note that the sampling width (frequency band) of the frequency domain in the first conversion processing of the first conversion unit 31 does not have to be divided into constant widths like the FFT processing. For example, the frequency band in the first conversion process of the first conversion unit 31 may be divided into 1/3 octave bands or bands based on the Mel scale. However, application of the Bark scale to frequency bands is not preferable from the viewpoint of computer load.
　スペクトログラム生成部３２は、第１変換部３１による第１変換処理の結果を用いて、スペクトログラムを生成する機能ブロックである。スペクトログラム生成部３２は、第１変換部３１によるデータ単位毎の第１変換処理の結果を、時系列に期間Ｔ２にわたって蓄積する。期間Ｔ２は、期間Ｔ１より長い。期間Ｔ２は、例えば、数百ミリ秒である。スペクトログラム生成部３２は、期間Ｔ２にわたって蓄積された複数の第１変換処理の結果に基づき、スペクトログラムを生成する。スペクトログラム生成部３２は、生成されたスペクトログラムを第２変換部３３に送信する。 The spectrogram generation unit 32 is a functional block that generates a spectrogram using the result of the first conversion processing by the first conversion unit 31. The spectrogram generation unit 32 accumulates the results of the first conversion processing for each data unit by the first conversion unit 31 in time series over the period T2. The period T2 is longer than the period T1. The period T2 is, for example, several hundred milliseconds. The spectrogram generator 32 generates a spectrogram based on the results of the multiple first conversion processes accumulated over the period T2. The spectrogram generator 32 transmits the generated spectrogram to the second converter 33 .
　図５は、実施形態に係る異常判定装置で生成されるスペクトログラムの一例を示す図である。スペクトログラムは、３次元データの集合である。スペクトログラムにおける各３次元データの３成分はそれぞれ、時間、周波数、及び信号強度（振幅）に対応する。図５の例では、Ｘ軸及びＹ軸がそれぞれ、時間及び周波数に対応する。そして、或る時間幅及び周波数帯域に対応する振幅の大小が、グリッド毎の色の濃淡で示される。スペクトログラムにおけるグリッドとは、１個の時間幅と１個の周波数帯域とで区切られるスペクトログラム内の矩形領域である。 FIG. 5 is a diagram showing an example of a spectrogram generated by the abnormality determination device according to the embodiment. A spectrogram is a collection of three-dimensional data. The three components of each three-dimensional data in the spectrogram correspond to time, frequency, and signal strength (amplitude), respectively. In the example of FIG. 5, the X and Y axes correspond to time and frequency, respectively. The magnitude of amplitude corresponding to a certain time width and frequency band is indicated by the shade of color for each grid. A grid in a spectrogram is a rectangular area in the spectrogram that is delimited by one time span and one frequency band.
　このスペクトログラムにおいて、或る時間幅に対応する（Ｙ方向に並ぶ）複数のグリッドが、１個のデータ単位についての第１変換処理の結果に対応する。このようなデータ単位についての第１変換処理の結果が時系列に（Ｘ方向に）期間Ｔ２に対応する個数並ぶことにより、スペクトログラムが形成される。言い換えると、スペクトログラムは、各々が対応する周波数帯域で時系列に並ぶ複数のデータ群の集合である。 In this spectrogram, a plurality of grids (aligned in the Y direction) corresponding to a certain time width correspond to the results of the first conversion processing for one data unit. A spectrogram is formed by arranging the results of the first conversion processing for such data units in time series (in the X direction) in a number corresponding to the period T2. In other words, a spectrogram is a set of multiple data groups arranged in time series in corresponding frequency bands.
　再び図４を参照して、異常判定装置４の機能構成について説明する。 The functional configuration of the abnormality determination device 4 will be described with reference to FIG. 4 again.
　第２変換部３３は、スペクトログラム生成部３２によって生成されたスペクトログラムを変動周波数領域に変換する機能ブロックである。変動周波数とは、或る周波数で振動する信号における、ピーク値の時間変化の周波数表現である。第２変換部３３は、スペクトログラムを、複数のデータ群に分割する。複数のデータ群は、互いに異なる周波数帯域に対応する。すなわち、複数のデータ群の各々は、スペクトログラムのうち、或る周波数帯域に対応する（Ｘ方向に並ぶ）複数のグリッドを含む。第２変換部３３は、例えば、データ群毎にＦＦＴ処理を実行し得る。また、第２変換部３３は、例えば、オクターブバンド分析のように、互いに異なる周波数帯を透過させる複数のバンドパスフィルタを用いることによって、スペクトログラムを変動周波数領域に変換してもよい。第２変換部３３は、データ群毎の変換処理の結果を、特徴量マップ生成部３４に送信する。以下の説明では、変動周波数領域への変換処理は、“第２変換処理”とも呼ぶ。 The second conversion unit 33 is a functional block that converts the spectrogram generated by the spectrogram generation unit 32 into the fluctuating frequency domain. Fluctuation frequency is a frequency representation of the time variation of the peak value in a signal that oscillates at a certain frequency. The second conversion unit 33 divides the spectrogram into multiple data groups. A plurality of data groups correspond to frequency bands different from each other. That is, each of the multiple data groups includes multiple grids (aligned in the X direction) corresponding to a certain frequency band in the spectrogram. The second conversion unit 33 can perform FFT processing for each data group, for example. Further, the second transforming unit 33 may transform the spectrogram into the variable frequency domain by using a plurality of bandpass filters that transmit different frequency bands, such as octave band analysis. The second conversion unit 33 transmits the result of conversion processing for each data group to the feature map generation unit 34 . In the following description, the conversion processing to the variable frequency domain is also called "second conversion processing".
　なお、第２変換部３３の第２変換処理における変動周波数帯域のサンプリング幅（変動周波数帯域）は、ＦＦＴ処理のように定幅に分割されなくてもよい。例えば、第２変換部３３の第２変換処理における変動周波数帯域は、１／３オクターブバンドやメル尺度に基づく帯域で分割されてもよい。ただし、変動周波数帯域へのバーク尺度の適用は、計算機負荷の観点から好ましくない。 Note that the sampling width (fluctuation frequency band) of the fluctuation frequency band in the second conversion processing of the second conversion unit 33 does not have to be divided into constant widths unlike the FFT processing. For example, the variable frequency band in the second conversion process of the second conversion unit 33 may be divided into 1/3 octave bands or bands based on the Mel scale. However, application of the Bark scale to fluctuating frequency bands is not preferable from the viewpoint of computer load.
　また、第２変換処理に先立ち、第２変換部３３は、ラウドネス算出時に実行されるようなレベル補正処理、時間マスキング処理、周波数マスキング処理、及び時間重み付け処理の各々を実行してもよい。ただし、計測データＳの異常の有無を人の聴感によらずに判定する観点から、レベル補正処理、時間マスキング処理、周波数マスキング処理、及び時間重み付け処理は、不要である。このため、第２変換部３３は、レベル補正処理、時間マスキング処理、周波数マスキング処理、及び時間重み付け処理を実行しなくてもよい。 Also, prior to the second conversion process, the second conversion unit 33 may perform each of the level correction process, the time masking process, the frequency masking process, and the time weighting process that are performed when calculating the loudness. However, from the viewpoint of determining the presence or absence of abnormality in the measurement data S without relying on human hearing, the level correction process, time masking process, frequency masking process, and time weighting process are unnecessary. Therefore, the second conversion unit 33 does not need to perform level correction processing, time masking processing, frequency masking processing, and time weighting processing.
　特徴量マップ生成部３４は、第２変換部３３による第２変換処理の結果を用いて、特徴量マップを生成する機能ブロックである。特徴量マップ生成部３４は、１個のスペクトログラムから生成される全てのデータ群に対する第２変換処理の結果を、特徴量として蓄積する。特徴量マップ生成部３４は、蓄積された特徴量をマッピングすることにより、特徴量マップを生成する。特徴量マップ生成部３４は、生成された特徴量マップを確率算出部３５に送信する。 The feature map generation unit 34 is a functional block that uses the result of the second conversion processing by the second conversion unit 33 to generate a feature map. The feature quantity map generation unit 34 accumulates the results of the second conversion processing for all data groups generated from one spectrogram as feature quantities. The feature quantity map generation unit 34 generates a feature quantity map by mapping the accumulated feature quantity. The feature map generation unit 34 transmits the generated feature map to the probability calculation unit 35 .
　図６は、実施形態に係る異常判定装置で生成される特徴量マップの一例を示す図である。特徴量マップは、３次元データの集合である。特徴量マップにおける各３次元データの３成分はそれぞれ、周波数、変動周波数、及び信号強度（振幅）に対応する。図６の例では、Ｘ軸及びＹ軸がそれぞれ、周波数及び変動周波数に対応する。そして、或る周波数帯域及び変動周波数帯域に対応する振幅の大小が、グリッド毎の色の濃淡で示される。特徴量マップにおけるグリッドとは、１個の周波数帯域と１個の変動周波数帯域とで区切られる特徴量マップ内の矩形領域である。 FIG. 6 is a diagram showing an example of a feature quantity map generated by the abnormality determination device according to the embodiment. A feature map is a set of three-dimensional data. Three components of each three-dimensional data in the feature quantity map respectively correspond to frequency, fluctuation frequency, and signal strength (amplitude). In the example of FIG. 6, the X and Y axes correspond to frequency and variation frequency, respectively. The magnitude of the amplitude corresponding to a certain frequency band and the variable frequency band is indicated by the shade of color for each grid. A grid in the feature map is a rectangular area in the feature map that is partitioned by one frequency band and one variable frequency band.
　この特徴量マップにおいて、或る周波数帯域に対応する（Ｙ方向に並ぶ）複数のグリッドが、１個のデータ群についての第２変換処理の結果に対応する。このようなデータ群の第２変換処理の結果が周波数方向に（Ｘ方向に）複数個並ぶことにより、特徴量マップが形成される。 In this feature quantity map, a plurality of grids (aligned in the Y direction) corresponding to a certain frequency band correspond to the result of the second transformation processing for one data group. A feature quantity map is formed by arranging a plurality of results of the second conversion processing of such a data group in the frequency direction (in the X direction).
　再び図４を参照して、異常判定装置４の機能構成について説明する。 The functional configuration of the abnormality determination device 4 will be described with reference to FIG. 4 again.
　確率算出部３５は、計測データＳに異常がある確率を算出する機能ブロックである。確率算出部３５は、動作モード信号ＭＯＤＥに基づき、機構２の動作モードに応じた判定モデルＭを選択する。確率算出部３５は、選択された判定モデルＭに、特徴量マップを入力する。確率算出部３５は、特徴量マップが入力された判定モデルＭからの出力結果に基づき、計測データＳの異常確率を算出する。異常確率は、例えば、０以上１以下の実数である。例えば、異常確率が大きいほど、計測データＳが異常である確率が高いことを示す。確率算出部３５は、算出された異常確率を判定部３６に送信する。 The probability calculation unit 35 is a functional block that calculates the probability that the measurement data S is abnormal. The probability calculator 35 selects the determination model M corresponding to the operation mode of the mechanism 2 based on the operation mode signal MODE. The probability calculation unit 35 inputs the feature quantity map to the selected determination model M. FIG. The probability calculation unit 35 calculates the abnormality probability of the measurement data S based on the output result from the determination model M to which the feature quantity map is input. The abnormality probability is, for example, a real number between 0 and 1 inclusive. For example, the higher the abnormality probability, the higher the probability that the measurement data S is abnormal. The probability calculation unit 35 transmits the calculated abnormality probability to the determination unit 36 .
　上述の通り、判定モデルＭは、ニューラルネットワークを含む。より具体的には、判定モデルＭに含まれるニューラルネットワークは、例えば、畳み込みニューラルネットワーク（ＣＮＮ：Convolutional Neural Network）である。この場合、特徴量マップは、あたかも画像情報として、判定モデルＭに入力される。これにより、特徴量マップの各グリッドにおける振幅値は、判定モデルＭによって、画像におけるピクセル値と同等に取り扱われる。 As described above, the judgment model M includes a neural network. More specifically, the neural network included in the judgment model M is, for example, a convolutional neural network (CNN). In this case, the feature quantity map is input to the determination model M as if it were image information. As a result, the amplitude values in each grid of the feature map are handled by the determination model M in the same way as the pixel values in the image.
　判定部３６は、異常確率に基づき、計測データＳの異常の有無を判定する機能ブロックである。判定部３６は、確率算出部３５から異常確率を受信するたびに、判定処理を実行する。異常確率が条件を満たす場合、判定部３６は、計測データＳに異常があると判定する。条件は、例えば、異常確率が第１閾値（例えば０．５）以上となることであってもよい。また、条件は、例えば、異常確率が第１閾値以上となった回数が第２閾値以上となることであってもよい。このように、条件は、機構２の動作モード及び機構２の特性に応じて、任意の条件が適用される。判定部３６は、判定処理の結果を時系列に並べた判定履歴Ｒを生成し、ユーザに出力する。また、計測データＳに異常があると判定された場合、判定部３６は、異常通知信号ＡＮＯＭを生成し、制御装置６に送信する。なお、計測データＳに異常があると判定された場合、判定部３６は、ブザーを鳴らしたり、ランプを点灯させたりする等の手段で、異常をユーザに通知してもよい。 The determination unit 36 is a functional block that determines the presence or absence of an abnormality in the measurement data S based on the abnormality probability. The determination unit 36 executes determination processing each time it receives an abnormality probability from the probability calculation unit 35 . When the abnormality probability satisfies the condition, the determination unit 36 determines that the measurement data S is abnormal. The condition may be, for example, that the abnormality probability is greater than or equal to a first threshold (eg, 0.5). Further, the condition may be, for example, that the number of times the abnormality probability becomes equal to or greater than the first threshold is equal to or greater than the second threshold. In this way, arbitrary conditions are applied according to the operation mode of the mechanism 2 and the characteristics of the mechanism 2 . The determination unit 36 generates a determination history R in which the results of determination processing are arranged in chronological order, and outputs it to the user. Further, when it is determined that the measurement data S has an abnormality, the determination unit 36 generates an abnormality notification signal ANOM and transmits it to the control device 6 . Note that, when it is determined that the measurement data S has an abnormality, the determination unit 36 may notify the user of the abnormality by sounding a buzzer, lighting a lamp, or the like.
　図７は、実施形態に係る異常判定装置で生成される判定履歴の一例を示す図である。図７の例では、判定履歴Ｒが、異常確率を時系列に並べたグラフとしてユーザに出力される場合が示される。判定履歴Ｒは、新たな異常確率が算出されるたびに、時間方向に更新されていく。 FIG. 7 is a diagram showing an example of determination history generated by the abnormality determination device according to the embodiment. The example of FIG. 7 shows a case where the determination history R is output to the user as a graph in which the abnormality probabilities are arranged in chronological order. The determination history R is updated in the time direction each time a new abnormality probability is calculated.
　図７の例において、異常確率が０．５以上の際に計測データＳに異常があると判定される場合、判定部３６は、時刻ｔ１～ｔ８の各々のタイミングで、異常通知信号ＡＮＯＭを制御装置６に送信する。 In the example of FIG. 7, when it is determined that there is an abnormality in the measurement data S when the abnormality probability is 0.5 or more, the determination unit 36 controls the abnormality notification signal ANOM at each timing from time t1 to t8. Send to device 6 .
　以上のように構成されることにより、異常判定装置４は、計測データＳに基づき、判定モデルＭを用いて、機構２の異常の有無を判定し、その結果をユーザ及び制御装置６に通知することができる。そして、制御装置６は、異常判定装置４からの異常通知信号ＡＮＯＭを受信することにより、機構２の動作モードを中断させることができる。 With the configuration as described above, the abnormality determination device 4 determines whether or not there is an abnormality in the mechanism 2 based on the measurement data S using the determination model M, and notifies the user and the control device 6 of the result. be able to. Then, the control device 6 can interrupt the operation mode of the mechanism 2 by receiving the abnormality notification signal ANOM from the abnormality determination device 4 .
　１．３．２　モデル学習装置
　実施形態に係るモデル学習装置の機能構成について説明する。 1.3.2 Model Learning Device A functional configuration of the model learning device according to the embodiment will be described.
　図８は、実施形態に係るモデル学習装置の機能構成の一例を示すブロック図である。制御回路２１のＣＰＵは、ストレージ２２又は記憶媒体２６に記憶されたモデル学習処理に関するプログラムをＲＡＭに展開する。そして、制御回路２１のＣＰＵは、ＲＡＭに展開されたプログラムを解釈及び実行する。これにより、モデル学習装置５は、第１変換部４１、スペクトログラム生成部４２、第２変換部４３、教師特徴量マップ生成部４４、確率算出部４５、及び更新部４６を備えるコンピュータとして機能する。また、モデル学習装置５は、教師データセットＴ及び学習前モデルＭ０を記憶する。 FIG. 8 is a block diagram showing an example of the functional configuration of the model learning device according to the embodiment. The CPU of the control circuit 21 loads a program related to model learning processing stored in the storage 22 or the storage medium 26 into the RAM. Then, the CPU of the control circuit 21 interprets and executes the program developed in the RAM. Thereby, the model learning device 5 functions as a computer including a first conversion unit 41 , a spectrogram generation unit 42 , a second conversion unit 43 , a teacher feature map generation unit 44 , a probability calculation unit 45 and an update unit 46 . The model learning device 5 also stores the teacher data set T and the pre-learning model M0.
　学習前モデルＭ０は、ニューラルネットワークを含む数学モデルである。学習前モデルＭ０の構成は、パラメタの値が異なる点を除いて、判定モデルＭと同等である。学習前モデルＭ０は、初期状態、すなわちパラメタが異常判定処理に対して最適化される前の状態の判定モデルである。 The pre-learning model M0 is a mathematical model including a neural network. The configuration of the pre-learning model M0 is the same as that of the judgment model M, except that the parameter values are different. The pre-learning model M0 is a determination model in an initial state, that is, before the parameters are optimized for the abnormality determination process.
　教師データセットＴは、複数の教師データを含む。複数の教師データの各々は、互いに対応づけられたデータ及びラベルを含む。 A teacher data set T includes multiple teacher data. Each of the plurality of teacher data includes data and labels associated with each other.
　各教師データ内のデータは、例えば、機構２からの音又は振動を予め測定したデータである。各教師データ内のデータの長さは、例えば、期間Ｔ３以上である。期間Ｔ３は、期間Ｔ２より長い。このため、各教師データ内のデータからは、複数の特徴量マップを生成することができる。 The data in each teacher data is, for example, data obtained by measuring the sound or vibration from the mechanism 2 in advance. The length of data in each teacher data is, for example, period T3 or longer. Period T3 is longer than period T2. Therefore, a plurality of feature quantity maps can be generated from the data in each teacher data.
　各教師データ内のラベルは、対応するデータに関する真の判定結果を示す情報である。具体的には、各教師データ内のラベルは、対応するデータの異常確率を含む。すなわち、異常な音又は振動を含むデータに対応するラベルは、異常確率“１”を示す。一方、異常な音又は振動を含まないデータに対応するラベルは、異常確率“０”を示す。 The label in each training data is information indicating the true judgment result for the corresponding data. Specifically, the labels within each training data include the anomaly probability of the corresponding data. That is, a label corresponding to data including abnormal sound or vibration indicates an abnormality probability of "1". On the other hand, a label corresponding to data that does not contain abnormal sound or vibration indicates an abnormality probability of "0".
　教師データセットＴは、教師データを処理単位として、モデル学習装置５によって処理される。以下では、各教師データに対するモデル学習装置５の機能構成について、説明する。 The teacher data set T is processed by the model learning device 5 using teacher data as a processing unit. The functional configuration of the model learning device 5 for each teacher data will be described below.
　第１変換部４１、スペクトログラム生成部４２、及び第２変換部４３の機能構成は、計測データＳに代えて教師データが入力である点を除き、第１変換部３１、スペクトログラム生成部３２、及び第２変換部３３と同等である。 The functional configurations of the first conversion unit 41, the spectrogram generation unit 42, and the second conversion unit 43 are the first conversion unit 31, the spectrogram generation unit 32, and the It is equivalent to the second converter 33 .
　第１変換部４１は、時系列の教師データを周波数領域に変換する機能ブロックである。第１変換部４１は、教師データを期間Ｔ１の長さのデータ単位に分割する。第１変換部４１は、例えば、教師データに対して、データ単位毎に第１変換処理（例えば、ＦＦＴ処理）を実行する。第１変換部４１は、データ単位毎の第１変換処理の結果をスペクトログラム生成部４２に送信する。なお、第１変換部４１の第１変換処理における周波数帯域は、判定モデルＭと学習前モデルＭ０との整合性の観点から、第１変換部３１と同じ周波数帯域が適用されることが望ましい。 The first conversion unit 41 is a functional block that converts time-series teacher data into the frequency domain. The first conversion unit 41 divides the teacher data into data units having a length of period T1. The first conversion unit 41 performs, for example, a first conversion process (for example, an FFT process) for each data unit on teacher data. The first conversion unit 41 transmits the result of the first conversion process for each data unit to the spectrogram generation unit 42 . From the viewpoint of consistency between the judgment model M and the pre-learning model M0, it is desirable that the same frequency band as that of the first conversion unit 31 is applied as the frequency band in the first conversion processing of the first conversion unit 41 .
　スペクトログラム生成部４２は、第１変換部４１による第１変換処理の結果を用いて、スペクトログラムを生成する機能ブロックである。スペクトログラム生成部４２は、第１変換部４１によるデータ単位毎の第１変換処理の結果を、時系列に期間Ｔ２にわたって蓄積する。スペクトログラム生成部４２は、期間Ｔ２にわたって蓄積された複数の第１変換処理の結果に基づき、スペクトログラムを生成する。スペクトログラム生成部４２は、生成されたスペクトログラムを第２変換部４３に送信する。 The spectrogram generation unit 42 is a functional block that generates a spectrogram using the result of the first conversion processing by the first conversion unit 41. The spectrogram generation unit 42 accumulates the results of the first conversion processing for each data unit by the first conversion unit 41 in time series over the period T2. The spectrogram generator 42 generates a spectrogram based on the results of a plurality of first conversion processes accumulated over the period T2. The spectrogram generator 42 transmits the generated spectrogram to the second converter 43 .
　第２変換部４３は、スペクトログラム生成部４２によって生成されたスペクトログラムを変動周波数領域に変換する機能ブロックである。第２変換部４３は、スペクトログラムを、複数のデータ群に分割する。第２変換部４３は、例えば、データ群毎に第２変換処理（例えば、ＦＦＴ処理）を実行する。第２変換部４３は、データ群毎の第２変換処理の結果を、教師特徴量マップ生成部４４に送信する。なお、第２変換部４３の第２変換処理における変動周波数帯域は、判定モデルＭと学習前モデルＭ０との整合性の観点から、第２変換部３３と同じ変動周波数帯域が適用されることが望ましい。 The second conversion unit 43 is a functional block that converts the spectrogram generated by the spectrogram generation unit 42 into the fluctuating frequency domain. The second conversion unit 43 divides the spectrogram into multiple data groups. The second conversion unit 43 performs, for example, second conversion processing (for example, FFT processing) for each data group. The second conversion unit 43 transmits the result of the second conversion processing for each data group to the teacher feature quantity map generation unit 44 . It should be noted that, from the viewpoint of consistency between the determination model M and the pre-learning model M0, the same fluctuation frequency band as that of the second conversion unit 33 may be applied to the fluctuation frequency band in the second conversion processing of the second conversion unit 43. desirable.
　教師特徴量マップ生成部４４は、第２変換部４３による第２変換処理の結果を用いて、教師特徴量マップを生成する機能ブロックである。教師特徴量マップ生成部４４は、１個のスペクトログラムから生成される全てのデータ群の第２変換処理の結果を、教師特徴量候補として蓄積する。教師特徴量マップ生成部４４は、蓄積された教師特徴量候補をマッピングすることにより、スペクトログラム毎に教師特徴量候補マップを生成する。そして、教師特徴量マップ生成部４４は、複数のスペクトログラムに対応する複数の教師特徴量候補マップに基づき、教師特徴量マップを生成する。教師特徴量マップ生成部４４は、生成された教師特徴量マップを確率算出部４５に送信する。 The teacher feature quantity map generation unit 44 is a functional block that uses the result of the second conversion processing by the second conversion unit 43 to generate a teacher feature quantity map. The teacher feature quantity map generation unit 44 accumulates the results of the second conversion processing of all data groups generated from one spectrogram as teacher feature quantity candidates. The teacher feature quantity map generating unit 44 generates a teacher feature quantity candidate map for each spectrogram by mapping the accumulated teacher feature quantity candidates. Then, the teacher feature quantity map generating unit 44 generates a teacher feature quantity map based on a plurality of teacher feature quantity candidate maps corresponding to a plurality of spectrograms. The teacher feature quantity map generation unit 44 transmits the generated teacher feature quantity map to the probability calculation unit 45 .
　図９は、実施形態に係るモデル学習装置で生成される複数の教師特徴量候補マップと教師特徴量マップとの関係の一例を示す図である。図９に示すように、教師特徴量マップにおける各グリッドの振幅値には、例えば、複数の教師特徴量候補マップの対応するグリッドにおける振幅値の最大値が適用される。 FIG. 9 is a diagram showing an example of the relationship between a plurality of teacher feature quantity candidate maps generated by the model learning device according to the embodiment and the teacher feature quantity map. As shown in FIG. 9, for the amplitude value of each grid in the teacher feature quantity map, for example, the maximum value of the amplitude values in the corresponding grids of the plurality of teacher feature quantity candidate maps is applied.
　再び図８を参照して、モデル学習装置５の機能構成について説明する。 The functional configuration of the model learning device 5 will be described with reference to FIG. 8 again.
　確率算出部４５の機能構成は、特徴量マップに代えて教師特徴量マップが入力である点、及び判定モデルＭに代えて学習前モデルＭ０が用いられる点を除き、確率算出部３５と同等である。 The functional configuration of the probability calculation unit 45 is the same as that of the probability calculation unit 35, except that the teacher feature map is input instead of the feature map, and that the pre-learning model M0 is used instead of the judgment model M. be.
　確率算出部４５は、教師データに異常がある確率を算出する機能ブロックである。確率算出部４５は、学習前モデルＭ０に教師特徴量マップを入力する。確率算出部４５は、教師特徴量マップが入力された学習前モデルＭ０からの出力結果に基づき、教師データの異常確率を算出する。確率算出部４５は、算出された異常確率を更新部４６に送信する。 The probability calculation unit 45 is a functional block that calculates the probability that teacher data is abnormal. The probability calculation unit 45 inputs the teacher feature quantity map to the pre-learning model M0. The probability calculation unit 45 calculates the abnormality probability of teacher data based on the output result from the pre-learning model M0 to which the teacher feature quantity map is input. The probability calculator 45 transmits the calculated abnormality probability to the updater 46 .
　更新部４６は、確率算出部４５によって算出された異常確率に基づき、学習前モデルＭ０のパラメタを更新する機能ブロックである。具体的には、例えば、更新部４６は、算出された異常確率とラベルとに基づき、評価関数を算出する。評価関数には、例えば、算出された異常確率とラベルとが一致した場合に最小となるような関数が適用される。算出された評価関数が閾値以上である場合、更新部４６は、学習前モデルＭ０のパラメタを更新する。パラメタの更新に際して、更新部４６は、例えば、誤差逆伝播法を用いる。更新部４６によってパラメタが更新された学習前モデルＭ０は、確率算出部４５にフィードバックされる。教師データセットＴに関して算出された全ての評価関数が閾値未満である場合、更新部４６は、学習前モデルＭ０を、判定モデルＭとして異常判定装置４に送信する。 The update unit 46 is a functional block that updates the parameters of the pre-learning model M0 based on the abnormality probability calculated by the probability calculation unit 45. Specifically, for example, the updating unit 46 calculates an evaluation function based on the calculated abnormality probability and label. For the evaluation function, for example, a function that minimizes when the calculated abnormality probability matches the label is applied. When the calculated evaluation function is equal to or greater than the threshold, the updating unit 46 updates the parameters of the pre-learning model M0. When updating the parameters, the updating unit 46 uses, for example, the error backpropagation method. The pre-learning model M<b>0 whose parameters have been updated by the update unit 46 is fed back to the probability calculation unit 45 . When all the evaluation functions calculated for the teacher data set T are less than the threshold, the update unit 46 transmits the pre-learning model M0 as the determination model M to the abnormality determination device 4.
　以上のように構成されることにより、モデル学習装置５は、異常判定処理に最適化された判定モデルＭを異常判定装置４に提供することができる。 By being configured as described above, the model learning device 5 can provide the abnormality determination device 4 with the determination model M optimized for the abnormality determination process.
　２．　動作
　次に、実施形態に係る機構制御システムの動作について説明する。 2. Operation Next, the operation of the mechanism control system according to the embodiment will be described.
　２．１　機構制御処理
　まず、実施形態に係る機構制御システムにおける機構制御処理について説明する。 2.1 Mechanism Control Processing First, mechanism control processing in the mechanism control system according to the embodiment will be described.
　図１０は、実施形態に係る機構制御システムにおける機構制御処理の一例を示すフローチャートである。 FIG. 10 is a flowchart showing an example of mechanism control processing in the mechanism control system according to the embodiment.
　ユーザから機構２の動作モードが入力されると（開始）、制御装置６は、入力された動作モードに応じた動作モード信号ＭＯＤＥを異常判定装置４に送信する（Ｓ１）。動作モード信号ＭＯＤＥは、異常判定装置４における判定モデルＭの選択に用いられる。 When the operation mode of the mechanism 2 is input by the user (start), the control device 6 transmits an operation mode signal MODE corresponding to the input operation mode to the abnormality determination device 4 (S1). The operation mode signal MODE is used for selecting the determination model M in the abnormality determination device 4 .
　Ｓ１の処理の後、制御装置６は、入力された動作モードに応じて機構２の動作を開始させる（Ｓ２）。 After the processing of S1, the control device 6 starts the operation of the mechanism 2 according to the input operation mode (S2).
　Ｓ２の処理の後、機構２は、制御信号ＣＮＴで指示された動作モードを開始する。異常判定装置４は、センサ３から計測データＳを受信すると、Ｓ１の処理で送信された動作モード信号ＭＯＤＥに基づき、計測データＳに対する異常判定処理を実行する。一方、制御装置６は、入力された動作モードが終了するまで、又は異常通知信号ＡＮＯＭを受信するまで待機する（Ｓ３）。 After the processing of S2, the mechanism 2 starts the operation mode indicated by the control signal CNT. Upon receiving the measurement data S from the sensor 3, the abnormality determination device 4 executes abnormality determination processing for the measurement data S based on the operation mode signal MODE transmitted in the process of S1. On the other hand, the control device 6 waits until the input operation mode ends or until it receives the abnormality notification signal ANOM (S3).
　動作モードの終了後、又は異常通知信号ＡＮＯＭの受信後、制御装置６は、異常通知信号ＡＮＯＭを受信したか否かを判定する（Ｓ４）。Ｓ４の処理は、制御装置６が機構２の動作モードが終了したか否かを判定することと同義である。 After the operation mode ends or after receiving the abnormality notification signal ANOM, the control device 6 determines whether or not the abnormality notification signal ANOM has been received (S4). The processing of S4 is synonymous with the control device 6 determining whether or not the operation mode of the mechanism 2 has ended.
　異常通知信号ＡＮＯＭを受信した場合（つまり、動作モードが終了していない場合）（Ｓ４；ｙｅｓ）、制御装置６は、実行中の動作モードを中断することにより、機構２の動作を停止させる（Ｓ５）。 When the abnormality notification signal ANOM is received (that is, when the operation mode has not ended) (S4; yes), the control device 6 stops the operation of the mechanism 2 by interrupting the operation mode being executed ( S5).
　異常通知信号ＡＮＯＭを受信していない場合（つまり、動作モードが終了した場合）（Ｓ４；ｎｏ）、又はＳ５の処理の後、機構制御処理は終了となる（終了）。 If the abnormality notification signal ANOM is not received (that is, if the operation mode ends) (S4; no), or after the process of S5, the mechanism control process ends (end).
　以上のように動作することにより、異常判定装置４が機構２の異常を判定した場合、機構２の動作モードを自動的に中断させることができる。 By operating as described above, when the abnormality determination device 4 determines that the mechanism 2 is abnormal, the operation mode of the mechanism 2 can be automatically interrupted.
　２．２　異常判定処理
　次に、実施形態に係る異常判定装置における異常判定処理について説明する。 2.2 Abnormality Determination Processing Next, the abnormality determination processing in the abnormality determination device according to the embodiment will be described.
　図１１は、実施形態に係る異常判定装置における異常判定処理の一例を示すフローチャートである。図１１の例では、異常判定装置４は、モデル学習装置５から判定モデルＭを予め受信しているものとする。 FIG. 11 is a flowchart showing an example of abnormality determination processing in the abnormality determination device according to the embodiment. In the example of FIG. 11, it is assumed that the abnormality determination device 4 has received the determination model M from the model learning device 5 in advance.
　図１１に示すように、制御装置６から動作モード信号ＭＯＤＥを受信すると（開始）、確率算出部３５は、受信した動作モード信号ＭＯＤＥに基づいて、判定モデルＭを選択する（Ｓ１１）。 As shown in FIG. 11, when the operation mode signal MODE is received from the control device 6 (start), the probability calculation unit 35 selects the determination model M based on the received operation mode signal MODE (S11).
　Ｓ１１の処理の後、異常判定装置４は、計測データＳの受信を開始するまで待機する（Ｓ１２）。 After the process of S11, the abnormality determination device 4 waits until it starts receiving the measurement data S (S12).
　計測データＳの受信が開始すると、第１変換部３１、スペクトログラム生成部３２、第２変換部３３、及び特徴量マップ生成部３４は、特徴量マップ生成処理を実行する（Ｓ１３）。特徴量マップ生成処理の詳細については、後述する。 When the reception of the measurement data S starts, the first conversion unit 31, the spectrogram generation unit 32, the second conversion unit 33, and the feature map generation unit 34 execute feature map generation processing (S13). Details of the feature map generation process will be described later.
　確率算出部３５は、Ｓ１３の処理で生成された特徴量マップを判定モデルＭに入力することにより、異常確率を算出する（Ｓ１４）。 The probability calculation unit 35 calculates an abnormality probability by inputting the feature quantity map generated in the process of S13 to the judgment model M (S14).
　判定部３６は、Ｓ１４の処理によって算出された異常確率に基づき、ユーザに判定履歴Ｒを出力する（Ｓ１５）。 The determination unit 36 outputs the determination history R to the user based on the abnormality probability calculated by the process of S14 (S15).
　また、判定部３６は、Ｓ１４の処理によって算出された異常確率に基づき、計測データＳに異常があるか否かを判定する（Ｓ１６）。 Further, the determination unit 36 determines whether or not there is an abnormality in the measurement data S based on the abnormality probability calculated by the process of S14 (S16).
　計測データＳに異常があると判定された場合（Ｓ１６；ｙｅｓ）、判定部３６は、異常通知信号ＡＮＯＭを制御装置６に送信する（Ｓ１７）。 When it is determined that there is an abnormality in the measurement data S (S16; yes), the determination unit 36 transmits an abnormality notification signal ANOM to the control device 6 (S17).
　計測データＳに異常がないと判定された場合（Ｓ１６；ｎｏ）、又はＳ１７の処理の後、異常判定装置４は、計測データＳの受信が終了したか否かを判定する（Ｓ１８）。 When it is determined that there is no abnormality in the measurement data S (S16; no), or after the process of S17, the abnormality determination device 4 determines whether or not the reception of the measurement data S has ended (S18).
　計測データＳの受信が終了していない場合（Ｓ１８；ｎｏ）、第１変換部３１、スペクトログラム生成部３２、第２変換部３３、及び特徴量マップ生成部３４は、新たに受信した計測データＳに基づく特徴量マップ生成処理を実行する（Ｓ１３）。そして、Ｓ１３の処理に続いてＳ１４～Ｓ１８の処理が実行される。このように、計測データＳの受信が終了するまで、Ｓ１３～Ｓ１８の処理が繰り返される。 When the reception of the measurement data S has not ended (S18; no), the first conversion unit 31, the spectrogram generation unit 32, the second conversion unit 33, and the feature map generation unit 34 receive the newly received measurement data S (S13). Then, following the process of S13, the processes of S14 to S18 are executed. In this way, the processes of S13 to S18 are repeated until the reception of the measurement data S is completed.
　なお、Ｓ１３～Ｓ１８の処理は、例えば、リアルタイムで実行される。より具体的には、或るループにおけるＳ１３～Ｓ１８の処理は、１つ前のループにおけるＳ１３～Ｓ１８の処理を実行中に受信した計測データＳに対して実行される。 Note that the processes of S13 to S18 are executed in real time, for example. More specifically, the processing of S13 to S18 in a certain loop is performed on the measurement data S received during the execution of the processing of S13 to S18 in the previous loop.
　計測データＳの受信が終了した場合（Ｓ１８；ｙｅｓ）、異常判定処理は終了となる（終了）。 When the reception of the measurement data S ends (S18; yes), the abnormality determination process ends (end).
　２．３　特徴量マップ生成処理
　次に、実施形態に係る異常判定装置における特徴量マップ生成処理について説明する。 2.3 Feature Amount Map Generation Processing Next, feature amount map generation processing in the abnormality determination device according to the embodiment will be described.
　図１２は、実施形態に係る異常判定装置における特徴量マップ生成処理の一例を示すフローチャートである。図１２におけるＳ２１～Ｓ２８の処理は、図１１におけるＳ１３の処理の詳細に対応する。 FIG. 12 is a flowchart showing an example of feature map generation processing in the abnormality determination device according to the embodiment. The processing of S21 to S28 in FIG. 12 corresponds to the details of the processing of S13 in FIG.
　図１２に示すように、計測データＳを受信すると（開始）、第１変換部３１は、期間Ｔ１分（すなわち、データ単位）の計測データＳが蓄積されるまで待機する（Ｓ２１）。 As shown in FIG. 12, when the measurement data S are received (start), the first conversion unit 31 waits until the measurement data S for the period T1 (that is, data unit) is accumulated (S21).
　期間Ｔ１分の計測データＳが蓄積された後、第１変換部３１は、蓄積された期間Ｔ１分の計測データＳを周波数領域に変換する（Ｓ２２）。 After the measurement data S for the period T1 is accumulated, the first conversion unit 31 converts the accumulated measurement data S for the period T1 into the frequency domain (S22).
　第１変換部３１は、期間Ｔ２分の計測データＳがＳ２２の処理によって変換されたか否かを判定する（Ｓ２３）。 The first conversion unit 31 determines whether or not the measurement data S for the period T2 has been converted by the process of S22 (S23).
　期間Ｔ２分の計測データＳが変換されていない場合（Ｓ２３；ｎｏ）、第１変換部３１は、期間Ｔ１分の新たな計測データＳが蓄積されるまで待機する（Ｓ２１）。そして、期間Ｔ１分の計測データＳが蓄積されると、第１変換部３１は、Ｓ２２の処理を実行する。このように、期間Ｔ２分の計測データＳがＳ２２の処理によって変換されるまで、Ｓ２１及びＳ２２の処理が繰り返される。 If the measurement data S for the period T2 has not been converted (S23; no), the first conversion unit 31 waits until new measurement data S for the period T1 is accumulated (S21). Then, when the measurement data S for the period T1 is accumulated, the first converter 31 executes the process of S22. In this manner, the processes of S21 and S22 are repeated until the measurement data S for the period T2 are converted by the process of S22.
　期間Ｔ２分の計測データＳが変換された場合（Ｓ２３；ｙｅｓ）、スペクトログラム生成部３２は、時系列に期間Ｔ２にわたって複数回実行されたＳ２２の処理の結果に基づき、スペクトログラムを生成する（Ｓ２４）。 When the measurement data S for the period T2 has been converted (S23; yes), the spectrogram generator 32 generates a spectrogram based on the results of the process of S22 executed multiple times over the period T2 in chronological order (S24). .
　第２変換部３３は、Ｓ２４の処理で生成されたスペクトログラムの周波数領域から、周波数帯域を選択する（Ｓ２５）。 The second conversion unit 33 selects a frequency band from the frequency domain of the spectrogram generated in the process of S24 (S25).
　第２変換部３３は、スペクトログラムのうち、Ｓ２５の処理で選択された周波数帯域に対応する時系列データ（すなわち、データ群）を変動周波数領域に変換する（Ｓ２６）。 The second transforming unit 33 transforms the time-series data (that is, data group) corresponding to the frequency band selected in the process of S25 from the spectrogram into the fluctuating frequency domain (S26).
　第２変換部３３は、スペクトログラムから全ての周波数帯域が選択されたか否かを判定する（Ｓ２７）。 The second conversion unit 33 determines whether or not all frequency bands have been selected from the spectrogram (S27).
　選択されていない周波数帯域がある場合（Ｓ２７；ｎｏ）、第２変換部３３は、Ｓ２４の処理で生成されたスペクトログラムの周波数領域から、未選択の周波数帯域を選択する（Ｓ２５）。そして、Ｓ２５の処理に続いて、Ｓ２６及びＳ２７の処理が実行される。このように、スペクトログラム内の全ての周波数帯域に対応するデータ群がＳ２６の処理によって変換されるまで、Ｓ２５～Ｓ２７の処理が繰り返される。 If there is an unselected frequency band (S27; no), the second conversion unit 33 selects an unselected frequency band from the frequency domain of the spectrogram generated in S24 (S25). Then, following the process of S25, the processes of S26 and S27 are executed. In this manner, the processes of S25 to S27 are repeated until the data groups corresponding to all frequency bands in the spectrogram are converted by the process of S26.
　全ての周波数帯域が選択された場合（Ｓ２７；ｙｅｓ）、特徴量マップ生成部３４は、同一のスペクトログラム内の全てのデータ群に対してそれぞれ実行された複数回のＳ２６の処理の結果に基づき、特徴量マップを生成する（Ｓ２８）。 If all frequency bands have been selected (S27; yes), the feature map generation unit 34 performs a plurality of S26 processes on all data groups within the same spectrogram. Based on the results, A feature quantity map is generated (S28).
　以上により、特徴量マップ生成処理は終了となる（終了）。 With the above, the feature map generation process ends (end).
　２．４　モデル学習処理
　次に、実施形態に係るモデル学習装置におけるモデル学習処理について説明する。 2.4 Model Learning Processing Next, model learning processing in the model learning device according to the embodiment will be described.
　図１３は、実施形態に係るモデル学習装置におけるモデル学習処理の一例を示すフローチャートである。図１３の例では、モデル学習装置５は、教師データセットＴ及び学習前モデルＭ０を予め記憶しているものとする。 FIG. 13 is a flowchart showing an example of model learning processing in the model learning device according to the embodiment. In the example of FIG. 13, the model learning device 5 pre-stores the teacher data set T and the pre-learning model M0.
　図１３に示すように、ユーザからモデル学習処理を開始する旨の指示が入力されると（開始）、制御回路２１は、教師データセットＴから教師データを選択する（Ｓ３１）。 As shown in FIG. 13, when the user inputs an instruction to start the model learning process (start), the control circuit 21 selects teacher data from the teacher data set T (S31).
　第１変換部４１、スペクトログラム生成部４２、第２変換部４３、及び教師特徴量マップ生成部４４は、Ｓ３１の処理で選択された教師データに基づき、教師特徴量マップ生成処理を実行する（Ｓ３２）。教師特徴量マップ生成処理の詳細については、後述する。 The first conversion unit 41, the spectrogram generation unit 42, the second conversion unit 43, and the teacher feature amount map generation unit 44 execute the teacher feature amount map generation process based on the teacher data selected in the process of S31 (S32 ). Details of the teacher feature quantity map generation processing will be described later.
　確率算出部４５は、Ｓ３２の処理で生成された教師特徴量マップを学習前モデルＭ０に入力することにより、異常確率を算出する（Ｓ３３）。 The probability calculation unit 45 calculates an abnormality probability by inputting the teacher feature quantity map generated in the process of S32 to the pre-learning model M0 (S33).
　更新部４６は、Ｓ３３の処理によって算出された異常確率が、Ｓ３１の処理で選択された教師データ内のラベルに対して条件を満たすか否かを判定する（Ｓ３４）。例えば、更新部４６は、異常確率及びラベルに基づいて評価関数を算出する。そして、更新部４６は、算出された評価関数を閾値と比較することにより、異常確率及びラベルの一致性を判定する。 The update unit 46 determines whether the abnormality probability calculated by the process of S33 satisfies the conditions for the label in the teacher data selected by the process of S31 (S34). For example, the updating unit 46 calculates an evaluation function based on the abnormality probability and the label. The updating unit 46 then compares the calculated evaluation function with a threshold value to determine the abnormality probability and label matching.
　異常確率がラベルに対して条件を満たさない（すなわち、異常確率がラベルと一致しない）と判定された場合（Ｓ３４；ｎｏ）、更新部４６は、学習前モデルＭ０を更新する（Ｓ３５）。 If it is determined that the abnormality probability does not satisfy the condition for the label (that is, the abnormality probability does not match the label) (S34; no), the updating unit 46 updates the pre-learning model M0 (S35).
　Ｓ３５の処理の後、確率算出部４５は、Ｓ３２の処理で生成された教師特徴量マップを、Ｓ３５の処理で更新された学習前モデルＭ０に入力する（Ｓ３３）。そして、Ｓ３３の処理に続いてＳ３４及びＳ３５の処理が実行される。このように、異常確率がラベルに対して条件を満たすと判定されるまで、Ｓ３３～Ｓ３５の処理が繰り返される。 After the process of S35, the probability calculation unit 45 inputs the teacher feature map generated in the process of S32 to the pre-learning model M0 updated in the process of S35 (S33). Then, following the process of S33, the processes of S34 and S35 are executed. In this manner, the processes of S33 to S35 are repeated until it is determined that the abnormality probability satisfies the condition for the label.
　異常確率がラベルに対して条件を満たす（すなわち、異常確率がラベルと一致する）と判定された場合（Ｓ３４；ｙｅｓ）、制御回路２１は、教師データセットＴ内の全ての教師データが選択済みであるか否かを判定する（Ｓ３６）。 If it is determined that the abnormality probability satisfies the condition for the label (that is, the abnormality probability matches the label) (S34; yes), the control circuit 21 determines that all the teacher data in the teacher data set T have been selected. (S36).
　選択されていない教師データがある場合（Ｓ３６；ｎｏ）、制御回路２１は、教師データセットＴから未選択の教師データを選択する（Ｓ３１）。そして、Ｓ３１の処理に続いてＳ３２～Ｓ３６の処理が実行される。このように、全ての教師データが選択されるまで、Ｓ３１～Ｓ３６の処理が繰り返される。 If there is unselected teacher data (S36; no), the control circuit 21 selects unselected teacher data from the teacher data set T (S31). Then, following the process of S31, the processes of S32 to S36 are executed. In this manner, the processes of S31 to S36 are repeated until all teaching data are selected.
　全ての教師データ選択済みである場合（Ｓ３６；ｙｅｓ）、更新部４６は、学習前モデルＭ０を判定モデルＭとして異常判定装置４に送信する（Ｓ３７）。 If all the teacher data have been selected (S36; yes), the update unit 46 transmits the pre-learning model M0 as the determination model M to the abnormality determination device 4 (S37).
　Ｓ３７の処理が終了すると、モデル学習処理は終了となる（終了）。 When the process of S37 ends, the model learning process ends (end).
　２．５　教師特徴量マップ生成処理
　次に、実施形態に係るモデル学習装置における教師特徴量マップ生成処理について説明する。 2.5 Teacher Feature Amount Map Generation Processing Next, teacher feature amount map generation processing in the model learning apparatus according to the embodiment will be described.
　図１４は、実施形態に係るモデル学習装置における教師特徴量マップ生成処理の一例を示すフローチャートである。図１４におけるＳ４１～Ｓ５０の処理は、図１３におけるＳ３２の処理の詳細に対応する。 FIG. 14 is a flowchart showing an example of teacher feature map generation processing in the model learning device according to the embodiment. The processing of S41 to S50 in FIG. 14 corresponds to the details of the processing of S32 in FIG.
　図１４に示すように、教師データセットＴ内の教師データが選択されると（開始）、第１変換部４１は、選択された教師データのうち期間Ｔ１分（すなわち、データ単位）を選択する（Ｓ４１）。 As shown in FIG. 14, when teacher data in the teacher data set T is selected (start), the first conversion unit 41 selects a period T1 (ie, data unit) from the selected teacher data. (S41).
　期間Ｔ１分の教師データが選択された後、第１変換部４１は、選択された期間Ｔ１分の教師データを周波数領域に変換する（Ｓ４２）。 After the teacher data for the period T1 is selected, the first conversion unit 41 transforms the selected teacher data for the period T1 into the frequency domain (S42).
　第１変換部４１は、期間Ｔ２分の教師データがＳ４２の処理によって変換されたか否かを判定する（Ｓ４３）。 The first conversion unit 41 determines whether or not the teacher data for period T2 has been converted by the process of S42 (S43).
　期間Ｔ２分の教師データが変換されていない場合（Ｓ４３；ｎｏ）、第１変換部４１は、期間Ｔ１分の未選択の教師データを更に選択する（Ｓ４１）。そして、期間Ｔ１分の教師データが選択されると、第１変換部４１は、新たに選択された期間Ｔ１分の教師データに対してＳ４２の処理を実行する。このように、期間Ｔ２分の教師データがＳ４２の処理によって変換されるまで、Ｓ４１及びＳ４２の処理が繰り返される。 If the teacher data for period T2 has not been converted (S43; no), the first conversion unit 41 further selects unselected teacher data for period T1 (S41). Then, when the teacher data for the period T1 is selected, the first conversion unit 41 performs the process of S42 on the newly selected teacher data for the period T1. In this way, the processes of S41 and S42 are repeated until the teacher data for period T2 is converted by the process of S42.
　期間Ｔ２分の教師データが変換された場合（Ｓ４３；ｙｅｓ）、スペクトログラム生成部４２は、時系列に期間Ｔ２にわたって複数回実行されたＳ４２の処理の結果に基づき、スペクトログラムを生成する（Ｓ４４）。 When the teacher data for period T2 has been converted (S43; yes), the spectrogram generation unit 42 generates a spectrogram based on the results of the process of S42 executed multiple times over period T2 in chronological order (S44).
　第２変換部４３は、Ｓ４４の処理で生成されたスペクトログラムの周波数領域から、周波数帯域を選択する（Ｓ４５）。 The second conversion unit 43 selects a frequency band from the frequency domain of the spectrogram generated in the process of S44 (S45).
　第２変換部４３は、スペクトログラムのうち、Ｓ４５の処理で選択された周波数帯域に対応する時系列データ（すなわち、データ群）を変動周波数領域に変換する（Ｓ４６）。 The second conversion unit 43 converts the time-series data (that is, the data group) corresponding to the frequency band selected in the process of S45 from the spectrogram into the fluctuation frequency domain (S46).
　第２変換部４３は、スペクトログラムから全ての周波数帯域が選択されたか否かを判定する（Ｓ４７）。 The second conversion unit 43 determines whether or not all frequency bands have been selected from the spectrogram (S47).
　選択されていない周波数帯域がある場合（Ｓ４７；ｎｏ）、第２変換部４３は、Ｓ４４の処理で生成されたスペクトログラムの周波数領域から、未選択の周波数帯域を選択する（Ｓ４５）。そして、Ｓ４５の処理に続いて、Ｓ４６及びＳ４７の処理が実行される。このように、スペクトログラム内の全ての周波数帯域に対応するデータ群がＳ４６の処理によって変換されるまで、Ｓ４５～Ｓ４７の処理が繰り返される。 If there is an unselected frequency band (S47; no), the second conversion unit 43 selects an unselected frequency band from the frequency domain of the spectrogram generated in S44 (S45). Then, following the process of S45, the processes of S46 and S47 are executed. In this way, the processes of S45 to S47 are repeated until the data groups corresponding to all frequency bands in the spectrogram are converted by the process of S46.
　全ての周波数帯域が選択された場合（Ｓ４７；ｙｅｓ）、教師特徴量マップ生成部４４は、同一のスペクトログラム内の全てのデータ群に対してそれぞれ実行された複数回のＳ４６の処理の結果に基づき、教師特徴量候補マップを生成する（Ｓ４８）。 If all frequency bands have been selected (S47; yes), the teacher feature map generation unit 44 performs multiple processing of S46 on all data groups within the same spectrogram. , a teacher feature value candidate map is generated (S48).
　Ｓ４８の後、教師特徴量マップ生成部４４は、教師データの全期間にわたって教師特徴量候補マップが生成されたか否かを判定する（Ｓ４９）。 After S48, the teacher feature quantity map generation unit 44 determines whether or not the teacher feature quantity candidate map has been generated over the entire period of the teacher data (S49).
　教師データ内に教師特徴量候補マップが生成されていない期間がある場合（Ｓ４９；ｎｏ）、第１変換部４１は、教師特徴量候補マップの生成に用いられていない期間の教師データから、期間Ｔ１分（すなわち、データ単位）を選択する（Ｓ４１）。そして、Ｓ４１の処理に続いて、Ｓ４２～Ｓ４９の処理が実行される。このように、教師データ内の全ての期間にわたって教師特徴量候補マップが生成されるまで、Ｓ４１～Ｓ４９の処理が繰り返される。 If there is a period during which no teacher feature quantity candidate map is generated in the training data (S49; no), the first conversion unit 41 converts the training data of the period not used for generating the teacher feature quantity candidate map into the period T1 minute (ie, data unit) is selected (S41). Then, following the process of S41, the processes of S42 to S49 are executed. In this way, the processing of S41 to S49 is repeated until the teacher feature quantity candidate map is generated over all the periods in the teacher data.
　教師データ内の全ての期間にわたって教師特徴量候補マップが生成された場合（Ｓ４９；ｙｅｓ）、教師特徴量マップ生成部４４は、複数の教師特徴量候補マップに基づいて、教師特徴量マップを生成する（Ｓ５０）。具体的には、例えば、教師特徴量マップ生成部４４は、複数の教師特徴量候補マップのグリッドにおける振幅値の最大値を、教師特徴量マップの対応するグリッドに適用する。 If the teacher feature quantity candidate map is generated over all periods in the teacher data (S49; yes), the teacher feature quantity map generation unit 44 generates a teacher feature quantity map based on a plurality of teacher feature quantity candidate maps. (S50). Specifically, for example, the teacher feature quantity map generation unit 44 applies the maximum value of the amplitude values in the grids of the plurality of teacher feature quantity candidate maps to the corresponding grid of the teacher feature quantity map.
　以上により、教師特徴量マップ生成処理は終了となる（終了）。 With the above, the teacher feature value map generation processing is completed (end).
　３．　効果
　実施形態によれば、異常判定装置４の第１変換部３１、スペクトログラム生成部３２、第２変換部３３、及び特徴量マップ生成部３４は、計測データＳに基づき、各々が周波数帯域及び変動周波数帯域の組に対応づけられた複数の特徴量を算出する。確率算出部３５は、算出された複数の特徴量を判定モデルＭに入力することにより、計測データＳの異常確率を算出する。判定モデルＭとしては、畳み込みニューラルネットワークが用いられる。これにより、異常判定装置４は、検査員を介することなく、客観的な判定処理の指標として異常確率を自動的に算出することができる。 3. Effect According to the embodiment, the first conversion unit 31, the spectrogram generation unit 32, the second conversion unit 33, and the feature amount map generation unit 34 of the abnormality determination device 4 are based on the measurement data S, and each of the frequency band and fluctuation A plurality of feature quantities associated with the set of frequency bands are calculated. The probability calculation unit 35 calculates the abnormality probability of the measurement data S by inputting the plurality of calculated feature amounts to the determination model M. FIG. As the judgment model M, a convolutional neural network is used. As a result, the abnormality determination device 4 can automatically calculate the abnormality probability as an index for objective determination processing without the intervention of an inspector.
　具体的には、第１変換部３１は、計測データＳ内の複数のデータ単位の各々を周波数領域に変換する。スペクトログラム生成部３２は、第１変換処理の結果に基づいて、各々が対応する周波数帯域で時系列に並ぶ複数のデータ群を生成する。第２変換部３３は、複数のデータ群の各々を変動周波数領域に変換する。特徴量マップ生成部３４は、第２変換処理の結果を、対応する周波数帯域及び変動周波数帯域にマッピングする。第１変換処理では、複数のデータ単位の各々は、バーク尺度を除く尺度に基づく周波数帯域で分割される。第２変換処理では、複数のデータ群の各々は、バーク尺度を除く尺度に基づく変動周波数帯域で分割される。これにより、ラウドネスのような心理量の算出を省略しつつ、特徴量マップを生成できる。このように、人の聴感によらない物理量として複数の特徴量を算出することで、より合理的かつ客観的な判定処理を実行することができる。加えて、ラウドネス算出に要する計算量の多い処理を省略することで、異常判定装置４に汎用計算機が適用される場合でも、計測データＳに基づいて、複数の特徴量をリアルタイムで算出することができる。 Specifically, the first conversion unit 31 converts each of the plurality of data units in the measurement data S into the frequency domain. The spectrogram generation unit 32 generates a plurality of data groups arranged in time series in corresponding frequency bands based on the result of the first conversion processing. The second transforming unit 33 transforms each of the plurality of data groups into the fluctuating frequency domain. The feature quantity map generator 34 maps the result of the second conversion process to the corresponding frequency band and fluctuation frequency band. In the first conversion process, each of the plurality of data units is divided into frequency bands based on a scale other than the Bark scale. In the second transform process, each of the plurality of data groups is divided into varying frequency bands based on the scale other than the Bark scale. As a result, a feature quantity map can be generated while omitting calculation of a psychological quantity such as loudness. In this way, by calculating a plurality of feature amounts as physical amounts that do not depend on human auditory sense, more rational and objective determination processing can be performed. In addition, by omitting the processing requiring a large amount of calculation required for loudness calculation, even when a general-purpose computer is applied to the abnormality determination device 4, it is possible to calculate a plurality of feature amounts in real time based on the measurement data S. can.
　また、判定部３６は、算出された異常確率に基づき、計測データＳの異常の有無を判定する。これにより、異常判定装置４は、検査員を介することなく、自動的に判定履歴Ｒ及び異常の通知をユーザに出力すると共に、異常通知信号ＡＮＯＭを制御装置６に送信することができる。 Also, the determination unit 36 determines whether or not there is an abnormality in the measurement data S based on the calculated abnormality probability. As a result, the abnormality determination device 4 can automatically output the determination history R and the notification of the abnormality to the user and transmit the abnormality notification signal ANOM to the control device 6 without the intervention of an inspector.
　また、制御装置６は、異常通知信号ＡＮＯＭに基づいて、機構２の動作モードを中断させる。これにより、機構２から発生する音又は振動に異常があると判定された場合には、機構２を自動的に中断させることができる。このように、機構制御システム１は、異常の有無の判定から機構２の動作制御までを自動的に行うことができる。 Also, the control device 6 suspends the operation mode of the mechanism 2 based on the abnormality notification signal ANOM. Thereby, when it is determined that the sound or vibration generated from the mechanism 2 is abnormal, the mechanism 2 can be automatically interrupted. In this way, the mechanism control system 1 can automatically perform everything from determining whether there is an abnormality to controlling the operation of the mechanism 2 .
　また、モデル学習装置５の教師特徴量マップ生成部４４は、時系列に並ぶ複数の特徴量マップに対応する複数の教師特徴量候補マップを生成する。教師特徴量マップ生成部４４は、複数の教師特徴量候補マップの最大値を対応するグリッドにマッピングすることにより、教師特徴量マップを生成する。これにより、機構２が正常な場合には発生しない特徴的な音又は振動が反映された特徴量を教師特徴量マップに反映させることができる。加えて、ラベルを教師データ内のデータ全体に対して１対１に対応づけることができるため、データに対するラベル付けの作業負荷を軽減することができる。 In addition, the teacher feature quantity map generation unit 44 of the model learning device 5 generates a plurality of teacher feature quantity candidate maps corresponding to the plurality of feature quantity maps arranged in time series. The teacher feature quantity map generation unit 44 generates a teacher feature quantity map by mapping the maximum value of a plurality of teacher feature quantity candidate maps to the corresponding grid. As a result, it is possible to reflect in the teacher feature quantity map a feature quantity reflecting a characteristic sound or vibration that does not occur when the mechanism 2 is normal. In addition, since labels can be associated one-to-one with all the data in the teacher data, the workload of labeling the data can be reduced.
　また、更新部４６は、教師特徴量マップが入力された学習前モデルＭ０から算出された異常確率と、ラベルとに基づき、学習前モデルＭ０のパラメタを更新する。これにより、学習前モデルＭ０から算出される異常確率をラベルの異常確率に近づけることができる。このため、モデル学習装置５は、異常判定処理に最適化された判定モデルＭを異常判定装置４に提供することができる。 Further, the updating unit 46 updates the parameters of the pre-learning model M0 based on the anomaly probability calculated from the pre-learning model M0 to which the teacher feature quantity map is input, and the label. As a result, the abnormality probability calculated from the pre-learning model M0 can be brought closer to the label abnormality probability. Therefore, the model learning device 5 can provide the abnormality determination device 4 with the determination model M optimized for the abnormality determination process.
　４．　その他
　なお、上記実施形態には、種々の変形が適用可能である。 4. Others Various modifications can be applied to the above embodiment.
　例えば、上記実施形態では、教師データ内のデータ全体に対してラベルが１対１で割り当てられる場合について説明したが、これに限られない。例えば、ラベルは、教師データ内のデータの期間Ｔ２の部分に対して１対１で割り当てられもよい。この場合、図１４に示されたＳ４９及びＳ５０の処理が省略される。そして、Ｓ４８の処理において生成された教師特徴量候補マップが、教師特徴量マップとして用いられる。これにより、教師データあたりの更新処理の回数を増やすことができる。このため、判定モデルＭの判定精度を向上させることができる。 For example, in the above embodiment, a case where labels are assigned to all data in the teacher data on a one-to-one basis has been described, but the present invention is not limited to this. For example, labels may be assigned on a one-to-one basis to portions of data in the training data for period T2. In this case, the processing of S49 and S50 shown in FIG. 14 is omitted. Then, the teacher feature quantity candidate map generated in the process of S48 is used as the teacher feature quantity map. As a result, it is possible to increase the number of update processes per teacher data. Therefore, the determination accuracy of the determination model M can be improved.
　また、例えば、上記実施形態では、異常通知信号ＡＮＯＭを受信した場合、制御装置６が機構２の動作モードを中断させる場合について説明したが、これに限られない。制御装置６は、機構２をセーフモードに移行させる等、各種制御動作を実行してもよい。また、制御装置６は、異常通知信号ＡＮＯＭに応じて、ユーザに異常の通知をしてもよい。 Also, for example, in the above embodiment, the case where the control device 6 suspends the operation mode of the mechanism 2 when the abnormality notification signal ANOM is received has been described, but the present invention is not limited to this. The control device 6 may perform various control operations such as shifting the mechanism 2 to the safe mode. Further, the control device 6 may notify the user of the abnormality in response to the abnormality notification signal ANOM.
　また、例えば、上記実施形態では、異常判定動作を実行するプログラムが異常判定装置４で、モデル学習動作を実行するプログラムがモデル学習装置５で、それぞれ実行される場合について説明したが、これに限られない。例えば、異常判定動作を実行するプログラム及びモデル学習動作を実行するプログラムは、同一の情報処理装置で実行されてもよい。また、例えば、異常判定動作を実行するプログラム及びモデル学習動作を実行するプログラムは、クラウド上の計算リソースで実行されてもよい。 Further, for example, in the above embodiment, a case has been described in which the program for executing the abnormality determination operation is executed by the abnormality determination device 4, and the program for executing the model learning operation is executed by the model learning device 5. However, this is not the only case. can't For example, the program for executing the abnormality determination operation and the program for executing the model learning operation may be executed by the same information processing device. Further, for example, the program for executing the abnormality determination operation and the program for executing the model learning operation may be executed by computing resources on the cloud.
　なお、本発明は、上記実施形態に限定されるものではなく、実施段階ではその要旨を逸脱しない範囲で種々に変形することが可能である。また、各実施形態は適宜組み合わせて実施してもよく、その場合組み合わせた効果が得られる。更に、上記実施形態には種々の発明が含まれており、開示される複数の構成要件から選択された組み合わせにより種々の発明が抽出され得る。例えば、実施形態に示される全構成要件からいくつかの構成要件が削除されても、課題が解決でき、効果が得られる場合には、この構成要件が削除された構成が発明として抽出され得る。 It should be noted that the present invention is not limited to the above-described embodiments, and can be variously modified in the implementation stage without departing from the gist of the present invention. Further, each embodiment may be implemented in combination as appropriate, in which case the combined effect can be obtained. Furthermore, various inventions are included in the above embodiments, and various inventions can be extracted by combinations selected from a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiments, if the problem can be solved and effects can be obtained, the configuration with the constituent elements deleted can be extracted as an invention.
　１…機構制御システム、２…機構、３…センサ、４…異常判定装置、５…モデル学習装置、６…制御装置、１１，２１…制御回路、１２，２２…ストレージ、１３，２３…通信モジュール、１４，２４…ユーザインタフェース、１５，２５…ドライブ、１６，２６…記憶媒体、３１，４１…第１変換部、３２，４２…スペクトログラム生成部、３３，４３…第２変換部、３４…特徴量マップ生成部、３５，４５…確率算出部、３６…判定部、４４…教師特徴量マップ生成部、４６…更新部、Ｓ…計測データ、Ｍ…判定モデル、Ｍ０…学習前モデル、Ｒ…判定履歴、Ｔ…教師データセット。 Reference Signs List 1 mechanism control system 2 mechanism 3 sensor 4 abnormality determination device 5 model learning device 6 control device 11, 21 control circuit 12, 22 storage 13, 23 communication module , 14, 24... user interface, 15, 25... drive, 16, 26... storage medium, 31, 41... first converter, 32, 42... spectrogram generator, 33, 43... second converter, 34... features Quantity map generation unit 35, 45 Probability calculation unit 36 Judgment unit 44 Teacher feature amount map generation unit 46 Update unit S Measurement data M Judgment model M0 Pre-learning model R Judgment history, T: teacher data set.

Claims (13)

  1. 　時系列データに基づき、各々が周波数帯域及び変動周波数帯域の組に対応づけられた複数の値を算出することと、
    　前記複数の値をモデルに入力して、前記時系列データに異常が含まれる確率を算出することと、
    　を実行するように構成されたプロセッサを備えた、
    　情報処理装置。     calculating a plurality of values, each associated with a set of frequency bands and variable frequency bands, based on the time series data;
    Inputting the plurality of values into a model to calculate a probability that the time series data contains an abnormality;
    with a processor configured to run
    Information processing equipment.
  2. 　前記プロセッサは、前記算出された確率に基づき、前記時系列データに異常が含まれるか否かを判定することを更に実行するように構成された、
    　請求項１記載の情報処理装置。     The processor is further configured to determine whether the time series data includes an anomaly based on the calculated probability.
    The information processing apparatus according to claim 1.
  3. 　前記時系列データは、時系列に並ぶ複数の第１部分データを含み、
    　前記複数の値を算出することは、
    　　前記複数の第１部分データの各々を周波数領域に変換することと、
    　　前記複数の第１部分データの周波数領域への変換結果に基づいて、各々が対応する周波数帯域で時系列に並ぶ複数の第２部分データを生成することと、
    　　前記複数の第２部分データの各々を変動周波数領域に変換することと、
    　　前記複数の第２部分データの変動周波数領域への変換結果を、対応する周波数帯域及び変動周波数帯域にマッピングすることと、
    　を含む、
    　請求項１記載の情報処理装置。     The time series data includes a plurality of first partial data arranged in time series,
    Calculating the plurality of values includes:
    transforming each of the plurality of first partial data into a frequency domain;
    generating a plurality of second partial data arranged in time series in corresponding frequency bands based on the results of transforming the plurality of first partial data into the frequency domain;
    transforming each of the plurality of second partial data into a variable frequency domain;
    mapping the result of transforming the plurality of second partial data into the variable frequency domain to the corresponding frequency band and the variable frequency band;
    including,
    The information processing apparatus according to claim 1.
  4. 　前記周波数領域に変換すること及び前記変動周波数領域に変換することの少なくとも一方は、バーク尺度を除く尺度に基づく帯域でデータを分割することを含む、
    　請求項３記載の情報処理装置。     at least one of the transforming to the frequency domain and the transforming to the varying frequency domain comprises dividing the data into bands based on a scale other than the Bark scale;
    4. The information processing apparatus according to claim 3.
  5. 　前記周波数領域に変換すること及び前記変動周波数領域に変換することの少なくとも一方は、高速フーリエ変換することを含む、
    　請求項３記載の情報処理装置。     at least one of transforming to the frequency domain and transforming to the varying frequency domain comprises fast Fourier transforming;
    4. The information processing apparatus according to claim 3.
  6. 　前記周波数領域に変換すること及び前記変動周波数領域に変換することの少なくとも一方は、オクターブバンド又はメル尺度に基づく帯域でデータを分割することを含む、
    　請求項３記載の情報処理装置。     at least one of the transforming to the frequency domain and the transforming to the varying frequency domain comprises dividing the data into octave bands or bands based on a Mel scale;
    4. The information processing apparatus according to claim 3.
  7. 　前記モデルは、畳み込みニューラルネットワークである、
    　請求項１記載の情報処理装置。     wherein the model is a convolutional neural network;
    The information processing apparatus according to claim 1.
  8. 　前記時系列データは、前記モデルの教師データであり、
    　前記プロセッサは、前記複数の値、及び前記時系列データに対応するラベルに基づいて、前記モデルを更新することを更に実行するように構成された、
    　請求項１記載の情報処理装置。     The time-series data is training data for the model,
    The processor is further configured to update the model based on the plurality of values and labels corresponding to the time series data.
    The information processing apparatus according to claim 1.
  9. 　前記複数の値は、各々が同一の周波数帯域及び変動周波数帯域の組に対応づけられ、かつ互いに異なる時間帯に対応づけられた第１値及び第２値を含み、
    　前記確率を算出することは、前記第１値及び前記第２値のうちの大きい方を前記モデルに入力することを含む、
    　請求項８記載の情報処理装置。     The plurality of values each include a first value and a second value each associated with a set of the same frequency band and variable frequency band and associated with mutually different time zones;
    calculating the probability includes inputting the greater of the first value and the second value into the model;
    The information processing device according to claim 8 .
  10. 　前記時系列データは、音データ又は振動データである、
    　請求項１記載の情報処理装置。     The time-series data is sound data or vibration data,
    The information processing apparatus according to claim 1.
  11. 　機構と、
    　前記機構の動作に応じて、前記時系列データを計測するセンサと、
    　前記センサに接続された請求項２記載の情報処理装置と、
    　前記判定の結果に基づき、前記機構の動作の制御及び判定の結果の送出の少なくとも一方を行うように構成された制御装置と、
    　を備えた、制御システム。     a mechanism;
    a sensor that measures the time-series data according to the operation of the mechanism;
    an information processing device according to claim 2 connected to the sensor;
    a controller configured to at least one of control operation of the mechanism and transmit a result of the determination based on the result of the determination;
    A control system with
  12. 　時系列データに基づき、各々が周波数帯域及び変動周波数帯域の組に対応づけられた複数の値を算出することと、
    　前記複数の値をモデルに入力して、前記時系列データに異常が含まれる確率を算出することと、
    　を備えた、情報処理方法。     calculating a plurality of values, each associated with a set of frequency bands and variable frequency bands, based on the time series data;
    Inputting the plurality of values into a model to calculate a probability that the time series data contains an abnormality;
    A method of processing information, comprising:
  13. 　時系列データに基づき、各々が周波数及び変動周波数の組に対応づけられた複数の値を算出することと、
    　前記複数の値をモデルに入力して、前記時系列データに異常が含まれる確率を算出することと、
    　をプロセッサ実行させるためのプログラム。
          calculating a plurality of values, each associated with a set of frequencies and variation frequencies, based on the time series data;
    Inputting the plurality of values into a model to calculate a probability that the time series data contains an abnormality;
    A program for executing a processor.
PCT/JP2022/041433 2021-11-16 2022-11-07 Information processing device, control system, information processing method, and program WO2023090193A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-186424 2021-11-16
JP2021186424A JP7230155B1 (en) 2021-11-16 2021-11-16 Information processing device, control system, information processing method, and program

Publications (1)

Publication Number Publication Date
WO2023090193A1 true WO2023090193A1 (en) 2023-05-25

Family

ID=85330596

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/041433 WO2023090193A1 (en) 2021-11-16 2022-11-07 Information processing device, control system, information processing method, and program

Country Status (2)

Country Link
JP (1) JP7230155B1 (en)
WO (1) WO2023090193A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7386278B2 (en) * 2022-02-28 2023-11-24 株式会社日立製作所 Inspection equipment and method
JP7471025B1 (en) 2023-06-05 2024-04-19 株式会社コジマプラスチックス Performance Inspection System
CN116906839B (en) * 2023-09-14 2023-12-01 浙江英集动力科技有限公司 Safety intelligent monitoring and early warning method for thermodynamic pipeline integrating physical measurement and soft measurement

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020064286A (en) * 2018-10-12 2020-04-23 株式会社東陽テクニカ Abnormal sound detection system, device, method, and program
US20210256991A1 (en) * 2020-02-11 2021-08-19 Purdue Research Foundation System and methods for machine anomaly detection based on sound spectrogram images and neural networks

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020064286A (en) * 2018-10-12 2020-04-23 株式会社東陽テクニカ Abnormal sound detection system, device, method, and program
US20210256991A1 (en) * 2020-02-11 2021-08-19 Purdue Research Foundation System and methods for machine anomaly detection based on sound spectrogram images and neural networks

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ICHIKAWA SHUTO, KUWAHARA, NORIAKI. : "A method of recognizing abnormal heart sound using CNN-LSTM network.", IPSJ SIG TECHNICAL REPORT (ASD), vol. 17, no. 7, 28 February 2020 (2020-02-28), pages 1 - 7, XP093068309 *
ISHIDA, KENJI; OKOSHI, MASARU: "Sound Quality Evaluation of Machine Sound and Natural Sound Using Variable Sound Analysis", PROCEEDINGS OF 2010 AUTUMN MEETING OF THE ACOUSTICAL SOCIETY OF JAPAN; SUITA, JAPAN; SEPTEMBER 14-16, vol. 2010, 2010, pages 1001 - 1004, XP009546409 *

Also Published As

Publication number Publication date
JP2023073761A (en) 2023-05-26
JP7230155B1 (en) 2023-02-28

Similar Documents

Publication Publication Date Title
WO2023090193A1 (en) Information processing device, control system, information processing method, and program
US11568244B2 (en) Information processing method and apparatus
US10228994B2 (en) Information processing system, information processing method, and program
WO2017112591A1 (en) Machine fault detection based on a combination of sound capture and on spot feedback
US10637751B2 (en) Methods and systems for online monitoring using a variable data sampling rate
CN105528652A (en) Method and terminal for establishing prediction model
CN108509325A (en) System time-out time is dynamically determined method and apparatus
US20160357886A1 (en) System for analytic model development
US11322124B2 (en) Chord identification method and chord identification apparatus
US20230325292A1 (en) Anomaly detection using multiple detection models
CN112463301A (en) Container-based model training test tuning and deployment method and device
CN116680146A (en) Method and device for guaranteeing safe and reliable operation of application software
CN114860542A (en) Trend prediction model optimization method, trend prediction model optimization device, electronic device, and medium
US11600252B2 (en) Performance analysis method
CN110322587B (en) Evaluation recording method, device and equipment in driving process and storage medium
JP5701251B2 (en) Pronunciation determination device, pronunciation determination method, and program
CN112463957A (en) Abstract extraction method and device for unstructured text log stream
CN115185819A (en) System testing method, device, equipment and computer readable storage medium
KR102456407B1 (en) Systems and methods for analyzing application loading times
WO2022134001A1 (en) Machine learning model framework development method and system based on containerization technology
US20220358953A1 (en) Sound model generation device, sound model generation method, and recording medium
CN115552380A (en) Performance event troubleshooting system
CN115244614A (en) Parameter inference method, parameter inference system, and parameter inference program
Ang et al. Emotional prosody analysis on human voices
WO2021130997A1 (en) Model compression device, learning system, model compression method, and program recording medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22895475

Country of ref document: EP

Kind code of ref document: A1