JP7142530B2 - Information processing system, information processing method, and program - Google Patents

Description

The present invention relates to an information processing system, an information processing method, and a program.

Machine learning is conventionally known.
In such machine learning, learning data (teacher data) is generated by adding labels such as correct answer information to data. For example, Patent Literature 1 discloses a data processing device that includes a training data complementing unit and is capable of adding a training data candidate to training data by assigning a label to the training data candidate. there is Further, Patent Literature 2 discloses a data learning device that displays a message prompting a user to label.

In Patent Document 3, when an operation abnormality occurs, the operation sound of the device being recorded, the operation sound at the time of failure when a defect occurs in the moving part where the operation abnormality occurred this time, and the operation abnormality this time Disclosed is an equipment failure confirmation assisting device that enables a user to hear normal operation sounds when no failure occurs in a moving part.

Patent Literature 4 discloses an abnormality detection device intended to reduce false alarms due to work sounds similar to abnormal sounds that should be detected as abnormalities.

JP 2016-76073 A JP 2018-13857 A JP 2004-326505 A JP 2009-259020 A

By the way, when a system for monitoring the equipment is used to determine whether the sound emitted by the equipment is a normal sound or an abnormal sound, the following problems arise.

It is not possible to grasp in advance what kind of sound is an abnormal sound before operating the system. In other words, normally, the system can learn only normal sounds before the system is put into operation.

Therefore, even if the system determines that the sound emitted by the equipment is not normal sound, the sound is not necessarily abnormal sound. In other words, even if the sound emitted by the equipment is actually normal sound, the sound may be erroneously detected as not normal sound.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an information processing system and an information processing system capable of easily obtaining learning data for enabling highly accurate abnormal noise determination. The object is to provide a method and a program.

According to one aspect of the present invention, an information processing system uses a first sound collector for collecting sound emitted from a first facility and a first model for the first facility to generate a first sound. A first judgment means for judging whether or not the sound emitted from the equipment is a normal sound; and a data processing means for processing as second sound data used for. The data processing means divides the second sound data into first unit data for each unit time. The first determination means uses the first model to determine whether each first unit data is normal sound data. When the first unit data is not determined to be normal sound data by the first model, the information processing system stores the first sound data that is not determined to be normal sound data out of the first sound data stored in the storage device. A reproduction means for outputting a portion corresponding to one unit data from a speaker, and a first label indicating normal sound data and a second label indicating abnormal sound data according to a user operation. and labeling means for applying a label to the portion output from the speaker.

According to the above invention, it is possible to easily obtain learning data for enabling highly accurate abnormal noise determination.

1 illustrates an information processing system and equipment monitored by the information processing system; FIG. FIG. 4 is a flow chart for explaining the flow of overall processing in the information processing system; FIG. 4 is a flowchart for explaining the flow of preparatory work for collecting sound with the sound collector. FIG. 10 is a flowchart for explaining the flow of processing for generating a reference model that is a normal sound model; FIG. 10 is a flowchart for explaining the flow of sound monitoring processing using a reference model; FIG. 4 is a diagram for explaining processing of sound data obtained by a sound collecting device; It is a figure showing the example of the display screen displayed with a terminal device. FIG. 5 is a diagram for explaining an example of data displayed on the screen of the terminal device when a tab related to sound monitoring is selected; FIG. 10 is a diagram for explaining another example of data displayed on the screen of the terminal device when a tab related to sound monitoring is selected; FIG. 10 is a diagram for explaining still another example of data displayed on the screen of the terminal device when a tab related to sound monitoring is selected; FIG. 4 is a flowchart for explaining the flow of labeling processing executed in the information processing system; FIG. 10 is a flow diagram for explaining the flow of processing for generating an erroneous detection model; FIG. 10 is a flowchart for explaining the flow of processing for generating an anomaly model; It is a figure showing the screen displayed in a terminal device. It is a figure showing the other screen displayed in a terminal device. FIG. 10 is a flowchart for explaining the flow of sound monitoring processing using a reference model, an erroneous detection model, and an anomaly model; FIG. 10 is a diagram showing still another screen displayed on the terminal device; It is a figure for demonstrating the functional structure of an information processing system. It is a figure showing the typical example of the hardware constitutions of a server apparatus. It is a figure showing the typical example of the hardware constitutions of a terminal device. It is a figure showing the screen of life-and-death monitoring.

Hereinafter, an information processing system according to each embodiment of the present invention will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<A. System configuration>
FIG. 1 is a diagram showing an information processing system 1 and a facility 900 monitored by the information processing system 1. As shown in FIG.

Referring to FIG. 1 , an information processing system 1 includes a cloud system 100 that forms the core of sound monitoring, a terminal device 200 and a plurality of sound collectors 300 . Note that the number of terminal devices 200 is not limited to one.

The information processing system 1 uses the sound collector 300 and the cloud system 100 to monitor in real time whether the equipment 900 is making an abnormal sound. Note that one sound collector 300 collects sounds emitted by one piece of equipment 900 .

The cloud system 100 is composed of, for example, a server device 100A and a storage device 100B. The server device 100A is communicably connected to the terminal device 200 . The cloud system 100 may be composed of a single server device or a plurality of server devices. The server device 100A and the storage device 100B are connected so as to be able to communicate with each other. Other device configurations are not particularly limited as long as the cloud system 100 includes at least one server device.

The sound collector 300 is provided near equipment 900 (for example, mechanical equipment, piping equipment) to be monitored. Sound collector 300 includes a microphone and a data communication device. The data communication device transmits sound data (packet data) collected by the microphone to the server device 100A via the network NW. Typically, the data communication device transmits sound data to server device 100A in real time. The sound data is sequentially stored in the storage device 100B. The sound collector 300 may be an integrated microphone and data communication device.

The server device 100A sequentially analyzes the sound data accumulated in the storage device 100B for each facility 900, and stores the analysis results in the storage device 100B. When the server device 100A determines that an abnormal sound (abnormal sound) is generated as a result of the analysis, it notifies the terminal device 200 in real time that the abnormal sound is generated in the equipment 900 . Specifically, the server device 100A notifies the user of the terminal device 200 (maintenance/monitoring staff 950) of which facility 900 is generating the abnormal sound.

The server device 100A uses artificial intelligence (AI) to determine whether the sound emitted by the facility 900 is normal or abnormal. Typically, the server device 100A uses the learned model to determine whether the sound emitted by the facility 900 is normal sound or abnormal sound.

The terminal device 200 is configured to be able to display various data related to sounds collected from the server device 100A. The terminal device 200 can display various monitoring screens (see FIG. 7). Terminal device 200 typically uses a browser to display various information provided from server device 100A.

Note that the above determination processing by the server device 100A is not limited to artificial intelligence, and may be processing based on general machine learning. From this point of view, the processing of the information processing system 1 will be described below using terms such as machine learning and model, without using restrictive terms such as artificial intelligence and trained model.

FIG. 2 is a flowchart for explaining the flow of overall processing in the information processing system 1. As shown in FIG. Note that steps S1 to S11, which will be described later, are not automatically executed in sequence, but typically the server device 100A receives a predetermined instruction (user instruction to execute each step) based on user operation. It is executed by Also, a reference model M1, an erroneous detection model M2, and an anomaly model M3, which will be described below, are generated for each facility 900. FIG.

Referring to FIG. 2, in step S1, cloud system 100 generates a reference model M1 (normal sound model) by machine learning using sound data of normal sound emitted by facility 900 as learning data. In step S2, the cloud system 100 performs sound monitoring using the reference model M1. Specifically, the cloud system 100 uses the reference model M1 to determine whether or not the sound collected by the sound collector 300 is normal sound.

In step S3, a human actually hears the sound judged to be not normal by the reference model M1, and if the human judges that the sound is abnormal, the data of the sound is labeled as abnormal. will be When a human determines that the sound is normal (that is, when the reference model determines an erroneous detection), the sound data is labeled as a normal sound.

In step S4, the cloud system 100 uses the sound data labeled as normal sounds to generate the false detection model M2. If there is no sound data labeled as normal sound in step S3, the process of step S4 is not performed.

In step S5, the cloud system 100 uses the sound data labeled as the abnormal sound to generate the abnormal model M3. If there is no sound data labeled as abnormal sound in step S3, the process of step S5 is not performed.

In step S6, the cloud system 100 performs sound monitoring using the reference model M1, the false positive model M2, and the abnormal model M3. In step S7, the cloud system 100 updates the reference model M1 using the false positive model M2. In step S8, the cloud system 100 performs sound monitoring using the updated reference model M1 and the abnormal model M3.

In step S9, labeling processing similar to that in step S3 is performed. In step S10, the cloud system 100 updates the false detection model M2 using the sound data labeled as normal sound in step S9. In step S11, the cloud system 100 updates the abnormal model M3 using the sound data labeled as abnormal sound in step S9.

Note that the process of step S7 is not essential and may not be performed.
FIG. 3 is a flowchart for explaining the flow of preparatory work for collecting sound with the sound collector 300. As shown in FIG.

Referring to FIG. 3, in step S21, cloud system 100 synchronizes the time of the clock of the data communication device of sound collector 300, the time of the clock of equipment 900, and the time of the clock of server device 100A. accepts the operation of In step S<b>22 , the cloud system 100 also receives a user operation for adjusting the sound volume of the microphone of the sound collector 300 when sound is collected.

In step S23, the cloud system 100 receives a user operation for setting transfer of sound data from the data communication device of the sound collector 300 to the server device 100A.

After the above adjustments and settings are completed, the sound collector 300 performs data sampling of the collected sound in step S24. The sampling period can be, for example, 44.1 kHz. In step S25, the sound collector 300 transforms the sound data into a PCM (Pulse Code Modulation) sound source. The data communication device of the sound collector 300 transmits PCM sound data (packets) to the server device 100A. Thereafter, the processes of steps S24, S25 and S26 are repeated.

Below, according to the flow of the flow chart shown in FIG. Steps S3, S4, S5)", "Use of erroneous detection model M2 and anomaly model M3 (step S6)", and "Update of reference model M1 (step S7)" will be described in this order.

<B. Generation of Reference Model M1>
FIG. 4 is a flowchart for explaining the flow of processing for generating the reference model M1, which is a normal sound model. Typically, the reference model M1 is generated by the processor 151 (see FIG. 19) of the server device 100A using learning data (feature values, which will be described later) and a predetermined learning program.

Referring to FIG. 4, in step S31, server device 100A uses sound collecting device 300 to acquire sound data (sound data) emitted from equipment 900 when equipment 900 is operating normally. Start.

In step S32, the server device 100A preprocesses the acquired sound data. Specifically, the server device 100A divides the sound data into unit data Du for each unit time. For example, the server device 100A divides the sound data into unit data Du of Tu seconds (predetermined number of seconds less than or equal to 1 second). Furthermore, as preprocessing, noise removal, setting of a frequency band to be analyzed, application of a bandpass filter, and the like are performed.

In step S33, the server device 100A extracts feature amounts from each unit data Du using a predetermined algorithm. In step S34, the server device 100A performs machine learning using the extracted feature amount. By repeating such machine learning, the reference model M1 is generated.

Assuming that the sampling frequency is 44.1 kHz, the measurable frequency is half the sampling frequency, and the frequency resolution is 10 Hz, the number of dimensions of the feature amount (the number of dimensions of the input vector) is 2205.

<C. Use of Normative Model M1>
FIG. 5 is a flowchart for explaining the flow of sound monitoring processing using the reference model M1. Since the erroneous detection model M2 and the abnormality model M3 have not been generated at the time when the reference model M1 is generated, the information processing system 1 performs sound monitoring using only the reference model M1.

Referring to FIG. 5, in step S41, server device 100A uses sound collecting device 300 to obtain sound data (sound data) emitted from equipment 900 when equipment 900 is operating normally. Start.

In step S51, the server device 100A stores the acquired sound data in the storage device 100B as sound data for reproduction (raw data). Specifically, the server device 100A stores the sound data in the storage device 100B in association with the time information.

In step S42, the server apparatus 100A pre-processes the acquired sound data in the same manner as in step S32 of FIG. Specifically, the server device 100A divides the sound data into unit data Du for each unit time.

After step S42, the server device 100A (more specifically, the processor 151) executes each process of steps S43 to S50 for each unit data Du.

In step S43, the processor 151 extracts feature amounts from the unit data Du using the predetermined algorithm described above. In step S44, the processor 151 inputs the extracted feature amount to the reference model M1. At step S45, the processor 151 obtains the output from the reference model M1. Specifically, an error with respect to the reference model M1 (cluster) is calculated and the error is output.

At step S46, the processor 151 determines whether the calculated error is within a predetermined range R1. When processor 151 determines that the error is within range R1 (YES in step S46), processor 151 determines unit data Du to be normal sound data in step S47. It should be noted that the 3σ method, for example, may be used to determine whether or not it is within the range R1.

When processor 151 determines that the error is outside range R1 (NO in step S46), processor 151 tentatively determines unit data Du to be abnormal sound data in step S48. Next, in step S49, the processor 151 executes abnormality notification processing. Specifically, the server device 100</b>A issues a predetermined warning notification to the terminal device 200 . In this case, the terminal device 200 performs various processes such as warning display. In step S50, the processor 151 stores the determination result in the storage device 100B. The processing of steps S49 and S50 may be performed in parallel, or the processing of step S50 may precede the processing of step S49.

FIG. 6 is a diagram for explaining processing of sound data obtained by the sound collector 300. FIG.

Referring to FIG. 6, collected sound data is stored in storage device 100B as sound data for reproduction (hereinafter also referred to as "storage data Ds"), while sound data used for determination (hereinafter also referred to as sound data Ds) is , also referred to as “determination data Dd”). That is, the server device 100A holds sound data (see (A)) obtained by one sound collecting device 300 as storage data Ds (see (B)), and separately from the storage data Ds, It is also held as determination data Dd (see (C)). For example, the processor 151 may generate the determination data Dd by using the storage data Ds as original data and copying the original data. The storage data Ds is the data itself (raw data) of the sound collected by the sound collecting device 300, and is data that has not undergone processing for removing sounds such as noise.

The processor 151 divides the determination data Dd into unit data Du for each unit time Tu. The processor 151 extracts the feature amount for each unit data Du, as described above.

FIG. 7 is a diagram showing an example of a display screen displayed on the terminal device 200. As shown in FIG.
Referring to FIG. 7 , screen 201 includes multiple tabs 211 , 212 , 213 , 214 and 215 and dashboard 270 .

The dashboard 270 includes status monitoring items 271 , communication log items 272 , tool items 273 , report items 274 , and alert items 275 .

When the user of the terminal device 200 selects the tab 211, a life-and-death monitoring screen is displayed. Alive monitoring is performed by monitoring “ping” from the sound collector 300 . When the user of the terminal device 200 selects the tab 212, a trap monitoring screen is displayed. When the user of the terminal device 200 selects the tab 213, a Syslog monitoring screen is displayed. When the user of terminal device 200 selects tab 214, a protocol monitoring screen is displayed. When the user of the terminal device 200 selects the tab 215, a sound monitoring screen is displayed.

FIG. 8 is a diagram for explaining an example of data displayed on the screen of the terminal device 200 when the tab 215 relating to sound monitoring is selected. Referring to FIG. 8, terminal device 200 displays sound waveform 221 in association with time.

FIG. 9 is a diagram for explaining another example of data displayed on the screen of terminal device 200 when tab 215 relating to sound monitoring is selected. Referring to FIG. 9 , terminal device 200 displays spectrum 222 .

FIG. 10 is a diagram for explaining still another example of data displayed on the screen of terminal device 200 when tab 215 relating to sound monitoring is selected. Referring to FIG. 10, terminal device 200 displays graph 223 showing the output of reference model M1 (denoted as "Loss") and the normal range of the output.

Waveform 221, spectrum 222, and graph 223 can be utilized in generating false positive model M2 and anomaly model M3.

When the processor 151 of the server device 100A determines that the unit data Du is abnormal sound data, the terminal device 200 displays a warning on the screen. For example, terminal device 200 may perform warning display by superimposing an object image showing warning display on screen 201 (see FIG. 7).

In addition, the terminal device 200 or the server device 100A emits light from an indicator lamp (for example, a stacked indicator lamp) installed near the terminal device 200, so that the user (maintenance/monitor 950) of the terminal device 200 can may be notified of the occurrence of an abnormality. Also, the server device 100A may send a warning mail to a predetermined address.

<D. Generation of False Positive Model M2 and Abnormal Model M3>
First, labeling processing to data will be described as preprocessing for generating the false positive model M2 and the abnormal model M3. Next, generation of the false positive model M2 and the abnormal model M3 will be described.

FIG. 11 is a flowchart for explaining the flow of labeling processing executed in the information processing system 1. As shown in FIG.

Referring to FIG. 11, in step S61, processor 251 of terminal device 200 selects a portion ("partial data Dp ”) is reproduced. Specifically, the processor 251 outputs the partial data Dp of the storage data Ds from the speaker. The terminal device 200 reproduces the partial data Dp by acquiring the partial data Dp from the server device 100A.

More specifically, based on the time information of the unit data Du determined as abnormal sound data, the server device 100A extracts the partial data Dp of the time zone indicated by the time information from the storage data Ds. The server device 100</b>A transmits the extracted partial data Dp to the terminal device 200 . The terminal device 200 reproduces the received partial data Dp.

For example, when there are a plurality of unit data Du tentatively determined to be abnormal sound data consecutively in time, the terminal device 200 may reproduce the partial data Dp corresponding to the plurality of unit data Du. . Typically, when the unit data Du temporarily determined to be abnormal sound data exists in a threshold value or more within a predetermined time (for example, within 10 seconds), the terminal device 200 controls all or a specified part of this period. (Fig. 15). Specifically, the terminal device 200 determines a section (period) to be reproduced by user operation, and reproduces the partial data Dp included in the determined section.

The user of the terminal device 200 listens to the sound output by the reproduction process in step S61 to determine whether or not the sound determined to be abnormal by the reference model M1 is really abnormal. In other words, the user himself/herself determines whether the sound was abnormal or erroneous detection (normal sound).

In step S62, the server device 100A determines whether or not the terminal device 200 has received a user operation to select the object 231 (see FIG. 15) indicating normal sound data (erroneous detection data). Specifically, server device 100A determines whether or not a signal indicating that object 231 has been selected has been received from terminal device 200 .

If object 231 is selected (YES in step S62), in step S63 server device 100A labels partial data Dp as normal sound, and returns partial data Dp labeled as normal sound. Stored in the storage device 100B.

If object 231 is not selected (NO in step S62), server device 100A causes terminal device 200 to cause terminal device 200 to perform a user operation to select object 232 (see FIG. 15) indicating abnormal sound data in step S64. Determine whether or not it has been accepted. Specifically, server device 100A determines whether or not a signal indicating that object 232 has been selected has been received from terminal device 200 .

If object 232 is selected (YES in step S64), in step S65 server device 100A labels partial data Dp as abnormal sound, and returns partial data Dp labeled as abnormal sound. Stored in the storage device 100B.

If object 232 is not selected (NO in step S64), terminal device 200 determines in step S66 whether or not an operation for repeatedly reproducing partial data Dp has been received. When terminal device 200 determines that the operation has been received (YES in step S66), the process proceeds to step S61. When terminal device 200 determines that the operation has not been received (NO in step S66), the process proceeds to step S62.

As described above, labeling of the reproduced partial data Dp (partial data Dp corresponding to the unit data Du temporarily determined to be abnormal sound) indicating normal sound or abnormal sound is completed.

By the way, in the process of FIG. 11, the case where the user of the terminal device 200 performs labeling only by the reproduction process has been described as an example, but the present invention is not limited to this.

If the user cannot determine whether the sound is normal or abnormal just by confirming the reproduced sound, he or she should select at least one of the waveform 221 shown in FIG. 8, the spectrum 222 shown in FIG. 9, and the graph 223 shown in FIG. can be displayed on the screen. In this case, the user can determine whether the sound judged to be abnormal by the reference model M1 is really an abnormal sound or a normal sound by confirming the reproduced sound and the screen information. Also in this case, the partial data Dp is labeled based on the determination result.

Note that FIG. 11 shows an example of a configuration in which a label of normal sound data or a label of abnormal sound data is added to each partial data Dp corresponding to each of all unit data Du tentatively determined to be abnormal sound data. However, it is not limited to this. Only the partial data Dp specified (selected) by the user may be labeled (FIGS. 14 and 15). In other words, it is not always necessary for the user to reproduce and listen to all the partial data Dp corresponding to all the unit data Du temporarily determined as abnormal sound data.

FIG. 12 is a flowchart for explaining the flow of processing for generating the false positive model M2.
Referring to FIG. 12, in step S71, processor 151 of server device 100A reads partial data Dp labeled as normal sound data from storage device 100B. In step S72, the processor 151 extracts feature amounts from the read partial data Dp using a predetermined algorithm. In step S73, the server device 100A performs machine learning using the extracted feature amount.

In step S74, the processor 151 determines whether or not all the partial data Dp labeled as normal sound data have been read. Partial data Dp is data relating to the same equipment 900 .

When processor 151 determines that all partial data Dp has not been read (NO in step S74), the process proceeds to step S72. When processor 151 determines that all partial data Dp have been read (YES in step S74), the series of processing ends.

The server device 100A repeats such machine learning to generate the false positive model M2.

FIG. 13 is a flowchart for explaining the flow of processing for generating the abnormal model M3.
Referring to FIG. 13, in step S81, processor 151 of server device 100A reads partial data Dp labeled with abnormal sound data from storage device 100B. In step S82, the processor 151 extracts feature amounts from the read partial data Dp using a predetermined algorithm. In step S83, the server device 100A performs machine learning using the extracted feature amount.

In step S84, the processor 151 determines whether or not all partial data Dp labeled with abnormal sound data have been read. Partial data Dp is data relating to the same equipment 900 .

When processor 151 determines that all partial data Dp have not been read (NO in step S84), the process proceeds to step S82. When processor 151 determines that all partial data Dp has been read (YES in step S84), the series of processes ends.

The server device 100A repeats such machine learning to generate the abnormality model M3.

In the following, instead of labeling each of the partial data Dp corresponding to one unit data Du, partial data Dp corresponding to all continuous unit data within a predetermined time (hereinafter, 10 seconds) An example of labeling (10 seconds of sound data) will be described. In other words, an example will be described in which the partial data Dp for 10 seconds is labeled instead of the time unit (one second or less) of division in the preprocessing.

FIG. 14 is a diagram showing a screen 202 displayed on the terminal device 200. As shown in FIG.
Referring to FIG. 14, dashboard 270 further includes an item 276 for sound monitoring. The items 276 include a device list item 2761 , a determination result item 2762 , a voice learning item 2763 , and an erroneously detected sound learning item 2764 . Items 2761 to 2764 are displayed by expanding the item 276 based on the user's operation.

The alert item 275 includes an abnormal noise history item 2751 . When the abnormal noise history item 2751 is selected by a user operation, a list of abnormal noise histories is displayed as illustrated. The abnormal sound history list includes a plurality of records. Each record includes at least device name, model, date and time, and evaluation value as fields. Each record is associated with partial data Dp.

In this example, in the abnormal sound history list, the partial data Dp corresponding to all the unit data Du temporarily judged to be abnormal sound data are not associated as records, but are temporarily judged to be abnormal sound data. is associated with partial data Dp corresponding to a period in which the unit data Du is concentrated at a certain rate or more. Specifically, each record is associated with partial data Dp corresponding to a period in which unit data Du provisionally determined to be abnormal sound data is greater than or equal to a threshold value within a predetermined time (within 10 seconds).

For example, if the time unit (unit time Tu) for division in preprocessing is 0.5 seconds, 20 units of data Du are included in 10 seconds. If, for example, 12 or more of the 20 units of data Du are tentatively determined to be abnormal data, the partial data Dp for 10 seconds are managed as one record. Also, as described above, the partial data Dp for 10 seconds are labeled.

FIG. 15 is a diagram showing a screen 203 displayed on the terminal device 200. As shown in FIG.
Referring to FIG. 15, when one record is selected from the abnormal noise history list of FIG. 14 by user operation, a determination result screen corresponding to the selected record is displayed as shown. be.

The terminal device 200 displays a determination graph corresponding to the device, model, date and time corresponding to the selected record (see FIG. 14). The judgment graph shows the waveform of the partial data Dp within a predetermined time (within 10 seconds). A section can be selected on the time axis (horizontal axis) by a user operation. Terminal device 200 can reproduce the sound of only the selected section (for example, section 235). Selection of the section is performed by a pointer device such as a mouse, for example.

The user uses the button 236 or the like to cause the terminal device 200 to reproduce the displayed partial data Dp.

When an erroneously detected object 231 (specifically, a button) is selected by a user operation, labeling of a normal sound to the displayed partial data Dp (data for 10 seconds in this example) is performed by the server device. 100A.

When an abnormal object 232 (specifically, a button) is selected by a user operation, labeling of an abnormal sound to the displayed partial data Dp (data for 10 seconds in this example) is performed by the server device 100A. is executed in

Note that in the determination graph, when a part of the partial data Dp is specified and the objects 231 and 232 are selected, the server device 100A determines whether the selected partial data Dp in the specified time domain is Labeling according to the objects 231 and 232 may be performed.

Further, in the information processing system 1, when the partial data Dp is labeled by selecting the objects 231 and 232, the reason for judging the partial data Dp as normal sound data or abnormal sound data can be written. Specifically, the reason can be written by the user selecting the object 234 . The written reason is transmitted to the server device 100A in association with the labeled partial data Dp.

According to such a labeling method, it is possible to easily obtain learning data (labeled partial data Dp) for enabling highly accurate abnormal noise determination. Note that the labeling method described above is an example, and is not limited to the configuration using the screen 203 described above. Labeling may be done by user's voice.

<E. Use of False Detection Model M2 and Abnormal Model M3>
FIG. 16 is a flowchart for explaining the flow of sound monitoring processing using the reference model M1, the false positive model M2, and the abnormal model M3.

Referring to FIG. 16, in step S101, server device 100A uses sound collecting device 300 to obtain sound data (sound data) emitted from equipment 900 when equipment 900 is operating normally. Start.

In step S115, the server device 100A stores the acquired sound data in the storage device 100B as sound data for reproduction (raw data). Specifically, the server device 100A stores the sound data in the storage device 100B in association with the time information.

In step S102, the server device 100A pre-processes the acquired sound data in the same manner as in step S42 of FIG. Specifically, the server device 100A divides the sound data into unit data Du for each unit time.

After step S102, the server device 100A (more specifically, the processor 151) executes each process of steps S103 to S114 for each unit data Du.

In step S103, the processor 151 extracts feature amounts from the unit data Du using the predetermined algorithm described above. In step S104, the processor 151 inputs the extracted feature amount to the reference model M1. At step S105, the processor 151 obtains the output from the reference model M1. Specifically, an error with respect to the reference model M1 (cluster) is calculated and the error is output.

At step S106, the processor 151 determines whether the calculated error is within a predetermined range R1. When processor 151 determines that the error is within range R1 (YES in step S106), processor 151 determines unit data Du to be normal sound data in step S107.

When processor 151 determines that the error is outside range R1 (NO in step S106), in step S108, processor 151 determines whether or not the extracted feature value matches false detection model M2. For example, when the erroneous detection model M2 is configured as a trained model, the output layer outputs a determination result as to whether or not there is a match.

If processor 151 determines that the feature quantity matches erroneous detection model M2 (YES in step S108), processor 151 advances the process to step S107. When processor 151 determines that the feature amount does not match false detection model M2 (NO in step S108), processor 151 determines whether the extracted feature amount matches abnormality model M3 in step S109. For example, when the abnormal model M3 is configured as a learned model, the output layer outputs a determination result as to whether or not there is a match.

When it is determined that the feature amount matches the abnormality model M3 (YES in step S109), processor 151 determines unit data Du to be abnormal sound data in step S110. Next, in step S111, the processor 151 executes abnormality notification processing.

When it is determined that the feature quantity does not match the abnormality model M3 (NO in step S109), the processor 151 tentatively determines the unit data Du to be abnormal sound data in step S112. Next, in step S113, the processor 151 executes abnormality notification processing. In step S114, the processor 151 stores the determination result in the storage device 100B.

<F. Normative model update>
FIG. 17 is a diagram showing a screen 204 displayed on the terminal device 200. As shown in FIG.

Referring to FIG. 17, when the user selects the false positive sound learning item 2764, a list of false positives is displayed as shown. The false detection list includes at least device name, model, date and time, evaluation value, and details. The details column contains an object indicating that labeling has been performed, and an object indicating which of the normal sound label and the abnormal sound label was assigned when labeling was performed. , is displayed for each record.

According to such a configuration, the user can easily determine whether or not the partial data Dp of each record has been labeled, and if so, which label has been given. .

<G. Functional configuration>
FIG. 18 is a diagram for explaining the functional configuration of the information processing system 1. As shown in FIG.

Referring to FIG. 18 , information processing system 1 includes cloud system 100 , terminal device 200 , and multiple sound collectors 300 . The cloud system 100 typically includes a server device 100A and a storage device 100B.

Each sound collector 300 collects sounds emitted from facilities near where each is installed. Each sound collecting device 300 transmits collected sound data to the cloud system 100 . The server device 100A of the cloud system 100 acquires the sound data.

(g1. Server device 100A)
The server device 100A includes a control unit 101 and a communication IF (Interface) 103 . The control unit 101 includes a data processing unit 111 , a model processing unit 112 and a sound determination unit 113 . The model processing unit 112 includes a model generating unit 1121 , a labeling unit 1122 and a model updating unit 1123 . Sound determination section 113 includes first determination section 1131 , second determination section 1132 , and third determination section 1133 .

The control unit 101 controls the overall operation of the server device 100A. Specifically, the control unit 101 controls the overall operation of the server device 100A by executing an operating system and application programs stored in a storage unit (not shown) or the like. Control unit 101 is typically implemented by processor 151 (see FIG. 19) executing these operating systems and application programs.

Communication IF 103 is an interface for communicating with an external device. The server device 100A communicates with each sound collector 300 and the terminal device 200 via the communication IF 103 . Also, the server device 100A communicates with the storage device 100B through a communication IF (not shown).

The data processing unit 111 causes the storage device 100B to store the sound collected by the sound collecting device 300 as storage data Ds, and processes the sound as determination data Dd used for determination. Specifically, the data processing unit 111 divides the determination data Dd into unit data Du for each unit time. The data processing section 111 sends the unit data Du to the sound determination section 113 .

The model generation unit 1121 of the model processing unit 112 acquires the normal sound unit data Du from the data processing unit 111 and generates the reference model M1 based on the unit data Du. The generated reference model M1 is sent to the first determination unit 1131 .

The model generation unit 1121 also generates an erroneous detection model M2 and an anomaly model M3. The generated false detection model M2 is sent to the second determination unit 1132 . The generated abnormal model M3 is sent to the third determination unit 1133 .

The false detection model M2 is generated using the partial data Dp labeled as normal sounds, as described above. The abnormal model M3 is generated using the partial data Dp labeled with the abnormal sound, as described above. Such labeling is performed by the labeling unit 1122 . The partial data Dp is stored in the storage device 100B as described above.

The labeling unit 1122 labels the partial data Dp based on an instruction from the terminal device 200 (objects 231 and 232 (see FIG. 15) by the user). The labeling unit 1122 assigns the label corresponding to the user's operation to the partial data Dp (more specifically, from the speaker 295 of the terminal device 200). Assigned to the output partial data Dp).

Typically, when the number of units of data Du not judged to be normal sound data for a predetermined time (predetermined unit period) is equal to or greater than a threshold, the labeling unit 1122 assigns , to assign a label according to the user's operation.

The model updating unit 1123 updates the reference model M1 using the erroneous detection data (that is, the partial data Dp labeled as normal sound). An update trigger is typically a user instruction at the terminal device 200 . Alternatively, the model update unit 1123 may automatically perform update processing after a predetermined time has elapsed since the previous update. Further, the model update unit 1123 may automatically perform update processing when the number of erroneously detected data is equal to or greater than a predetermined number. The model updating unit 1123 may make it a condition to obtain the user's permission from the terminal device 200 when performing automatic updating.

The sound determination unit 113 determines whether the collected sound is normal sound or abnormal sound.
The first determination unit 1131 has the reference model M1 described above. Specifically, the first determination unit 1131 has a reference model M1 for each facility 900 . The first determination unit 1131 uses the reference model M1 to determine whether or not the sound emitted from the target facility 900 is normal sound. Specifically, the first determination unit 1131 uses the reference model M1 to determine whether each unit data Du is normal sound data.

The second determination unit 1132 has the erroneous detection model M2 generated using the partial data Dp labeled as normal sound, as described above. Specifically, the second determination unit 1132 has an erroneous detection model M2 for each facility 900 . The false positive model M2 is generated after operation as described above. When the unit data Du is not determined to be normal sound data by the reference model M1, the second determination unit 1132 uses the false detection model M2 to determine whether or not the unit data Du is normal sound data.

The third determination unit 1133 has the abnormal model M3 generated using the partial data Dp labeled as abnormal sound, as described above. Specifically, the third determination unit 1133 has an abnormality model M3 for each facility 900 . The anomaly model M3 is generated after operation as described above. The third determination unit 1133 uses the abnormality model M3 to determine whether the unit data Du is abnormal sound data.

When the control unit 101 determines (including provisional determination) that the unit data Du is abnormal sound data, the control unit 101 issues a predetermined warning notification to the terminal device 200 .

(g2. Terminal device 200)
Terminal device 200 includes control unit 291 , input device 292 , communication IF 293 , display 258 , and speaker 295 .

The control unit 291 controls overall operations of the terminal device 200 . Specifically, the control unit 291 controls the overall operation of the terminal device 200 by executing an operating system and application programs stored in a storage unit (not shown) or the like of the terminal device 200 . Control unit 291 is typically implemented by processor 251 (see FIG. 20) executing these operating systems and application programs.

Communication IF 293 is an interface for communicating with an external device. The terminal device 200 communicates with the server device 100A through the communication IF 293 .

The input device 292 is a device that receives user operations. Input device 292 is typically a keyboard, mouse, or touch panel. The input device 292 sends the received input to the control section 291 . The input includes a selection operation for items displayed on each screen described above, input of data, and the like.

The display control unit 2911 displays various screens on the display. The display control unit 2911 switches screens to be displayed on the display 258 based on user operations using the input device 292 . The display control unit 2911, for example, displays screens 201 to 204 (see FIGS. 7, 14, 15, and 17), a waveform 221 (see FIG. 8), a spectrum 222 (see FIG. 9), and a graph 223 (see FIG. 10). is displayed on display 258 . Note that the waveform 221 includes the waveform of the unit data Du that has not been determined as normal sound data.

The reproduction processing unit 2912 reproduces the partial data Dp using a speaker based on the user's operation via the input device 292 . Specifically, when the unit data Du is not determined to be normal sound data by the reference model M1, the reproduction processing unit 2912 determines that the storage data Ds stored in the storage device 100B is not determined to be normal sound data. The partial data Dp corresponding to the unit data Du is output from the speaker 295 . Further, on the condition that the unit data Du is not determined to be normal sound data by the erroneous detection model M2, the reproduction processing unit 2912 determines that the save data Ds stored in the storage device 100B is normal sound data. The partial data Dp corresponding to the missing unit data Du is output from the speaker 295 .

If the unit data Du is not determined to be normal sound data by the first determination unit 1131 of the server device 100A, the control unit 291 makes a predetermined notification based on the warning notification from the server device 100A. Typically, control unit 291 outputs a predetermined warning sound from speaker 295 . Also, the control unit 291 displays a predetermined warning on the display 258 .

(g3. Labeling)
Next, regarding the aspect of the labeling process, the processes of the terminal device 200 and the server device 100A will be described in more detail.

The display control unit 2911 of the terminal device 200 causes the display 258 to display an operation screen (see FIG. 15) for outputting the partial data Dp corresponding to the unit data Du not determined as normal sound data from the speaker 295 . This operation screen has objects 231 and 232 .

When the object 231 is selected after the partial data Dp corresponding to the unit data Du is output from the speaker 295 by operating the operation screen, the terminal device 200 instructs the server device 100A to transmit the A notification indicating that the partial data Dp is normal sound data is transmitted. In this case, the label assigning unit 1122 of the server device 100A assigns a normal sound label to the partial data Dp on condition that the server device 100A has received the notification.

When the object 232 is selected after the partial data Dp corresponding to the unit data Du is output from the speaker 295 by the operation on the operation screen, the terminal device 200 instructs the server device 100A to transmit the relevant A notification indicating that the partial data Dp is abnormal sound data is transmitted. In this case, the label assigning unit 1122 of the server device 100A assigns an abnormal sound label to the partial data Dp on condition that the server device 100A has received the notification.

According to such a configuration, it is possible to easily obtain learning data (labeled partial data Dp) for enabling highly accurate abnormal noise determination.

<H. Hardware configuration>
FIG. 19 is a diagram showing a typical example of the hardware configuration of the server device 100A.

Referring to FIG. 19, server device 100A includes, as main components, processor 151 that executes a program, ROM 152 that stores data in a non-volatile manner, data generated by execution of the program by processor 151, or an input device. RAM 153 for volatilely storing data input via , HDD 154 for nonvolatilely storing data, communication IF 155 , operation keys 156 , power supply circuit 157 , and display 158 . Each component is connected to each other by a data bus. Communication IF 155 is an interface for communicating with other devices.

The processing in server device 100A is implemented by each piece of hardware and software executed by processor 151 . Such software may be stored in HDD 154 in advance. Software may also be stored in other storage media and distributed as program products. Alternatively, the software may be provided as a downloadable program product by a so-called Internet-connected information provider. Such software is temporarily stored in HDD 154 after being read from the storage medium by a reading device or downloaded via communication IF 155 or the like. The software is read from HDD 154 by processor 151 and stored in RAM 153 in the form of an executable program. Processor 151 executes the program.

Each component constituting the server device 100A shown in the figure is a general one. Therefore, it can be said that the essential part of the present invention is software stored in RAM 153, HDD 154, storage media, or software that can be downloaded via a network. Since the operation of each piece of hardware of server device 100A is well known, detailed description thereof will not be repeated.

FIG. 20 is a diagram showing a typical example of the hardware configuration of the terminal device 200. As shown in FIG.
Referring to FIG. 20, terminal device 200 includes, as main components, processor 251 that executes a program, ROM 252 that stores data in a non-volatile manner, data generated by execution of the program by processor 251, or an input device. RAM 253 volatilely stores data input via , HDD 254 nonvolatilely stores data, communication IF 255 , operation keys 256 , power supply circuit 257 , and display 258 . Each component is connected to each other by a data bus. Communication IF 255 is an interface for communicating with other devices.

Processing in the terminal device 200 is realized by each piece of hardware and software executed by the processor 251 . Such software may be stored in HDD 254 in advance. Software may also be stored in other storage media and distributed as program products. Alternatively, the software may be provided as a downloadable program product by a so-called Internet-connected information provider. Such software is temporarily stored in the HDD 254 after being read from the storage medium by a reading device or downloaded via the communication IF 255 or the like. The software is read from HDD 254 by processor 251 and stored in RAM 253 in the form of an executable program. Processor 251 executes the program.

Each component constituting the terminal device 200 shown in the figure is a general one. Therefore, it can be said that the essential part of the present invention is software stored in RAM 253, HDD 254, storage media, or software that can be downloaded via a network. Since the operation of each hardware of terminal device 200 is well known, detailed description will not be repeated.

<I. Other functions>
(1) FIG. 21 is a diagram showing a life-and-death monitoring screen. Referring to FIG. 21 , when item 2711 of life and death monitoring is selected, a life and death monitoring screen is displayed on display 258 of terminal device 200 . When the ping exceeds the threshold, the terminal device 200 notifies of life-and-death monitoring abnormality. For example, terminal device 200 displays item 2711 in a manner different from normal display. For example, terminal device 200 changes the color of the item.

Further, when a life-and-death monitoring abnormality is detected while the terminal device 200 is displaying the screen 201 shown in FIG. 7, the tab 211 may be displayed in a manner different from the normal mode.

(2) The number of input dimensions of the false positive model M2 and the abnormal model M3 is lower than the number of input dimensions of the reference model M1.

(3) Algorithms differ between the reference model M1, the false positive model M2, and the abnormal model M3. For example, the algorithms of the false positive model M2 and the anomaly model M3 can be SVM (Support Vector Machine).

(4) By integrating not only the erroneous detection model M2 but also the abnormality model M3 into the reference model M1, the server device 100A may determine whether or not the sound emitted by the facility 900 is abnormal.

(5) If the reference model M1 is not updated using the erroneous detection data within a predetermined period after the normal sound label is assigned, the server device 100A sends a predetermined notification to the terminal device 200. The information processing system 1 may be configured to allow

(6) The program uses the equipment model to determine whether or not the sound emitted from the equipment is a normal sound; and storing it in a storage device as second sound data used for the determination. The step of processing the second sound data includes dividing the second sound data into unit data for each unit time. The program uses the model to determine whether or not each unit data is normal sound data, and if the unit data is not determined to be normal sound data by the model, a step of outputting, from a speaker, a portion of the first sound data stored in the unit corresponding to the unit data that has not been determined as normal sound data; a first label indicating normal sound data and an abnormality; The processor further causes the processor to assign a second label indicating sound data and a label corresponding to a user's operation to the portion output from the speaker.

(7) The information processing system 1 includes the cloud system 100 and the terminal device 200. The information processing system 1 is configured so that the functions of the cloud system 100 and the terminal device 200 are executed by one information processing device. may be configured.

(8) In the above, the case where the division time unit (Tu seconds) in the preprocessing is 1 second or less has been described as an example, but it is not limited to this, and even if it is 1 second or more good.

The embodiments disclosed this time are examples, and are not limited to the above contents. The scope of the present invention is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope of equivalence to the scope of claims.

1 information processing system, 100 cloud system, 100A server device, 100B storage device, 101, 291 control unit, 111 data processing unit, 112 model processing unit, 113 sound determination unit, 151, 251 processor, 158, 258 display, 200 terminal Device, 201, 202, 203, 204 Screen, 211-215 Tab, 221 Waveform, 222 Spectrum, 223 Graph, 231, 232, 234 Object, 235 Time domain, 241, 242 Icon, 270 Dashboard, 271-276, 2711 , 2751, 2761 to 2764 items, 292 input device, 295 speaker, 300 sound collector, 900 equipment, 1121 model generation unit, 1122 label assignment unit, 1123 model update unit, 1131 first determination unit, 1132 second determination unit, 1133 third determination unit, 2911 display control unit, 2912 reproduction processing unit, Dd determination data, Dp partial data, Ds storage data, Du unit data, M1 reference model, M2 false positive model, M3 abnormal model, NW network.

Claims (15)

An information processing system,
a first sound collector for collecting sound emitted from the first equipment;
a first determination means for determining whether or not the sound emitted from the first equipment is a normal sound using the first model for the first equipment;
Data processing means for storing the sound collected by the first sound collecting device in a storage device as first sound data and processing it as second sound data used for the determination,
The data processing means partitions the second sound data into first unit data for each unit time,
The first determination means uses the first model to determine whether each of the first unit data is normal sound data,
The information processing system is
When the first unit data is not determined to be normal sound data by the first model, the first sound data among the first sound data stored in the storage device is not determined to be normal sound data. reproduction means for outputting a portion corresponding to one unit data from a speaker;
A label corresponding to a user's operation out of a first label indicating normal sound data and a second label indicating abnormal sound data is assigned to the portion output from the speaker. a granting means ;
When the first unit data is not determined to be normal sound data by the first model, using the second model generated using the portion to which the first label is assigned, the a second determination means for determining whether the first unit data is normal sound data;
On condition that the first unit data is not determined to be normal sound data by the second model, the reproducing means reproduces normal sound data out of the first sound data stored in the storage device. an information processing system that causes the speaker to output a portion corresponding to the first unit data that is not determined to be An information processing system,
a first sound collector for collecting sound emitted from the first equipment;
a first determination means for determining whether or not the sound emitted from the first equipment is a normal sound using the first model for the first equipment;
Data processing means for storing the sound collected by the first sound collecting device in a storage device as first sound data and processing it as second sound data used for the determination,
The data processing means partitions the second sound data into first unit data for each unit time,
The first determination means uses the first model to determine whether each of the first unit data is normal sound data,
The information processing system is
When the first unit data is not determined to be normal sound data by the first model, the first sound data among the first sound data stored in the storage device is not determined to be normal sound data. reproduction means for outputting a portion corresponding to one unit data from a speaker;
A label corresponding to a user's operation out of a first label indicating normal sound data and a second label indicating abnormal sound data is assigned to the portion output from the speaker. a granting means ;
An information processing system , further comprising update means for updating the first model using the portion to which the first label is assigned . An information processing system,
a first sound collector for collecting sound emitted from the first equipment;
a first determination means for determining whether or not the sound emitted from the first equipment is a normal sound using the first model for the first equipment;
Data processing means for storing the sound collected by the first sound collecting device in a storage device as first sound data and processing it as second sound data used for the determination,
The data processing means partitions the second sound data into first unit data for each unit time,
The first determination means uses the first model to determine whether each of the first unit data is normal sound data,
The information processing system is
When the first unit data is not determined to be normal sound data by the first model, the first sound data among the first sound data stored in the storage device is not determined to be normal sound data. reproduction means for outputting a portion corresponding to one unit data from a speaker;
A label corresponding to a user's operation out of a first label indicating normal sound data and a second label indicating abnormal sound data is assigned to the portion output from the speaker. further comprising a granting means ,
When the number of pieces of the first unit data not determined as normal sound data in a predetermined unit period exceeds a threshold value, the labeling means assigns a label to the portion corresponding to the unit period according to the user's operation. An information processing system that assigns labels . third determining means for determining whether or not the first unit data is abnormal sound data using a third model generated using the portion to which the second label is assigned; The information processing system according to any one of claims 1 to 3, comprising: 5. The information processing system according to any one of claims 1 to 4, wherein when said first unit data is not determined to be normal sound data by said first determining means, a predetermined notification is made. The information processing system further includes a server device and a terminal device communicably connected to the server device,
The server device includes the labeling means,
The terminal device
including the speaker, the reproduction means, and a display;
displaying on the display an operation screen for outputting from the speaker a portion corresponding to the first unit data that is not determined to be normal sound data;
The operation screen has a first object,
When the first object is selected after the portion corresponding to the first unit data is output from the speaker by an operation on the operation screen, the terminal device sends the portion to the server device. is normal sound data;
6. The labeling means according to any one of claims 1 to 5, wherein said labeling means gives said first label to said portion on condition that said server device has received said first notification. information processing system. The operation screen has a second object,
When the second object is selected after the portion corresponding to the first unit data is output from the speaker by an operation on the operation screen, the terminal device instructs the server device to transmit the portion is abnormal sound data;
7. The information processing system according to claim 6, wherein said label assigning means assigns said second label to said portion on condition that said server apparatus has received said second notification. 8. The information processing system according to claim 6, wherein said terminal device causes said display to display a waveform of said first unit data that has not been determined as normal sound data. further comprising a second sound collector that collects sound emitted from the second equipment,
The data processing means stores the sound collected by the second sound collecting device in a storage device as third sound data, processes the sound as fourth sound data, and processes the fourth sound data. Separated into second unit data for each unit time,
The first determination means uses a fourth model for the second equipment to determine whether each of the second unit data is normal sound data,
The reproducing means determines that the third sound data stored in the storage device is normal sound data when the second unit data is not determined to be normal sound data by the fourth model. outputting from a speaker a portion corresponding to the second unit data that has not been processed;
The label assigning means assigns a label corresponding to a user operation, out of the first label and the second label, to the portion corresponding to the second unit data output from the speaker. The information processing system according to claim 1 or 2 . An information processing method,
a step of collecting sound emitted from the equipment using a sound collecting device;
determining whether a sound emitted from the equipment is normal sound using a first model for the equipment;
storing the sound collected by the sound collecting device in a storage device as first sound data and processing it as second sound data used for the determination, and processing it as the second sound data. The step includes dividing the second sound data into unit data for each unit time;
The information processing method includes:
determining whether each unit data is normal sound data using the first model;
When the unit data is not determined as normal sound data by the first model, it corresponds to the unit data that is not determined as normal sound data among the first sound data stored in the storage device. a step of outputting the portion to be performed from a speaker;
a step of assigning to the portion output from the speaker a label corresponding to a user's operation, out of a first label indicating normal sound data and a second label indicating abnormal sound data; When,
When the unit data is not determined to be normal sound data by the first model, the unit data is generated using a second model generated using the portion to which the first label is assigned. a step of determining whether the sound data is normal;
The unit data that is not determined to be normal sound data among the first sound data stored in the storage device, provided that the unit data is not determined to be normal sound data by the second model. and outputting a portion corresponding to from the speaker . An information processing method,
a step of collecting sound emitted from the equipment using a sound collecting device;
determining whether the sound emitted from the equipment is a normal sound using the model for the equipment;
storing the sound collected by the sound collecting device in a storage device as first sound data and processing it as second sound data used for the determination, and processing it as the second sound data. The step includes dividing the second sound data into unit data for each unit time;
The information processing method includes:
determining whether each unit data is normal sound data using the model;
When the unit data is not determined to be normal sound data by the model, a part of the first sound data stored in the storage device corresponding to the unit data not determined to be normal sound data is a step of outputting from a speaker;
a step of assigning to the portion output from the speaker a label corresponding to a user's operation, out of a first label indicating normal sound data and a second label indicating abnormal sound data; When,
and updating the model using the portion to which the first label has been assigned . An information processing method,
a step of collecting sound emitted from the equipment using a sound collecting device;
determining whether the sound emitted from the equipment is a normal sound using the model for the equipment;
storing the sound collected by the sound collecting device in a storage device as first sound data and processing it as second sound data used for the determination, and processing it as the second sound data. The step includes dividing the second sound data into unit data for each unit time;
The information processing method includes:
determining whether each unit data is normal sound data using the model;
When the unit data is not determined to be normal sound data by the model, a part of the first sound data stored in the storage device corresponding to the unit data not determined to be normal sound data is a step of outputting from a speaker;
a step of assigning to the portion output from the speaker a label corresponding to a user's operation, out of a first label indicating normal sound data and a second label indicating abnormal sound data; When,
and adding a label corresponding to the user's operation to the portion corresponding to the unit period when the number of the unit data not determined as normal sound data in a predetermined unit period is equal to or greater than a threshold. , information processing methods. a program,
determining whether a sound emitted from the equipment is normal sound using a first model for the equipment;
storing the sound collected by the sound collecting device in a storage device as first sound data and processing the sound as second sound data used for the determination by a processor of the device;
The step of processing as the second sound data includes dividing the second sound data into unit data for each unit time,
Said program
determining whether each unit data is normal sound data using the first model;
When the unit data is not determined as normal sound data by the first model, it corresponds to the unit data that is not determined as normal sound data among the first sound data stored in the storage device. a step of outputting the portion to be performed from a speaker;
a step of assigning to the portion output from the speaker a label corresponding to a user's operation, out of a first label indicating normal sound data and a second label indicating abnormal sound data; When,
When the unit data is not determined to be normal sound data by the first model, the unit data is generated using a second model generated using the portion to which the first label is assigned. a step of determining whether the sound data is normal;
The unit data that is not determined to be normal sound data among the first sound data stored in the storage device, provided that the unit data is not determined to be normal sound data by the second model. and causing the processor to output a portion corresponding to from the speaker . a program,
determining whether the sound emitted from the equipment is a normal sound using a model for the equipment;
storing the sound collected by the sound collecting device in a storage device as first sound data and processing the sound as second sound data used for the determination by a processor of the device;
The step of processing as the second sound data includes dividing the second sound data into unit data for each unit time,
Said program
determining whether each unit data is normal sound data using the model;
When the unit data is not determined to be normal sound data by the model, a part of the first sound data stored in the storage device corresponding to the unit data not determined to be normal sound data is a step of outputting from a speaker;
a step of assigning to the portion output from the speaker a label corresponding to a user's operation, out of a first label indicating normal sound data and a second label indicating abnormal sound data; When,
and updating the model using the portion with the first label . a program,
determining whether the sound emitted from the equipment is a normal sound using a model for the equipment;
storing the sound collected by the sound collecting device in a storage device as first sound data and processing the sound as second sound data used for the determination by a processor of the device;
The step of processing as the second sound data includes dividing the second sound data into unit data for each unit time,
Said program
determining whether each unit data is normal sound data using the model;
When the unit data is not determined to be normal sound data by the model, a part of the first sound data stored in the storage device corresponding to the unit data not determined to be normal sound data is a step of outputting from a speaker;
a step of assigning to the portion output from the speaker a label corresponding to a user's operation, out of a first label indicating normal sound data and a second label indicating abnormal sound data; When,
the step of assigning a label according to the user's operation to the portion corresponding to the unit period when the number of the unit data not determined as normal sound data in a predetermined unit period is greater than or equal to a threshold ; A program that causes the processor to run further.

Images