JP2021170204A - Information processor, information processing method, and information processing program - Google Patents

Abstract

To provide an information processor, an information processing method, and an information processing program which are capable of determining whether or not an event is abnormal.SOLUTION: An information processor comprises: a reception unit for receiving image data; an acquisition unit for acquiring the degree of deviation between an event recorded in the image data and a first state based on a first learning model, on the basis of the first learning model in which a first arbitrary state of the event has been learned and the image data received by the reception unit; a classification unit for classifying events recorded in the image data into a plurality of states, on the basis of a second learning model in which at least a plurality of second states, different from the first state, of the event have been learned and the image data received by the reception unit; and an output control unit for outputting states of the events recorded in the image data on the basis of the degrees of deviation acquired by the acquisition unit and classification results of the classification unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

Conventionally, a neural network may be used when determining an abnormality of an event. The technique described in Patent Document 1 determines the occurrence of engine knocking (abnormal combustion) by utilizing the sound pressure of the engine and a neural network.

Japanese Patent No. 6605170

As described above, the feature amount of an event may be learned by using a neural network, and the abnormality of the event may be determined by the learning model which is the learning result. However, when determining the abnormality of the event based on the image data generated by imaging the event by the camera unit, the abnormality of the event cannot be determined only by the feature amount analysis which is the result of learning. In some cases. For this reason, there is a demand for a technique capable of determining whether or not an event is abnormal even for an event that could not be determined whether or not it is abnormal by the conventional technique.

An object of the present invention is to provide an information processing device, an information processing method, and an information processing program capable of determining whether or not an event is abnormal.

The information processing device of one aspect is recorded in image data based on a reception unit that receives image data, a first learning model that learns an arbitrary first state of an event, and image data received by the reception unit. Accepted by the acquisition unit that acquires the degree of deviation between the event and the first state based on the first learning model, the second learning model that learns at least a plurality of second states of the event, which is different from the first state, and the reception unit. Based on the image data, the classification unit that classifies the events recorded in the image data into multiple states, the degree of deviation acquired by the acquisition unit, and the result classified by the classification unit. It includes an output control unit that outputs the state of the event recorded in the image data from the output unit.

In one aspect of the information processing device, the acquisition unit acquires the degree of divergence by using the first learning model obtained by learning the moving image recording the first state of the event, and the classification unit is the event. Of the first state and the second state of the event, the event recorded in the image data is the second state by using the second learning model obtained by learning at least the moving image recording the second state of the event. It may be classified according to whether or not.

In the information processing device of one aspect, the output control unit may have a degree of deviation acquired by the acquisition unit of the first predetermined value or more and a second predetermined value classified by the classification unit. In the above case, the output unit may be controlled so as to output the information indicating the second state.

In one aspect of the information processing device, the reception unit receives the image data in which the flame is recorded, and the acquisition unit uses the first learning model in which the flame in the normal state is learned as the first state as an event to convert the image data. Obtaining whether the recorded flame deviates from the normal flame, the classification unit uses a second learning model in which the non-normal flame, which is different from the normal one, is learned as the second state as an event. , It may be possible to classify whether the flame recorded in the image data may become an unusual flame.

In one aspect of the information processing device, the reception unit receives image data recording the river flow, and the acquisition unit uses the first learning model that learns the normal river flow as the first state as an event. , Acquires whether the river flow recorded in the image data deviates from the normal river flow, and the classification unit determines whether the river flow is different from the normal river flow as an event. The second learning model learned by using the flow of the river as the second state may be used to classify whether the flow of the river recorded in the image data may become the flow at the abnormal time.

In the information processing device of one aspect, the receiving unit receives image data recording the state of mixing a plurality of materials, and the acquiring unit learns the mixed state of materials that is normal as an event as the first state. Is used to obtain whether the mixed state of the materials recorded in the image data deviates from the mixed state of the normal materials, and the classification unit determines the mixed state of the materials at the time of abnormality different from the normal time as an event. Using the second learning model learned as the second state, it may be possible to classify whether or not the mixed state of the materials recorded in the image data may become the mixed state at the time of abnormality.

In one aspect of the information processing apparatus, the reception unit receives image data recording the state of food during fermentation, and the acquisition unit uses a first learning model that learns the fermentation state of normal food as the first state as an event. By using it, it is obtained whether the fermentation state of the food recorded in the image data deviates from the normal fermentation state, and the classification unit secondly determines the fermentation state of the food during the abnormal state, which is different from the normal state, as an event. The second learning model learned as a state may be used to classify whether the fermentation state of the food recorded in the image data may be the fermentation state at the time of abnormal.

In one aspect of the information processing device, the reception unit receives image data recording the passage of one or more persons, and the acquisition unit learns the normal passage state of the person as the first state as an event. Using the model, it is acquired whether the traffic state of the person recorded in the image data deviates from the normal traffic state, and the classification unit sets the normal traffic state different from the normal one as an event. Using the second learning model learned as the two states, it may be possible to classify whether or not the passing state of the person recorded in the image data may become the passing state at the abnormal time.

In one aspect of the information processing device, the reception unit receives image data recording the state of water processed in the water treatment facility, and the acquisition unit sets the treatment state of water normally treated as an event as the first state. Using the learned first learning model, it is acquired whether the state of the treated water recorded in the image data deviates from the normal processing state, and the classification unit is abnormal as an event different from the normal processing state. Using the second learning model that trained the water treatment state of the time processing state as the second state, it is possible to check whether the state of the treated water recorded in the image data may become the processing state at the abnormal time. It may be classified.

In one aspect of the information processing device, the reception unit receives image data recording the state of smoke discharged from the facility, and the acquisition unit learns the state of smoke normally discharged as an event as the first state. 1 Using the learning model, it is acquired whether the smoke state recorded in the image data deviates from the first state, and the classification unit determines whether the smoke state at the time of abnormal smoke is different from the normal smoke state as an event. The second learning model learned as the second state may be used to classify whether the smoke state recorded in the image data may be the smoke state at the time of abnormal state.

In one aspect of the information processing device, the reception unit receives image data recording a vehicle traveling on the road, and the acquisition unit learns the traveling state of the vehicle normally traveling on the road as the first state as an event. Using the learning model, it is acquired whether the running state of the vehicle recorded in the image data deviates from the first state, and the classification unit determines whether the running state of the vehicle is different from the normal running state as an event. As a method of classifying whether the running state of the vehicle recorded in the image data may be the running state of the vehicle at the time of abnormal state by using the second learning model learned by using the running state of May be good.

In one aspect of the information processing device, the reception unit receives image data recording the movement of the person or animal, and the acquisition unit learns the normal operation state of the person or animal as the first state as an event. Using the model, it is obtained whether the motion state of the person or animal recorded in the image data deviates from the first state, and the classification unit is abnormal as an event, which is different from the normal movement of the person or animal. It may be possible to classify whether the motion state of a person or animal recorded in the image data becomes the abnormal motion state by using it as the second learning model in which the motion state at the time is learned as the second state.

In one aspect of the information processing method, the computer converts the image data into image data based on the reception step of receiving the image data, the first learning model in which the arbitrary first state of the event is learned, and the image data received by the reception step. An acquisition step for acquiring the degree of deviation between the recorded event and the first state based on the first learning model, a second learning model different from the first state, which has learned at least a plurality of second states of the event, and reception. Based on the image data received by the step, the classification step that classifies the events recorded in the image data into multiple states, the degree of divergence acquired by the acquisition step, and the result classified by the classification step. Based on this, the output control step of outputting the state of the event recorded in the image data from the output step is executed.

The information processing program of one aspect converts the image data into image data based on the reception function of receiving the image data in the computer, the first learning model in which the arbitrary first state of the event is learned, and the image data received by the reception function. An acquisition function that acquires the degree of deviation between the recorded event and the first state based on the first learning model, a second learning model that learns at least a plurality of second states of the event, which is different from the first state, and reception. Based on the image data received by the function, the classification function that classifies the events recorded in the image data into multiple states, the degree of divergence acquired by the acquisition function, and the result classified by the classification function. Based on this, the output control function that outputs the state of the event recorded in the image data from the output function is realized.

The information processing apparatus of one aspect includes a first learning model that learns an arbitrary first state of an event, an event recorded in the image data based on the image data, and a first state based on the first learning model. Based on the second learning model that acquired the degree of divergence and learned at least a plurality of second states of events, which are different from the first state, and the image data, the events recorded in the image data are classified into a plurality of states. Since the state of the event recorded in the image data is output based on the degree of divergence and the result of the classification, it is possible to determine whether or not the event is abnormal.
The information processing method and the information processing program of one aspect can exert the same effect as the information processing program of one aspect described above.

It is a block diagram for demonstrating the information processing apparatus which concerns on one Embodiment. It is a figure for demonstrating the step in generating the 1st learning model. It is a figure for demonstrating the step in generating the 2nd learning model. It is a figure for demonstrating an example of a state in which a river flow is normal. It is a figure for demonstrating an example of an abnormal state of a river flow. (A) shows an example in which the river is flooded, (B) shows an example in which an object is flowing in the river, and (C) shows an example immediately before the embankment collapses. It is a 1st flowchart for demonstrating the information processing method which concerns on one Embodiment. It is a 2nd flowchart for demonstrating the information processing method which concerns on one Embodiment.

Hereinafter, an embodiment of the present invention will be described.
Although the wording of "information" is used in this specification, the wording of "information" can be paraphrased as "data", and the wording of "data" can be paraphrased as "information".

FIG. 1 is a block diagram for explaining the information processing device 1 according to the embodiment.
The information processing device 1 outputs the state of an event based on a plurality of learning models. The information processing device 1 generates a first learning model by learning an arbitrary first state of an event. The information processing device 1 generates a second learning model by learning at least the second state of the event among the first state of the above-mentioned event and the second state different from the first state. As the second state, the information processing device 1 learns the state before the second state (not the first state, but the state before the second state (immediately before, etc.), etc.), and the second learning model. May be generated.

When the information processing device 1 receives the image data by the reception unit 12, it determines whether the state of the event recorded in the image data deviates from the first state based on the image data and the first learning model. ..
Further, the information processing apparatus 1 determines whether the state of the event recorded in the image data may be the second state based on the image data and the second learning model. In this case, the information processing apparatus 1 may, for example, classify the first state to the second state into a plurality of stages and determine which stage the state recorded in the image data is.
In the following, the normal state of the event will be illustrated as the first state of the event, and the abnormal state of the event will be illustrated as the second state of the event.

Next, the information processing device 1 will be described in detail.
The information processing device 1 includes a communication unit 16, a storage unit 17, a display unit 18, a reception unit 12, an acquisition unit 13, a classification unit 14, and an output control unit 15. The reception unit 12, the acquisition unit 13, the classification unit 14, and the output control unit 15 may be realized as one function of the control unit 11 (for example, an arithmetic processing unit or the like) of the information processing device 1. The communication unit 16, the storage unit 17, and the display unit 18 may be realized as an embodiment of the "output unit" of the present invention.

The communication unit 16 can send and receive information to and from a device (for example, a server, an external camera, a mobile terminal, etc.) (not shown) outside the information processing device 1.

The storage unit 17 is a device that stores various information and programs.

The display unit 18 is, for example, a device for displaying characters, images, and the like.

The reception unit 12 receives the image data. That is, the reception unit 12 receives image data via, for example, the communication unit 16. For example, the reception unit 12 is generated by image data stored in an external server (not shown) outside the information processing device 1 or by an external camera (not shown) outside the information processing device 1. The image data is received via the communication unit 16. Alternatively, the reception unit 12 receives the image data generated by the camera unit (not shown) arranged in the information processing device 1. The reception unit 12 may accept moving images as image data, or may accept a plurality of still images that are continuous in time or at predetermined time intervals.

The acquisition unit 13 determines the event recorded in the image data and the normal state based on the first learning model based on the first learning model in which the normal state of the event is learned and the image data received by the reception unit 12. Get the degree of divergence. An example of an event is a combustion state (flame state), a river state, a mixed state of a plurality of substances, a fermentation state of food, or a flow state of a person (passage state), as described in Examples described later. It may be an event other than these. When the event is in a combustion state (flame state), the information processing device 1 may detect, for example, the possibility of abnormal combustion or the like. When the event is in the state of a river, the information processing device 1 may detect, for example, the flooding of the river and the possibility of the embankment breaking. When the event is a mixed state of a plurality of substances, the information processing apparatus 1 may detect, for example, the possibility that the substances are not properly mixed. When the event is a fermented state of food, the information processing apparatus 1 may detect, for example, the possibility that the food is not fermented properly. When the event is in a state of flow of people (traffic state), the information processing device 1 may output, for example, the possibility that a person stays or falls.

The acquisition unit 13 may acquire the degree of divergence by using the first learning model obtained by learning the moving image recording the normal state of the event. Further, the acquisition unit 13 may calculate, for example, the possibility that the event is normal, and determine that the event is abnormal when the result of the calculation (for example, a numerical value) is low. Here, the moving image to be learned may be a plurality of still images acquired continuously in time, or a plurality of still images acquired at predetermined time intervals, in addition to the moving image itself.

FIG. 2 is a diagram for explaining steps in generating the first learning model.
For example, the control unit 11 learns the normal state of the event and generates the first learning model. The control unit 11 acquires a moving image (image data) 101 in which the normal state of the event is recorded (FIG. 2 (A)), learns the moving image (image data) 101 (FIG. 2 (B)), and first 1 A learning model is generated (Fig. 2 (C)). The control unit 11 may learn the normal state for each of a plurality of different time zones and generate a first learning model for each time zone. Further, the control unit 11 may learn a plurality of patterns of normal states for the event and generate a first learning model. The first learning model generated by the control unit 11 may be stored in the storage unit 17, for example. In this case, the acquisition unit 13 may acquire the first learning model from the storage unit 17 when acquiring the degree of divergence. Further, the first learning model generated by the control unit 11 may be stored in the acquisition unit 13, for example.
The first learning model is not limited to the example generated by the control unit 11 as described above, and may be generated by an external server (not shown) or the like. In this case, the acquisition unit 13 acquires the first learning model generated by the control unit 11.

Based on the first learning model and the image data, the acquisition unit 13 acquires a degree of deviation indicating how much the state of the event recorded in the image data deviates from the normal state. The acquisition unit 13 may acquire, for example, a numerical value indicating the degree of deviation.

The classification unit 14 classifies the events recorded in the image data into a plurality of states based on the second learning model in which at least a plurality of abnormal states of the events are learned and the image data received by the reception unit 12. do. In this case, the classification unit 14 uses a second learning model obtained by learning a moving image recording at least the abnormal state of the event among the normal state of the event and the abnormal state of the event, and uses the image. It may be classified according to whether or not the event recorded in the data is an abnormal state.

An example of the event is a combustion state (flame state), a river state, a mixed state of a plurality of substances, a fermentation state of food, as described in Examples described later, as in the case of the acquisition unit 13 described above. Alternatively, it may be a state of flow of people (traffic state), or may be an event other than these.
The moving image to be learned may be a plurality of still images acquired continuously in time, or a plurality of still images acquired at predetermined time intervals, in addition to the moving image itself.

FIG. 3 is a diagram for explaining a step in generating the second learning model.
The control unit 11 generates the second learning model based on the same event as the event when the first learning model is generated. For example, the control unit 11 learns at least the abnormal state among the normal state and the abnormal state related to the above-mentioned event to generate the second learning model. In this case, it is preferable that the control unit 11 learns at least a plurality of patterns of abnormal states to generate a second learning model. That is, the control unit 11 uses moving images (moving images (1) to (3) shown in FIG. 3A as an example) (image data) 102a to 102c in which at least a plurality of patterns of abnormal states of events are recorded. It is acquired (FIG. 3 (A)), the moving image (image data) 102a to 102c is trained (FIG. 3 (B)), and a second learning model is generated (FIG. 3 (C)). As an abnormal state, the control unit 11 may learn a state before the abnormal state (a state that is not a normal state but before the abnormal state, etc.). For example, the control unit 11 may learn the state at the time of transition from the normal state to the abnormal state. As a specific example, the control unit 11 sets a predetermined time before (time before the reference time) when the abnormal state occurs when transitioning from the normal state to the abnormal state as a reference (reference time). It may be possible to learn the state of (within a predetermined time). Various times are set in the predetermined time according to the event.

For example, the classification unit 14 may classify from a normal state to an abnormal state into a plurality of stages, and classify the state recorded in the image data into any stage.
Further, the classification unit 14 may indicate, for example, the classification result numerically. For example, the classification unit 14 sets the normal state as "0", sets the abnormal state as "100", and sets any numerical value from "0" to "100" as the classification indicating whether or not it is abnormal. It may be output.
Further, for example, the classification unit 14 sets the normal state as "0", sets the abnormal state as "1", and outputs either "0" or "1" as the classification indicating whether or not the state is abnormal. May be.

The output control unit 15 determines the state of the event (whether or not the state of the event is abnormal) recorded in the image data based on the degree of deviation acquired by the acquisition unit 13 and the result classified by the classification unit 14. Output from the output section. The content output from the output unit may be, for example, a numerical value. As an example, the output control unit 15 sets the normal state to "0", sets the abnormal state to "100", and sets the event state recorded in the image data to any of "0" to "100". The numerical value may be output.

Here, the output control unit 15 has a degree of deviation acquired by the acquisition unit 13 of the first predetermined value or more, and the possibility of an abnormal state classified by the classification unit 14 is greater than or equal to the second predetermined value. In this case, the output unit may be controlled so as to output information indicating an abnormal state. The first predetermined value and the second predetermined value are appropriately set according to the event.
As described above, the output unit may be the communication unit 16, the storage unit 17, and the display unit 18. That is, the output control unit 15 provides the display unit 18 with information indicating the state of the event (whether or not the state of the event is abnormal) recorded in the image data or information indicating that the event is in the abnormal state. It may be displayed. The output control unit 15 stores in the storage unit 17 information indicating the state of the event (whether or not the state of the event is abnormal) recorded in the image data, or information indicating that the event is in the abnormal state. It may be that. The output control unit 15 externally (for example, an external server) provides information indicating the state of the event (whether or not the state of the event is abnormal) recorded in the image data or information indicating that the event is in the abnormal state. And the communication unit 16 may be controlled so as to transmit to a mobile terminal or the like (not shown).
The mobile terminal may be a terminal owned by a company, a national institution or a local public body, or a terminal owned by a citizen. Since the output control unit 15 has set the transmission destination of the information indicating the abnormal state according to the event, the output control unit 15 appropriately transmits the information to the mobile terminal according to the setting.

Next, a specific example (event) in which the above-mentioned information processing device 1 is used will be described.

[Example 1]
The reception unit 12 receives the image data in which the flame is recorded. For example, the reception unit 12 receives image data generated by imaging a flame when a substance burns. As an example, the reception unit 12 receives image data generated by capturing an image of the current flame state by a camera unit (not shown).

The acquisition unit 13 acquires whether or not the flame recorded in the image data deviates from the normal flame by using the first learning model in which the normal flame is learned as a normal state as an event. For example, when the image data is a moving image, the acquisition unit 13 divides a plurality of frames constituting the image data (moving image) into a predetermined number of frames (frame groups), and divides the frame group into a first learning model. Based on this, it is sequentially acquired whether or not the flame recorded in the frame group deviates from the normal flame.

The number of frames (predetermined number) constituting the frame group can be appropriately set. If the number of frames is relatively small, the influence of noise becomes large. If the number of frames is relatively large, the influence of the true abnormality of the flame is weakened, and the degree of deviation may not be discernible. In the determination of the flame when a substance burns in a specific example scene, it has been experimentally sought that it is optimal to divide into a frame group having 15 frames per second. However, in the acquisition unit 13, for example, the number of frames constituting the frame group is appropriately set for each specific case when the substance is burned, and based on the frame rate and the like. The number of frames constituting the frame group is appropriately set for each event of the embodiment described later.

In the above case, the example in which the image data is a moving image has been described, but even if the image data is a plurality of still images, the acquisition unit 13 can process in the same manner as described above.

The normal flame (normal state) is, for example, the state of the flame when there is no abnormal combustion. An abnormal flame (abnormal state) is, for example, a flame state at the time of abnormal combustion, the flame is swaying more than the normal time, the flame is darker (or brighter) than the normal time, and The color of the flame is different from that of the normal one.

The classification unit 14 may use the second learning model in which the flame in the abnormal time different from the normal time is learned as an abnormal state as an event, and the flame recorded in the image data may become the flame in the abnormal time. Classify if there is. The classification unit 14 is a flame recorded in the image data based on the second learning model in which at least the abnormal state is learned among the normal state and the abnormal state of the flame when the substance is burned, and the image data. Classify if is likely to be in an abnormal state before the present. The "abnormal state" of the flame learned when the second learning model is generated includes the transition state at the time of transition from the normal state to the abnormal state and the transition state immediately before the abnormal state. It may be.
Similar to the acquisition unit 13 described above, the classification unit 14 divides a plurality of frames constituting the image data (moving image) into a predetermined number of frames (frame groups), for example, when the image data is a moving image. Then, based on the frame group and the second learning model, it may be possible to sequentially classify whether or not the flame recorded in the frame group may be in an abnormal time. The number of frames constituting the frame group is appropriately set for each of the events of this embodiment and the events of the examples described later. Even if the image data is a plurality of still images, the classification unit 14 can process them in the same manner as described above.

The output control unit 15 determines whether or not the flame state is abnormal based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14 (the possibility that the flame state becomes abnormal at a time ahead). Whether or not there is) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 controls whether the substance may burn abnormally (whether the state of the flame may become abnormal) to be output from the output unit.

[Example 2]
FIG. 4 is a diagram for explaining an example of a state in which the flow of a river is normal.
FIG. 5 is a diagram for explaining an example of a state in which the flow of a river is abnormal. FIG. 5 (A) shows an example in which the river is flooded, FIG. 5 (B) shows an example in which an object (here, "tree") is flowing in the river, and FIG. 5 (C) shows an example in which the embankment breaks. The previous example is shown.

The reception unit 12 receives image data recording the flow of the river. The reception unit 12 receives, for example, image data generated by a camera unit (not shown) that monitors a river. As an example, the reception unit 12 receives the image data generated by capturing the current river flow with the camera unit.

The acquisition unit 13 uses the first learning model in which the normal river flow is learned as an event in a normal state, and the river flow recorded in the image data deviates from the normal river flow. To get. The acquisition unit 13 is the state of the river 201 excluding abnormal states such as flooding, flooding, objects flowing in the river, and the embankment breaking as the normal state of the river flow. The state of the river recorded in the image data (for example, the current state of the river) is normal based on the first learning model learned (see FIG. 4) and the flow of the river recorded in the image data. Get if it deviates from.

The classification unit 14 uses a second learning model in which the flow of the river before the flow of the river during the abnormal time, which is different from the normal time, is learned as an abnormal state as an event, and the classification unit 14 uses the second learning model to record the river in the image data. Classify if the flow can be an unusual flow. In the classification unit 14, the river flow recorded in the image data is currently recorded based on the second learning model in which at least the abnormal state is learned among the normal state and the abnormal state of the river flow and the image data. Classify if there is a possibility of an abnormal condition at a point earlier than that. The "abnormal state" of the river flow learned when the second learning model is generated includes the transition state that transitions from the normal state to the abnormal state and the transition state immediately before the abnormal state. It may be. As a transition state immediately before the river flow becomes abnormal, for example, the river 201 is flooded (see FIG. 5 (A)), and the object 202 accompanying the flood is flowing in the river 201 (FIG. 5 (Fig. 5)). B)), the state immediately before the embankment 203 collapses (see FIG. 5C), and the transition state immediately before the river overflows.

The output control unit 15 determines whether or not the river condition is abnormal based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14 (the river condition may become abnormal at a time ahead). Whether or not there is) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 outputs from the output unit whether the river may overflow and whether the embankment may collapse (whether the river condition may become abnormal). Control.

[Example 3]
The reception unit 12 receives image data recording a state in which a plurality of materials are mixed. As an example, the reception unit 12 receives image data generated by capturing an image of the current mixed state by a camera unit (not shown).

The acquisition unit 13 uses the first learning model that learns the normal material mixing state as an event to acquire whether the material mixing state recorded in the image data deviates from the normal material mixing state. .. The acquisition unit 13 mixes, for example, as a normal state of the mixed state, excluding abnormal states such as separation of a plurality of materials, bias in the plurality of materials, and lumps of materials. Based on the first learning model that learned the state and the mixed state recorded in the image data, it is acquired whether the mixed state recorded in the image data (for example, the current mixed state) deviates from the normal state. do.

The classification unit 14 can use the second learning model that learns the mixed state of the material at the time of abnormality different from the normal time as an event, and the mixed state of the material recorded in the image data can be changed to the mixed state at the time of abnormality. Classify if there is sex. The classification unit 14 records a mixed state recorded in the image data based on the second learning model in which at least the abnormal state is learned among the normal state and the abnormal state in which the mixed state of the plurality of materials is learned, and the image data. Classify if is likely to be in an abnormal state before the present. The "abnormal state" of the mixture of a plurality of materials learned when the second learning model is generated includes a transition state that transitions from a normal state to an abnormal state and a transition state immediately before the abnormal state. May be included.

The output control unit 15 determines whether or not the mixed state of the plurality of materials is abnormal based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14 (the mixed state becomes abnormal at a time point ahead). Whether or not there is a possibility) is controlled to be output from the above-mentioned output unit. For example, in the output control unit 15, is there a possibility that a plurality of materials are separated, a material is biased, or a lump of material is formed (mixed state)? Is possible to become abnormal) is controlled to be output from the output section.

[Example 4]
The reception unit 12 receives image data recording the state of food during fermentation. As an example, the reception unit 12 receives image data generated by taking an image of the currently fermented food by a camera unit (not shown).

The acquisition unit 13 uses the first learning model in which the fermentation state of the normal food is learned as an event to acquire whether the fermentation state of the food recorded in the image data deviates from the normal fermentation state. The acquisition unit 13 includes a first learning model in which the normal fermentation state of the food is learned, for example, the fermentation state excluding the abnormal state such as the food being separated and impurities being emitted from the food. Based on the food recorded in the image data, it is acquired whether the fermented state of the food recorded in the image data (for example, the fermented state of the current food) deviates from the normal state.

The classification unit 14 uses a second learning model in which the fermentation state of the food during the abnormal state different from the normal state is learned as an event, and the fermentation state of the food recorded in the image data is changed to the fermentation state during the abnormal state. Classify what is likely to be. The classification unit 14 determines the fermentation state of the food recorded in the image data based on the second learning model in which at least the abnormal fermentation state is learned among the normal fermentation state and the abnormal fermentation state of the food and the image data. Classify if is likely to be in an abnormal state before the present. The "abnormal fermentation state" of the food learned when the second learning model is generated includes a transition state in which the normal fermentation state changes to the abnormal fermentation state and a transition state immediately before the abnormal fermentation state. May be included.

Based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14, the output control unit 15 determines whether or not the fermentation state of the food is abnormal (the fermentation state of the food becomes abnormal at a time ahead). Whether or not there is a possibility) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 outputs from the output unit whether the food may be separated and whether impurities may be emitted from the food (whether the fermentation state of the food may be abnormal). Control to do.

[Example 5]
The reception unit 12 receives image data recording the passage of one or more persons. As an example, the reception unit 12 receives image data generated by capturing an image of the current passage of a person (flow of people) by a camera unit (not shown).

The acquisition unit 13 uses the first learning model that learns the normal passage state of the person as an event, and acquires whether the passage state of the person recorded in the image data deviates from the normal passage state. .. In the acquisition unit 13, a plurality of people are staying (crowded and people cannot move forward), and a plurality of people have fallen (become a domino) as the passing state of the person. Based on the first learning model that learned the traffic state of the person excluding the abnormal state of, and the traffic state of the person recorded in the image data, the traffic state of the person recorded in the image data (for example, the current current state) Acquires whether the traffic state of a person) deviates from the normal state.

The classification unit 14 can change the traffic state of a person recorded in the image data to the normal traffic state by using the second learning model that learns the normal traffic state different from the normal one as an event. Classify if you have sex. The classification unit 14 describes the traffic state of the person recorded in the image data based on the second learning model in which at least the abnormal state is learned among the normal state and the abnormal state of the traffic state of the person and the image data. Classify if is likely to be in an abnormal state before the present. The "abnormal state" of the flow of the person learned when the second learning model is generated includes the transition state in which the normal state is changed to the abnormal state and the transition state immediately before the abnormal state is reached. It may be.

The output control unit 15 determines whether or not the passage state of the person is abnormal based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14 (the possibility that the passage state becomes abnormal at a time ahead). Whether or not there is) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 determines from the output unit whether a plurality of persons may stay and whether a plurality of persons may fall (whether the traffic state may become abnormal). Control to output.

[Example 6]
The reception unit 12 receives image data recording the state of the water treated in the water treatment facility. As an example, the reception unit 12 receives image data generated by capturing an image of the current water treatment state by a camera unit (not shown). The facility may be various facilities, for example, a factory, a plant, a water purification plant, a sewage treatment plant, or the like.

The acquisition unit 13 acquires whether or not the state of the treated water recorded in the image data deviates from the normal state by using the first learning model in which the state of the water normally treated as an event is learned. As the water treatment state, the acquisition unit 13 has a color of the treated water different from that of the normal time, the presence of foreign matter that is not normally contained in the treated water, and bubbles that are not normally contained in the treated water. Recorded in the image data based on the first learning model that learned the discharge state of the treated water excluding the abnormal state such as the presence of (or the amount of bubbles) and the state of the treated water recorded in the image data. Acquires whether the state of the treated water to be treated (for example, the current state of the treated water) deviates from the normal state.

The classification unit 14 uses a second learning model that learns the state of water in an abnormal state different from the normal state as an event, and the state of the treated water recorded in the image data is the state in the abnormal state. Classify what is likely to be. The classification unit 14 is based on image data based on a second learning model that has learned at least the state of the treated water in the normal state and the (abnormal) state in the abnormal state, and the image data. Classify whether the state of treated water recorded in is likely to become an abnormal (abnormal) state at a point earlier than the present. The "normal (abnormal) state" of the treated water that is learned when the second learning model is generated includes a transition state that transitions from a normal state to a non-normal state, and an abnormal state. The transition state immediately before the state may be included.

Based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14, the output control unit 15 determines whether or not the state of the treated water is abnormal (the state of the treated water becomes abnormal at a time ahead). Whether or not there is a possibility) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 may change the color of the treated water, may contain foreign matter in the treated water, and may contain air bubbles in the treated water (the amount of air bubbles emitted from the treated water). Controls to output from the output unit whether there is a possibility (whether the state of treated water may become abnormal).

[Example 7]
The reception unit 12 receives image data recording the state of smoke discharged from the facility. As an example, the reception unit 12 receives image data generated by capturing the current smoke emission state with a camera unit (not shown). The facility may be various facilities, for example, a cleaning factory, a factory, a plant, or the like.

The acquisition unit 13 acquires whether the smoke state recorded in the image data deviates from the normal state by using the first learning model in which the smoke state normally discharged as an event is learned. The acquisition unit 13 has, for example, a smoke color different from the normal one, the presence of a foreign substance that is not normally contained in the smoke, and a flame (spark) that is not normally contained in the smoke. The smoke state recorded in the image data (for example, the present) based on the first learning model in which the smoke emission state excluding the abnormal state such as the existence of the smoke is learned and the smoke state recorded in the image data. (Smoke state) is obtained as to whether it deviates from the normal state.

The classification unit 14 uses a second learning model that learns a smoke state at an abnormal time (abnormal time) that is different from the normal smoke state as an event, and the smoke state recorded in the image data is abnormal. Classify if there is a possibility of smoke conditions at the time. The classification unit 14 converts the smoke data into image data based on the second learning model that learned at least the abnormal state of the smoke state in the normal state and the (abnormal) state in the abnormal state, and the image data. Classify whether the recorded smoke condition is likely to be an abnormal (abnormal) condition at a point earlier than the present. The "abnormal (abnormal) state" of smoke learned when the second learning model is generated includes a transition state that transitions from a normal state to an abnormal state, and a state at the time of abnormal. The transition state immediately before becomes may be included.

The output control unit 15 determines whether or not the smoke state is abnormal based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14 (the smoke state may become abnormal at a time ahead). Whether or not there is) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 may change the color of smoke, may emit foreign matter together with smoke, and may emit flames (sparks) together with smoke (whether the output control unit 15 may emit flames (sparks) together with smoke. Is there a possibility that the smoke condition may become abnormal?) Is controlled to be output from the output unit.

[Example 8]
The reception unit 12 receives image data recording a vehicle traveling on the road. As an example, the reception unit 12 receives image data generated by capturing an image of the traveling state of a vehicle traveling on the current road by a camera unit (not shown). The above-mentioned camera unit may be, for example, a road surveillance camera or the like.

The acquisition unit 13 uses a learning model learned as the first state of the running state of the vehicle normally traveling on the road as an event, and whether the running state of the vehicle recorded in the image data deviates from the first state. To get. In the acquisition unit 13, for example, the vehicle is running in reverse, the vehicle is congested, the emergency stop vehicle (vehicle with the hazard lamp turned on) is stopped, or an accident caused by the vehicle is caused. The first learning model that learns the running state of the vehicle in the normal state excluding the abnormal (normal time) state such as the occurrence or the vehicle is driving in a meandering manner, and the running of the vehicle recorded in the image data. Based on the state, it is acquired whether the running state of the vehicle recorded in the image data (for example, the running state of the vehicle traveling on the current road) deviates from the normal (normal) state.

The classification unit 14 uses a second learning model that learns the running state of the vehicle at the time of abnormality, which is different from the running state of the normal vehicle as an event, and when the running state of the vehicle recorded in the image data is abnormal. Classify whether there is a possibility that the vehicle will be in a running state. The classification unit 14 is based on image data based on a second learning model in which at least a second learning model in which the running state of the vehicle is in a normal state and a (abnormal) state in a non-normal state is learned, and an image data. It is classified whether the running state of the vehicle recorded in is likely to become an abnormal (abnormal) state at a time earlier than the present. The "(abnormal) running state" of the vehicle that is learned when the second learning model is generated includes a transition state that transitions from a normal state to a non-normal state, and a non-normal state. The transition state immediately before the state may be included.

Based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14, the output control unit 15 determines whether or not the running state of the vehicle is abnormal (the running state of the vehicle becomes abnormal at a time ahead). Whether or not there is a possibility) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 determines whether the vehicle may run in reverse, the vehicle may be congested, or the vehicle may cause an accident (the running state of the vehicle is abnormal (the running state of the vehicle is abnormal (). Controls to output from the output section (whether there is a possibility of abnormalities).

[Example 9]
The reception unit 12 receives image data recording the movement of a person or an animal. As an example, the reception unit 12 receives image data generated by photographing a person or an animal with a camera unit (not shown) indoors, outdoors, public facilities, public transportation, or the like.

The acquisition unit 13 uses a learning model learned as a first state of the normal operating state of the person or animal as an event to acquire whether the operating state of the person or animal recorded in the image data deviates. .. The acquisition unit 13 has abnormal movement states of the person or animal, such as the movement of a sick person or animal, the movement of a person with a physical disability, the movement of a pregnant woman, and the movement of a drunken person. Based on the first learning model that learned the normal operating state (for example, the operating state of a healthy person) excluding the operating state of, and the operating state of a person or animal recorded in the image data, the image data Acquires whether the recorded motion state of the person or animal deviates from the normal (normal) state.

The classification unit 14 uses as a learning model learned as a second state an abnormal motion state different from the normal motion of the person or animal as an event, and the motion state of the person or animal recorded in the image data. Is classified as to whether or not is in the operating state at the time of abnormal. The classification unit 14 is based on a second learning model in which at least the abnormal state is learned among the normal state and the abnormal (abnormal) state of the movement state of the person or animal, and the image data. Classify whether the motion state of a person or animal recorded in the image data may become an abnormal (abnormal) state at a time earlier than the present. The "abnormal (abnormal) state" of a person or animal learned when the second learning model is generated includes a transition state that transitions from a normal state to an abnormal state, and an abnormal state. The transition state immediately before the state of is may be included.

Based on the acquisition result by the acquisition unit 13 and the classification result by the classification unit 14, the output control unit 15 determines whether or not the operation state of the person or animal is abnormal (abnormal) (the operation state of the person or animal is temporal). Whether or not there is a possibility of (abnormal) abnormality at the previous point in time) is controlled to be output from the above-mentioned output unit. For example, the output control unit 15 identifies a person with dementia, a pregnant woman, a person who needs assistance, a person who is drunk, an animal suffering from mad cow disease, and the like, and controls to output from the output unit. Further, the output control unit 15 controls to output from the output unit whether the state of the person or animal may become abnormal (for example, whether the person who is drunk may fall from the platform, etc.). do.

Next, the information processing method according to the embodiment will be described.
First, a method of generating the first learning model and the second learning model will be described.
FIG. 6 is a first flowchart for explaining an information processing method according to an embodiment.

In step ST101, the control unit 11 learns the normal state of the event and generates the first learning model. The control unit 11 acquires image data (moving image or a plurality of still images) in which the normal state of the event is recorded, and generates the first learning model by learning the image data. In this case, the control unit 11 may learn a plurality of patterns of normal states for the event and generate a first learning model.

In step ST102, the control unit 11 learns at least the abnormal state among the normal state and the abnormal state related to the same event as in step ST101 to generate the second learning model. In this case, it is preferable that the control unit 11 learns at least a plurality of patterns of abnormal states to generate a second learning model. That is, the control unit 11 acquires image data (moving image or a plurality of still images) in which an abnormal state of at least a plurality of patterns of events is recorded, and generates a second learning model by learning the image data. .. The control unit 11 may learn the state at the time of transition from the normal state to the abnormal state. As a specific example, the control unit 11 sets a predetermined time before (time before the reference time) when the abnormal state occurs when transitioning from the normal state to the abnormal state as a reference (reference time). It may be possible to learn the state of (within a predetermined time). Various times are set in the predetermined time according to the event.

Next, a method of outputting information indicating whether or not the event is abnormal will be described.
FIG. 7 is a second flowchart for explaining the information processing method according to the embodiment.

In step ST201, the reception unit 12 receives the image data. As an example, the reception unit 12 is generated by being imaged by a camera unit (not shown) arranged in the information processing device 1 or an external camera (not shown) arranged outside the information processing device 1. Image data (image data generated by being imaged at present) may be accepted. Alternatively, as an example, the reception unit 12 may receive the image data stored in the external server (not shown) via the communication unit 16.

In step ST202, the acquisition unit 13 sets the event recorded in the image data and the first learning model based on the first learning model generated in step ST101 shown in FIG. 6 and the image data received in step ST201. Obtain the degree of deviation from the normal state based on. That is, the acquisition unit 13 acquires the degree of deviation indicating how much the state of the event recorded in the image data deviates from the normal state based on the first learning model and the image data. The acquisition unit 13 may acquire, for example, a numerical value indicating the degree of deviation.

In step ST203, the classification unit 14 classifies the events recorded in the image data into a plurality of states based on the second learning model generated in step ST102 of FIG. 6 and the image data received in step ST201. Classify. In this case, the classification unit 14 may use the second learning model to classify whether or not the event recorded in the image data is in an abnormal state. For example, the classification unit 14 may classify the state from the normal state to the abnormal state into a plurality of stages, and classify the state recorded in the image data into any stage.

In step ST204, the output control unit 15 determines the state of the event (event state) recorded in the image data received in step ST201 based on the degree of deviation acquired in step ST202 and the result classified in step ST203. Controls the output section to output (whether or not is abnormal).
Here, the output control unit 15 outputs information indicating an abnormal state when the degree of deviation is equal to or greater than the first predetermined value and the possibility that the classification result is an abnormal state is equal to or greater than the second predetermined value. The output unit may be controlled.
Further, the output control unit 15 determines from the output unit whether or not the event is abnormal (whether or not the state of the event may become abnormal at a time earlier) based on the degree of deviation and the classification result. It may be controlled to output.
One embodiment of the above-mentioned "output unit" may be a communication unit 16, a storage unit 17, a display unit 18, and the like.

Next, the effect of this embodiment will be described.
The information processing device 1 includes an event recorded in the image data based on the reception unit 12 that receives the image data, the first learning model that learns the normal state of the event, and the image data received by the reception unit 12. Based on the acquisition unit 13 that acquires the degree of deviation from the normal state based on the first learning model, the second learning model that has learned at least a plurality of abnormal states of the event, and the image data received by the reception unit 12. , Recorded in the image data based on the classification unit 14 that classifies the events recorded in the image data into a plurality of states, the degree of deviation acquired by the acquisition unit 13, and the result classified by the classification unit 14. It includes an output control unit 15 that outputs the state of the event to be performed from the output unit.
Thereby, the information processing apparatus 1 can determine whether the event is in an abnormal state (or whether the event is in a normal state).
Further, since the information processing device 1 uses two of the degree of deviation acquired by the acquisition unit 13 and the result classified by the classification unit 14, the determination accuracy is higher than the case where only one of them is used. Can be raised. Further, the information processing apparatus 1 includes the acquisition unit 13 and the classification unit 14, so that when the event is transitioning from the normal state to the abnormal state, the information processing device 1 is in a stage earlier than this in terms of time. It is possible to acquire that the event becomes an abnormal state.

In the information processing device 1, the acquisition unit 13 acquires the degree of divergence by using the first learning model obtained by learning the moving image in which the normal state of the event is recorded. In this case, the classification unit 14 uses a second learning model obtained by learning a moving image recording at least the abnormal state of the event among the normal state of the event and the abnormal state of the event, and uses the image. It may be classified according to whether or not the event recorded in the data is an abnormal state.
As a result, the information processing apparatus 1 acquires that the event becomes an abnormal state at a stage earlier than this, particularly when the event is transitioning from a normal state to an abnormal state. Can be done.

In the information processing device 1, the output control unit 15 may be in an abnormal state in which the degree of deviation acquired by the acquisition unit 13 is equal to or greater than the first predetermined value and is classified by the classification unit 14. If it is greater than or equal to the value, the output unit may be controlled to output information indicating an abnormal state.
As a result, the information processing apparatus 1 can output information indicating an abnormal state (for example, the degree of the abnormal state or the degree of transition to the abnormal state).

In the information processing device 1, the reception unit 12 receives the image data in which the flame is recorded, and the acquisition unit 13 uses the first learning model in which the flame in the normal state is learned as a normal state as an event to obtain the image data. Acquiring whether the recorded flame deviates from the normal flame, the classification unit 14 uses the second learning model in which the non-normal flame different from the normal one is learned as an abnormal state as an event. Therefore, it may be possible to classify whether or not the flame recorded in the image data may become an unusual flame.
As a result, the information processing device 1 can output an abnormal state of the flame. That is, the information processing device 1 can output whether or not there is an abnormality in the combustion state of the substance. Alternatively, the information processing device 1 can output the possibility that an abnormality may occur in the combustion state of the substance.

In the information processing device 1, the reception unit 12 may receive image data recording the flow of the river. In this case, the acquisition unit 13 uses the first learning model in which the river flow in the normal state is learned as a normal state as an event, and the river flow recorded in the image data is obtained from the river flow in the normal time. It may be possible to acquire whether or not there is a divergence. The classification unit 14 uses a second learning model in which the flow of the river before the flow of the river during the abnormal time, which is different from the normal time, is learned as an abnormal state as an event, and the classification unit 14 uses the second learning model to record the river in the image data. It may be possible to classify whether the flow may become an abnormal flow.
As a result, the information processing device 1 can output the possibility that an abnormal state of the river, for example, flooding of the river or breakage of the embankment occurs.

In the information processing device 1, the reception unit 12 may receive image data recording a state in which a plurality of materials are mixed. In this case, the acquisition unit 13 uses the first learning model in which the normal material mixing state is learned as an event, and whether the material mixing state recorded in the image data deviates from the normal material mixing state. May be obtained. The classification unit 14 can use the second learning model that learns the mixed state of the material at the time of abnormality different from the normal time as an event, and the mixed state of the material recorded in the image data can be changed to the mixed state at the time of abnormality. It may be classified as having sex.
As a result, the information processing device 1 can output whether or not the mixing of substances is in an abnormal state. For example, the information processing device 1 can output a state in which substances are not evenly mixed, or a possibility that substances are not evenly mixed.

In the information processing device 1, the reception unit 12 may receive image data recording the state of food during fermentation. In this case, the acquisition unit 13 acquires whether the fermentation state of the food recorded in the image data deviates from the normal fermentation state by using the first learning model in which the fermentation state of the normal food is learned as an event. You may do it. The classification unit 14 uses a second learning model in which the fermentation state of the food during the abnormal state different from the normal state is learned as an event, and the fermentation state of the food recorded in the image data is changed to the fermentation state during the abnormal state. It may be classified as to whether or not there is a possibility of becoming.
As a result, the information processing device 1 can output the abnormal fermentation state of the food. For example, the information processing device 1 can output the possibility that the food is not fermented normally or the food is not fermented normally.

In the information processing device 1, the reception unit 12 may receive image data recording the passage of one or more persons. In this case, the acquisition unit 13 uses the first learning model in which the normal passage state of the person is learned as an event, and whether the passage state of the person recorded in the image data deviates from the normal passage state. May be obtained. The classification unit 14 can change the traffic state of a person recorded in the image data to the normal traffic state by using the second learning model that learns the normal traffic state different from the normal one as an event. It may be classified as having sex.
As a result, the information processing device 1 can output whether or not the passage of a person on a road or the like is in an abnormal state. For example, the information processing device 1 may have an abnormal passing state of a person (as a specific example, an abnormal state in which a plurality of people have fallen due to congestion) or an abnormal passing state of a person (as an example). , The possibility that multiple people will fall due to congestion) can be output.

In the information processing device 1, the reception unit 12 may receive image data recording the state of water processed in the water treatment facility. In this case, the acquisition unit 13 uses the first learning model that learns the state of the water that has been normally treated as an event to determine whether the state of the treated water recorded in the image data deviates from the normal state. It may be acquired. The classification unit 14 uses a second learning model that learns the state of water in an abnormal state different from the normal state as an event, and the state of the treated water recorded in the image data is the state in the abnormal state. It may be classified as to whether or not there is a possibility of becoming.
As a result, the information processing device 1 can output whether the state of the water treated in the water treatment facility is abnormal. For example, the information processing apparatus 1 includes an abnormal state of water (as a specific example, an abnormal state in which the color of the treated water is discolored, an abnormal state in which the treated water contains foreign matter, and bubbles in the treated water (as a specific example). It is possible to output the possibility that the abnormal state (the amount of bubbles contained in the treated water is larger than usual) or the state of the treated water becomes abnormal.

In the information processing device 1, the reception unit 12 may receive image data recording the state of smoke discharged from the facility. In this case, the acquisition unit 13 uses the first learning model that learns the state of smoke normally discharged as an event to acquire whether the state of smoke recorded in the image data deviates from the normal state. You may do it. The classification unit 14 uses a second learning model that learns a smoke state at an abnormal time different from the normal smoke state as an event, and uses a second learning model to learn the smoke state at the time when the smoke state recorded in the image data is abnormal. It may be possible to classify whether there is a possibility of becoming a state.
As a result, the information processing device 1 can output whether the smoke discharged from the facility is in an abnormal state. For example, the information processing device 1 has an abnormal state of emitted smoke (as a specific example, a smoke color different from the normal time, the presence of foreign matter that is not normally contained in the smoke, and the normal time. It is possible to output the possibility that the state of smoke (abnormal state such as the presence of flame (sparks) that is not contained in smoke) or the state of smoke becomes abnormal.

In the information processing device 1, the reception unit 12 may receive image data recording a vehicle traveling on the road. In this case, the acquisition unit 13 uses the first learning model that learns the traveling state of the vehicle normally traveling on the road as an event, and the traveling state of the vehicle recorded in the image data deviates from the normal traveling state. It may be possible to obtain whether or not it is. The classification unit 14 uses a second learning model that learns the running state of the vehicle at the time of abnormality, which is different from the running state of the normal vehicle as an event, and when the running state of the vehicle recorded in the image data is abnormal. It may be possible to classify whether or not there is a possibility that the vehicle will be in a running state.
As a result, the information processing device 1 can output whether the traveling state of the vehicle traveling on the road is abnormal. For example, the information processing device 1 can output whether the vehicle may run in reverse, whether the vehicle may be congested, whether the vehicle may cause an accident, or the like.

In the information processing device 1, the reception unit 12 may receive image data recording the movement of a person or an animal. In this case, the acquisition unit 13 uses the first learning model in which the normal operating state of the person or animal is learned as an event, and the operating state of the person or animal recorded in the image data is changed from the normal operating state. It may be possible to acquire whether or not there is a divergence. The classification unit 14 uses the second learning model as a second learning model that learns the abnormal motion state different from the normal motion of the person or animal as an event, and the motion state of the person or animal recorded in the image data is non-existent. It may be possible to classify whether or not the operating state is normal.
As a result, the information processing device 1 can output whether the operating state of the person or animal is in an abnormal (abnormal) state or may be in an abnormal (abnormal) state.

In the information processing method, an event recorded in the image data based on the reception step in which the computer receives the image data, the first learning model in which the normal state of the event is learned, and the image data received by the reception step. An image based on an acquisition step for acquiring the degree of deviation from the normal state based on the first learning model, a second learning model for learning at least a plurality of abnormal states of an event, and image data received by the reception step. The states of the events recorded in the image data based on the classification step that classifies the events recorded in the data into multiple states, the degree of deviation acquired by the acquisition step, and the results classified by the classification step. Is executed from the output step to the output control step.
Thereby, the information processing method can determine whether the event is in an abnormal state (or whether the event is in a normal state).
In addition, since the information processing method uses two, the degree of deviation acquired by the acquisition step and the result classified by the classification step, the accuracy of the determination is higher than when only one of them is used. be able to. Further, the information processing method includes two steps, an acquisition step and a classification step, so that when the event is transitioning from the normal state to the abnormal state, the event is abnormal at a stage earlier than this. You can get to be in a state of being.

The information processing program is a reception function that receives image data on a computer, a first learning model that learns the normal state of the event, and an event recorded in the image data based on the image data received by the reception function. An image based on an acquisition function that acquires the degree of deviation from the normal state based on the first learning model, a second learning model that has learned at least a plurality of abnormal states of an event, and image data received by the reception function. The state of the event recorded in the image data based on the classification function that classifies the events recorded in the data into multiple states, the degree of deviation acquired by the acquisition function, and the result classified by the classification function. Is realized with an output control function that outputs from the output function.
As a result, the information processing program can determine whether the event is in an abnormal state (or whether the event is in a normal state).
In addition, since the information processing program uses two, the degree of deviation acquired by the acquisition function and the result classified by the classification function, the accuracy of the determination is higher than when only one of them is used. be able to. Further, the information processing program is provided with two functions, an acquisition function and a classification function, so that when an event is transitioning from a normal state to an abnormal state, the event is abnormal at a stage earlier than this. You can get to be in a state of being.

Each part of the information processing device 1 described above may be realized as a function of a computer arithmetic processing unit or the like. That is, the reception unit 12, the acquisition unit 13, the classification unit 14, and the output control unit 15 (control unit 11) of the information processing device 1 have a reception function, an acquisition function, a classification function, and an output control function (control unit 11) by a computer arithmetic processing unit or the like. It may be realized as a control function).
The information processing program can realize each of the above-mentioned functions in a computer. The information processing program may be recorded on a non-temporary recording medium that can be read by a computer, such as an external memory or an optical disk.
Further, as described above, each part of the information processing device 1 may be realized by a computer arithmetic processing device or the like. The arithmetic processing unit or the like is composed of, for example, an integrated circuit or the like. Therefore, each part of the information processing device 1 may be realized as a circuit constituting an arithmetic processing unit or the like. That is, the reception unit 12, the acquisition unit 13, the classification unit 14, and the output control unit 15 (control unit 11) of the information processing device 1 are the reception circuit, the acquisition circuit, the classification circuit, and the output control that constitute the arithmetic processing unit of the computer. It may be realized as a circuit (control circuit).
Further, the communication unit 16, the storage unit 17, and the display unit 18 (output unit) of the information processing device 1 may be realized as, for example, a communication function including a function of an arithmetic processing unit or the like, a storage function, and a display function (output function). good. Further, the communication unit 16, the storage unit 17, and the display unit 18 (output unit) of the information processing device 1 are realized as a communication circuit, a storage circuit, and a display circuit (output circuit) by being composed of, for example, an integrated circuit or the like. You may. Further, the communication unit 16, the storage unit 17, and the display unit 18 (output unit) of the information processing device 1 are configured as a communication device, a storage device, and a display device (output device) by being composed of, for example, a plurality of devices. You may.

1 Information processing device 11 Control unit 12 Reception unit 13 Acquisition unit 14 Classification unit 15 Output control unit 16 Communication unit 17 Storage unit 18 Display unit

Claims (14)

The reception department that accepts image data and
Degree of divergence between the event recorded in the image data and the first state based on the first learning model based on the first learning model that learned an arbitrary first state of the event and the image data received by the reception unit. And the acquisition department to acquire
A plurality of events recorded in the image data are recorded based on the second learning model that has learned at least a plurality of second states of the event, which is different from the first state, and the image data received by the reception unit. A classification unit that classifies into states and
An output control unit that outputs the state of an event recorded in image data from the output unit based on the degree of deviation acquired by the acquisition unit and the result classified by the classification unit.
Information processing device equipped with. The acquisition unit acquires the degree of divergence by using the first learning model obtained by learning the moving image recording the first state of the event.
The classification unit uses a second learning model obtained by learning a moving image recording at least the second state of the event among the first state of the event and the second state of the event. The information processing apparatus according to claim 1, wherein the event recorded in the image data is classified into whether or not it is in the second state. The output control unit is the second when the degree of deviation acquired by the acquisition unit is equal to or greater than the first predetermined value and the possibility of being in the second state classified by the classification unit is greater than or equal to the second predetermined value. 2. The information processing apparatus according to claim 1 or 2, which controls the output unit so as to output information indicating a state. The reception unit receives the image data recording the flame and receives it.
The acquisition unit uses a first learning model that learns the flame in the normal state as the first state as the event, and acquires whether the flame recorded in the image data deviates from the flame in the normal time. ,
The classification unit uses a second learning model in which a flame in an abnormal time different from the normal time is learned as a second state as the event, and the flame recorded in the image data becomes a flame in the abnormal time. The information processing apparatus according to any one of claims 1 to 3, which classifies the possibility. The reception section receives image data recording the flow of the river and receives it.
The acquisition unit uses the first learning model in which the normal river flow is learned as the first state as the event, and the river flow recorded in the image data deviates from the normal river flow. Get what you are doing
The classification unit is recorded in image data by using a second learning model in which the flow of the river before becoming the flow of the river in the abnormal time different from the normal time as the event is learned as the second state. The information processing apparatus according to any one of claims 1 to 3, which classifies whether a river flow may become an abnormal flow. The reception unit receives image data recording a state in which a plurality of materials are mixed, and receives the image data.
The acquisition unit uses the first learning model in which the normal material mixed state is learned as the first state as the event, and the material mixed state recorded in the image data deviates from the normal material mixed state. Get what you are doing
The classification unit uses a second learning model in which the mixed state of materials at an abnormal time different from the normal state is learned as a second state as the event, and the mixed state of materials recorded in the image data is abnormal. The information processing apparatus according to any one of claims 1 to 3, which classifies whether or not there is a possibility of being in a mixed state. The reception section receives image data recording the state of food during fermentation, and receives the image data.
The acquisition unit uses the first learning model in which the fermentation state of normal food is learned as the first state as the event, and the fermentation state of food recorded in the image data deviates from the normal fermentation state. Get or
The classification unit uses a second learning model in which the fermentation state of the food during the abnormal state different from the normal state is learned as the second state as the event, and the fermentation state of the food recorded in the image data is non-normal. The information processing apparatus according to any one of claims 1 to 3, which classifies whether or not there is a possibility of a normal fermentation state. The reception unit receives image data recording the passage of one or more persons, and receives the image data.
The acquisition unit uses a first learning model in which the normal passage state of a person is learned as the first state as the event, and the passage state of the person recorded in the image data deviates from the normal passage state. Get what you are doing
The classification unit uses a second learning model in which the second learning model is learned as the second state of the normal traffic state different from the normal time as the event, and the traffic state of the person recorded in the image data is the abnormal time. The information processing apparatus according to any one of claims 1 to 3, which classifies whether or not there is a possibility of being in a passing state. The reception unit receives image data recording the state of water treated in the water treatment facility, and receives the image data.
The acquisition unit uses the first learning model in which the state of water normally processed as the event is learned as the first state, and the state of the treated water recorded in the image data deviates from the normal state. Get what you are
The classification unit uses a second learning model in which the state of water in an abnormal state different from the normal state as the event is learned as the second state, and the state of the treated water recorded in the image data. The information processing apparatus according to any one of claims 1 to 3, which classifies whether or not there is a possibility of a state in an abnormal state. The reception section receives image data recording the state of smoke emitted from the facility, and receives the image data.
The acquisition unit uses a first learning model in which the state of smoke normally discharged as the event is learned as the first state, and the state of smoke recorded in the image data deviates from the first state. Get or
The classification unit uses a second learning model in which the smoke state at an abnormal time different from the normal smoke state is learned as the second state as the event, and the smoke state recorded in the image data is determined. The information processing apparatus according to any one of claims 1 to 3, which classifies whether or not there is a possibility of a smoke state at an abnormal time. The reception unit receives image data recording vehicles traveling on the road, and receives the image data.
The acquisition unit uses the first learning model in which the traveling state of the vehicle normally traveling on the road is learned as the first state as the event, and the traveling state of the vehicle recorded in the image data is changed from the first state. Get if there is a divergence,
The classification unit uses a second learning model in which the running state of the vehicle at an abnormal time different from the running state of the normal vehicle as the event is learned as the second state, and the vehicle is recorded in the image data. The information processing device according to any one of claims 1 to 3, which classifies whether or not the traveling state may be the traveling state of the vehicle when the traveling state is abnormal. The reception unit receives image data recording the movement of a person or an animal, and receives the image data.
The acquisition unit uses the first learning model in which the normal operating state of the person or animal is learned as the first state as the event, and the operating state of the person or animal recorded in the image data is the first state. Get if it deviates from
The classification unit uses the person or animal as the second learning model in which the normal movement state different from the normal movement of the person or animal is learned as the second state, and the person or animal is recorded in the image data. The information processing apparatus according to any one of claims 1 to 3, which classifies whether the operating state of an animal becomes an abnormal operating state. The computer
The reception step that accepts image data and
Degree of divergence between the event recorded in the image data and the first state based on the first learning model based on the first learning model that learned an arbitrary first state of the event and the image data received by the reception step. To get the steps and
A plurality of events recorded in the image data are recorded based on the second learning model that has learned at least a plurality of second states of the event, which is different from the first state, and the image data received by the reception step. Classification steps to classify into states and
An output control step that outputs the state of the event recorded in the image data from the output step based on the degree of deviation acquired by the acquisition step and the result classified by the classification step.
Information processing method to execute. On the computer
A reception function that accepts image data and
Degree of divergence between the event recorded in the image data and the first state based on the first learning model based on the first learning model that learned an arbitrary first state of the event and the image data received by the reception function. And the get function to get
A plurality of events recorded in the image data are recorded based on the second learning model in which at least a plurality of second states of the event are learned, which is different from the first state, and the image data received by the reception function. Classification function to classify into states and classification
An output control function that outputs the state of the event recorded in the image data from the output function based on the degree of deviation acquired by the acquisition function and the result classified by the classification function.
Information processing program that realizes.

Images