JP2021111027A - Information output program, information output device and information output method - Google Patents

Abstract

To provide an information output program, an information output device and an information output method which can efficiently monitor an object reflected in a dynamic image.SOLUTION: An information output device acquires a dynamic image; acquires a classification result about each type of an object included in a plurality of images constituting the dynamic image, and a first degree of certainty indicating a degree of certainty of the classification result, by using a first learning model which has learned a type candidate of the object included in the image and the degree of certainty that a type of the object is the type candidate; determines whether or not each object satisfies a predetermined condition, based on the classification result corresponding to each object and the first degree of certainty, for each object included in the plurality of images; and outputs information indicating that the object which has been determined to satisfy the predetermined condition is a monitoring object, among the objects included in the plurality of images.SELECTED DRAWING: Figure 18

Description

The present invention relates to an information output program, an information output device, and an information output method.

For example, in fields such as maritime patrols and coastal surveillance, workers can check the contents of moving images taken in the monitoring area to detect objects (for example, suspicious ships) and identify the types of objects reflected in the moving images. And so on. Specifically, for example, when a plurality of objects are detected in a captured moving image, the operator visually identifies the type of each object (see, for example, Patent Document 1).

International Publication No. 2018/173800

However, if all the detected objects are output as monitoring targets, there is a problem that the objects displayed in the moving image cannot be efficiently monitored.

Therefore, in one aspect, it is an object of the present invention to provide an information output program, an information output device, and an information output method that enable efficient monitoring of an object displayed in a moving image.

In one aspect of the embodiment, the first learning model is used by acquiring a moving image and learning the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate. The categorization result for each type of the object included in the plurality of images constituting the moving image and the first accuracy indicating the accuracy of the categorization result are acquired, and for each object included in the plurality of images, Based on the categorization result corresponding to each object and the first accuracy, it is determined whether or not each object satisfies a predetermined condition, and among the objects included in the plurality of images, the predetermined object is determined. Outputs information indicating that the object determined to satisfy the condition of is the monitoring target.

According to one aspect, it is possible to efficiently monitor the object shown in the moving image.

FIG. 1 is a diagram illustrating a configuration of the information processing system 10. FIG. 2 is a diagram illustrating a hardware configuration of the information processing device 1. FIG. 3 is a block diagram of the function of the information processing device 1. FIG. 4 is a flowchart illustrating an outline of the information output process according to the first embodiment. FIG. 5 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 6 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 7 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 8 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 9 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 10 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 11 is a flowchart illustrating the details of the information output processing according to the first embodiment. FIG. 12 is a diagram illustrating a specific example of the first learning data 131. FIG. 13 describes a specific example of the process of S22. FIG. 14 is a diagram illustrating a specific example of the second learning data 132. FIG. 15 is a diagram illustrating a specific example of the process of S42. FIG. 16 is a diagram illustrating a specific example of the processing of S53. FIG. 17 is a diagram illustrating a specific example of the detection result information 135. FIG. 18 is a diagram illustrating a specific example of the processing of S61. FIG. 19 is a diagram illustrating a specific example of the classification result information 134. FIG. 20 is a diagram illustrating a specific example of the processing of S64. FIG. 21 is a diagram illustrating a specific example of the process of S72.

[Information processing system configuration]
First, the configuration of the information processing system 10 will be described. FIG. 1 is a diagram illustrating a configuration of the information processing system 10.

The information processing system 10 shown in FIG. 1 includes an information processing device 1 and an operation terminal 3. The operation terminal 3 is, for example, a PC (Personal Computer) in which an operator inputs necessary information and the like, and can access the information processing device 1 via a network NW such as the Internet.

The information processing device 1 is, for example, a candidate (hereinafter, type) of an object type included in each image (hereinafter, also referred to as image data) constituting a moving image (hereinafter, also referred to as moving image data) captured in a monitoring area. It is included in each image data by using a learning model (hereinafter, also referred to as a first learning model) that learns (also referred to as a candidate) and the probability (for example, probability) that the type candidate is the correct answer as the type of the object. Among the objects to be processed, identify the objects that the operator needs to visually check.

Specifically, the information processing apparatus 1 acquires, for example, a categorized result for each type of an object included in a plurality of image data and the accuracy of the categorized result by using the first learning model. Then, the information processing device 1 determines whether or not each object satisfies a predetermined condition based on the classification result and the accuracy corresponding to each object for each object included in the plurality of image data. conduct. After that, the information processing device 1 outputs information indicating that, among the objects included in the plurality of image data, the object determined to satisfy a predetermined condition is the monitoring target.

That is, in the past, for example, when moving images that require detection of an object or the like frequently occur, there is a possibility that the burden on the operator due to the detection of the object or the like will increase. Further, for example, if the number of objects that need to be confirmed increases further in the future, it may become difficult to secure the required number of workers in the future.

Therefore, the information processing device 1 in the present embodiment is, for example, an object whose type corresponding to the classification result output by the first learning model is not included in the type of the monitoring target, and is based on the first learning model. When there is an object whose accuracy of the output classification result is higher than a predetermined threshold value, it is determined that it is not necessary to monitor the existing object. Further, for example, an object in which the type corresponding to the classification result output by the first learning model is included in the type of the monitoring target, or the accuracy of the classification result output by the first learning model is higher than a predetermined threshold value. When a low object exists, the information processing apparatus 1 determines that it is necessary to monitor the existing object.

Then, the information processing device 1 outputs, for example, only the information about the object determined to need to be monitored to the operation terminal 3. After that, the operator visually identifies the type of the object whose information is output to the operation terminal 3, for example.

As a result, the information processing device 1 can suppress the burden on the operator due to the monitoring of the object.

[Hardware configuration of information processing system]
Next, the hardware configuration of the information processing system 10 will be described. FIG. 2 is a diagram illustrating a hardware configuration of the information processing device 1.

As shown in FIG. 2, the information processing device 1 includes a CPU 101 which is a processor, a memory 102, a communication device 103, and a storage medium 104. The parts are connected to each other via the bus 105.

The storage medium 104 is, for example, a program storage area (not shown) for storing a program 110 for performing a process of outputting information indicating an object that an operator needs to monitor (hereinafter, also referred to as an information output process). Has. Further, the storage medium 104 has, for example, a storage unit 130 (hereinafter, also referred to as an information storage area 130) for storing information used when performing information output processing. The storage medium 104 may be, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

The CPU 101 executes the program 110 loaded from the storage medium 104 into the memory 102 to perform information output processing.

Further, the communication device 103 communicates with the operation terminal 3 via, for example, the network NW.

[Information processing system functions]
Next, the function of the information processing system 10 will be described. FIG. 3 is a block diagram of the function of the information processing device 1.

As shown in FIG. 3, the information processing device 1 is, for example, an information receiving unit 111 (hereinafter, also referred to as an information acquisition unit 111) by organically collaborating with hardware such as a CPU 101 and a memory 102 and a program 110. ), The information management unit 112, the first model generation unit 113, the second model generation unit 114, the result acquisition unit 115, the first condition determination unit 116, the second condition determination unit 117, and the result output unit. Realizes various functions including 118.

Further, as shown in FIG. 3, the information processing apparatus 1 stores, for example, the first learning data 131, the second learning data 132, the moving image data 133, the classification result information 134, and the detection result information 135. Store in area 130.

The information receiving unit 111 receives, for example, various information input by the operator via the operation terminal 3. Specifically, the information receiving unit 111 receives, for example, the first learning data 131 used for generating the first learning model. The first learning data 131 is, for example, learning data including a learning image including an object and information indicating a type of the object included in the learning image.

Further, the information receiving unit 111 receives, for example, the second learning data 132 used for generating the second learning model. The second learning data 132 is, for example, learning data including a learning image including an object and position information of the object included in the learning image.

Further, the information receiving unit 111 receives the moving image data 133 acquired by, for example, a photographing device (not shown) such as a camera that photographs an object at sea. Then, the information management unit 112 stores, for example, the first learning data 131, the second learning data 132, and the moving image data 133 received by the information receiving unit 111 in the information storage area 130.

The information receiving unit 111 may acquire various information stored in the information storage area (not shown) of the operation terminal 3 by voluntarily accessing the operation terminal 3, for example. good. Further, the information receiving unit 111 may acquire the moving image data 133 stored in the information storage area (not shown) of the imaging device by voluntarily accessing the imaging device, for example. ..

The first model generation unit 113 generates the first learning model by learning the first learning data 131 stored in the information storage area 130.

The second model generation unit 114 generates the second learning model by learning the second learning data 132 stored in the information storage area 130.

The result acquisition unit 115 acquires the moving image data 133 stored in the information storage area 130. Then, the result acquisition unit 115 uses the second learning model generated by the second model generation unit 114 to indicate the accuracy that the object exists in each of the plurality of image data constituting the moving image data 133 (hereinafter referred to as the accuracy). , Also called the second accuracy). Specifically, the result acquisition unit 115 acquires the second accuracy that is output when a plurality of image data are input to the second learning model. After that, the information management unit 112 generates, for example, the detection result information 135 indicating the second accuracy acquired by the result acquisition unit 115 and stores it in the information storage area 130.

In addition, the result acquisition unit 115 acquires a plurality of partial image data for each of the objects included in the plurality of image data constituting the moving image data 133. Specifically, the result acquisition unit 115 acquires (extracts) a plurality of partial image data from the plurality of image data constituting the moving image data 133 by referring to the detection result information 135 stored in the information storage area 130, for example. )do. Then, the result acquisition unit 115 uses the first learning model generated by the first model generation unit 113 to obtain the categorized results for each type of the object included in the plurality of partial image data and the categorized results. Acquire the accuracy (hereinafter, also referred to as the first accuracy). Specifically, the result acquisition unit 115 acquires the categorized result and the first accuracy that are output when a plurality of image data are input to the first learning model. After that, the information management unit 112 generates, for example, the classification result information 134 indicating the classification result acquired by the result acquisition unit 115 and the first accuracy, and stores it in the information storage area 130.

The first condition determination unit 116 refers to, for example, the classification result information 134 stored in the information storage area 130, and whether or not the classification result corresponding to each object satisfies the first condition for each of the plurality of partial image data. And, it is determined whether or not the first accuracy corresponding to each object satisfies the second condition. Specifically, the first condition determination unit 116, for example, when the classification result indicates that each object is an object having a predetermined attribute (for example, a warship), the classification result corresponding to each object. Is determined to satisfy the first condition. Further, in the first condition determination unit 116, for example, when the first accuracy corresponding to each object is larger than the threshold value (hereinafter, also referred to as the first threshold value), the first accuracy corresponding to each object is the second condition. Is determined to satisfy.

The second condition determination unit 117 refers to, for example, the classification result information 134 and the detection result information 135 stored in the information storage area 130, and for each of the plurality of partial image data, the first accuracy corresponding to each object is the first. It is determined whether or not the two conditions are satisfied, and whether or not the second accuracy corresponding to each object satisfies the third condition. Specifically, in the second condition determination unit 117, for example, when the second accuracy corresponding to each object is larger than the threshold value (hereinafter, also referred to as the second threshold value), the second accuracy corresponding to each object is the second. It is determined that the three conditions are satisfied.

The result output unit 118 includes an object for which the classification result is determined to satisfy the first condition and an object for which the first accuracy is determined to satisfy the second condition among the objects included in the plurality of partial image data. Information (for example, alert) indicating the above is output.

Further, in the result output unit 118, among the objects included in the plurality of partial image data, the objects whose first accuracy satisfies the second condition and whose second accuracy satisfies the third condition are unlearned. Output information indicating that.

[Outline of the first embodiment]
Next, the outline of the first embodiment will be described. FIG. 4 is a flowchart illustrating an outline of the information output process according to the first embodiment.

As shown in FIG. 4, the information processing device 1 waits until the information output timing is reached (NO in S1). The information output timing may be, for example, the timing at which the worker inputs information to the effect that the monitoring of the object at sea is started.

Then, when the information output timing is reached (YES in S1), the information processing device 1 acquires the moving image data 133 (S2). Specifically, the information processing device 1 acquires, for example, the moving image data 133 stored in the information storage area 130. Further, the information processing device 1 acquires, for example, moving image data 133 stored in an external storage device (not shown).

Subsequently, the information processing apparatus 1 processes S2 by using the first learning model that learns the type candidate of the object included in the image data and the probability that the type candidate is the correct answer as the type of the object. The categorization result for each type of the object included in the plurality of image data constituting the moving image data acquired in the above and the first accuracy indicating the accuracy of the categorization result are acquired (S3).

Next, the information processing device 1 sets each of the objects included in the plurality of image data constituting the moving image data acquired in the process of S2 based on the categorization result acquired in the process of S3 and the first accuracy. It is determined whether or not the object satisfies a predetermined condition (S4).

After that, the information processing device 1 monitors the objects included in the plurality of image data constituting the moving image data acquired in the process of S2, which are determined to satisfy a predetermined condition in the process of S4. Information indicating that is output (S5).

As a result, the information processing device 1 can identify an object that the operator needs to visually confirm by using the first learning model. Therefore, the information processing device 1 can suppress the burden on the operator due to the monitoring of the object.

[Details of the first embodiment]
Next, the details of the first embodiment will be described. 5 to 11 are flowcharts illustrating the details of the information output processing according to the first embodiment. Further, FIGS. 12 to 21 are diagrams for explaining the details of the information output processing in the first embodiment.

[1st learning data storage processing]
First, among the information output processes, a process of storing the first learning data 131 (hereinafter, also referred to as a first learning data storage process) will be described. FIG. 5 is a flowchart illustrating the first learning data storage process.

As shown in FIG. 5, the information receiving unit 111 of the information processing device 1 waits until, for example, receives a plurality of first learning data 131 transmitted from the operation terminal 3 (NO in S11). That is, the information receiving unit 111 waits until, for example, the first learning data 131 generated by the operator is transmitted via the operation terminal 3.

When a plurality of first learning data 131s are received (YES in S11), the information management unit 112 of the information processing apparatus 1 stores the plurality of first learning data 131s received in the process of S11 in the information storage area 130. (S12). Hereinafter, a specific example of the first learning data 131 will be described.

[Specific example of the first training data]
FIG. 12 is a diagram illustrating a specific example of the first learning data 131.

The first learning data 131 shown in FIG. 12 includes a "item number" indicating each identification information of the first learning data 131, an "image data" indicating each learning image data itself, and each learning image data. Includes "type" indicating the type of object to be included (type of correct answer). In the following description, it is assumed that any one of "AAA", "BBB" and "CCC" is set for the "type".

Specifically, the first learning data 131 whose "item number" is "1" in FIG. 12 includes learning image data whose name is "IMAGEa1" and whose "type" is "AAA". It is data.

Further, the first learning data 131 whose "item number" is "2" in FIG. 12 is learning data which includes learning image data whose name is "IMAGEa2" and whose "type" is "BBB". be. The description of the other first learning data 131 included in FIG. 12 will be omitted.

[First model generation process]
Next, among the information output processes, a process for generating the first model (hereinafter, also referred to as a first model generation process) will be described. FIG. 6 is a flowchart illustrating the first model generation process.

As shown in FIG. 6, the first model generation unit 113 of the information processing apparatus 1 waits until, for example, the first model generation timing (NO in S21). The first model generation timing may be, for example, the timing at which the worker inputs information to the information processing device 1 to the effect that the first model is generated.

Then, when the first model generation timing is reached (YES in S21), the first model generation unit 113 learns the plurality of first learning data 131 stored in the information storage area 130, thereby performing the first learning. Generate a model (S22). Hereinafter, a specific example of the processing of S22 will be described.

[Specific example of processing of S22]
FIG. 13 describes a specific example of the process of S22.

As shown in FIG. 13, the first model generation unit 113 corresponds to, for example, a plurality of output neurons (hereinafter, also referred to as first output neurons) NE11 corresponding to each of the object type candidates and the first accuracy. A first model MD01 having one output neuron (hereinafter, also referred to as a second output neuron) NE12 is generated. In the example shown in FIG. 13, the first output neuron NE11 includes first output neurons NE11a, NE11b, and NE11c corresponding to the type candidates "AAA", "BBB", and "CCC", respectively.

Then, for example, "IMAGEa1" is set as the "image data" and "AAA" is set as the "type" for the information in which the "item number" is "1" in the first learning data 131 described with reference to FIG. Has been done. Therefore, for example, when the feature amount of the image data whose name is "IMAGEa1" is input from the input neuron NE13, the first model generation unit 113 has a value output from each of the first output neurons NE11 and "AAA". The value corresponding to each weight in the first learning model is updated so that the difference from the correct answer value (“1”, “0”, and “0”) corresponding to “” becomes small. Specifically, in this case, the first model generation unit 113 has a difference between the value output from the first output neuron NE11a and "1", and a difference between the value output from the first output neuron NE11b and "0". , And, the value corresponding to each weight in the first learning model is updated so that the difference between the value output from the first output neuron NE11c and “0” becomes smaller.

Further, for example, in the information in which the "item number" is "2" in the first learning data 131 described with reference to FIG. 12, "IMAGEa2" is set as the "image data" and "BBB" is set as the "type". Has been done. Therefore, for example, when the image data whose name is "IMAGEa2" or its feature amount is input from the input neuron NE13, the first model generation unit 113 has a value output from each of the first output neurons NE11 and " The value corresponding to each weight in the first learning model is updated so that the difference from the correct answer value (“0”, “1”, and “0”) corresponding to “BBB” becomes small. Specifically, in this case, the first model generation unit 113 has a difference between the value output from the first output neuron NE11a and "0", and a difference between the value output from the first output neuron NE11b and "1". , And, the value corresponding to each weight in the first learning model is updated so that the difference between the value output from the first output neuron NE11c and “0” becomes smaller.

The first model generation unit 113 uses the output value from the first output neuron NE11 as the output value from the second output neuron NE12 before comparing the output value from the first output neuron NE11 with the correct answer value. It may be the one that makes a correction.

Specifically, in this case, the first model generation unit 113 corrects so that the closer the output value from the second output neuron NE12 is to "0", the closer the output value from the first output neuron NE11 is to the correct answer value. conduct. On the other hand, the first model generation unit 113 reduces the correction width of the output value from the first output neuron NE11 as the output value from the second output neuron NE12 is closer to "1". Further, in this case, the first model generation unit 113 sets a penalty value of a larger value as the correction width of the output value from the first output neuron NE11 is larger. Then, the first model generation unit 113 sets each weight in the first learning model so that both the difference between the output value (corrected value) from the first output neuron NE11 and the correct answer value and the penalty value become small. Update the corresponding value.

[Second learning data storage processing]
Next, among the information output processes, a process of storing the second learning data 132 (hereinafter, also referred to as a second learning data storage process) will be described. FIG. 7 is a flowchart illustrating the second learning data storage process.

As shown in FIG. 7, the information receiving unit 111 waits until, for example, receives a plurality of second learning data 132 transmitted from the operation terminal 3 (NO in S31). That is, the information receiving unit 111 waits until, for example, the second learning data 132 generated by the operator is transmitted via the operation terminal 3.

Then, when a plurality of first learning data 131s are received (YES in S31), the information management unit 112 stores the plurality of second learning data 132s received in the process of S31 in the information storage area 130 (S32). Hereinafter, a specific example of the second learning data 132 will be described.

[Specific example of the second learning data]
FIG. 14 is a diagram illustrating a specific example of the second learning data 132.

The second learning data 132 shown in FIG. 14 includes a "item number" indicating each identification information of the second learning data 132, an "image data" indicating each learning image data itself, and each learning image data. "Position" indicating the upper left position of the included object (frame including the object), "width" indicating the width of the object (frame including the object) included in each learning image data, and each learning It has an item "height" indicating the height of the object (frame including the object) included in the image data.

Specifically, the second learning data 132 in which the "item number" is "1" in FIG. 14 includes the learning image data whose name is "IMAGEb1", and further, the "position" is "10, 32". Information about the object whose "width" is "10" and "height" is "8", and "position" is "24,30" and "width" is "4". Information about the object whose "height" is "2", and "position" is "28,20", "width" is "8", and "height" is "6". It is learning data including information about an object. That is, the second learning data 132 whose “item number” is “1” in FIG. 14 includes information about three objects included in the learning image data whose name is “IMAGEb1”.

Further, the second learning data 132 in which the "item number" is "2" in FIG. 14 includes the learning image data whose name is "IMAGEb2", and further, the "position" is "34,36". Information about an object whose "width" is "4" and whose "height" is "2", and whose "position" is "10,12" and whose "width" is "12" and " It is learning data including information about an object whose "height" is "6". That is, the second learning data 132 whose "item number" is "2" in FIG. 14 includes information about two objects included in the learning image data whose name is "IMAGEb2". The description of the other second learning data 132 included in FIG. 14 will be omitted.

[Second model generation process]
Next, among the information output processes, a process for generating the second model (hereinafter, also referred to as a second model generation process) will be described. FIG. 8 is a flowchart illustrating the second model generation process.

As shown in FIG. 8, the first model generation unit 113 of the information processing apparatus 1 waits until, for example, the second model generation timing (NO in S41). The second model generation timing may be, for example, the timing at which the operator inputs information to the information processing device 1 to the effect that the second model is generated. Further, the second model generation timing may be, for example, the timing immediately after the first model is generated.

Then, when the second model generation timing is reached (YES in S41), the second model generation unit 114 learns the plurality of second learning data 132 stored in the information storage area 130, thereby performing the second learning. Generate a model (S42). Hereinafter, a specific example of the processing of S42 will be described.

[Specific example of processing of S42]
FIG. 15 is a diagram illustrating a specific example of the process of S42.

As shown in FIG. 15, the second model generation unit 114 generates, for example, a second model MD02 having a plurality of output neurons NE21 corresponding to each of the plurality of divided image data obtained by dividing the learning image data. In the example shown in FIG. 15, the output neuron NE21 includes output neurons NE21a, NE21b, NE21c, NE21d, NE21e, NE21f, NE21g, NE21h, NE21i corresponding to each of the 16 divided image data obtained by dividing the learning image data. , NE21j, NE21k, NE21l, NE21m, NE21n, NE21о and NE21p, respectively.

Then, for example, "IMAGEb3" is set as "image data" for the information in which the "item number" is "3" in the second learning data 132 described with reference to FIG. 14, and the "position", "width", and "width" are set. Information about one object is set as "height". Therefore, the second model generation unit 114 has a name, for example, when the divided image data of the image data whose name is "IMAGEb3" includes an object in the divided image data corresponding to the output neuron NE21d. When the feature amount of the image data of "IMAGEb3" is input from the input neuron NE23, the difference between the output value from the output neuron NE21d and "1" and the output from each of the output neurons NE21 other than the output neuron NE21d. The value corresponding to each weight in the second learning model is updated so that the difference between the value and "0" becomes small.

Further, for example, "IMAGEb2" is set as "image data" for the information in which the "item number" is "2" in the second learning data 132 described with reference to FIG. 14, and the "position", "width", and "width" are set. Information about two objects is set as "height". Therefore, in the second model generation unit 114, for example, when the divided image data of the image data whose name is "IMAGEb2" includes an object in each of the divided image data corresponding to the output neurons NE21f and NE21m. In, when the feature amount of the image data whose name is "IMAGEb2" is input from the input neuron NE23, the difference between the output value from each of the output neurons NE21f and NE21m and "1", and the output neurons other than these. The value corresponding to each weight in the second learning model is updated so that the difference between the output value from each of NE21 and "0" becomes small.

[Main processing of information output processing]
Next, the main process of the information output process will be described. 9 to 11 are flowcharts illustrating the main process of the information output process.

As shown in FIG. 9, the result acquisition unit 115 of the information processing device 1 waits until, for example, the information output timing (NO in S51). The information output timing may be, for example, the timing at which the worker inputs information to the effect that the monitoring of the object at sea is started.

Then, when the information output timing is reached (YES in S51), the result acquisition unit 115 acquires the moving image data 133 (S52).

Subsequently, the result acquisition unit 115 uses the second learning model generated in the process of S42 to add an object to each image data for each of the plurality of image data included in the moving image data 133 acquired in the process of S52. The second accuracy indicating the existing accuracy is acquired (S53). Hereinafter, a specific example of the processing of S53 will be described.

[Specific example of processing of S53]
FIG. 16 is a diagram illustrating a specific example of the processing of S53.

As shown in FIG. 16, the result acquisition unit 115 inputs, for example, the image data IMAGEc1 included in the moving image data 133 into the second learning model MD02. Then, the result acquisition unit 115 acquires, for example, the values output from each of the output neurons NE21 of the second learning model MD02 in response to the input of the image data IMAGEc1.

After that, the result acquisition unit 115 identifies the position of the object included in the image data IMAGEc1 from each of the values acquired from, for example, the second learning model MD02. Specifically, as shown in FIG. 16, the result acquisition unit 115 includes, for example, partial image data IMAGEd1, partial image data IMAGEd2, and partial image data IMAGEd3 as partial image data including an object included in the image data IMAGEc1. Identify the position of.

Further, for example, the result acquisition unit 115 sets the maximum value of the value corresponding to the position of each object among the values output from each of the output neurons NE2 of the second learning model MD02 for each specified object. 2 Obtained as accuracy.

The position and the second accuracy of the object included in the image data IMAGEc1 may be output from the second model MD02. Then, in this case, the result acquisition unit 115 may acquire each information output from the second model MD02.

Returning to FIG. 9, the information management unit 112 generates detection result information 135 including the second accuracy acquired in the process of S53 for each image data included in the moving image data 133 acquired in the process of S52, for example (S54). ). Then, the information management unit 112 stores the generated detection result information 135 in the information storage area 130. Hereinafter, a specific example of the detection result information 135 will be described.

[Specific example of detection result information]
FIG. 17 is a diagram illustrating a specific example of the detection result information 135.

The detection result information 135 shown in FIG. 17 includes a "item number" indicating each information included in the detection result information 135, a "partial image data" indicating identification information of partial image data including each object, and each object. It has a "second accuracy" indicating the second accuracy corresponding to the item.

Specifically, in the detection result information 135 shown in FIG. 17, “IMAGEd1” is set as the “partial image data” and “0. 9 ”is set.

Further, in the detection result information 135 shown in FIG. 17, "IMAGEd2" is set as the "partial image data" and "0.7" is set as the "second accuracy" for the information whose "item number" is "2". Is set. The description of other information included in FIG. 17 will be omitted.

The detection result information 135 may include information indicating the position of the partial image data specified in the process of S53.

Returning to FIG. 9, the information management unit 112 acquires partial image data including an object included in each image data from the image data included in the moving image data 133 acquired in the process of S52 (S55).

Specifically, the information management unit 112 acquires partial image data from each image data included in the moving image data 133 acquired in the process of S52 by referring to the position of the partial image data specified in the process of S53, for example. I do.

When the detection result information 135 includes information indicating the position of the partial image data specified in the process of S53, the information management unit 112 refers to the detection result information 135 stored in the information storage area 130. Partial image data may be acquired.

Then, as shown in FIG. 10, the result acquisition unit 115 uses the first learning model generated in the process of S22 to include the target included in each partial image data for each partial image data acquired in the process of S55. The categorization result of the type of the object and the first accuracy indicating the accuracy of the categorization result are acquired (S61). Hereinafter, a specific example of the processing of S61 will be described.

[Specific example of processing of S61]
FIG. 18 is a diagram illustrating a specific example of the processing of S61.

As shown in FIG. 18, the result acquisition unit 115 inputs, for example, the partial image data IMAGEd1 acquired in the process of S55 into the first learning model MD01. Then, the result acquisition unit 115 acquires, for example, the values output from the output neurons NE11 and the output neurons NE12 of the first learning model MD01 in response to the input of the partial image data IMAGEd1.

Specifically, as shown in FIG. 18, the result acquisition unit 115 has an accuracy of "0.6" that the type of the object included in the partial image data IMAGEd1 is "AAA", and is "BBB". Information indicating that a certain accuracy is "0.3" and the accuracy of "CCC" is "0.1" is acquired as a classification result.

Further, as shown in FIG. 18, the result acquisition unit 115 acquires "0.9" as the first accuracy of the classification result for the type of the object included in the partial image data IMAGEd1, for example.

Returning to FIG. 10, the information management unit 112 generates, for example, the classification result information 134 including the classification result acquired in the processing of S61 and the first accuracy for each partial image data acquired in the processing of S55 (S62). Then, the information management unit 112 stores the generated classification result information 134 in the information storage area 130. Hereinafter, a specific example of the classification result information 134 will be described.

[Specific examples of categorized result information]
FIG. 19 is a diagram illustrating a specific example of the classification result information 134.

The classification result information 134 shown in FIG. 19 includes a "item number" indicating each information included in the classification result information 134, a "partial image data" indicating identification information of partial image data including each object, and each object. The items include a "classification result" indicating the categorization result of each type of the object and a "first probability" indicating the first probability of the categorization result for each type of object.

Specifically, in the classification result information 134 shown in FIG. 19, "IMAGEd1" is set as "partial image data" for the information in which the "item number" is "1". Further, in the classification result information 134 shown in FIG. 19, the information in which the "item number" is "1" has the types of objects included in the partial image data IMAGEd1 as "classification results" of "AAA" and "BBB". The information that the accuracy of each of "" and "CCC" is "0.6", "0.3", and "0.1" is set. Further, in the classification result information 134 shown in FIG. 19, "0.8" is set as the "first accuracy" for the information in which the "item number" is "1".

Further, in the classification result information 134 shown in FIG. 19, "IMAGEd2" is set as "partial image data" for the information in which the "item number" is "2". Further, in the classification result information 134 shown in FIG. 19, in the information in which the "item number" is "2", the types of objects included in the partial image data IMAGEd2 are "AAA" and "BBB" as the "classification result". The information that the accuracy of each of "" and "CCC" is "0.2", "0.7", and "0.1" is set. Further, in the classification result information 134 shown in FIG. 19, "0.9" is set as the "first accuracy" for the information in which the "item number" is "2". Description of the other information contained in FIG. 19 will be omitted.

Returning to FIG. 10, the second condition determination unit 117 of the information processing apparatus 1 determines whether or not the classification result corresponding to each partial image data satisfies the first condition for each partial image data acquired in the process of S55. , It is determined whether or not the first accuracy corresponding to each partial image data satisfies the second condition (S63).

Specifically, the first condition determination unit 116, for example, when the classification result indicates that each object is an object having a predetermined attribute (for example, a warship), the classification result corresponding to each object. Is determined to satisfy the first condition. Further, the first condition determination unit 116 determines that, for example, when the first accuracy corresponding to each object is larger than the first threshold value, the first accuracy corresponding to each object satisfies the second condition. ..

Then, the result output unit 118 of the information processing apparatus 1 is included in each of the partial image data acquired in the process of S55 and which is determined that the classification result acquired in the process of S61 satisfies the first condition. An alert about the object and the object included in each of the partial image data determined that the first accuracy acquired in the process of S61 satisfies the second condition is output to the operation terminal 3 (S64). Hereinafter, a specific example of the processing of S64 will be described.

[Specific example of S64 processing]
FIG. 20 is a diagram illustrating a specific example of the processing of S64.

Of the information shown in FIG. 20, the information whose "item number" is "1" is alerted when the first accuracy is higher than the first threshold value and the object needs to be monitored. It is shown that. Further, in the information shown in FIG. 20 in which the "item number" is "3", if the first accuracy is lower than the first threshold value and the object needs to be monitored, an alert output is output. Indicates that it will be done. Further, in the information shown in FIG. 20 in which the "item number" is "4", if the first accuracy is lower than the first threshold value and the object does not require monitoring, an alert is output. It shows that it will be done.

On the other hand, in the information shown in FIG. 20 in which the "item number" is "2", if the first accuracy is higher than the first threshold value and the object does not require monitoring, an alert is output. It shows that it will not be broken.

That is, it is highly possible that an object having a high first accuracy and does not require monitoring is an object that the worker does not need to detect (an object that the worker does not need to deal with). It can be judged that it is expensive. Therefore, when the result output unit 118 detects an object having a high first accuracy and does not require monitoring, the result output unit 118 does not notify the operator (output of an alert).

This makes it possible for the information processing device 1 to reduce the work load associated with monitoring an object at sea, for example.

Further, as shown in FIG. 11, the first condition determination unit 116 of the information processing device 1 satisfies the second condition with the first accuracy corresponding to each partial image data for each partial image data acquired in the process of S55. It is determined whether or not and whether or not the second accuracy corresponding to each partial image data satisfies the third condition (S71).

Specifically, the second condition determination unit 117 states that, for example, when the second accuracy corresponding to each object is larger than the second threshold value, the second accuracy corresponding to each object satisfies the third condition. judge.

Then, in the result output unit 118, among the partial image data acquired in the process of S55, the first accuracy acquired in the process of S61 does not satisfy the second condition, and the second accuracy acquired in the process of S53 is the second. Information indicating that the object included in the partial image data satisfying the three conditions is unlearned is output (S72). Hereinafter, a specific example of the processing of S72 will be described.

[Specific example of processing of S72]
FIG. 21 is a diagram illustrating a specific example of the process of S72. In the graph shown in FIG. 21, the horizontal axis corresponds to the first accuracy and the vertical axis corresponds to the second accuracy.

The graph shown in FIG. 21 shows, for example, that an object whose first accuracy exceeds the first threshold value is determined to be an object that has already been learned in the first learning model and the second learning model.

Further, in the graph shown in FIG. 21, for example, it is determined that an object having a first accuracy lower than the first threshold value is an object that may not be learned in the first learning model and the second learning model. Is shown.

Further, in the graph shown in FIG. 21, an object whose first accuracy is lower than the first threshold value and whose second threshold value is lower than the second threshold value is a learned object, but noise and the like included in the moving image data 133. It is shown that it is judged that the object is difficult to judge the type due to the influence of. Further, the graph shown in FIG. 21 shows that an object having a first accuracy lower than the first threshold value and a second threshold value exceeding the second threshold value is determined to be an unlearned object. ..

That is, it can be determined that an object having a low first accuracy and a high second accuracy is extremely likely to be an object that has not been learned even in the first learning model and the second learning model. Therefore, when the result output unit 118 has an object having a low first accuracy and a high second accuracy, the result output unit 118 identifies the object as an unlearned object.

As a result, the information processing apparatus 1 can classify each of the objects having a low first accuracy into either a learned object or an unlearned object. Then, for example, the worker can improve the accuracy of each learning model by having the first learning model and the second learning model learn only the object specified as the unlearned object.

As described above, the information processing apparatus 1 in the present embodiment uses the first learning model to obtain categorized results for each type of the object included in the plurality of image data constituting the moving image data 133, and the categorized results thereof. Obtain the first accuracy of the categorization result. Then, the information processing device 1 determines whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy for each object included in the plurality of image data. Make a judgment. After that, the information processing device 1 outputs information indicating that, among the objects included in the plurality of image data, the object determined to satisfy a predetermined condition is the monitoring target.

That is, for example, the type corresponding to the classification result output by the first learning model is an object that is not included in the type of the monitoring target (for example, the predetermined type of the monitoring target), and the first learning model. When there is an object whose accuracy of the classification result output by is higher than a predetermined threshold value (for example, a predetermined threshold value), the information processing apparatus 1 does not need to monitor the existing object. to decide. On the other hand, for example, an object in which the type corresponding to the classification result output by the first learning model is included in the type of the monitoring target, or the accuracy of the classification result output by the first learning model is higher than a predetermined threshold value. When a low object exists, the information processing apparatus 1 determines that it is necessary to monitor the existing object.

As a result, the information processing device 1 can identify an object that the operator needs to visually monitor by using the first learning model. Therefore, the information processing device 1 can suppress the burden on the operator due to the monitoring of the object.

The above embodiments can be summarized as follows.

(Appendix 1)
Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
For each object included in the plurality of images, it is determined whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy.
Among the objects included in the plurality of images, information indicating that the object determined to satisfy the predetermined condition is the monitoring target is output.
An information output program characterized by having a computer execute processing.

(Appendix 2)
In Appendix 1, further
Prior to the process of acquiring the classification result and the first accuracy, a plurality of first learnings including a learning image including an object and information indicating a type candidate of the object included in the learning image. By training the data, the first training model is generated.
An information output program characterized by having a computer execute processing.

(Appendix 3)
In Appendix 1,
In the process of performing the determination, for each object included in the plurality of images, whether or not the classification result corresponding to each object satisfies the first condition, and the first accuracy corresponding to each object. Determines if the second condition is met,
In the output process, among the objects included in the plurality of images, the object for which the classification result is determined to satisfy the first condition and the object for which the first accuracy is determined to satisfy the second condition are satisfied. Output an alert about and
An information output program characterized by this.

(Appendix 4)
In Appendix 3,
In the process of performing the determination, when the classification result indicates that each object is an object having a predetermined attribute, the classification result corresponding to each object satisfies the first condition. judge,
An information output program characterized by this.

(Appendix 5)
In Appendix 3,
In the process of performing the determination, when the first accuracy corresponding to each object is larger than the first threshold value, it is determined that the first accuracy corresponding to each object satisfies the second condition.
An information output program characterized by this.

(Appendix 6)
In Appendix 3, further
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
Let the computer perform the process
In the process of performing the determination, for each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the second condition, and the object corresponds to each object. It is determined whether or not the second accuracy is satisfied with the third condition.
In the output process, among the objects included in the plurality of images, the objects whose first accuracy does not satisfy the second condition and whose second accuracy satisfies the third condition are unlearned. Output information indicating that there is,
An information output program characterized by this.

(Appendix 7)
In Appendix 6, further
Prior to the process of acquiring the second accuracy, a plurality of second learning data including the learning image including the object and the position information of the object in the learning image are learned, thereby performing the second learning. 2 Generate a learning model,
An information output program characterized by having a computer execute processing.

(Appendix 8)
In Appendix 6,
In the process of performing the determination, when the second accuracy corresponding to each object is larger than the second threshold value, it is determined that the second accuracy corresponding to each object satisfies the third condition.
An information output program characterized by this.

(Appendix 9)
Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the condition, and the second accuracy corresponding to each object satisfies other conditions. Determine if it meets or not,
Among the objects included in the plurality of images, information indicating that the object whose first accuracy does not satisfy the above condition and whose second accuracy satisfies the other condition is unlearned is output. ,
An information output program characterized by having a computer execute processing.

(Appendix 10)
The information acquisition unit that acquires moving images and
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. A result acquisition unit that acquires the categorized results for each type and the first accuracy indicating the accuracy of the categorized results.
A first condition for determining whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy for each object included in the plurality of images. Judgment unit and
Among the objects included in the plurality of images, the result output unit that outputs information indicating that the object determined to satisfy the predetermined condition is the monitoring target is provided.
An information output device characterized by the fact that.

(Appendix 11)
In Appendix 10,
The first condition determination unit determines whether or not the classification result corresponding to each object satisfies the first condition for each object included in the plurality of images, and the first condition corresponding to each object. Determine if the accuracy meets the second condition,
Among the objects included in the plurality of images, the result output unit is an object for which the classification result is determined to satisfy the first condition and an object for which the first accuracy is determined to satisfy the second condition. Output an alert about and
An information output device characterized by the fact that.

(Appendix 12)
In Appendix 11,
By using the second learning model in which the position information of the object in the image is learned, the result acquisition unit acquires the second accuracy indicating the probability that the object exists in each of the plurality of images, and further,
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the second condition, and the second accuracy corresponding to each object is the second. It has a second condition determination unit that determines whether or not the three conditions are satisfied.
Among the objects included in the plurality of images, the result output unit has not learned the objects whose first accuracy does not satisfy the second condition and whose second accuracy satisfies the third condition. Output information indicating that there is,
An information output device characterized by the fact that.

(Appendix 13)
The information acquisition unit that acquires moving images and
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. By acquiring the categorization result for each type and the first accuracy indicating the accuracy of the categorization result, and further using the second learning model in which the position information of the object in the image is learned, the plurality of images are described. The result acquisition unit that acquires the second accuracy, which indicates the probability that the object exists in each,
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the condition, and the second accuracy corresponding to each object satisfies other conditions. A second condition determination unit that determines whether or not the condition is satisfied, and
Among the objects included in the plurality of images, information indicating that the object whose first accuracy does not satisfy the above condition and whose second accuracy satisfies the other condition is unlearned is output. Has a result acquisition unit,
An information output device characterized by the fact that.

(Appendix 14)
Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
For each object included in the plurality of images, it is determined whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy.
Among the objects included in the plurality of images, information indicating that the object determined to satisfy the predetermined condition is the monitoring target is output.
An information output method characterized by having a computer execute processing.

(Appendix 15)
In Appendix 14,
In the process of performing the determination, for each object included in the plurality of images, whether or not the classification result corresponding to each object satisfies the first condition, and the first accuracy corresponding to each object. Determines if the second condition is met,
In the output process, among the objects included in the plurality of images, the object for which the classification result is determined to satisfy the first condition and the object for which the first accuracy is determined to satisfy the second condition are satisfied. Output an alert about and
An information output method characterized by the fact that.

(Appendix 16)
In Appendix 15, further
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
Let the computer perform the process
In the process of performing the determination, for each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the second condition, and the object corresponds to each object. It is determined whether or not the second accuracy is satisfied with the third condition.
In the output process, among the objects included in the plurality of images, the objects whose first accuracy does not satisfy the second condition and whose second accuracy satisfies the third condition are unlearned. Output information indicating that there is,
An information output method characterized by the fact that.

(Appendix 17)
Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the condition, and the second accuracy corresponding to each object satisfies other conditions. Determine if it meets or not,
Among the objects included in the plurality of images, information indicating that the object whose first accuracy does not satisfy the above condition and whose second accuracy satisfies the other condition is unlearned is output. ,
An information output method characterized by having a computer execute processing.

1: Information processing device 3: Operation terminal 10: Information processing system NW: Network

Claims (13)

Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
For each object included in the plurality of images, it is determined whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy.
Among the objects included in the plurality of images, information indicating that the object determined to satisfy the predetermined condition is the monitoring target is output.
An information output program characterized by having a computer execute processing. In claim 1, further
Prior to the process of acquiring the classification result and the first accuracy, a plurality of first learnings including a learning image including an object and information indicating a type candidate of the object included in the learning image. By training the data, the first training model is generated.
An information output program characterized by having a computer execute processing. In claim 1,
In the process of performing the determination, for each object included in the plurality of images, whether or not the classification result corresponding to each object satisfies the first condition, and the first accuracy corresponding to each object. Determines if the second condition is met,
In the output process, among the objects included in the plurality of images, the object for which the classification result is determined to satisfy the first condition and the object for which the first accuracy is determined to satisfy the second condition are satisfied. Output an alert about and
An information output program characterized by this. In claim 3,
In the process of performing the determination, when the classification result indicates that each object is an object having a predetermined attribute, the classification result corresponding to each object satisfies the first condition. judge,
An information output program characterized by this. In claim 3,
In the process of performing the determination, when the first accuracy corresponding to each object is larger than the first threshold value, it is determined that the first accuracy corresponding to each object satisfies the second condition.
An information output program characterized by this. In claim 3, further
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
Let the computer perform the process
In the process of performing the determination, for each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the second condition, and the object corresponds to each object. It is determined whether or not the second accuracy is satisfied with the third condition.
In the output process, among the objects included in the plurality of images, the objects whose first accuracy does not satisfy the second condition and whose second accuracy satisfies the third condition are unlearned. Output information indicating that there is,
An information output program characterized by this. In claim 6, further
Prior to the process of acquiring the second accuracy, a plurality of second learning data including the learning image including the object and the position information of the object in the learning image are learned, thereby performing the second learning. 2 Generate a learning model,
An information output program characterized by having a computer execute processing. In claim 6,
In the process of performing the determination, when the second accuracy corresponding to each object is larger than the second threshold value, it is determined that the second accuracy corresponding to each object satisfies the third condition.
An information output program characterized by this. Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the condition, and the second accuracy corresponding to each object satisfies other conditions. Determine if it meets or not,
Among the objects included in the plurality of images, information indicating that the object whose first accuracy does not satisfy the above condition and whose second accuracy satisfies the other condition is unlearned is output. ,
An information output program characterized by having a computer execute processing. The information acquisition unit that acquires moving images and
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. A result acquisition unit that acquires the categorized results for each type and the first accuracy indicating the accuracy of the categorized results.
A first condition for determining whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy for each object included in the plurality of images. Judgment unit and
Among the objects included in the plurality of images, the result output unit that outputs information indicating that the object determined to satisfy the predetermined condition is the monitoring target is provided.
An information output device characterized by the fact that. The information acquisition unit that acquires moving images and
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. By acquiring the categorization result for each type and the first accuracy indicating the accuracy of the categorization result, and further using the second learning model in which the position information of the object in the image is learned, the plurality of images are described. The result acquisition unit that acquires the second accuracy, which indicates the probability that the object exists in each,
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the condition, and the second accuracy corresponding to each object satisfies other conditions. A second condition determination unit that determines whether or not the condition is satisfied, and
Among the objects included in the plurality of images, information indicating that the object whose first accuracy does not satisfy the above condition and whose second accuracy satisfies the other condition is unlearned is output. Has a result acquisition unit,
An information output device characterized by the fact that. Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
For each object included in the plurality of images, it is determined whether or not each object satisfies a predetermined condition based on the classification result corresponding to each object and the first accuracy.
Among the objects included in the plurality of images, information indicating that the object determined to satisfy the predetermined condition is the monitoring target is output.
An information output method characterized by having a computer execute processing. Get a moving image,
By using the first learning model in which the type candidate of the object included in the image and the accuracy that the type of the object is the type candidate are learned, the object included in the plurality of images constituting the moving image is used. Obtain the categorization result for each type and the first accuracy indicating the accuracy of the categorization result.
By using the second learning model in which the position information of the object in the image is learned, the second accuracy indicating the probability that the object exists in each of the plurality of images is acquired.
For each object included in the plurality of images, it is determined whether or not the first accuracy corresponding to each object satisfies the condition, and the second accuracy corresponding to each object satisfies other conditions. Determine if it meets or not,
Among the objects included in the plurality of images, information indicating that the object whose first accuracy does not satisfy the above condition and whose second accuracy satisfies the other condition is unlearned is output. ,
An information output method characterized by having a computer execute processing.

Images