WO2024079902A1

WO2024079902A1 - Processing control system, processing control device, and processing control method

Info

Publication number: WO2024079902A1
Application number: PCT/JP2022/038457
Authority: WO
Inventors: 浩一二瓶; 昌治森本; 勇人逸身; フロリアンバイエ; 孝法岩井; 誠也柴田
Original assignee: 日本電気株式会社
Priority date: 2022-10-14
Filing date: 2022-10-14
Publication date: 2024-04-18

Abstract

A processing control system (100) includes: a selection means (101) that selects, for each item of data to be analyzed acquired by first processing units, a method for sharing the analysis of the item of data to be analyzed according to the calculation capacity of each of the first processing units; and a processing control means (102) that controls the first processing units and a second processing unit according to the result of selection.

Description

Processing control system, processing control device, and processing control method

The present invention relates to a processing control system, a processing control device, and a processing control method.

Technology is being used to analyze data to be analyzed acquired by cameras, sensors, etc., and to confirm targets such as people and objects and their movements, for example from a remote location. For example, Patent Document 1 describes a system consisting of one or more image sensors and one or more operation terminals, in which field operators each use a different operation terminal to access the image sensors and view images of targets acquired by the image sensors and the processing results analyzed by the image sensors.

International Publication No. WO2021/161647

However, when the amount of data to be analyzed increases due to improvements in camera resolution, or when the calculation costs increase due to more advanced analysis content, it may not be possible to perform the analysis adequately. Therefore, there is a demand for technology to control the analysis process in order to perform the analysis efficiently. Patent Document 1 describes how the image sensor adaptively changes the resolution of the image sent to the operation terminal depending on the performance of the operation terminal, but does not describe technology to control the analysis process.

One aspect of the present invention has been made in consideration of the above problems, and one example of its objective is to provide a process control system, a process control device, and a process control method that can control an analysis process in order to perform an analysis efficiently.

A process control system according to one aspect of the present invention is a process control system that controls one or more first processing units each acquiring one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing the analysis of the analysis target data with each of the first processing units, and includes a selection means that selects a sharing method for the analysis of the analysis target data for each of the one or more pieces of analysis target data acquired by each of the first processing units in accordance with the computing capacity of the first processing unit, and a process control means that controls each of the first processing units and the second processing units so that analysis is performed for each piece of analysis target data using the sharing method selected by the selection means.

A processing control device according to one aspect of the present invention is a processing control device that controls one or more first processing units that each acquire one or more pieces of analysis target data, and a second processing unit that is capable of communicating with each of the first processing units and shares the analysis of the analysis target data with each of the first processing units, and is equipped with a selection unit that selects a sharing method for the analysis of the analysis target data for each of the one or more pieces of analysis target data acquired by each of the first processing units in accordance with the computing capacity of the first processing unit, and a processing control unit that controls each of the first processing units and the second processing units so that analysis is performed for each piece of analysis target data using the sharing method selected by the selection unit.

A process control method according to one aspect of the present invention is a process control method for controlling one or more first processing units, each of which acquires one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing the analysis of the analysis target data with each of the first processing units, and for each of the one or more pieces of analysis target data acquired by each of the first processing units, a sharing method for the analysis of the analysis target data is selected according to the computing power of the first processing unit, and each of the first processing units and the second processing unit are controlled to perform analysis of each piece of analysis target data using the selected sharing method.

According to one aspect of the present invention, the analysis process can be controlled to perform the analysis efficiently.

1 is a block diagram showing an example of the configuration of a process control system according to a first embodiment. 1 is a block diagram showing an example of the configuration of a processing system controlled by a processing control system. FIG. 2 is a flowchart showing an example of the flow of a process control method according to the first embodiment. 2 is a block diagram showing an example of the configuration of a processing control device according to the first embodiment; FIG. 11 is a block diagram showing an example of the configuration of a process control system and a processing system according to a second embodiment. FIG. 13 is a schematic diagram illustrating an example of extraction of analysis target data. 11 is a table showing an example of information referenced for selecting a sharing method. FIG. 13 is a block diagram showing an example of the configuration of a process control system and a processing system according to a third embodiment. FIG. 13 is a block diagram showing an example of the configuration of a process control system and a processing system according to a fourth embodiment. FIG. 13 is a block diagram showing an example of the configuration of a process control system and a processing system according to a fifth embodiment. FIG. 1 is a block diagram illustrating an example of the configuration of a computer.

First Embodiment
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A first embodiment of the present invention will be described in detail with reference to the accompanying drawings. This embodiment is a basic form of the embodiments described below.

(Configuration of the Processing Control System)
The configuration of a process control system according to this embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of a process control system 100 according to a first embodiment. The process control system 100 includes a selection unit 101 and a process control unit 102, and controls the processing system.

FIG. 2 is a block diagram showing an example configuration of a processing system controlled by a processing control system. The processing system 1 includes one or more first processing units 20 and a second processing unit 30. For ease of viewing, FIG. 2 shows a configuration with one first processing unit 20, but there may be multiple first processing units 20.

Each of the first processing units 20 is connected to, for example, a camera or a sensor such as LiDAR (Light Detection and Ranging), and acquires one or more pieces of analysis target data from the camera or sensor. In one example, the analysis target data may be video data captured by a camera. It is sufficient for the video data to include the analysis target within the field of view of the video. The analysis target may be, for example, a worker (person) working at a construction site, work equipment (object), and the behavior (movement) of the worker and work equipment. The analysis target data may also be sensing data from a sensor that detects the analysis target.

The first processing unit 20 may also be connected to multiple cameras, sensors, etc., and acquire multiple pieces of analysis target data. The first processing unit 20 may also acquire multiple pieces of analysis target data from a single camera, sensor, etc. In this case, the first processing unit 20 may acquire multiple pieces of analysis target data by extracting the multiple pieces of analysis target data from data acquired from a single camera, sensor, etc.

The first processing unit 20 and the second processing unit 30 may each be configured with one or more computers. The first processing unit 20 and the second processing unit 30 are capable of communicating via a network NW, and share the analysis processing of the data to be analyzed. The network NW may be wireless or wired, and if wireless, may be a wireless communication system such as Wi-Fi, LTE, 4G, or 5G.

In one embodiment, the first processing unit 20 may be an edge processing unit, and the second processing unit 30 may be a cloud processing unit. In this specification, "edge" refers to a place where data is collected. The first processing unit 20, which is an edge processing unit, is an information processing device (computer) or a group of information processing devices installed at or around the location where the analysis target is present (e.g., a construction site, a factory, etc.), and acquires video data from the imaging device 10 installed at the location where the analysis target is present. The first processing unit 20 may be integrated with a camera, a sensor, etc. Also, in this specification, "cloud" refers to a place where data is processed, stored, etc. The second processing unit 30, which is a cloud processing unit, may be an information processing device (computer) or a group of information processing devices installed at a location that can provide large computational resources, such as a data center or a server farm. Note that the second processing unit 30 may be a processing unit located at a location connected to the first processing unit 20 via a network, and may be a computational resource connected to a base station such as 5G (e.g., MEC (Multi-access Edge Computing)), or a server installed in an office at the site (on-premises server), etc.

The first processing unit 20 may perform an analysis process on at least a portion of the one or more pieces of analysis target data acquired to generate an analysis result. The first processing unit 20 may also calculate features for at least a portion of the one or more pieces of analysis target data acquired and transmit the calculated features to the second processing unit 30 via the network NW. The first processing unit 20 may also transmit at least a portion of the one or more pieces of analysis target data acquired to the second processing unit 30 via the network NW. When the first processing unit 20 transmits the features or analysis target data to the second processing unit 30, it may compress or encrypt the features or analysis target data before transmitting them to the second processing unit 30, or it may transmit the features or analysis target data to the second processing unit 30 without compressing or encrypting them.

The second processing unit 30 receives the features or data to be analyzed sent from the first processing unit 20, performs restoration processing as necessary, and performs analysis processing.

The analysis process is, for example, detection, identification, tracking, and time series analysis of the analysis target (object, person) based on the analysis target data. A learning model may be used for this analysis process. One or both of the first processing unit 20 and the second processing unit 30 may use the learning model.

The processing control system 100 (selection means 101, processing control means 102) controls the processing system 1 (i.e., the first processing unit 20, the second processing unit 30) to divide the analysis of the data to be analyzed between the first processing unit 20 and the second processing unit 30. Note that the processing control system 100 does not have to cause the processing system 1 to analyze data to be analyzed that is determined not to require analysis.

The analysis of the data to be analyzed can be shared between the first processing unit 20 and the second processing unit 30 in various ways. For example, the first processing unit 20 that acquired the data to be analyzed performs all of the analysis processing of the data to be analyzed, the first processing unit 20 that acquired the data to be analyzed performs a certain amount of analysis processing and the second processing unit 30 performs the remaining analysis processing, or the first processing unit 20 performs the minimum necessary processing such as compression and the second processing unit 30 performs all of the analysis processing of the data to be analyzed. By using these sharing methods appropriately, the analysis processing can be performed efficiently according to the situation.

Here, the selection means 101 selects a sharing method for each of one or more pieces of analysis target data acquired by each first processing unit 20. In detail, the selection means 101 can select a sharing method for the analysis of the analysis target data according to the calculation capacity of the first processing unit 20. For example, the selection means 101 may select a sharing method such that the higher the calculation capacity of the first processing unit 20, the more analysis is performed in the first processing unit 20 for the analysis target data acquired by the first processing unit 20. Note that the calculation capacity of the first processing unit 20 is an index of the ability to execute analysis processing, and is based on the calculation resources such as the CPU, GPU, and memory that the first processing unit 20 has.

In general, the second processing unit 30 has a much greater computational capacity than the first processing unit 20. On the other hand, the first processing unit 20 can instantly process the acquired data to be analyzed. Therefore, it can be said that it is efficient overall to have the second processing unit 30 process the analysis processing that the first processing unit 20 cannot process. In other words, how to efficiently share the analysis processing depends on the computational capacity of the first processing unit 20. Therefore, an efficient analysis can be realized by having the selection means 101 select a sharing method according to the computational capacity of the first processing unit 20 that acquired the data to be analyzed.

The processing control means 102 controls each of the first processing units 20 and the second processing units 30 so that each analysis target data is analyzed using the sharing method selected by the selection means 101.

The processing control means 102 may be provided, for example, in the second processing unit 30, and may directly control the second processing unit 30 and may control each first processing unit 20 via communication between the second processing unit 30 and each first processing unit 20. The processing control means 102 may be provided, for example, in a device capable of communicating with the first processing unit 20 and the second processing unit 30, and may control each first processing unit 20 and the second processing unit 30 via communication between the device and each first processing unit 20 and the second processing unit 30. The processing control means 102 may be provided, for example, in each first processing unit 20 and the second processing unit 30, and may directly control each first processing unit 20 and the second processing unit 30. The processing control means 102 may be provided, for example, in each first processing unit 20, and may directly control each first processing unit 20 and the second processing unit 30, and may control the second processing unit 30 via communication between the first processing unit 20 and the second processing unit 30.

As described above, the process control system 100 according to this embodiment selects a sharing method based on the computational capacity of the first processing unit. As a result, the process control system 100 according to this embodiment can efficiently perform analysis processing in the first processing unit 20 and the second processing unit 30.

(Flow of the processing control method)
The flow of the process control method S100 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a flow diagram showing the flow of the process control method S100 according to the first embodiment.

In step S101, the selection means 101 selects a sharing method for each of the one or more pieces of analysis target data acquired by each first processing unit 20. In detail, the selection means 101 selects a sharing method for the analysis of the one or more pieces of analysis target data acquired by each first processing unit 20 in accordance with the computing capacity of the first processing unit.

In step S102, the process control means 102 controls each of the first processing units 20 and the second processing unit 30 to analyze each analysis target data using the sharing method selected by the selection means 101.

As described above, in the process control method S100 according to this embodiment, a sharing method is selected based on the computational capacity of the first processing unit. As a result, the process control system 100 according to this embodiment can efficiently perform analysis processing in the first processing unit 20 and the second processing unit 30.

(Configuration of Processing Control Device)
The configuration of the process control device 200 according to this embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the process control device 200 according to the first embodiment. The process control device 200 has a selection unit 201 and a process control unit 202, and controls the processing system 1 (each of the first processing units 20 and the second processing unit 30).

The selection unit 201 has a function equivalent to the selection means 101, and selects a sharing method for each of one or more analysis target data acquired by each first processing unit 20 according to the calculation capacity of the first processing unit 20. The process control unit 202 has a function equivalent to the process control means 102, and controls each first processing unit 20 and second processing unit 30 so that each analysis target data is analyzed using the sharing method selected by the selection unit 201.

The selection unit 201 and the processing control unit 202 may be a computer device in which processing is performed by a processor executing a program stored in a memory. For example, the selection unit 201 and the processing control unit 202 may be a single computer device, or may be a computer device group in which multiple computer devices operate in cooperation with each other, or a server device group in which multiple server devices operate in cooperation with each other. The processing control device 200 can provide the same effects as the processing control system 100.

Second Embodiment
A second embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first embodiment are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

FIG. 5 is a block diagram showing an example configuration of a processing control system 100 and a processing system 1 according to the second embodiment. The processing control system 100 controls the processing system 1. The processing system 1 according to this embodiment includes one or more first processing units 20 and a second processing unit 30 connected to the one or more first processing units 20 via a network NW. The first processing unit 20 is connected to one or more cameras 10.

The camera 10 is an imaging device that captures the subject of analysis and generates video data. As described above, the subject of analysis may be, for example, a worker (person), work equipment (object), and the behavior (movement) of the worker and work equipment working at a construction site, and the camera 10 needs only to be installed so as to be able to capture these. One or more cameras 10 are connected to each first processing unit 20, and one camera 10 is basically connected to only one first processing unit 20.

(First Processing Section)
The first processing unit 20 includes an input unit 21, a communication unit 22, and a main control unit 23. Video data captured by each camera 10 connected to the first processing unit 20 is input to the input unit 21. The communication unit 22 exchanges data with the second processing unit 30 and the processing control system 100 via the network NW.

The main control unit 23 includes an analysis target data acquisition unit 230, a feature calculation unit 231, an analysis unit 232, and an encoding unit 233.

The analysis target data acquisition unit 210 acquires one or more pieces of analysis target data. The analysis target data acquisition unit 210 may acquire the entirety of each piece of video data input to the input unit 21 as the analysis target data, but may also acquire one or more areas extracted from each piece of video data as the analysis target data, as described below.

In one aspect, the analysis target data acquisition unit 210 extracts one or more regions from the video data input to the input unit 21, and acquires each region as analysis target data. There are no particular limitations on the method for extracting one or more regions from the video data, but for example, the analysis target data acquisition unit 210 may perform object detection on the video data and acquire the region in which the object is detected as analysis target data. At this time, the analysis target data acquisition unit 210 may perform object detection using a learning model for object detection. Furthermore, the analysis target data acquisition unit 210 may divide the video data into a predetermined grid, and acquire each divided region as analysis target data.

FIG. 6 is a diagram showing an example of analysis target data acquired by the analysis target data acquisition unit 210. In the example shown in FIG. 6, the analysis target data acquisition unit 210 acquires areas detected by object detection for a frame image F of video data as analysis target data T1 and T2. The areas may be selected to include multiple objects, such as T2.

The feature calculation unit 231 calculates the feature of the analysis target data. The method of calculating the feature of the analysis target data is not particularly limited, but in one embodiment, the feature calculation unit 231 may calculate the feature from the analysis target data using a learning model having a convolutional layer. In one embodiment, when the learning model used by the analysis target data acquisition unit 210 for object detection outputs object identification result information (class information), the identification result information may be used as the feature. In addition, tracking of the detected or identified object may be performed, and an identifier indicating whether or not the object is the same as the object shown in the previous or next frame image may be added to the feature.

The analysis unit 232 analyzes the data to be analyzed and generates an analysis result. In this specification, "analysis" means detecting that an event to be detected has occurred in the analysis target, or that an object to be detected is present. For example, if the analysis targets are workers (people), work equipment (objects), and the behavior (motion) of the workers and work equipment working at a construction site, the analysis results may include the occurrence of events such as inefficient work, procedural errors, and dangerous behavior, or the detection result of the presence of a specific object such as a specific worker or specific work equipment. For example, the analysis target data T2 shown in Figure 6 may be analyzed as a dangerous behavior of approaching heavy machinery.

The analysis unit 232 can perform the analysis, for example, using a learning model that has been trained in advance to output an analysis result. The learning method of the learning model for analysis is not particularly limited, but for example, a pair of an event to be detected and analysis target data that indicates the analysis target when the event occurs may be trained as teacher data, or reinforcement learning may be performed to give a reward when the event to be detected is detected. Furthermore, as an input to the learning model, the analysis target data or the feature calculated by the feature calculation unit 231 may be used.

The encoding unit 233 performs processes such as compression and encryption on the data (data to be analyzed, features, etc.) sent via the communication unit 22.

(Second Processing Section)
The second processing unit 30 includes a communication unit 31 and a main control unit 32. The communication unit 31 exchanges data with each of the first processing units 20 and the processing control system 100 via the network NW.

The main control unit 32 includes a decoding unit 320, an analysis unit 321, and an output unit 322.

The decoding unit 320 performs processing such as expansion and decryption on the data received via the communication unit 31, and provides the data to the analysis unit 321 or the output unit 322.

The analysis unit 321 analyzes the analysis target data or feature quantities received via the communication unit 31, and generates an analysis result. Like the analysis unit 232, the analysis unit 321 can perform analysis using, for example, a learning model that has been trained in advance to output an analysis result. The learning model used by the analysis unit 321 may be the same as the learning model used by the analysis unit 232, or it may be different.

The output unit 322 performs processing according to the analysis results of the analysis unit 321 or the analysis results of the analysis unit 232 received via the communication unit 31. For example, the output unit 322 may notify a predetermined notification destination of the above-mentioned analysis results via the communication unit 31, or may send a warning signal to the predetermined notification destination via the communication unit 31 if the above-mentioned analysis results indicate a predetermined event.

Furthermore, this analysis result (work content) may be displayed, for example, on a terminal held by a supervisor (as an example, a site supervisor) or on a large display at a monitoring center, together with the analyzed video, via communication from the first processing unit 20 or the second processing unit 30. As a result, the supervisor can check the video of the work site together with the work analysis results, accurately grasp the status of the work, and give accurate instructions to the site.

The network NW may be a wireless or wired network. The communication bandwidth in the network NW may be pre-allocated to each first processing unit 20. The network NW may also detect or predict the communication quality between each first processing unit 20 and the second processing unit 30, and provide communication quality information indicating the communication quality to the processing control system 100.

(Processing Control System)
The process control system 100 includes a selection unit 101, a process control unit 102, and a communication unit 103, and controls the processing system 1. The communication unit 103 exchanges data with each of the first processing units 20 and the second processing units 30 via the network NW.

The selection means 101 includes an importance determination means 1010, a processing cost calculation means 1011, and a communication quality information determination means 1012, and as described above, selects a sharing method for each of one or more pieces of analysis target data acquired by each first processing unit 20. Details of how the selection means 101 selects the sharing method will be described later.

The importance determination means 1010 determines the importance of each analysis target data. In this specification, "importance" means the necessity of performing an analysis, and an event or object that is likely to occur or exist as a target of the analysis may be determined to be of high importance. "Importance" can also be expressed as "attention" or "necessity of attention". Events with high importance include, but are not limited to, actions according to a process, actions that are different from a process, and actions that are highly dangerous. Objects with high importance include, but are not limited to, people and heavy machinery. The importance may also be determined based on whether or not they can be detected. For example, the importance of a person or object that is very small in the image and difficult to detect may be reduced. The method of expressing importance is not particularly limited, and may be expressed, for example, as two values of "0" (low importance) and "1" (high importance), or as a multi-value of three or more values (for example, high, medium, low), or as a continuous numerical value.

The importance determination means 1010 may determine that the importance is high when the action indicated by the analysis target data indicates the above-mentioned event, for example, but may also perform the determination using a learning model. For example, the importance determination means 1010 may determine the importance using a learning model that inputs the feature amount of each analysis target data and outputs the importance. In this case, the feature amount of each analysis target data may be calculated by the feature amount calculation unit 231 of the first processing unit 20, and the importance determination means 1010 may acquire the feature amount of each analysis target data from the first processing unit 20 via the communication means 103. The learning method of the learning model that inputs the feature amount of each analysis target data and outputs the importance is not particularly limited, but learning may be performed using analysis target data for learning labeled with importance as teacher data. Also, the importance determination means 1010 may determine the importance using a learning model that inputs video data and outputs the importance. In this case, the importance determination means 1010 may obtain the analysis target data itself from the first processing unit 20 as video data, but may also obtain from the first processing unit 20 data in which the frame rate of the video of the analysis target data has been lowered or the image quality has been reduced in order to reduce the amount of communication.

Also, for example, the importance determination means 1010 may determine the importance of each part by inputting input data in which the feature amounts of each part of the data to be analyzed are calculated and each feature amount is combined into the trained model. The trained model used may receive input data in which the feature amounts of each part are combined, generate relationship information indicating the relationship between the feature amounts of each part based on the input data, and output the importance of each area based on the relationship information and the input data. In one aspect, the relationship information indicates the degree to which areas other than the area are related to the importance of each area. In other words, the relationship information indicates the relationship between areas such that the relationship is large for areas necessary for determining the importance of the area and small for areas not necessary for determining the importance of a specific area. Such relationship information includes, for example, attention weights used in attention mechanisms such as self-attention mechanisms. The trained model includes, for example, one or more layers that generate relationship information based on input data, and one or more layers that generate the importance of each region based on the relationship information and the input data. The trained model can be trained, for example, by reinforcement learning using training input images labeled with an analysis result and an analysis engine that analyzes the input images using the importance.

The processing cost calculation means 1011 calculates or predicts the calculation cost in the first processing unit 20 and the communication cost between the first processing unit 20 and the second processing unit 30 when each analysis target data is analyzed by each sharing method. The calculation cost in the first processing unit 20 is, for example, a value indicating the ratio of the calculation capacity of the first processing unit 20 used for the analysis target data, and may be expressed as a relative value when the calculation capacity of the first processing unit 20 is set to "1". The communication cost between the first processing unit 20 and the second processing unit 30 is, for example, a value indicating the amount of data transported between the first processing unit 20 and the second processing unit 30, and may be expressed as a data bit rate (Mbps). The communication cost may also be the required resource amount, such as the number of resource blocks in LTE or 5G. In this case, even if the data bit rate is the same, the required resource amount differs between an environment with good communication quality and an environment with poor communication quality. When the processing cost calculation means 1011 predicts each cost, it may predict each cost using a learning model that inputs the features of the data to be analyzed and outputs a predicted value of each cost.

FIG. 7 is a table showing an example of information that the selection means 101 refers to in order to select an allocation method for each analysis target data acquired by each first processing unit 20, and shows, for each analysis target data, the first processing unit, the importance, and the calculation cost and communication cost when analyzed using each allocation method. How the selection means 101 refers to this table and selects the allocation method will be described later.

The communication quality information acquisition means 1012 acquires communication quality information indicating the communication quality between each first processing unit 20 and the second processing unit 30. The communication quality information may be, for example, communication throughput, MCS (Modulation and Coding Scheme) index, SINR (Signal to Interference plus Noise Ratio), etc., or may be a predicted value of these. Furthermore, when a predicted value is used as the communication quality information, the worst value (lowest value) of the predicted fluctuation range may be used.

The process control means 102 controls each of the first processing units 20 and the second processing units 30 so that each analysis target data is analyzed using the sharing method selected by the selection means 101. In one embodiment, the process control means 102 may perform the above control by transmitting, via the communication means 103, to each of the first processing units 20 and the second processing units 30, information identifying the analysis target data and information indicating the sharing method selected for the identified analysis target data.

In another embodiment, the processing control means 102 may select a sharing method to switch between the first processing unit 20 and the second processing unit 30 to analyze the data to be analyzed, based on a prediction of the processing load of the data to be analyzed in the first processing unit 20 and a prediction of the communication bandwidth between the first processing unit 20 and the second processing unit 30. The processing control means 102 may also determine a portion of the data to be analyzed to be discarded based on the predicted communication bandwidth. The processing control means 102 may also cause the first processing unit 20 and the second processing unit 30 to complement frames that were processed before the switching in the unit frame set when switching from a state in which the data to be analyzed is not being processed to a state in which the data to be analyzed is being processed. The processing control means 102 may also cause the processing unit that is not analyzing the data to be analyzed, among the first processing unit 20 and the second processing unit 30, to buffer the data to be analyzed, and when the processing unit that is not processing the data to be analyzed is switched to processing the data to be analyzed, the buffered data may be used to analyze the data to be analyzed. The process control means 102 may execute the above-mentioned discard process, complement process, and buffering process based on the importance of the data to be analyzed, the reliability of the processing of the data to be analyzed, the communication bandwidth allocated for transmitting the data to be analyzed, etc. The reliability is an index indicating the degree of confidence in the predicted analysis result, and may be, for example, a confidence value output from the trained model that performed the analysis.

(Selection of sharing method)
Next, a detailed description will be given of the selection of the sharing method by the selection unit 101. The selection unit 101 in this embodiment selects a sharing method from the following sharing methods 1 to 3.

　Sharing method 1: The first processing unit 20 acquires the data to be analyzed and generates the analysis results of the data to be analyzed.

　Sharing method 2: The first processing unit 20 acquires the data to be analyzed and calculates the features of the data to be analyzed. The features are then sent from the first processing unit 20 to the second processing unit 30, which then generates an analysis result for the data to be analyzed from the features.

　Sharing method 3: The first processing unit 20 acquires the data to be analyzed and transmits the data to the second processing unit 30, which then generates analysis results from the data to be analyzed.

From one perspective, sharing methods 1 to 3 differ in how the analysis processing of the analysis target data is shared between the first processing unit 20 and the second processing unit 30 that acquired the analysis target data. In other words, sharing method 1 is a method in which the first processing unit 20 that acquired the analysis target data performs all of the analysis processing of the analysis target data. Sharing method 2 is a method in which the first processing unit 20 and the second processing unit 30 that acquired the analysis target data share the analysis processing of the analysis target data. Sharing method 3 is a method in which the first processing unit 20 performs the minimum necessary processing such as compression, and the second processing unit 30 performs all of the analysis processing of the analysis target data. By using these sharing methods appropriately, analysis processing can be performed efficiently according to the situation.

In one embodiment, the processing control system 100 may cause the processing system 1 to analyze each piece of analysis target data using an allocation method selected from at least two of allocation methods 1 to 3. That is, the processing control system 100 may cause the processing system 1 to analyze each piece of analysis target data using an allocation method selected from allocation methods 1 to 3, may cause the processing system 1 to analyze each piece of analysis target data using an allocation method selected from allocation methods 1 and 2, may cause the processing system 1 to analyze each piece of analysis target data using an allocation method selected from allocation methods 1 and 3, or may cause the processing system 1 to analyze each piece of analysis target data using an allocation method selected from allocation methods 2 and 3.

The selection means 101 selects a sharing method for each of one or more pieces of analysis target data acquired by each first processing unit 20. In detail, the selection means 101 selects a sharing method from at least two sharing methods among a first sharing method for causing the first processing unit to generate an analysis result for the analysis target data for each of one or more pieces of analysis target data acquired by each first processing unit 20, a second sharing method for causing the first processing unit to calculate a feature amount of the analysis target data, transmit the feature amount from the first processing unit to the second processing unit, and cause the second processing unit to generate the analysis result from the feature amount, and a third sharing method for causing the first processing unit to transmit the analysis target data to the second processing unit, and cause the second processing unit to generate the analysis result from the analysis target data.

The constraints to be considered when selecting the sharing method are as follows:
- Because video data is large in size, it is not possible in a large-scale system to transmit all camera footage from the first processing unit 20 to the second processing unit 30 due to limitations on the amount of communication resources.- The amount of communication can be reduced by performing analysis or calculating features in the first processing unit 20 and transmitting the results to the second processing unit 30, but since the amount of computational resources available to the first processing unit 20 is limited, it is not possible for the first processing unit 20 to perform all analyses.- The amount of computational resources required to analyze the data obtained from each camera, the amount of communication resources required to transmit data with image quality that can be analyzed by the second processing unit 30, and the communication quality between the first processing unit 20 and the second processing unit 30 change from moment to moment, so it is necessary to respond as needed.

In order to select a sharing method taking into account the above constraints, in one embodiment, the selection means 101 operates as follows.

First, the selection means 101 collects, for each of the one or more pieces of analysis target data acquired by each first processing unit 20, the importance determined by the importance determination means 1010, as well as the calculation cost and communication cost calculated by the processing cost calculation means 1011 when analyzing using each sharing method. Figure 7 shows an example of information collected by the selection means 101. Note that in Figure 7, in order to distinguish between different first processing units 20, they are written as "first processing unit (1)" and so on.

In one embodiment, the selection means 101 refers to the collected information and first determines the analysis target data for which the first allocation method is to be selected. At this time, the selection means 101 determines the analysis target data selected to be analyzed using the first allocation method, among one or more analysis target data acquired by each first processing unit 20, such that the total calculation cost of the first processing unit 20 when analyzed using the first allocation method does not exceed the upper limit of the calculation cost based on the calculation capacity of the first processing unit 20. This makes it possible to prevent the first processing unit 20 from being unable to process all the analysis target data. Note that the upper limit of the calculation cost based on the calculation capacity of the first processing unit 20 may be the calculation capacity of the first processing unit 20 itself, or may be a value set with a certain margin.

In one embodiment, the selection means 101 determines the priority of each analysis target data, and determines the analysis target data for which the first allocation method is to be selected based on the determined priority of each analysis target data. For example, the selection means 101 may select the first allocation method from the analysis target data with high priority, within a range in which the total calculation cost selected as the first allocation method does not exceed the upper limit of the calculation cost based on the calculation capacity of the first processing unit 20. This makes it possible to select the first allocation method within a range that can be processed by the first processing unit 20. The first allocation method performs analysis in the first processing unit 20 that acquires the analysis target data, so that the analysis can be performed quickly, which is advantageous when detecting events with high urgency, etc.

The method of determining the priority is not particularly limited, but for example, the selection means 101 may determine a priority for each of one or more analysis target data acquired by each first processing unit 20 based on the calculation cost when the analysis target data is analyzed using the first allocation method and the importance of the analysis target data. For example, the selection means 101 may determine the priority based on the following formula.

Priority=(α1×importance)/calculation cost (α1 is a predetermined value)
7, the selection means 101 will give priority to and select, as the first allocation method, analysis target data 1 among the analysis target data acquired by the "first processing unit (1)." By determining the priority in this manner, it is possible to perform the analysis efficiently by balancing the necessity for analysis with the calculation cost.

Then, the selection means 101 selects an allocation method for analysis target data for which the first allocation method was not selected, among the one or more analysis target data acquired by each first processing unit 20. In one aspect, the selection means 101 may select either the second allocation method or the third allocation method based on the communication quality information of each first processing unit 20 acquired by the communication quality information acquisition means 1012.

In other words, although the third sharing method involves a larger amount of communication than the second sharing method, by selecting the third sharing method for data to be analyzed by a first processing unit 20 with good communication quality, it is possible to improve the efficiency of wireless resource utilization. In particular, for example, when multiple first processing units 20 each acquire one or more areas of video data input from a camera 10 as data to be analyzed, and data that could not be fully processed by each first processing unit 20 is offloaded to the second processing unit 30, the efficiency of wireless resource utilization can be improved by prioritizing the first processing unit 20 with good communication quality. This allows the analysis process to be efficiently offloaded to the second processing unit 30, making it possible to analyze video data acquired from a greater number of cameras 10.

In addition, when the total communication cost of the analysis target data for which the first allocation method was not selected, among one or more analysis target data acquired by each first processing unit 20, exceeds the communication bandwidth between the first processing unit 20 and the second processing unit 30, the selection means 101 may, for example, not analyze the analysis target data of low importance.

The second embodiment has been described above as a process control system 100, but the process control system 100 according to the second embodiment may be mounted on a single device as a process control device. Furthermore, the operation of the process control system 100 according to the second embodiment may be the process control method according to the second embodiment.

Third Embodiment
A third embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second embodiments are given the same reference numerals, and the description thereof will be omitted as appropriate.

FIG. 8 is a block diagram showing an example of the configuration of a process control system 100 and a process system 1 according to the third embodiment. The process control system 100 according to this embodiment differs from the process control system 100 according to the second embodiment in that the selection means 101 includes a risk determination means 1013, so the function of the risk determination means 1013 will be described.

The risk assessment means 1013 assesses the risk indicated by each analysis target data. In this specification, "risk" means the need to perform analysis quickly. For example, a high risk may be determined for a highly likely dangerous event. High risk events include, but are not limited to, people in high risk positions (e.g., people at high altitudes, people working near holes dug at the site, people working near heavy machinery, people near roads, railroad tracks, high-voltage power lines, etc.), areas with high density of people and equipment (high density is judged to be high risk), people who move a lot (higher risk than people who are not moving), etc. There are no particular limitations on the way the risk is expressed, but it may be expressed as two values, 0 (low risk) and 1 (high risk), or as multiple values of three or more values, or as continuous numerical values.

The risk determination means 1013 may determine that the risk is high when the relationship between the person and the surrounding objects indicated by the analysis target data indicates the above-mentioned event, for example, but may also perform the determination using a learning model like the importance determination means 1010. For example, the risk determination means 1013 may determine the importance using a learning model that inputs the feature amount of each analysis target data and outputs the risk amount. In this case, the feature amount of each analysis target data may be calculated by the feature amount calculation unit 231 of the first processing unit 20, and the risk determination means 1013 may obtain the feature amount of each analysis target data from the first processing unit 20 via the communication means 103. The learning method of the learning model that inputs the feature amount of each analysis target data and outputs the risk amount is not particularly limited, but learning may be performed using the analysis target data for learning labeled with the risk amount as teacher data.

Then, for each of the one or more pieces of analysis target data acquired by each first processing unit 20, the selection means 101 collects the importance determined by the importance determination means 1010, the calculation cost and communication cost calculated by the processing cost calculation means 1011 when analyzing using each sharing method, and the risk determined by the risk determination means 1013.

Furthermore, the selection means 101 may determine a priority for each of the one or more analysis target data acquired by each first processing unit 20, based on the calculation cost when the analysis target data is analyzed using the first allocation method, the importance of the analysis target data, and the risk level indicated by the analysis target data. For example, the selection means 101 may determine the priority based on the following formula.

Priority=(α1×importance)×(α2×risk)/calculation cost (α1 and α2 are predetermined values)
By determining the priority in this manner, the first processing unit 20 can detect risky behavior that requires an immediate alert to the person in question or an administrator, thereby preventing communication delays and, for example, reducing the delay until an alert is issued.

As mentioned above, the risk determination means 1013 may be provided in each first processing unit 20, independent of the selection means 101. In this case, if the risk determination means 1013 of the first processing unit 20 determines that the risk indicated by certain analysis target data is high, the analysis target data may be analyzed in the first processing unit 20 (using the first sharing method) without being controlled by the processing control means 102.

The third embodiment has been described above as a process control system 100, but the process control system 100 according to the third embodiment may be mounted on a single device as a process control device. Furthermore, the operation of the process control system 100 according to the third embodiment may be the process control method according to the third embodiment.

Fourth embodiment
A fourth embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first, second and third embodiments are denoted by the same reference numerals and will not be described repeatedly.

FIG. 9 is a block diagram showing an example of the configuration of a processing control system 100 and a processing system 1 according to the fourth embodiment. The processing control system 100 according to this embodiment differs from the processing control system 100 according to the second embodiment in that the selection means 101 includes a compression efficiency calculation means 1014, so the function of the compression efficiency calculation means 1014 will be described.

The compression efficiency calculation means 1014 calculates the compression efficiency of each analysis target data. In this specification, "compression efficiency" is an index of the degree to which the analysis target data can be compressed by compression processing, and may be a predicted value.

For example, if the image represented by the data to be analyzed shows a state in which it is raining or snowing, a state in which an object is moving significantly, or a state in which an object is moving randomly, the compression efficiency tends to decrease. Therefore, in one aspect, the compression efficiency calculation means 1014 determines whether the data to be analyzed corresponds to any of the states of a state in which it is raining or snowing, a state in which an object is moving significantly, or a state in which an object is moving randomly, and if it does not correspond to any of the states, the compression efficiency may be set to, for example, 1, and if it corresponds to the state, the compression efficiency may be set to, for example, a value between 0 and 1 depending on the amount of rain or snow or the amount of movement.

Furthermore, for data to be analyzed that shows outdoors, weather information may be referenced to determine whether it is raining or snowing. In one aspect, the compression efficiency calculation means 1014 may set the compression efficiency to, for example, 1 for data to be analyzed that corresponds to areas showing indoors, depending on the imaging range of the camera 10, and may set the compression efficiency to, for example, a value between 0 and 1 for data to be analyzed that corresponds to areas showing outdoors, by reference to weather information, when it is raining or snowing.

The compression efficiency may also be calculated by referring to past actual measurements, etc. In one embodiment, the compression efficiency calculation means 1014 may obtain the compressed bit rate in the encoding unit 233 of the first processing unit 20 and predict the compression efficiency. In another embodiment, the compression efficiency calculation means 1014 may obtain analysis feasibility information for past analysis target data compressed at an arbitrary compression rate in the analysis unit 321 of the second processing unit 30, and if analysis is not possible, may set a low compression rate and calculate the compression efficiency, assuming that image quality needs to be improved.

Then, for each of the one or more analysis target data acquired by each first processing unit 20, the selection means 101 collects the importance determined by the importance determination means 1010, the calculation cost and communication cost calculated by the processing cost calculation means 1011 when analyzing using each allocation method, and the compression efficiency determined by the compression efficiency calculation means 1014.

Furthermore, the selection means 101 may determine a priority for each of the one or more analysis target data acquired by each first processing unit 20, based on the calculation cost when the analysis target data is analyzed using the first allocation method, and the importance and compression efficiency of the analysis target data. For example, the selection means 101 may determine the priority based on the following formula.

Priority=(α1×importance)×(α3/compression efficiency)/calculation cost (α1 and α3 are predetermined values)
By determining the priority in this manner, data to be analyzed that has a high compression efficiency and a low cost of transmitting the data to the second processing unit 30 can be sent to the second processing unit 30 preferentially, thereby enabling analysis to be performed efficiently.

Furthermore, the selection means 101 selects an allocation method for analysis target data for which the first allocation method was not selected, among one or more analysis target data acquired by each first processing unit 20. In one aspect, the selection means 101 may select one of the second and third allocation methods based on the compression efficiency of the analysis target data in addition to the communication quality information of each first processing unit 20 described above.

In other words, the third sharing method involves a larger amount of communication traffic than the second sharing method, but by selecting the third sharing method for data to be analyzed that has high compression efficiency, it is possible to improve the efficiency of wireless resource utilization.

In addition, the compression efficiency calculation means 1014 may take into account the density of people and objects in the data to be analyzed when calculating the compression efficiency. In one aspect, the compression efficiency calculation means 1014 may increase the value of the compression efficiency the greater the number of people or objects to be analyzed that exist within a range of a predetermined size (number of pixels) in the data to be analyzed. In this way, densely populated areas can be analyzed even with low resolution, and the compression efficiency can be improved by transmitting data corresponding to such areas to the second processing unit 30.

In addition, the compression efficiency calculation means 1014 may take into account the size (number of pixels) of the person or object being analyzed in the data being analyzed when calculating the compression efficiency. In one aspect, the compression efficiency calculation means 1014 may reduce the value of the compression efficiency the smaller the size (number of pixels) of the person or object being analyzed in the data being analyzed. If a person or object that appears small is compressed and sent to the second processing unit 30 for analysis, there is a high possibility that the analysis accuracy will decrease, so by setting the compression efficiency low, it is possible to perform the analysis in the first processing unit 20 as much as possible.

The selection means 101 may further take into account the degree of danger when determining the priority. In one aspect, the selection means 101 according to this embodiment may further include the degree of danger determination means 1013 according to the third embodiment, and may determine the priority based on the following formula:

Priority=(α1×importance)×(α2×risk)×(α3/compression efficiency)/computation cost (α1, α2, and α3 are predetermined values)
Although the fourth embodiment has been described above as the process control system 100, the process control system 100 according to the fourth embodiment may be mounted on a single device to form a process control device. Furthermore, the operation of the process control system 100 according to the fourth embodiment may be the process control method according to the fourth embodiment.

Fifth embodiment
A fifth embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first embodiment are denoted by the same reference numerals, and the description thereof will not be repeated.

FIG. 10 is a block diagram showing an example of the configuration of a processing control system 100 and a processing system 1 according to the fifth embodiment. The selection means 101 according to this embodiment includes an importance determination means 1010, a processing cost calculation means 1011, and an analysis accuracy information acquisition means 1015, and selects a sharing method for each of one or more pieces of analysis target data acquired by each first processing unit 20 in the following manner.

The selection means 101 according to this embodiment solves a combinatorial optimization problem so as to maximize the objective function and selects an allocation method for each analysis target data. In detail, the selection means 101 selects an allocation method for each analysis target data so as to satisfy all of the constraint conditions below and maximize or minimize one of the objective functions below, or decides not to perform analysis. This allows for efficient analysis that takes into account the overall situation.

(Constraint 1) The total calculation cost in each first processing unit 20 is equal to or less than the upper limit based on the calculation capacity of the first processing unit 20. (Constraint 2) For all first processing units 20 sharing the same communication line in communication with the second processing unit 30, the total communication cost of all analysis target data is equal to or less than the communication bandwidth of the communication line. (Objective function 1) When the sharing method is selected for each analysis target data, maximize the sum of the importance of each analysis target data x analysis accuracy. (Objective function 2) When the sharing method is selected for each analysis target data, maximize the number of analysis target data whose analysis accuracy is equal to or greater than a specified value (e.g., 80%). The selection means 101 may search all options to find an optimal solution, or may apply a commonly used heuristic algorithm to find an approximate solution. The analysis accuracy of the analysis target data may be obtained by the analysis accuracy information acquisition means 1015.

In one embodiment, the analysis accuracy information acquisition means 1015 may use, as the analysis accuracy, analysis accuracy information (confidence value) obtained when the analysis is performed using a learning model. For example, the analysis accuracy information acquisition means 1015 may assume that analysis target data corresponding to a specific analysis target is continuously obtained, have the analysis unit 212 of the first processing unit 20 and the analysis unit 311 of the second processing unit 30 analyze the first few pieces of analysis target data corresponding to the specific analysis target, acquire analysis accuracy information (confidence value) of the analysis results, and use the acquired analysis accuracy information as the analysis accuracy information of the analysis target data for the specific analysis target. Alternatively, a model may be created that predicts the analysis accuracy from inputs including the analysis target data and the video compression parameters, and the analysis accuracy information acquisition means 1015 may predict the analysis accuracy using this model.

In one embodiment, the selection means 101 may not include the analysis target data corresponding to a specific camera 10 in the calculation of the objective function or may weight the analysis target data corresponding to the specific camera 10 less. For example, when a specific camera is covered with water droplets or debris, or when the specific camera is installed in a dark environment where nothing can be seen, for example, by not performing analysis of the analysis target data corresponding to the camera 10 that is likely to produce a specific result even without analysis, resources can be allocated to other analysis target data, resulting in improved efficiency. The method of determining whether a specific camera is in a situation where a specific result is likely to produce a specific result even without analysis is not particularly limited, and may be determined and set by a person, or may be set according to the results of analyzing the captured image, for example.

In one embodiment, the selection means 101 may switch the objective function depending on the type of behavior to be analyzed. For example, if risky behavior detection is desired, the objective function may be set to importance x analysis accuracy x delay coefficient, and the delay coefficient may be set to decrease when the delay until detection exceeds a preset value. This allows an analysis appropriate for the type of behavior to be analyzed.

The analysis accuracy information acquisition means 1015 may also estimate the amount of degradation in analysis accuracy due to a change in processing load and reflect this in the analysis accuracy information. That is, the analysis engines used by each analysis unit include engines that perform analysis based on changes in successive video frames, not just a single video frame. When using such engines, frequent changes in processing load between the first processing unit 20 and the second processing unit 30 will result in a degradation in analysis accuracy. By correcting the analysis accuracy information to take into account such degradation in analysis accuracy, more accurate analysis accuracy information can be obtained.

The fifth embodiment has been described above as a process control system 100, but the process control system 100 according to the fifth embodiment may be mounted on a single device to form a process control device. Furthermore, the operation of the process control system 100 according to the fifth embodiment may be the process control method according to the fifth embodiment.

[Modifications]
Below, modified examples that can be commonly applied to the first to fifth embodiments will be described.

Although the above describes a configuration in which the processing control system 100 is independent of each of the first processing units 20 and the second processing units 30, each embodiment is not limited to this. For example, a part or all of the processing control system 100 may be provided in each of the first processing units 20, the second processing unit 30, or in each of the first processing units 20 and the second processing unit 30, distributed therein.

Furthermore, for example, each component of the selection means 101 (e.g., importance determination means 1010, processing cost calculation means 1011, communication quality information acquisition means 1012, risk determination means 1013, compression efficiency calculation means 1014, analysis accuracy information acquisition means 1015, etc.) may be provided in different devices, and a main part of the selection means 101 that ultimately performs selection for each analysis target data may exist separately from these components. For example, at least one of the importance determination means 1010, processing cost calculation means 1011, communication quality information acquisition means 1012, risk determination means 1013, compression efficiency calculation means 1014, and analysis accuracy information acquisition means 1015 may be provided in each first processing unit 20, and the main part of the selection means 101 may be provided in the second processing unit 30.

In addition, when the selection means 101 cannot uniquely select an allocation method for each analysis target data (for example, when there is a large amount of analysis target data with the same priority), the selection means 101 may, for example, select the analysis targets to be analyzed by the first processing unit 20 (select the first allocation method) in order (round robin method).

The selection means 101 may select the sharing method each time analysis target data is acquired, but may also periodically execute the process of selecting the sharing method and change the processing method in response to changes in communication quality, risk level, video content, etc.

The selection means 101 may also make a selection taking into consideration the results of past analysis of analysis target data corresponding to the same analysis target. For example, if there is prior information that a particular behavior X will (is highly likely to) continue for a predetermined period of time or more, once behavior X has been detected as an analysis result, the selection means 101 may decide not to analyze the analysis target data corresponding to the same analysis target for the predetermined period of time.

Furthermore, for example, when the analytical accuracy (confidence value) of the analysis results of past analysis target data is high, the selection means 101 may lower the importance of the analysis target data corresponding to the same analysis target, and when the analytical accuracy of the analysis results of past analysis target data is low, the selection means 101 may raise the importance of the analysis target data corresponding to the same analysis target. This is because when the analytical accuracy is high, the analysis is performed correctly and it is highly likely that the same analysis result will continue, but when the analytical accuracy is low, the analysis is not performed correctly and it is possible that a different analysis result will be obtained.

In addition, for analysis target data for which the third sharing method has been selected, the encoding unit 213 may generate an image that cuts out only the area in which the analysis target exists and provide it to the communication unit 22, or may generate an image with reduced image quality for areas other than the area in which the analysis target exists and provide it to the communication unit 22.

In one embodiment, the selection means 101 may predict in advance the number of analysis subjects (e.g., people) that can be processed by the first processing unit 20, and select a sharing method to perform analysis in the first processing unit 20 as much as possible.

For example, the number of subjects that can be analyzed simultaneously with the computational capabilities (amount of computational resources) of the first processing unit 20 is profiled in advance, and the first processing unit 20 analyzes up to that number (selecting the first sharing method), and the excess is offloaded to the second processing unit 30 (selecting the second or third sharing method).

In this way, by analyzing as many analysis targets as possible in the first processing unit 20, the amount of processing by the second processing unit 30 can be reduced, and the second processing unit 30 can analyze other camera images or use them for other purposes.

This disclosure is not limited to the above-described embodiments, and various modifications are possible. The technical scope of this disclosure also includes embodiments obtained by appropriately combining the configurations, operations, and processes disclosed in the different embodiments. In addition, the technical scope of this disclosure also includes embodiments in which the order of the operations and processes disclosed in the different embodiments is appropriately changed.

Each of the configurations according to the first to fifth embodiments may be realized by (1) one or more pieces of hardware, (2) one or more pieces of software, (3) a combination of hardware and software, or (4) a cloud server. Each device, function, and process may be realized by at least one computer having at least one processor and at least one memory. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 11. For example, each of the functions described in the first to fifth embodiments may be realized by storing a program for implementing the processing control method described in the first to fifth embodiments in memory C2, and having processor C1 read and execute program P stored in memory C2.

The program P includes a set of instructions for causing the computer C to execute one or more of the functions described in the first to fifth embodiments when the program P is loaded into the computer C. The program P is stored in the memory C2. The processor C1 may be, for example, a CPU (Central Processing Unit). The memory C2 may be, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory, a Solid State Drive (SSD), etc.

The program P can also be recorded on a non-transitory, tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can obtain the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communications network or broadcast waves. The computer C can also obtain the program P via such a transmission medium.

The present disclosure is not limited to the above-described embodiments. In other words, the present invention can be applied in various aspects that a person skilled in the art can understand within the scope of the present disclosure. Note that some or all of the above-described embodiments can also be described as follows. However, the present invention is not limited to the aspects described below.

(Appendix 1)
A process control system for controlling one or more first processing units each acquiring one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing an analysis of the analysis target data with each of the first processing units,
a selection means for selecting, for each of the one or more pieces of analysis target data acquired by each first processing unit, a sharing method for analysis of the analysis target data in accordance with the calculation capacity of the first processing unit;
a process control means for controlling each of the first processing means and the second processing means so as to execute analysis for each analysis target data using the sharing method selected by the selection means.

(Appendix 2)
The selection means is
For each of the one or more pieces of analysis target data acquired by each first processing unit,
a first sharing method for causing the first processing unit to generate an analysis result of the analysis target data;
a second sharing method in which the first processing unit calculates features of the data to be analyzed, the first processing unit transmits the features to the second processing unit, and the second processing unit generates the analysis result from the features; and a third sharing method in which the first processing unit transmits the data to be analyzed to the second processing unit, and the second processing unit generates the analysis result from the data to be analyzed.

(Appendix 3)
The processing control system described in Appendix 2, wherein the selection means determines the analysis target data for which the first allocation method is to be selected from the one or more analysis target data acquired by each first processing unit, such that the total computational cost in the first processing unit when the analysis target data is analyzed using the first allocation method does not exceed an upper limit of computational cost based on the computational capacity of the first processing unit.

(Appendix 4)
The processing control system described in Appendix 3, wherein the selection means determines a priority for each of the one or more analysis target data acquired by each first processing unit based on the computational cost of the analysis target data and the importance of the analysis target data, and determines the analysis target data for which the first allocation method is selected based on the determined priority of each analysis target data.

(Appendix 5)
The processing control system of claim 4, wherein the selection means determines the priority for each of the one or more analysis target data acquired by each first processing unit based further on at least one of the risk level indicated by each analysis target data and the compression efficiency of each analysis target data.

(Appendix 6)
The processing control system of any one of Appendices 3 to 5, wherein the selection means selects one of the second sharing method and the third sharing method for analysis target data for which the first sharing method was not selected among the one or more analysis target data acquired by each first processing unit, based on communication quality between the first processing unit and the second processing.

(Appendix 7)
The processing control system described in Appendix 6, wherein the selection means selects one of the second allocation method and the third allocation method for analysis target data other than the analysis target data for which the first allocation method was selected among the one or more analysis target data acquired by each first processing unit, further based on the compression efficiency of the analysis target data.

(Appendix 8)
The processing control system of claim 2 or 3, wherein the selection means selects the sharing method based on the computational cost in the first processing unit, the communication cost between the first processing unit and the second processing unit, and the analytical accuracy of the analysis target data when each of the one or more analysis target data acquired by each first processing unit is analyzed using each of the at least two sharing methods.

(Appendix 9)
A process control device that controls one or more first processing units that respectively acquire one or more pieces of analysis target data, and a second processing unit that is capable of communicating with each of the first processing units and shares analysis of the analysis target data with each of the first processing units,
a selection unit that selects, for each of the one or more pieces of analysis target data acquired by each first processing unit, a sharing method for analysis of the analysis target data in accordance with a calculation capacity of the first processing unit;
a processing control unit that controls each of the first processing units and the second processing unit so that analysis is performed for each analysis target data using the sharing method selected by the selection unit.

(Appendix 10)
The processing control device described in Appendix 9, wherein the selection unit determines the analysis target data for which the first allocation method is selected from the one or more analysis target data acquired by each first processing unit, such that the total computational cost in the first processing unit when the analysis target data is analyzed using the first allocation method does not exceed an upper limit of computational cost based on the computational capacity of the first processing unit.

(Appendix 11)
The processing control device described in Appendix 10, wherein the selection unit determines the analysis target data for which the first allocation method is selected from the one or more analysis target data acquired by each first processing unit, such that the total computational cost in the first processing unit when the analysis target data is analyzed using the first allocation method does not exceed an upper limit of computational cost based on the computational capacity of the first processing unit.

(Appendix 12)
The processing control device described in Appendix 11, wherein the selection unit determines a priority for each of the one or more analysis target data acquired by each first processing unit based on the calculation cost of the analysis target data and the importance of the analysis target data, and determines the analysis target data for which the first allocation method is selected based on the determined priority of each analysis target data.

(Appendix 13)
The processing control device described in Appendix 12, wherein the selection unit determines the priority for each of the one or more analysis target data acquired by each first processing unit based further on at least one of the risk level indicated by each analysis target data and the compression efficiency of each analysis target data.

(Appendix 14)
The processing control device according to any one of appendices 9 to 13, wherein the selection unit selects one of the second sharing method and the third sharing method for analysis target data other than the analysis target data for which the first sharing method has been selected among the one or more analysis target data acquired by each first processing unit, based on communication quality between the first processing unit and the second processing.

(Appendix 15)
The processing control device described in Appendix 14, wherein the selection unit selects one of the second allocation method and the third allocation method for analysis target data other than the analysis target data for which the first allocation method was selected among the one or more analysis target data acquired by each first processing unit, further based on the compression efficiency of the analysis target data.

(Appendix 16)
The processing control device described in Appendix 9 or 10, wherein the selection unit selects the allocation method based on the computational cost in the first processing unit, the communication cost between the first processing unit and the second processing, and the analytical accuracy of the analysis target data when each of the one or more analysis target data acquired by each first processing unit is analyzed using each of the at least two allocation methods.

(Appendix 17)
A process control method for controlling one or more first processing units each acquiring one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing an analysis of the analysis target data with each of the first processing units, comprising:
selecting a sharing method for analysis of the one or more pieces of analysis target data acquired by each first processing unit in accordance with the computing capacity of the first processing unit;
A process control method for controlling each of the first processing units and the second processing unit so as to execute analysis for each analysis target data in a selected sharing method.

(Appendix 18)
For each of the one or more pieces of analysis target data acquired by each first processing unit,
a first sharing method for causing the first processing unit to generate an analysis result of the analysis target data;
a second sharing method in which the first processing unit calculates features of the data to be analyzed, the first processing unit transmits the features to the second processing unit, and the second processing unit generates the analysis result from the features, and a third sharing method in which the first processing unit transmits the data to be analyzed to the second processing unit, and the second processing unit generates the analysis result from the data to be analyzed.

(Appendix 19)
The processing control method described in Appendix 18, wherein, among the one or more pieces of analysis target data acquired by each first processing unit, analysis target data for which the first allocation method is selected is determined such that the total calculation cost in the first processing unit when the analysis target data is analyzed using the first allocation method does not exceed an upper limit of the calculation cost based on the calculation capacity of the first processing unit.

(Appendix 20)
20. The processing control method of claim 19, further comprising: determining a priority for each of the one or more analysis target data acquired by each first processing unit based on the computational cost of the analysis target data and the importance of the analysis target data; and determining the analysis target data for which the first allocation method is to be selected based on the determined priority of each analysis target data.

(Appendix 21)
A processing control method as described in Appendix 20, in which the priority is determined for each of the one or more analysis target data acquired by each first processing unit based further on at least one of the risk level indicated by each analysis target data and the compression efficiency of each analysis target data.

(Appendix 22)
A processing control method according to any one of appendices 19 to 21, in which, for analysis target data other than the analysis target data for which the first allocation method has been selected among the one or more analysis target data acquired by each first processing unit, one of the second allocation method and the third allocation method is selected based on communication quality between the first processing unit and the second processing.

(Appendix 23)
The processing control method described in Appendix 22, wherein for analysis target data other than the analysis target data for which the first allocation method was selected, among the one or more analysis target data acquired by each first processing unit, a allocation method of either the second allocation method or the third allocation method is selected based further on the compression efficiency of the analysis target data.

(Appendix 24)
20. The processing control method of claim 18, wherein the allocation method is selected based on the calculation cost in the first processing unit, the communication cost between the first processing unit and the second processing unit, and the analytical accuracy of the analysis target data when each of the one or more analysis target data acquired by each first processing unit is analyzed using each of the at least two allocation methods.

(Appendix 25)
The above-described process control system can also be expressed as follows.

A process control system for controlling one or more first processing units each acquiring one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing an analysis of the analysis target data with each of the first processing units,
At least one processor, the processor comprising:
a selection process for selecting, for each of the one or more analysis target data acquired by each first processing unit, a sharing method for analysis of the analysis target data in accordance with the calculation capacity of the first processing unit;
and a process control process for controlling each of the first processing units and the second processing unit so as to perform analysis on each analysis target data using the sharing method selected in the selection process.

The processing control system may further include at least one memory, and this memory may store a program for causing the processor to execute the selection process and the processing control process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

(Appendix 26)
The above-mentioned processing control device can also be expressed as follows.

A process control device that controls one or more first processing units that respectively acquire one or more pieces of analysis target data, and a second processing unit that is capable of communicating with each of the first processing units and shares analysis of the analysis target data with each of the first processing units,
At least one processor, the processor comprising:
a selection process for selecting, for each of the one or more analysis target data acquired by each first processing unit, a sharing method for analysis of the analysis target data in accordance with the calculation capacity of the first processing unit;
and a process control process for controlling each of the first processing units and the second processing unit so that analysis is performed on each analysis target data using the sharing method selected in the selection process.

The processing control device may further include at least one memory, and this memory may store a program for causing the processor to execute the selection process and the processing control process. The program may also be recorded on a computer-readable, non-transitory, tangible recording medium.

1 Processing system 10 Camera 20 First processing unit 21 Input unit 22 Communication unit 23 Main control unit 30 Second processing unit 31 Communication unit 32 Main control unit 100 Processing control system 101 Selection means 102 Processing control means 103 Communication means 200 Processing control device 201 Selection unit 202 Processing control unit 230 Analysis target data acquisition unit 231 Feature amount calculation unit 232 Analysis unit 233 Encoding unit 320 Decoding unit 321 Analysis unit 322 Output unit 1010 Importance determination means 1011 Processing cost calculation means 1012 Communication quality information acquisition means 1013 Risk determination means 1014 Compression efficiency calculation means 1014
1015 Analysis accuracy information acquisition means

Claims

A process control system for controlling one or more first processing units each acquiring one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing an analysis of the analysis target data with each of the first processing units,
a selection means for selecting, for each of the one or more pieces of analysis target data acquired by each first processing unit, a sharing method for analysis of the analysis target data in accordance with the calculation capacity of the first processing unit;
a process control means for controlling each of the first processing means and the second processing means so as to execute analysis for each analysis target data using the sharing method selected by the selection means.
The selection means is
For each of the one or more pieces of analysis target data acquired by each first processing unit,
a first sharing method for causing the first processing unit to generate an analysis result of the analysis target data;
2. The processing control system according to claim 1, wherein the allocation method is selected from at least two of: a second allocation method in which the first processing unit calculates features of the data to be analyzed, the first processing unit transmits the features to the second processing unit, and the second processing unit generates the analysis result from the features; and a third allocation method in which the first processing unit transmits the data to be analyzed to the second processing unit, and the second processing unit generates the analysis result from the data to be analyzed.
The processing control system according to claim 2, wherein the selection means determines the analysis target data for which the first sharing method is selected from the one or more analysis target data acquired by each first processing unit such that the total calculation cost in the first processing unit when the analysis target data is analyzed using the first sharing method does not exceed an upper limit of the calculation cost based on the calculation capacity of the first processing unit.
The processing control system according to claim 3, wherein the selection means determines a priority for each of the one or more analysis target data acquired by each first processing unit based on the calculation cost of the analysis target data and the importance of the analysis target data, and determines the analysis target data for which the first sharing method is to be selected based on the determined priority of each analysis target data.
The processing control system according to claim 4, wherein the selection means determines the priority for each of the one or more analysis target data acquired by each first processing unit based on at least one of the risk level indicated by each analysis target data and the compression efficiency of each analysis target data.
The processing control system according to any one of claims 3 to 5, wherein the selection means selects one of the second and third sharing methods for analysis target data for which the first sharing method has not been selected among the one or more analysis target data acquired by each first processing unit, based on the communication quality between the first processing unit and the second processing unit.
The processing control system according to claim 6, wherein the selection means selects one of the second and third sharing methods for analysis target data other than the analysis target data for which the first sharing method has been selected among the one or more analysis target data acquired by each first processing unit, based further on the compression efficiency of the analysis target data.
A process control device that controls one or more first processing units that respectively acquire one or more pieces of analysis target data, and a second processing unit that is capable of communicating with each of the first processing units and shares analysis of the analysis target data with each of the first processing units,
a selection unit that selects, for each of the one or more pieces of analysis target data acquired by each first processing unit, a sharing method for analysis of the analysis target data in accordance with a calculation capacity of the first processing unit;
a processing control unit that controls each of the first processing units and the second processing unit so that analysis is performed for each analysis target data using the sharing method selected by the selection unit.
The selection unit is
For each of the one or more pieces of analysis target data acquired by each first processing unit,
a first sharing method for causing the first processing unit to generate an analysis result of the analysis target data;
9. The processing control device according to claim 8, wherein the allocation method is selected from at least two of: a second allocation method in which the first processing unit calculates features of the data to be analyzed, the first processing unit transmits the features to the second processing unit, and the second processing unit generates the analysis result from the features; and a third allocation method in which the first processing unit transmits the data to be analyzed to the second processing unit, and the second processing unit generates the analysis result from the data to be analyzed.
The processing control device according to claim 9, wherein the selection unit determines the analysis target data for which the first sharing method is selected from among the one or more analysis target data acquired by each first processing unit, such that the total calculation cost in the first processing unit when the analysis target data is analyzed using the first sharing method does not exceed an upper limit of the calculation cost based on the calculation capacity of the first processing unit.
The processing control device according to claim 10, wherein the selection unit determines a priority for each of the one or more analysis target data acquired by each first processing unit based on the calculation cost of the analysis target data and the importance of the analysis target data, and determines the analysis target data for which the first sharing method is to be selected based on the determined priority of each analysis target data.
The processing control device according to claim 11, wherein the selection unit determines the priority for each of the one or more analysis target data acquired by each first processing unit based on at least one of the risk level indicated by each analysis target data and the compression efficiency of each analysis target data.
The processing control device according to any one of claims 9 to 12, wherein the selection unit selects one of the second and third sharing methods for analysis target data other than the analysis target data for which the first sharing method has been selected, among the one or more analysis target data acquired by each first processing unit, based on the communication quality between the first processing unit and the second processing unit.
A process control method for controlling one or more first processing units each acquiring one or more pieces of analysis target data, and a second processing unit capable of communicating with each of the first processing units and sharing an analysis of the analysis target data with each of the first processing units, comprising:
selecting a sharing method for analysis of the one or more pieces of analysis target data acquired by each first processing unit in accordance with the computing capacity of the first processing unit;
A process control method for controlling each of the first processing units and the second processing unit so as to execute analysis for each analysis target data in a selected sharing method.
For each of the one or more pieces of analysis target data acquired by each first processing unit,
a first sharing method for causing the first processing unit to generate an analysis result of the analysis target data;
15. The process control method according to claim 14, further comprising selecting a sharing method from at least two of a second sharing method in which the first processing unit calculates features of the data to be analyzed, the first processing unit transmits the features to the second processing unit, and the second processing unit generates the analysis result from the features, and a third sharing method in which the first processing unit transmits the data to be analyzed to the second processing unit, and the second processing unit generates the analysis result from the data to be analyzed.
The processing control method according to claim 15, wherein the analysis target data for which the first sharing method is selected is determined from among the one or more analysis target data acquired by each first processing unit such that the total calculation cost in the first processing unit when the analysis target data is analyzed using the first sharing method does not exceed an upper limit of the calculation cost based on the calculation capacity of the first processing unit.
The process control method according to claim 16, further comprising: determining a priority for each of the one or more analysis target data acquired by each first processing unit based on the calculation cost of the analysis target data and the importance of the analysis target data; and determining the analysis target data for which the first allocation method is to be selected based on the determined priority of each analysis target data.
The process control method according to claim 17, further comprising determining the priority for each of the one or more analysis target data acquired by each first processing unit based on at least one of the risk level indicated by each analysis target data and the compression efficiency of each analysis target data.
The processing control method according to any one of claims 16 to 18, wherein for analysis target data other than the analysis target data for which the first sharing method has been selected among the one or more analysis target data acquired by each first processing unit, one of the second sharing method and the third sharing method is selected based on the communication quality between the first processing unit and the second processing unit.
20. The processing control method according to claim 19, further comprising the step of selecting one of the second and third sharing methods for analysis target data other than the analysis target data for which the first sharing method has been selected among the one or more analysis target data acquired by each first processing unit, further based on the compression efficiency of the analysis target data.