WO2021100546A1

WO2021100546A1 - Artificial intelligence processing system, upload management device, method, and program

Info

Publication number: WO2021100546A1
Application number: PCT/JP2020/041869
Authority: WO
Inventors: 宏典松政
Original assignee: 富士フイルム株式会社
Priority date: 2019-11-20
Filing date: 2020-11-10
Publication date: 2021-05-27

Abstract

Provided are an artificial intelligence processing system, upload management device, method, and program for properly uploading input data to a resource that executes inference or the like using artificial intelligence. Current states of a plurality of resources capable of executing inference or the like using artificial intelligence are acquired. Types of input data required to execute the inference or the like are discerned on the basis of a table. Status of acquisition of input data to be used to execute the inference or the like is acquired on the basis of the types of input data discerned. The input data is uploaded to at least one resource among the plurality of resources on the basis of the current states of the plurality of resources and the status of acquisition of the input data.

Description

Artificial intelligence processing system, upload management device, method, program

The present invention relates to an artificial intelligence processing system, an upload management device, a method, and a program, and particularly relates to a technique for uploading input data to a resource for executing inference using artificial intelligence.

In the medical field, the performance of image analysis technology is improving. In particular, in recent years, the accuracy of analysis processing for recognizing lesions and identifying lesions has improved by using artificial intelligence (AI: Artificial Intelligence) that uses neural networks learned by deep learning. doing. A high-performance computer is required to execute such analysis processing.

On the other hand, in recent years, various cloud services using multiple servers have come to be provided. For example, services using the cloud are being provided without owning an expensive computer.

Patent Document 1 describes an API (Application Programming Interface), which is an application that acquires sensor data from an edge device and processes the acquired sensor data by exchanging the edge device with a computer in a cloud solution provided by a company. ) Is selected, the API of the selected computer is used to control the computer to perform the calculation, and the computer system that provides the calculation result is described.

Patent No. 6404529

When the analysis processing using AI and the learning processing of AI are distributed to the cloud or an external platform such as a computer of another facility, the transfer network band and the resources such as the CPU (Central Processing Unit) of the platform are finite. Therefore, there is a problem that the operation rate is lowered by not uploading the data at the intended timing.

The present invention has been made in view of such circumstances, and provides an artificial intelligence processing system, an upload management device, a method, and a program for appropriately uploading input data to a resource for executing inference using artificial intelligence. With the goal.

One aspect of the artificial intelligence processing system for achieving the above object is a plurality of resources capable of performing inference or artificial intelligence learning using artificial intelligence that outputs inference results with respect to input data, and inference or learning. Acquires the reception unit that accepts the instruction to execute the instruction, the discriminator that determines the type of input data required to execute the inference or learning that received the instruction, and the input data that is used to execute the inference or learning that received the instruction. An input data acquisition unit, an acquisition status acquisition unit that acquires the acquisition status of input data used for executing inference or learning based on the determined type of input data, and a state acquisition unit that acquires the current status of a plurality of resources. An artificial intelligence processing system including a unit and an upload control unit that uploads input data to be used for at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data. Is.

According to this aspect, the input data used for at least one resource among the plurality of resources is uploaded based on the current state of the plurality of resources and the acquisition status of the input data. Input data can be appropriately uploaded to the resource that executes the inference used.

The plurality of resources can execute inference or learning of artificial intelligence using a plurality of different artificial intelligences, and the reception unit accepts the artificial intelligence used for executing inference or re-learning among the plurality of artificial intelligences. Is preferable. This makes it possible to use artificial intelligence that is appropriate for executing inferences from multiple artificial intelligences.

The current state of a plurality of resources preferably includes the processing capacity of each resource, the operating status of the resource, and the network bandwidth of the network connected to the resource. As a result, at least one resource can be appropriately selected from the plurality of resources.

There are a plurality of types of input data required for executing inference or learning that has received an instruction, and it is preferable that the upload control unit uploads the input data after all the input data of the plurality of types of input data are prepared. As a result, the input data can be uploaded appropriately.

There are a plurality of types of input data required for executing inference or learning that have received instructions, and it is preferable that the upload control unit uploads the acquired input data in order from the plurality of types of input data. As a result, the input data can be uploaded appropriately.

There are multiple types of input data required to execute inference or learning that has received instructions, and the upload control unit includes a judgment unit that performs calculation processing on at least one of the multiple types of input data. It is preferable to upload a plurality of types of input data based on the result of the calculation process. As a result, the input data can be uploaded appropriately.

It is preferable that the upload control unit uploads the input data used for the resource that receives the instruction and finishes the execution of inference or learning earliest among the plurality of resources. As a result, inference or learning can be executed quickly.

The discriminator discriminates the type of input data required for executing inference or learning that has received an instruction, based on a table in which the inference and learning are associated with the types of input data required for executing the inference and learning. It is preferable to do so. As a result, it is possible to appropriately determine the type of input data required for executing the inference or learning that has received the instruction.

Artificial intelligence is preferably a machine-learned learning model. As a result, the inference result can be appropriately output for the input data.

The input data preferably includes medical data. This aspect can be applied to an artificial intelligence processing system that executes inference using artificial intelligence or learning of artificial intelligence on medical data. The medical data is data related to a subject, and is data including at least one of a medical image, a pathological image, diagnostic information, and finding information.

One aspect of the upload management device for achieving the above object is to acquire the current state of a plurality of resources capable of performing inference using artificial intelligence that outputs inference results for input data or learning of artificial intelligence. An acquisition status acquisition unit that determines the type of input data required for execution of inference or learning that has received an execution instruction and acquires the acquisition status of input data used for execution of inference or learning. It is an upload management device including an upload control unit that uploads input data to be used to at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data.

According to this aspect, since the input data is uploaded to at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data, artificial intelligence is used. Input data can be appropriately uploaded to the resource that executes inference.

One aspect of the upload management device for achieving the above object includes a memory for storing an instruction to be executed by the processor and a processor for executing the instruction stored in the memory, and the processor receives the input data. The type of input data required to execute inference or learning that receives the execution instruction by acquiring the current state of multiple resources that can execute inference or artificial intelligence learning using artificial intelligence. To determine the acquisition status of the input data used for executing inference or learning, and to at least one resource among the multiple resources based on the current state of the plurality of resources and the acquisition status of the input data. It is an upload management device that uploads the input data to be used.

One aspect of the upload management method for achieving the above object is to acquire the current state of a plurality of resources capable of performing inference using artificial intelligence that outputs inference results for input data or learning of artificial intelligence. The state acquisition process to be performed, the acquisition status acquisition process to determine the type of input data required to execute the inference or learning that received the execution instruction, and to acquire the acquisition status of the input data used to execute the inference or learning. This is an upload management method including an upload control step of uploading input data to be used to at least one resource among a plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data.

According to this aspect, since the input data is uploaded to at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data, artificial intelligence is used. Input data can be appropriately uploaded to the resource that executes inference. A program for causing a computer to execute the above upload management method is also included in this embodiment. The program for causing the computer to execute the above upload management method may be provided by storing it in a non-temporary recording medium that can be read by the computer.

According to the present invention, input data can be appropriately uploaded to a resource for executing inference or the like using artificial intelligence.

FIG. 1 is a block diagram showing a functional configuration of an artificial intelligence processing system. FIG. 2 is a block diagram showing a main functional configuration of the AI process unit. FIG. 3 is a block diagram showing a functional configuration of the AI process dispatcher. FIG. 4 is a flowchart showing an upload management method.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<Configuration of artificial intelligence processing system>
FIG. 1 is a block diagram showing a functional configuration of the artificial intelligence processing system 10. The artificial intelligence processing system 10 includes HIS (Hospital Information System) 12, PACS (Picture Archiving and Communication Systems) 14, VNA (Vendor Neutral Archive) 16, RIS (Radiology Information System) 18, and endoscope 20. , A genome analysis system 22, a hospital network 24, a hospital computer 26, a network adapter 28, a cloud 30, a cloud computer 32, and an AI (Artificial Intelligence) process dispatcher 100.

HIS12 is a computer system that supports medical services and accounting services for the entire hospital. HIS12 may include an automatic reception system, an ordering system, a medical accounting system, an electronic medical record system, and the like.

PACS14 is an image storage communication system. The PACS 14 receives a medical image taken by an imaging apparatus (modality) and stores it in a database (not shown). Further, the PACS 14 causes the designated medical image to be displayed on a display device (not shown).

The modality includes a device that generates a medical image representing the part to be inspected of the subject by photographing the part to be inspected, adds incidental information specified by the DICOM standard to the medical image, and outputs the medical image. Specific examples include CT equipment (Computed Tomography: Computed Tomography), MRI equipment (magnetic resonance imaging), PET equipment (Positron Emission Tomography), positron emission tomography equipment, and ultrasonic diagnostic equipment. Examples thereof include a CR device (Computed Radiography: computer X-ray imaging device) using a flat panel detector (FPD).

The PACS14 may treat a pathological image of a tissue collected from a subject taken with a camera as a medical image.

VNA16 is a distributor-neutral archive. The VNA 16 collectively manages various data managed by PACS 14s of different manufacturers.

RIS18 is a radiological information system. RIS18 manages radiological examination requests, examination appointments, patient information, examination reports, and the like.

The endoscope 20 is an optical system device for observing a luminal region such as the esophagus and intestine of a subject. Further, the endoscope 20 includes an optical system device for observing the inside of the incision site.

The genome analysis system 22 is a system that analyzes the genetic information of a subject. The genome analysis system 22 performs genome analysis on a cell sample of a subject.

The in-hospital network 24 is realized by, for example, a LAN (Local Area Network). The HIS12, PACS14, VNA16, RIS18, endoscope 20, genome analysis system 22, and AI process dispatcher 100 are connected via an in-hospital network 24.

The in-hospital computer 26 (an example of a resource) is a computer used in the hospital. The in-hospital computer 26 includes a hardware configuration (not shown) such as a CPU (Central Processing Unit), memory, storage, input / output interface, communication interface, input device, display device, and data bus. Further, the in-hospital computer 26 has a keyboard, a mouse, etc. (not shown) as an input device, and a display (not shown) as a display device. A well-known operation system or the like is installed in the in-hospital computer 26, and a viewer application for displaying a medical image is executed.

The AI process unit 50A and the table 50B are stored in the in-hospital computer 26. The AI process unit 50A is an example of artificial intelligence, and is a learning model that is learned by machine learning so as to output an inference result with respect to input data. Input data includes medical data. The medical data is data relating to a subject, and is data including at least one of a medical image, a pathological image, diagnostic information, and finding information. The diagnostic information includes information such as genome analysis results, electrocardiogram waveform data, and vital data. The finding information includes text data indicating the type and progress of the disease. The medical data may include personal information such as the sex and age of the subject, and clinical information such as medical history.

Here, the AI process unit 50A is a learning model that recognizes lung cancer. The in-hospital computer 26 can perform lung cancer recognition (an example of inference) using the AI process unit 50A and re-learning of the AI process unit 50A using predetermined resources.

In the table 50B, the recognition of lung cancer using the AI process unit 50A and the type of input data necessary for executing the recognition of lung cancer are stored in association with each other. Further, in the table 50B, the re-learning of the AI process unit 50A and the type of input data required for executing the re-learning are stored in association with each other. Here, the types of input data required for performing lung cancer recognition using the AI process unit 50A are, for example, a CR image of the chest and a CT image of the chest. Further, the types of input data required for executing the re-learning of the AI process unit 50A are, for example, a CR image of the chest and a CT image of the chest.

The network adapter 28 realizes communication with the cloud 30 via the Internet 34.

Cloud 30 is a service that provides computer resources. The cloud 30 is configured to be able to communicate with a client such as a medical institution via the Internet 34. The cloud 30 includes a cloud computer 32.

The cloud computer 32 (an example of resources) includes a hardware configuration (not shown) such as a CPU, memory, storage, input / output interface, communication interface, input device, display device, and data bus. Further, the cloud computer 32 has a keyboard, a mouse and the like (not shown) as an input device, and a display and the like (not shown) as a display device. A well-known operating system or the like is installed in the cloud computer 32 and has a server function. The cloud 30 may include a plurality of cloud computers 32.

The cloud computer 32 stores the AI process unit 50A, the AI process unit 52A, the table 50B, and the table 52B. The AI process unit 50A and the table 50B are the same as the AI process unit 50A and the table 50B stored in the in-hospital computer 26. The AI process unit 52A is an example of artificial intelligence, and is a learning model that is learned by machine learning so as to output an inference result with respect to input data. Here, the AI process unit 52A recognizes a brain tumor. The cloud computer 32 uses predetermined resources to perform inference using the AI process unit 50A, inference using the AI process unit 52A, re-learning of the AI process unit 50A, and re-learning of the AI process unit 52A. , Is feasible. The cloud computer 32 has a predetermined ratio of CPU processing that can be occupied between the execution of inference and relearning using the AI process unit 50A and the execution of inference and relearning using the AI process unit 52A. There is.

In the table 52B, the recognition of the brain tumor using the AI process unit 52A and the type of input data necessary for executing the recognition of the brain tumor are stored in association with each other. Further, in the table 52B, the re-learning of the AI process unit 52A and the type of input data required for executing the re-learning are stored in association with each other. The types of input data required to execute the recognition of the brain tumor using the AI process unit 52A are, for example, a CT image of the head and an MRI image of the head. Further, the types of input data required for executing the re-learning of the AI process unit 52A are, for example, a CT image of the head and an MRI image of the head.

The AI process dispatcher 100 is a program that serves as a gateway, and is located at the entrance / exit of the hospital network 24. The AI process dispatcher 100 operates on a computer (not shown). The AI process dispatcher 100 may operate on the in-hospital computer 26. The AI process dispatcher 100 manages the upload of data to the in-hospital computer 26 and the cloud computer 32.

<Structure of AI process section>
FIG. 2 is a block diagram showing a main functional configuration of the AI process unit. Here, the AI process unit 50A that recognizes lung cancer will be described as an example. The AI process unit 50A is a learning model that learns using a CR image and a CT image and performs recognition processing using the CR image and the CT image.

The AI process unit 50A has a plurality of layer structures and holds a plurality of weight parameters. The AI process unit 50A can change from an unlearned model to a trained model by updating the weight parameter from the initial value to the optimum value.

The AI process unit 50A has a convolutional neural network (CNN) configuration, and includes an input layer 60, an intermediate layer 62, and an output layer 64. The input layer 60, the intermediate layer 62, and the output layer 64 each have a structure in which a plurality of "nodes" are connected by "edges".

A CR image and a CT image are input to the input layer 60 as input data.

The intermediate layer 62 is a layer for extracting features from the image input from the input layer. The intermediate layer 62 has a plurality of sets including a convolution layer and a pooling layer as one set, and a fully connected layer. The convolution layer performs a convolution operation using a filter on nearby nodes in the previous layer and acquires a feature map. The pooling layer reduces the feature map output from the convolution layer to a new feature map. The fully connected layer connects all the nodes of the immediately preceding layer (here, the pooling layer). The convolution layer plays a role of feature extraction such as edge extraction from an image, and the pooling layer plays a role of imparting robustness so that the extracted features are not affected by translation or the like. The intermediate layer 62 is not limited to the case where the convolution layer and the pooling layer are set as one set, and the convolution layer may be continuous or may include a normalization layer.

The output layer 64 is a layer that outputs a recognition result for detecting lung cancer based on the characteristics extracted by the intermediate layer 62. In addition, the recognition result that classifies the detected lung cancer as benign or malignant may be output.

The learned AI process unit 50A classifies lung cancer into three categories, for example, "malignant", "benign", and "other", and the recognition results are classified into "malignant", "benign", and "other". Output as the corresponding three scores. The sum of the three scores is 100%.

Arbitrary initial values are set for the coefficient of the filter applied to each convolution layer of the AI process unit 50A before learning, the offset value, and the weight of the connection with the next layer in the fully connected layer.

The AI process unit 50A adjusts the weight parameter of the AI process unit 50A by the error back propagation method based on the error between the recognition result output from the output layer 64 and the correct answer data. This parameter adjustment process is repeated, and repeated learning is performed until the difference between the output of the AI process unit 50A and the correct answer data becomes small.

The AI process unit 50A does not have to use the correct answer data depending on the recognition process to be realized. Further, the AI process unit 50A may extract features by an algorithm designed in advance such as edge extraction, and use the information to learn with a support vector machine or the like.

The AI process unit 50A may learn using the CR image, the CT image, and the blood test result, and perform the recognition process using the CR image and the CT image. That is, the input data used for the recognition process may be a subset of the input data used for the learning process.

<Configuration of AI process dispatcher>
FIG. 3 is a block diagram showing a functional configuration of the AI process dispatcher 100. The AI process dispatcher 100 (an example of an upload management device) includes a reception unit 102, a determination unit 104, an acquisition status acquisition unit 106, a state acquisition unit 108, and an upload control unit 110.

The reception unit 102 receives an instruction to execute inference using the AI process unit 50A, inference using the AI process unit 52A, re-learning of the AI process unit 50A, or re-learning of the AI process unit 52A input from the user. .. The execution instruction is given by, for example, the user using an input device (not shown) of the in-hospital computer 26. The execution instruction may be automatically issued by the in-hospital computer 26. The reception unit 102 acquires an execution instruction from the in-hospital computer 26 and accepts it.

The instruction to execute the re-learning may be automatically transmitted from the in-hospital computer 26 when the time suitable for the re-learning or when the input data used for the re-learning is prepared.

The discrimination unit 104 discriminates from the table 50B or the table 52B the type of input data required for executing the inference or relearning received by the reception unit 102. When the instruction received by the reception unit 102 is inference using the AI process unit 50A or re-learning of the AI process unit 50A, the determination unit 104 determines the type of input data required from the table 50B of the in-hospital computer 26. It may be acquired, or it may be acquired from the table 50B of the cloud computer 32.

The acquisition status acquisition unit 106 acquires the acquisition status (alignment) of the input data used for executing inference or learning based on the type of input data determined by the determination unit 104.

The status acquisition unit 108 acquires the current status of a plurality of resources. Here, the status of the in-hospital computer 26 and the status of the cloud computer 32 are acquired. The status refers to the processing capacity (operating clock speed, etc.) of each CPU of the in-hospital computer 26 and the cloud computer 32, the ratio of CPU processing that can be occupied, the operating status of the CPU, the network bandwidth of the connected network, and the like. Including. The network bandwidth means the transmission line capacity (bit rate: bps) of the communication network, that is, the data transmission capacity of the communication network. A known method may be used to measure the network bandwidth.

The upload control unit 110 selects at least one resource from the plurality of resources based on the current state of the plurality of resources acquired by the state acquisition unit 108 and the acquisition status of the input data acquired by the acquisition status acquisition unit 106. select. Further, the upload control unit 110 inputs input data to be used for the selected resource based on the current state of the plurality of resources acquired by the state acquisition unit 108 and the acquisition status of the input data acquired by the acquisition status acquisition unit 106. Upload.

The AI process dispatcher 100 may include a storage unit that stores the table 50B and the table 52B.

<Upload management method>
FIG. 4 is a flowchart showing an upload management method. Here, a case where the reception unit 102 of the AI process dispatcher 100 receives an instruction for recognition processing of lung cancer using the AI process unit 50A will be described. It is assumed that the AI process dispatcher 100 has previously identified that the in-hospital computer 26 and the cloud computer 32 can execute the lung cancer recognition process using the AI process unit 50A.

In step S1, the AI process dispatcher 100 acquires all the data combination information necessary for executing the recognition and relearning that can be processed by the artificial intelligence processing system 10 from the calculation processing information management unit. Here, the AI process dispatcher 100 uses the cloud computer 32 to perform data information necessary for performing lung cancer recognition and relearning using the AI process unit 50A, and brain tumor recognition and relearning using the AI process unit 52A. Get the data information required for execution. For example, the discrimination unit 104 reads out the table 50B, and the types of input data required for performing lung cancer recognition and relearning using the AI process unit 50A are two types, a CR image of the chest and a CT image of the chest. Acquire (determine) that it is data (multi-data). Further, the discrimination unit 104 reads out the table 52B, and the types of input data required for executing the recognition and relearning of the brain tumor using the AI process unit 52A are 2 of the CT image of the head and the MRI image of the head. Get that it is a kind of data. The AI process dispatcher 100 may acquire all the data combination information necessary for executing the recognition and relearning that can be processed by the artificial intelligence processing system 10 from the in-hospital computer 26.

In step S2, the AI process dispatcher 100 (an example of the input data acquisition unit) corresponding to the data processing management unit acquires the data necessary for executing recognition or relearning from the PACS 14. For example, the AI process dispatcher 100 receives a CR image of the chest of a subject and a CT image of the chest. The AI process dispatcher 100 may receive the CR image of the chest of the subject and the CT image of the chest directly from the CR device and the CT device, or may be received from the VNA 16.

If only one of the CR image of the chest and the CT image of the chest of the subject has been captured, the AI process dispatcher 100 receives at least the captured image.

In step S3, the AI process dispatcher 100 narrows down the recognition or relearning process to be executed from the input data acquired in step S2 with respect to the combination information of the data acquired in step S1, and the data requested from the cloud computer 32. To get. Here, since the CR image of the chest and the CT image of the chest were received in step S2, the AI process dispatcher 100 determines that the process to be executed is the lung cancer recognition process. As the process to be executed, a plurality of recognition and relearning processes may be candidates.

In step S4, the determination unit 104 determines whether or not the data required in the table 50B is multi-data, that is, whether or not it is a plurality of types of data. If the required data is multi-data, the AI process dispatcher 100 performs the process of step S5. If the requested data is one type of data, the AI process dispatcher 100 performs the process of step S10. Here, the required data is a multi-data of a CR image of the chest and a CT image of the chest. Therefore, the AI process dispatcher 100 performs the process of step S5.

Note that multi-data is not limited to multiple types of medical images. The multi-data is a combination of a CT image and a pathological image, a CT image and a PET image, an MR image and a genome analysis result, a CT image and a blood test result, and the like according to the purpose of the examination.

In step S5 (an example of the acquisition status acquisition process), the acquisition status acquisition unit 106 determines whether or not all the input data of the required multi-data are available. When all the input data are prepared, the AI process dispatcher 100 performs the process of step S10. If not all the input data are available, the AI process dispatcher 100 performs the process of step S6.

Here, the required multi-data is a CR image of the chest of the subject and a CT image of the chest. For example, in step S2, only the CR image is received and the CT image is not received, only the CT image is received and the CR image is not received, and both the CR image and the CT image are received. If not, the AI process dispatcher 100 determines that all the input data are not available. Further, when both the CR image and the CT image are received in step S2, the AI process dispatcher 100 determines that all the input data are available.

In step S6, the acquisition status acquisition unit 106 acquires the scheduled completion date and time of the input data that has not been received among the requested input data from the data creation source. For example, when the CT image is not received, the acquisition status acquisition unit 106 acquires the shooting completion date and time of the CT image. The acquisition completion date and time of the CT image can be obtained from HIS12 or RIS18. The imaging completion date and time is, for example, 20 minutes for an ultrasonic diagnostic image, 1 hour for a CT image, and several days for a pathological image.

In step S7, the AI process dispatcher 100 sets the deadline for uploading the input data. Here, the AI process dispatcher 100 sets the completion deadline for uploading the input data to the completion time of all the input data acquisition. For example, when the CT image is not received, the AI process dispatcher 100 sets the deadline for uploading the CT image after one hour.

In step S8, the upload control unit 110 determines whether or not there is input data being uploaded. If there is input data being uploaded, the AI process dispatcher 100 performs the process of step S5. If there is no input data being uploaded, the AI process dispatcher 100 performs the process of step S9.

In step S9 (an example of the upload control process), the upload control unit 110 uploads a part of the input data. Here, the upload control unit 110 uploads in order from the acquired input data among the plurality of types of input data. For example, when only the CR image is received and the CT image is not received, the AI process dispatcher 100 uploads the CR image to the in-hospital computer 26 and the cloud computer 32, respectively. After that, the AI process dispatcher 100 performs the process of step S5.

As described above, in the present embodiment, when all the input data are not prepared, the upload control unit 110 lowers the upload order of the unprepared data. For example, in the case of multi-data consisting of a CT image and a blood test, the upload order of the CT images can be lowered in priority until a few days after the blood test is completed. However, if the resource (hospital computer 26 or cloud computer 32) is not used, it may be the target of upload.

If it is determined in the process of step S5 that all the input data are not prepared, the upload control unit 110 steps until the acquisition of all the input data is completed without performing the processes of steps S6 to S9. It is also possible to repeat the process of S5. For example, when only the CR image is received and the CT image is not received, the AI process dispatcher 100 performs the process of step S10 after receiving the CT image. As a result, the input data can be uploaded after all the input data are collected, so that it is possible to prevent the data required for processing from being uploaded in an incomplete state.

Further, when the AI process unit corresponding to the initial diagnosis data is created from the data for definitive diagnosis, the process of step S5 is performed after the state of one or more of the devices from which the multi-data is acquired is determined. You may. For example, in the case of multi-data consisting of a PET image for definitive diagnosis and a CT image for the first diagnosis, the upload control unit 110 determines whether uploading including the CT image is necessary based on the diagnosis result of the PET image. The multi-data may be a combination of a CT image for definitive diagnosis and a CR image for initial diagnosis.

In step S10 (an example of the state acquisition process), the state acquisition unit 108 acquires the operating status of all resources. Here, the state acquisition unit 108 acquires the operating status of the in-hospital computer 26, the operating status of the cloud computer 32, and the network bandwidth to the cloud 30.

The state acquisition unit 108 inquires about the operating status of the in-hospital computer 26 and acquires the operating status of the in-hospital computer 26. Further, the state acquisition unit 108 inquires about the operating status of the cloud computer 32 and acquires the operating status of the cloud computer 32. Further, the state acquisition unit 108 measures the transfer speed to the cloud computer 32 and infers the transfer time to the cloud computer 32.

In step S11, the state acquisition unit 108 calculates the local processing time, that is, the processing time for lung cancer recognition processing using the AI process unit 50A on the in-hospital computer 26. In addition, the state acquisition unit 108 calculates the processing time of the lung cancer recognition process using the AI process unit 50A on the cloud computer 32. The processing time here includes the time required for transmitting data from the AI process dispatcher 100 to the cloud computer 32.

In step S12, the state acquisition unit 108 determines whether or not the local processing time is longer than the processing time of another resource. Here, the state acquisition unit 108 uses the AI process unit 50A of the in-hospital computer 26 to process the lung cancer recognition process based on the processing results of steps S10 and S11, and the processing time of the lung cancer recognition process is the AI process on the cloud computer 32. It is determined whether or not the processing time of the lung cancer recognition process using the part 50A is longer than the processing time.

If the processing time on the in-hospital computer 26 is shorter, the AI process dispatcher 100 performs the processing in step S13. If the processing time on the in-hospital computer 26 is longer, the AI process dispatcher 100 performs the processing in step S14.

By comparing and judging the processing time of each resource in this way, it is possible to prevent the timing of uploading data from being delayed and the processing from being delayed. In addition, it is possible to prevent the operation rate from being lowered due to the occurrence of time when uploading is not performed.

In step S13, the AI process dispatcher 100 performs local execution processing. That is, the upload control unit 110 uploads the CR image and the CT image to the in-hospital computer 26 (an example of the upload control process). If some data has already been uploaded in step S9, the data that has not been uploaded is uploaded.

The in-hospital computer 26 inputs a CR image and a CT image and performs a lung cancer recognition process by the AI process unit 50A. Further, the in-hospital computer 26 transmits the processing result to the AI process dispatcher 100. This completes the processing of this flowchart. The processing result can be obtained quickly by performing the execution processing on the in-hospital computer 26 having a relatively short processing time.

On the other hand, in step S14, the upload control unit 110 determines whether or not data can be uploaded by controlling the number of simultaneous uploads possible. The number of simultaneous uploads is a predetermined number of simultaneous uploads, and refers to the number of data that can be uploaded at the same time. The communication speed of the upload line is finite (fixed), and uploading multiple data at the same time slows down and becomes inefficient. Therefore, in the artificial intelligence processing system 10, the number of simultaneous uploads is predetermined.

If the data can be moved, the AI process dispatcher 100 performs the process of step S15. If the data is not movable, the AI process dispatcher 100 performs the process of step S10.

In step S15 (an example of the upload control process), the upload control unit 110 uploads data. That is, the upload control unit 110 uploads the CR image and the CT image to the cloud computer 32. If some data has already been uploaded in step S9, the data that has not been uploaded is uploaded.

The cloud computer 32 receives the CR image and the CT image as inputs, performs lung cancer recognition processing by the AI process unit 50A, and transmits the processing result to the AI process dispatcher 100. This completes the processing of this flowchart. By executing the execution process on the cloud computer 32 having a relatively short processing time, the processing result can be quickly acquired.

The AI process dispatcher 100 sets the input data in step S7 for the resource for which a part of the data was uploaded in step S9 but the remaining data was not uploaded in step S13 or step S15. When the completion deadline for uploading has passed, a part of the data uploaded in step S9 is deleted. For example, when the CR image is uploaded to the hospital computer 26 and the cloud computer 32 in step S9 and the CT image is uploaded to the cloud computer 32 in step S15, the AI process dispatcher 100 uploads the CR image to the hospital computer 26. Erase the image. As a result, it is possible to prevent unused data from remaining in the resource.

As described above, according to this aspect, data can be uploaded to the resource in which the execution of inference or learning that receives the instruction finishes earliest among the plurality of resources.

<Filtering process>
In the above example, when the required data is multi-data, the upload control unit 110 has uploaded all the data, but the data to be uploaded may be filtered by preprocessing. Data may be uploaded when one or more calculation processes of the multi-data source are performed and the probability of being diagnosed as positive is equal to or higher than a certain value. For example, when the multi-data includes a CT image, the in-hospital computer 26 performs the CT image inference processing, and if the probability of being diagnosed as positive is equal to or higher than a certain value, the upload control unit 110 uploads the CT image. Do.

In addition, the in-hospital computer 26 (an example of the judgment unit) performs calculation processing of one or a plurality of data (an example of at least one input data) of the multi-data, and the probability of being diagnosed as positive is a certain value or more. In this case, uploading may be performed as soon as related data are available, or the acquired data may be uploaded in order.

In this way, by filtering the data to be uploaded in advance, it is possible to prevent uploading and processing unnecessary data.

<Others>
The above upload management method is configured as a program for realizing each process on a computer, and a non-temporary recording medium such as a CD-ROM (Compact Disk-Read Only Memory) in which this program is stored may be configured. It is possible.

In the embodiments described so far, for example, the hardware structure of the processing unit that executes various processes of the AI process dispatcher 100 is various processors as shown below. Various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, and a GPU (Graphics Processing Unit), which is a processor specialized in image processing. Dedicated to execute specific processing such as programmable logic device (PLD), ASIC (Application Specific Integrated Circuit), which is a processor whose circuit configuration can be changed after manufacturing FPGA (Field Programmable Gate Array), etc. A dedicated electric circuit or the like, which is a processor having a designed circuit configuration, is included.

One processing unit may be composed of one of these various processors, or two or more processors of the same type or different types (for example, a plurality of FPGAs, or a combination of a CPU and an FPGA, or a CPU and a CPU. It may be composed of a combination of GPUs). Further, a plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, one processor is configured by a combination of one or more CPUs and software, as represented by a computer such as a server and a client. There is a form in which the processor functions as a plurality of processing units. Secondly, as typified by System On Chip (SoC), there is a form in which a processor that realizes the functions of the entire system including a plurality of processing units with one IC (Integrated Circuit) chip is used. is there. As described above, the various processing units are configured by using one or more various processors as a hardware-like structure.

Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

The technical scope of the present invention is not limited to the scope described in the above-described embodiment. The configurations and the like in each embodiment can be appropriately combined between the respective embodiments without departing from the spirit of the present invention.

10 ... Artificial intelligence processing system 20 ... Endoscope 22 ... Genome analysis system 24 ... In-hospital network 26 ... In-hospital computer 28 ... Network adapter 30 ... Cloud 32 ... Cloud computer 34 ... Internet 50A ... AI process department 50B ... Table 52A ... AI process unit 52B ... Table 60 ... Input layer 62 ... Intermediate layer 64 ... Output layer 100 ... AI process dispatcher 102 ... Reception unit 104 ... Discrimination unit 106 ... Acquisition status acquisition unit 108 ... Status acquisition unit 110 ... Upload control units S1 to S15 … Steps on how to manage uploads

Claims

A plurality of resources capable of performing inference using artificial intelligence that outputs inference results for input data or learning of the artificial intelligence, and
A reception unit that receives instructions for executing the inference or learning, and
A discriminator that determines the type of input data required to execute the inference or learning that has received the instruction, and a discriminator.
An input data acquisition unit that acquires input data used for executing the inference or learning that has received the instruction, and an input data acquisition unit.
An acquisition status acquisition unit that acquires an acquisition status of input data used for executing the inference or learning based on the determined type of input data.
A state acquisition unit that acquires the current state of the plurality of resources, and
An upload control unit that uploads the input data to be used to at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data.
Artificial intelligence processing system equipped with.
The plurality of resources can perform inference or learning of artificial intelligence using a plurality of different artificial intelligences.
The artificial intelligence processing system according to claim 1, wherein the reception unit receives an artificial intelligence used for executing inference or re-learning among the plurality of artificial intelligences.
The artificial intelligence processing according to claim 1 or 2, wherein the current state of the plurality of resources includes the processing capacity of each resource, the operating status of the resource, and the network bandwidth of the network connected to the resource. system.
There are a plurality of types of the input data required for executing the inference or the learning that received the instruction.
The artificial intelligence processing system according to any one of claims 1 to 3, wherein the upload control unit uploads the input data after all the input data of the plurality of types of input data are collected.
There are a plurality of types of the input data required for executing the inference or the learning that received the instruction.
The artificial intelligence processing system according to any one of claims 1 to 3, wherein the upload control unit uploads the acquired input data in order from the plurality of types of input data.
There are a plurality of types of the input data required for executing the inference or the learning that received the instruction.
A determination unit that performs calculation processing on at least one of the plurality of types of input data is provided.
The artificial intelligence processing system according to any one of claims 1 to 3, wherein the upload control unit uploads the plurality of types of input data based on the result of the calculation process.
The upload control unit according to any one of claims 1 to 6 for uploading the input data to be used to the resource in which the inference or the learning execution finishes earliest among the plurality of resources. The artificial intelligence processing system described.
The discriminating unit receives the instruction and executes the inference or the learning based on a table in which the inference and the learning are associated with the types of input data necessary for executing the inference and the learning. The artificial intelligence processing system according to any one of claims 1 to 7, which determines the type of input data required for the above.
The artificial intelligence processing system according to any one of claims 1 to 8, which is a machine-learned learning model.
The artificial intelligence processing system according to any one of claims 1 to 9, wherein the input data includes medical data.
A state acquisition unit that acquires the current states of a plurality of resources that can perform inference using artificial intelligence or learning of the artificial intelligence that outputs inference results with respect to input data.
An acquisition status acquisition unit that determines the type of input data required to execute the inference or learning that has received an execution instruction, and acquires the acquisition status of the input data used for executing the inference or learning.
An upload control unit that uploads the input data to be used to at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data.
Upload management device equipped with.
A state acquisition process for acquiring the current states of a plurality of resources capable of performing inference using artificial intelligence or learning of the artificial intelligence that outputs inference results for input data, and
An acquisition status acquisition step of determining the type of input data required for executing the inference or learning that has received an execution instruction and acquiring the acquisition status of input data used for executing the inference or learning.
An upload control step of uploading the input data to be used to at least one resource among the plurality of resources based on the current state of the plurality of resources and the acquisition status of the input data.
Upload management method with.
A program for causing a computer to execute the upload management method according to claim 12.
A non-temporary, computer-readable recording medium on which the program according to claim 13 is recorded.