JP7510760B2 - Medical information processing device and medical information processing system - Google Patents

Description

The embodiments disclosed in this specification relate to a medical information processing device and a medical information processing system.

In recent years, many AI technologies have been applied in the medical field. With such AI technologies in the medical field, medical data used in actual medical treatment is used as training data with the patient's permission. Therefore, the learning model built into the medical system is trained using training data with the subject's permission.

However, if a patient subsequently requests to restrict or prohibit the use of their data, it may become impossible to use the trained model. At the same time, in order to deal with such a problem, manually replacing the data with the usage restrictions from the vast amount of data contained in the training dataset and retraining the trained model would be a huge burden.

JP 2013-537326 A

One of the problems that the embodiments disclosed in this specification aim to solve is to make it possible to easily reconstruct a training data set even if restrictions are subsequently imposed on the training data used in a learning model.

The medical information processing device according to the present embodiment includes a database, an information management unit, and a processing unit. The database stores a first dataset including a plurality of training data managed for each patient, which are used as training data for a learning model, and a second dataset including a plurality of spare data having the same data structure as the training data. The information management unit generates identification information for identifying unusable privacy information among the plurality of training data included in the first dataset, according to a usage restriction of at least a part of privacy information associated with each patient. The processing unit replaces the unusable training data identified by the identification information generated by the information management unit among the plurality of training data included in the first dataset with the spare data that is usable among the plurality of spare data included in the second dataset, and processes the first dataset.

FIG. 1 is a diagram for explaining a situation in which a medical image processing apparatus according to an embodiment is used. FIG. 2 is a block diagram showing the configuration of a medical image processing apparatus according to this embodiment. FIG. 3 is a diagram for explaining the data structure of the training data and the preliminary data. FIG. 4 is a flowchart for explaining the flow of the reconstruction process and the post-learning process of the learning model dataset.

The medical information processing device according to the embodiment will be described below with reference to the attached drawings.

First, a situation in which the medical information processing device according to the embodiment is used will be described. FIG. 1 is a diagram for explaining a situation in which the medical information processing device according to the embodiment is used. In FIG. 1, company A develops and manufactures learning models used in medicine (for example, a super-resolution model that takes a low-resolution image as input and outputs a high-resolution image, a CAD model that takes an image used for diagnosis as input and determines the presence or absence of a tumor, a model for region extraction, etc.). Such learning models are implemented in medical image diagnostic devices and medical image processing devices. In addition, such learning models may be implemented in a server device owned by company A and used via network N in response to a request from a client device owned by hospital B.

Hospital B uses a medical image diagnostic device or a medical image processing device that has a built-in learning model manufactured by company A, or a learning model implemented on a server device owned by company A via network N.

The learning model created by company A was produced by using medical information provided by multiple medical information providers as training data. Medical information provider C is one of the providers (typically patients) of the medical information used as this training data, and can express to company A at his/her own discretion and at any time whether or not the medical information he/she provided can be used (including whether or not part of the medical information provided can be used).

Note that in Figure 1, only one medical information provider C is shown for the sake of convenience. Naturally, there will be as many medical information providers as there are people who have received medical information. Furthermore, it is not necessary for the medical information provider himself to be the one who informs company A of whether or not the medical information provided by the medical information provider can be used; it could be his/her representative (e.g., a relative, legal representative, agent, etc.). In the following explanation, for the sake of concreteness, we will use as an example a case where the information provider is the patient himself/herself.

Figure 2 is a block diagram showing the configuration of a medical information processing device 1 according to this embodiment. The medical information processing device 1 is typically owned and managed by company A, and is a dedicated or general-purpose computer.

The medical information processing device 1 includes a memory circuit 10, a processing circuit 11, an input circuit 12, a communication I/F unit 13, and a display circuit 14.

The memory circuit 10 is composed of semiconductor memory elements such as RAM (Random Access Memory) and flash memory, a hard disk, an optical disk, etc. The memory circuit 10 may also be composed of portable media such as a USB (Universal Serial Bus) memory and a DVD (Digital Video Disk).

The memory circuitry 10 stores various processing programs (including application programs, an OS (Operating System), etc.) used in the processing circuitry 11, as well as data required to execute the programs, volume data, and medical images. The OS can also include a GUI (Graphical User Interface) that makes extensive use of graphics to display information to the operator on the display circuitry 14 and allows basic operations to be performed by the input circuitry 12.

The memory circuit 10 also stores multiple learning model data sets 10-1 to 10-n generated for each learning model.

The learning model data sets 10-1 to 10-n are databases that register medical data for each patient according to a predetermined data structure, and are managed for each learning model, such as a super-resolution model or a CAD model. The learning model data sets 10-1 to 10-n are each composed of training data sets 10-1a to 10-na that are composed of multiple training data sets, and spare data sets 10-1b to 10-nb that are composed of multiple spare data sets.

Here, training data refers to training data used for training (or learning) an AI model, such as a deep learning model, and is managed for each patient. In addition, multiple pieces of training data in the amount required for training the corresponding learning model are called a training dataset. Therefore, a training dataset is composed of training data provided by multiple patients.

The spare data has a data structure similar to the training data, and is not used directly to train the AI model, but is used to replace (substitute) the unusable data if the training dataset contains unusable data after the learning model is constructed. For this purpose, the spare data is generated with reference to the training data (for example, so that the content contained therein is similar to that of the training data). Furthermore, multiple spare data are called spare datasets. Similarly, a spare dataset is composed of spare data provided by multiple patients. Note that the spare dataset may be managed, for example, in association with the training dataset that is the target for replacing the unusable data.

Figure 3 is a diagram for explaining the data structure of training data and spare data, and is an example of meta information attached to the training data and spare data. As shown in Figure 3, the training data and spare data are managed for each patient, i.e., for each patient ID, and are systematically classified according to various information about each patient, such as gender, date of birth, height, weight, genetic information, medical history, and examination information (CT imaging, MR imaging, etc.).

A privacy tag P is attached to the patient ID and each piece of information associated with the patient ID. Here, a privacy tag is a tag that indicates whether the corresponding information can be used. For example, the privacy tag (first tag) of the patient ID itself is a tag that collectively manages whether all information associated with the patient can be used. In the example shown in FIG. 3, the privacy tag P of the patient ID is checked to indicate "usable". Therefore, the information related to the patient is managed for usability for each piece of information associated below it. On the other hand, if, in the example shown in FIG. 3, the privacy tag P of the patient ID is unchecked to indicate "unusable", all information related to the patient is collectively managed as unusable.

In the example shown in FIG. 3, the privacy tags P (second tags) attached to information related to the patient ID, such as gender, date of birth, height, weight, medical history, and examination information, are checked to indicate that they are "usable." On the other hand, the privacy tag P for genetic information is unchecked to indicate that it is "unusable."

Therefore, for the training data or spare data for the patient shown in Figure 3, only the genetic information will be treated as "unusable."

The processing circuit 11 is a processor that reads out the programs from the memory circuit 10 and executes them to realize the functions corresponding to each program. The processing circuit 11 has, for example, a privacy management function 111, a data set processing function 112, and a post-learning function 113. The processing circuit 11 reads out various control programs stored in the memory circuit 10 to realize the privacy management function 111, the data set processing function 112, and the post-learning function 113, and also comprehensively controls the processing operations in the memory circuit 10, the input circuit 12, the communication I/F unit 13, and the display circuit 14. In other words, the processing circuit 11 in a state in which each program has been read out has the privacy management function 111, the data set processing function 112, and the post-learning function 113.

The privacy management function 111 performs processing (privacy protection processing) to protect the privacy of medical information related to each patient. Data that has undergone this privacy protection processing becomes information that, in principle, cannot identify an individual. Furthermore, if at least a portion of specific training data becomes unusable (subsequently) after a learning model has been generated (if use restrictions are placed on at least a portion of the training data), the privacy management function 111 generates information (specification information) for identifying the unusable data. Furthermore, together with the specific information, the privacy management function 111 instructs the dataset processing function 112 to reconstruct a training dataset that includes the unusable data. The privacy management function 111 is an example of an information management unit.

The dataset processing function 112 automatically collects data required for the training dataset and the spare dataset from a database (not shown; this database is also referred to as the "patient-provided information database") that stores information provided by patients for the purpose of training the learning model, based on conditions that are preset according to the type of learning model, and automatically constructs a dataset for the learning model. The dataset processing function 112 is an example of a processing unit.

That is, the dataset processing function 112 generates training data or spare data having the data structure shown in FIG. 3 for each patient from the information stored in the patient-provided information database, and automatically constructs a training dataset and a spare dataset for each learning model. The dataset processing function 112 can arbitrarily set the proportion of data used to construct the training dataset and the spare dataset to be used as training data and the proportion to be used as spare data.

The learning model dataset can be constructed manually, rather than automatically. It is also possible to set information in advance about data that is to be actively used as training data. Furthermore, some data may be shared between the training dataset and the backup dataset.

Furthermore, the dataset processing function 112 automatically reconstructs the learning model dataset when triggered by a command from the privacy management function 111. That is, the dataset processing function 112 identifies unusable data in the training dataset based on the specific information received from the privacy management function 111. Furthermore, the dataset processing function 112 automatically reconstructs the learning model dataset by replacing the identified unusable data in the training dataset with data (candidate data) that is highly similar to the unusable data in the spare dataset and is a candidate for replacement.

Furthermore, the dataset processing function 112 calculates the similarity between each piece of usable data in the spare dataset and the unusable data to be replaced. For each piece of usable data in the spare dataset, the dataset processing function 112 replaces the unusable data in the training dataset with the usable data that has the highest similarity.

The dataset processing function 112 can calculate similarity based on various indicators related to unusable data.

Indices used in similarity calculations include, for example, "imaging protocol (Cardiac, Neuro, etc.)", "imaging program (whether contrast agent was used or not, DA, DSA, rotational imaging, etc.)", "patient information (gender, age group, physique, case, etc.)", "device setting information at the time of imaging (tube voltage value, tube current value, bed height, arm angle, etc.)", and "image information (purpose of examination, diagnostic area, modality, image type, resolution, etc.)".

To calculate similarity using multiple indices, the following similarity evaluation formula can be used, for example:

(Similarity) = 1/{(Weighting coefficient α) x (difference in first index) + (Weighting coefficient β) x (difference in second index) + ...}

Here, "difference in index" refers to the difference in index between the unusable data and the backup data that is the subject of the similarity calculation. Also, each weighting coefficient is a coefficient for adjusting how much each index is reflected in the similarity.

For example, in a learning model that automatically extracts the area of a specific organ, image data in which the area of the target organ is marked is used as input data. When replacing the training data used in this learning model, a similarity calculation is performed that includes, for example, the size of the target area and the position of the cross section as an index. In this case, the weighting coefficient for the size of the target area and the position of the cross section can be increased in consideration of the type of learning model before the similarity calculation is performed.

The dataset processing function 112 notifies the post-learning function 113 and company B's management server, as necessary, that the construction or reconstruction of the learning model dataset has been completed.

The post-learning function 113 uses the training dataset reconstructed by the dataset processing function 112 to perform post-learning on the corresponding learning model.

In addition, the post-learning function 113 notifies the data set processing function 112 and company B's management server of the results of post-learning using the reconstructed training data set, as necessary.

The term "processor" used in the above description refers to circuits such as a CPU (Central Processing Unit), a GPU (Graphical processing unit), or an application specific integrated circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), and a Field Programmable Gate Array (FPGA)). The processor realizes its functions by reading and executing the program stored in the memory circuit 10. Note that instead of storing the program in the memory circuit 10, the program may be directly built into the processor circuit. In this case, the processor realizes its functions by reading and executing the program built into the circuit.

The input circuitry 12 is a circuit that inputs signals from input devices such as a pointing device (such as a mouse) or a keyboard that can be operated by an operator, and here, the input devices themselves are also included in the input circuitry 12. When the input device is operated by the operator, the input circuitry 12 generates an input signal corresponding to the operation and outputs it to the processing circuitry 11. The medical information processing device 1 may also be equipped with a touch panel in which the input device is integrated with the display circuitry 14.

The input circuitry 12 is realized by a trackball for setting a region of interest (ROI), a switch button, a mouse, a keyboard, a touchpad for performing input operations by touching the operation surface, a touchpad for performing input operations by touching the operation surface, a touch screen in which a display screen and a touchpad are integrated, a non-contact input circuit using an optical sensor, and a voice input circuit and a touch panel display in which a display screen and a touchpad are integrated, etc.

The input circuit 12 is not limited to being equipped with physical operating parts such as a mouse and a keyboard. For example, an example of the input circuit 12 includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.

The display circuit 14 is a display that displays images, and is configured with an LCD (Liquid Crystal Display, etc.). The display circuit 14 displays various display information such as various operation screens and image data on the LCD in response to instructions from the processing circuit 11.

The communication I/F (interface) circuit 13 communicates with an external device in accordance with a specified communication standard. When the medical information processing device 1 is provided on a network, the communication I/F circuit 13 transmits and receives information to and from external devices on the network. For example, the communication I/F circuit 13 receives data obtained by imaging from a medical image diagnostic device such as an MRI device or a medical image management device.

(Reconstruction process of the data set for the learning model)
Next, the reconstruction process of the learning model dataset will be described.

First, we will explain a typical scenario in which the process of reconstructing a dataset for a learning model is performed.

When the initial training dataset is constructed by the dataset processing function 112, each piece of data contained in the training dataset and the spare dataset is usable data. However, there are cases where this data becomes unusable data later, for example, when the patient who provided the data or their relatives request that the data not be used. In such cases, there is a possibility that the learning model trained using the training dataset that includes the unusable data will also become unusable.

For this reason, it is necessary to reconstruct the training dataset by replacing the unusable data in the training dataset with the usable data in the spare dataset, and then to post-train the learning model using the reconstructed training dataset. Furthermore, it is necessary to ensure that the results of such post-training have the same level of accuracy as the learning model before the post-training.

Below, we explain the reconstruction process of the learning model dataset and the post-learning process.

Figure 4 is a flowchart explaining the process of reconstructing the learning model dataset and the post-learning process.

As shown in FIG. 4, first, the dataset processing function 112 receives a data processing instruction for unusable data from the privacy management function 111 (step S1). The dataset processing function 112 also acquires information about the unusable data (i.e., information for identifying which data of which patient is unusable data) (step S2).

The dataset processing function 112 uses the acquired information about the unusable data to identify the unusable data in the training dataset (step S3).

The dataset processing function 112 calculates the similarity between each data in the spare dataset and the identified unusable data, and determines candidate data to replace the identified unusable data according to the calculated similarity (step S4). At this time, the spare dataset may contain data provided by the same provider as the identified unusable data. Such data provided by the same provider as the identified unusable data is excluded from the similarity calculation and is not subject to replacement.

The dataset processing function 112 replaces the first candidate data with the highest similarity in the spare dataset with the unusable data in the training dataset (step S5). At this time, the privacy tag of the patient ID is removed from the replaced unusable data to indicate "unusable," and the data is stored and managed in the spare dataset as spare data.

In addition, the replaced unusable data may be deleted from the backup data set as necessary after the privacy tag of the patient ID is removed to indicate that the data is "unusable."

If other unusable data exists (Yes in step S6), the dataset processing function 112 also performs the processing of steps S4 and S5 on the other unusable data.

If there is no other unusable data (No in step S6), the post-learning function 113 uses the reconstructed training dataset to perform post-learning on the corresponding learning model (step S7).

The post-learning function 113 determines whether the accuracy of the new learning model obtained as a result of the post-learning is equal to or greater than a certain value (step S8).

On the other hand, if the dataset processing function 112 determines that the accuracy of the new learning model is below a certain level (No in step S8), it replaces the first candidate data with the highest similarity with the second candidate data with the second highest similarity, and reconstructs the training dataset (step S9). The post-learning function 113 uses the training dataset in which the first candidate data has been replaced with the second candidate data to perform post-learning on the corresponding learning model (step S7).

The post-learning function 113 replaces candidate data based on the similarity calculated in step S3 and repeatedly executes the processes from steps S7 to S9 until the accuracy of the learning model newly generated by post-learning reaches a certain value.

On the other hand, if the post-learning function 113 determines that the accuracy of the new learning model is equal to or greater than a certain value (Yes in step S8), it ends the post-learning and completes the generation of the learning model using the reconstructed training dataset. In addition, the post-learning function 113 notifies the management server of company B, as necessary, that the post-learning using the reconstructed training dataset has been completed.

The medical information processing device according to this embodiment includes a learning model dataset as a database, a dataset processing function 112 as a processing unit, and a post-learning function 113 as a post-learning unit. The learning model dataset stores a training dataset as a first dataset including multiple training data used as training data for the learning model, and a spare dataset as a second dataset including multiple spare data having the same data structure as the training data. The dataset processing function 112 replaces unusable training data among the multiple training data included in the training dataset with usable spare data among the multiple spare data included in the spare dataset, and processes (reconstructs) the training dataset.

Therefore, even if restrictions are subsequently placed on the use of data included in the training dataset, the training dataset can be easily reconstructed, and the total number and variation of data used in training (learning) can be kept constant. As a result, the dataset required for post-learning can be easily and quickly updated, and even if restrictions are subsequently placed on the use of training data, it is possible to ensure that the newly retrained model has the same performance as the learning model that was previously used, and it is possible to provide a learning model that always has a certain level of accuracy.

It also eliminates the need to replace data used in learning models used in medical systems and the need to re-train learning models. As a result, even if restrictions on the use of data included in the training dataset are subsequently imposed, users can ensure the accuracy of the learning model without having to perform any specific tasks.

(Variation 1)
In the above embodiment, the medical information processing device 1 has the learning model data sets 10-1 to 10-n as a database, the privacy management function 111, and the post-learning function 113. In contrast, the learning model data sets 10-1 to 10-n, the privacy management function 111, and the post-learning function 113 may be provided in a distributed or collectively manner in at least one device that can communicate with the medical information processing device 1 via a network, such as a server on a cloud.

In this configuration, the medical information processing device 1 is configured as a medical image information processing system.

(Variation 2)
In the above embodiment, the data set processing function 112 reconstructs the training data set in response to a command from the privacy management function. Alternatively, the data set processing function 112 may reconstruct the training data set in response to a command from another server device managed by company B.

In the above embodiment, the post-learning function 113 automatically executes post-learning when the training dataset is reconstructed by the dataset processing function 112b. In contrast, the post-learning function 113 can also start post-learning of the learning model at any time after the learning model database is reconstructed. For example, the post-learning function 113 may execute post-learning when triggered by a command from another server device managed by company B or a separate manual input instruction.

(Variation 3)
In the above embodiment, the training dataset is reconstructed when a data processing instruction for unusable data is received from the privacy management function 111. Alternatively, the training dataset may be reconstructed when the number of newly registered patient-provided information in the patient-provided information database reaches a certain value. In this case, not only the training dataset but also the spare dataset may be replaced with new data in the patient-provided information database. In addition, it is preferable that the old spare data that is replaced from the spare dataset is released from management as a learning model dataset.

In this modified example, it is of course preferable that the post-learning function 113 repeatedly reconstructs the training dataset until the accuracy of the new training data is guaranteed.

(Variation 4)
When the ratio of usable data of the spare dataset in the learning model dataset falls below a threshold, the dataset processing function 112 preferably reconstructs the spare dataset. At this time, the dataset processing function 112 reconstructs the spare dataset by acquiring new usable data from the patient-provided information database and replacing the unusable data in the spare dataset with the new usable data. In addition, in such reconstruction, it is preferable that the old spare data that is to be replaced from the spare dataset is released from management as a learning model dataset.

(Variation 5)
The training data in the learning model data sets 10-1 to 10-n may include, for example, information acquired by a device that is no longer in use, information collected under inappropriate conditions due to changes in medical guidelines, etc. Such training data (called "disqualified data") cannot be used for training the learning model even if permission to use it has been obtained from the patient.

Therefore, when it is found that such disqualified data is included in the learning model dataset, it is preferable that the dataset processing function 112 automatically reconstructs the training model database by replacing it with the unusable data in the spare dataset. In addition, in such reconstruction, it is preferable that the old spare data that is the target of replacement from the spare dataset is released from management as a learning model dataset.

In this modified example, it is of course preferable that the post-learning function 113 repeatedly reconstructs the training dataset until the accuracy of the new training data is guaranteed.

According to at least one of the embodiments described above, even if restrictions are subsequently imposed on the use of training data used in a learning model, it is possible to easily reconstruct the training data set.

Although several embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, modifications, and combinations of embodiments can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as set forth in the claims, as well as in the scope and spirit of the invention.

1 Medical information processing device 10 Memory circuit 10-1 to 10-n Learning model data set 10-1a to 10-na Training data set 10-1b to 10-nb Spare data set 11 Processing circuit 12 Input circuit 13 Communication I/F unit 14 Display circuit 111 Privacy management function 112 Data set processing function 113 Post-learning function

Claims (12)

A database storing a first dataset including a plurality of training data managed for each patient, which are used as training data for a learning model, and a second dataset including a plurality of preliminary data having the same data structure as the training data;
an information management unit that generates identification information that identifies unusable privacy information among the plurality of training data included in the first dataset according to a usage restriction on at least a portion of the privacy information associated with each patient; and
a processing unit that replaces the unusable training data identified by the identification information generated by the information management unit, among the plurality of training data included in the first data set, with the usable spare data among the plurality of spare data included in the second data set, and processes the first data set;
A medical information processing device comprising: the training data includes at least one of patient information, genetic information, and image information relating to a patient , and a tag for collectively managing whether or not privacy information included in the training data can be used for each patient ;
The processing unit processes the first data set based on the tag.
The medical information processing device according to claim 1 . the training data includes at least one of patient information, genetic information, and image information relating to a patient, and a tag for individually managing the availability of privacy information included in the training data for each patient;
The processing unit processes the first data set based on the tag.
The medical information processing device according to claim 1 . The medical information processing device according to any one of claims 1 to 3, wherein the processing unit performs the replacement based on the similarity between the unusable training data included in the first data set and each of the multiple spare data included in the second data set. The medical information processing device according to claim 4, wherein the processing unit calculates the similarity using at least one of an imaging protocol, an imaging program, patient information, imaging device setting information at the time of imaging, and image information. The medical information processing device according to claim 4 or 5, further comprising a post-learning unit that post-learns the learning model using the processed first data set. The processing unit further processes the first data set based on the similarity when the post-trained learning model does not satisfy a predetermined accuracy,
The medical image processing apparatus according to claim 6 , wherein the post-learning unit performs post-learning on the learning model using the first data set that has been further processed. The medical information processing device according to any one of claims 1 to 7, wherein, when there are multiple pieces of unusable training data among the multiple pieces of training data included in the first data set, the processing unit replaces each of the multiple pieces of unusable training data with the spare data that is usable among the multiple spare data included in the second data set, and processes the first data set. The medical information processing device according to any one of claims 1 to 8, wherein the processing unit reconstructs the first data set when a certain number of pieces of patient-provided information are newly registered in the patient-provided information database. The medical information processing device according to any one of claims 1 to 9, wherein the processing unit reconstructs the first data set when the ratio of usable data in the second data set falls below a threshold value. The medical information processing device according to any one of claims 1 to 10, wherein the processing unit reconstructs at least one of the first data set and the second data set when at least one of the first data set and the second data set includes disqualification data that is inappropriate for use in training. A database storing a first dataset including a plurality of training data managed for each patient, which are used as training data for a learning model, and a second dataset including a plurality of preliminary data having the same data structure as the training data;
an information management unit that generates identification information that identifies unusable privacy information among the plurality of training data included in the first dataset according to a usage restriction on at least a portion of the privacy information associated with each patient; and
a processing unit that replaces the unusable training data identified by the identification information generated by the information management unit, among the plurality of training data included in the first data set, with the usable spare data among the plurality of spare data included in the second data set, and processes the first data set;
A medical information processing system comprising:

Images