CN111382132A - Medical image data cloud storage system - Google Patents
Medical image data cloud storage system Download PDFInfo
- Publication number
- CN111382132A CN111382132A CN201811623760.2A CN201811623760A CN111382132A CN 111382132 A CN111382132 A CN 111382132A CN 201811623760 A CN201811623760 A CN 201811623760A CN 111382132 A CN111382132 A CN 111382132A
- Authority
- CN
- China
- Prior art keywords
- archive
- node
- preprocessing
- slave
- cluster
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
Landscapes
- Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Engineering & Computer Science (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a medical image data cloud storage system, which comprises: a database; the file transmission cluster is used for storing files related to the medical image data and transmitting the files; the filing cluster is used for filing the files in the file transmission cluster and storing filing results to the database; and the preprocessing cluster is used for preprocessing the medical image data. The medical image data cloud storage system adopts the cheap storage server to store mass images, so that the storage cost is reduced; each functional block is allocated with work and is definitely deployed by adopting a cluster, the cluster characteristics are exerted, and the problem of efficiency of storage and reading of mass data is solved by distributed deployment; the traditional film storage approach is finalized.
Description
Technical Field
The invention mainly relates to the field of data storage, in particular to a medical image data cloud storage system.
Background
In the conventional medical image data storage scheme, various kinds of medical image data are concentrated in the hospitals or several hospitals having a hierarchical relationship. In the storage scheme, image data is basically stored in a medical image archiving and communication system (PACS) by using MySQL as a database, if image data is expanded in the future, data migration is performed, the storage cost is more and more expensive along with the increase of data, the efficiency of data migration is more and more low, the data cannot be shared, and local solidification is not favorable for the development of an image cloud of a telemedicine network system in a large trend.
Disclosure of Invention
The invention aims to provide a medical image data cloud storage system which efficiently solves the storage problem of medical image big data.
In order to solve the technical problem, the invention provides a medical image data cloud storage system, which comprises: a database; the file transmission cluster is used for storing files related to the medical image data and transmitting the files; the filing cluster is used for filing the files in the file transmission cluster and storing filing results to the database; and the preprocessing cluster is used for preprocessing the medical image data.
In an embodiment of the present invention, the file transfer cluster includes: a plurality of file transfer slave nodes for storing and transferring the files; the file transmission master node is used for managing the file transmission slave nodes; and the file transmission standby node is used for taking over the file transmission main node to manage the file transmission slave node when the file transmission main node fails.
In an embodiment of the present invention, the file transfer slave node further sends a heartbeat packet to the file transfer master node or the file transfer slave node, so as to feed back the state of the file transfer slave node to the file transfer master node or the file transfer slave node.
In an embodiment of the present invention, the archive cluster includes: a plurality of filing slave nodes, configured to file the file, and store the filing result in the database; an archive master node for managing the archive slave nodes; and the archiving node is used for taking over the archiving master node to manage the archiving slave node when the archiving master node fails.
In an embodiment of the invention, after archiving the file, the archiving slave node stores the file to a different area in the database according to the type of the business data.
In an embodiment of the present invention, the archive slave node further sends a heartbeat packet to the archive master node or the archive backup node to feed back the status of the archive slave node to the archive master node or the archive backup node.
In an embodiment of the present invention, the archive master node or the archive backup node receives a heartbeat packet of the archive slave node to monitor a status of the archive slave node.
In an embodiment of the present invention, the archive master node or the archive backup node assigns an archive task to one or more of the archive slave nodes depending on the state of the archive slave nodes.
In an embodiment of the present invention, when the archive slave node succeeds in archiving the file, the archive master node or the archive backup node writes an archive success flag to the database.
In an embodiment of the present invention, the preprocessing cluster includes: the plurality of preprocessing slave nodes are used for preprocessing the medical image data and storing preprocessing results to the database; the preprocessing master node is used for managing the preprocessing slave nodes; the preprocessing standby node is used for taking over the preprocessing master node to manage the preprocessing slave node when the preprocessing master node fails; when the preprocessing slave node finishes executing preprocessing, the preprocessing master node or the preprocessing slave node writes a preprocessing completion mark into the database.
Compared with the prior art, the invention has the following advantages: the medical image data cloud storage system adopts the cheap storage server to store mass images, so that the storage cost is reduced; each functional block is allocated with work and is definitely deployed by adopting a cluster, the cluster characteristics are exerted, and the problem of efficiency of storage and reading of mass data is solved by distributed deployment; the traditional film storage approach is finalized.
Drawings
Fig. 1 is a schematic diagram of a medical image data cloud storage system according to some embodiments of the invention.
Fig. 2 is a schematic block diagram of a file transfer master node of some embodiments of the present invention.
FIG. 3 is a schematic deployment diagram of an archive cluster of some embodiments of the invention.
FIG. 4 is an archive flow diagram of an archive cluster of some embodiments of the invention.
Fig. 5 is a schematic flow diagram of a timing detection task of some embodiments of the present invention.
FIG. 6 is a schematic flow chart diagram of assigning archiving tasks in accordance with some embodiments of the present invention.
FIG. 7 is a schematic flow chart diagram of archiving task state changes in accordance with some embodiments of the present invention.
FIG. 8 is a schematic flow diagram of an archive retrieval request from a node in accordance with some embodiments of the invention.
FIG. 9 is a schematic flow diagram of an archive processing by an archive slave node of some embodiments of the invention.
Fig. 10 is an icebox server deployment diagram of an archival host node of some embodiments of the invention.
FIG. 11 is a schematic deployment diagram of an archive cluster of some embodiments of the invention.
Fig. 12 is a schematic block diagram of a preprocessing master node of some embodiments of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described herein, and thus the present invention is not limited to the specific embodiments disclosed below.
As used in this application and the appended claims, the terms "a," "an," "the," and/or "the" are not intended to be inclusive in the singular, but rather are intended to be inclusive in the plural unless the context clearly dictates otherwise. In general, the terms "comprises" and "comprising" merely indicate that steps and elements are included which are explicitly identified, that the steps and elements do not form an exclusive list, and that a method or apparatus may include other steps or elements.
Fig. 1 is a schematic diagram of a medical image data cloud storage system according to some embodiments of the invention. Referring to fig. 1, the medical image data cloud storage system 100 includes a database 110, a file transfer cluster 130, an archive cluster 140, and a preprocessing cluster 150. The medical image data cloud storage system 100 mainly has a function of archiving patient diagnosis reports, cloud films, DICOM files and preprocessed data in the cloud gateway.
In some embodiments, medical image data cloud storage system 100 further includes a management cluster 120. Management cluster 120 is used to manage file transfer cluster 130 and archive cluster 140 to ensure high availability of management file transfer cluster 130 and archive cluster 140. In some embodiments, the management cluster 120 may be a Zookeeper cluster.
The file transfer cluster 130 is used to store files related to medical images and transfer the files. In some embodiments, file transfer cluster 130 employs a distributed architecture to support dynamic capacity expansion, that is, the number of file transfer slave nodes (described in detail below) may be dynamically increased. In some embodiments, file transfer cluster 130 employs a master-standby mode to ensure high availability of file transfer cluster 130.
In some embodiments, file transfer cluster 130 may include a file transfer master node 131, a file transfer standby node 132, and a plurality of file transfer slave nodes 133. The file transfer master node 131 is used to manage the file transfer slave node 133. The file transfer standby node 132 is configured to take over the file transfer master node 131 to manage the file transfer slave node 133 when the file transfer master node 131 fails. The file transfer slave node 133 is used to store and transfer files related to medical images.
In some embodiments, the file transfer slave node 133 also sends a heartbeat packet to the file transfer master node 131 or the file transfer slave node 132 to feed back the status of the file transfer slave node 133 to the file transfer master node 131 or the file transfer slave node 132. The state of the file transfer slave node 133 may include the operational state of the file transfer slave node 133 and/or hardware resource usage. The running state may include, for example, a state that a file is being forwarded, a state that a file is not being forwarded, and the like. Hardware resource usage may include, for example, one or more of memory usage, CPU utilization, disk IO usage, and network bandwidth occupancy.
The file transfer master node 131 or the file transfer standby node 132 receives a heartbeat packet of the file transfer slave node 133 to monitor the status of the file transfer slave node 133. The file transfer master node 131 or the file transfer standby node 132 may also establish a file transfer pipe for one or more file transfer slave nodes 133 according to the status of the file transfer slave nodes 133. For example, a less heavily loaded file transfer slave node 133 may be selected to establish a file transfer pipe based on the usage of hardware resources of the file transfer slave node 133. In some embodiments, the primary file transfer node 131 or the standby file transfer node 132 may use rack sensing to establish the file transfer pipe. After the file transfer pipe is established, the file transfer slave node 133 acquires the file transfer pipe from the file transfer master node 131 or the file transfer standby node 132, and transfers a file based on the file transfer pipe. In some embodiments, the file transfer master node 131 or the file transfer standby node 132 also stores the file transfer pipe to the database 110 for use by the archive cluster 140 and/or the pre-processing cluster 150.
In some embodiments, the file transfer master node 131 registers a corresponding master node with the management cluster 120, and the file transfer slave node 132 listens to the corresponding master node in the management cluster 120, so that when the file transfer master node 131 fails (e.g., goes down), the file transfer slave node 132 receives the message and replaces the file transfer master node 131, and synchronizes the data stored in the management cluster 120 by the file transfer master node 131. Specifically, the file transfer master node 131 stores critical data to the management cluster. The critical data may include file transfer pipe and/or rack information. File transfer standby node 132 monitors the status of file transfer master node 131 via management cluster 120 and synchronizes critical data from management cluster 120 on a timed basis. When the file transfer standby node 132 detects that the file transfer master node 131 fails, the file transfer standby node 132 acquires key data from the management cluster 120 to take over the management of the file transfer slave node 133 by the file transfer master node 131.
Fig. 2 is a schematic block diagram of a file transfer master node of some embodiments of the present invention. Referring to fig. 2, the file transfer master node 131 includes a file transfer slave node status monitoring module 131a, a management cluster data import module 131b, a database data import module 131c, a rack sensing module 131d, and a file transfer pipe establishment module 131 e. The file transfer slave node status monitoring module 131a is used for monitoring the status of the file transfer slave node 133. Management cluster data import module 131b is used to import data from management cluster 120. The database data import module 131c is used for importing data from the database 110. The rack sensing module 131d is used for sensing rack information. The file transfer pipe establishing module 131e is used for establishing a file transfer pipe.
The archive cluster 140 is used to archive the files in the file transfer cluster 130 and store the archive results to the database 110. In some embodiments, archive cluster 140 employs a distributed architecture to support dynamic expansion, that is, the number of archive slave nodes (described in detail below) may be dynamically increased. In some embodiments, archive cluster 140 employs a primary-standby mode to ensure high availability of archive cluster 140.
FIG. 3 is a schematic deployment diagram of an archive cluster of some embodiments of the invention. Referring collectively to fig. 1 and 3, archive cluster 140 may include an archive master node 141, an archive slave node 142, and a plurality of archive slave nodes 143. The archive master node 141 is used to manage the archive slave node 143. The archive backup node 142 is used to take over the management of the archive slave node 143 by the archive master node 141 when the archive master node 141 fails. The plurality of archive slave nodes 143 are used to archive files and store archive results to the database 110. In some embodiments, after archiving the file, the archive is stored from node 143 to a different area in database 110 according to the business data type, for example to one or more of HBase index table 111, HDFS file system 112, and SOLR enterprise search platform 113 according to the business data type.
FIG. 4 is an archive flow diagram of an archive cluster of some embodiments of the invention. Referring to fig. 4, an archive flow of an archive cluster may include:
step 401: the archiving slave node 143 reports the heartbeat packet to the archiving master node 141;
step 402: the archiving master node 141 obtains a list of files to be downloaded on the file transfer cluster 130 from the database 110;
step 403: the database 110 returns a list of files to be downloaded to the filing master node 141;
step 404: the archiving master node 141 caches a list of files to be downloaded;
step 405: the archiving master node 141 returns archiving task information to the archiving slave node 143 according to a heartbeat protocol;
step 406: the filing slave node 143 requests the file transfer cluster 130 to establish a file transfer pipeline and requests to download files in a file list to be downloaded;
step 407: file transfer cluster 130 returns file data to archive slave node 143;
step 408: the filing slave node 143 decompresses the downloaded file data and screens a data source to perform filing operation;
step 409: after the archiving operation is completed, the archiving slave node 143 writes the archiving result into the designated area of the database 110 according to different business needs;
step 410: the filing slave node 143 reports the task completion condition to the filing master node 141;
step 411: after receiving the task completion reported by the filing slave node 143, the filing master node 141 writes a corresponding task completion flag into the database 110.
The communication schemes of the archive master node 141, archive backup node 142, archive slave node 143, and archive cluster 140 will be described in detail below.
Archiving host node
When the archive master node 141 is started, it is determined whether it is the master node of the archive cluster 140 according to the API provided by the management cluster 120, and if so, the archive master node 141 starts to operate with the master node.
In step 402, the file to be downloaded list obtained by the filing master node 141 is a list of filenames whose flag bits are not downloaded in the database 110. In some embodiments, archive master node 141 reads the list of filenames whose flag bits are not downloaded in HBase index table 111.
At step 404, the archiving master node 141 may store the list of files to be downloaded to a bounded data structure, such as a bounded blocked queue. The upper limit of the bounded data structure may be, for example, 30 files. In some embodiments, the archiving master node 141 may periodically monitor the bounded data structure and when the bounded data structure is empty, the archiving master node 141 re-reads the list of files from the database 110. More specifically, the archiving master node 141 may initiate a timed detection task to periodically query the status of the bounded data structure, and when the bounded data structure is queried to be empty, the archiving master node 141 re-reads the list of files from the database 110.
In some embodiments, the archival master node 141 also caches addresses and timestamps of the archival slave node 143 and monitors the validity of the cache in a timed detection task. The time stamp may be, for example, 60 seconds.
Fig. 5 is a schematic flow diagram of a timing detection task of some embodiments of the present invention. Referring to fig. 5, the process of timing detection task includes:
step 501: starting a timing task;
step 502: when the time arrives, detecting whether the filing file list is empty; if the result is null, executing step 503, otherwise, executing step 505;
step 503: acquiring an undelivered file list from a database;
step 504: filling the filing queue with the file list;
step 505: detecting whether expired archive slave node information exists in a cache; if yes, executing step 506, and if not, returning to step 501;
step 506: the expired cache is cleared and the process returns to step 501.
With continued reference to fig. 4, at step 401, the archiving master node 141 receives a heartbeat packet reported by the archiving slave node 143 to monitor the status of the archiving slave node 143. The state of the archive slave node 143 includes one or more of the address, load, filename currently archived, and archive condition of the archive slave node 143.
At step 405, archive master node 141 assigns archive tasks to one or more archive slave nodes 143 based on the status of archive slave nodes 143. In some embodiments, a simple load policy may be set, that is, the archive slave node 143 does not archive any one of the CPU load, the memory load, and the network IO load more than a predetermined ratio. The predetermined ratio may be, for example, 80%.
FIG. 6 is a schematic flow chart diagram of assigning archiving tasks in accordance with some embodiments of the present invention. Referring to FIG. 6, the process by which archive master node 141 assigns archive tasks includes:
step 601: the archive master node 141 listens for a heartbeat of the archive slave node 143;
step 602: the archival master node 141 updates the cached time stamp of the archival slave node 143;
step 603: the archive master node 141 detects whether the archive file queue is empty; if the status is empty, returning to the step 601, otherwise, executing the step 604;
step 604: the archiving master node 141 acquires an archiving file and an archiving condition of the archiving slave node 143 in the heartbeat package;
step 605: judging whether the archiving slave node 143 has an archiving task; if yes, go to step 606, otherwise go to step 608;
step 606: determining whether the archive task of the archive slave node 143 was successful; if successful, go to step 607, if unsuccessful, go to step 608;
step 607: the archiving master node 141 writes an archiving success flag to the database 110;
step 608: the archive master node 141 assigns a new archive task to the archive slave node 143 and returns to step 601.
With continued reference to fig. 4, in step 411, when the archiving master node 141 receives the heartbeat packet reported by the archiving slave node 143 and knows that the archiving condition of the archiving slave node 143 is successful from the heartbeat packet, the archiving master node 141 writes an archiving success flag into the database 110.
FIG. 7 is a schematic flow chart diagram of archiving task state changes in accordance with some embodiments of the present invention. Referring to FIG. 7, the process of archiving a task state change includes:
step 701: the archive slave node 143 reports its initial state with a heartbeat. The initial state includes: the archive file name is null and the archive status is "unprocessed".
Step 702: after receiving the heartbeat packet, the archive master node 141 replies with a new archive task to the archive slave node 143. The archiving task includes: the archive file name is an archive file, and the archive state is "unprocessed".
Step 703: upon receiving the archive task from the archive slave node 143, the archive state is set to "in process".
Step 704: the archive slave node 143 performs an archive process on the archive file.
Step 705: the archive slave node 143 sets the archive status to "success/failure" after the archive is completed.
Step 706: the archive slave node 143 reports its current archive state on a heartbeat.
Step 707: after receiving the heartbeat packet, the archive master node 141 replies with a new archive task to the archive slave node 143. The archiving task includes: the archive file name is a new archive file and the archive status is "unprocessed".
Archival node
Archiving slave nodes
The archive slave node 143 sends a heartbeat packet to the archive master node 141 to feed back the status of the archive slave node 143 to the archive master node 141. The state of the archive slave node 143 includes one or more of the address, load, filename currently archived, and archive condition of the archive slave node 143. In some embodiments, the archival slave node 143 may periodically send heartbeat packets to the archival master node 141. The period may be, for example, every 15 seconds.
If the archive file name in the heartbeat packet reported by the archive slave node 143 is null, that is, the archive slave node 143 is in a state without an archive task, the archive master node 141 issues an archive task to the archive slave node 143, and the archive slave node 143 performs the archive. In some embodiments, the archive slave node 143 may perform archive processing on the file through a thread pool. The number of thread pools may be, for example, one more than the number of processors owned by the archive slave node 143. It is understood that the number of the thread pool may be other values, and the invention is not limited thereto.
FIG. 8 is a schematic flow diagram of an archive retrieval request from a node in accordance with some embodiments of the invention. Referring to fig. 8, the process of acquiring an archive task from a node by an archive includes:
step 801: the archive slave node 143 monitors its own load;
step 802: judging whether to report the heartbeat; if yes, go to step 803, if no, return to step 801;
step 803: the archiving slave node 143 reports the heartbeat to the archiving master node 141;
step 804: receiving a response of the correspondent master node 141 from the archive slave node 143;
step 805: the archive slave node 143 determines whether an archive task is issued in the response; if yes, executing step 806, otherwise, returning to step 801;
step 806: the archive slave node 143 processes archive tasks asynchronously.
In step 802, the archival slave node 143 determines whether to report a heartbeat based on whether a set period has arrived. Taking the 15 second period as an example, when the period is 1-14 seconds, the archive slave node 143 determines not to report the heartbeat, and when the 15 th second arrives, the archive slave node 143 determines to report the heartbeat.
FIG. 9 is a schematic flow diagram of an archive processing by an archive slave node of some embodiments of the invention. Referring to fig. 9, the process of performing the archive processing by the archive slave node includes:
step 901: the archive slave node 143 establishes a connection with the file transfer cluster 130;
step 902: archive slave node 143 downloads the data source from file transfer cluster 130;
step 903: archive slave node 143 decompresses the data source;
step 904: the archive slave node 143 parses the data source;
step 905: archive traverses the data source from node 143;
step 906: archiving different types of files in the data source processed by the slave node 143;
step 907: the archiving slave node 143 writes the archiving result into the HBase index table;
step 908: the archive slave node 143 writes the archive results to the SOLR enterprise search platform;
step 909: archiving short messages sent from node 143 to the portal;
step 910: the archive slave node 143 records the completion of the archive task.
At step 906, the archival slave node 143 may perform one or more of processing reports, processing films, processing Jpeg lossy photos, processing dicom images, and processing pre-processed data.
It should be noted that the archive slave node 143 may write the archive result to the HBase index table and/or the SOLR enterprise search platform according to the type of the business data. That is, the process of performing the archive processing at the archive slave node may include one or more of steps 907, 908. Furthermore, step 907 and step 908 may be in any order, for example, step 907 and step 908 are performed simultaneously, and step 908 is performed before step 907.
In some embodiments, the archival slave node 143 may also report a heartbeat to the archival master node 141 after the archival is complete. In some alternative embodiments, the archive slave node 143 may only report the status of the archive task as successful or failed.
In some embodiments, the archive slave node 143 may also flush the archive task from the cache after a heartbeat report succeeds. For an archive task completed between two heartbeat reports (i.e., one heartbeat interval), the archive slave node 143 reports the archive task completed within the heartbeat interval when the latter of the two heartbeat packets is reported.
Communication scheme for archive clusters
The archive cluster 140 may communicate based on IceBoxServer and IceRegistry, so the archive cluster 140 environment requires installation of an Ice. The archival master node 141 provides a Service (Service) for listening to the reports of the archival slave node 143. The archive slave node 143, as a Client (Client), calls a heartbeat report Remote Procedure Call (RPC) and processes a reply of the archive master node 141. The IceRegistry is used as a monitoring registration agent of the Ice Service, and the problem of communication address drift during the master-slave switching is solved. The client and the server communicate by locating to a locator (locator) pointed to by the IceRegistry. IceRegistry is deployed in a main standby mode, so that the stability is improved.
Fig. 10 is an icebox server deployment diagram of an archival host node of some embodiments of the invention. Referring to fig. 10, the IceBox server in the archive main node 141 provides StatusReport as a Service (Service) in the IceBox. StatusReport listens for requests from the fixed port. Properties as the startup configuration of icebox server.
FIG. 11 is a schematic deployment diagram of an archive cluster of some embodiments of the invention. Referring to fig. 11, the archive master node 141 starts up as a Server (Server) of the IceBox. The archival slave node 143 also starts as the Server (Server) of the IceBox, but does not listen to any Service (Service), which can take advantage of the configurable advantages of the IceBox to monitor the communication anomalies and optimization terms of the IceBox. IceRegistry is deployed in a master-slave mode. The server side of the archive master node 141 locates the IceRegistry master-slave address. The slave node 143 is archived as a client's location configuration to the IceRegistry master-slave address. The IceRegistry is responsible for positioning conversion when the client and the server communicate.
In some embodiments, the communication protocol for the archive cluster is defined as follows:
the archive cluster 140 uses the communication principles of remote procedure calls to facilitate communication among the archive cluster internal servers. The archive cluster 140 uses a cluster approach to download data from a data source, making full use of network bandwidth to archive the data in time to the maximum extent. The archive cluster 140 uses the IceBox to modify the synchronous communication mode between the original remote procedure call interfaces, thereby improving the concurrency performance of the transceiving ends in the cluster. IceRegistry is used as a monitoring registration agent of IceService, and the problem of communication address drift during cluster master-slave switching is solved. Archive cluster 140 supports stand-alone mode and cluster mode operation. The archive cluster 140 is responsible for reading configuration related to the database 110 environment, and is used as a configuration item written by the database 110, so that the archive cluster is convenient to deploy to different cluster environments.
With continued reference to fig. 1, the pre-processing cluster 150 is used to pre-process medical image data. In some embodiments, the pre-processing cluster 150 employs a distributed architecture to support dynamic capacity expansion, that is, the number of pre-processing slave nodes (described in detail below) may be dynamically increased. In some embodiments, pre-processing cluster 150 employs a master-standby mode to ensure high availability of pre-processing cluster 150.
In some embodiments, preprocessing slave node 153 also sends a heartbeat packet to preprocessing master node 151 or preprocessing slave node 152 to feed back the status of preprocessing slave node 153 to preprocessing master node 151 or preprocessing slave node 152. The state of the preconditioning slave node 153 includes the operational state and/or hardware resource usage of the preconditioning slave node 153. The run state may include, for example, a state of being preprocessed file, not being preprocessed file, etc. Hardware resource usage may include, for example, one or more of memory usage, CPU utilization, disk IO usage, and network bandwidth occupancy.
The preprocessing master node 151 or the preprocessing backup node 152 receives the heartbeat packet of the preprocessing slave node 153 to monitor the status of the preprocessing slave node 153. The preprocessing master node 151 or the preprocessing standby node 152 may assign preprocessing tasks to one or more preprocessing slave nodes 153 according to the state of the preprocessing slave nodes 153. Specifically, the preprocessing master node 151 or the preprocessing standby node 152 periodically obtains files that are not preprocessed from the database 110, and distributes the files to one or more preprocessing slave nodes 153 for downloading according to the states of the preprocessing slave nodes 153. In some embodiments, the pre-processing master node 151 or the pre-processing standby node 152 may select the less loaded pre-processing slave node 153 to perform the pre-processing according to the utilization rate of the hardware resources of the pre-processing slave node 153.
After the preprocessing slave node 153 receives the preprocessing task, it issues a start command to the preprocessing program to start preprocessing. After the preprocessing is completed, the preprocessing slave node 153 reports the preprocessing result to the preprocessing master node 151.
When the preprocessing slave 153 finishes preprocessing, the preprocessing master 151 or the preprocessing slave 152 writes a preprocessing completion flag into the database 110.
Fig. 12 is a schematic block diagram of a preprocessing master node of some embodiments of the present invention. Referring to fig. 12, the preprocessing master node 151 includes a preprocessing slave node status monitoring module 151a, a data distribution module 151b, and a database data modification module 151 c. The preprocessing slave node status monitoring module 151a is used to monitor the status of the preprocessing slave node 153. The data distribution module 151b is used to distribute data. For example, the preprocessing task is distributed to the preprocessing slave node 153. The database data modification module 151c is used for modifying corresponding data in the database 110. For example, after receiving the message that the preprocessing slave node 153 completes preprocessing, the preprocessing master node 151 modifies the corresponding record in the database 110.
This application uses specific words to describe embodiments of the application. Reference throughout this specification to "one embodiment," "an embodiment," and/or "some embodiments" means that a particular feature, structure, or characteristic described in connection with at least one embodiment of the present application is included in at least one embodiment of the present application. Therefore, it is emphasized and should be appreciated that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, some features, structures, or characteristics of one or more embodiments of the present application may be combined as appropriate.
Although the present invention has been described with reference to the present specific embodiments, it will be appreciated by those skilled in the art that the above embodiments are merely illustrative of the present invention, and various equivalent changes and substitutions may be made without departing from the spirit of the invention, and therefore, it is intended that all changes and modifications to the above embodiments within the spirit and scope of the present invention be covered by the appended claims.
Claims (10)
1. A medical image data cloud storage system, comprising:
a database;
the file transmission cluster is used for storing files related to the medical image data and transmitting the files;
the filing cluster is used for filing the files in the file transmission cluster and storing filing results to the database; and
and the preprocessing cluster is used for preprocessing the medical image data.
2. The medical image data cloud storage system of claim 1, wherein the file transfer cluster comprises:
a plurality of file transfer slave nodes for storing and transferring the files;
the file transmission master node is used for managing the file transmission slave nodes; and
and the file transmission standby node is used for taking over the file transmission main node to manage the file transmission slave node when the file transmission main node fails.
3. The medical image data cloud storage system according to claim 2, wherein the file transfer slave node further sends a heartbeat packet to the file transfer master node or the file transfer slave node to feed back the state of the file transfer slave node to the file transfer master node or the file transfer slave node.
4. The medical image data cloud storage system of claim 2, wherein the archive cluster comprises:
a plurality of filing slave nodes, configured to file the file, and store the filing result in the database;
an archive master node for managing the archive slave nodes; and
and the archiving node is used for taking over the archiving master node to manage the archiving slave node when the archiving master node fails.
5. The medical image data cloud storage system according to claim 4, wherein after the file is archived, the archive slave node stores the archived file into a different area in the database according to a business data type.
6. The medical image data cloud storage system of claim 4, wherein the archive slave node further sends a heartbeat packet to the archive master node or the archive backup node to feed back the status of the archive slave node to the archive master node or the archive backup node.
7. The medical image data cloud storage system of claim 4, wherein the archive master node or the archive backup node receives a heartbeat packet of the archive slave node to monitor a status of the archive slave node.
8. The medical image data cloud storage system of claim 7, wherein the archive master node or the archive backup node assigns an archive task to one or more of the archive slave nodes according to the state of the archive slave nodes.
9. The medical image data cloud storage system of claim 4, wherein the archive master node or the archive preparation node writes an archive success flag to the database when the archive slave node succeeds in archiving the file.
10. The medical image data cloud storage system of claim 1, wherein the preprocessing cluster comprises:
the plurality of preprocessing slave nodes are used for preprocessing the medical image data and storing preprocessing results to the database;
the preprocessing master node is used for managing the preprocessing slave nodes; and
the preprocessing standby node is used for taking over the preprocessing master node to manage the preprocessing slave node when the preprocessing master node fails;
when the preprocessing slave node finishes executing preprocessing, the preprocessing master node or the preprocessing slave node writes a preprocessing completion mark into the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811623760.2A CN111382132A (en) | 2018-12-28 | 2018-12-28 | Medical image data cloud storage system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811623760.2A CN111382132A (en) | 2018-12-28 | 2018-12-28 | Medical image data cloud storage system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111382132A true CN111382132A (en) | 2020-07-07 |
Family
ID=71218062
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811623760.2A Pending CN111382132A (en) | 2018-12-28 | 2018-12-28 | Medical image data cloud storage system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111382132A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671594A (en) * | 2021-02-21 | 2021-04-16 | 新疆医科大学第一附属医院 | Medical image data cloud storage filing monitoring system |
| CN115174447A (en) * | 2022-06-27 | 2022-10-11 | 京东科技信息技术有限公司 | Network communication method, device, system, equipment and storage medium |
| CN118210757A (en) * | 2024-05-20 | 2024-06-18 | 杭州政云数据技术有限公司 | Credential processing method, apparatus, device and storage medium |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103036956A (en) * | 2012-11-30 | 2013-04-10 | 航天恒星科技有限公司 | Filing system and implement method of distributed configured massive data |
| CN103365972A (en) * | 2013-06-27 | 2013-10-23 | 北京中科金财科技股份有限公司 | Intelligent processing system for mass data |
| CN105049504A (en) * | 2015-07-09 | 2015-11-11 | 国云科技股份有限公司 | Big data transit transmission synchronization and storage method |
| US20160378844A1 (en) * | 2015-06-25 | 2016-12-29 | International Business Machines Corporation | Data synchronization using redundancy detection |
| CN106534249A (en) * | 2016-09-21 | 2017-03-22 | 苏州市广播电视总台 | File transmission system based on file straight-through technology |
| CN107391936A (en) * | 2017-07-25 | 2017-11-24 | 南京慧目信息技术有限公司 | A kind of ophthalmology image based on distributed website integrates and dissemination method |
| CN108540511A (en) * | 2017-03-03 | 2018-09-14 | 全球能源互联网研究院 | A kind of method of data synchronization towards Hadoop clusters |
| CN108845865A (en) * | 2018-06-28 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of monitoring service dispositions method, system and storage medium |
-
2018
- 2018-12-28 CN CN201811623760.2A patent/CN111382132A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103036956A (en) * | 2012-11-30 | 2013-04-10 | 航天恒星科技有限公司 | Filing system and implement method of distributed configured massive data |
| CN103365972A (en) * | 2013-06-27 | 2013-10-23 | 北京中科金财科技股份有限公司 | Intelligent processing system for mass data |
| US20160378844A1 (en) * | 2015-06-25 | 2016-12-29 | International Business Machines Corporation | Data synchronization using redundancy detection |
| CN105049504A (en) * | 2015-07-09 | 2015-11-11 | 国云科技股份有限公司 | Big data transit transmission synchronization and storage method |
| CN106534249A (en) * | 2016-09-21 | 2017-03-22 | 苏州市广播电视总台 | File transmission system based on file straight-through technology |
| CN108540511A (en) * | 2017-03-03 | 2018-09-14 | 全球能源互联网研究院 | A kind of method of data synchronization towards Hadoop clusters |
| CN107391936A (en) * | 2017-07-25 | 2017-11-24 | 南京慧目信息技术有限公司 | A kind of ophthalmology image based on distributed website integrates and dissemination method |
| CN108845865A (en) * | 2018-06-28 | 2018-11-20 | 郑州云海信息技术有限公司 | A kind of monitoring service dispositions method, system and storage medium |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112671594A (en) * | 2021-02-21 | 2021-04-16 | 新疆医科大学第一附属医院 | Medical image data cloud storage filing monitoring system |
| CN115174447A (en) * | 2022-06-27 | 2022-10-11 | 京东科技信息技术有限公司 | Network communication method, device, system, equipment and storage medium |
| CN115174447B (en) * | 2022-06-27 | 2023-09-29 | 京东科技信息技术有限公司 | Network communication method, device, system, equipment and storage medium |
| CN118210757A (en) * | 2024-05-20 | 2024-06-18 | 杭州政云数据技术有限公司 | Credential processing method, apparatus, device and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107861686B (en) | File storage method, server and computer readable storage medium | |
| WO2019137320A1 (en) | Resource scheduling method, apparatus, device and system | |
| US8108623B2 (en) | Poll based cache event notifications in a distributed cache | |
| JP2023532947A (en) | Data transfer method, proxy server, storage medium and electronic device | |
| CN113168404B (en) | System and method for replicating data in a distributed database system | |
| CN101751415B (en) | Metadata service system, metadata synchronized method and writing server updating method | |
| CN103905537A (en) | System for managing industry real-time data storage in distributed environment | |
| CN107623703B (en) | Synchronization method, device and system for Global Transaction Identifier (GTID) | |
| JP5686034B2 (en) | Cluster system, synchronization control method, server device, and synchronization control program | |
| CN106933550B (en) | Global information obtaining, processing and updating method, device and system | |
| CN107368369B (en) | Distributed container management method and system | |
| CN107818111B (en) | Method for caching file data, server and terminal | |
| CN112052230B (en) | Multi-machine room data synchronization method, computing device and storage medium | |
| CN103870570A (en) | HBase (Hadoop database) data usability and durability method based on remote log backup | |
| US8156177B2 (en) | Fail-safe system for managing of client-server communication | |
| CN109639773B (en) | Dynamically constructed distributed data cluster control system and method thereof | |
| CN105493474A (en) | System and method for supporting partition level journaling for synchronizing data in a distributed data grid | |
| CN111382132A (en) | Medical image data cloud storage system | |
| CN111225003B (en) | NFS node configuration method and device | |
| CN113094430B (en) | Data processing method, device, equipment and storage medium | |
| CN111190547A (en) | Distributed container mirror image storage and distribution system and method | |
| CN111966482B (en) | Edge computing system | |
| US9298765B2 (en) | Apparatus and method for handling partially inconsistent states among members of a cluster in an erratic storage network | |
| CN107295030B (en) | Data writing method and device, data processing method, device and system | |
| US10545667B1 (en) | Dynamic data partitioning for stateless request routing |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information |
Address after: 201807 Shanghai City, north of the city of Jiading District Road No. 2258 Applicant after: Shanghai Lianying Medical Technology Co., Ltd Address before: 201807 Shanghai City, north of the city of Jiading District Road No. 2258 Applicant before: SHANGHAI UNITED IMAGING HEALTHCARE Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |