CN113111130A - Distributed medical image data storage method based on cloud platform - Google Patents

Distributed medical image data storage method based on cloud platform Download PDF

Info

Publication number
CN113111130A
CN113111130A CN202110484228.2A CN202110484228A CN113111130A CN 113111130 A CN113111130 A CN 113111130A CN 202110484228 A CN202110484228 A CN 202110484228A CN 113111130 A CN113111130 A CN 113111130A
Authority
CN
China
Prior art keywords
file
medical image
storage
cloud
fastdfs
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110484228.2A
Other languages
Chinese (zh)
Inventor
李晖
冯刚
张大斌
施若
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Guizhou Lianke Weixin Technology Co ltd
Guizhou University
Original Assignee
Guizhou Lianke Weixin Technology Co ltd
Guizhou University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guizhou Lianke Weixin Technology Co ltd, Guizhou University filed Critical Guizhou Lianke Weixin Technology Co ltd
Priority to CN202110484228.2A priority Critical patent/CN113111130A/en
Publication of CN113111130A publication Critical patent/CN113111130A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/24Querying
    • G06F16/245Query processing
    • G06F16/2455Query execution
    • G06F16/24552Database cache management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

The invention relates to a distributed medical image data storage method based on a cloud platform, which comprises the following steps: designing a cloud storage architecture and generating a local-cloud two-stage storage mode; constructing a distributed medical image storage framework according to a local-cloud two-stage storage mode; constructing a ProxmoxVE virtual resource environment, and establishing Ngi nx on a cloud fastDFS according to the access characteristics of massive medical image files; and optimizing the cloud FastDFS storage, and generating the medical image data and the metadata for storage. According to the invention, a large-scale distributed medical image storage architecture is provided through a local-cloud two-stage storage mode, the medical image cloud storage service is realized aiming at the special format DICOM and application scene of the medical image, the problem of storage node synchronization delay in a cluster is solved, and the cache hit rate and the medical image access rate are improved.

Description

Distributed medical image data storage method based on cloud platform
Technical Field
The invention relates to the technical field of big data, in particular to a distributed medical image data storage method based on a cloud platform.
Background
With the popularization of information technology and internet technology, people have stepped into the big data age, which means that the volume and the dimensionality of data are large, the data form is rich, and the variety is wide, such as text, images, sound, numbers and the like.
When the information technology is introduced into the medical industry, the informatization and automation degree of the medical industry is continuously improved, and the medical data shows the increase of TB and even PB level. According to the existing data statistical analysis, in 2020, the medical data is predicted to be increased sharply to 35ZB (1ZB is 230TB), which is equivalent to 44 times of the data volume in 2009, and the number of national medical and health institutions reaches 98.7 ten thousand by the time when the data is released by the authority of the committee of china defense and 5 months in 2015, wherein: 2.6 thousands of hospitals, 92.2 thousands of primary medical and health institutions, 3.5 thousands of professional public and health institutions and 0.3 thousands of other institutions, and how to reasonably manage and store the massive medical data and the complicated data types brings huge pressure to the medical industry.
At present, medical image data accounts for over 90 percent and is an important component of medical data, the medical image data has the characteristics of special file format (international unified DICOM format), most of the medical image data are small files (between 2KB and 1 MB), large data volume, high growth speed, long storage time and the like, the medical image technology is developed rapidly in more than ten years, new technology and new equipment are continuously emerged, and medical image information is digitalized and digitalized to present abundant and diverse medical large data with large storage amount.
According to research, after daily image compression, hospital radiology department has more than 40 GB, about 10TB in 1 year, and medical images generally require to be stored for 15 years, and general domestic hospitals all adopt an "online-nearline-offline" three-level storage mode, a storage mode of a disk array and a tape drive (or an optical disk library) with commercial centralized storage as a core, and adopt three storage architectures of NAS, DAS and SAN, which belong to a centralized file system of a private network.
The appearance and the rise of the cloud computing technology provide a new effective way for processing massive medical data, have the remarkable advantages of resource integration, high availability, high performance, easiness in expansion and the like, and provide a new method for data storage, retrieval, processing and analysis, so that the method is very suitable for long-term storage and quick and effective access of medical image data.
Disclosure of Invention
In order to overcome the technical defects in the prior art, the invention provides a distributed medical image data storage method based on a cloud platform, which can effectively solve the problems in the background art.
In order to solve the technical problems, the technical scheme provided by the invention is as follows:
the embodiment of the invention discloses a distributed medical image data storage method based on a cloud platform, which is characterized by comprising the following steps: the method comprises the following steps:
designing a cloud storage architecture and generating a local-cloud two-stage storage mode;
constructing a distributed medical image storage framework according to a local-cloud two-stage storage mode;
constructing a ProxmoxVE virtual resource environment, and establishing Nginx on a cloud FastDFS according to the access characteristics of massive medical image files;
and optimizing the cloud FastDFS storage, and generating the medical image data and the metadata for storage.
In any of the above solutions, preferably, the cloud storage architecture includes an application layer, a storage layer and a platform layer, where the application layer includes clients of the HIS system and the PACS system; the storage layer is in a two-stage storage mode of a local end and a cloud end, the local end comprises an HIS server and a PACS server, and the cloud end is constructed for a FastDFS large-scale distributed cluster; the platform layer is a virtual platform constructed above the infrastructure and realized by a virtualization technology.
In any of the above schemes, preferably, the memory-based key-value database system Redis built on Orthanc as a cache system, and a server of Orthanc is used as a server for locally storing medical images, and is used for storing recent files and periodically transmitting data to the cloud FastDFS according to a time threshold determined by the system.
In any of the above scenarios, it is preferred that GNU Health comprises 12 modules.
In any of the above schemes, preferably, the FastDFS cluster is composed of 9 nodes, and a mainstream web Server Nignx is built on each node, wherein one Tracker Server is a coordinating node of other 8 Storage servers, and every two Storage nodes are taken as a group, which are 4 groups in total, and are backed up with each other in the group.
In any of the above schemes, preferably, when there is a request for downloading, the FastDFS downloading flow is as follows:
firstly, analyzing a path to obtain a group number and a file ID;
acquiring metadata information according to the file ID, comprising: source storage ip, file path, name, size;
judging whether the file exists or not by calling trunk _ file _ stat _ ex1, and outputting the file if the file exists;
when the file does not exist, a validity check is performed, checking item a: the source storage is that the difference between the local time or the current time and the file creation time exceeds a threshold value, and an error is reported; examination item B: if the scene is the scene after redirect, the same error is reported.
In any of the above schemes, preferably, by virtualizing the Docker container, an application runs in each container, different containers are isolated from each other, and a communication mechanism may be established between the containers.
In any of the above schemes, preferably, the Redis database is designed as 5 dictionaries, the dictionary (0) stores meta information of original files, the dictionary (1) stores data of the serialized images, the dictionary (2) stores a prediction set list, the dictionary (3) stores file information corresponding to patients, and the dictionary (4) stores an LRU list for cache replacement.
In any of the above schemes, preferably, the downloading process of the medical image file is as follows:
a user sends a file downloading request to a system;
traversing the cache data index by the system according to the ID, judging whether the buffer zone hits the file or not according to the file ID, if so, reading the Value of the HashMap from the buffer zone, deserializing and returning to the user, adding 1 to the corresponding data access frequency, and if not, executing the next step;
traversing the prediction set, if the file ID is in the prediction set and the cache is not full, sending a file request to a tracker server of a management node of the FastDFS, returning the file to a user after obtaining the file, serializing the file and storing the file in Redis, adding 1 to the corresponding data access frequency, if the cache is full, deleting N data which are accessed least in the data, and if the file ID is not in the prediction set, entering the next step;
and sending a file downloading request to a tracker server of the FastDFS. After the file is obtained, returning the file to the user, and adding 1 to the corresponding data access times;
inquiring whether the updating time of the prediction set is reached, and if the updating time of the prediction set is reached, regenerating a new prediction set through an access log according to whether the updating time is within m years and the access times;
and closing the connection and finishing the file downloading.
In any of the above schemes, preferably, the uploading process of the medical image file is as follows:
a user sends a file uploading request to a system;
the system traverses the cache data index according to the ID, if the repeated ID exists, the False is returned, otherwise, the next step is carried out;
traversing the metadata index according to the ID, if the repeated ID exists, returning to False, otherwise, entering the next step;
sending a file uploading request to a tracker server of a management node of the FastDFS, returning false by the server, returning false to a user by the system, otherwise, storing the metadata of the data into the metadata, and returning true to the user;
and closing the connection and finishing the file uploading.
Compared with the prior art, the invention has the beneficial effects that:
according to the invention, through a local-cloud two-stage storage mode, an open source HIS system GNU Health, an open source Web PACS system Orthanc and an open source distributed file system FastDFS are integrated, a large-scale distributed medical image storage framework is provided, the medical image cloud storage service is realized aiming at a special format DICOM and an application scene of medical images, and aiming at the access characteristic of massive medical image files, the problem of storage node synchronization delay in a cluster is solved by building Nginx on the FastDFS;
aiming at cloud fast DFS storage in a two-stage storage mode, medical image storage optimization based on Redis is provided, and meanwhile, by means of a prediction set cache strategy of Redis, cache hit rate and access rate of medical images are further improved.
Drawings
The drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification.
FIG. 1 is a diagram of a medical image storage architecture according to the present invention;
FIG. 2 is a diagram of a medical image storage architecture according to the present invention;
FIG. 3 is a schematic diagram of the interface-HIS main function module of the GNU Health system of the present invention;
FIG. 4 is a diagram of the Orhtanc system interface-video information and image presentation of the present invention;
FIG. 5 is a schematic diagram of a FastDFS cluster of the present invention;
FIG. 6 is a flow chart of Fastfs download according to the present invention
FIG. 7 is a schematic diagram of the GNU Health system interface-image management module according to the present invention;
FIG. 8 is a cloud storage architecture diagram of medical images according to the present invention;
FIG. 9 is a Docker-based multi-tenant architecture diagram of the present invention;
FIG. 10 is a flowchart of the medical image file download of the present invention;
fig. 11 is a flowchart of uploading medical image files according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly on the other element or be indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or be indirectly connected to the other element.
In the description of the present invention, it is to be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on the orientations or positional relationships illustrated in the drawings, and are used merely for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention.
Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless specifically defined otherwise.
For better understanding of the above technical solutions, the technical solutions of the present invention will be described in detail below with reference to the drawings and the detailed description of the present invention.
The invention provides a distributed medical image data storage method based on a cloud platform, which comprises the following steps:
step 1, designing a cloud storage architecture and generating a local-cloud two-stage storage mode.
Specifically, as shown in fig. 1, the storage architecture includes an application layer, a storage layer, and a platform layer, where the application layer includes clients of an HIS system and a PACS system, and is used to provide functions such as an operation interface, information management, and image viewing for a user; the storage layer is in a two-stage storage mode of a local end and a cloud end, the local end comprises an HIS server and a PACS server and is used for storing and managing structural information data of a hospital and recent image data, the cloud end is constructed by a FastDFS large-scale distributed cluster and is used for permanently storing long-term files, and data of the storage layer is uploaded by a user and can be periodically uploaded to the cloud end from the local end; the platform layer is a virtual platform constructed on the infrastructure and realized through a virtualization technology, and in actual operation, cloud service is conveniently provided through reasonably and effectively utilizing server resources.
Furthermore, a local-cloud two-stage medical data storage mode is generated through an HIS system GNU Health, a PACS system ortho and a lightweight distributed file system FastDFS, and is built on a cloud platform, so that permanent storage and real-time online of data are realized.
Furthermore, a key-value database system Redis based on an internal memory is built on Orthanc as a cache system, a server side of Orthanc is used as a server for locally storing medical images and is used for storing recent files and periodically transmitting data to a cloud FastDFS according to a time threshold value determined by the system, only recent data with a certain size is stored locally, the cloud side is used for storing all long-term permanent data, and in actual operation, the FastDFS can achieve better elastic expansion and fully meet the storage requirement of the long-term files.
And 2, constructing a distributed medical image storage framework according to a local-cloud two-stage storage mode.
Specifically, the GNU Health comprises 12 modules, each of which can add a new function to the system, and all the modules are completely installed to support various service flows and related functions in the HIS system.
As shown in fig. 2 and fig. 3, firstly, the Modules Perform GNU Health to select the required Modules, then the Modules enter a form Pending introduction/Upgrade to install or Upgrade the selected Modules, and the Modules can also be selectively installed according to the hospital scale, the use habit and the like of the user.
For example, the GNU Health system is used for acquiring information of doctors, Patients, institutions, electronic medical records and the like, a request for radiography examination is generated through the GNU Health, and image examination results of all Patients can be uploaded, examined and downloaded through the GNU Health.
The plug-in the form of a dynamic library developed by the SDK can be imported into the ortho service, responds to the HTTP request of the browser by registering a callback function, and the callback function can access the ortho database to extract the information of the target DICOM file.
As shown in FIG. 4, the information of the patient is viewed through the WEB of ortho, not only can the image display of the medical image be realized, but also the information in all DICOM Tags can be listed in detail, including all detailed information and data in the image, so that the user can know the content of the viewed image more directly and clearly.
In a more specific embodiment, as shown in fig. 5, the FastDFS cluster is composed of 9 nodes, and a mainstream web Server Nignx is built on each node, wherein only one Tracker Server is selected as a coordinating node of other 8 Storage servers, and every two Storage nodes are taken as a group, which are 4 groups in total, and are backed up with each other in the group.
When data storage operation is carried out, the starting state, the storage condition and the like in each group are reported to the Tracker Server node, data are transmitted according to the capacity condition of each group, the data are transmitted to a certain source data Server in the group, and after the data are uploaded, the group synchronization is carried out.
The Nginx is a high-performance Web and anti-proxy server, and compared with Apache, the Nginx uses fewer resources, supports more concurrent connections and embodies higher efficiency.
As an anti-proxy Server, the problem of synchronous delay of Storage is solved, Nginx and a FastDFS extension module fdfs-Nginx-module are deployed on each Storage Server, the Nginx module provides http downloading service for files in Storage nodes, and a redirect or proxy action is initiated to a source Storage Server only when the Storage nodes cannot find the files, so that the problem caused by copying delay between the Storage nodes can be avoided.
The download process after building a Nginx is shown in figure 6,
when there is a request for downloading, the FastDFS download flow is as follows:
(1) firstly, analyzing a path to obtain a group number and a file ID;
(2) acquiring metadata information according to the file ID, comprising: source storage ip, file path, name, size, etc.;
(3) judging whether the file exists or not by calling trunk _ file _ stat _ ex1, and outputting the file if the file exists;
(4) when the file does not exist, a validity check is performed, checking item a: the source storage is that the difference between the local time or the current time and the file creation time exceeds a threshold value, and an error is reported; examination item B: if the scene is the scene after redirect, the same error is reported;
(5) a redirection mode: the configuration item response _ mode is redirect, and the server returns a response code 302, url is as follows:
http:// { source storage address }: { current port } { current url } { parameter "redirect ═ 1" } (mark redirected)
(6) The proxy mode comprises the following steps: the configuration item response _ mode is proxy, the mode works as reverse proxy, and only the source storage address is used as the host of proxy, and the rest remains unchanged.
As shown in fig. 7, the GNU Health image management module Imaging includes three parts of image examination request, processing of image examination request, and viewing of image examination result, and the image examination result module can select a module interface through which the patient can be directly connected to orthoanc to view image information.
And 3, constructing a ProxmoxVE virtual resource environment, and building Nginx on the cloud FastDFS according to the access characteristics of the massive medical image files.
Specifically, as two Virtual Machine technologies for underlying supporting Promox VE, a KVM (Kernel-based Virtual Machine) is x86 hardware based on virtualization extension (Intel VT or AMD-V), is a full virtualization solution completely native to Linux, and a partial para-virtualization support is mainly used for Linux and Windows client systems in the form of a para-Virtual network driver, the KVM is designed to support a wide range of client operating systems through a loadable Kernel module, and LXC (Linux providers) is an operating system level virtualization environment, and can run a plurality of independent Linux systems to be controlled on a Linux host, and LXC is a Kernel and is a user space interface control function, so that a Linux user can easily create and manage a system or an application container and a strong API and simple tool. The user can choose according to his own needs, and if KVM is needed, the CPU must support hardware virtualization technology similar to Intel VT or AMD-V.
As shown in fig. 8, in the present invention, all the servers and storage clusters are built on the platform through virtual nodes implemented based on KVM technology in Proxmox VE, a user performs operations such as viewing, uploading, downloading, etc. of image information through Tryton and ortho-web clients, the client transmits a command to the server, and performs a related management operation of image data on the server or a cloud storage layer, if the command is long-term data, the command is obtained from a FastDFS distributed file system cluster in the cloud, otherwise, the command is obtained from a local storage layer.
Furthermore, all related functions of the HIS system and the PACS system and the cloud storage service composed of FastDFS clusters are provided through a SaaS (software as a service) mode, a multi-tenant architecture of Share Hardware is adopted, the cost of constructing, using and maintaining software applications by clients is reduced, in the mode, a user group with common requirements is called as a tenant, an end user rents software by taking the tenant as a unit, each tenant provides a virtual system only used for trial use, meanwhile, software services are provided for a plurality of tenants, and cost and benefit can be reduced and improved through manpower, equipment and other resources shared among the tenants.
By virtualizing the Docker container, a user may simply understand the Docker container as a kind of Sandbox (Sandbox). An application runs in each container, different containers are isolated from each other, and a communication mechanism can be established between the containers.
As shown in fig. 9, the application layer and the data layer can be isolated according to the requirements of the user, so as to implement multiple multi-tenant service providing manners:
(1) an application layer: a user can directly access the server of the cloud through the network client and the webpage in a remote mode, so that the same server can be shared; the server side can also be separately allocated to the tenant, and the address space is isolated.
(2) And (3) a data layer: because the distributed file system cluster is used, the data of the tenant can be placed in different groups of the same cluster, the child nodes in the groups are mutually backed up, and no association or image exists between the groups; isolation and security can be further improved, and a cluster and an independent address space are independently provided for a single tenant.
And 4, optimizing the cloud fast DFS storage, and generating the medical image data and the metadata storage.
Specifically, when a high concurrent access request to an image system in a short time is met in a hospital, in order to accelerate the data processing speed and avoid frequent access to a database by a server side, a Redis cache layer is introduced to optimize the system, and medical image data of a hot spot is serialized and then placed into Redis with a higher processing speed, so that a user can quickly obtain the hot spot data without accessing the database with higher I/O cost.
In a more specific embodiment, since the metadata of the FastDFS is managed by the Tracker node, when a CRUD (add-drop-and-delete-check) operation occurs in the system, the system writes the operation into the binlog file, and records the file meta-information.
After the file is successfully uploaded, the FastDFS renames the file according to self-defined rules and returns the renamed file to a path consisting of a group name, a disk, a directory and a new file name to a user, the original file name of the file is lost, the path is very troublesome to manage, and great damage is caused to user experience.
Secondly, metadata management of Tracker is stored in a memory, an externally operable API is not provided, and great difficulty is caused to data management.
Aiming at the problem of difficult metadata management, the metadata storage structure of the system is generated based on the rich data structure of Redis.
Redis is a dictionary-structured storage system that does not have the notion of a table, but provides multiple dictionaries for storing data, and a user can specify which particular dictionary data is stored in, similar to the design in which multiple databases can be created in our well-known relational database instance. The dictionary can be understood as a separate database. The number of dictionaries can be set by setting a database parameter in the configuration file Redis. config, and the number of dictionaries is 16 in a default case.
Preferably, the Redis database is designed as 5 dictionaries, wherein the dictionary 0 stores meta information of original files, the dictionary 1 stores data of the serialized images, the dictionary 2 stores a prediction set list, the dictionary 3 stores file information corresponding to patients, and the dictionary 4 stores an LRU list for cache replacement.
Furthermore, as data is completely stored in the memory, the possibility of power failure loss can occur, and the system can be persisted in a snapshot manner, the snapshot manner is that data is stored once in a certain operation within a specified time, for example, Save 9001 means that a snapshot is performed when at least one key is changed within 15 minutes (900 seconds), Redis defaults to store snapshot data in dump.
Further, the medical image file access process includes an image file downloading process and an image file uploading process, where the image file downloading process is shown in fig. 10:
(1) the user sends a file download request to the system.
(2) The system traverses the cache data index according to the ID, judges whether the buffer zone hits the file according to the file ID, if so, reads the Value of the HashMap from the buffer zone, returns the result to the user after deserialization, and adds 1 to the corresponding data access frequency. If there is no hit, the next step is performed.
(3) And traversing the prediction set, and if the file ID is in the prediction set and the cache is not full, sending a file request to a tracker server of a management node of the FastDFS. And returning the obtained file to the user, serializing the file and storing the serialized file in Redis, and adding 1 to the corresponding data access times. And if the cache is full, deleting the N data which are accessed least in the data. If the file ID is not in the prediction set, the next step is performed.
(4) And sending a file downloading request to a tracker server of the FastDFS. And returning the obtained file to the user, and adding 1 to the corresponding data access times.
(5) And inquiring whether the updating time of the prediction set is reached, and if the updating time is reached, regenerating a new prediction set through the access log according to whether the updating time is within m years and the access times.
(6) And closing the connection and finishing the file downloading.
The image file uploading process is shown in fig. 11:
(1) and the user sends a file uploading request to the system.
(2) And traversing the cache data index by the system according to the ID, if the repeated ID exists, returning False, and if not, entering the next step.
(3) And traversing the metadata index according to the ID, if the repeated ID exists, returning False, and if not, entering the next step.
(3) And sending a file uploading request to a tracker server of a management node of the FastDFS, returning a false by the server, returning the false to the user by the system, and otherwise, storing the metadata of the data into the metadata and returning true to the user.
(4) And closing the connection and finishing the file uploading.
Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that various changes, modifications and substitutions can be made without departing from the spirit and scope of the invention as defined by the appended claims. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (10)

1. A distributed medical image data storage method based on a cloud platform is characterized by comprising the following steps: the method comprises the following steps:
designing a cloud storage architecture and generating a local-cloud two-stage storage mode;
constructing a distributed medical image storage framework according to a local-cloud two-stage storage mode;
constructing a ProxmoxVE virtual resource environment, and establishing Nginx on a cloud FastDFS according to the access characteristics of massive medical image files;
and optimizing the cloud FastDFS storage, and generating the medical image data and the metadata for storage.
2. The cloud platform-based distributed medical image data storage method according to claim 1, wherein: the storage architecture comprises an application layer, a storage layer and a platform layer, wherein the application layer comprises clients of an HIS system and a PACS system; the storage layer is in a two-stage storage mode of a local end and a cloud end, the local end comprises an HIS server and a PACS server, and the cloud end is constructed for a FastDFS large-scale distributed cluster; the platform layer is a virtual platform constructed above the infrastructure and realized by a virtualization technology.
3. The cloud platform-based distributed medical image data storage method according to claim 2, wherein: the key-value database system Redis based on the memory is used as a cache system and built on Orthanc, and a server side of Orthanc is used as a server for locally storing medical images and is used for storing recent files and periodically transmitting data to the cloud FastDFS according to a time threshold value determined by the system.
4. The cloud platform-based distributed medical image data storage method according to claim 3, wherein: GNU Health contains 12 modules.
5. The cloud platform-based distributed medical image data storage method according to claim 4, wherein: the FastDFS cluster is composed of 9 nodes, a main stream web Server Nignx is set up on each node, one Tracker Server is a coordinating node of other 8 Storage servers, every two Storage nodes are taken as a group, 4 groups are provided, and mutual backup is carried out in the group.
6. The cloud platform-based distributed medical image data storage method according to claim 5, wherein: when there is a request for downloading, the FastDFS download flow is as follows:
firstly, analyzing a path to obtain a group number and a file ID;
acquiring metadata information according to the file ID, comprising: source storage ip, file path, name, size;
judging whether the file exists or not by calling trunk _ file _ stat _ ex1, and outputting the file if the file exists;
when the file does not exist, a validity check is performed, checking item a: the source storage is that the difference between the local time or the current time and the file creation time exceeds a threshold value, and an error is reported; examination item B: if the scene is the scene after redirect, the same error is reported.
7. The cloud platform-based distributed medical image data storage method according to claim 6, wherein: by virtualizing the Docker containers, each container runs an application, different containers are isolated from each other, and a communication mechanism can be established between the containers.
8. The cloud platform-based distributed medical image data storage method according to claim 7, wherein: the Redis database is designed into 5 dictionaries, wherein a dictionary (0) stores meta information of original files, a dictionary (1) stores data of cache image serialization, a dictionary (2) stores a prediction set list, a dictionary (3) stores file information corresponding to patients, and a dictionary (4) stores an LRU list for cache replacement.
9. The cloud platform-based distributed medical image data storage method according to claim 8, wherein: the downloading process of the medical image file is as follows:
a user sends a file downloading request to a system;
traversing the cache data index by the system according to the ID, judging whether the buffer zone hits the file or not according to the file ID, if so, reading the Value of the HashMap from the buffer zone, deserializing and returning to the user, adding 1 to the corresponding data access frequency, and if not, executing the next step;
traversing the prediction set, if the file ID is in the prediction set and the cache is not full, sending a file request to a tracker server of a management node of the FastDFS, returning the file to a user after obtaining the file, serializing the file and storing the file in Redis, adding 1 to the corresponding data access frequency, if the cache is full, deleting N data which are accessed least in the data, and if the file ID is not in the prediction set, entering the next step;
and sending a file downloading request to a tracker server of the FastDFS. After the file is obtained, returning the file to the user, and adding 1 to the corresponding data access times;
inquiring whether the updating time of the prediction set is reached, and if the updating time of the prediction set is reached, regenerating a new prediction set through an access log according to whether the updating time is within m years and the access times;
and closing the connection and finishing the file downloading.
10. The cloud platform-based distributed medical image data storage method according to claim 9, wherein: the uploading process of the medical image file is as follows:
a user sends a file uploading request to a system;
the system traverses the cache data index according to the ID, if the repeated ID exists, the False is returned, otherwise, the next step is carried out;
traversing the metadata index according to the ID, if the repeated ID exists, returning to False, otherwise, entering the next step;
sending a file uploading request to a tracker server of a management node of the FastDFS, returning false by the server, returning false to a user by the system, otherwise, storing the metadata of the data into the metadata, and returning true to the user;
and closing the connection and finishing the file uploading.
CN202110484228.2A 2021-04-30 2021-04-30 Distributed medical image data storage method based on cloud platform Pending CN113111130A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110484228.2A CN113111130A (en) 2021-04-30 2021-04-30 Distributed medical image data storage method based on cloud platform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110484228.2A CN113111130A (en) 2021-04-30 2021-04-30 Distributed medical image data storage method based on cloud platform

Publications (1)

Publication Number Publication Date
CN113111130A true CN113111130A (en) 2021-07-13

Family

ID=76720795

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110484228.2A Pending CN113111130A (en) 2021-04-30 2021-04-30 Distributed medical image data storage method based on cloud platform

Country Status (1)

Country Link
CN (1) CN113111130A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113674837A (en) * 2021-07-20 2021-11-19 中电通商数字技术(上海)有限公司 High-performance, high-fault-tolerance and extensible medical data acquisition method and system
CN113793697A (en) * 2021-09-07 2021-12-14 山东健康医疗大数据有限公司 Network deployment method for realizing doctor-patient remote communication
CN114697314A (en) * 2022-03-25 2022-07-01 宁波全网云医疗科技股份有限公司 Medical image downloading system and method
CN115098538A (en) * 2022-08-25 2022-09-23 北京永洪商智科技有限公司 Database query optimization method and system
CN116741346A (en) * 2023-05-11 2023-09-12 无锡蓝影医疗科技有限公司 Medical intelligent cloud image management system and method based on cloud computing
CN116881049A (en) * 2023-07-21 2023-10-13 无锡隆云数字技术有限公司 Distributed data backup method based on terminal and cloud host under cloud framework

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108806773A (en) * 2018-05-21 2018-11-13 上海熙业信息科技有限公司 Medical image cloud storage platform designing method

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108806773A (en) * 2018-05-21 2018-11-13 上海熙业信息科技有限公司 Medical image cloud storage platform designing method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘秋彤: "基于云平台的分布式医学影像数据存储技术的研究与应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113674837A (en) * 2021-07-20 2021-11-19 中电通商数字技术(上海)有限公司 High-performance, high-fault-tolerance and extensible medical data acquisition method and system
CN113793697A (en) * 2021-09-07 2021-12-14 山东健康医疗大数据有限公司 Network deployment method for realizing doctor-patient remote communication
CN114697314A (en) * 2022-03-25 2022-07-01 宁波全网云医疗科技股份有限公司 Medical image downloading system and method
CN115098538A (en) * 2022-08-25 2022-09-23 北京永洪商智科技有限公司 Database query optimization method and system
CN115098538B (en) * 2022-08-25 2022-11-25 北京永洪商智科技有限公司 Database query optimization method and system
CN116741346A (en) * 2023-05-11 2023-09-12 无锡蓝影医疗科技有限公司 Medical intelligent cloud image management system and method based on cloud computing
CN116741346B (en) * 2023-05-11 2024-03-26 无锡蓝影医疗科技有限公司 Medical intelligent cloud image management system and method based on cloud computing
CN116881049A (en) * 2023-07-21 2023-10-13 无锡隆云数字技术有限公司 Distributed data backup method based on terminal and cloud host under cloud framework
CN116881049B (en) * 2023-07-21 2024-01-30 无锡隆云数字技术有限公司 Distributed data backup method based on terminal and cloud host under cloud framework

Similar Documents

Publication Publication Date Title
CN113111130A (en) Distributed medical image data storage method based on cloud platform
JP7410181B2 (en) Hybrid indexing methods, systems, and programs
US10013185B2 (en) Mapping systems and methods of an accelerated application-oriented middleware layer
US8458239B2 (en) Directory traversal in a scalable multi-node file system cache for a remote cluster file system
US20170147243A1 (en) Profile-guided data preloading for virtualized resources
US9348842B2 (en) Virtualized data storage system optimizations
US8489676B1 (en) Technique for implementing seamless shortcuts in sharepoint
US11652883B2 (en) Accessing a scale-out block interface in a cloud-based distributed computing environment
US20210303537A1 (en) Log record identification using aggregated log indexes
US10922280B2 (en) Policy-based data deduplication
CN114391141A (en) Automatic derivation of sharded key values and transparent multi-sharded transaction and query support
CN107818111A (en) A kind of method, server and the terminal of cache file data
CN115827907A (en) Cross-cloud multi-source data cube discovery and integration method based on distributed memory
Won et al. Moving metadata from ad hoc files to database tables for robust, highly available, and scalable HDFS
CN105022779A (en) Method for realizing HDFS file access by utilizing Filesystem API
US20230169079A1 (en) Scaling query processing resources for efficient utilization and performance
US20230169048A1 (en) Detecting idle periods at network endpoints for management actions at processing clusters for managed databases
US20230081324A1 (en) Shared cache for multiple index services in nonrelational databases
US8356016B1 (en) Forwarding filesystem-level information to a storage management system
Iwazume et al. Big data in memory: Benchimarking in memory database using the distributed key-value store for machine to machine communication
US10685046B2 (en) Data processing system and data processing method
US11842085B1 (en) Up-sized cluster performance modeling for a tiered data processing service
Shrivastava Hadoop-CC (Collaborative Caching) in Real Time HDFS Thesis
US20240004867A1 (en) Optimization of application of transactional information for a hybrid transactional and analytical processing architecture
US20240004860A1 (en) Handshake protocol for efficient exchange of transactional information for a hybrid transactional and analytical processing architecture

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210713