CN110349254B - C/S architecture-oriented adaptive medical image three-dimensional reconstruction method - Google Patents

C/S architecture-oriented adaptive medical image three-dimensional reconstruction method Download PDF

Info

Publication number
CN110349254B
CN110349254B CN201910623642.XA CN201910623642A CN110349254B CN 110349254 B CN110349254 B CN 110349254B CN 201910623642 A CN201910623642 A CN 201910623642A CN 110349254 B CN110349254 B CN 110349254B
Authority
CN
China
Prior art keywords
layer
client
reconstruction
server
medical image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201910623642.XA
Other languages
Chinese (zh)
Other versions
CN110349254A (en
Inventor
栗伟
于鲲
冯朝路
周文萍
覃文军
赵大哲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Northeastern University China
Original Assignee
Northeastern University China
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Northeastern University China filed Critical Northeastern University China
Priority to CN201910623642.XA priority Critical patent/CN110349254B/en
Publication of CN110349254A publication Critical patent/CN110349254A/en
Application granted granted Critical
Publication of CN110349254B publication Critical patent/CN110349254B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • G06F18/243Classification techniques relating to the number of classes
    • G06F18/24323Tree-organised classifiers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/02Protocols based on web technology, e.g. hypertext transfer protocol [HTTP]

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention provides a C/S architecture-oriented adaptive medical image three-dimensional reconstruction method, and relates to the technical field of medical image three-dimensional reconstruction. The method provides a universal three-dimensional reconstruction method comprising four layers of structures at a client and a server through a universal three-dimensional reconstruction method based on a four-layer architecture, wherein the four layers of structures are a remote transmission protocol layer, a preprocessing volume data layer, a reconstruction scene algorithm layer and a rendering visual model layer from bottom to top, and then the reconstruction mode is switched in a self-adaptive mode according to different operation platforms, data volumes, network types and network speeds dynamically through a self-adaptive reconstruction mode selection method based on a decision tree. The invention can unify the current medical image three-dimensional reconstruction mechanisms of various types, and the medical image system realized based on the method can adapt to different operation platforms, different transmission networks and different medical image data volumes, and dynamically switches the medical image three-dimensional reconstruction mechanism according to different operation real-time conditions, thereby having strong flexibility.

Description

C/S architecture-oriented adaptive medical image three-dimensional reconstruction method
Technical Field
The invention relates to the technical field of three-dimensional reconstruction of medical images, in particular to a C/S (client/server) architecture-oriented adaptive three-dimensional reconstruction method of medical images.
Background
Medical image imaging equipment is continuously developed, the precision and the speed of image generation are higher and higher, and a large amount of data (200 MB-2 GB) is generated as a result of one-time imaging, and the data bring challenges to doctors and image storage equipment. The number of two-dimensional images read by doctors is multiplied, the workload is increased, and missed diagnosis is easy to cause. Therefore, the three-dimensional reconstruction technique gives the doctor a new observation dimension to observe the lesion or the tissue organ from different angles and different scales. The three-dimensional reconstruction not only intuitively shows the results to doctors, but also reduces the amount of work and foot for reading the film, and brings great convenience to later treatment, auxiliary operation planning, doctor-patient communication and the like.
The existing three-dimensional reconstruction functional modules are integrated in a PACS system or an image workstation, such as a heart software package, a lung function software package, and the like, and the three-dimensional reconstruction methods are operated based on local image data. With the rapid development of technologies such as cloud computing and internet of things, the remote-radiology technology (Tele-radiology) promotes the development of technologies such as remote consultation and remote film reading; meanwhile, with the increase of the speed of the mobile network and the improvement of the performance of the mobile equipment, mobile medical treatment is rapidly developed, doctors are not limited to a film reading room, and clinicians are not only assisted with diagnosis by local images. Therefore, the research on remote three-dimensional reconstruction technology is also receiving more and more attention.
Three-dimensional reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of a three-dimensional object, is the basis for processing, operating and analyzing the properties of the three-dimensional object in a computer environment, and is also a key technology for establishing virtual reality expressing an objective world in a computer. The three-dimensional reconstruction of medical images is to study a two-dimensional image fault sequence acquired by various medical imaging devices to construct a three-dimensional geometric model of tissues or organs, and to draw and display the three-dimensional geometric model on a computer screen. From the computer system division, the three-dimensional reconstruction method can be divided into three types, i.e., client reconstruction, server reconstruction and hybrid reconstruction, and as shown in fig. 1, three reconstruction mechanisms (reconstruction modes) A, B, C are shown: the mechanism A is to perform a three-dimensional reconstruction process at an external network client, interact a three-dimensional reconstruction model with a Web server of a hospital service network through a wide area network or a mobile network and the like, and simultaneously connect a background image database with the Web server to transmit medical image data; the mechanism B is used for performing a three-dimensional reconstruction process at a Web server side of a hospital service network, simultaneously connecting the Web server with a background image database to transmit medical image data, and interacting a three-dimensional reconstruction model with an external network client side through a wide area network or a mobile network and the like; and in the mechanism C, three-dimensional reconstruction processes are carried out at both ends of the Web server in the extranet client and the hospital service network, but the three-dimensional reconstruction effects of the extranet client and the Web server are different, the three-dimensional reconstruction carried out by the Web server is based on the medical image with the original resolution, and the three-dimensional reconstruction carried out by the client is based on the medical image with the reduced resolution.
The above methods have advantages and disadvantages, and have great differences in applicability under different operating environments and different application scenarios, as shown in table 1.
TABLE 1 comparison of advantages and disadvantages of the three existing reconstruction mechanisms
Figure BDA0002126331000000021
The three reconstruction mechanisms are analyzed respectively in the aspects of client resource requirements, network speed requirements, operation fluency, platform adaptability, data volume adaptability, client image quality, mobile universality, user experience and UI display capacity. The mechanism A has low requirement on network speed, good operation fluency, good image quality display effect of the client, strong user experience and system display capability and relatively poor other aspects. The mechanism B does not need too much client resources, has good cross-platform adaptability and good data volume adaptability, can be easily applied to mobile terminal equipment, and is relatively poor in other aspects. The mechanism C has better operation fluency, high client quality and good user experience, but has poorer applicability to data volume and general performance in other aspects. The existing methods have advantages and disadvantages, and a unified modeling method which can be used for the methods is lacked, and a reconstruction mechanism can be automatically switched according to an actual operation scene.
Currently, systems such as a PACS (picture archiving and communication system) and an image workstation adopt a client reconstruction mechanism (including a plug-in Web reconstruction mode), and a pure server drawing, a mixed drawing and a pure Web reconstruction mechanism are limited by the influence and the limitation of mobile terminal performance, UI (user interface) display capacity, network speed and the like, so that the application of products is still in a prototype research stage at present. However, with the development of technologies such as 5G and service architecture, the future on-line three-dimensional reconstruction technology will be widely applied based on the urgent needs of the existing actual clinical on-line film reading, remote diagnosis and the like. How to provide a cross-platform three-dimensional reconstruction method with wide applicability and a self-adaptive switching reconstruction mechanism according to the running environment and scene of a client is the current technical difficulty and challenge.
Disclosure of Invention
The technical problem to be solved by the invention is to provide a C/S architecture-oriented adaptive medical image three-dimensional reconstruction method aiming at the defects of the prior art, which can operate on various existing application platforms, has wider applicability, can meet the requirements of clinical online film reading, remote diagnosis and the like, and solves the defects of the traditional three-dimensional reconstruction mechanism
In order to solve the technical problems, the technical scheme adopted by the invention is as follows:
a C/S architecture-oriented adaptive medical image three-dimensional reconstruction method comprises a four-layer architecture universal three-dimensional reconstruction method and a decision tree-based adaptive reconstruction mode selection method;
in the universal three-dimensional reconstruction method of the four-layer architecture, a client and a server both comprise four-layer structures, and a remote transmission protocol layer R1, a preprocessing volume data layer R2, a reconstruction scene algorithm layer R3 and a rendering visual model layer R4 are respectively arranged from bottom to top; the remote transmission protocol layer is used for providing a three-dimensional reconstruction communication transmission protocol and content semantics and format definition of data/control messages; the preprocessing volume data layer is used for providing three-dimensional reconstruction volume data interpolation processing, and comprises a layer thickness, a pixel depth and a sampling resolution; the reconstruction scene algorithm layer is used for providing different three-dimensional reconstruction algorithms and reconstruction model output; the rendering visual model layer is used for providing rendering visualization and picture display of a three-dimensional reconstruction result; according to different application scenes and requirements, the user can flexibly jump from the bottom layer R1 to any layer R2, R3 or R4, layers at two ends of the customer service are simplified, and different application modes are formed, wherein the application modes comprise the following four modes:
(a) For a remote transmission protocol layer, both the client and the server can select HTTP, FTP, RMI and HTTPS protocols to carry out remote data transmission; for the preprocessing volume data layer, the client and the server both store medical images in a DICOM format; for a reconstructed scene algorithm layer and a rendered visual model layer, as the model is reconstructed at a client, only the client has a two-layer structure, the client performs three-dimensional reconstruction on a medical image by using MPR, VR, SSD, MIP, minIP and AvgIP algorithms, and displays the three-dimensional model on a screen of electronic equipment of the client by using a medical image workstation or ActiveX, applet and Flash plug-in programs;
(b) For a remote transmission protocol layer, both the client and the server can select HTTP, AJAX, HTTPS, FTP and RMI protocols or technologies to carry out remote data transmission; for the preprocessing volume data layer, the service end is a medical image in a DICOM format, and the client end is a model file; for a reconstructed scene algorithm layer, because the three-dimensional reconstruction process is carried out in a server, only the server has the structure, and the medical image is three-dimensionally reconstructed by using a Surface Rendering (Surface Rendering) algorithm in the layer; for the rendering visual model layer, because three-dimensional reconstruction display needs to be carried out on the client, only the client has the structure, and the layer carries out browsing display on the three-dimensional model data reconstructed by the server reconstruction scene algorithm layer on the screen of the electronic equipment of the client by using a player plug-in;
(c) For the remote transmission protocol layer, both the client and the server can select HTTP, webSocket, webWork protocol or technology to perform remote data transmission; for the preprocessing Volume data layer and the reconstruction scene algorithm layer, because the mode is only reconstructed in the server, only the server has two-layer structure, the server stores medical images in DICOM format, and then three-dimensional reconstruction is carried out on the medical images by using Volume Rendering (Volume Rendering) or Surface Rendering (Surface Rendering) algorithm; for the rendering visual model layer, the technologies used by the client and the server are different, the client uses HTML, canvas and WebGL technologies, and the server uses VTK and OpenGL technologies;
(d) For a remote transmission protocol layer, both the client and the server can select AJAX, webSocket, webWork and HTTP protocols or technologies to perform remote data transmission; for the preprocessing volume data layer, the server stores the DICOM medical image with the original resolution, and the client stores the DICOM medical image with the down-sampling resolution; for the reconstructed scene algorithm layer, the client uses an MPR, VR or Surface Rendering (Surface Rendering) algorithm, and the server uses a VR or Surface Rendering (Surface Rendering) algorithm; for rendering the visual model layer, the client side adopts a WebGL technology, and the server adopts a VTK technology and an OpenGL technology;
the self-adaptive reconstruction mode selection method based on the decision tree is based on a user-defined reconstruction process, and dynamically and automatically switches modes according to different operation platforms, data volumes, network types and network speeds; a decision tree-based method is adopted in the switching process, and attributes involved in the decision tree construction process comprise an operation platform, data volume, network type and network speed; the operation platform is divided into a PC local application, a PC Web application, a tablet local application, a tablet Web application, a smart phone local application and a smart phone Web application; the data volume is divided into two types, namely the number of image layers is less than 200 layers and the data volume is less than 100M, the number of image layers is more than 200 layers or the data volume is more than 100M; the network types comprise wide area networks, local area networks, wireless networks and mobile networks; the network speed is the actual transmission speed of the current network; the network speed is a dynamic parameter, the defining basis is two threshold parameters, an upper limit and a lower limit, different network types have different thresholds, and the thresholds are set according to requirements;
the decision making process starts from the operation platform; then judging the data quantity, and respectively measuring the data quantity from the number and the size; then checking the current network type; and finally, selecting a corresponding reconstruction model according to the current dynamic network speed.
The decision tree longitudinally comprises four layers of structures, and the operation platform, the data volume of the medical image, the network type and the current network speed are respectively arranged from top to bottom; viewed from the transverse direction, the first layer structure divides the client into 6 types according to the operation platform, wherein the 6 types are respectively a PC local App, a PC Web App, a Pad local App, a Pad Web App, a Phone local App and a Phone Web App; secondly, judging the size of the data volume of the medical image by a second layer structure, and dividing the medical image into two types according to the size of the data volume, wherein one type is that the number of image layers is less than 200 layers and the data volume is less than 100M, and the other type is that the number of image layers is more than 200 layers or the data volume is more than 100M, and each client side respectively analyzes the two types of conditions; next, the third layer structure checks the current network type, and for the PC, the network type comprises a wide area network, a local area network and a wireless network; for Pad and Phone, the network types include mobile and wireless networks; and the last layer of structure selects a corresponding reconstruction mode according to the current dynamic network speed.
Adopt the produced beneficial effect of above-mentioned technical scheme to lie in: the C/S architecture-oriented adaptive medical image three-dimensional reconstruction method provided by the invention can unify the current various different types of medical image three-dimensional reconstruction mechanisms, and the medical image system realized based on the method has the capability of adapting to different operation platforms, different transmission networks and different medical image data volumes, dynamically switches the medical image three-dimensional reconstruction mechanisms according to different operation real-time conditions, and has strong flexibility.
Drawings
FIG. 1 is a schematic diagram of a conventional three-dimensional reconstruction mechanism;
fig. 2 is a schematic diagram of a four-layer structure in a general three-dimensional reconstruction method based on a layered architecture according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of an application mode of a C/S-oriented architecture according to an embodiment of the present invention;
fig. 4 is a schematic diagram of different application forms of a general three-dimensional reconstruction method based on a layered architecture according to an embodiment of the present invention;
fig. 5 is a schematic structural diagram of a decision tree according to an embodiment of the present invention.
Detailed Description
The following detailed description of embodiments of the present invention is provided in connection with the accompanying drawings and examples. The following examples are intended to illustrate the invention but are not intended to limit the scope of the invention.
A C/S architecture-oriented adaptive medical image three-dimensional reconstruction method comprises a four-layer architecture universal three-dimensional reconstruction method and a decision tree-based adaptive reconstruction mode selection method.
In the universal three-dimensional reconstruction method with the four-layer architecture, a client and a server both have four-layer structures, as shown in fig. 2, a remote transmission protocol layer, a preprocessing volume data layer, a reconstruction scene algorithm layer and a rendering visual model layer are respectively arranged from bottom to top.
The remote transmission protocol layer is used for providing the content semantics and format definitions of the three-dimensional reconstruction communication transmission protocol and the data/control message, such as: webSocket, HTTP, socket, FTP, and the like.
The preprocessing volume data layer provides three-dimensional reconstruction volume data interpolation processing, including layer thickness, pixel depth and sampling resolution, such as: image interpolation, image segmentation, resampling, etc.
The reconstructed scene algorithm layer provides different three-dimensional reconstruction algorithms such as surface rendering, volume rendering and the like, and reconstructed model output such as: volume rendering, surface rendering, multi-planar reconstruction, maximum intensity projection, etc.
The rendering visual model layer provides rendering visualization of the three-dimensional reconstruction result, and comprises visual model data display, GPU data drawing or picture data display technical modes, such as: various plug-ins, webGL, image rendering Canvas, and the like.
As can be seen from fig. 2, according to different application scenarios and requirements, the lowest layer R1 can be directly jumped to any one of the layers R2, R3, or R4, thereby achieving flexibility of the method. The method of the present embodiment is oriented to a Client/Server (C/S) software application, so that the universal three-dimensional reconstruction method with a four-layer architecture of the present embodiment includes a four-layer architecture at both the Client and the Server, and the application mode is as shown in fig. 3.
According to different application scenarios and requirements, layers at two ends of a customer service can be simplified to form different application modes, according to the four-layer architecture described in this embodiment, a currently-occurring three-dimensional reconstruction method can be divided into four typical application modes as shown in fig. 4, which include the following four modes:
(a) For a remote transmission protocol layer, both the client and the server can select HTTP, FTP, RMI and HTTPS protocols to carry out remote data transmission; for the preprocessing volume data layer, the client and the server both store medical images in a DICOM format; for a reconstructed scene algorithm layer and a rendered visual model layer, as the model is reconstructed at a client, only the client has a two-layer structure, the client performs three-dimensional reconstruction on a medical image by using MPR, VR, SSD, MIP, minIP and AvgIP algorithms, and displays the three-dimensional model on a screen of electronic equipment of the client by using a medical image workstation or ActiveX, applet and Flash plug-in programs;
(b) For a remote transmission protocol layer, both the client and the server can select HTTP, AJAX, HTTPS, FTP and RMI protocols or technologies to carry out remote data transmission; for the preprocessing volume data layer, the service end is a medical image in a DICOM format, and the client end is a model file; for a reconstructed scene algorithm layer, because the three-dimensional reconstruction process is carried out in a server, only the server has the structure, and the medical image is three-dimensionally reconstructed by using a Surface Rendering (Surface Rendering) algorithm in the layer; for the rendering visual model layer, because three-dimensional reconstruction display needs to be carried out on the client, only the client has the structure, and the three-dimensional model data reconstructed by the server reconstruction scene algorithm layer is browsed and displayed on a screen of the electronic equipment of the client by using a player plug-in;
(c) For the remote transmission protocol layer, both the client and the server can select HTTP, webSocket, webWork protocol or technology to perform remote data transmission; for the preprocessing Volume data layer and the reconstruction scene algorithm layer, because the mode is only reconstructed in the server, only the server has two-layer structure, the server stores medical images in DICOM format, and then three-dimensional reconstruction is carried out on the medical images by using Volume Rendering (Volume Rendering) or Surface Rendering (Surface Rendering) algorithm; for the rendering visual model layer, the technologies used by the client and the server are different, the client uses HTML, canvas and WebGL technologies, and the server uses VTK and OpenGL technologies;
(d) For a remote transmission protocol layer, both the client and the server can select AJAX, webSocket, webWork and HTTP protocols or technologies to perform remote data transmission; for the preprocessing volume data layer, the server stores the DICOM medical image with the original resolution, and the client stores the DICOM medical image with the down-sampling resolution; for the reconstructed scene algorithm layer, the client uses an MPR, VR or Surface Rendering (Surface Rendering) algorithm, and the server uses a VR or Surface Rendering (Surface Rendering) algorithm; for rendering the visual model layer, the client adopts WebGL technology, and the server adopts VTK and OpenGL technology.
As can be seen from FIG. 4, A, B, C in FIG. 1 are exemplary reconstruction mechanisms in the form of application (a), (c) and (d) in FIG. 4. In the mode (b) in fig. 4, after the server three-dimensionally reconstructs the model, the model data is pushed to the client and browsed by using the player plug-in. The layered architecture of the embodiment can flexibly correspond to different application scenes.
Based on the above C/S hierarchical structure, this embodiment proposes a decision tree-based adaptive reconstruction mode automatic selection method for the three typical reconstruction processes A, B, C in fig. 1. In practical applications, a user may predefine a variety of reconstruction processes according to actual requirements and the above-mentioned hierarchical structure.
The self-adaptive reconstruction mode selection method based on the decision tree refers to a user-defined reconstruction process, and the mode is automatically switched dynamically according to the difference of a network, an operation platform, data volume, network type and network speed, so that the best use experience can be provided for a user, and the method is not fixed to a certain reconstruction process mode.
The switching process adopts a decision tree-based method, and the attributes involved in the decision tree construction process comprise an operation platform, data volume, network type and network speed. The operation platform is divided into a PC local application, a PC Web application, a tablet local application, a tablet Web application, a smart phone local application and a smart phone Web application; the data volume is divided into two types, namely the number of image layers is less than 200 layers and the data volume is less than 100M, the number of image layers is more than 200 layers or the data volume is more than 100M; network classes include wide area networks, local area networks, wireless networks, mobile networks (cellular networks); the network speed is the actual transmission speed of the current network. The construction structure of the decision tree is shown in fig. 5.
The decision making process starts from the operation platform; then judging the data quantity, and respectively measuring the data quantity from the number and the size; then checking the current network type; and finally, selecting a corresponding reconstruction model according to the current dynamic network speed. The network speed is a dynamic parameter and the defining basis is two threshold parameters, an upper limit and a lower limit, different network types have different thresholds, and the thresholds are set according to requirements. The present embodiment has four types of networks, 8 thresholds, and the set thresholds are T1=100K, T2=1M, T3=1M, T =10M, T5=1M, T6=5M, T7=500K, T =1M.
The decision tree longitudinally comprises four layers of structures, and the operation platform, the data volume of the medical image, the network type and the current network speed are respectively arranged from top to bottom; viewed from the transverse direction, the first layer structure divides the client into 6 types according to the operation platform, wherein the 6 types are respectively a PC local App, a PC Web App, a Pad local App, a Pad Web App, a Phone local App and a Phone Web App; secondly, judging the size of the data volume of the medical image by a second layer structure, and dividing the medical image into two types according to the size of the data volume, wherein one type is that the number of image layers is less than 200 layers and the data volume is less than 100M, and the other type is that the number of image layers is more than 200 layers or the data volume is more than 100M, and each client side respectively analyzes the two types of conditions; next, the third layer structure checks the current network type, and for the PC, the network type comprises a wide area network, a local area network and a wireless network; for Pad and Phone, the network types include mobile and wireless networks; and the last layer of structure selects the corresponding reconstruction mode in fig. 1 according to the current dynamic network speed.
The following is a detailed description of the two branches of the decision tree in fig. 5.
The first branch is that when the local App of the PC uses the wide area network to operate the medical image with the number of image layers less than 200 and the data volume less than 100M, different reconstruction processes are selected according to the network speed. T1 and T2 are the upper and lower limits of the wide area network speed respectively, and if the network speed is less than T1, a B reconstruction model is automatically selected; if the network speed is greater than T1 and less than T2, automatically selecting a C reconstruction model; and if the network speed is greater than T2, automatically selecting the A reconstruction model. Similarly, the local App of the PC uses the local area network and the wireless network to operate the medical image with the number of image layers less than 200 and the data volume less than 100M, so that the corresponding reconstruction model is automatically selected according to the network speed.
The second branch is that when the Pad Web App uses the mobile network to operate the medical image with the image layer number larger than 200 or the data volume larger than 100M, different reconstruction processes are selected according to the network speed. T7 and T8 are respectively the upper limit and the lower limit of the network speed of the mobile network, and if the network speed is less than T7, a B reconstruction model is automatically selected; if the network speed is greater than T7 and less than T8, automatically selecting a B reconstruction model; and if the network speed is higher than T8, automatically selecting the C reconstruction model. Similarly, the Pad Web App operates the medical image with the number of image layers greater than 200 or the data volume greater than 100M by using the wireless network, and thus automatically selects the corresponding reconstruction model according to the network speed.
The other branches also automatically select the corresponding reconstruction model according to the network speed.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions and scope of the present invention as defined in the appended claims.

Claims (2)

1. A C/S architecture-oriented adaptive medical image three-dimensional reconstruction method is characterized by comprising the following steps: the method comprises a universal three-dimensional reconstruction method of a four-layer framework and a self-adaptive reconstruction mode selection method based on a decision tree;
in the universal three-dimensional reconstruction method of the four-layer architecture, a client and a server both comprise four-layer structures, and a remote transmission protocol layer R1, a preprocessing volume data layer R2, a reconstruction scene algorithm layer R3 and a rendering visual model layer R4 are respectively arranged from bottom to top; the remote transmission protocol layer is used for providing a three-dimensional reconstruction communication transmission protocol and content semantics and format definition of data/control messages; the preprocessing volume data layer is used for providing three-dimensional reconstruction volume data interpolation processing, and comprises a layer thickness, a pixel depth and a sampling resolution; the reconstruction scene algorithm layer is used for providing different three-dimensional reconstruction algorithms and reconstruction model output; the rendering visual model layer is used for providing rendering visualization and picture display of a three-dimensional reconstruction result; according to different application scenes and requirements, the user can flexibly jump from the bottom layer R1 to any layer R2, R3 or R4, layers at two ends of the customer service are simplified, and different application modes are formed, wherein the application modes comprise the following four modes:
(a) For a remote transmission protocol layer, both the client and the server can select HTTP, FTP, RMI and HTTPS protocols to carry out remote data transmission; for the preprocessing volume data layer, the client and the server both store medical images in a DICOM format; for a reconstructed scene algorithm layer and a rendered visual model layer, as the model is reconstructed at a client, only the client has a two-layer structure, the client performs three-dimensional reconstruction on a medical image by using MPR, VR, SSD, MIP, minIP and AvgIP algorithms, and displays the three-dimensional model on a screen of electronic equipment of the client by using a medical image workstation or ActiveX, applet and Flash plug-in programs;
(b) For a remote transmission protocol layer, both the client and the server can select HTTP, AJAX, HTTPS, FTP and RMI protocols or technologies to carry out remote data transmission; for the preprocessing volume data layer, the service end is a medical image in a DICOM format, and the client end is a model file; for a reconstruction scene algorithm layer, because the three-dimensional reconstruction process is carried out in a server, only the server has the structure of the layer, and the medical image is subjected to three-dimensional reconstruction by using a surface rendering algorithm in the layer; for the rendering visual model layer, because three-dimensional reconstruction display needs to be carried out on the client, only the client has the structure, and the three-dimensional model data reconstructed by the server reconstruction scene algorithm layer is browsed and displayed on a screen of the electronic equipment of the client by using a player plug-in;
(c) For the remote transmission protocol layer, both the client and the server can select HTTP, webSocket, webWork protocol or technology to perform remote data transmission; for the preprocessing volume data layer and the reconstruction scene algorithm layer, because the mode is only reconstructed in the server, only the server has two-layer structure, the server stores medical images in DICOM format, and then three-dimensional reconstruction is carried out on the medical images by using a volume rendering or surface rendering algorithm; for the rendering visual model layer, the technologies used by the client and the server are different, the client uses HTML, canvas and WebGL technologies, and the server uses VTK and OpenGL technologies;
(d) For a remote transmission protocol layer, both the client and the server can select AJAX, webSocket, webWork and HTTP protocols or technologies to perform remote data transmission; for the preprocessing volume data layer, the server stores the DICOM medical image with the original resolution, and the client stores the DICOM medical image with the down-sampling resolution; for the reconstructed scene algorithm layer, the client adopts an MPR (maximum reduction processor), VR (virtual reality) or surface rendering algorithm, and the server adopts a VR or surface rendering algorithm; for rendering the visual model layer, the client side adopts a WebGL technology, and the server adopts a VTK technology and an OpenGL technology;
the self-adaptive reconstruction mode selection method based on the decision tree is based on a user-defined reconstruction process, and dynamically and automatically switches modes according to different operation platforms, data volumes, network types and network speeds; a decision tree-based method is adopted in the switching process, and attributes involved in the decision tree construction process comprise an operation platform, data volume, network type and network speed; the operation platform is divided into a PC local application, a PC Web application, a tablet local application, a tablet Web application, a smart phone local application and a smart phone Web application; the data volume is divided into two types, namely the number of image layers is less than 200 layers and the data volume is less than 100M, the number of image layers is more than 200 layers or the data volume is more than 100M; the network category comprises wide area network, local area network, wireless network and mobile network; the network speed is the actual transmission speed of the current network; the network speed is a dynamic parameter, the defining basis is two threshold parameters, an upper limit and a lower limit, different network types have different thresholds, and the thresholds are set according to requirements;
the decision making process starts from the operation platform; then judging the data quantity, and respectively measuring the data quantity from the number and the size; then checking the current network type; and finally, selecting a corresponding reconstruction model according to the current dynamic network speed.
2. The C/S architecture-oriented adaptive medical image three-dimensional reconstruction method according to claim 1, wherein: the decision tree longitudinally comprises four layers of structures, and the operation platform, the data volume of the medical image, the network type and the current network speed are respectively arranged from top to bottom; viewed from the transverse direction, the first layer structure divides the client into 6 types according to the operation platform, wherein the 6 types are respectively a PC local App, a PC Web App, a Pad local App, a Pad Web App, a Phone local App and a Phone Web App; secondly, judging the size of the data volume of the medical image by a second layer structure, and dividing the medical image into two types according to the size of the data volume, wherein one type is that the number of image layers is less than 200 layers and the data volume is less than 100M, and the other type is that the number of image layers is more than 200 layers or the data volume is more than 100M, and each client side respectively analyzes the two types of conditions; next, the third layer structure checks the current network type, and for the PC, the network type comprises a wide area network, a local area network and a wireless network; for Pad and Phone, the network types include mobile and wireless networks; and the last layer of structure selects a corresponding reconstruction mode according to the current dynamic network speed.
CN201910623642.XA 2019-07-11 2019-07-11 C/S architecture-oriented adaptive medical image three-dimensional reconstruction method Active CN110349254B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910623642.XA CN110349254B (en) 2019-07-11 2019-07-11 C/S architecture-oriented adaptive medical image three-dimensional reconstruction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910623642.XA CN110349254B (en) 2019-07-11 2019-07-11 C/S architecture-oriented adaptive medical image three-dimensional reconstruction method

Publications (2)

Publication Number Publication Date
CN110349254A CN110349254A (en) 2019-10-18
CN110349254B true CN110349254B (en) 2022-12-06

Family

ID=68175871

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910623642.XA Active CN110349254B (en) 2019-07-11 2019-07-11 C/S architecture-oriented adaptive medical image three-dimensional reconstruction method

Country Status (1)

Country Link
CN (1) CN110349254B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113808051A (en) * 2021-09-26 2021-12-17 武汉联影生命科学仪器有限公司 Multi-modal medical image reconstruction system and method

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102592310A (en) * 2011-12-28 2012-07-18 中国人民解放军第四军医大学唐都医院 Three-dimensional reconstruction method of image of mobile terminal for medical image teaching
CN107835436A (en) * 2017-09-25 2018-03-23 北京航空航天大学 A kind of real-time virtual reality fusion live broadcast system and method based on WebGL

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120084350A1 (en) * 2010-10-05 2012-04-05 Liang Xie Adaptive distributed medical image viewing and manipulating systems

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102592310A (en) * 2011-12-28 2012-07-18 中国人民解放军第四军医大学唐都医院 Three-dimensional reconstruction method of image of mobile terminal for medical image teaching
CN107835436A (en) * 2017-09-25 2018-03-23 北京航空航天大学 A kind of real-time virtual reality fusion live broadcast system and method based on WebGL

Also Published As

Publication number Publication date
CN110349254A (en) 2019-10-18

Similar Documents

Publication Publication Date Title
US10671255B2 (en) Facilitating transitioning between viewing native 2D and reconstructed 3D medical images
CN103136785B (en) For the implementation method of the human medical model three-dimension visible sysem of mobile terminal
JP3954596B2 (en) Image processing apparatus, image processing method, and image processing program
CN101976458A (en) Medical image three-dimensional visualization method applied to mobile handheld equipment
CN109215764B (en) Four-dimensional visualization method and device for medical image
CN110517238A (en) CT medical image AI three-dimensional reconstruction and human-computer interaction visual network system
CN106055881A (en) Cloud platform for medical data processing
CN110349254B (en) C/S architecture-oriented adaptive medical image three-dimensional reconstruction method
US20220358735A1 (en) Method for processing image, device and storage medium
CN113409333B (en) Three-dimensional image cutting method and electronic equipment
CN106845477B (en) Method and device for establishing region of interest based on multiple reconstructed images
CN115080886A (en) Three-dimensional medical model GLB file analysis and display method based on mobile terminal
CN113379929B (en) Bone tissue repair virtual reality solution method based on physical simulation
Ji et al. Research on telemedicine technology and implement based on Virtual Reality
JP4422161B2 (en) Image processing apparatus and image processing program
CN110334027A (en) Game picture test method and device
CN114388145A (en) Online inquiry method and device
Zhang et al. Server-aided 3d dicom viewer for mobile platforms
Yin et al. Remote three-dimensional reconstruction for medical images based on Browser/Server framework
Bohak et al. Web-based 3D visualisation of biological and medical data
CN113409433A (en) Medical three-dimensional model display and cutting system based on mobile terminal
CN113096779A (en) Medical image cloud image system based on distributed CT terminal
Mao Three-dimensional tree visualization of computer image data based on Louvain algorithm
Li et al. R4: a client server architecture oriented open three dimensional reconstruction layered reference model for medical images
CN109799936A (en) A kind of image generating method, device, equipment and medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant