CN100477717C - Method and system for remote and adaptive visualization of graphical image data - Google Patents

Method and system for remote and adaptive visualization of graphical image data Download PDF

Info

Publication number
CN100477717C
CN100477717C CNB038266539A CN03826653A CN100477717C CN 100477717 C CN100477717 C CN 100477717C CN B038266539 A CNB038266539 A CN B038266539A CN 03826653 A CN03826653 A CN 03826653A CN 100477717 C CN100477717 C CN 100477717C
Authority
CN
China
Prior art keywords
data
compression
client
screen picture
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CNB038266539A
Other languages
Chinese (zh)
Other versions
CN1839618A (en
Inventor
安德鲁·布鲁诺·多伯斯
尼尔斯·赫斯特德·谢尔
亚历山大·迪米特罗夫·卡雷瓦诺夫
莫登·西尔韦斯特·奥尔森
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Medical Informatics AS
Original Assignee
Medical Insight AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Medical Insight AS filed Critical Medical Insight AS
Publication of CN1839618A publication Critical patent/CN1839618A/en
Application granted granted Critical
Publication of CN100477717C publication Critical patent/CN100477717C/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/414Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/4143Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a Personal Computer [PC]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/327Initiating, continuing or ending a single-mode communication; Handshaking therefor
    • H04N1/32765Initiating a communication
    • H04N1/32771Initiating a communication in response to a request, e.g. for a particular document
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/327Initiating, continuing or ending a single-mode communication; Handshaking therefor
    • H04N1/32765Initiating a communication
    • H04N1/32771Initiating a communication in response to a request, e.g. for a particular document
    • H04N1/32776Initiating a communication in response to a request, e.g. for a particular document using an interactive, user-operated device, e.g. a computer terminal, mobile telephone
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/333Mode signalling or mode changing; Handshaking therefor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/333Mode signalling or mode changing; Handshaking therefor
    • H04N1/33353Mode signalling or mode changing; Handshaking therefor according to the available bandwidth used for a single communication, e.g. the number of ISDN channels used
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N1/333Mode signalling or mode changing; Handshaking therefor
    • H04N1/33376Mode signalling or mode changing; Handshaking therefor according to characteristics or state of one of the communicating parties, e.g. available memory capacity
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2347Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving video stream encryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4405Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video stream decryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/47202End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting content on demand, e.g. video on demand
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/60Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client 
    • H04N21/61Network physical structure; Signal processing
    • H04N21/6106Network physical structure; Signal processing specially adapted to the downstream path of the transmission network
    • H04N21/6125Network physical structure; Signal processing specially adapted to the downstream path of the transmission network involving transmission via Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/8146Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics
    • H04N21/8153Monomedia components thereof involving graphical data, e.g. 3D object, 2D graphics comprising still images, e.g. texture, background image
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17318Direct or substantially direct transmission and handling of requests
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N2201/00Indexing scheme relating to scanning, transmission or reproduction of documents or the like, and to details thereof
    • H04N2201/32Circuits or arrangements for control or supervision between transmitter and receiver or between image input and image output device, e.g. between a still-image camera and its memory or between a still-image camera and a printer device
    • H04N2201/333Mode signalling or mode changing; Handshaking therefor
    • H04N2201/33307Mode signalling or mode changing; Handshaking therefor of a particular mode
    • H04N2201/33342Mode signalling or mode changing; Handshaking therefor of a particular mode of transmission mode
    • H04N2201/33357Compression mode

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Compression Of Band Width Or Redundancy In Fax (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention relates to a method and system for remote visualization and data analysis of graphical data, in particular graphical medical data. A user operates a client machine (21) such as a thin client, a PC, a PDA, etc., and the client machine is connected to a server machine (20) through a computer network. The server machine runs an adaptive streaming module (ASM) which handles the connection between the client and the server. All data and data applications are stored and run on the server. A user at the client side requests data to be shown on the screen of the client, this request (24) is transferred to the server. At the server side the request is interpreted as a request for a particular screen image, and a data application generates the requested screen image and estimates a present available bandwidth (26) of a connection between the client and the server. Based on the estimated available bandwidth, the generated screen image is compressed using a corresponding compression method so that a compressed screen image is formed. The screen image may also be encrypted. The compressed (and possible encrypted) screen image is forwarded (22) to the client, and shown on the screen of the client (23). The compression method depends foremost upon the available bandwidth, however also the type of client machine (28), the type of request, etc. may be taken into account.

Description

The method and system that is used for the long-range and self adaptation demonstration of graphic image data
Technical field
The present invention relates to be used for the long-range demonstration of graph data and the method and system of data analysis, the invention particularly relates to the long-range demonstration and the data analysis of graphical medical.
Background technology
In order to show the various internal features of human body, for example various medical image scanners have been developed in the position of tumour.The volume scan instrument, be spatial digitizer, such as computerized tomography (CT), magnetic resonance imaging (MRI), ultrasonic wave (US), positron emission tomography (PET) and single photon emission computerized tomography (SPECT) and two-dimensional scanner, be available such as computer irradiation art (CR) and numerical control X line photography (DR).Scanner utilizes different biophysical mechanism to produce human body image.For example, the X ray in the specific elementary volume, volume element of CT scan device detection institute scan patients absorbs, and the MRI scanner uses magnetic field to detect the existence of water in the specific elementary volume, volume element of institute's scan patients.These two scanners all provide the tangent plane of human body, can make up to form the complete three dimensional image that the patient is scanned part.The common factor of most of medical scanners is that the data set that obtained (especially with data set that spatial digitizer obtained) is quite big, all comprises hundreds of million byte for each patient.Big like this data set needs powerful computing ability so that video data, in particular for handling and operating data.And this class view data of transmission has proposed the related challenge of safety and traffic congestion on common network.
Usually manage and store via the electronic database system under picture archiving and communication system (PACS system) wide class with the view data that medical image scanners produces, the PACS system use in the medical science standard (dicom standard) digital imagery with communicate by letter.Scanner is connected to the centralized server computer of store patient data set, or on one group of server computer.In traditional system, data can visit from the display workstation of single or several special uses then.This class work station is quite expensive, therefore can only visit in the separate diagnostic suite usually, and clinician's office, hospital ward or operating room then can not be visited.
In the environment that uses conventional client terminal/server structure, there is another kind of more cheap system.Here still need the considerable volume server of computing capability, but centralized server computer can visit from various client type (for example thin-client).In this type systematic, display routine moves on central server, and the output of this program connects the remote display that sends to client via network.
An example of client/server system is by Silicon Graphics, the OpenGLVizserver that Inc. provides TMSystem ( Http:// www.sqi.com/software/vizserver/).This system makes such as Silicon
Figure C0382665300071
And so on client and can obtain based on the work station of PC
Figure C0382665300072
The graphic capability of server.In this solution, need specific software to be installed in client.This has not only limited the client-class that can be used to access server, but also has increased additional maintenance needs, because Vizserver TMClient software must be installed on each client workstation this locality.In addition, Vizserver TMInformation in the frame that sends before server software is not attempted reusing.Therefore, have only when using just suitable this type systematic of operation of special-purpose high speed data network.But not this situation usually for many hospitals; And this class network installation also is the task of needing the writing spending.
At United States Patent (USP) 6,014, in 694, disclosed the system that is used for transmitting adaptively video by network, available bandwidth changes in time in this network.This system comprises video/audio encoder/decoder, and it is used to compress, encodes, decodes and decompresses and is connected the video flowing that transmits by network.This system adjusts compression ratio according to channel width, to adapt to various bandwidth.Provide bandwidth adjustability by the compromise proposal that provides between video resolution, frame rate and the individual frames quality.Original video source is divided framing, and wherein, each frame all comprises a plurality of data hierarchies of expression different quality degree.Videoconference client receives some levels of each frame according to bandwidth, and the received level of each frame is high more, and the quality of this frame is just high more.Have only known data stream will be sent out many times (as under the situation of video flowing), this type systematic could be worked best.If this data flow is unique all when it is sent out at every turn, then this system all can produce a large amount of redundant datas for each session, and can not be before the request of receiving the branch framing, produce redundant data thereby taken computing capability.
Summary of the invention
The present invention seeks to overcome the problem relevant with operation with the long-range demonstration of big digital data sets.
According to first form of the present invention, a method is provided, be used for graph data being sent at least one second from first device and install in computer network system, this method comprises the following steps:
Generation is to the request of screen picture,
In first device, as long as receive request, just to screen picture:
Produce the screen picture of being asked,
The current available bandwidth of the connection between estimation first and second device of at least one,
Based on the available bandwidth that estimates, the screen picture that is produced with corresponding compression method compression so that form compressed screen image and
Compressed screen image is forwarded to second device of at least one.
Graph data can be the graph data of any kind, but medical image preferably, the data that for example obtain in conjunction with patient's medical scanning.Graph data is stored on first device, and first device can be a central computer, or the central group of computer.First device can comprise the computer or the computer cluster of any kind, and they have necessary total memory capacity with storage mass data group (for example producing by a large amount of patients of scanning in hospital).First device also will be equipped with essential computing capability, with can Treatment Analysis and the demanding task of a large amount of three-dimensional data groups of operation (such as the 3-D view at positions such as people's head, chest).
At least the second device can be the computer of any kind, and it is equipped with the screen that is used for graphical display.The term demonstration should be interpreted into and comprise that simultaneously two dimension shows and three-dimensional display.Second device of at least one can for example be a thin-client, the wireless handheld device such as PDA(Personal Digital Assistant), personal computer (PC), flat computer, laptop computer or work station.Second device of at least one can only serve as the graphic terminal of first device.Second device of at least one can receive request action and transmit this request to first device from the user, and receives and show screen picture by the first device generation.The screen of second device of at least one can be regarded as being connected to the screen of first device in many aspects.
For example, asked an action by second user who installs of at least one or by routine call.This action can for example be presented on second screen that installs of at least one tabulation of possible selection, and perhaps this action can make the image relevant with patient data be presented on second screen that installs of at least one.This request can be based on from the user instruction that moves such as button, mouse, user interactive incident the click receives.
As long as receive request, first device is just according to this request is explained in the request of particular screen image.First device obtains relevant patient data from the storage medium of its connection.This storage medium can be the storage medium of any kind, such as hard disk.Result as this request produces screen picture.Estimate current connection bandwidth, and, compress this screen picture with corresponding compression method based on available bandwidth that estimates and request type.First device is forwarded to compressed screen image in second device of at least one.
Yet the request that first device also may not receive second device that comes from least one just produces the screen picture of a non-request.The screen picture of this non-request may be based on corresponding patient data, and perhaps the screen picture of non-request also may be irrelevant with any request that patient data or user make.The screen picture of this non-request may produce owing to the instruction that the first device place exists.
The generation of screen picture also may according at least one second the device type as condition.If for example second device of at least one is PDA, it may be unnecessary then producing high-resolution image, because current available PDA is restricted on their resolution.Therefore, the image that produces PDA is compared with the identical image that produces thin-client, and the resolution that has is lower.
Compression method may be a condition with the type of request also.The compression of graph image may relate to loss, and the image that obtains after the promptly compressed and decompressed processing is inconsistent with the image before the compression resolution process, and these class methods are commonly referred to the lossy compression method method.The lossy compression method method is usually carried out very fast, and can be with higher compression compressed image recently.Can consider request type in these situations, promptly not have loss be important to the image of Xie Yasuoing, or loss is unessential.The type of request can be such as being: display image, image rotating, enlarged image, mobile image etc.
Compression method may be a condition with second type of installing of at least one also.Especially can consider the computing capability of second device of at least one.For example, if to make that decompression tasks is estimated as too consuming time for the computing capability that has of second device of at least one, then can use different and require lower compression method.
Because system may be used for transmitting responsive personal information on data network, can encryptedly may be very important so be transmitted data.Therefore, first device may comprise a device, is used for before screen picture being sent to second device of at least one it being encrypted.Similarly, second device of at least one may have a device, is used for deciphering the screen picture that receives before on the screen of second device that screen picture is created at least one.And this system may comprise a feature, and wherein, the user artificially is provided with secret grade, or system may automatically be provided with suitable secret grade.The time of the screen picture that deciphering receives may depend on that processing unit, the especially hand-held device of second appliance arrangement of at least one may be subject to processing the restriction of ability.Therefore in some cases, this may be to use the limiting factor of the encipheror of high request.Therefore, the encipheror that is used to enciphered data depends on the type of second device of at least one.
Except view data, the application program that is used for data analysis, data manipulation and data demonstration also can be stored in first device, and can operation from first device.This application program can also be stored in via the computer network connection and be connected on the device of first device, and can therefrom move.A large amount of application programs can be from the first device visit.This application program can comprise and is suitable for operating 3-D graphic medical data (such as the data that come from MRI, CT, US, PET and SPECT) and X-Y scheme medical data (such as the data that come from CR and DR), and the software that is suitable for operating the data that come from other device that produces medical image.Operation can be any standard operation of data, such as rotation, amplify and dwindle, zone of montage or data subset or the like.Operation also can be non-type operation, perhaps can be to be the specifically developed unique operation of native system.
In order to obtain system flexibly, can use different compression methods.Compression method can be selected or automatically be selected by software in the artificially when session begin.The compression method more different than adopting according to desired compression.Compression method may be in difference aspect compression time, compression ratio and their the optimal data types.Can use various compression methods, comprise two standard methods and be the specifically developed method of native system.
For example, a specific compression method is so-called gray cell compression (GCC) method, wherein, and compression rgb color graph image or GTG graph image.This compression method comprises the following steps:
Graph image is divided into the unit that comprises the 4x4 pixel again,
Determine the averaging unit color of each unit,
At the averaging unit color is under the situation of gray-scale colors, with 1 bit this unit is labeled as GTG, and represents gray-scale colors with 7 bits, or
At the averaging unit color is not under the situation of gray-scale colors, with 1 bit this unit is labeled as non-GTG, and represents color with 15 bits.
The GCC method especially is particularly suitable for compressing the image that most of image is a GTG.Because many medical objects usually may be with gray scale imaging, so the GCC method is well suited for the compression medical image.
As long as session begins, just can create and keep second appliance arrangement of at least one and the session between first device at the session manager of first setting position, and control assembly is uploaded to second device of at least one.Second device of at least one can be the computer that does not have operating system (OS), for example thin-client.Can upload OS in this case, can accept and send the request action, and can receive and show the screen picture that produces by first device so that second device of at least one becomes.Yet second device of at least one also can be a computer that OS is arranged, for example PDA or PC.For these equipment, OS works on second device of at least one, and it only need upload computer applied algorithm to start session in this case.Yet, can upload to computer applied algorithm second device of at least one and create and/or keep session from first device.For example, may be enough to allow second device of at least one to receive screen picture from first device.Unnecessary at least one second the device on the operation computer applied algorithm receive, check and/or even at least one second the device on operate screen picture.
Provide a frame device is set, it is according to detected available bandwidth and the resolution of frame buffer of second device of at least one optionally is set according at least one the specification requirement of second device.That is, if detected bandwidth is very low, then the resolution of frame buffer may be set at low value, and can produce screen picture according to the resolution of this frame buffer.The method that low resolution is Fast Compression data is arranged in frame buffer.The graphic hardware of most computers system has such function, if promptly received the screen picture that resolution is lower than screen resolution, then automatically amplifies screen picture and is full of whole screen.In this situation, second device of at least one is gone up last screen output and is restricted on resolution naturally.Under the acceptable situation of detected bandwidth, the resolution of frame buffer can be set at the screen resolution of second device of at least one.In this case, though taken more bandwidth, but kept resolution completely.Because at present available PDA screen resolution is limited,, then can consider the specification of second device of at least one if second device of at least one for example is a PDA.Only because its second device at least one can be downsampled, transmitting the too high image of resolution is the waste of bandwidth.
Target double sampling device can be provided, its according to detected available bandwidth, and optionally also be provided with and show and painted parameter according at least one the specification of second device.Can change the color depth of the screen picture that is produced, when bandwidth is low, can use 8 colors, if bandwidth allows then can use 16,24 or 32.In addition, also can consider the computing capability of second device of at least one.The time of the screen picture that decompression receives may depend on that processing unit, the especially hand-held device of second appliance arrangement of at least one may be subject to processing the restriction of ability.Therefore, in some cases, not compressing or slightly compress screen picture may be faster.
Through setting, double sampling, compression and perhaps ciphered data be sent to the incoming/outgoing management device of second device end of at least one by the incoming/outgoing management device of first device end, it is the transmission of process user interactive operation to the first device also.
In many cases, the screen picture of being asked is compared with the screen picture on second device screen that is presented at least one and is had only very little variation.In this situation advantageously, the screen picture that produces at least one second device end be based on the screen picture that receives from first device or based at least one second device end frame buffer content or based on the combination of screen picture that receives and frame buffer content.Promptly, the screen picture that receives comprises the change with respect to the screen picture of former transmission, so as the screen picture that shows be by at least one second device frame buffer and can obtain before the two the stack of screen picture that shows and the image modification that receives.
Most of networks are shared resources, and the available bandwidth on any particular moment, network connected in time and change in location.Estimate current available bandwidth, and change rate of data signalling in view of the above.When not receiving request action, second device at least one sends screen frame, and second device of at least one is clear screen from second frame buffer of installing of at least one in this case.What therefore, the network connection took is variable amount of bandwidth.
Data network has been installed by many hospitals, clinic or other medical institutions, and the clinical medicine doctor can be in or can not visit jumbo network in little medical office.So importantly, second device of at least one and first device can be connected communication via a plurality of possible common networks, and the Internet connection or the in-house network that connect such as for example Ethernet connect, or by the cable connection or by wireless connections.Especially, second device and first device can be by the network services of any kind, and these networks use the Internet protocol (IP) such as the Internet or other TCP/IP network.Second device can be communicated by letter with non-dedicated Internet access by special-purpose with first device.
Graph data can be based on to be followed picture archiving and communication system (PACS system) and goes up the digital imagery in the medical science standard of implementing and the graphical medical of the data of communicate by letter (dicom standard).Most of medical scanners are all supported dicom standard, and it is a standard of handling the compatibility between the different system.Can provide text data with graph data.Preferably, text data be based on follow health and fitness information exchange layer 7 agreement (HL7) standard or be used to manage, the data of commercial and electronic data interchange (EDIFACT) standard that transmits.The exchange of figure and/or medical data is based on International Health information exchange (IHE) framework that is used for data interchange.
According to second form of the present invention, be provided in computer network system, transmitting the system of graph data.This system comprises:
At least one second device is equipped with device that is used for the recording user input and the display unit that is used for the display graphics data,
First device is equipped with:
Be applicable to the software that produces screen picture,
Be used to estimate first and at least one second device between the device of the available bandwidth that connects,
Be applicable to the software that compresses screen picture with a plurality of compression methods, with form compressed screen image and
Be used for compressed screen image is sent to the device of second device of at least one.
First device can also comprise a device, be used to encrypt to connect the data that send by the computer between first device and second device of at least one, and second device of at least one can comprise the device that is used to decipher the data that receive.
Second device of at least one can be connected communication by common network with first device.First device can be a computer server system, and second device of at least one can for example be thin-client, work station, PC, flat computer, laptop computer or wireless handheld device.First device can be the part of PACS system or this system.
Description of drawings
Describe the preferred embodiments of the present invention in detail referring now to accompanying drawing, wherein:
Fig. 1 shows the schematic diagram of the preferred embodiments of the present invention;
Fig. 2 illustrates schematic flow diagram, and the function of self adaptation flow module (ASM) has been described;
Fig. 3 illustrates the rotation example and the corresponding bandwidth of data object;
Fig. 4 has illustrated the contact between compression time, employed compression method and the obtainable compression ratio of lossless compress; With
Fig. 5 has illustrated the contact between compression quality, employed compression method and the obtainable compression ratio of lossy compression method.
Embodiment
The invention provides the method and system that is used for graph data being sent to from first device second device of at least one in computer network system.Followingly describe the present invention with reference to preferred embodiment, wherein, graph data is a graphical medical, and computer network system is a client/server system.Fig. 1 has provided a schematic diagram.
Obtain medical image with the medical scanners 1 that is connected to server computer 2.A large amount of clients 3 can be connected to this server.This server is the part of PACS system.When the patient stands to scan, the image 16 of acquisition can be automatically or the artificially be sent to and be stored on the server apparatus.Although only reference server or server apparatus, this server can be independent computer, computer cluster or the computer system that connects by computer.Can be in the visit of whenever setting up thereafter to image.Except view data, the application program 15 that is used for data analysis and demonstration also is stored on the server apparatus and therefrom operation.Server is equipped with necessary computing capability, with can Treatment Analysis and the task of the high request of the big three-dimensional data group of operation, such as the 3-D view at positions such as people's head, chest.All data and all storage on server, operation and the processing of data-application 15 that are used to show and analyze.
Client 3 can be the computer equipment that is equipped with any kind of the screen that is used for graphical display.Client can for example be thin-client (Thin Client) 5, the wireless handheld device such as PDA(Personal Digital Assistant) 6, personal computer (PC), laptop computer or work station 7 etc.
Self adaptation flow module (ASM) 4 is used for guaranteeing data flow continuous between the server and client side.ASM can estimate current available bandwidth, and can change the transfer rate of data in view of the above.ASM 4 is parts of server apparatus 2.
Client can comprise ASM 5,6,7, and perhaps it can not comprise ASM 17.Customer end A SM is not that system works is necessary.
ASM comprises session manager 8.The session between client device and the server is created and kept to session manager.Session manager 8 uploads to control assembly in second device of at least one.For example, if client is a thin-client 5, then upload operation system (OS) at first can accept and send request action so that thin-client becomes, and can receive and show the screen picture that is produced by server.In client is under the situation of PDA 6 or PC, and operating system works on this client, and it only need upload computer program to start session in this case.
ASM also comprises Bandwidth Broker 9, and it constantly measures available bandwidth.Frame is provided with device 10, and it is provided with the resolution of the frame buffer of client.Target double sampling device 11 is provided with and shows and painted parameter.Condensing encoder 12 compressed images.Encryption equipment 13 comprises and being used for before data are sent to client 3 its device of encrypting.Carrying out setting, double sampling, compression and ciphered data is transmitted by incoming/outgoing management device 14.
The schematic flow diagram of explanation ASM module 20 functions shown in Figure 2.The user of medical data for example can be the surgeon, he will be on the background of 3-D view of scanning planning surgery.The user at first sets up a connection from graphic interface 21 (such as the thin-client its office).At first the user should sign in to system so that the identification identity.Then, present a tabulation and give the user, the user can ask to visit the associated picture that will be presented on the computer screen 23 from this tabulation.In another example, the user of medical data is the clinician who makes a round of visits in hospital ward.For the ease of discussing or being convenient to the understanding of patient to its state of an illness, the clinician can be with a PDA, and he can at first log in the system by PDA, visits this patient's associated picture then.
The user of client is asking an action, such as patient's specific image.To ask 24 to send to server, server is according to this request is explained in the request of specific screen picture.Server obtains relevant view data 25 from the storage medium that it connects.The current bandwidth 26 that estimation connects, and, screen picture is compressed to corresponding compression ratio based on detected available bandwidth and a plurality of other parameter.For instance, can be with two other parameter generating screen pictures.First parameter can be a color depth 27.If the user for example asks the image of cerebral veins, venae cerebri, then can use 24 RGB color depth, and if the user for example asks the image of skull, then 8 color depths may be just enough.Second parameter can be a client type 28.If requesting client equipment is thin-client, then can do the figure interface with 19 inches screen.In this case, can produce the image of 768 * 1024 pixels.If but requesting service is a PDA, then will be produced a little bit smaller a little image, the image of one 300 * 400 pixel for example is because most of PDA has limited screen resolution.
Produce, compress and encrypt screen picture 22.Image is sent to client device, wherein, and before it is presented on the screen 23 that the request user uses, at first to its deciphering with decompress 29.
The surgeon can use a large amount of 3-D graphic programs, such as rotation, convergent-divergent etc., for example is used for obtaining the understanding to the target location that will have an operation.In Fig. 3, provide the rotation example and the corresponding bandwidth of data object.
By using the step of explaining in conjunction with Fig. 2, the user has asked the 3-D view of skull 30.During image transmits, use a certain amount of bandwidth 34, but needed only transmitted image, do not have so or very little bandwidth occupied 35.Now, the user wants image rotating to obtain different view 31,32,33.For example, the user can click image, keeps mouse button to be pressed then and comes rolling mouse by the desired shape direction of revolving.Thereby request type is the object rotation, and when pressing the mouse button maintenance, software is treated to rotation to request.
The compression of graph image is trading off between resolution and the compression ratio.Desired resolution is low more, and operable compression ratio is just high more.When object of rotation, during rotation 31,32, when target rotation, only need an image indication, and when stopping the rotation, just need to transmit high-quality image 33.Image 31 and 32 uses the step of explaining in conjunction with Fig. 2 to transmit, but higher image compression rate causes required bandwidth lower.When mouse button unclamped, the image 33 of transmission was not re-used as rotation, and used lower compression.
Use the compression method of two types, lossless compression method and the compression method or the lossy compression method method that provide loss.Two dissimilar compression methods all are used.The compression method more different than adopting according to desired compression.Compression method can be in difference aspect compression time, compression ratio and their the optimal image types.Image compression depends primarily on available bandwidth to be determined, but request type also is very important, especially is related to use to can't harm when still damage method being arranged.Lossless compression method for three standards: PackBits (or run-length encoding), BZIP2 and Lempel-Ziv-Obenhumer (LZO) provide the example of getting in touch between compression time and the compression ratio in Fig. 4.In Fig. 5, compression method for two standards: colour cell compression (CCC) and expansion colour cell compression (XCCC), and specific compression method, promptly so-called GTG cell compression (GCC) illustrates the picture quality of lossy compression method method and the contact between the compression ratio.
Can distinguish or use these methods continuously, to obtain higher compression ratio.For example, can be with CCC compression and LZO compression (CCC::LZO) combination.
In Fig. 4,,, compared compression time with obtainable compression sizes 40 or compression ratio for PackBits compression method 41, BZIP2 method 42 and LZO method 43.The structure that is compressed image is depended in correct contact between compression time and the compression ratio.Illustrate by the specific area extension that is taken by each method.
In Fig. 5,,, compared picture quality with obtainable compression sizes 50 for various compression methods (combinations of a method or a plurality of methods).
If image comprises very big gray-scale areas, it is useful using so and utilizing the special compression method of this information.Gray cell compression (GCC) method is an example of this class compression method.GCC is a variant of standard CC C technology.It has utilized this fact, comprises that promptly the unit of GTG pixel has GTG averaging unit color.Utilize this more effectively two averaging unit colors to be encoded being used for.If the averaging unit color is a gray-scale colors, coming this color indicia with 1 bit so is gray-scale colors, and represents the GTG value with 7 bits.If the averaging unit color is non-gray-scale colors, with 1 bit this unit is labeled as non-gray-scale colors so, and represents this color self with 15 bits.
The compression ratio of GCC method depends on that the grey exponent part of image has much.In the poorest situation, the neither one average color is a gray-scale colors.In this case, compression ratio is 1: 8.In optimal cases, all average color all are gray-scale colors, draw 1: 12 compression ratio.Than the CCC method of standard, the advantage of GCC method is to comprise that the image of very big gray-scale areas can transmit with lower bandwidth and higher image quality.
Although described the present invention in conjunction with the preferred embodiments, yet it is not to be intended to be subject to particular form set forth herein.Now anti-, scope of the present invention is only limited by subsidiary claim.

Claims (26)

1. be used for graph data being sent to from first device method of second device of at least one in computer network system, so that show, analyze and operation three-dimensional data group, wherein be used for the demonstration of data set, the storage on first device of all data of analysis and operation and data-application, operation and processing, described demonstration, analyze and operate the interactive operation of carrying out at least one second device based on the user, this method comprises the following steps:
Generation is to the request of screen picture,
In first device, as long as receive request, just to screen picture:
Obtain the computing capability of at least one second device,
The current available bandwidth of the connection between estimation first and second device of at least one,
Based on the available bandwidth that estimates, by target double sampling device demonstration and painted parameter are set,
Produce the screen picture of being asked,
Based on the available bandwidth that estimates, the computing capability of request type and at least one second device, with the combination of corresponding compression method or corresponding compression method, the screen picture that compression is produced so that form compressed screen image and
Compressed screen image is sent to second device of at least one.
2. method according to claim 1 wherein, does not have to receive first of request from second device of at least one and is installing:
The current available bandwidth of the connection between estimation first and second device of at least one,
Based on the available bandwidth that estimates, by target double sampling device demonstration and painted parameter are set,
Instruction according on first device produces the screen picture of non-request,
Based on the available bandwidth that estimates, with the combination of corresponding compression method or corresponding compression method, the screen picture that produced of compression, so as to form compressed screen image and
Compressed screen image is sent to second device of at least one.
3. according to any described method in claim 1 or 2, wherein, the generation of screen picture also with at least one the type of second device as condition.
4. according to any described method in claim 1 or 2, wherein, the combination of employed compression method or corresponding compression method is a condition with second type of installing of at least one also.
5. according to any described method in claim 1 or 2, wherein, the graph data that transmits between first device and second device of at least one is encrypted.
6. according to any described method in claim 1 or 2, wherein, graph data is a graphical medical.
7. according to any described method in claim 1 or 2, wherein, graph data and lot of data analysis and display application procedure stores/operate in first device, or by on computer network and the device that first device is connected.
8. according to any described method in claim 1 or 2, wherein, adopt different compression methods according to the desired compression ratio.
9. according to any described method in claim 1 or 2, wherein, select compression method or automatically select in the artificially at first by software in session.
10. according to any described method in claim 1 or 2, wherein, control assembly is uploaded to second device of at least one from first device.
11. according to any described method in claim 1 or 2, wherein, device is set according to the available bandwidth that estimates at the frame of first device end, and optionally according to the specification of second device of at least one, the resolution of frame buffer at the second device place of at least one is set.
12. according to any described method in claim 1 or 2, wherein, target double sampling device is provided with according to the specification of second device of at least one and shows and painted parameter.
13. according to any described method in claim 1 or 2, wherein, the incoming/outgoing management device at the first device end place sends at least one second to the frame buffer data that also might encrypt through setting, double sampling, compression and installs, and the incoming/outgoing management device at the second device end place of at least one receives graph data.
14. according to any described method in claim 1 or 2, wherein, the screen picture that produces at least one second device end is based on the screen picture that receives from first device, or based on the frame buffer content of second device end, or based on the combination of screen picture that receives and frame buffer content at least one.
15. according to any described method in claim 1 or 2, wherein, the computer network connection takies variable amount of bandwidth, and takies minimum bandwidth when at least one second installed when not transferring data to from first device.
16. according to any described method in claim 1 or 2, wherein, second device of at least one and first device are connected by common network and communicate, and perhaps by the cable connection, perhaps by wireless connections, perhaps connect by mobile phone.
17. method according to claim 16, wherein, connection protocol is an ICP/IP protocol.
18. according to any described method in claim 1 or 2, wherein, the generation of screen picture is based on follows the DICOM, the HL7 that implement or the data of EDIFACT standard in the PACS system.
19. method according to claim 1, wherein, the graph image or the GTG graph image of compression RGB look, described compression method comprises the following steps:
Graph image is divided into the unit that comprises 4 * 4 pixels again,
Determine the averaging unit color of each unit,
At the averaging unit color is under the situation of gray-scale colors, with 1 bit this unit is labeled as GTG, and represents gray-scale colors with 7 bits, or
At the averaging unit color is not under the situation of gray-scale colors, with 1 bit this unit is labeled as non-GTG, and represents color with 15 bits.
20. be used between device, transmitting graph data so that show in computer network system, analyze and operate the system of three-dimensional data group, wherein be used for the demonstration of data set, the storage on first device of all data analyzing and operate and data-application, operation and processing, the interactive operation of carrying out based on the user is analyzed and is operated in described demonstration at least one second device, described system comprises:
At least one second device is equipped with device that is used for the recording user input and the display unit that is used for the display graphics data,
First device is equipped with:
Be applicable to the software that produces screen picture,
Be used to obtain the device of at least one second computing capability of installing,
Be used to estimate first and at least one second device between the device of available bandwidth of connection,
Be used to be provided with the target double sampling device of demonstration and painted parameter,
Be applicable to the software that uses a plurality of compression methods to compress screen picture, so as to form compressed screen image and
Be used for compressed screen image is sent to the device of second device of at least one.
21. system according to claim 20, wherein, first device also comprises being used to encrypt and will connect the device of the data that send by the computer between first device and second device of at least one, and second device of at least one comprises the device that is used to decipher the data that receive.
22. system according to claim 20, wherein, second device and first of at least one installs to be connected by common network and communicates.
23. system according to claim 20, wherein, it is non-dedicated Internet access that network connects.
24. system according to claim 20, wherein, first device is a computer server system.
25. system according to claim 20, wherein, second device of at least one is thin-client, workstation computer, PC, laptop computer, flat computer, mobile phone or wireless handheld device.
26. system according to claim 20, wherein, first device is the part of PACS system or PACS system.
CNB038266539A 2003-05-13 2003-05-13 Method and system for remote and adaptive visualization of graphical image data Expired - Fee Related CN100477717C (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/DK2003/000312 WO2004102949A1 (en) 2003-05-13 2003-05-13 Method and system for remote and adaptive visualization of graphical image data

Publications (2)

Publication Number Publication Date
CN1839618A CN1839618A (en) 2006-09-27
CN100477717C true CN100477717C (en) 2009-04-08

Family

ID=33442589

Family Applications (1)

Application Number Title Priority Date Filing Date
CNB038266539A Expired - Fee Related CN100477717C (en) 2003-05-13 2003-05-13 Method and system for remote and adaptive visualization of graphical image data

Country Status (5)

Country Link
EP (1) EP1634438A1 (en)
CN (1) CN100477717C (en)
AU (1) AU2003223929A1 (en)
CA (1) CA2566638A1 (en)
WO (1) WO2004102949A1 (en)

Families Citing this family (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102004038668A1 (en) * 2004-08-09 2006-02-23 Siemens Ag Image data object transmitting method for use in medical area, involves classifying object-based data into safety and non-safety critical data, sending safety data after authorization and non-safety data to target device
US20070183493A1 (en) * 2005-02-04 2007-08-09 Tom Kimpe Method and device for image and video transmission over low-bandwidth and high-latency transmission channels
US7996495B2 (en) * 2006-04-06 2011-08-09 General Electric Company Adaptive selection of image streaming mode
DE102006053261B4 (en) * 2006-11-11 2015-04-16 Visus Technology Transfer Gmbh System for the reproduction of medical images
JP4935385B2 (en) * 2007-02-01 2012-05-23 ソニー株式会社 Content playback method and content playback system
EP2308189B1 (en) 2008-06-04 2019-04-17 Koninklijke Philips N.V. Adaptive data rate control
CN101606863B (en) * 2008-06-19 2011-05-11 深圳市巨烽显示科技有限公司 Method and system for controlling medical display network
TWI364220B (en) 2008-08-15 2012-05-11 Acer Inc A video processing method and a video system
CN101686382B (en) * 2008-09-24 2012-05-30 宏碁股份有限公司 Video signal processing method and video signal system
US8145001B2 (en) 2008-10-01 2012-03-27 International Business Machines Corporation Methods and systems for proxy medical image compression and transmission
EP2399380B1 (en) * 2009-02-20 2017-05-10 Citrix Systems, Inc. Systems and methods for intermediaries to compress data communicated via a remote display protocol
JP5067409B2 (en) * 2009-09-28 2012-11-07 カシオ計算機株式会社 Thin client system and program
FR2975556B1 (en) * 2011-05-19 2014-05-16 Keosys SYSTEM FOR REMOTE RESTRUCTURE OF MEDICAL DATA PROCESSED
CN102857791B (en) * 2012-09-14 2015-07-08 武汉善观科技有限公司 Method for processing and displaying image data in PACS system by mobile terminal
CN103121324B (en) * 2013-02-06 2015-09-16 心医国际数字医疗系统(大连)有限公司 A kind of medical imaging concentrates the system of printing
CN104423783B (en) * 2013-09-02 2019-03-29 联想(北京)有限公司 The method and electronic equipment of information transmission
CN103475953B (en) 2013-09-13 2017-11-17 华为技术有限公司 A kind of media control method and equipment based on desktop cloud
CN103914998A (en) * 2014-04-21 2014-07-09 中国科学院苏州生物医学工程技术研究所 Remote medical consultation training system with medical image processing function
WO2018050247A1 (en) 2016-09-16 2018-03-22 Siemens Healthcare Gmbh Controlling cloud-based image processing by assuring data confidentiality
US10820151B2 (en) * 2016-10-06 2020-10-27 Mars, Incorporated System and method for compressing high fidelity motion data for transmission over a limited bandwidth network
CN106485079B (en) * 2016-10-12 2019-06-07 南京巨鲨医疗科技有限公司 A kind of medical image cloud processing method
WO2020030488A1 (en) * 2018-08-07 2020-02-13 Signify Holding B.V. Compressive sensing systems and methods using edge nodes of distributed computing networks
CN111833788B (en) * 2019-04-19 2023-08-04 北京小米移动软件有限公司 Screen dimming method and device, terminal and storage medium

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1056273A3 (en) * 1999-05-25 2002-01-02 SeeItFirst, Inc. Method and system for providing high quality images from a digital video stream
CA2312333A1 (en) * 2000-06-21 2001-12-21 Kimihiko E. Sato Multimedia compression, coding and transmission method and apparatus
GB2367219A (en) * 2000-09-20 2002-03-27 Vintage Global Streaming of media file data over a dynamically variable bandwidth channel
AU2002232634A1 (en) * 2000-12-18 2002-07-01 Agile Tv Corporation Method and processor engine architecture for the delivery of audio and video content over a broadband network
US8107524B2 (en) * 2001-03-30 2012-01-31 Vixs Systems, Inc. Adaptive bandwidth footprint matching for multiple compressed video streams in a fixed bandwidth network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
图像存档和通信系统的要求条件. 曹新华.国外医学生物医学工程手册,第16卷第5期. 1993
图像存档和通信系统的要求条件. 曹新华.国外医学生物医学工程手册,第16卷第5期. 1993 *

Also Published As

Publication number Publication date
AU2003223929A1 (en) 2004-12-03
WO2004102949A1 (en) 2004-11-25
CA2566638A1 (en) 2004-11-25
EP1634438A1 (en) 2006-03-15
CN1839618A (en) 2006-09-27

Similar Documents

Publication Publication Date Title
CN100477717C (en) Method and system for remote and adaptive visualization of graphical image data
US20040240752A1 (en) Method and system for remote and adaptive visualization of graphical image data
CN101334818B (en) Method and apparatus for efficient client-server visualization of multi-dimensional data
US8422770B2 (en) Method, apparatus and computer program product for displaying normalized medical images
EP1236082B1 (en) Methods and apparatus for resolution independent image collaboration
US6711297B1 (en) Methods and apparatus for dynamic transfer of image data
US11664115B2 (en) Volumetric imaging technique for medical imaging processing system
JP5681706B2 (en) Method and system for advanced visualization and high-speed access of medical scans using a dedicated web portal
US20060122482A1 (en) Medical image acquisition system for receiving and transmitting medical images instantaneously and method of using the same
US7706626B2 (en) Digital image reconstruction using inverse spatial filtering
US8417043B2 (en) Method, apparatus and computer program product for normalizing and processing medical images
CA2547014A1 (en) Improved system for video digitization and image correction for use with a computer management system
AU2016204080B2 (en) Presentation and manipulation of high depth images in low depth image display systems
CN102665075A (en) Method and system for extracting and displaying medical images for mobile handheld device
US20100114603A1 (en) Systems and Methods for Obtaining Readings of Diagnostic Imaging Studies
US20070223793A1 (en) Systems and methods for providing diagnostic imaging studies to remote users
US20170228918A1 (en) A system and method for rendering a video stream
JP2005287927A (en) Image processor, image processing method and medical image system
Pohjonen et al. Pervasive access to images and data—the use of computing grids and mobile/wireless devices across healthcare enterprises
KR20110004086A (en) System and method for sharing web page of representing health information
US20030095712A1 (en) Method for determining a data-compression method
CN1392505A (en) Medical information service system
Prabhu et al. Secured transmission of medical images in radiology using aes technique
Akhtar et al. Significance of Region of Interest applied on MRI image in Teleradiology-Telemedicine
Kandasamy et al. Radiology Information System and Picture Archiving and Communication System

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
C14 Grant of patent or utility model
GR01 Patent grant
C17 Cessation of patent right
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20090408

Termination date: 20100513