CN102883660A

CN102883660A - Advanced multimedia structured reporting

Info

Publication number: CN102883660A
Application number: CN2010800592273A
Authority: CN
Inventors: D·J·维宁
Original assignee: University of Texas System
Current assignee: University of Texas System
Priority date: 2010-09-20
Filing date: 2010-11-27
Publication date: 2013-01-16
Also published as: JP2013527503A; KR20140024788A

Abstract

Embodiments of methods, systems, and apparatuses for generating a composited multimedia-based report are described. In one embodiment, a method includes capturing a medical image configured to be displayed on a medical image display device. The present methods may be independent of the medical image display, and may be able to capture images from any proprietary medical image viewer. The method may also include capturing description data related to the medical image. Additionally, the method may include processing the medical image and the description data related to the medical image on a data processing device. Also, the method may include storing the medical image and the description data related to the medical image in a data storage device.

Description

Senior multi-media architecture report

Background of invention

The application requires the U.S. Provisional Application S/N.61/264 of submission on November 25th, 2009, the U.S. Provisional Application S/N.61/384 of 577 and 2010 on JIUYUE submission in 20,, and 599 priority, the full content of these applications is incorporated herein by reference.

2. description of related art

Radiology report is usually by making the expert of radiology department visually check image or a series of images and give an oral account subsequently the narrative description that image finds and consist of.Language description can be transcribed to the computer system of literal by the record person of writing or voice, to be created on the literal report (Sobel etc., 1996) that is changing aspect content, definition and the style between the radiologist.Be used for transmitting the guide that diagnosing image is found although ACR has announced, this guide is not stipulated general reporting format (ACR, 2005).

Structured report (SR) is advocated by professional organization (such as Radiological Society of North America), but is organized into search database (Kahn etc., 2009 with the information content of image being found and be associated; Reiner etc., 2007).The advantage of SR is that it can be convenient to the application such as data mining, disease tracking and purposes management.Proposed many SR solutions, but two main challenges have hindered the general of these schemes adopted.At first, most of SR solution manages to change the mode that the radiologist puts into practice naturally.For example, some SR solutions need the radiologist to finish the predefine report template or with the click of computer mouse to image; Yet radiologist's natural workflow is to check image, is the oral account of the language description of the image discovery that sometimes can occur after initial the observation afterwards.Secondly, the employed various image display systems of radiologist are the special-purpose commodity by the FDA supervision, although and proposed the SR standard, ask the seller to adopt and these standards of fulfiling SR are great comprehensive business challenges.

Previous SR solution has some defectives.A this defective is need to be integrated with the software of special-purpose commercial graphic display system (for example, picture filing and transfer system or PACS) and other information systeies (for example, radiology information system (RIS) and/or electronic medical record (RMR)).Another defective of current method is that the radiologist will repeat mouse action and click the image discovery, and this can cause people's fatigue and complication of wrist.Another defective is that the radiologist can take sb's mind off sth, and comes marking image to find because they need to see to the report generation screen from image display screen to use from the cascade of drop-down menu set or from the item of the speech recognition with limited speech pattern.Equally, current method generally includes strides process that the link of a series of structured reports or connection layout picture find (namely, the process of difficulty in based on the situation of the report of literal), even and in the situation of computer based report scheme, also need a large amount of user interactions.

1. technical field

The present invention relates generally to the radiation field.More specifically, it relates to for device, system and method in conjunction with the senior multi-media architecture report of radiation image.The embodiment of the invention needing to can be used in other image-based fields that picture material and descriptive information be linked, for example, and dermatological, pathology, photography, satellite reflection, military target etc.

Summary of the invention

Each embodiment that is used for generating based on the method for multimedia structured report is described.In one embodiment, a kind of method comprises: catch and to be configured to the medical image that shows at medical image display device.The method also can comprise: catch the data of description relevant with medical image.In addition, the method can comprise: process medical image and the data of description relevant with medical image in data handling equipment.Equally, the method can comprise: medical image and the data of description relevant with medical image are stored in the data storage device.

In addition, method can comprise: create data association between the medical image in data storage device and the data of description relevant with medical image.For example, an embodiment can comprise: medical image is linked to patient's identifier.Equally, the method embodiment can comprise: medical image is linked to one or more medical images that link.In one embodiment, medical image and can link medical image and can link according to public verification.In another embodiment, from the medical image of difference check with can link medical image and can link according to linking criteria.In addition, medical image can be linked to accounting code.Persons of ordinary skill in the art will recognize that other data can advantageously be linked to medical image according to present embodiment.

In one embodiment, the method also can comprise: generate the synthetic medical report that comprises medical image.Synthetic medical report also can comprise linking and is linked at least one of medical image in the medical image.In one embodiment, medical image and each can link medical image and comprises that patient's whole radiation is historical.In other embodiments, result of the test, laboratory work result, case history etc. also can represent in this report.In one embodiment, synthetic medical report is arranged in form.This form can comprise at least a portion relevant with medical image of medical image and data of description.In another embodiment, synthetic medical report can be the figure report that comprises anthropometric dummy (homunculus).In another embodiment, synthetic medical report can be timeline.Timeline can comprise similarly that medical image and at least one can link medical image.

In one embodiment, medical image display device comprises picture filing and transfer system (PACS).

In one embodiment, data of description can comprise voice data, video data, literal etc.In addition, data of description can comprise the eye-tracking data.The eye-tracking data can comprise one or more eye gazes position and one or more eye gaze time of staying.In addition, data of description can comprise at least one that pointer position and pointer point hit.

Processing medical image can comprise: the medical image that automatically montage captures is with isolation diagnostic image component.Image through montage can be included in the synthetic medical report.In another embodiment, processing medical image can comprise: extract Word message by optical character recognition (OCR) utility from medical image; And the extraction literal that will be associated with medical image is stored in the data storage device.In addition, this processing can comprise: show to have the graphic user interface that graphical representation and data of description represent; And reception is for the user command that image and data of description are linked.In addition, graphic user interface can comprise timeline.Equally, can comprise at server processing image and data of description: at least one in the time of staying in response to eye gaze position and eye gaze automatically links image and data of description.For example, an embodiment can comprise: reach threshold value in response to the eye gaze time of staying of staring the position in particular eye, automatically trigger picture catching.

In one embodiment, the method can comprise: show translucent pop-up window, this translucent pop-up window shows the previous check discovery that is associated with the feature of medical image.

In another embodiment, processing medical image can comprise: to medical image operation image matching algorithm to generate the unique digital signature that is associated with medical image.Processing medical image also can comprise: the feature that quantizes medical image with automatic quantification tool.

Processing medical image also can comprise: but in response to a plurality of medical images that link that are linked to the medical image data of description that is associated with one or more concatenated image, automatically follow the tracks of progression of disease.In one embodiment, this processing comprises: in response to medical image and the data of description relevant with medical image, automatically computational entity tumor the standard of curative effect evaluation (RECIST) is worth.This processing also can comprise: in response to the feature of medical image and the data of description that is associated with medical image, automatically determine the disease phase.

In one embodiment, the data of description that is associated with medical image comprises the label that is associated with medical image.This label can be associated with the feature of medical image.In one embodiment, this label can be determined from the speech segment through isolation according to the natural language processing algorithm.This label also can be determined according to the optical character recognition that appears at the literal on the image.In another embodiment, this label can be determined according to the computer input that the user receives.

In another embodiment, the method can comprise: whether have the repetition medical image in the specified data memory device; Whether there are the data that are repeated in this description that are associated with medical image in the specified data memory device; And merge the repetition medical image and be repeated in this description data.

A kind of each embodiment that comprises the tangible computer program of the computer-readable medium with instruction, these instructions when carrying out so that computer is carried out the operation that is associated with above-described method step.For example, these operations can comprise: be received in the medical image that catches on the medical image display device; Receive the data of description relevant with medical image; Process medical image and the data of description relevant with medical image in data handling equipment; And medical image and the data of description relevant with medical image be stored in the data storage device.

Description comprises another embodiment of the tangible computer program of the computer-readable medium with instruction.In one embodiment, these operations of being carried out by computer can comprise: catch the medical image on the medical image display device; Catch the data of description relevant with medical image; And medical image and the data of description relevant with medical image be sent to treatment facility, this treatment facility is configured to process medical image and the data of description relevant with medical image in data handling equipment, and medical image and the data of description relevant with medical image are stored in the data storage device.

Same describe a kind of for each embodiment based on the device of multimedia structured report.An embodiment of this device can comprise: the interface that is configured to receive medical image and the data of description relevant with medical image.In addition, this device can comprise: be coupled to the treatment facility of interface, this treatment facility is configured to process medical image and the data of description relevant with medical image.This device also can comprise: be coupled to the data memory interface for the treatment of facility, this data memory interface is configured to store medical image and the data of description relevant with medical image.

In each embodiment, this device can comprise: be configured in response to being configured to so that this device RUGEN goes up one or more software definition modules that instruction that the tangible computer program of the described executable operations of method stores comes executable operations according to this.

Another device embodiment can comprise: the medical image display device that is configured to the display of medical image.Present embodiment also can comprise: be coupled to the picture catching utility of medical image display device, this picture catching utility is configured to catch medical image.In addition, this device can comprise: be configured to collect from the user interface apparatus of user's data of description.In one embodiment, this device also can comprise: the communication adapter that is coupled to image-capturing apparatus and user interface apparatus, this communication adapter is configured to medical image and the data of description relevant with medical image are sent to treatment facility, this treatment facility is configured to process medical image and the data of description relevant with medical image in data handling equipment, and medical image and the data of description relevant with medical image are stored in the data storage device.

In one embodiment, image-capturing apparatus can comprise the computer that is coupled to display device, and this computer has the operating system that is equipped with the screen capture function.In one embodiment, medical image display device can be picture filing and transfer system (PACS).For example, PACS can be dedicated system.An advantage of the embodiment of the invention is, image-capturing apparatus can catch the medical image from this special use medical image display in the direct integrated situation that does not need with special-purpose medical image display.In this, the embodiment of the invention can be pervasive, because it can directly not use with this dedicated system in the integrated situation with any dedicated system.This advantage has greatly reduced cost and the complexity of the embodiment of the invention, and the standardized report platform of making peace is provided more.

In one embodiment, user interface apparatus can comprise eye-tracking equipment.User interface apparatus can be video camera.In another embodiment, user interface apparatus can be voice recording equipment.For example, voice recording equipment can be to have the oral account equipment that triggers assembly.

In other embodiments, this device can comprise: the one or more software definition modules that are configured to come in response to the instruction that tangible computer program is stored executable operations.In such an embodiment, operation can comprise: catch the medical image on the medical image display device; Catch the data of description relevant with medical image; And medical image and the data of description relevant with medical image be sent to treatment facility, this treatment facility is configured to process medical image and the data of description relevant with medical image in data handling equipment, and medical image and the data of description relevant with medical image are stored in the data storage device.

Each embodiment that also presents a kind of system.An embodiment can comprise server, data storage device and medical image viewer.In one embodiment, server can comprise the interface that is configured to receive medical image and the data of description relevant with medical image.Server also can comprise the treatment facility that is coupled to this interface, and this treatment facility is configured to process medical image and the data of description relevant with medical image.In addition, server can comprise the data memory interface that is coupled to treatment facility, and this data memory interface is configured to store medical image and the data of description relevant with medical image.

Data storage device can be coupled to data memory interface.In one embodiment, data storage device can be configured to receive and store medical image and the data of description relevant with medical image.

In one embodiment, the medical image viewer can be coupled at least one in server and the data storage device.The medical image viewer can comprise the medical image display device that is configured to the display of medical image.The medical image viewer also can comprise the picture catching utility that is coupled to medical image display device, and this picture catching utility is configured to catch medical image.For example, the picture catching utility can comprise Microsoft

The screen capture function of operating system.The medical image viewer also can comprise the user interface apparatus that is configured to collect from user's data of description.In addition, the medical image viewer also can comprise the communication adapter that is coupled to image-capturing apparatus and user interface apparatus, and this communication adapter is configured to medical image and the data of description relevant with medical image are sent to server.

In each embodiment, this system can comprise: be configured to the one or more software definition modules according to each embodiment executable operations of above-described method.

In one embodiment, this system can comprise X-ray production apparatus.This medical imaging devices can be computer tomography (CT) scanner.This medical imaging devices can be nuclear magnetic resonance (MRI) machine.Alternatively, this medical imaging devices can be supersonic imaging apparatus.Persons of ordinary skill in the art will recognize that various medical imaging devices can be combined with the present embodiment of device, system and method.

In one embodiment, this system can comprise: the PACS server that is configured to receive the DICOM data that represent medical image.This system also can comprise: be coupled to the PACS data storage device of PACS server, this PACS data storage device is configured to store the view data of expression medical image.

This system also can comprise: be configured to reception server in response to the report viewer based on the report of media that medical image and the data of description relevant with medical image generate, should comprise based on report of media the patient of single figure in observing whole radiation history.

Term " coupling " is defined as needn't be directly and mechanically " connection ".

Term " link " is defined as by or connects to form relation via intermediate module.For example, chained list can have the metadata of one group of data link to another group data, wherein this metadata creation logical relation.Equally, two computers can link by cable.

Term " one " and " one " are defined as one or more, unless present disclosure clearly indicates in addition.

Term " basically " and variant thereof be defined as those of ordinary skills understand and appointment mainly but need not to be fully, and in a non-limiting example, " basically " refer to appointment 10% in, in preferred 5%, the more preferably scope in 0.5% in 1% and most preferably.

Term " comprises " (and any form that comprises, such as " comprising " and " including "), " having " (and any form that has, such as " having " and " possessing ") and " comprising " (and any form that comprises, such as " comprising " and " containing ") be open catenative verb.Thus, method or the equipment of " comprising ", " having ", " comprising " or " containing " one or more steps or element have this one or more steps or element, but are not limited to only have these one or more elements.Equally, method or the element of " comprising ", " having ", " comprising " or " containing " one or more features have this one or more features, but are not limited to only have these one or more features.In addition, the equipment or the structure that dispose with ad hoc fashion dispose in this mode at least, but also can unlisted mode dispose.

Reference is by reference to the accompanying drawings to the following detailed description of specific embodiment, and other features and the advantage that is associated will become apparent.

The accompanying drawing summary

Accompanying drawing consists of the part of this description and is included, further to illustrate the particular aspects of present embodiment.These embodiment can be by understanding with reference to the one or more accompanying drawing in conjunction with the detailed description of the specific embodiment that presents herein better.

Fig. 1 is the schematic block diagram that illustrates for an embodiment of the system of senior multi-media architecture report.

Fig. 2 is the schematic block diagram that an embodiment of medical image observer system is shown.

Fig. 3 is the schematic block diagram that an embodiment of computer system is shown.

Fig. 4 is the schematic block diagram that illustrates for an embodiment of the client computer of senior multi-media architecture report.

Fig. 5 is the schematic block diagram that an embodiment of senior multimedia reports server is shown.

Fig. 6 is the schematic block diagram that another embodiment of senior multimedia reports server is shown.

Fig. 7 is the indicative flowchart that illustrates for an embodiment of the method for senior multi-media architecture report.

Fig. 8 is the indicative flowchart that illustrates for another embodiment of the method for senior multi-media architecture report.

Fig. 9 is the axonometric chart of an embodiment of speech capture device.

Figure 10 is for the logic chart for an embodiment of the method for using at synthetic medical report of montage medical image automatically.

Figure 11 is the logic chart for an embodiment of the method that generates synthetic medical report.

Figure 12 catches medical image and store medical image for the logic chart of an embodiment of the method for using in synthetic medical report.

Figure 13 is link medical image and the logic chart of finding with an embodiment who forms the method for synthesizing medical report.

Figure 14 is the screenshot capture of an embodiment of the synthetic medical report of tabulation figure.

Figure 15 is the screenshot capture of an embodiment of the anthropometric dummy figure of synthetic medical report.

Figure 16 is the screenshot capture of another embodiment of the anthropometric dummy figure of synthetic medical report.

Figure 17 is the logic chart that other embodiment of the synthetic report that comprises timeline and image metric are shown.

Figure 18 A is the curve chart of RECIST result's a embodiment.

Figure 18 B is the curve chart of an embodiment of RECIST percentage ratio result of variations.

Figure 19 is the screenshot capture of an embodiment that comprises the figure RECIST result of the image that catches according to present embodiment.

Figure 20 A is the screenshot capture with embodiment of the tabulation figure report that is labeled as urgent discovery.

Figure 20 B is the front view that has for the mobile device of the application that receives the emergency notice corresponding with the urgent discovery shown in Figure 20 A.

Figure 21 A is the schematic block diagram that is suitable for an embodiment of the Arithmetic of Eye-tracking System that uses with present embodiment.

Figure 21 B is the diagram of figure and the eye-tracking data that are associated.

Figure 21 C is the logical expressions for an embodiment of the method that the seizure medical image is associated with the label of deriving from the speech segment through isolation by natural language processing.

The description of illustrative embodiment

More fully explain each feature and favourable details with reference to non-limiting example shown in the drawings and that describe in detail in the following description.Omit the description of known raw material, treatment technology, assembly and equipment, in order to avoid aspect details, unnecessarily obscure the present invention.Yet, should be appreciated that detailed description when specifying each embodiment of the present invention and concrete example are just as example but not provide as restriction.According to the disclosure, the various replacements in the spirit of following inventive concept and/or the scope, modification, interpolation and/or rearrangement will become apparent those skilled in the art.

Discrete cell described in this description has been noted as module, more specifically to emphasize the independence of its realization.Module is " with separate hardware or the component software of larger system interaction " (Alan Freedman, " computerese table (The Computer Glossary) " 268(the 8th edition, 1998)).Module comprises machine or machine-executable instruction.For example, module can be implemented as the hardware circuit that comprises self-defined VLSI circuit or gate array, ready-made quasiconductor (such as logic chip, transistor or other discrete assemblies).Module also can realize in programmable hardware devices such as field programmable gate array, programmable logic array, programmable logic device.

When module also can be included in by handling machine or equipment execution the data of storing on the data storage device are transformed into software definition unit or the instruction of the second state from the first state.For example, the identification module of executable code can comprise one or more physics or the logical block of the computer instruction that can be identified as object, process or function.Yet, but the executive item of identification module does not need to be positioned at physically together, and can comprise the different instruction that is stored in the diverse location, these instructions comprise this module and realize stating when logically combining data conversion when being carried out by processor.

In fact, the module of executable code can be single instruction, MIMD perhaps, and even can and stride some memory devices on some different code segments, between distinct program and distribute.Similarly, operating data can be in this article in module sign or illustrate, and can be presented as any suitable form and in the data structure inner tissue of adequate types.Operating data can be collected as the individual data collection, and can distribute at the diverse location that comprises different storage device.

In the following description, provide a large amount of details (such as the example of programming, software module, user selection, network trading, data base querying, database structure, hardware module, hardware circuit, hardware chip etc.) that the thorough understanding to present embodiment is provided.Yet, those skilled in the art will recognize that the present invention can put into practice in the situation of neither one or a plurality of details or by additive method, assembly, material etc.In other examples, be not shown specifically or describe known structure, material or operation in order to avoid obscure aspects of the present invention.

Fig. 1 illustrates an embodiment for the system 100 of senior multi-media architecture report.System 100 can comprise server 114, data storage device 116 and medical image viewer 112.In additional embodiment, system 100 can comprise medical imaging devices 102 and Medical Image Processing equipment 104.Medical imaging devices 102 can generate medical image, and this medical image is sent to Medical Image Processing equipment 104 for further processing.In a particular embodiment, medical image can format according to professional format scheme or industry standard format scheme (such as, digital imaging and communications in medicine (DICOM)).Persons of ordinary skill in the art will recognize that various format schemes can use in conjunction with the embodiment of the invention.

Comprise among the embodiment of PACS 112 that in system 100 system 100 also can comprise the PACS server 108 that is configured to receive the view data that represents medical image.System 100 also can comprise the PACS data storage device 110 that is coupled to PACS server 108, and this PACS data storage device 110 is configured to store the view data of expression medical image.In one embodiment, each in each assembly of system 100 can be coupled by network 106.For example, network 106 can comprise Local Area Network, wide area network (WAN), storage area network (SAN), individual territory net (PAN) and the Internet separately or with various combinations.

In one embodiment, medical image viewer 112 can be coupled at least one in server 114 and the data storage device 116.Medical image viewer 112 can comprise the medical image display device 112 that is configured to the display of medical image.For example, Fig. 2 illustrates an embodiment of medical image viewer 112.In one embodiment, medical image viewer 112 can comprise a PACS viewer 204, the 2nd PACS viewer 206, RIS display 202 and treatment facility 208.Medical image viewer 112 also can comprise one or more user interface apparatus, and it comprises mouse pointer 210, voice recording equipment 212, the video capturing device (not shown) such as video camera or web camera, the eye-tracking equipment shown in Figure 21 A etc.User interface apparatus can be collected from user's iamge description data.For example, the radiation image on radiologist's observable the one PACS viewer 204, and give an oral account his discovery on speech recording apparatus 212.

Fig. 9 illustrates an embodiment of the speech recording apparatus 212 that can use according to present embodiment.Particularly, this speech recording apparatus can comprise microphone 1202 for the recording of voice data, be used for playback speech segment speaker 1204, be used for connecting by interface trigger button 1206, client computer 400 and/or the treatment facility 208 of PACS.

Medical image viewer 112 also can comprise the treatment facility 208 such as computer.Further describe such as Fig. 4, picture catching utility 406 can be coupled to medical image display device 112.For example, picture catching utility 406 can be to be configured in treatment facility 208 operation and to be configured to catch from least one the software client 400 of medical image in a PACS viewer 204 and the 2nd PACS viewer 06.An embodiment of client computer 400 is shown in Figure 4.Alternatively, picture catching utility 406 can be independent equipment or the computer that is configured to be connected by interface and catch with medical image viewer 112 copy of medical image or medical image.In one embodiment, picture catching utility 406 can comprise the Microsoft for the treatment of facility 208 The screen capture function of operating system or be coupled to another computer of medical image viewer 112.The advantage of these embodiment is that client computer 400 does not need to install or be directly integrated with PACS viewer 204,206.Therefore, the embodiment of the invention can be used for catching the image from any medical image viewer, no matter manufacturer, model or special-purpose how the requirement.Thus, the embodiment of the invention can be platform independence.

In addition, medical image viewer 112 also can comprise the communication adapter 314 that is coupled to picture catching utility 406 and user interface apparatus 212, and this communication adapter 314 can be sent to server 114 with medical image and the data of description relevant with medical image.

Fig. 3 illustrates according to the specific embodiment of each server 108,114, treatment facility 208 and/or according to the report viewer 118 of the embodiment of the invention and adaptive computer system 300.CPU (CPU) 302 is coupled to system bus 304.CPU 302 can be universal cpu or microprocessor.The embodiment of the invention is not subjected to the restriction of the architecture of CPU 302, as long as CPU 302 supports as described herein module and operation.CPU 302 can carry out various logical instructions according to the embodiment of the invention.For example, CPU 302 can be according to carrying out machine level instruction below with reference to Fig. 7 and 8 described exemplary operations.

Computer system 300 also can comprise random-access memory (ram) 308, and this RAM 308 can be SRAM, DRAM, SDRAM etc.Computer system 300 can utilize RAM 308 to come stored configuration to become to generate the employed various data structures of software application of the synthetic report of patient medical record.Computer system 300 also can comprise read only memory (ROM) 306, and this ROM 306 can be PROM, EPROM, EEPROM, optical storage etc.This ROM can store the configuration information for booting computer system 300.RAM 308 and ROM 306 preserve the data of user and system 100.

Computer system 300 also can comprise I/O (I/O) adapter 310, communication adapter 314, user interface adapter 316 and display adapter 322.In a particular embodiment, I/O adapter 310 and/or user interface adapter 316 can make the user can be mutual with computer system 300, thereby input message is with the input data of description relevant with medical image and other discoveries that are associated with check.In another embodiment, display adapter 322 can demonstration and software or the graphic user interface that is associated based on the application of web, for transmission metric, to image classification etc.

I/O adapter 310 can be with in one or more memory device 312(such as hard drive, compact disk (CD) driving, disk drive, the magnetic tape drive one or more) be connected to computer system 300.Communication adapter 314 can be suitable for computer system 300 is coupled to network 106, and this network 106 can be one or more in LAN and/or WAN and/or the Internet.User interface adapter 316 will be coupled to computer system 300 such as the user input device in keyboard 320 and the pointing device 318.Display adapter 322 can drive to be controlled at demonstration on the display device 324 by CPU 302.

The embodiment of the invention is not limited to the architecture of system 300.On the contrary, computer system 300 is set to be suitable for to carry out the example of a class computing equipment of the function of server 102 and/or user interface apparatus 110.For example, can use any suitable equipment based on processor of the server that includes but not limited to PDA(Personal Digital Assistant), tablet PC, game console and multiprocessor.In addition, the embodiment of the invention can realize at special IC (ASIC) or great scale integrated (VLSI) circuit.In fact, those of ordinary skills can use can be according to the suitable construction of any amount of described embodiment actuating logic operation.

In all each embodiment as shown in Figure 5, server 114 can comprise the interface that is configured to receive medical image and the data of description relevant with medical image such as receiver 502.Server 114 also can comprise the data processor 506 that is coupled to receiver 502, and this data processor 506 can be configured to process medical image and the data of description relevant with medical image.In addition, server 114 can comprise the data memory interface 512 that is coupled to data processor 506.Data memory interface 512 can be configured to medical image and the data of description relevant with medical image are stored in the data storage device 116.

Data storage device 116 can be coupled to data memory interface 512.In one embodiment, data storage device 116 can be configured to receive and store medical image and the data of description relevant with medical image.For example, data storage device 116 can comprise the one or more data storage mediums according to the database scheme configuration.The data base can be configured to according to logical data association store medical image and data of description.For example, a plurality of medical images can be according to public verification or according to another linking criteria link.For example, if obtain a plurality of images from identical testing data, then they can link.These images can be linked to the image discovery that the Medical Technologist such as the radiologist records.In another embodiment, image and the data of description from the first check can be linked to image and the data of description of checking from second.For example, such link can be used for progression of disease analysis, RECIST calculating etc.

In one embodiment, system 100 can comprise medical imaging devices 102.For example, this medical imaging devices can be X-ray production apparatus.This medical imaging devices can be computer tomography (CT) scanner.This medical imaging devices can be radio frequency (RF) imaging device.This medical imaging devices can be nuclear magnetic resonance (MRI) machine.Alternatively, this medical imaging devices can be supersonic imaging apparatus.Persons of ordinary skill in the art will recognize that various medical imaging devices can be combined with the present embodiment of device, system and method.

System 100 also can comprise and be configured to reception server 114 in response to the report viewer 118 based on the report of media that medical image and the data of description relevant with medical image generate, and should comprise based on report of media the patient of single figure in observing whole radiation history.In a particular embodiment, the report viewer can be tablet PC for example.Tablet PC can be configured to operational report and use.For example, the report application can be can be by the application based on web of reporting that viewer visits by sign in to server 114 via the Internet.Alternatively, report is used can be installed on the report viewer 118 and is used as local.In each embodiment, the report viewer can be desk computer, laptop computer, tablet PC or PDA.Persons of ordinary skill in the art will recognize that various suitable hardware platforms can be configured to report viewer 118.

In one embodiment, system 100 can comprise the client-server configuration.For example, as described in Figure 4 client computer 400 can be installed on the treatment facility 208.In such an embodiment, client computer 400 can comprise input interface 402, authentication module 404, picture catching utility 406 and transmitter 414.In addition, client computer 400 can comprise that speech catches at least one in utility 408, video capture utility 410 and the input capture utility 412.

Server 114 can dispose according to the described embodiment of Fig. 5.For example, server 114 can comprise receiver 502, authentication module 504, data processor 506, Report Builder 508, find linker 510, data memory interface 512 and transmitter 514.

In one embodiment, the patient can receive the check from CT scanner 102, as shown in Figure 1.View data from CT scan can be sent to image processing equipment 104.Then, image processing equipment 104 can be sent to PACS server 108 by network 106 with view data.Then, PACS server 108 can be stored in view data in the PACS data storage device 110.

Then, the addressable PACS viewer 112 of the Medical Technologist such as the radiologist.Then, the radiologist can sign in to client computer 400 by the authentication module 404 transmission Service Ticket (such as username and password) to client computer 400.The radiologist also can send Service Ticket by the authentication module 504 to server 114 and sign in to senior multimedia server 114.

Patient's record on the addressable RIS display 202 of radiologist, and to PACS server 108 requested image data.Then, PACS server 108 can be sent to a PACS viewer 204 by network 106 with view data.Then, the radiologist can catch with picture catching utility 406 copy of the medical image that shows at a PACS viewer 112.For example, the radiologist can click trigger or the functional keys that is integrated on the voice recording equipment 212.The radiologist also can record to be transfused to respectively with mouse pointer 210, voice recording equipment 212, video capturing device (not shown) etc. and catch that utility 412, speech catch speech information that utility 408 and video capture utility 410 catch and about other data of description of medical image.

Then, client computer 400 can be sent to server 114 with medical image and data of description by transmitter 414.Receiver 502 on the server 114 can receive medical image and data of description.If need further to process, then data processor 506 can automatically be processed medical image and data of description.Medical image and data of description also can be by finding that linker 510 is linked to other discoveries.Data memory interface 512 can be stored in medical image and data of description in the data storage device 116.Medical image and data of description can be passed through the links such as patient's identifier, test number, record number.

Then, but user's operation report viewer 118 to the synthetic medical report of server 114 request.Receiver 502 can receive the report request.For example, in one embodiment, receiver 502 can receive the web request of passing through the Internet 106 access servers 114 from report viewer 118.Then, Report Builder 508 can be according to the request of parameter generating database or the inquiry of report request.Parameter can comprise patient's identification information, link parameter etc.Then, data memory interface 512 can be retrieved the information of asking from data storage device.Then, Report Builder can generate synthetic medical report.This report can be tabulation figure report or the report of the anthropometric dummy pattern shown in Figure 18-19 as shown in figure 14.Then, transmitter 514 can be sent to report viewer 118 with this report 6 by the Internet 10, for presenting.

Fig. 6 shows the another embodiment of server 114.As described above with reference to Figure 5, server 114 can comprise receiver 502, authentication module 504, data processor 506, Report Builder 508, find linker 510, data memory interface 512 and transmitter 514.

In one embodiment, find to create data association between medical image that linker 510 can be in data storage device 116 and the data of description relevant with medical image.For example, find that linker 510 can be linked to medical image patient's identifier.Equally, find that linker can be linked to medical image one or more medical images that link.In one embodiment, medical image and can link medical image and can link according to public verification.In another embodiment, from the medical image of difference check with can link medical image and can link according to linking criteria.In addition, medical image can be linked to accounting code.Persons of ordinary skill in the art will recognize that other data can advantageously be linked to medical image according to the embodiment of the invention.

In another embodiment, data processor 506 can comprise image clips device 602, image tag machine 604, RECIST computer 614, disease tracking utility 616, staging utility 618 and repeat to merge utility 620.In one embodiment, data processor 506 can be CPU 302 as described in Figure 3.Data processor 506 can be coupled to receiver 502.Data processor 506 generally can be processed medical image and the data of description relevant with medical image.

For example, data processor 506 can comprise image clips device 602.Image clips device 602 automatically the montage medical image with isolation diagnostic image component.In alternative embodiment, image clips device 602 can be integrated with client computer 400.Figure 10 illustrates an embodiment of the function of image clips device 602.In one embodiment, image clips device 602 can come montage to catch the medical image that utility 406 catches with the image coordinate of hard coded.For example,

The PACS system or

The PACS system can comprise the known pixels coordinate system.Image clips device 602 can be short in the subset of PACS pixel so that image is cut out by hard coded.The optical imagery coordinate can change according to the brand of PACS or 3D work station and according to image layout.In another embodiment, graphic user interface (GUI) instrument can be configured to allow the manager by the rubber belt frame (rubber-band box) of drawing for the particular station configuration montage (croppy) coordinate to be set.As shown in figure 10, rubber belt frame can be regulated by the user.Then, can be stored in the data storage device through the image of montage, for such as synthetic report based on multimedia report in use.

In one embodiment, image tag machine 604 can comprise one or more in natural language processing device 606, optical character recognition (OCR) utility 608, user's input processor 610 or the database link utility 612.Generally speaking, image tag machine 604 can comprise for the utility that data of description is added to the image that picture catching utility 406 catches.Add data of description and can comprise the new data of description of collecting from medical expert (such as the radiologist).In another embodiment, add data of description and can comprise seizure, transmit or otherwise obtain existing data of description, and this data of description is associated with the medical image that catches.

For example, image tag machine 604 can comprise natural language processing device 606.Figure 21 C illustrates an embodiment of the method that links for the data of description that will catch in the speech segment through isolation and medical image.Natural language processing module 606 solves medical expert's public workflow issues.For example, the radiologist can check the first image, and identifies the marked feature in the first image.Then, when describing this marked feature, the radiologist can be scanned the second image simultaneously to identify the second marked feature.In one embodiment, the radiologist can catch utility 408 with speech and comes the recording of voice segment.Then, natural language processing device 606 can use public speech recognition program that speech is transcribed into literal.Then, but natural language processing device 606 scan texts are described the tolerance of this feature with sign, perhaps can identify keyword and equivalent.For example, some keywords can comprise " stable ", " not changing ", " through what improve ", " deterioration " etc.For example, natural language processing can be used for sign and distributes dissection, pathology and priority feature.For example, the radiologist who observes the CT image of lung can state " this image comprises the vegetation that needs urgent attention in the left lung ".Natural language processing device 606 can identify keyword " lung ", " vegetation " and " promptly ", and correspondingly distributes dissection, pathology and precedence field.

In one embodiment, image tag machine 604 can comprise OCR utility 608.But the medical image that OCR utility 608 scanograms seizure utility 406 catches is to identify the literal in present this image.In one embodiment, can scan whole medical image.Alternatively, can scan the known interested specific region that comprises literal.In another embodiment, the OCR for using image to process can strengthen this literal.OCR utility 608 also can automatically be determined to specific what literal of data of description field distribution.For example, OCR utility 608 can automatically identify name, medical records number, data, time, picture position of patient etc.OCR utility 608 determined literal can be stored in the data storage device 116.

In one embodiment, image tag machine 604 can comprise user's input processor 610.User's input processor 610 can generate and allow user selection to one or more menus of the label of medical image distribution.For example, this menu can be cascade menu, combobox menu, text selecting frame etc.In another embodiment, this menu can comprise one or more text entry fields.For example, can distribute the one or more tolerance that limit the feature sizes in the medical image with text entry field.In another embodiment, can dissect field, pathology field, precedence field etc. with distributing such as the cascade menu of selecting.Each selection can be filled the next stage of cascade menu, thereby provides the relevant additional aggregates of selecting to the user.

In one embodiment, shown in Figure 21 A-C, user's input processor 610 can receive and process the eye-tracking data.At an embodiment of Arithmetic of Eye-tracking System shown in Figure 21 A.The user can stare its of ad-hoc location special time amount that keeps.The eye-tracking camera can be followed the tracks of the eye gaze position, and makes those positions relevant with the part of medical image.For example, Figure 21 B illustrates an embodiment of the determined eye gaze of the eye-tracking equipment position of Figure 21 A.Except the eye-tracking position, user's input processor 610 can be followed the tracks of the change sequential of the eye gaze position shown in Figure 21 C.In a particular embodiment, user's input processor 610 and natural language processing device 606 can be in conjunction with work with the characteristic allocation labels to the indicated medical image in eye gaze position.An one embodiment is shown in Figure 21 C.In one embodiment, the eye gaze positional information isolation that the speech segment can be collected with eye-tracking equipment.In such an embodiment, can be according to time series analysis speech segment, and can be according to time series analysis eye gaze positional information.

Different from common eye-tracking technology, the embodiment of the invention comprises being associated from the key images of the information content of radiologist's language description (and intrinsic medical importance of this information content) and the certain significance level of image that gives to catch.In radiologist's typical workflow, the long time of staying can appear at the radiologist and check when intricate but final unessential image is found, checks more obvious important discovery but the radiologist can spend the less time.Compared with prior art, information content and linking of key images provide the more accurate means to the specific image apportioning cost.

In another embodiment, independent eye-tracking module can include client computer 400.In another embodiment, when the user remained on its eye gaze position ad-hoc location and reaches the persistent period that reaches predetermined threshold, this event can automatically trigger picture catching.

In another embodiment, image tag machine 604 can comprise database link utility 612.For example, the data of description with the primitive medicine image correlation that shows at a for example PACS viewer 204 can be stored in PACS data storage device 110.In one embodiment, data of description can be by database link utility 601 automatically from 110 retrievals of PACS data storage device.In another embodiment, the medical image and the data of description that are stored in the data storage device 116 can be stored among the independent data base based on such as dissection, mode etc.In one embodiment, database link utility 612 can make index of reference or critical field link or retrieve information from a plurality of data bases.For example, all images and the data of description relevant with patient's name, patient ID etc. can link and retrieve by database link utility 612.

In another embodiment, RECIST computer 614 can automatically be carried out RECIST calculating.For example, Figure 18 A-21C illustrates the sampled result of RECIST computer 614.In one embodiment, RECIST computer 614 can when the cancer patient improves (" response "), keeps the publication rule of identical (" stablizing ") or deterioration (" progress ") to come result of calculation during treating according to defining.RECIST computer 614 can measure evaluation based on the tumor that comprises in the data of description.In another embodiment, RECIST computer 628 can generate based on the result that RCIST computer 614 calculates expression tumor level of response shown in Figure 18 A-B or the curve of percentage ratio change level.In another embodiment, RECIST computer 628 can calculate to generate the RECIST report based on RECIST computer 614 performed RECIST, and this RECIST report can comprise the link medical image that the picture catching utility 406 shown in Figure 21 C catches.

In each embodiment, server 114 also can comprise disease tracking utility 616 and staging utility 618.The RECIST value that RECIST computer 614 generates can be used for disease and follows the tracks of and staging.In a particular embodiment, the staging report can be generated by staging utility 618.The disease phase can comprise stadium 0, stadium 1, stadium 2, stadium 3, stadium 4 and recurrence.For example, if patient diagnosis suffers from colon cancer, then the stadium of cancer can automatically be determined in response to data of description by staging utility 618.In this example, stadium 0 can indicate cancer only to find in colon or the innermost lining of rectum.Stadium 1 can indicate tumor long in the inwall of colon or rectum.Tumor does not also grow up to passes this wall.Stadium 2 can be indicated tumor deeper to expand in the wall of colon or rectum or be passed this wall, perhaps indicate near its tissue having attacked, but cancer cell also is not diffused into lymph node.Stadium 3 can indicate cancer to be diffused near lymph node, but also is not diffused into other parts of health.Stadium 4 can indicate cancer to be diffused into other parts of health, such as liver or lung.It is the cancer of having treated and having reappeared after a period of time that may not detect cancer that recurrence can be indicated this, and the indication disease can be reappeared in colon or rectum or in another part of health.The criterion of the corresponding stadium of the cancer of these stadium and other types is determined by American National health association.Disease tracking module 616 can be measured the progress of automatically following the tracks of disease with other that comprise in by stages information, RECIST information and the data of description.The forms such as disease tracking module 616 can curve, form, timeline are followed the tracks of disease.

Repeating to merge utility 620 can merge and repeat to find.When at the upper identification discovery of an above image sequence (for example, the delay phase of the CT scan of tremulous pulse, vein and imaging), the discovery through merging is useful.In one embodiment, merging utility 620 can automatically detect by a stack features of analyzing each medical image and repeat to find.Alternatively, repeating to merge utility 620 can provide user interface to find to merge to allow the user manually to select to repeat.

In one embodiment, Report Builder 508 can comprise tabulation diagram generator 622, anthropometric dummy diagram generator 624, timeline maker 626, RECIST Report Builder 628 and emergency notice maker 630.Generally speaking, the database retrieval that medical image and the data of description that is associated with medical image can be from data storage devices 116 is to generate one or more in tabulation figure report, anthropometric dummy figure report, timeline report, the RECIST report etc.In a particular embodiment, tabulation figure report and/or anthropometric dummy figure report can be synthetic reports.Synthetic report can be the congeries that all images are found, wherein (for example be linked to the existing specific image discovery of finding by illustrating, stacking picture appearance) designator is found to show in (in the anthropometric dummy report) on the concrete anatomical position or in anatomical categories (in the report of tabulation type) from the nearest image of any mode.This finds that with comprising the image of dissecting (the chest x ray that for example, obtains in specific date and time) about concrete mode/date/time/imaging the Routine report of tabulation is different.Yet, according to the data base that the image that is stored on the data base 116 is found, can filter out the discovery subset that is equivalent to the conventional planning report about the discovery of concrete check with establishment.

Figure 14 illustrates an embodiment of composition-seies chart report.As shown in figure 14, tabulation figure report can form present.Tabulation figure report can comprise one or more medical image thumbnails.This report can be assigned to dissection, pathology, time or any other criterion that tabulation figure reports maker 622 according to the user and organize.In the embodiment of Figure 14, tabulation figure report comprises that the thumbnail image, the indication in orientation, anatomical position, pathology designator, priority indicator, characteristic measure, change designator (following the tracks of utility 616 such as disease generates), the medical expert that find classification, medical image describe the literal that the video of discovery or audio frequency, medical expert find and transcribe and the additional designator of supporting image.Certainly, persons of ordinary skill in the art will recognize that more or less field can be included in the tabulation figure report.

Figure 15 illustrates an embodiment of the anthropometric dummy figure report that anthropometric dummy diagram generator 624 generates.Figure 16 illustrates alternative embodiment.Those of ordinary skill will be recognized the many different embodiment of anthropometric dummy and anthropometric dummy figure report.In an embodiment of the anthropometric dummy figure of Figure 18 and 19 report, recent findings can appear on the anthropometric dummy and dissect in the relevant position with patient's physics.In one embodiment, if there be the additional discovery relevant with the dissection of recent findings, then exist the additional designator of finding to appear in the anthropometric dummy report.For example, shown in Figure 18 and 19, a plurality of discoveries can show as stacking image.Alternatively, frame, star or other designators can be indicated and be had additional the discovery.Then, the user can click at the thumbnail of finding, and about finding or the additional additional information of finding can appear in the new observation panel or in same observation panel.

As shown in figure 17, timeline maker 626 can generate the timeline of these images.In one embodiment, timeline maker 626 can generate and comprise from the image of a plurality of different modalities and the disease timeline of discovery.For example, the disease timeline can be included in the link of CT discovery, ultrasonic discovery, laboratory discovery etc.In one embodiment, these links can comprise the thumbnail image corresponding with medical image.

Additional information can be included in the detailed view shown in Figure 17.For example, detailed view can comprise other included information in characteristic measure, curve, RECIST information, disease stadium information, disease trace information and the data of description.

In one embodiment, Report Builder 508 can comprise emergency notice maker 630.Have the definite of urgent priority in response to finding, emergency notice maker 630 can automatically generate the notice to for example medical expert.For example, the radiologist can check that abdominal CT is to determine whether the patient has appendicitis and patient's vermiform appendix whether to be in the danger of breaking.If radiologist's precedence field is set to promptly, then emergency notice maker 630 can notify consultant internist, surgeon, operation room work personnel etc. to need urgent attention.Emergency notice maker 630 can generate automatic call, paging, Email, text message etc.In another embodiment, emergency notice maker 630 can be carried in mobile device on mobile the application be connected by interface.For example, shown in Figure 20 A and 20B, when precedence field is set as when urgent, but the mobile trigger notice of using on the remote-moving apparatus.In one embodiment, this notice can comprise medical image copy, priority designator and in order to monitor the link of the audio or video that the radiologist finds.

Following exemplary process diagram generally is described as logical flow chart.An embodiment of the step indication institute rendering method of the order of describing thus, and institute's labelling.Can be envisaged in function, logic or effect aspect be equivalent to shown in one or more steps of method or other step and methods of its part.In addition, the form that adopts and symbol are configured to explain the logic step of the method, and are understood to not limit the scope of the method.Although can adopt various arrow types and line type in flow chart, it is understood to not limit the scope of correlation method.In fact, some arrows or other connectors can be used for only indicating the logic flow of the method.For example, arrow can be indicated the wait of not specifying the persistent period or the Looking Out Time section between the step enumerated at institute's plotting method.The order of the corresponding steps shown in addition, the order of ad hoc approach appearance may or may strictly not followed.

Fig. 7 illustrates an embodiment for the method 700 that generates synthetic medical report.In one embodiment, when picture catching utility 406 catches the medical image (702) that is configured in medical image display device 112 demonstrations, method 700 beginnings.In one embodiment, picture catching utility 406 is duplicated in the image that shows on the commercially available PACS viewer 204.For example, picture catching utility 406 can comprise the screen capture function.Then, speech catches utility 408, video capture utility 410 and input capture utility 412 and can catch the data of description relevant with medical image (704).For example, speech catches utility 408 and can catch the speech segment that the medical expert gives an oral account discovery.Video capture utility 410 can comprise and is configured to catch the web-camera (not shown) that the medical expert describes the videograph of discovery.The input capture utility can comprise eye-tracking data, menu setecting, text input etc.In addition, method 700 can be included in upper medical image and the data of description relevant with medical image processed of data handling equipment (such as server 114).Particularly, the data processor 506 on the server 114 can be processed medical image and data of description.Equally, method 700 can comprise medical image and the data of description relevant with medical image are stored in (708) in the data storage device 116.For example, data memory interface 512 can be stored in medical image and data of description in the data storage device 116.

Another embodiment of method 800 describes in Fig. 8.Method 800 can begin when the user accesses PACS viewer (802).Then, the senior multimedia reports client computer of user-accessible (804).For example, the user can sign in to client computer 400 by voucher being sent to authentication module 404.Then, the user can be chosen in the upper patient (806) who observes of PACS.For example, the user can select the patient in the RIS system 202.Then, the senior multimedia reports server of user-accessible 114(808).Then, the user can trigger picture catching utility 406 on the client computer to catch the copy (702) of the image that shows at PACS viewer 204.This screen capture 702 can with the cooperation of any image viewing platform, and may not need integrated with the PACS viewer.For example, the user can use trigger or the function of oral account equipment 212, such as

Speechmike.Alternatively, the user can trigger seizure by the click of mouse 210 or the keystroke on the keyboard.Then, speech catches the one or more data of description that are associated with medical image that catch in utility 408, video capture utility 410 and the input capture utility 412.This process is generally shown in Figure 11.

Can use transmitter 414 that medical image and the data of description that is associated are sent to server 114, as shown in figure 12.Server 114 can be processed medical image and data of description (706), described at above embodiment.For example, data of description can further be generated or refinement by OCR utility 608, natural language processing device 606 and user's input processor 610.Then data memory interface 512 can be stored in (708) in the data storage device 116 with medical image and the data of description relevant with medical image.In another embodiment, but find that the chain field etc. in linker 510 based on databases is linked to other medical images and data of description with medical image and data of description.This process is generally shown in Figure 13.

Then, the second user can be to server 114 request reports.For example, the second user can send to server 114 via report viewer 118 with the request to the synthetic report that is associated with selected patient.Server 114 can receive the request (810) to synthetic report, and Report Builder 508 can generate synthetic report (812) from the data base's who is stored in medical image on the data storage device 116 and data of description medical image and data of description by access.Then, transmitter 514 can will synthesize report by network 106 and send to report viewer 118(814).This synthetic report can be tabulation figure report or the report of the anthropometric dummy figure shown in Figure 15-16 as shown in figure 14.In response to the image thumbnails of clicking in the synthetic report, the report viewer can be to the additional information of server 114 requests about selected discovery.Server 114 can be inquired about the data base who is stored on the data storage device 116, and additional report information is returned report viewer 118.

In another embodiment, method 800 can comprise that also generation comprises the synthetic medical report of medical image.Synthetic medical report also can comprise linking and is linked at least one of medical image in the medical image.In one embodiment, medical image and each can link medical image and comprises that patient's whole radiation is historical.In other embodiments, result of the test, laboratory work result, case history etc. also can represent in this report.In one embodiment, synthetic medical report is arranged in form.This form can comprise medical image and data of description at least a portion relevant with medical image.In another embodiment, synthetic medical report can be the figure report that comprises anthropometric dummy.In another embodiment, synthetic medical report can be timeline.Timeline can comprise similarly that medical image and at least one can link medical image.

Process medical image that medical image (706) can comprise that automatically montage captures with isolation diagnostic image component.Image through montage can be included in the synthetic medical report.In another embodiment, processing medical image (706) can comprise: extract Word message by optical character recognition (OCR) utility from medical image; And the extraction literal that will be associated with medical image is stored in the data storage device 116.In addition, this processing can comprise: show to have the graphic user interface that graphical representation and data of description represent; And reception is for the user command that image and data of description are linked.In addition, graphic user interface can comprise timeline.Equally, can comprise at server 114 processing images and data of description: at least one in the time of staying in response to eye gaze position and eye gaze automatically links image and data of description.For example, an embodiment can comprise: reach threshold value in response to the eye gaze time of staying of staring the position in particular eye, automatically trigger picture catching.

In another embodiment, processing medical image (706) can comprise: to medical image operation image matching algorithm to generate the unique digital signature that is associated with medical image.Processing medical image (706) also can comprise: the feature that quantizes medical image with automatic quantification tool.

Processing medical image (706) also can comprise: but in response to a plurality of medical images that link that are linked to the medical image data of description that is associated with one or more concatenated image, automatically follow the tracks of progression of disease.In one embodiment, this processing comprises: in response to medical image and the data of description relevant with medical image, automatically computational entity tumor the standard of curative effect evaluation (RECIST) is worth.This processing also can comprise: in response to the feature of medical image and the data of description that is associated with medical image, automatically determine the disease stadium.

In another embodiment, method 700 can comprise: whether have the repetition medical image in the specified data memory device 116; Whether there are the data that are repeated in this description that are associated with medical image in the specified data memory device 116; And merge the repetition medical image and be repeated in this description data.

In one embodiment, a kind of tangible computer program of computer-readable medium that comprises can comprise instruction, these instructions when carrying out so that computer (such as server 114) is carried out the operation that is associated with the step of above-described method 700.For example, these operations can comprise: be received in the medical image that catches on the medical image display device 112; Receive the data of description relevant with medical image; Process medical image and the data of description (706) relevant with medical image in data handling equipment; And medical image and the data of description relevant with medical image be stored in (708) in the data storage device 116.

In another embodiment of a kind of tangible computer program that comprises the computer-readable medium with instruction, the operation of being carried out by computer (such as treatment facility 208) can comprise: catch the medical image (702) on the medical image display device 112; Catch the data of description (704) relevant with medical image; And medical image and the data of description relevant with medical image be sent to treatment facility; This treatment facility is configured to process medical image and the data of description relevant with medical image in data handling equipment, and medical image and the data of description relevant with medical image are stored in the data storage device 116.

According to the disclosure, openly need not too much experiment with claimed all devices, system and/or method herein can make and carry out.Although described combination of the present invention and method with reference to some embodiment, but will it will be apparent for a person skilled in the art that, can to herein described combination and method and herein each step of described method or the order of each step apply variant, and do not deviate from concept of the present invention, spirit and scope.More specifically, it is evident that, can realize identical or similar as a result the time, the replaceable described reagent herein of particular agent that chemistry is relevant with physiology.All these that it will be apparent to those skilled in the art are similarly replaced and are changed and are considered to be in spirit of the present invention, scope and the concept that claims limit.

Claims

1. method comprises:

Seizure is configured to the medical image that shows at medical image display device;

Catch the data of description relevant with described medical image;

Process described medical image and the data of description relevant with described medical image in data handling equipment; And

Described medical image and the data of description relevant with described medical image are stored in the data storage device.

2. the method for claim 1 is characterized in that, also comprises: create data association between the described medical image in described data storage device and the data of description relevant with described medical image.

3. such as each described method among the claim 1-2, it is characterized in that, also comprise: described medical image is linked to patient's identifier.

4. such as each described method among the claim 1-3, it is characterized in that, also comprise: described medical image is linked to one or more medical images that link.

5. such as each described method among the claim 1-4, it is characterized in that described medical image and the described medical image that links link according to public verification.

6. such as each described method among the claim 1-5, it is characterized in that described medical image and the described medical image that links checked from difference link according to linking criteria.

7. such as each described method among the claim 1-6, it is characterized in that, also comprise: described medical image is linked to accounting code.

8. such as each described method among the claim 1-7, it is characterized in that, also comprise: generate synthetic medical report, described synthetic medical report comprises described medical image.

9. such as each described method among the claim 1-8, it is characterized in that, also comprise: generate and to comprise described medical image and described at least one the synthetic medical report that is linked to described medical image that links in the medical image.

10. such as each described method among the claim 1-9, it is characterized in that, also comprise: generate the whole synthetic medical report that radiates each historical linked medical image that comprises described medical image and comprise the patient.

11., it is characterized in that described synthetic medical report comprises form such as each described method among the claim 1-10, described form comprises at least a portion relevant with described medical image of described medical image and described data of description.

12., it is characterized in that described synthetic medical report comprises the figure report that comprises the anthropometric dummy with associated image discovery such as each described method among the claim 1-11.

13., it is characterized in that described synthetic medical report comprises the figure report that comprises timeline such as each described method among the claim 1-12, described timeline comprises described medical image and at least one described medical image that links.

14., it is characterized in that described medical image display device comprises picture filing and transfer system (PACS) such as each described method among the claim 1-13.

15., it is characterized in that described medical image display device is configured to show the medical image according to special-purpose medical image format configuration such as each described method among the claim 1-14.

16. such as each described method among the claim 1-15, it is characterized in that described data of description comprises voice data.

17. such as each described method among the claim 1-16, it is characterized in that described data of description comprises video data.

18. such as each described method among the claim 1-17, it is characterized in that described data of description comprises literal.

19. such as each described method among the claim 1-18, it is characterized in that described data of description comprises the eye-tracking data, described eye-tracking data comprise:

One or more eye gazes position; And

One or more eye gaze time of staying.

20., it is characterized in that described data of description comprises at least one that pointer position and pointer point hit such as each described method among the claim 1-19.

21., it is characterized in that the described medical image of described processing comprises such as each described method among the claim 1-20: the medical image that automatically montage captures is with isolation diagnostic image component.

22. such as each described method among the claim 1-21, it is characterized in that, the described medical image of described processing comprises: extract Word message by optical character recognition (OCR) utility from described medical image, and the extraction literal that will be associated with described medical image is stored in the described data storage device.

23. such as each described method among the claim 1-22, it is characterized in that, process described medical image and process described data of description in described data handling equipment in described data handling equipment and also comprise: show to have the graphic user interface that graphical representation and data of description represent, and receive and be used for user command that described image and described data of description are linked.

24. such as each described method among the claim 1-23, it is characterized in that described graphic user interface comprises timeline.

25. such as each described method among the claim 1-24, it is characterized in that, also comprise at the described server described image of processing with in the described data of description of described server processing: at least one in the time of staying in response to eye gaze position and eye gaze automatically links described image and described data of description.

26. such as each described method among the claim 1-25, it is characterized in that, comprising: reach threshold value in response to stare the position eye gaze time of staying in particular eye, automatically trigger picture catching.

27. such as each described method among the claim 1-26, it is characterized in that, comprising: show translucent pop-up window, described translucent pop-up window shows the previous check discovery that is associated with the feature of described medical image.

28., it is characterized in that the described medical image of described processing comprises such as each described method among the claim 1-27: to described medical image operation image matching algorithm to generate the unique digital signature be associated with described medical image.

29., it is characterized in that the described medical image of described processing comprises: the feature that quantizes described medical image with automatic quantification tool such as each described method among the claim 1-28.

30. such as each described method among the claim 1-29, it is characterized in that, the described medical image of described processing comprises: but in response to a plurality of medical images that link that are linked to the described medical image data of description that is associated with described one or more concatenated image, automatically follow the tracks of progression of disease.

31. such as each described method among the claim 1-30, it is characterized in that, the described medical image of described processing comprises: in response to described medical image and the data of description relevant with described medical image, automatically computational entity tumor the standard of curative effect evaluation (RECIST) is worth.

32., it is characterized in that the described medical image of described processing comprises such as each described method among the claim 1-31: in response to the feature of described medical image and the data of description that is associated with described medical image, automatically determine the disease phase.

33., it is characterized in that the data of description that is associated with described medical image comprises the label that is associated with described medical image such as each described method among the claim 1-32.

34., it is characterized in that described label is associated with the feature of described medical image such as each described method among the claim 1-33.

35. such as each described method among the claim 1-34, it is characterized in that, comprising: according to the natural language processing algorithm from through the isolation the speech segment determine described label.

36. such as each described method among the claim 1-35, it is characterized in that, comprising: determine described label according to the optical character recognition that appears at the literal on the described image.

37. such as each described method among the claim 1-36, it is characterized in that, comprising: determine described label according to the computer input that the user receives.

38. such as each described method among the claim 1-37, it is characterized in that, comprising:

Determine whether there is the repetition medical image in the described data storage device;

Determine whether to exist in the described data storage device data that are repeated in this description that are associated with described medical image; And

Merge and repeat medical image and be repeated in this description data.

39. a tangible computer program that comprises the computer-readable medium with instruction, described instruction so that described computer operates below carrying out, comprising when carrying out:

Be received in the medical image that catches on the medical image display device;

Receive the data of description relevant with described medical image;

40. tangible computer program as claimed in claim 39 is characterized in that, described operation also comprises: create data association between the described medical image in described data storage device and the data of description relevant with described medical image.

41. such as each described tangible computer program among the claim 39-40, it is characterized in that described operation also comprises: described medical image is linked to patient's identifier.

42. such as each described tangible computer program among the claim 39-41, it is characterized in that described operation also comprises: described medical image is linked to one or more medical images that link.

43., it is characterized in that described medical image and the described medical image that links link according to public verification such as each described tangible computer program among the claim 39-42.

44., it is characterized in that described medical image and the described medical image that links checked from difference link according to linking criteria such as each described tangible computer program among the claim 39-43.

45. such as each described tangible computer program among the claim 39-44, it is characterized in that described operation also comprises: described medical image is linked to accounting code.

46. such as each described tangible computer program among the claim 39-45, it is characterized in that described operation also comprises: generate synthetic medical report, described synthetic medical report comprises described medical image.

47. such as each described tangible computer program among the claim 39-46, it is characterized in that described operation also comprises: generate and to comprise described medical image and described at least one the synthetic medical report that is linked to described medical image that links in the medical image.

48. such as each described tangible computer program among the claim 39-47, it is characterized in that described operation also comprises: generate the synthetic medical report of each linked medical image comprise described medical image and to comprise patient's whole radioactivity history.

49. such as each described tangible computer program among the claim 39-48, it is characterized in that, described synthetic medical report comprises form, and described form comprises at least a portion relevant with described medical image of described medical image and described data of description.

50., it is characterized in that described synthetic medical report comprises the figure report that comprises anthropometric dummy such as each described tangible computer program among the claim 39-49.

51. such as each described tangible computer program among the claim 39-50, it is characterized in that, described synthetic medical report comprises the figure report that comprises timeline, and described timeline comprises described medical image and at least one described medical image that links.

52., it is characterized in that described medical image display device comprises picture filing and transfer system (PACS) such as each described tangible computer program among the claim 39-51.

53. such as each described tangible computer program among the claim 39-52, it is characterized in that described medical image display device is configured to show the medical image according to special-purpose medical image format configuration.

54. such as each described tangible computer program among the claim 39-53, it is characterized in that described data of description comprises voice data.

55. such as each described tangible computer program among the claim 39-54, it is characterized in that described data of description comprises video data.

56. such as each described tangible computer program among the claim 39-55, it is characterized in that described data of description comprises literal.

57. such as each described tangible computer program among the claim 39-56, it is characterized in that described data of description comprises the eye-tracking data, described eye-tracking data comprise:

One or more eye gazes position; And

One or more eye gaze time of staying.

58., it is characterized in that described data of description comprises at least one that pointer position and pointer point hit such as each described tangible computer program among the claim 39-57.

59. such as each described tangible computer program among the claim 39-58, it is characterized in that the described medical image of described processing comprises: the described medical image of automatically montage is with isolation diagnostic image component.

60. such as each described tangible computer program among the claim 39-59, it is characterized in that, the described medical image of described processing comprises: extract Word message by optical character recognition (OCR) utility from described medical image, and the extraction literal that will be associated with described medical image is stored in the described data storage device.

61. such as each described tangible computer program among the claim 39-60, it is characterized in that, process described medical image and process described data of description in described data handling equipment in described data handling equipment and also comprise: show to have the graphic user interface that graphical representation and data of description represent, and receive and be used for user command that described image and described data of description are linked.

62. such as each described tangible computer program among the claim 39-61, it is characterized in that described graphic user interface comprises timeline.

63. such as each described tangible computer program among the claim 39-62, it is characterized in that, also comprise at the described server described image of processing with in the described data of description of described server processing: at least one in the time of staying in response to eye gaze position and eye gaze automatically links described image and described data of description.

64. such as each described tangible computer program among the claim 39-63, it is characterized in that described operation comprises: reach threshold value in response to the eye gaze time of staying of staring the position in particular eye, automatically trigger picture catching.

65. such as each described tangible computer program among the claim 39-64, it is characterized in that described operation comprises: show translucent pop-up window, described translucent pop-up window shows the previous check discovery that is associated with the feature of described medical image.

66. such as each described tangible computer program among the claim 39-65, it is characterized in that the described medical image of described processing comprises: to described medical image operation image matching algorithm to generate the unique digital signature be associated with described medical image.

67. such as each described tangible computer program among the claim 39-66, it is characterized in that the described medical image of described processing comprises: the feature that quantizes described medical image with automatic quantification tool.

68. such as each described tangible computer program among the claim 39-67, it is characterized in that, the described medical image of described processing comprises: but in response to a plurality of medical images that link that are linked to the described medical image data of description that is associated with described one or more concatenated image, automatically follow the tracks of progression of disease.

69. such as each described tangible computer program among the claim 39-68, it is characterized in that, the described medical image of described processing comprises: in response to described medical image and the data of description relevant with described medical image, automatically computational entity tumor efficiency interpretational criteria (RECIST) is worth.

70. such as each described tangible computer program among the claim 39-69, it is characterized in that, the described medical image of described processing comprises: in response to the feature of described medical image and the data of description that is associated with described medical image, automatically determine the disease phase.

71. such as each described tangible computer program among the claim 39-70, it is characterized in that the data of description that is associated with described medical image comprises the label that is associated with described medical image.

72. such as each described tangible computer program among the claim 39-71, it is characterized in that described label is associated with the feature of described medical image.

73. such as each described tangible computer program among the claim 39-72, it is characterized in that described operation comprises: according to the natural language processing algorithm from through the isolation the speech segment determine described label.

74. such as each described tangible computer program among the claim 39-73, it is characterized in that described operation comprises: determine described label according to the optical character recognition that appears at the literal on the described image.

75. such as each described tangible computer program among the claim 39-74, it is characterized in that described operation comprises: determine described label according to the computer input that the user receives.

76. such as each described tangible computer program among the claim 39-75, it is characterized in that described operation comprises:

Merge and repeat medical image and be repeated in this description data.

77. a tangible computer program that comprises the computer-readable medium with instruction, described instruction so that described computer operates below carrying out, comprising when carrying out:

Catch the medical image on the medical image display device;

Catch the data of description relevant with described medical image; And

Described medical image and the data of description relevant with described medical image are sent to treatment facility, described treatment facility is configured to process described medical image and the data of description relevant with described medical image in data handling equipment, and described medical image and the data of description relevant with described medical image are stored in the data storage device.

78. a device comprises:

Be configured to receive the interface of medical image and the data of description relevant with described medical image;

Be coupled to the treatment facility of described interface, described treatment facility is configured to process described medical image and the data of description relevant with described medical image; And

Be coupled to the data memory interface of described treatment facility, described data memory interface is configured to store described medical image and the data of description relevant with described medical image.

79. such as the described device of claim 78, it is characterized in that, comprising: the one or more software definition modules that are configured to come in response to the instruction of storing such as the described tangible computer program of claim 39-76 executable operations.

80. a device comprises:

The medical image display device that is configured to the display of medical image;

Be coupled to the picture catching utility of described medical image display device, described picture catching utility is configured to catch described medical image;

Be configured to collect from the user interface apparatus of user's data of description; And

Be coupled to the communication adapter of described image-capturing apparatus and described user interface apparatus, described communication adapter is configured to described medical image and the data of description relevant with described medical image are sent to treatment facility, described treatment facility is configured to process described medical image and the data of description relevant with described medical image in data handling equipment, and described medical image and the data of description relevant with described medical image are stored in the data storage device.

81., it is characterized in that described image-capturing apparatus comprises the computer that is coupled to described display device such as the described device of claim 80, described computer has the operating system that is equipped with the screen capture function.

82., it is characterized in that described medical image display device comprises picture filing and transfer system (PACS) such as each described device among the claim 80-81.

83. such as each described device among the claim 80-82, it is characterized in that described user interface apparatus comprises eye-tracking equipment.

84. such as each described device among the claim 80-83, it is characterized in that described user interface apparatus comprises video camera.

85. such as each described device among the claim 80-84, it is characterized in that described user interface apparatus comprises voice recording equipment.

86., it is characterized in that described voice recording equipment comprises having the oral account equipment that triggers assembly such as each described device among the claim 80-85.

87. such as each described device among the claim 80-86, it is characterized in that, comprising: the one or more software definition modules that are configured to come in response to the instruction of storing such as the described tangible computer program of claim 77 executable operations.

86. a system comprises:

Server comprises:

Be coupled to the data memory interface of described treatment facility, described data memory interface is configured to store described medical image and the data of description relevant with described medical image;

Be coupled to the data storage device of described data memory interface, described data storage device is configured to receive and stores described medical image and the data of description relevant with described medical image; And

Be coupled at least one the medical image viewer in described server and the described data storage device, described medical image viewer comprises:

Be coupled to the communication adapter of described image-capturing apparatus and described user interface apparatus, described communication adapter is configured to described medical image and the data of description relevant with described medical image are sent to described server.

87. such as the described system of claim 86, it is characterized in that, comprising: the one or more software definition modules that are configured to come in response to the instruction of storing such as the described tangible computer program of claim 39-77 executable operations.

88. such as each described system among the claim 86-87, comprising: the medical imaging devices that is coupled to described medical image viewer.

89. such as each described system among the claim 86-88, it is characterized in that described medical imaging devices comprises X-ray production apparatus.

90., it is characterized in that described medical imaging devices comprises computer tomography (CT) scanner such as each described system among the claim 86-89.

91., it is characterized in that described medical imaging devices comprises radio frequency (RF) imaging device such as each described system among the claim 86-90.

92., it is characterized in that described medical imaging devices comprises nuclear magnetic resonance (MRI) machine such as each described system among the claim 86-91.

93. such as each described system among the claim 86-92, it is characterized in that described medical imaging devices comprises supersonic imaging apparatus.

94. such as each described system among the claim 86-93, it is characterized in that, also comprise: the PACS server that is configured to receive the view data that represents described medical image.

95. such as each described system among the claim 86-94, it is characterized in that, also comprise: be coupled to the PACS data storage device of described PACS server, described PACS data storage device is configured to store the view data of the described medical image of expression.

96. such as each described system among the claim 86-95, it is characterized in that, also comprise: be configured to the report viewer based on the report of media that receives in response to described medical image and the data of description relevant with described medical image that described server generates, described report based on media comprises that the patient of single figure in observing whole radioactivity is historical.