JP2004517380A - Diagnosis support system and method using cross-referenced knowledge and image database - Google Patents

Abstract

The present invention is a method and apparatus for enhancing the usefulness of visual knowledge in a number of applications. It derives the relationship between properties and hypotheses into a database format (104), which helps users to investigate various hypotheses (medical diagnosis, drug identification, plant / animal identification, cause of death, cause of accident, etc.) The visual information in a way that is suitable for The present invention avoids the open problems around knowledge engineering by not automating the decision making process. Rather, the present invention aims to assist the user in reaching a reasonable conclusion (120) based on the information available by direct observation and comparison of the stored image with the text data (116, 118). As a support tool, it uses a relational database (104) to dynamically respond to text and visual findings (110).

Description

[0001]
(Technical field)
The present invention relates generally to systems and methods for supporting diagnosis, and more particularly to cross-referenced knowledge and image databases, wherein an image-centric database is cross-referenced with text database information to provide a diagnostic hypothesis. And narrowed down to at least one subset of the possible diagnoses.
[0002]
(Copyright notice)
Some of the disclosures in this patent document include those that are subject to copyright protection. The copyright holder has no objection to the facsimile duplication of the patent document or patent disclosure as it appears in the Patent and Trademark Office patent files or records, but otherwise retains all copyright.
[0003]
(Background of the Invention)
The present invention is a system and method for automatically controlling the selection and display of visual information to assist in testing and / or reaching a diagnostic hypothesis. One embodiment of the present invention is used as a visual medical browser, and in particular herein is a visual diagnostic system (VisualDx).^TM). The VisualDx system is intended for use in patient care or care settings to assist the user in identifying possible diagnoses based on a set or arrangement of patient findings.
[0004]
Although a portion of the description below focuses on a particular dermatological application (adult dermatology) due to the features of the invention, it is further understood that various other applications of the underlying system and components are possible. Should. One such other application is a system (eg, an emergency medicine) that assists in identifying the drug or other oral medication that a patient may have received during treatment. A similar drug identification application is where a police technician uses the system as a first line to identify drugs or other oral drugs seized during arrest or interrogation.
[0005]
The present invention seeks to improve the paper atlas that is examined by a physician when investigating possible diagnoses. Focusing on the appropriate classification or arrangement of findings that best directs the diagnosis not only speeds up access to information but also uses a computer-driven interface to assist the user in the diagnostic process. Visual, textual information is provided to the user. The system believes that it is distinguishable from other medical diagnostic systems that simply search for a set of symptoms and input the symptoms into an artificial brain engine to create a ranked (by probability) list of diagnoses . Conversely, the present invention uses the classification or arrangement of patient findings or sample characteristics to examine or consider multiple diagnoses so that the user is not "led" to one diagnosis over another. Provide users with a combination of text and graphic / image information where possible. In other words, the system provides knowledge sources (medical or otherwise) in multiple formats that allow a user to test a diagnostic hypothesis against an image database using patient findings or sample characteristics.
[0006]
For vision-centric medical professionals (eg, dermatology, radiology, ophthalmology, pathology), physicians have the ability to classify, accurately identify and correlate disease visual features against a knowledge base of diagnostic features, criteria, etc. Is polished. Accordingly, an object of the present invention is a software-based scheme or strategy that assists inexperienced medical users in interpreting visual evidence (conditions and / or findings) when relating diagnostic hypotheses. Use graphical displays, drawings and select pictorial icons (PICONS) to augment medical keywords in the software database.
[0007]
One object of the present invention is to provide almost instantaneous access to diagnostically relevant images when used, for example when diagnosing a patient. The system uses a combinatorial, non-expert system approach to diagnosis or identification, along with access to relevant images. Images are often diagnostics and signs or indications of disease, whether under a microscope, a radiologist viewing box or a computer monitor. Users of the medical diagnostic features of the present invention can access data at patient care locations (eg, outpatient offices, emergency departments, walk-in clinics, hospitals, military medical facilities, corporate clinics, long-term care facilities, telemedicine fields). It is intended for health care workers who need real-time access. However, the invention can also be used in educational or reference-based environments.
[0008]
The present invention builds, to a considerable extent, the innate human ability to match patterns. It is the basis for pictorial handbooks or guides, such as field readings of plants, birds and animals and medical atlases. In addition, all visual identification problems benefit from user experience and knowledge. When previous visual knowledge is limited, painting or image "centered" reference material can assist inexperienced persons. Paper-based painting references have a linear structure and do not allow for user-defined grouping or painting alignment. Software-based imaging systems offer the possibility of combinatorial search with user-defined comparison possibilities.
[0009]
The visual diagnostic embodiment of the present invention, VisualDX, assembles text and visual knowledge and creates the ability to display "pre-sorted" images so that users can more effectively engage in pattern matching. These unique functional, organizational and graphical display capabilities are useful in any discipline where individuals must perform visual "diagnosis" or identification, or reconstruct visual features.
[0010]
Features of the present invention are believed to have particular relevance within the traditional medical and healthcare industry, including, but not limited to:
·Dermatology
・ Dermatology
・ Ophthalmology
・ Dental
・ Pathology including all sub-disciplines such as hematology, kidney, neuropathology, etc.
・ Obstetrics / Gynecology
・ Otorhinolaryngology
・ Gastroenterology (endoscopic image)
・ Surgery (interoperative image)
・ Urology (endoscope image)
・ Pulmonary Medicine (Endoscope image)
・ Microbiology (culture, microscope slides, eg Gram stain method)
・ Oral medicine
・ Patient self-use (diagnosis of own rash or diaper rash in children)
[0011]
(Disclosure of the Invention)
According to the present invention, there is provided a system for supporting a visual diagnosis process, comprising: an image database; a knowledge database cross-referenced to the image database to support the diagnosis process; A user interface requesting a plurality of descriptive characteristics, and a diagnostic engine responsive to the characteristics, wherein the characteristics of the sample are used by the engine to determine, from a plurality of possible diagnoses, diagnostics matching the characteristics The engine for identifying a subset and reconstructing an information space of the image database for simultaneous display of a plurality of images for user review via a user interface.
[0012]
According to another aspect of the present invention, there is provided a method of assisting a visual diagnostic process, the method comprising the steps of creating an image database from a set of images related to a particular subject matter; Creating a knowledge database with other data, wherein the knowledge database is cross-referenced with the image database to support a diagnostic process; and a user adapted to a particular subject matter is provided. Collecting from the user a plurality of descriptive features of the sample requiring diagnosis via the interface; and collecting from the user a plurality of possible diagnoses contained in the knowledge database in response to the descriptive features. Identifying a subset of diagnoses that match the described descriptive characteristics, and using the subset of diagnostics to Including the steps of reconstructing the information space of the image database for simultaneous display of a plurality of images associated with the description characteristics for users considering through the interface.
[0013]
The features of the present invention are based on the finding that the general primary caregiver lacks or is not well-informed about sufficient tools and resources to investigate multiple alternative diagnostic hypotheses. This finding recognizes the problems that arise in traditional patient diagnostic situations, where health care professionals first identify a set of possible diagnoses based on symptoms and other patient findings, and then at least one specific diagnostic Face to work to select for treatment.
[0014]
This is further based on the discovery of a technique that can properly match the appropriate visual information to the unique characteristics of the individual patient presentation or sample being analyzed. It is an object of various embodiments of the present invention to provide near-instantaneous access to appropriate images at or near the location of use of the information. For example, the identification of street drugs is useful to emergency room personnel along with police investigators at crime scenes or crime scenes. Similarly, at locations very close to the necropsy area, where the actual specimen / sample comparison will be performed on the images classified in the system, the coroner will use the system in assisting in determining the cause of death It is possible. The system uses a combined non-expert approach to identification / diagnosis and image access. Wherever found, visual characteristics or evidence is often a sign or indicator of identification and / or diagnosis. Temporal access to the appropriate visual evidence can be performed when correlating relevant findings or characteristics to the profile, and then the images are displayed as "stacks" or "clusters" in relation to the entered findings or characteristics. Is desirable. Displaying the appropriate images for each type of diagnosis, or displaying the images as a stack for a particular diagnosis, allows the practitioner to visually compare and contrast the patient's own observations with the images, thereby providing accurate and appropriate diagnostics. Improve the possibilities.
[0015]
It is an object of the present invention to create a streamlined process of obtaining and tracking images to support diagnosis and identification, referencing all items in the knowledge base for relevant explanatory literature, easy to use, efficient Implementing efficient, network-distributed data entry / access systems, centralizing network connections to integrate knowledge sources, publishing on media and network connections capable of carrying images for public and private use To create specific subset modules of the knowledge base to serve important areas of need for diagnosis and identification (eg, adult dermatology, fever and rash, drug identification, plant identification, etc.).
[0016]
A feature of the present invention is based on the fact that experts lack sufficient tools and resources to investigate multiple alternatives when searching for specimens to identify or verify diagnostic hypotheses. I have. This finding is that experts (physicians, nurses-practitioners, coroners, criminal, etc.) first identify, based on a given set of characteristics and findings, a set of possible identifications or diagnoses to the extent that the expert can work. We have found the problems that arise in traditional situations that have been reduced to a minimum and then faced when working to complete the identification or diagnosis.
[0017]
One aspect of the invention is generally referred to as a visual browser, and the system associates the identification of a particular feature or common visual manifestation with data in a particular research area (eg, drug, drug control, etc.). It is intended to be used as a supporting tool. Various embodiments of the present invention provide a method for identifying physical elements or investigating hypotheses (e.g., diagnoses) for paper atlases or incomplete or nonexistent databases referenced by physicians and other professionals. Intended as an improvement. Throughout all of the various embodiments, similar visual or textual information is provided to the user using a computer-driven interface that accelerates access to the information, and appropriate characterizations or findings that will have a guide to the conclusion. It is also possible to provide assistance in the diagnosis / identification process by concentrating the user on the sequence (eg, identification, confirmation of diagnosis, loss assessment, etc.).
[0018]
The techniques described herein are useful because they are adaptable to a number of diagnostic situations that a practitioner / user may encounter when making a diagnosis or testing multiple diagnostic hypotheses as well. As a result of the present invention, a practitioner faced with the above situation is assisted by image / graphic and text tools that allow for the scope of each possible diagnosis and allow for review of each detail.
[0019]
(Embodiment of the invention)
For a general understanding of the present invention, reference is made to the drawings. In the drawings, the same reference numbers are used to refer to the same elements throughout. In describing the present invention, the following terminology will be used in the description.
[0020]
The “user input circuit” is a circuit that provides a signal based on a user operation. The user input circuit is capable of receiving signals from one or more "user input devices," such as a keyboard or mouse, that provide signals based on user actions. The set of signals provided by the user input circuit therefore includes data indicating a mouse operation and data indicating a keyboard operation. The signal from the user input circuit may include a "request" for an operation, in which case the system responds and performs the requested operation.
[0021]
An "image" is a pattern of physical light rays. A "display" is an image output device that provides information in a visual format. The display may include, for example, a cathode ray tube operatively driven by an operatively connected computer, an array of light emitting, reflecting, or absorbing elements, structures that provide marks on paper or other media, or visual Other structures that can define the image in a format. To "present an image" on the display means to operate or drive the display so that a viewer can recognize the image.
[0022]
For example, although a wide variety of display technologies are available for data processing devices, including various graphic user interfaces, despite their versatility, these technologies tend to have certain common characteristics. One fundamental common property is that displays produce human perception. In this application example, “display function” refers to some human perception generated by the display.
[0023]
“Display object” or “object” is a display function that can be recognized as a coherent unit. An “object plane” or “plane” is a display function that can be recognized as the surface of a display object, for example, the outer boundary of a three-dimensional display object is a plane. A "region" on a surface is an area bounded by the surface, for example, a single point is the smallest possible region of any surface. A "shape" is a display object having an identifiable contour, for example, a circular display object is a shape.
[0024]
"Structure" is a display feature that includes other display features therein, all of which appear to be connected to a display or a unitary element of a user interface depicted thereon.
[0025]
A "selectable unit" is a display function that is recognized as being surrounded by a selectable display area. For example, an arbitrary text link displayed in the area 288 of the interface screen in FIG. When used in connection with a selectable unit, the term "select" means a user input operation that includes a signal from a user input circuit that indicates the selectable unit differently. The user selects a selectable unit using a pointing device such as a mouse, for example, by pointing to that position via a cursor on a display and clicking a button on the pointing device. In general, a selectable unit may take any appearance and is not limited to a visually identifiable function or set of functions that are viewed as a coherent unit.
[0026]
"Findings" are factual data about a patient or characteristic associated with the diagnosis. There are two basic classifications of findings. Findings that can be described effectively only by words are called text findings, while findings that can be described most effectively by icons or illustrations are called visual findings.
[0027]
Turning now to the essential description of the elements, features, and functions of the present invention, the following description is divided into sections, each of which is a specific description of a system used in implementing an embodiment of the present invention. It is intended to highlight and describe features and / or methods in detail. While this description focuses on the use of systems and methods for diagnosing adult diseases with signs of dermatological findings (eg, VisualDX: Adult Dermatology), similar techniques are used for drugs (tablets, capsules, etc.). It should be understood that such oral drugs can be used for identification or in various important areas where the medical information needs are mentioned above. Accordingly, the following description first describes the system requirements and architecture of the preferred and alternative embodiments, and then describes the use of the present invention with the VisualDX system.
[0028]
I. System requirements
Referring to FIG. 1, an exemplary embodiment of the present invention is illustrated. The present invention operates in two configurations: as a network-based client / server application for medium to large customers, and as a single user product with both client and server components on the same computer. . However, these configurations may be implemented on the same software basis. In particular, the present invention may be implemented in connection with a network or distributed computer system 50. Such a system preferably includes a host computer 54 having a mass storage device that maintains and is operatively associated with a master database 58.
[0029]
System data and operating software components may be distributed to one or more remote computers 70 via a removable medium such as a network connection 60 (eg, the Internet) or a compact disk 62. Although various other system configurations are possible, the computer system 70 includes a color monitor or display 72 capable of displaying image information at an appropriate resolution, for example, 1024 × 768 pixel resolution using 16-bit color (65536 colors). It is desirable. System 70 further includes a system unit (desktop, tower or mini-tower configuration) with a Pentium II processor (Intel) or equivalent operating at 233 MHz or higher. The system unit must further include a CD-ROM drive for software installation, a hard disk or equivalent mass storage device, and at least 32 megabytes of RAM (not shown) to run the software. In a preferred configuration, the software runs on a system using Microsoft's Windows-based operating system-Windows 95 or higher and Java or Macromedia Director. However, the functionality of the system may be further enhanced using a compatible browser interface (eg, Windows® Explorer, Netscape Navigator).
[0030]
System 70 also preferably includes one or more user input circuitry that allows a user to interact with the software. Examples of suitable devices include a keyboard 80 and a mouse 82, as well as other devices such as wireless communication tools (eg, a palm pilot or remote control using an infrared link). As will be described in detail below, software running on a remote computer preferably illustrates various objects and selectable display units for selection to characterize a patient, diagnostic sample or other object. . It will be appreciated that the drawings consider the diagnosis or characterization according to the specifications of a standard characterization device 74, which is illustrated as a characterization peripheral. Such a device is an imaging device in one embodiment of the invention (e.g., pill identification), or the output of the device is suitable for providing one or more characteristics of a sample obtained under standard conditions. The inspection apparatus of another embodiment may be used. Based on the user's input or the arrangement of the findings or characteristics indicated by the user, both directly given by the characterization device, a subset of the possible diagnostics or identifications may be displayed by the user for further review. Selected from a database-where the display preferably includes at least one or more reduced size or thumbnail images illustrating an example of a diagnosis (eg, a skin lesion map, tablet diagram).
[0031]
System 50 may be a stand-alone system, but it is also completely possible to install and operate the present invention on a network (LAN, WAN or Internet) where one or more users access a separate "view" of the system at any particular time. Is possible. In such a system, the software is expected to be installed on a server in a client / server system, and multiple users access and use the software simultaneously. Further, it is conceivable that the user implements or hosts system 50 using an application service provider (ASP) that accesses the system through a subscription or similar business relationship.
[0032]
In yet another embodiment, the adult dermatology module included as a detailed example herein is merely one of many such modules, some of which may be interrelated. Other modules include, but are not limited to, pediatrics (including pediatric dermatology), geriatrics, genetics, occupational medicine, human immunodeficiency virus (HIV), nevus, wound treatment, infectious diseases, diabetes, Including environmental exposure, body area specific modules, forensics, plant identification, toxic plant identification, viper identification. The user can access one of the modules via a higher level menu or a set of icons not shown. Further, menu content or icons may be displayed as a function of those modules to which the user and / or the user's experience, organization or company have subscribed. In a multi-module environment, passing modules and subscriptions from an application service provider (ASP) from a common host location (possibly including mirrors or redundant sites to ensure access) via the Internet or equivalent wide area network Is also conceivable. Each of the modules uses a similar user interface "look and feel" whenever possible to ensure that a user familiar with the operation of one module can quickly become proficient with another module. In other words, the user interface remains the same and the user options, selections and data change based on the module and the most appropriate information needed to perform a possible diagnostic search. In addition, the sum of multiple modules, when thoroughly developed, allows the placement of possible diagnostic categories or findings for a particular medical technology (dermatology, radiology, etc.) or sample type (tablets, plants, etc.) Produces a "complete" list of
[0033]
Referring now to FIG. 2, system 50 operates according to the overall steps of the flowchart. In particular, starting at step 100, capture an image for use in the system from an existing digital image obtained either directly using digital photographs or digitizing existing photographs using known scanning techniques. Or identify. It will be appreciated that various methods can be used to capture the photo, as is known in the art of digital images and digital photography. In one embodiment, each digital image is available in at least two, and preferably three, dimensions (pixel width x pixel height): 900x675, 384x288, and 160x120 pixels. In addition to the images, the database 58 of the present invention includes the characteristics of the items that are the subject of the images (eg, diseases, pills, etc.) as described in more detail in connection with FIGS. Database 58 is preferably a relational database that supports Structured Query Language (SQL) queries using Open Database Connectivity (ODBC). Step 102 represents the corresponding item of the property information associated with the image.
[0034]
As described below with reference to the database architecture, the image and characteristic information is implemented in a plurality of relational tables in the database created in step 104. Once the database has been created, it is available for distribution or transfer to the user's computer, as shown at step 106.
[0035]
Following the transfer or download of the database and associated integration software performed to manage the use of the user of the system, the VisualDx software starts and, for example, a user working on a remote computer 70 as shown in FIG. At 110, one or more observable characteristics are entered for the patient or sample (eg, skin lesion type, drug shape). Based on the characteristics entered by the user, the system then automatically analyzes the user input at step 112 and automatically selects one or more relevant diagnostics included in the database, where the selection comprises at least one identification. Including properties. At step 114, output to the user for review is preferably performed and includes a graphical representation of the diagnostic subset identified via the database.
[0036]
As shown at steps 116 and 118, the user selects one or more diagnoses in a review subset that includes a review of one or more images associated with the particular diagnosis and a review of textual information associated with the diagnosis. You may. Further, as shown in step 118, the user interface described in detail below can compare and contrast another diagnosis in the subset to the user to improve the likelihood of an accurate diagnosis by the user. Finally, the user completes the diagnosis or identification process by making a decision as represented by step 120.
[0037]
In some implementations of the invention, it is further contemplated that the diagnostics and associated textual and image data are returned to the host computer for inclusion in subsequent publications in the database. In this way, the system improves the rate of identifying new diseases, street drugs, etc., and fully characterizes the addition to the database.
[0038]
II Database / Knowledge Base Architecture
In one embodiment, the VisualDx software includes a diagnostic module as a sub-component that seeks to find a diagnostic set that matches or partially matches user input information (text and visual findings). Desirably, the diagnostic module has a theme (eg, general adult dermatology, or reaction with drug rash). In addition, VisualDx also includes an Image Atlas Module subcomponent whose function is to find sets of images that match or partially match user input information (keywords, diagnostics, etc.).
[0039]
As implemented in accordance with features of the present invention, a knowledge database, or knowledge base, is a relational database that uses various database tables in the schema 200 of FIGS. 3 and 4 as the basis for diagnostics presented by the system. is there.
[0040]
Knowledge base 200 preferably includes a plurality of relational tables representing icons, text and images used by the diagnostic engine and associated user interface. As described below, the user selects visual and textual properties or findings without prior knowledge of the specialized vocabulary associated with these findings, and via the knowledge base 200, the selection can be , Translated into queries and outputs containing images and their relationships to each other. Preferably, the knowledge base schema 200 is optimized in a way that significantly reduces the size of the database. With the ever-increasing speed of computers, this alternative to reducing the size of the database and improving delivery capabilities, even if speed is compromised, should be used unless faster processors are used to run the software. A schema has been developed.
[0041]
The relationships between the tables are indicated by the lines between them. In addition, PK indicators having the values FK1, FK2, and FK3 represent foreign keys to the association table.
[0042]
A brief description of the table is included below to characterize the functionality built into the knowledge base by the various objects and their interrelationships:
[0043]
Image table 210: Each of the image records represented via table 210 includes image characteristics, one or more of which may be unavailable to users of the system (eg, PATENT_CONSENT). For example, the CAPTION field displays the heading assigned to the image viewed by the end user.
[0044]
Case_Diagnosis_Image table 212: This table links cases, their images and diagnosis to each other. The case can have multiple images and diagnoses.
[0045]
Case table 214: The case record represents a particular patient having an association definition, an association definition characteristic found in the case findings table 216 and the case diagnosis table 218. There is a specific set of images associated with each case. There are also findings associated with the case, such as patient symptoms, experimental values, etc., which are used as additional characteristics to be considered for diagnosis.
[0046]
Image_Source Table 222: The image source record contains all information about the source of the image, the photographer, and the license to the image to ensure that the system only allows the images that have been licensed for use. .
[0047]
Image_Findings table 224: Links together images and their unique morphological characteristics. The images are assigned a form by a professional reviewer. In this table, the images are linked to these features (the concept is a concept, which is assigned a finding type in the findings_type table 226; for example, the lesion type (vesicle, pustules, etc., composition, distribution, Abnormal forms) In a preferred embodiment, an image can have multiple forms.
[0048]
Case_Findings Table 228: Link specific patient findings to case records (eg, actual patient symptoms, signs, etc.)
[0049]
Case_Diagnosis Table 218: Links diagnosis 232 to a specific case. A patient (case) can have multiple concurrent diagnoses.
[0050]
Concept table 234: A concept is a basic entity. It is unique, discrete and not redundant. Each entity in the database is a concept. The concept applies to places that fit within the web structure. It has parents and children. Parent-child relationships allow for the placement of diagnoses and findings within the logical knowledge hierarchy. For example, inguinal lymphadenopathy (swollen lymph nodes in the groin) is a subtype of lymphadenopathy, which is a more general sentence; swollen lymph nodes develop, but specific locations are designated It has not been. The end user may only want to be specific, for example, that the inguinal lymph nodes are swollen, or alternatively, the user may want to say that all lymph nodes are swollen. The system must handle both scenarios exactly. Most symptoms, signs, and diagnoses fall within such a hierarchical structure. For both diagnostics and findings, such a knowledge structure allows the end user to search for specific or general ones, and for wider or more limited exploration results (more Inguinal lymphadenopathy has a limited set of related diagnoses, as compared to the search for lymphadenopathy with related diagnoses).
[0051]
Concept_synonym table 236: This table links concepts to a number of synonyms found in synonym table 238.
[0052]
Synonym table 238: Synonyms can interface terms with expert and amateur end users and foreign language users. This table is a meta-dictionary / thesaurus, allowing multiple versions of the end-user application without having to create a new database to contain or house the end-user application.
[0053]
Relationship table 242: facilitates the creation of a network or "web" of concepts. The diagnoses are arranged in a plurality of parent-child, child-parent relationships. Allows end users to explore the hierarchical tree up and down.
[0054]
Term table 244: facilitates natural language processing of all words in end user tools and databases. In this table, concepts are associated with common terms and related terms and synonyms that reflect the language. Some terms are not necessarily included in the synonym table 238 or the concept table 234.
[0055]
Module Table 246: The system allows for multiple problem-oriented modules, each with a unique author, editor. For example, there may be an adult dermatology module and an ethnic dermatology module that may have overlapping data. Also, over time, multiple versions of the module may be published, meaning that alternative authors, editors, etc. may be used. The revision Revision_ID field of the table links to the revision details of the module.
[0056]
Module_Diagnostics Table 248: Multiple diagnostics are included in the module. In this table, the diagnosis is linked to the module, and the text specific to the diagnosis in the module is also linked by a foreign key. In other words, the text of a particular diagnostic can be changed between modules, or perhaps even between users in another embodiment (eg, layman or technical terminology).
[0057]
Diagnosis table 252: Diagnosis is the type of concept. Diagnostic records are listed separately from the concept table because they need to be uniquely limited from individual modules. By way of example, the degree of the disease may be contextual (severity_level) and thus needs to be associated with a concept, but is uniquely identified.
[0058]
Diagnostic_Module_Text Table 254: Each module has multiple diagnostics, each with unique text seen by the end user. The diagnostics can have different text depending on which module the text appears in. For example, herpes simplex in the ethnic skin module has a different "exploration" text than the Caucasian skin module, but this is the same diagnosis. The associated text table (below) serves as a container for multiple versions of the text.
[0059]
Text Table 256: Text is written module-specific, but certain shared elements can be "repurposed" to other modules. The text table contains general text that can be used in new or different modules.
[0060]
Diagnosis_Findings Table 258: Diagnosis is related to findings, findings such as laboratory values can have ranges (value_min and value_max), and relationships are entered by “creator” (CREATOR) ID) and “editor” (EDITOR_ID) will be reviewed accordingly.
[0061]
Concept_Findings_Type table 262: Findings can have one or many "Finding Types". Thus, the concept is hereby linked to the finding type. (For example, fever can be a symptom or sign.) In this table, concepts are given a finding context.
[0062]
Findings_Type Table 226: In this table, concepts are grouped into logical categories of medicine (or other fields: special domains that are practical for end users). Finding types that are logically related to medical diagnostic systems such as VisualDx include:
·age
・ Appearance
・ A-bomb
・ Medical history
·sex
[0063]
Further, although not specifically shown, various images used and displayed by the system are maintained according to the ID field of the image table 210. The image is preferably stored at a designated location indicated by one or more fields in table 210. Although the image (not shown in FIG. 3 or 4) is preferably stored as a JPEG file, any known or equivalent image file or image compression format may be used (eg, TIFF, CPC, PDF, BMP, etc.). Further, each image desirably has a reduced sized or thumbnail display (PICON) included in a user interface display that illustrates the plurality of images. It is important to note that the present invention uses images as an indication of a diagnosis, as the diagnosis is the output of some query in the knowledge base. This is in contrast to using an image database where the purpose of the search is simply to find images with relevant keywords.
[0064]
Referring to FIG. 4, the different types of characteristics or findings illustrated in the VisualDx example have a complex and rich hierarchy that is desirably represented by a separate table structure for each type of diagnostic finding. The VisualDx; Adult Dermatology example, described further herein, is merely one example of a feature of the present invention, and provides a number of uses of cross-reference knowledge and image databases to assist in diagnosis or other identification. It should be recognized that other examples exist.
[0065]
It should be appreciated that other database designs may be used to implement the invention. For example, some modules may maintain a speed optimization schema (flat hierarchy), while others may use a space optimization schema, and the choice of schema type may be partially dependent on the needs of the module and user population. Depends.
[0066]
III Findings / Characteristics
In the VisualDx example described herein, the knowledge base 200 desirably includes a table of medical findings appropriate for diagnosis and images available for viewing. In another embodiment, the knowledge base 200 may include a table of characteristics appropriate to the sample to be identified. The user explores the diagnosis and its associated images by entering the patient's findings via the findings menu. Findings can be in the tens of thousands in the master knowledge base, from which specific modules can be derived and limited to hundreds of specific problem-oriented modules, including a subset of the knowledge base. A module, or subset of a knowledge base, is a specific grouping of diagnoses around a visual diagnostic area or clinical problem area (eg, adult dermatology).
[0067]
Due to the large number of findings, the user needs a way to quickly reach a possible set of diagnostics (and relevant images for review) using the available findings or characteristics. The list of possible findings is constrained by the number of diagnoses found within a particular module. The visual problem area presented by each module is, by definition, limited in the number of diagnoses to be covered and its associated findings to be entered or explored, thus facilitating user input of patient findings. Basically, a limited findings list defines the "questions" that a medical user asks a patient for a particular module. The overwhelming cognitive load imposed by the complexity of medical decisions and the thousands of possible relationships dictates a problem-oriented approach.
[0068]
Findings may be broadly categorized into groups such as medical history, family history, travel history, age, physical location, patient symptoms, visual signs, radiation signs, and the like. In one embodiment, each of these groupings is given its own table structure in the database, thereby optimizing speed. Alternatively, the structure of the database may be modified to replace a flatter table structure with more levels of indexed findings and diagnostics as illustrated in FIGS. 3 and 4 (a more compact structure).
[0069]
The findings displayed in the system are composed of the smallest perceptible units that can be associated with a diagnosis, preferably by a medical professional (author and editor of the illustrated embodiment). Different modules of the system require different levels of granularity for their findings. For example, in certain fields of medical practice, a complex finding such as "a blister of 3 mm or less containing a yellow liquid" is the smallest perceptible unit. In other fields, it is meaningful to analyze the findings as "blisters", "3 mm or less", "including yellow liquid".
[0070]
Some of the findings fineness problems are solved by considering the meaning of the parent's findings linked to the detailed findings. The parent finding of "a blister containing a yellow liquid" may be "a blister with a liquid." This holds even more closely for the purpose of the present invention of trying a basic finding based on one adjective structure for a certain subject. The above example would then look like this:
-Blisters with liquid
-Yellow liquid
-3 mm or less in diameter
[0071]
The user interface software illustrated starting with FIG. 5 tracks this semantic link while maintaining the parental finding information being viewed as part of the set or all information related to the finding. In other words, using one or more tables in the database and the linked tables associated therewith, the user interface continually lists the selected or possible diagnoses associated with a particular patient along with the findings. Update.
[0072]
Referring to FIG. 5, at the start of the VisualDx system software, a user interface screen 280 illustrating one of a plurality of module selections, including but not limited to general diagnostics and special diagnostics modules, is provided to the user. Be presented. The user accesses the module via a top level menu or a set of icons. Further, menu selections may be displayed as a function of the user and / or the user's experience, an organization or company subscribed to the module. In a multi-module environment, it is further conceivable that the modules and subscriptions may be applied via the Internet or an equivalent wide area network from a common host location. Each of the modules uses a similar user interface "look and feel" wherever possible to ensure that users accustomed to one module will quickly become proficient in using the other module. In addition, the sum of a plurality of modules, when thoroughly developed, ultimately produces a "complete" list of possible diagnostic classes or sequences of findings. The initial VisualDx interface screen also includes the ability for the user to immediately explore an image atlas (eg, a dermatology atlas) or explore help (eg, a quick start) when using the VisualDx software. .
[0073]
The VisualDx example described herein assumes the user's selection of the adult dermatology module as a result of a "click" on the user's "adult dermatology" link 282. The user's selection now results in the display of screen 286 as shown in FIG. The screens and most of the screens used illustrate the look and feel of the VisualDx software. In particular, the user interface screen, in the case of the adult dermatology module, uses a drop-down or pull-down menu bar 288 from which skin lesion type, lesion distribution, other findings, and another menu selection can be performed. On the left side of the screen 286, there are a view selection button 290, a patient finding window 292, and a proposal diagnosis window 294. In the lower left part of the screen, there is a sort type selection button 296. The functions of the various buttons and windows are described in further detail below. Even more appropriate is that no particular diagnosis is selected as a subset of the matched diagnosis on a particular patient basis (all diagnoses are shown in a scrollable window 294) and therefore the area of the user interface screen 298 is that no image or additional information is shown. However, the user may select from any of the diagnoses presented in the scrollable window 294 to view the relevant images. Such a function provides a function similar to "indexing" or "chapter" of a book, and allows a user to quickly view images related to one or more selected diagnoses.
[0074]
Referring now to FIG. 7, the preferred database has the appropriate granularity and individual modules that are customized to use the appropriate search area for the task of narrowing the visual diagnostic search. For example, within the "Adult Dermatology" software module, it is sufficient to use the term "fluid-filled lesion" to limit visual exploration of vesicular (fluid-filled) lesions.
[0075]
The basic principles of the design of the database structure of FIGS. 3 and 4 and the various user interface screens described herein include photo or image icons (schematics, photos, videos, thumbnails and icons) of a particular disease or diagnosis. (Including images) must be associated with the findings. This is generally performed by authors who are knowledgeable in the field and will be subsequently confirmed by the editor. Therefore, the preferred user interface as illustrated in FIG. 13 is to display the classification or arrangement of findings in a pull-down menu by MS-Windows®, or on a screen illustrating detailed information about the findings. Is illustrated. In general, diagrams and artist illustrations are considered to be more useful at an advanced, more abstract level of visual concept, e.g., for "blisters" or "skin lesions" such as the distribution of lesions on a patient's body. Finding is a subclass. Photo or image icons are more useful when dealing with finer visual terms such as "hollow blisters". Since these images are on the diagnostic side, they are not made up of the finding cases, but the links back to the original cases are preserved as previously described with respect to the database table of FIG. Images and picons are preferably linked directly to the findings they represent.
[0076]
Successful diagnosis of pattern recognition dependence in dermatology (and in other visual-centric areas such as medicine, radiology, pathology, ophthalmology, etc.) is determined by a number of factors, the most important of which is visual An examiner's ability to accurately recognize, define, and sort findings into a well-established classification scheme. Often, specific visual evidence correlates with an accurate diagnosis, and the ability to observe, classify, and associate visual findings with existing visual knowledge can create a split secondary diagnosis due to specialization. More importantly, the present invention is directed to the ability of a family physician, general practitioner, pediatrician, obstetrician / gynecologist, emergency physician, registered nurse, and physician assistant to diagnose unskilled primary healthcare professionals. Is believed to improve significantly.
[0077]
For example, dermatologists refer to primary forms. This refers to the overall type of lesion without secondary changes. These are basic features such as papules (raised lesions, usually less than 1 cm), vesicles (raised, fluid-filled lesions, usually less than 1 cm), and plaques (plain lesions, usually less than 1 cm) . These are examples of general terms that are not very specific and create at most a large diagnostic group. Lesions can be further classified into secondary morphologies, such as scales, scabs, scabs, etc., which are surface changes that typically occur in primary lesions such as papules. A scaled papule is slightly more specific, however, its size, shape, color, and scale location are further instructions for accuracy and help narrow the diagnostic description to a more manageable number . Further descriptive terms used include the number of lesions, the shape of the group of lesions, the distribution on the body, the color, etc.
[0078]
As shown in FIG. 7, screen 300 includes a pull-down, visual menu that is activated by the user placing a pointer or cursor on “lesion type” on pull-down menu bar 288. This causes the display of various icons and images in area 304 to illustrate the range of lesion types selected by the user with a mouse or other pointing device. Schematic representations describe unskilled users without the prior knowledge of the specialized vocabulary associated with these findings, i.e., subtypes of fluid-filled lesions, such as "hollow vesicles" It allows the user to select or enter visual findings without having to enter more specific terms. Conversely, the module designated "Blister Skin Disease" provides highly detailed and detailed sub-menus, such as "hollow", "liquid color", "hemorrhagic fluid", "serum", etc. May have. Therefore, this level of fineness is not needed in modules where the diagnosis is of little concern with fluid-filled lesions. The user's selections are translated into queries on the database, including findings, diagnoses, images, and relationships to each other as described herein.
[0079]
It is noted that the adult dermatology examples described herein do not include each of the finding classifications. Rather, it uses a subset of the classifications, where the subsets include the classifications that are deemed most important in distinguishing and / or identifying possible diagnoses. VisualDx: The classification used in adult dermatology is illustrated in response to a user selection of the "findings" area on menu bar 288.
[0080]
Considering the example of FIG. 7, upon user selection of a lesion type, such as "elevated non-scaled lesion" at location 302 of lesion pull-down menu 304, the lesion type is displayed as a characteristic of the patient. Hereinafter, the user interface screen of FIG. 8 is illustrated as a result of the user selection of the location 302. Referring to FIG. 8, the lesion type selection is reflected in the user interface screen 330, where the patient findings window 292 is updated to show the lesion type, and the possible diagnoses have been rearranged and matched in the diagnostic window 294. The type (raised, non-scale) is indicated. Further, a scrollable image window 334 is provided in the area 298 of the screen. Within the scrollable window 334 are illustrated a plurality of diagnostic image windows 338, each having its own function.
[0081]
Each of the windows 338 includes a central area 340 that displays a digital image illustrating a diagnostic case of the disease indicated in the title bar 342 associated with the window. Along the bottom of each window 338 is an image control bar 344, where the central area of the bar displays the number of images available for a particular diagnosis. On the left side of the bar 344 is an image stack button 348 that allows a user to view a plurality of images 356 in the image stack of the scrollable window area 358 as shown in FIG. As FIG. 9 illustrates, the images associated with a particular diagnosis are displayed in a scrollable “contact sheet” which is also selected (by clicking on the reduced size image) and shown in FIG. To display an enlarged full screen view. Returning to FIG. 7, the details button 350 changes the view of the user interface shown in FIG. 10, where the area 298 includes not only the diagnostic image window 338 but also additional information and details that allow the user to associate with a particular diagnosis. Also includes a scrollable text window 354.
[0082]
Returning now to FIG. 11, the user interface screen 330 of FIG. 8 is illustrated in a modified form in response to a user selection of the “Distribution” item on the menu bar 288. As a result of such a selection, a painting menu 360 is displayed on the image window 334 in the area 298. The pull-down painting menu 360 illustrates various options for the distribution of possible dermatological rash patterns. In the menu, each selection area 362 includes a graphical description of the distribution 364, as well as a brief descriptive description in area 368. Here again, through the pictorial representation, the system allows the user to add features (eg, where disease symptoms are distributed throughout the body) by "matching" the visual characteristics of the patient with the pictorial representation. Assuming that the user has selected item 370 (distributed bullshit), the user interface screen of FIG. 12 is displayed.
[0083]
Returning to FIG. 12, a user interface screen 400 is illustrated, wherein the distribution information selected by the user is illustrated in a window 292. In addition, the diagnostic window 294 now reflects at its top a diagnosis consistent with the two patient characteristics currently entered into the system-raised, non-scaly, and distributed random distribution. Similarly, within the scrollable window 334, different diagnostic image windows 338 are shown. Again, diagnostic image window 338 is consistent with the possible diagnostics indicated in window 294.
[0084]
In addition to the lesion type and symptom distribution, to further refine the possible diagnosis, the user may also enter one or more findings associated with the patient using the "findings" selection on the pull-down menu bar 288. Good. Referring to FIG. 13, an exemplary pull-down menu for the adult dermatology module is illustrated. The different types of findings, some of which are illustrated by pull-down menus 410, 420, 430, have a complex and rich hierarchy that is better represented by a separate database table structure for each type or classification of finding. ing. However, it should be understood that equivalent database structures and methodologies may be used to implement equivalent features and functions as described and claimed in accordance with the present invention. The types of observations used in various VisualDx modules or embodiments contemplated by the present invention include one or more of the following:
・ Signs
・ Symptoms
・ Travel history
・ Medical history
・ Surgery history
·Family history
·age
·sex
·jobs
・ A-bomb
・ Radiation signs
·dosage
・ Habit
・ Inspection findings
・ Subcutaneous signs
・ Form
・ Abnormal form
・ Subcutaneous form
·distribution
[0085]
Dermatological rich descriptive terms have similar contrasts with other visual morphological terms. For example, radiologists classify visual findings within categories such as density (hypodense, hyperdense, etc.), pattern (interstitial, etc.) location specific, etc., as well as pathologists on glass slides. Utilize visual terms that correspond to the invading cell type, color, and overall "architecture" of the tissue or the like found.
[0086]
As another example, a pathologist must correlate microscopic morphological patterns with a diagnosis, which often depends on clinical history. It is common for pathologists to use the "clinical history" (found on the sample donation slip) to guide their interpretation process. The pathologist must look at the shape of the cells, the overall architecture of the examined tissue, the color changes, and interpret the changes within the context of the available patient findings. For example, hematologists receive bone marrow biopsies to help evaluate patients with very low blood cell counts. Patient history (findings), such as medical history of patients with a history of malignancy, the patient's current medication, and related medical conditions and signs, such as jaundice, may help pathologists interpret visual findings and provide diagnostic meaning It is indispensable when.
[0087]
Similarly, in the field of radiation, the referring physician will assist radiologists if they provide similar clinical history and relevant findings. For example, if a radiologist receives a request for a film or scan that marks a patient with HIV, heavy cough, fever, he is told to have a CT scan to assist in the diagnostic evaluation of a sharp recurrence of back pain. Compared to a healthy 75-year-old CT scan, radiologists consider in their diagnostic list a very wide range of possibilities for masses found on abdominal computed tomography (CT) scans. In this example, HIV findings, massive coughing, and fever help shape, expand the diagnosis considered, and assist in the association of pattern recognition.
[0088]
As mentioned above, vision-oriented specialists (dermatologists, radiologists, pathologists, ophthalmologists, endoscopy doctors) spend many years polishing their abilities, diagnosing visual symptoms, Accurately classify and relate to knowledge bases such as standards. In software systems that present a similar knowledge base in a usable manner when there are more than 100 possible visual evidences, we believe that a hierarchical menu system is desirable to facilitate picon exploration. It is desirable for the image representation to evolve from an abstract or general schematic representation to a computerized schematic and even more specific items to photographic icons. In some ways, the most important step is to find the way for the user to follow the correct "branch" of the hierarchy (and to provide enough redundancy to assist the user if the user is traversing an incorrect branch). To provide). The divergence opportunity in such a system is kept on a minimum number of screens, such as lesion type, secondary morphology and lesion composition, and other screens that represent the distribution of lesions on the body.
[0089]
Returning to the example embodiment illustrated in FIG. 13, after selecting "findings" in the pull-down menu bar 288, the user is presented with a higher level list of finding categories as described above. Moving the mouse or pointer to one of the categories in the menu 410 creates an adjacent menu 420 that is displayed in a known manner under the MS-Windows® environment. Finally, the illustrated menu represents a series of selectable observations in menu 430. In the particular example shown, the general "symptoms" under the findings are the "general" classification (menu 420) and finally the selection of menu 430 (cachetic, dehydration, hypothermia, obesity). Refined into a list. In this example, a user selection of "obesity" results in the user interface of FIG.
[0090]
Referring briefly to FIG. 14, a window 450 is now shown in which window 292 has been updated to add the "obesity" finding. In addition, the diagnostic subset displayed in both diagnostic window 294 and image window 334 (region 298) is updated. In accordance with the invention described, additional remarks change the subset of possible diagnoses and move one or more possible diagnoses to the “top” of the list, where the subset is in text form in window 294 and in image form. It is displayed in both windows 334.
[0091]
In a preferred embodiment, a browser-based visual medical diagnostic tool (VisualDx) implemented in accordance with aspects of the present invention is intended to enable both simple queries and complex, multi-view queries. Make exploration as simple as entering a name such as a drug name or a visual image of a related drug rash, or all possible patient findings (symptoms, signs, previous medical history, exposure, medication listings, laboratory findings) , Etc.) is intended to provide maximum flexibility to the user to make it as complex as possible. The user interface as described in accordance with this embodiment is intended for each user to interact with the application in different ways and under different circumstances, and even differently. Application software that drives the user interface and interacts with the knowledge base does not require the user to answer all questions (insights on all findings) to access the knowledge base. Rather, the software provides possible diagnostic information (a subset of the database) as the findings and characteristics are updated or changed by the user. Therefore, the application needs to dynamically display the search results (diagnosis). User input of each search item or patient finding, as illustrated on the user interface screen, results in an immediate relisting and redisplay of the diagnostic list in window 294 and its associated image in window 334.
[0092]
As described, after entering the patient findings, the application program provides the user with a list of possible diagnoses. There are at least three basic options for viewing the images associated with a possible diagnosis:
(A) The user can view the results in a “contact sheet” format as shown in FIGS. 8, 12, and 14, including reduced size image thumbnails of the stack, grouped by diagnostics. This interface represents one of the basic functions of the present invention. Diagnostic groupings of thumbnails, as contrasted with simple image databases or image atlases, allow users to visually scan and review images in terms of diagnostic possibilities. Each diagnosis "stack" allows the user to sort images by "stack" by body location (this facilitates comparison of similar lesions between diagnoses), and all thumbnails for a particular diagnosis are displayed in a separate window As can be seen (eg, FIG. 9), a set of controls that allow the thumbnails to “expand” and display relevant findings of the diagnosis are enabled. The thumbnails are also easily expanded to full screen images with a mouse click, as shown in screen 352 of FIG. According to the image display, the diagnostic image stack may be configured to display a natural progression through the stages of disease progression.
(B) The user can select specific diagnoses from the diagnostics list window, and then compare images from these diagnoses in the view contact sheet and expand the images to full screen if needed.
(C) The user can select one diagnosis and view information from the text window along with the image in both thumbnail and full screen versions.
[0093]
Second, the user may want to access reference material related to the diagnosis. Brief text sentences summarizing the disease progression and providing symptoms and management options are available in the knowledge base and are specifically displayed as shown in FIG. Other reference texts are also available through Internet links such as medications.
[0094]
IV Link between findings and diagnosis
A tightly defined module creates a concise knowledge base with solution problem picklists for the end user. The link between the module and the diagnosis is based on a thorough search of the medical literature and is reviewed (author / editor) by a specialist in the medical field. Diagnosis is possible in any number of modules. This organization of the knowledge base has a cumulative effect. As modules are developed, diagnostic profiles are leveraged into new modules.
[0095]
To identify a diagnosis from a set of findings, the present invention preferably represents the finding-to-diagnosis link by a finding-diagnosis-link (see the diagnosis_finding link table 258 in FIG. 4). The diagnosis_finding link is constituted by the type of the corresponding finding. The examples described herein find a diagnosis_finding link in a finding type table linked to a working set of symptoms, signs, travel history findings, etc., and then generate a diagnostic set. The relationship with the findings is many-to-many in two directions. In other words, a diagnosis can have a number of relevant findings among all the different types of findings, and a particular finding can be found through multiple diagnoses.
[0096]
In the preferred embodiment, the diagnosis_finding link represents the existence of a relationship between the finding and the diagnosis. These relationships can be used to construct a visual representation of the information, or to direct the search for relevant information. Relationships contemplated by the present invention include "associated," "predisposing," "relapse," "exacerbate," "complicated," "cause," "caused by." . These are important in the areas of work and exposure, family and medical history, and medication use. These are mainly applicable in elucidating a diagnosis-to-diagnosis relationship, and are most useful when findings of one diagnosis include others.
[0097]
For example, the "exposure" vs. diagnostic link relationship is exemplified by smoking as a risk factor for lung cancer. In the "type of habit" finding, the "smoking" finding is "predisposing" to the diagnosis of lung cancer. The diagnosis-to-diagnosis link is illustrated by the relationship between lung cancer and pneumonia. Inclusion of lung cancer is pneumonia. From the other side of the relationship, lung cancer predisposes patients to pneumonia. In the “medical history” findings type, findings of pneumonia point to lung cancer as a predisposing factor. Also, lung cancer findings in the "medical history" point to the possible inclusion of pneumonia. The system structured in this way represents a temporal relationship to medication and provides a wealth of connectivity between findings and diagnoses and their associated database objects.
[0098]
In the preferred embodiment, the probability or intensity of the probabilities, based on a statistical or stochastic approach, is not included in the knowledge base to assist the user in reaching a possible diagnosis. The diagnosis_findings link table includes a "diagnosis significance" field to include information about the relative importance of findings in the diagnosis as an aid to the user. In one embodiment, each finding_diagnosis record is classified into three categories at the time of data entry:
-Key or common findings of the diagnosis;
-Minor or uncommon findings
-Case report findings or very rare
[0099]
In addition, a flag may be set to indicate whether these are publicly accepted standards for making a diagnosis. An example of a publicly accepted standard is the American College of Rheumatology standard for diagnosing lupus erythematosus, which is the de facto standard. This very inaccurate "importance" data is not used in the application program to automate the weighting of query results, since the application program is not expected to attempt the diagnosis on its own. Diagnostic significance is the intention to allow the practitioner to re-sort the diagnostic list by applying accepted diagnostic criteria, only the main findings, or the overall findings view, including micro and case report findings . In general, however, this data is provided to inform the end user and to help form an idea of how seriously the presence or absence of a finding needs to be considered when looking at a possible diagnosis. is there.
[0100]
V Diagnosis / Identification
In the VisualDx system, the diagnostic module accepts input from a user and generates a list of diagnostics that match or partially match the input. The search for diagnostics occurs via two basic schemes: a diagnostic query that matches textual findings; and a diagnostic query that matches visual findings.
[0101]
Entry of textual findings is made from a hierarchical picklist in the form of free text entry or, preferably, a drop-down menu presented to the user. Free text entry presents the user with all partial match findings when there is no exact or synonym match. Implemented with respect to the drop-down menu of FIG. 13, and as described above, the findings list may be organized into a hierarchy of classifications to minimize the number of choices presented to the user at one time during the selection process. In one embodiment, the finding classification is determined by reading the hierarchical relationship from the database, and thus may be generated dynamically upon updating the database. If the user specifies an exact value (eg, “creatinine elevation,> 1.8”), certain findings may include or specify relative comparisons of numerical data (eg, “creatinine elevation” levels). Is recognized.
[0102]
Thus, in addition to the drop-down picklists 410, 420, 430 and, in the case of FIG. 13, the summary window 292 of the user interface screen 400, the text finding interface screen includes a free text entry area and other navigation tools. But it is conceivable. Since the characteristics of the findings include visual and textual information, the search may allow any combination of textual and visual findings. As a result of any selection, a search is initiated and a display of matched and partially matched diagnostics is displayed, for example, as shown in FIG. This display is updated whenever you add, change, or remove findings.
[0103]
In one embodiment, images associated with a particular diagnosis are associated as a "stack." The images in the stack are sorted based on view similarity, body location, and lesion type. The user can select which sorting method to use, but the default is used for the active displayed image stack, where the default may or may not be related to the lesion type or other defined findings .
[0104]
Displaying an image of the appropriate diagnosis (or identification in a non-medical embodiment) to suggest is a central objective and focus of the invention, including its knowledge base and its application. The diagnosis may be displayed in text and / or visually. The text is included in the diagnostic record. As described above, visual display of diagnostic information is provided via links to images associated with the knowledge base. In the above example, the diagnosis has a summary painting icon, a close-up view that usually defines the most visually characteristic, an abstract icon or a specified characteristic thumbnail image in situations where multiple diagnostics are arrayed in a menu.
[0105]
The diagnoses may be sorted for display according to user-specified preferences (eg, match or urgency order). These occasionally need to be sorted into a broad category. Under these broad categories, or when search results occur relating to a set of input findings, they need to be sortable by their various attributes. The user may change the sort order in real time to assist in the process of examining the results and refining the search. At a minimum, diagnostics must be sortable by one or more of the following:
-Number of reconciliations (default sort order)
The severity of the disease (emergency or non-emergency)
-Abnormal physiology (infection, malignancy, genetic disease, etc.)
The type of alignment finding
-Disease class
The number or class of inconsistent findings
[0106]
We believe that the default sort order, the number of matching findings, that displays the diagnosis is the best way to show how well the diagnosis fits the current set of findings. Each time an additional finding is specified by the user, the knowledge base query is re-run to update the display and immediately knows how closely each diagnosis of the module matches the selected set of findings I can do it.
[0107]
The preferred embodiment allows for a natural process of displaying the set of findings associated with the diagnosis, since non-coincidence findings contain crucial information in the diagnostic process. For example, FIG. 14 illustrates the findings (in patient window 292) corresponding to the particular diagnosis indicated. This allows the user to determine which findings were not selected or indicated in the initial search for a particular diagnosis.
[0108]
The vastness of the field of medical knowledge and the specialization of physicians has mandated the creation of applications focused on the knowledge base structure described herein. Diagnosis is categorized as an attachment to a problem-oriented module so that a focused subset of the knowledge base can be quickly viewed by the user. Other possible modules include:
-Easy eye and eye guidance for general physicians
-Simple ENT guidance for general physicians
-Pediatrician:
・ Newborn skin diseases and visual guidance for premature babies
・ Infant skin disease
・ Congenital abnormality (structure) noticed at birth
・ Generalized rash and rash in infants and children
・ Localized skin problems and children's products
・ Scalp and hair disorders (from infants to children)
・ Hemangiomas and vascular diseases in children
・ Nail problems
・ Local skin problems including genital lesions
-Geriatrician
・ Aging skin problem
・ Skin ulcer and skin care
−Genetics
・ Congenital syndrome Children and adults
・ Cranial abnormalities
-Occupational medicine
・ Contact dermatitis (occupation-related skin allergy)
・ Conduct of arts / sports medicine
-HIV
・ HIV and AIDS skin problems
-Nevus
・ Hemangiomas (before and after)
・ Melanin nevus (before and after surgery for patient information)
-Wound care
·burn
·injury
・ Cutting surgery care
・ Trauma
-Infectious diseases
・ Travel medical care (country-specific guidance for skin and infectious diseases prevalent in each country)
・ Male genital lesions including sexually transmitted diseases
・ Female genital lesions including sexually transmitted diseases
-Diabetes
・ Diabetes ulcer and skin care
-Adult dermatology for general medicine
・ Subcutaneous and epidermal pigmentation diseases (flat lesions)
・ Papules and plaque (elevated lesions)
-Ulcers and atrophy (hollow lesions)
・ Vesicles and blister disease (bullous lesions)
・ Suppuration and bleeding lesions
・ Individual lesions or tumors
・ Skin manifestation of internal disease
・ Ethnic diseases (hereditary skin diseases)
・ Nail disease-Adult
・ Ethnic skin disease
・ Acne and related diseases
・ Ostomy surgery care
・ Hair and scalp problems
-Environmental exposure
・ Body penetration and bite (images of microorganisms, spiders, etc.)
・ Marine and water injuries (microorganism images)
・ Plant dermatology (image of plant causing disease)
− Body area specific
・ Palm and heel disorders / foot disorders
・ Oral mucosal lesion including tongue
・ Intertribe (armpit hair, groin, lower breast)
・ Head and neck tumors
・ Face rash
-Patients at special risk
・ Transplantation and immunocompromised patients
-Forensic
-Autopsy / Cause of death
・ Criminal scene analysis
[0109]
VI Another Example
While the details of the invention have been described with reference to a visual medical diagnostic system (VisualDx), particularly an adult dermatology embodiment, the following description is directed to another embodiment in which the invention or features thereof find particular application. Is pointed.
[0110]
Returning to FIG. 16, there is illustrated another special embodiment contemplated by the present invention, an application called Drug Identification (PiIIID).
[0111]
Again, it will be appreciated that one or more features of drug identification, or other embodiments described herein, may be implemented with a similar hierarchical menu / list methodology. For example, it is possible to implement the following hierarchy for display to the user to help identify street or other dangerous drugs:
Top level menu list:
Form, administration method, mark, color, shape, overall weight, location of use, user symptoms, major chemical composition
Second level menu list
Format->
Powder, capsule, tablet, liquid, patch, gas
Administration method->
Oral, injection, epidermal, smoking / inhalation
[0112]
Referring to FIG. 16, various elements of the user interface screen 550 are functionally consistent with those described above, and a series of images 570 are also shown, where each image has been entered or selected by the user. Figure 3 illustrates possible matching agents based on characteristics. Each window in the area 298 further includes not only the image 572 but also a stack development button 574 and a text display button 576. Selection of the display stack button 574 under any particular image will generate the image in the “stack” being displayed on the interface as described above in connection with FIG. 9 of the VisualDx embodiment. Selection of the display text button 576 creates a text window with descriptive details and other information related to the particular drug. Beneath each image are illustrated stack navigation buttons 578 and 580 that can be used to move back and forth through the stack of images of the particular drug. For example, a stack of images may include different brands of medication to improve the likelihood of a user being able to identify a particular medication, as well as different views (back, side, etc.).
[0113]
In one embodiment, it should be appreciated that whatever the figure being presented, it is possible to double-click the image to present the user with a full screen, high resolution view of that particular image. Subsequent clicks or selections of the full screen image will return the user to the previous view or screen. Alternatively, the navigation arrows allow the user to continuously view high resolution images in full screen mode.
[0114]
Continuing with the reference of FIG. 16, the system illustrated by the user interface screen 550 is used for cross-reference access to the image and knowledge databases to assist in the identification of misuse and other drugs. Those familiar with digital imaging technology will recognize that such systems require standardized or controlled systems for review of image and visual information. In particular, the system includes an imaging station where the drug sample or sample is located in a cabinet or container having a sample table or table, lighting and high resolution, color digital image capture devices (digital cameras, CCD scanners, etc.). May be. The system may take the form of the characterization peripheral shown in FIG.
[0115]
One aspect of the present invention contemplates a system for cross-referencing access to image and knowledge databases to assist in identifying street drugs. As described above, the system includes a user interface for requesting a plurality of characteristics of the sample from a user. For example, properties include drug form (powder, capsule, tablet, liquid, patch, gas), method of administration (if known) (oral, injection, epidermis, smoking / inhalation), mark, color, shape, overall weight ( mg), dimensions (mm), geographical location of use, user symptoms, major chemical components (Marquis test, Mandelins test, Liebermans test) Determined by drug testing). Based on the characteristics indicated by the user, a diagnostic engine similar to that used in the VisualDx system identifies a subset of street or other dangerous drugs that match the characteristics from a plurality of possible misused drugs. Using a subset of misused drugs, the information space of the image database is sorted for presentation to the user, where presentation is performed by simultaneous display of images for user review in identifying misused drugs. .
[0116]
In the case of abused drugs such as pills, the image is photographed in standard size against a defined background, possibly containing a 2.5 x 2.5 cm grid printed to help the observer assess the size of the drug. It is further conceivable to be taken (in the case of a pill). This standard reference image is uniformly dispensed in the lower right corner so that the user has an immediate standard visual reference for comparison. Textual information about the product or chemical name or ingredient, symptoms, treatment (if any), and other tests that identify the substance also appears in response to user selection on the interface screen.
[0117]
In addition, a second feature of the present embodiment is that the database will be strengthened by collecting information from users of the drug identification system in the future (the situation of illegal drugs is constantly changing). This group of users (forensic chemists or other crime squads, drug enforcement investigators, and other public safety or police officers) have image terminals / image units or have standard photographic guidelines and techniques to acquire images. (Standard grid, lighting, background, position on grid, distance from object). Thereafter, images acquired using the system or the method may include other property details (weight (mg), dimensions (mm), laboratory analysis results, date items received (eg, date and time stamp), quantity obtained, Along with location (jurisdiction), product name, and listed contents (from the label if in the container), they are sent electronically to the central image database. The central database is also a searchable database in a manner similar to the invention described above (adding geographic searches and displays, for example, to track the spread of a particular drug from site to site).
[0118]
Further, in addition to providing the images from the database, the system may also include means for comparing the actual sample image information side by side with that of the database image information. As noted above, samples or sample data may be added to keep the information held up. For example, a drug abuse that was previously unknown in one region of the country (eg, Cincinnati, Ohio or Rochester, New York) could be viewed as similar to a drug manufactured in another region (eg, Toronto, Canada). Identified by characteristics. Alternatively, unknown misused drugs may be added to the database once the analysis including the chemical composition is completed.
[0119]
The preferred system utilizes a user interface in which the user enters descriptive properties of the sample via a series of pull-down menus as illustrated in menu bar 288 of FIG. The properties include form (powder, capsule, tablet, liquid, patch, gas), mode of administration (oral, injection, epidermis, smoking / inhalation), color, geographical location of use, user symptoms, and chemical composition . Once entered, a diagnostic engine within the system processes the entered information to identify a subset of the misused drugs that match the characteristics entered by the user from a plurality of possible misused drugs. This subset is then used to reconstruct the information space of the image database for display to the user via the user interface, where the display is performed by the simultaneous display of images for user review during the identification process. You.
[0120]
The information gathered by the system will be used by Criminology Institute to add information on the clear identification of the chemical nature of the drug to help emergency personnel and first responders (or campsite personnel, schools and other relevant personnel). It can be expected to be shared by law enforcement bureaus. The system tracks the geographic spread of new illicit drugs and believes it will aid in source identification (Los Angeles, New York, Canada).
[0121]
As an example, a future capture of a photo of a drug and its logo or mark (linked to a database containing other information such as volume, location, identity, capture environment) can be performed in police jurisdictions. . The identity (the chemical nature of the drug) is then confirmed at the Criminological Institute. The database grows continuously and is available through multiple jurisdictions. Such information can be shared with other states, countries or international law enforcement agencies via secure network connections, so that warnings about the release and spread of new hazardous drugs will be more timely.
[0122]
In yet another embodiment, it is believed that one or more features of the present invention are applicable for use by a coroner or an inspector in investigating causes of death. Here again, the system is used in conjunction with traditional equipment and technology to provide the coroner or the inspector with a variety of resources to assist with the investigation. In particular, the images are used at the time of necropsy to examine the gross and whole external images including microscopic pathology, trauma, causes of natural death, signs of bruising, the whole internal images of diseased organs and wounds, and microscopic images of diseased tissue. In implementing such a system, the property search fields include trauma type-blunt, cut, gunshot, etc., laboratory information-drug screen, chemistry, blood count, and other information such as dentist records. Including.
[0123]
In particular, a system for cross-referencing access to images and knowledge databases to assist in investigating causes of death includes a user interface that requests multiple characteristics of the cause of death. Other characteristics include methods of death (accidents, nature, allegations), weapon types (for guns such as handguns, shotgun rifles, homemade, assault weapons, machine guns), secondary injuries (burns in close proximity to shooting, etc.), Includes wound iconic representations (hole dimensions, cut patterns) modalities (devices or objects associated with death, such as guns, trains, fires), and medical dictionaries (acute, chronic, obese). Based on the characteristics, a diagnostic engine is used to identify a subset of the causes of death that match the characteristics from a plurality of possible causes of death. The image database is then reconfigured for display to the user using the subset of causes, wherein the display is performed by simultaneous display of a plurality of images for user review in identifying the cause of death.
[0124]
As in the previous embodiment, once the user has entered the data, the entered characteristics are processed using a pre-programmed diagnostic engine, and a plurality of possible causes of death are used to enter the entered characteristics and description. Identify a subset of matched causes. Using the subset of causes, the information space of the image database is reconstructed for display to the user, where the display is performed through simultaneous display of images for user review when determining the cause of death.
[0125]
Yet another embodiment contemplated by the present invention is a system for cross-referencing access to images and knowledge databases to assist in the identification of plants, fungi or other organisms (eg, reptiles, arthropods, etc.). For plants and fungi, the system consists of dimensions, shapes (leaves, seeds / berries, flowers, fruits, etc.), vein patterns, coloring (leaves, stems, roots, autumn coloring), stem types (trees, vines), Includes a user interface that requests from the user a number of descriptive properties of the plant sample (including fungi such as mushrooms), including where they were found. Once such information has been collected in a manner consistent with standard nomenclature, a diagnostic engine is used to identify a subset of plants from the plurality of possible plants that match the input characteristics. A subset of plants is then selected from the database and displayed in the information space, where the display is performed by simultaneous display of a plurality of images for user review to identify the plants.
[0126]
Another example where the features of the present invention find utility is in the diagnosis and treatment of plant and garden problems. For example, people often do not understand the causes of changes in the appearance of healthy plants, lawns, decorative shrubs or trees. A combination of text and visual evidence can also be used to help the person identify the source of change and resolve the problem.
[0127]
As mentioned above, the system has particular application in human dermatology. However, there is also a special application in veterinary dermatology, in which case the invention can be used for the diagnosis of animal skin and subcutaneous signs of infectious and systemic diseases. Accident investigation (aviation, automotive, military) is also supported using the present invention, where patterns or characteristics of automobile accidents (eg, recent tire safety issues, for example) can be used to arrive at an accident cause determination. . Factors such as skid marks, damage to vehicles, damage to occupants, and other factors can be used to assemble a case with a stack of images.
[0128]
Or within the aviation industry, the exact cause of an accident is not immediately known, classifying evidence of metal fatigue, and metadata and related damaged parts and accident details allow for a more efficient review of "visual evidence" I do. As with medical diagnostics, ambiguity and uncertainty are rules, and so "cases" can be similarly assembled to help the user identify the cause of the accident and reach final conclusions. In a similar manner, police crime scene investigations can be facilitated by the features of the present invention. In criminal work, the criminal assembles all evidence of the crime and associates visual information (footprints, fingerprints, signs of intrusion, pictures of the suspect) with the textual data of the crime, location, time, victim, etc. The details of the crime are then accurately categorized and accessed using the present invention.
[0129]
In summary, the present invention is a method and apparatus for enhancing the usefulness of visual knowledge in a number of applications. This is a suitable method to derive the relationship between properties and hypotheses into a database format, thereby assisting the user in investigating various hypotheses (drug identification, plant / animal identification, cause of death, cause of accident, etc.). Compose visual information. The present invention has enormous potential because it avoids the open problems around knowledge engineering by not automating the decision making process. Rather, the present invention is a support tool that assists the user in examining and reaching reasoned conclusions based on information available by direct observation and comparison of stored images with textual data.
[0130]
Thus, the present invention provides a cross-reference knowledge and image database method and apparatus, wherein multiple hypotheses can be used to display at least in an image-centric format for further review by a user. Obviously, it is made narrower to at least one subset of the identification. Although the present invention has been described in connection with a preferred embodiment, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. It is therefore intended to cover all such alternatives, modifications and changes as fall within the spirit and scope of the appended claims.
[Brief description of the drawings]
FIG.
1 is an exemplary block diagram illustrating an embodiment of a system in which the present invention operates.
FIG. 2
5 is a flowchart illustrating the general steps of making and using an embodiment of the present invention.
FIG. 3 to FIG. 4
4 is an illustrative example of a database schema that may be used in implementing an embodiment of the present invention.
FIG. 5 to FIG.
4 is a diagram illustrating an example of a user interface of a visual diagnostic device implementing various features and functions of the present invention according to an adult dermatology embodiment.
FIG.
FIG. 10 is an illustrative example of a user interface of a medicine identification device for explaining another embodiment of the present invention.

Claims (30)

In a system that supports the visual diagnosis process,
An image database,
A knowledge database cross-referenced to the image database to support a diagnostic process;
A user interface for requesting a plurality of descriptive properties of the sample requiring diagnosis from a user;
A diagnostic engine responsive to the characteristic, wherein the characteristic of the sample is used by the engine to identify, from a plurality of possible diagnostics, a subset of diagnostics that match the characteristic;
Including
Reconstructing the information space of the image database for simultaneous display of a plurality of images for user review via a user interface using a subset of the diagnostics;
A system that supports the visual diagnosis process. The system of claim 1, wherein the diagnostic engine dynamically operates to reconstruct an information space using a subset of diagnostics upon modification of at least one of the plurality of explanatory characteristics. In a method for assisting a visual diagnostic process,
Creating an image database from a collection of images related to a particular subject matter;
Creating a knowledge database of other data related to a particular subject matter, wherein the knowledge database creates the knowledge database cross-referenced to the image database to support a diagnostic process. When,
Collecting from the user a plurality of descriptive characteristics of the sample requiring diagnosis through a user interface adapted to the particular subject matter;
Responsive to the explanatory characteristics, identifying a subset of diagnoses from the plurality of possible diagnoses included in the knowledge database that match the explanatory characteristics collected from the user;
Reconstructing the information space of the image database for simultaneous display of a plurality of images associated with descriptive characteristics for user review via a user interface using a subset of the diagnostics. How to help. 4. The method of claim 3, wherein the diagnostic engine dynamically operates to reconstruct the information space using a subset of diagnostics upon user modification of at least one of the plurality of explanatory characteristics. In a system for reducing diagnostic uncertainty using cross-reference knowledge and an image database,
A user interface for requesting multiple characteristics of the diagnosis from the user;
A diagnostic engine, wherein the diagnostic engine identifies the subset of diagnostics that match the characteristic from a plurality of possible diagnoses using the characteristic of the diagnostic;
Including
Using a subset of the diagnostics to reconstruct the information space of the image database for display to the user, wherein the display is performed by simultaneous display of multiple images for user review;
System for reducing diagnostic uncertainty using cross-reference knowledge and image databases. The system of claim 5, wherein the plurality of images are presented as a diagnostic image stack. 7. The system of claim 6, wherein the diagnostic image stack comprises:
A subset of the plurality of images, each image of the subset being associated with a common diagnosis;
An index to the subset of images, the index to the subset independent of a common diagnosis;
Including the system. 7. The system of claim 6, wherein the diagnostic image stack is displayed to illustrate stages of disease progression. 7. The system of claim 6, wherein the diagnostic image stack is displayed as illustrating a plurality of images associated with a particular diagnosis. 6. The system of claim 5, wherein when the user selects a portion of the displayed image, the at least one image presented to the user includes an indication of relevant characteristics of the diagnosis. 6. The system of claim 5, wherein the display to the user is performed via a display, wherein the display simultaneously displays textual information associated with at least one subset of the diagnostics. 6. The system of claim 5, wherein the diagnostic engine performs pattern recognition operations on the knowledge database using characteristics of the diagnostics and identifies the diagnostics by matching characteristics. 6. The system of claim 5, wherein the system for reducing diagnostic uncertainty is applicable to and includes characteristics of a disease having a dermatological manifestation. 6. The system of claim 5, wherein the system for reducing diagnostic uncertainty is applicable to and includes a disease characteristic that is of a visual finding type that is visible to the unsupported human eye. 6. The system of claim 5, wherein the system for reducing diagnostic uncertainty is applicable to and includes a disease characteristic determined based on findings determined by a mechanical testing device. The system of claim 5, wherein the user interface requesting a plurality of characteristics includes at least one symptom displayed as an icon. 17. The system of claim 16, wherein the icon is an image illustrating a type of dermatological lesion. 17. The system of claim 16, wherein the icon is an image illustrating the distribution of a dermatological lesion on a patient's body. 6. The system of claim 5, wherein the system for reducing diagnostic uncertainty is applicable to and includes oral drug properties. 10. The system according to claim 9, wherein the icon display is an image illustrating the shape of the oral drug. 20. The system of claim 19, wherein the iconic representation is an image illustrating the color of the oral medication. 6. The system of claim 5, wherein the system for reducing diagnostic uncertainty is applicable and includes characteristics determined at necropsy. 6. The system of claim 5, wherein the system for reducing diagnostic uncertainty is applicable to and includes crime scene characteristics. 6. The system of claim 5, wherein the plurality of diagnostic characteristics are selected from a group consisting of:
Patient medical history Patient symptoms Patient drug system. A system for supporting identification of orally administered drugs using cross-reference knowledge and an image database, comprising:
A user interface requesting a plurality of orally administered drug characteristics;
A diagnostic engine that uses the characteristics to identify a subset of identities that match the characteristics from a plurality of possible identities;
Including
Using a subset of the identification, reconstruct the information space of the image database for display to the user, where the display is performed via simultaneous display of multiple images of the orally administered drug for user review as an image stack Done,
A system that supports the identification of orally administered drugs using cross-reference knowledge and an image database. 26. The system of claim 25, wherein the tablet characteristics include at least one selected from the group consisting of:
color,
mark,
shape,
Size,
system. A system for cross-reference access to image and knowledge databases to assist in street drug identification,
Request multiple properties of a sample from a user, including one of a group of properties consisting of format, mode of administration, mark, color, geographical location of use, user symptoms, and chemical composition A user interface,
A diagnostic engine, wherein the diagnostic engine uses the characteristics of the sample to identify a subset of misused drugs that match the characteristics from a plurality of possible misused drugs;
Including
Using a subset of misused drugs to reconstruct the information space of the image database for display to the user, where the display is performed through simultaneous display of multiple images for user review to identify misused drugs Done,
A system for cross-reference access to image and knowledge databases to assist in identifying misused drugs. In a system for cross-reference access to image and knowledge databases to assist in investigating causes of death,
A user interface requesting a plurality of characteristics of the cause of death, including one of a group of characteristics consisting of aspects of death, type of wound, type of accessory wound, modality, and medical dictionary;
A diagnostic engine that uses the characteristic of the cause of death to identify a subset of causes that match the characteristic from a plurality of possible causes of death;
Including
Use the subset of causes to reconstruct the information space of the image database for display to the user, where the display is performed through the simultaneous display of multiple images for user review to identify the cause of death ,
A system for cross-reference access to image and knowledge databases to assist in investigating causes of death. A system for cross-referencing access to an image and knowledge database to assist in plant identification,
A user interface for requesting multiple descriptive properties of the plant sample from a user;
A diagnostic engine, wherein the diagnostic engine uses the property of the plant sample to identify, from a plurality of possible plants, a subset of plants that match the property;
Including
Using a subset of the plants to reconstruct the information space of the image database for display to the user, wherein the display is performed through the simultaneous display of multiple images for user review to identify the plants;
A system for cross-reference access to image and knowledge databases to assist in plant identification. 30. The system of claim 29, wherein at least one of the explanatory characteristics is selected from the group consisting of:
Size,
Leaf shape,
Leaf dimensions,
Leaf vein pattern,
Coloring,
Stem type,
The geographic location that was discovered,
System.

Images