JPH06251005A - Report information input/output system - Google Patents

Abstract

PURPOSE:To enable a user to intuitively recognize picture information by visu ally displaying out picture information of a report or the like as characters, a list, a graphic, notes, or their combination. CONSTITUTION:This report information input/output consist of a report information input means 10 which takes a displayed medium as an input medium as it is and directly inputs a voice and pointing data as report information to generate an input report, an input report analysis means 20 which consists of a voice information analysis means 121 and a pointing data information analysis means 125 and analyzes the input report, an overlay generating means 60 which generates an overlay, and a report output means 80'' which generates and outputs a report as the combination of the picture and the overlay.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information input / output system for converting input information and visually displaying it.
In particular, the present invention relates to a report information input / output system for inputting medical report information in a medical image communication information system and visually outputting medical information as a report by conversion processing of the input information.

[0002]

2. Description of the Related Art In a medical image communication information system, an image-reading doctor reads an image-captured image and inputs a report, and a reference doctor later reads out or takes out the image-interpretation report (in the case of a memo) to obtain the image-captured image. It is referred to along with the report and case image and used for diagnosis. In this case, conventionally, the image interpretation report is displayed as character information printed by a printer, character information output and displayed on a display device such as a display, or handwritten characters or graphics (figure) written as a memo. It had been.

Further, as a method of inputting to a workstation, handwritten characters and graphics (figures) are viewed and characters are input from a keyboard, figures are input from a figure input device such as a digitizer, or characters and figures are input from a tablet. There was something to enter directly.

Further, a method of recording a voice report (such as a dictation by voice) on a recording tape, listening to the type, and typing to create a report is mainly performed in the United States.

As a reference, on page 231 of the special issue of "Medical Electronics and Biotechnology" Vol. 28, Special Issue (1990), sketches by Noriko Inaoka, Hideo Suzuki, Hiroshi Takahata and Masaki Mori are automatically added. There is an introductory article on the prototype of a device (interpretation and diagnostic finding description support system) that inputs and describes characters (such as findings) on it, extracts features by image processing, and creates a report. On page S285 of the special edition of "Ikuhokai", Vol. 50, February, there is a report article by Tadayuki Maehara, Yasushi Katsumata, Masanobu Noda, Ichiro Shiramizu and Noboru Sugamura on a voice input word processor for head CT reports. Yes, on page S285 of the same magazine, the current image diagnostic report generator (voice reporter) using a voice recognition device by Kenkyu Honda, Kikuo Machida, Toshio Mamiya, Taku Takahashi, Teruo Takishima, Tsuyoshi Kamano, Satoshi Tamaki. , A report article, is about.

[0006]

However, the medical report according to the above-mentioned conventional technique is (on the display)
Since it is displayed or output (from the printer) only as text information, for a doctor who wants to diagnose a photographed image, what (the symptom on the image) can be diagnosed just by looking at the interpretation report of the text information. Therefore, it is not possible to immediately understand what (what kind of symptoms on the image) was not diagnosed, which part of the image shows detailed symptoms, and even if it is limited to one patient, For a reference doctor who has to make a diagnosis while referring to a series of (several number of) images and an image interpretation report, it is a burden for the reference doctor to make a diagnosis while carefully reading the character report and comparing the image and the character report. It is inconvenient to make an accurate diagnosis for a large number of patients within a limited time.

The present invention has been made in view of the above-mentioned inconveniences, and a visual image is directly displayed as image information such as a report as a character, a list, a figure, or an annotation, or a combination thereof, so that a direct image can be obtained. It is an object to provide a report information input / output system capable of grasping information (report, etc.).

[0008]

In order to achieve the above object, a report information input / output system as a first invention of the present invention comprises a report information input means for inputting report information and creating an input report. And input report analysis means for analyzing the input report, and report output means for creating and outputting an output report based on the analysis result of the input report and the analysis means for the input report.

Further, in the first invention, the report information input means includes a report creating means for creating an input report as character information including a character string, and the input report analyzing means analyzes the meaning of the context of the character information. The report output means may include the information context analysis means and the list creation means for creating a list based on the analysis result by the character information context analysis means, and the report output means may create and output the output report as a combination of the word information and the list.

Further, in the above-mentioned first invention, the input report analyzing means comprises a character information context analyzing means, a graphic creating means and a graphic editing means for creating a graphic based on an analysis result by the character information context analyzing means, The report output means may be configured to generate and output the output report as a combination of character information and graphics.

Furthermore, in the first aspect of the present invention, the report information input means includes a report creating means for creating an input report as a combination of character information including a character string and graphic information, and the input report analyzing means includes a context of the character information. Character information context analyzing means for analyzing the meaning of, a graphic information analyzing means for analyzing graphic information, a first list creating means for creating a first list based on an analysis result by the character information context analyzing means, and a graphic A second list creating means for creating a second list based on the analysis result by the information analyzing means, wherein the report output means combines the output report with character information, graphics, the first list, and the second list. Can be created and output as.

A report information input / output system as a second invention of the present invention is a report information input means for inputting report information to create an input report, an input report analysis means for analyzing the input report, and an image storage means. An image retrieval means for retrieving an image from the image storage means, an image analysis means for analyzing an image obtained by retrieval, an input report, an analysis result by the input report analysis means, and an analysis result by the image analysis means. It is characterized by having an overlay creating means for creating an overlay by means of the above, and a report output means for creating and outputting a report as a combination of the searched image and overlay.

In the second invention, the input report analysis means creates a list based on the character information context analysis means for analyzing the meaning of the context of the character information and the analysis result by the character information context analysis means. It may include a creating means.

Further, in the above-mentioned second invention, the input report analyzing means comprises a character information context analyzing means, a graphic creating means and a graphic editing means for creating a graphic based on an analysis result by the character information context analyzing means, The report output means may be configured to generate and output the output report as a combination of character information and graphics.

Furthermore, in the above-mentioned second invention, the report information input means includes a report creating means for creating an input report as a combination of character information including a character string and graphic information, and the input report analyzing means includes the character information. Character information context analyzing means for analyzing meaning of context, graphic information analyzing means for analyzing graphic information, first list creating means for creating a first list based on an analysis result by the character information context analyzing means, and graphic Second list creating means for creating a second list based on the analysis result by the information analyzing means, and the report output means can be configured to create and output an output report as a combination of the searched image and the overlay. .

In the second invention, it is preferable that the image analysis means includes a contour extraction means for extracting the contour of the image and a pattern recognition means for recognizing the extracted contour pattern.

A report information input / output system as a third invention of the present invention is the report information input / output system according to the first invention, wherein the report information has an overlay creating means, and the report information is time-synchronized audio information, an image, and pointing information. Report information input means, pointing data creating means for creating pointing data from the pointing information, audio data creating means for creating audio data based on audio information synchronized with the pointing information, image data, pointing data And input report data creating means for creating input report data as a combination of voice data, wherein the input report analyzing means analyzes voice information from the input report data and converts it into character information, and input Pointing information from report data Pointing information analysis means for discriminating between the graphic data and the voice input data, the overlay creating means creates an overlay from the outputs of the voice information analyzing means and the pointing information analyzing means, and the report output means outputs the overlay. The report is created and output as a combination of a character overlay, a graphic overlay and an image.

The report information input / output system as a third invention is the report information input / output system according to the second invention, wherein the report information includes time-synchronized audio information, an image, and pointing information. , Pointing data creating means for creating pointing data from the pointing information, audio data creating means for creating audio data based on audio information synchronized with the pointing information, and input report as a combination of image data, pointing data and audio data Input report data creating means for creating data, wherein the input report analyzing means analyzes the voice information from the input report data and converts it into character information, and the pointing information from the input report data. Graphic data and audio input data Pointing information analysis means for discriminating between the input and output data, the overlay creation means creates an overlay from the outputs of the voice information analysis means and the pointing information analysis means, and the report output means includes a character overlay and a graphic overlay for the output report. And creating and outputting as a combination of images.
In this case, it is preferable that the image analysis unit includes a contour extraction unit that extracts the contour of the image and a pattern recognition unit that recognizes the pattern of the extracted contour.

[0019]

In the report information input / output system as the first invention of the present invention having the above structure, an input report is created from the report information input by the report information input means, the input report analysis means analyzes the input report, and the report is input. The output means creates and outputs an output report based on the input report and the analysis result of the input report analysis means.

In the first aspect of the invention, the report information inputting means creates the input report as character information including a character string by the report creating means, and the input report analyzing means uses the character information context analyzing means to convert the character information. The meaning of the context is analyzed, the list creating means creates a list based on the analysis result, and the report output means creates and outputs the output report as a combination of word information and the list.

Further, in the first invention, the input report analysis means analyzes the meaning of the context of the character information by the character information context analysis means, and based on the analysis result, the graphic is generated by the graphic creation means and the graphic editing means. The report output means can create and output an output report as a combination of character information and graphics.

Furthermore, in the first aspect of the invention, the report information input means creates the input report by the report creating means as a combination of character information including a character string and graphic information, and the input report analyzing means analyzes the character information context analysis. By analyzing the meaning of the context of the character information by means, analyze the graphic information by graphic information analysis means,
The first list creating means creates a first list based on the analysis result by the character information context analyzing means, the second list creating means creates a second list based on the analysis result by the graphic information analyzing means, and reports The output means can create and output the output report as a combination of character information, graphics, the first list, and the second list.

In the report information input / output system as the second aspect of the present invention, the report information input means creates an input report from the input report information, the input report analysis means analyzes the input report, and the image search means is provided. Retrieves the image from the image storage means, analyzes the image obtained by the image analysis means, and the overlay creation means compares the input report and the analysis result of the input report analysis means with the analysis result of the image analysis means. An overlay is created, and the report output means creates and outputs a report as a combination of the searched image and overlay.

In the second aspect of the invention, the input report analysis means may analyze the meaning of the context by the character information context analysis means and create the list by the list creation means based on the analysis result.

Further, in the second invention, the input report analysis means analyzes the context by the character information context analysis means, creates a graphic by the graphic creation means and the graphic editing means based on the analysis result, and outputs the report. The means can generate and output the output report as a combination of character information and graphics.

Furthermore, in the second invention, the report information input means creates the input report by the report creating means as a combination of character information including a character string and graphic information, and the input report analyzing means causes the character information context. The analyzing means analyzes the meaning of the context of the character information, the graphic information analyzing means analyzes the graphic information, and the first list creating means creates a first list based on the analysis result by the character information context analyzing means. The 2 list creation means can create a second list based on the analysis result by the graphic information analysis means, and the report output means can create and output an output report as a combination of the searched image and the overlay.

In the second aspect of the invention, the image analysis means extracts the contour of the image by the contour extraction means and recognizes the contour pattern extracted by the pattern recognition means.

In the report information input / output system as the third invention of the present invention, the report information input means creates pointing data from the pointing information by the pointing data creating means, and synchronizes with the pointing information by the voice data creating means. The voice data is created based on the voice information, and the input report data creating means creates the input report data as a combination of the image data, the pointing data and the voice data. And
The input report analysis means analyzes the voice information from the input report data by the pointing information analysis means and converts it to character information, and the pointing information from the input report data to discriminate the graphic data from the voice input data. The overlay creating means creates an overlay from the outputs of the voice information analyzing means and the pointing information analyzing means, and the report output means creates and outputs the output report as a combination of a character overlay, a graphic overlay and an image.

[0029]

1 is a block diagram showing the configuration of the first invention of the report information input / output system of the present invention, 10 is a report information input means for inputting report information and creating an input report, 20 is a An input report analysis means for analyzing the created input report, and 80 is a report output means for creating and outputting an output report based on the input report and the analysis result by the input report analysis means 20.

FIG. 2 is a block diagram showing the configuration of the second invention of the report information input / output system of the present invention.
Is a report information input means for inputting report information to create an input report, 20 is an input report analysis means for analyzing the input report, 30 is an image storage means such as a database, and 40 is from the image storage means 30. An image retrieval unit for retrieving an image, 50 an image analysis unit for analyzing an image obtained by retrieval, and 60 an overlay comparing the input report and the analysis result of the input report analyzing unit 20 with the analysis result of the image analyzing unit 50. And 80 'is a report output means for creating and outputting a report as a combination of the searched image and overlay.

FIG. 21 is a block diagram showing the configuration of the third invention of the report information input / output system of the present invention. Reference numeral 10 designates the displayed medium as the input medium as it is, and outputs the voice and the report information. Report information input means for inputting pointing data directly to create an input report, 20 is an input report analysis means for analyzing an input report consisting of voice information analysis means 121 and pointing data information analysis means 125, and 60 is an overlay. The overlay creating means 80 ″ is a report outputting means for creating and outputting a report as a combination of an image and an overlay.

Examples of the report information input / output system according to the present invention, which are based on the first basic form, will be described below as examples of application in a medical image communication system.
The second embodiment of the report information input / output system according to the present invention described in Section 3
-5, and another embodiment of the report information input / output system of the present invention will be described in a third embodiment. The symbols used in the following description of the embodiments are the same for the same components.

<Embodiment 1-1> FIG. 3 is a block diagram showing the configuration of the first embodiment of the first invention shown in FIG. 1. Report information input means 10 inputs handwritten characters. It has a tablet 11 or a digitizer 12 for coding and inputting characters, and a character recognition unit 16 as a report creating means. Also, the input report analysis means 20
Has a context analysis part 21 and a list creation means 23. The context analysis part 21 analyzes the input report from the report information input means 10 to obtain an analysis result, and the list creation means 23 provides a list (table ) Is created. Then, the report output means 81 displays the report in characters and a list (table).
Output to the printer 91, or
It is displayed on the display device (display) 92.

FIG. 4 is a block diagram showing a configuration example of the input report analysis means 20. In FIG. 4, 210 is a context semantic analysis unit, 220 is a semantic dictionary, 230 is a semantic frame, 240 is an inference / learning unit, and 250 is a learning database. In addition, the context semantic analysis unit 210 interprets the meaning of a word by matching the divided unit 211 that divides a character string with a significant word (word) and the separated word and the meaning dictionary 220, and outputs the meaning frame 230. Semantic analysis unit 21
2, the inference / learning unit 240 uses the meaning frame 230
Error content, lack of information, common sense, default value,
Interpolation inference unit 24 that infers and interpolates from the input habits of the operator
1 and a learning unit 242 that learns the input habit of the operator and registers (or searches for learning) the learning database.

As the tablet 11 for the user to input handwritten characters, for example, many products such as Wacom Handwriting Workstation (product name) are known, and handwritten characters directly input using the tablet 11 are data input. Character information 71 encoded (binarized) by a well-known method in the unit 15 and including a character string in the character recognition unit 16
(See FIG. 7), and are sequentially stored in the storage unit 17. In addition, the user inputs memos and printed materials, etc. Report (information)
When inputting as, the digitizer 12 digitizes (codes) the characters written on them and the character recognition unit 1
The character information including the character string is set in 6 and sequentially stored in the storage unit 17. Next, the character information stored in the storage unit 17 is shown in FIG.
The context analysis means 21 of the input report analysis means 20 shown in FIG.
As described above, after being read out and divided into words as described above, they are given meanings and output as analysis results. For example, if the input character information is "... no more heart and ribs ..." and "Pneumonia in lung field ...", an analysis result 51 as shown in FIG. 5 is obtained.

The list creating means 23 creates a list based on the context analysis result. The list is, as shown in Figure 6,
The system (for example, the storage unit 17 or the like) has a site name / diagnosis result correspondence table 61 which is a correspondence table of the site name and the diagnosis result, and matches the context analysis result 51 ′ with the site name / diagnosis result correspondence table 61. A list 62 is created by comparing the site name in the context with the site name in the site name / diagnosis result correspondence table 61, and writing the diagnostic result in the context for the matching one in the result column of the correspondence table.

For example, in FIG. 6, the site name in the site name / diagnosis result correspondence table 61 that matches all site names in the analysis result 51 'is searched. First, when the site name "right lung field" is searched from the site name / diagnosis result correspondence table 61, it is found that it exists in the first line. Therefore, "pneumonia" which is the context analysis result is written in the result column beside it. Such an operation is repeated. If there is no result that applies to the site name / diagnosis result correspondence table 61, "no description" is written.

Further, the report output means 81 converts the report 72 as shown in FIG. 7B into the character information 71 and the list 62.
As shown in FIG. 7A, the combination is output to the printer 91 or displayed on the display device (display) 92. In FIG. 7B, the report 72 displays characters and a list (table) at the same time.

<Embodiment 1-2> FIG. 8 is a block diagram showing the construction of the second embodiment of the first invention shown in FIG. 1. The construction of the report information input means 10 is the same as that of the above-mentioned embodiment. 1
Same as -1. The input report analysis unit 20 has a context analysis unit 21, a graphic creation unit 24 and a graphic editing unit 25. The configuration and function of the context analysis unit 21 are the same as those in the above-described Embodiment 1-1, and thus the description thereof will be omitted. Report information input means 10 by the context analysis unit 21
While analyzing the input report from to get the analysis result,
Graphic (graphic) by the graphic creating means 24
Create the data. And the graphic editing means 25
Edit the graphic data by. Then, the report output means 82 outputs the report as a combination of character information and graphics to the printer 91 or displays it on the display device (display) 92.

The input report such as handwritten characters or graphics input by the user from the tablet 11 or the memo or printed matter input from the digitizer 12 is coded as described above, and the character recognition unit 16 stores character information including a character string. And are sequentially stored in the storage unit 17. Next, the character information stored in the storage unit 17 is read out by the context analysis means 21 of the input report analysis means 20 shown in FIG. Is output as. For example, the entered text information is
Assuming that "... there are no more hearts and ribs ..." and "There is pneumonia in the lung field ...", an analysis result 51 as shown in FIG. 5 is obtained. Also. The graphic creating means 24 creates graphic data based on the context analysis result.
As shown in FIG. 9, the graphic editing means 25 has a correspondence table 91 between the site name and the address (area) on the graphic data in the system (for example, the storage unit 17 or the like), and the context name 51 Correspondence of the diagnosis result is made, the part name of the context analysis result 51 ′ and the part name of the correspondence table 91 are compared, and if they match, the context analysis corresponding to the region 93 on the graphic data corresponding to the part name. A graphic (figure) 94 is created by writing the result. For example, in the example of FIG. 9, the graphic data editing means displays "pneumonia" at the address (0F25, EEF5) of the area 93.
Create and write graphic data.

The report output means 82 is shown in FIG. 10B.
The report 100 as shown in FIG. 10 is output as a combination of the character information 101 and the graphic (drawing) 102 to the printer 91 as shown in FIG. 10A, or is displayed on the display device (display) 92. In FIG. 10B, the report 100 displays the characters 101 and FIG. 102 at the same time.

<Embodiment 1-3> FIG. 11 is a block diagram showing the configuration of the third embodiment of the first invention shown in FIG. 1, in which the report information input means 10 includes handwritten characters and graphics ( Tablet 11 'for inputting (figure) or
It has a digitizer 12 'for coding and inputting characters or graphics, a character recognition and graphic creation data generation unit 16' and a character recognition unit 16 as report generation means.

Further, the input report analysis means 20 is a character information context analysis means 21, a list creation means 23 as a first list creation means, a graphic information analysis section 26, a graphic pattern recognition section 27, a character pattern recognition section 28 and a second. It has a character-part correspondence list creating means 29 as a list creating means. Then, the context analysis unit 21 analyzes the input report from the report information input unit 10 to obtain the analysis result and creates the first list by the list creation unit 23, or the graphic analysis unit 26 creates the report information input unit. The input report from 10 is analyzed, the analysis result is obtained, and the character-site correspondence list creating means 29 creates a character-site correspondence list as a second list.

Then, the report output means 83 outputs the report as a combination of characters and graphics and the first list or the character-part correspondence list to the printer 91 or displays it on the display device (display) 92.

As the tablet 11 'for the user to input handwritten characters, for example, a product such as a paper image personal workstation (trade name) is known, and the tablet 11' is directly input. The input report consisting of handwritten characters and graphics or handwritten characters and graphics input from the digitizer 11 'is coded (binarized) in the data input unit 15', and the character information and graphic data is generated in the character recognition and graphic data creation unit 16 '. The input report is a combination of and is sequentially stored in the storage unit 17.

When the user inputs an input report consisting of handwritten characters, graphics, printed characters, memos with graphics, printed matter, etc. from the digitizer 12 ', these input reports are digitized (coded). The character recognition unit 16 forms an input report as a combination of character information including a character string and graphic data, and sequentially stores the input report in the storage unit 17.

Next, the character information stored in the storage unit 17 is read out by the context analysis unit 21 of the input report analysis unit 20 shown in FIG. 4 as described above (Example 1-1) and divided into words. After that, the meaning is given and the result is output as an analysis result, and an analysis result 51 as shown in FIG. 5 is obtained.
Further, the list creating means 23 creates a list (first list) as shown in FIG. 6 based on the context analysis result as described above (Example 1-1).

On the other hand, when the graphic pattern recognition unit 27 recognizes the graphic data from the report input from the report information input unit 10 by the graphic pattern recognition unit 27, the graphic analysis unit 26 stores a certain pattern and its part name in association with each other, and stores the character. When the pattern recognition unit 28 recognizes the pattern of the character information of the report input from the report information input means 10, the pattern information is stored in association with the part name which is the result of the pattern recognition of the graphic data near the character information. FIG. 12 is an explanatory diagram showing these relationships. The character-region correspondence list creating means 29 obtains the storage result (analysis result) and creates the character-region correspondence list 129 as shown in FIG.

The pattern recognition method of the graphic pattern recognition section may be a well-known method, and the method of recognizing the "part name which is the result of pattern recognition of graphic data in the vicinity of character information" by the character pattern recognition section is , Determine a distance from an address with text information. For example,
When the distance between (X1, Y1) and (X2, Y2) is ε,
Closest within the distance ε (that is, the shortest distance)
This is because the part name is found from the graphic data address / part correspondence table 125 as shown in FIG. Specifically, the distance between the address A0 of the character information and the graphic address Ai (i = 1 to n) of the correspondence table 125 is obtained, and the part name corresponding to the address having the minimum value is obtained. In addition, data 1 other than the parts shown in FIG.
Reference numeral 26 indicates a diagnosis result of what remains after subtracting each part pattern 136 from the original graphic data 135 as shown in FIGS. 13A and 13B, for example, an arrow or a diagram in which a region is shaded. It is a figure.

Also, the report output means 83 is shown in FIG. 14B.
10 as a combination of the character information 141, the graphic (figure) 142 and the list.
It is output to the printer 91 as shown in A or displayed on the display device (display) 92. In FIG. 10B, the report 140 includes characters 141, FIG. 142, and Table 5.
1'is displayed at the same time.

<Embodiment 2-1> FIG. 15 is a block diagram showing the configuration of the first embodiment of the second invention shown in FIG. 2. Report information input means 10 and input report analysis means 20. The configuration is the same as that of the above-mentioned Embodiment 1-1, and the description thereof is omitted.

The input report of the handwritten characters or graphics input by the user from the tablet 11 or the memo or printed matter input from the digitizer 12 is coded as described above, and the character recognition unit 16 stores character information including a character string. And are sequentially stored in the storage unit 17. Next, the character information stored in the storage unit 17 is read out by the context analysis unit 21 of the input report analysis unit 20 shown in FIG. It is output as the analysis result as shown in. Also. The list creating means 23 creates a list as shown in FIG.

On the other hand, the image retrieving means 40 retrieves the image database 31 via the network. The image analysis means 50 extracts the contour of the image obtained by the search as shown in FIG. 16A by the contour extraction means 51, and the pattern recognition means 55 performs pattern recognition on the graphic data of the contour,
A certain pattern and its part name are stored in the correspondence table 169 (FIG. 16B).

As shown in FIG. 17, the overlay creating means 60 compares the input report, the analysis result 171 of the input report analyzing means 20 and the analysis result 172 of the image analyzing means 50 by the comparing means 51, and the editing means 65. Create an overlay with. Context analysis result 171
The method of comparing the image analysis result 172 with the image analysis result 172 is to create an overlay by storing the correspondence between the site name on the image and the address (area) on the overlay data from the image analysis result 172, and then the internal system (for example, the storage unit 17). Correspondence table 169 of region names and addresses (areas) on overlay data
As described above (Embodiment 1-3), the site and the diagnosis result are associated from the context, and the site name in the context and the site name in the correspondence table 169 are compared by the comparing means 61 and coincide with each other. Converts the character code of the diagnostic result of the context corresponding to the address on the overlay data corresponding to the part name into graphic data to create write overlay data 179.

The report display unit 84 creates a report 180 as shown in FIG. 18B as a combination of the searched image and overlay, and simultaneously displays the table, the overlay data and the image on the display 92.

In the present embodiment, if the input report analysis means 20 is composed of the context analysis part 21, the graphic creation means 24 and the graphic editing means 25 (see Embodiment 1-2), the report display means 84 will be A table, a figure, overlay data and an image can be simultaneously displayed as a combination of overlays including an image and a figure retrieved as a report.

<Embodiment 2-2> FIG. 19 is a block diagram showing a structure of a second embodiment of the second invention of FIG. Report information input means 10 and input report analysis means 20
The configuration is the same as the configuration of the above-described (Embodiment 1-3), and description thereof will be omitted. Further, the configurations of the image search means 40 and the image analysis means 50 are the same as those described above (Example 2-2), and thus the description thereof is omitted.

When the user inputs an input report consisting of handwritten characters, graphics, printed characters and memos with graphics, printed matter, etc. from the digitizer 12 ', these input reports are digitized (coded),
The character recognition unit 16 forms an input report as a combination of character information including a character string and graphic data, and sequentially stores the input report in the storage unit 17.

Next, the character information stored in the storage unit 17 is read out by the context analysis unit 21 of the input report analysis unit 20 shown in FIG. 4 as described above (Example 1-1) and divided into words. After that, the meaning is given and the result is output as an analysis result, and an analysis result 51 as shown in FIG. 5 is obtained.
Further, the list creating means 23 creates a list (first list) as shown in FIG. 6 based on the context analysis result as described above (Example 1-1).

On the other hand, when the graphic pattern recognition section 27 recognizes the graphic data from the report input from the report information input means 10 by the graphic pattern recognition section 27, the graphic analysis means 26 stores a certain pattern and its part name in association with each other, and stores the character. When the pattern recognition unit 28 recognizes the pattern of the character information of the report input from the report information input means 10, the pattern information is stored in association with the part name which is the result of the pattern recognition of the graphic data near the character information. FIG. 12 is an explanatory diagram showing these relationships. The character-region correspondence list creating means 29 obtains the storage result (analysis result) and creates the character-region correspondence list 129 as shown in FIG.

As shown in FIG. 20, the overlay creating means 60 includes an input report and graphic data 201 as an analysis result of the input report analyzing means 20, a part name-address correspondence table 202, character information-address correspondence table 203, character information. -Part name correspondence table 204, address 205 of data other than the part, and pattern address as an analysis result by the image analysis unit 50-Part name correspondence table 207
And the comparison means 51 'to compare, and the editing means 65' creates the overlay data 209. The method of comparing the context analysis result 171 and the image analysis result 172 to create an overlay is also as described above (Example 1-3).

The report displaying means 84 creates a report 180 as shown in FIG. 18B as a combination of the retrieved image and overlay, and simultaneously displays the table, the overlay data and the image on the display 92.

Also in this embodiment, if the input report analyzing means 20 is composed of the context analyzing portion 21, the graphic creating means 24 and the graphic editing means 25 (see the first and second embodiments), the report displaying means 84 becomes As a report, a table, a figure, overlay data and an image can be simultaneously displayed as a combination of the searched image and an overlay including the figure.

<Embodiment 3> FIG. 22 is a block diagram showing the configuration of the third invention of the report information input / output system of the present invention. In FIG. 22, the report information input / output system uses the displayed medium as an input medium as it is, and inputs the voice and pointing data directly as the report information to create an input report, and analyzes the input report. Input report analysis means 20, an overlay creation means 60 for creating an overlay, and a report output means 80 ″ for creating and outputting a report as a combination of an image and an overlay.

The report information input means 10 associates the voice input microphone 111, the digitizer 112 for digitizing the output (analog signal) from the voice input microphone 111, and the correspondence between the voice digital signal input from the digitizer 112 and the time. 23. As the time / speech data, the sound data creating means 113 is stored in the time / speech correspondence table 230 as shown in FIG. In the case of touch sensor 114 as a pointing device for pointing (positioning) a position where an annotation or the like (converted voice is converted) should be displayed, analog data input from touch sensor 114 is coordinate data that is digital data. Pointing data input unit 11 for converting into (X, Y) , Pointing data input unit 11
24. Pointing data creating means 116 and voice data creating means that associates the coordinate data (X, Y) input from No. 5 with time and stores them as time / coordinate data in the time / coordinate correspondence table 250 as shown in FIG. A timer 117 for outputting time (timing) to 113 and pointing data creating means 116, time / voice data from the voice data creating means 113, and pointing data creating means 11
Image data management unit (described later) from 6 to time / coordinate data
It has input report data creating means 118 for inputting image data from 182 and creating input report data 250 as shown in FIG.

The input report analyzing means 20 comprises a report data creating means 125 ', a voice recognizing means 122, and an information discriminating means 125' as the pointing data information analyzing means 125.
1'and voice recognition means 122 are voice information analysis means 121.
Make up.

The report data generating means 121 'receives the input report data 250 or the second output report data which is the output of the information discriminating means 125', and outputs the first output report data. In this case, the report data creation means 121 'recognizes the voice data portion of the input report data 250 by the voice recognition means 122 and replaces it with character information. At this time, the character information must be synchronized (corresponding) with the input time of the original voice data. Therefore, as shown in FIG. 26, the time (b) of the first voice data in the time / voice correspondence table 230 is set as the start time, and the time (b) of the voice data that has been recognized is set as the last time. Write to the report data 270. If the portion (c) of the time / voice correspondence table 230 shown in FIG. 26 is “pneumonia”, it is written in the first output report data 270 as (d). FIG. 27 shows an example of the first output report data 270.

The information discriminating means 125 inputs the input report data 250 or the first output report data, analyzes the pointing information, and discriminates the graphic data from the voice input data. In this case, the time / coordinate data is used to calculate the distance traveled by the coordinate data from the arbitrary time t1 to the predetermined time Δt. As shown in the time / coordinate correspondence table 240 shown in FIG. 28A, assuming that there are n pieces of coordinate data during this Δt, the distances between all these coordinates and the initial values (X1, Y1) are calculated, and If at least one of them exceeds the predetermined distance ε, the coordinates (X1, Y
1) is considered to be data handwritten by the user as graphic data. Then, the second shown in FIG. 28B.
Coordinates (X1, Y1) are added to the graphic data area of the output report data 280 of FIG. Next, the same operation is performed for t2, and this operation is repeated.

The overlay creating means 60 comprises a character overlay creating means 161, a graphic overlay creating means 162 and an overlay memory 163.

The character overlay creating means 161 uses the first
The output report data 270 and the second output report data 280 are input and an overlay is created from the pointing information and the voice data. In this case, the overlay creating means 60 looks at the pointing information of the second output report data 280. For example, the pointing information of the second output report data 280 has the time tn + 1 to the first line.
When tn and the coordinates (Xn + 1, Yn + 1) are pointed, the times tn + 1 to tn of the first output data 270.
And search for the corresponding letter (eg "pneumonia")
Read out. Then, the overlay data of the image data management unit 182 is searched based on the image ID, and the coordinates (Xn + 1, Yn + 1) of the overlay data 291 are searched as shown in FIG.
From the right to add the overlay data 292 that is displayed as “pneumonia” to create a new overlay data 2
Get 90. The character overlay creating means 161 uses the second
This operation is performed for all the pointing information of the output data 280 of. The created overlay is sent to the display means and displayed via the image management means 182.

Graphic overlay creating means 162
Inputs the second output report data 280 and creates an overlay from the graphic data. The graphic overlay creating means 162 looks at the graphic data of the second output report data 280 and coordinates (X1,
It is considered that the portion Y1), ... Is handwritten, and the overlay is searched from the memory of the overlay data based on the image ID in the second output report data 280, and the graphic data is added to the data. The created overlay is sent to the display means and displayed via the image management means 182.

The report output means 80 "has an image data management unit 182 as a report input / preparation terminal, a keyboard 183 as an input unit, an image display unit 184, and an overlay display unit 185. The combination is displayed on the display 92. In this case,
When the image ID is input from the keyboard and a display request is issued, the overlay corresponding to the image ID is displayed on the display 9
Display on 2.

The operation of the report information input / output system having the above configuration will be described below.

[Report Input] The image data management unit 182 as a report input / preparation terminal displays an image on the display 92 although a doctor should input a diagnostic finding. The doctor inputs the findings while looking at the image by using the voice and the image pointing operation by the pointing device (touch sensor) 114. This is mainly (1) inputting character data by voice at the pointed location, or (2) drawing a figure exactly as you pointed,
And so on.

The report inputting / creating terminal 182 inputs a voice from the microphone 111 and pointing information from the touch sensor 114 and processes it.

First, the sound is sent from the microphone 111 to the digitizer 112 as an analog signal, and the digitizer 11
In step 2, it is A / D converted into digital data and sent to the audio data creation unit 113. Voice data creation unit 113
Then, the digital data sent from the digitizer 112 is correlated with the time data from the timer 117 to obtain the time /
The audio correspondence table 230 is sequentially stored. Specifically, the digital data sent from the digitizer 112 and the timer 117
The time data from 1 is stored in the buffer memory and stored in the time / voice correspondence table 230 in synchronization with the clock.

Next, the pointing data from the touch sensor 114 is input to the pointing data input unit 115 and converted into coordinate data. The pointing data creation unit 116, which has received the coordinate data from the pointing data input unit 115, inputs the time data from the sensor 117 and sequentially stores the time data in the time / coordinate correspondence table 240 in synchronization with the clock as in the case of the voice data.

When the doctor finishes the input of the diagnostic finding and inputs the end request command from the keyboard 183, the input command is sent to the image data management unit 182, and the image data management unit 182 is currently displayed and the finding is input. The image ID of the selected image is sent to the data creation unit 118. Also,
This end request command is sent to the voice data creation unit 113 and the pointing data creation unit 116, and the voice data creation unit 113 and the pointing data creation unit 116 that received the end request command created the time / voice data and time / coordinates, respectively. Data is created by the data creation unit 11
Output to 8.

The data creating unit 118 creates the input report data 250 by sequentially combining the input data. The created input report data 250 is sent to the report data creating means 121 'and the information discriminating means 125'.

[Creation of Character Information] The report data creating means 121 'converts the voice data into the character code by the voice recognizing means 122 by the well-known voice recognition method for the voice data part of the input report data 250 input. . At this time, the voice recognition unit 122 converts the word or sentence as a word or minute delimiter into a character code, for example, when the silent state is reached, the word or sentence up to that point is delimited.
As shown in FIG. 31A, times t1, ..., Tn corresponding to voice data V1, ..., Vn required for code conversion
And the converted character code C are associated with each other and output to the report data creating means 121 ′. The data format at this time is an array of the first time t1, the last time tn, and the character code C of the recognized voice as shown in FIG. 31B.
Similarly, for a portion recognized as a silent state, a character code (null) is associated with the first and last times and output.

Upon receiving the signal from the voice recognizing means 122, the report data creating means 121 'displays the image ID, time /
The coordinate data and the time (first and last) -character code data of one format (first output data 28
0) is output to the character overlay creating means 161.

[Discrimination between graphic data and voice input data] The information discriminating means 125 'focuses on the time / coordinate data of the input report data 250 input from the data creating section 118 and advances the processing. As mentioned above, time Δ
The distance D moved during t is calculated, all D of the coordinate data between Δt are calculated, and whether or not each D exceeds a predetermined value ε is used to judge whether it is handwritten graphic data. Separately, it is stored in the handwritten graphic table 320 shown in FIG.

[Creation of Character Overlay] From the information discriminating means 125 'and the report data generating means 121' to the first
The character overlay creating means 161 which receives the second output report data and creates the overlay data based on these output report data. The coordinates of the touch sensor 114 are in the range of (0, 0) to the maximum (Xmax, Ymax), and the character overlay creating means 161 also holds this data. The character overlay creating means 161 sends the image management data section 182 an image ID of the first output report data or the second output report data.
Is used as a parameter to request the character overlay data of the image.

The image data management unit 182 preliminarily sets the pixel size Xmax, Y of the overlay data for each image.
An overlay format of max, 1 bit in the density direction (Z direction) is created (however, the initial value is 0). Then, it has a correspondence table (not shown) of the overlay data. Therefore, the character overlay creating means 161
When a command (overlay request, image ID) is sent to the image management data section 182, the image management data section 182 refers to the correspondence table of the overlay data held and displays the result (image ID, overlay data). Character overlay creating means 1 from overlay memory 163
Send to 61.

When the character overlay creating means 161 receives the overlay data and its pixel size,
Referring to the pointing data of the second output report data, the coordinates (Xn + 1, Yn + 1) are set between the times tn + 1 and tn.
) Is pointed and it is judged that there is a voice input during that time, and the corresponding character information is searched. as a result,
For example, for the voice input "haen", "pneumonia"
If the character code is entered, the coordinates (Xn + 1 · (X'max / Xmax), Yn + of the overlay format are entered.
1 ・ (Y'max / Ymax)) The binarized character data corresponding to the character "pneumonia" is created in the neighborhood starting from (Y'max / Ymax), that is, within a predetermined number of pixels ε'and create. This operation is performed on the pointing data of the second output report data. The overlay data created in this manner is used as the overlay memory 163.
Sent to and stored.

[Creation of Graphic Overlay] Second
The graphic overlay creating means 162 that has obtained the output report data of 1 creates an overlay by referring to the graphic data portion of the second output report data. First, using the image ID of the second output report data as a parameter, a command requesting graphic overlay of the image is sent to the image data management unit 182. Upon receiving the command and parameter, the image data management unit 182 receives overlay data and pixel size (X
max, Ymax, and 1 bit of the density direction (Z direction) are sent to the graphic overlay creating means 162.

Graphic overlay creating means 162
Receives the overlay data and the pixel size from the image data management unit 182, and creates overlay data based on the coordinates (X1, Y1) of the graphic data of the second output report data. In this case, the coordinates of the overlay data (X1. (X'max / Xmax), Y1. (Y'max
/ Ymax)) data is changed from 0 to 1. This is performed for all the graphic data of the second output report data. The overlay data thus created is sent to and stored in the overlay memory 163.

In the overlay memory 163, the overlay data for one image ID may be divided into a character overlay memory and a graphic overlay memory. Further, the character overlay memory and the graphic overlay memory may be the same, but in that case, the character overlay creating means 161 and the graphic overlay creating means 162 update at the same time so that one does not erase the other over example. In addition, exclusive control by the overlay memory 163 is required. In this embodiment, the overlay data is divided and stored in the character overlay memory and the graphic overlay memory.

[Display of image and overlay] The character overlay and graphic overlay created by the character overlay creating means 161 and the graphic overlay creating means 162 are both in the image data management section 182.
Is associated with the image ID in the overlay memory 163.
Stored in. The reference request and the image ID are the keyboard 18
When input from 3, the image data management unit 182 adds the character overlay and the graphic overlay for each same coordinate, creates one overlay data, and sends it to the overlay display unit 185. Further, the image corresponding to the image ID is read out from the image memory 181, and the image display unit 1
To 84. Due to this, the image is overlaid and displayed on the display 92.

By each of the above operations, the voice and the report input from the pointing device become an overlay and can be referred to.

In the present invention, different means existing in the same CPU are activated as tasks (processes), and data exchange between them is performed by inter-task communication. Further, as shown in FIG. 33, when data is exchanged via the LAN, the data is transmitted to the LAN via the LAN interface (not shown), received by the other party's interface (not shown), and received. It FIG. 33
In this example, the first and second output report data are LAN3
It is sent from the report input terminal 331 to the report reference terminal 332 via 39.

The embodiments of the present invention have been described above.
It is needless to say that the present invention is not limited to the above embodiment and various modifications can be made.

[0093]

As described above, according to the present invention, it is possible to display a report in which images, character information, graphics, etc. are combined, so that the reader of the report can understand the results more directly. You can get easy reports.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a first invention of a report information input / output system of the present invention.

FIG. 2 is a block diagram showing a configuration of a second invention of the report information input / output system of the present invention.

FIG. 3 is a block diagram showing a configuration of a first exemplary embodiment of the first invention.

FIG. 4 is a block diagram showing a configuration example of an input report analysis unit.

FIG. 5 is an example of an analysis result by context analysis means.

FIG. 6 is an explanatory diagram of a list creating process by a list creating unit.

FIG. 7 is an explanatory diagram of report output by a report output unit.

FIG. 8 is a block diagram showing a configuration of a second exemplary embodiment of the first invention.

FIG. 9 is an explanatory diagram of a graphic editing process by a graphic editing unit.

FIG. 10 is an explanatory diagram of report output by a report output unit.

FIG. 11 is a block diagram showing a configuration of a third exemplary embodiment of the first invention.

FIG. 12 is an example of an analysis result by the graphic analysis unit.

FIG. 13 is an explanatory diagram of data other than parts.

FIG. 14 is an explanatory diagram of report output by a report output unit.

FIG. 15 is a block diagram showing a configuration of a first exemplary embodiment of the second invention.

FIG. 16 is an example of an analysis result by the image analysis means.

FIG. 17 is an explanatory diagram of creating an overlay by overlay creating means.

FIG. 18 is an explanatory diagram of report display by the report display means.

FIG. 19 is a block diagram showing a configuration of a second exemplary embodiment of the second invention.

FIG. 20 is an explanatory diagram of creating an overlay by overlay creating means.

FIG. 21 is a third part of the report information input / output system of the present invention.
3 is a block diagram showing the configuration of the invention of FIG.

FIG. 22 is a block diagram showing the configuration of an embodiment of the third invention.

FIG. 23 is an example of a time / voice correspondence table.

FIG. 24 is an example of a time / coordinate correspondence table.

FIG. 25 is an example of input report data.

FIG. 26 is an explanatory diagram of first output report data creation by report data creation means.

FIG. 27 is an example of first output report data.

FIG. 28 is an example of an analysis result by the information discriminating means.

FIG. 29 is an explanatory diagram of overlay output by a character overlay creating means.

FIG. 30 is an explanatory diagram of report output by the report output means.

FIG. 31 is a correspondence table of time, voice data, and character code.

FIG. 32 is an example of a handwritten graphic table.

[FIG. 33] LA of the report information input / output system of the present invention
It is explanatory drawing of the application example to N.

[Explanation of symbols]

 10 Report Information Input Means 20 Input Report Analysis Means 30 Image Storage Means 40 Image Retrieval Means 50 Image Analysis Means 60 Overlays 80, 80 ', 80 "Report Output Means 121 Voice Information Analysis Means 125 Pointing Data Information Analysis Means

Front page continuation (51) Int.Cl. ⁵ Identification code Office reference number FI Technical display location G06F 15/42 D 9194-5L 15/62 R 8125-5L 390 A 9287-5L

Claims (12)

[Claims] 1. A report information input means for inputting report information to create an input report, an input report analysis means for analyzing the input report, and an analysis result by the input report and the analysis means for the input report. And a report output means for creating and outputting an output report, and a report information input / output system. 2. The report information input / output system according to claim 1, wherein the report information input means includes a report creation means for creating an input report as character information including a character string, and the input report analysis means includes a context of the character information. Includes a text information context analysis means for analyzing the meaning of the text information and a list creation means for creating a list based on the analysis result by the text information context analysis means, and the report output means creates an output report as a combination of the word information and the list. A report information input / output system, which is characterized by being output. 3. The report information input / output system according to claim 2, wherein the input report analysis means creates character information context analysis means, and a graphic creation means and a graphic that creates a graphic based on an analysis result by the character information context analysis means. A report information input / output system comprising an editing means, wherein the report output means creates and outputs an output report as a combination of the character information and the graphic. 4. The report information input / output system according to claim 1, wherein the report information input means includes a report creation means for creating an input report as a combination of character information including a character string and graphic information, and the input report analysis means is provided. Character information context analysis means for analyzing the meaning of the context of the character information, graphic information analysis means for analyzing the graphic information, and a first list created based on the analysis result by the character information context analysis means. 1 list creation means and second list creation means for creating a second list based on the analysis result by the graphic information analysis means, wherein the report output means outputs the output report as character information, graphics, the first Report information input / output, characterized by being created and output as a combination of a list and the second list system. 5. A report information input means for inputting report information to create an input report, an input report analysis means for analyzing the input report, an image storage means, and an image search means for searching an image from the image storage means. An image analysis means for analyzing an image obtained by searching; an overlay creation means for creating an overlay by comparing the analysis result by the input report and the analysis means by the analysis means of the input report with the analysis result by the image analysis means; A report information input / output system comprising: a report output unit that creates and outputs a report as a combination of the searched image and the overlay. 6. The report information input / output system according to claim 5, wherein the input report analysis means is based on a character information context analysis means for analyzing the meaning of the context of the character information and an analysis result by the character information context analysis means. A report information input / output system characterized by including a list creating means for creating a list. 7. The report information input / output system according to claim 5, wherein the input report analysis means creates a graphic based on the character information context analysis means and the analysis result by the character information context analysis means. A report information input / output system comprising an editing means, wherein the report output means creates and outputs an output report as a combination of the character information and the graphic. 8. The report information input / output system according to claim 5, wherein the report information input means includes a report creation means for creating an input report as a combination of character information including a character string and graphic information, and the input report analysis means is provided. Character information context analysis means for analyzing the meaning of the context of the character information, graphic information analysis means for analyzing the graphic information, and a first list created based on the analysis result by the character information context analysis means. 1 list creation means and second list creation means for creating a second list based on the analysis result by the graphic information analysis means, and the report output means outputs an output report of the searched image and the overlay. A report information input / output system characterized by being created and output as a combination. 9. The report information input / output system according to claim 5, wherein the image analysis unit extracts a contour of the image, and a pattern for recognizing the extracted contour pattern. A report information input / output system comprising: a recognition means. 10. The report information input / output system according to claim 1, further comprising an overlay creating unit, wherein the report information includes audio information, an image, and pointing information synchronized in time. Including,
Report information input means, pointing data creating means for creating pointing data from the pointing information, voice data creating means for creating voice data based on voice information synchronized with the pointing information, image data, pointing data and voice data Input report data creating means for creating input report data as a combination of the above, and the input report analyzing means analyzes voice information from the input report data and converts it into character information, and the input report. Pointing information analyzing means for analyzing pointing information from the data and discriminating graphic data from voice input data, wherein the overlay creating means creates an overlay from the outputs of the voice information analyzing means and the pointing information analyzing means. , Port output means, character overlay the output report, the report information output system that created as a combination of graphic overlay and the image output, and wherein the. 11. The report information input / output system according to claim 5, wherein the report information includes time-synchronized audio information, an image, and pointing information, and the report information input means includes: , Pointing data creating means for creating pointing data from the pointing information, audio data creating means for creating audio data based on audio information synchronized with the pointing information, and input report as a combination of image data, pointing data and audio data Input report data creating means for creating data, wherein the input report analyzing means analyzes voice information from the input report data and converts it into character information, and pointing information from the input report data. Analyzing graphic data and voice input Pointing information analyzing means for discriminating the force data, the overlay creating means creates an overlay from outputs of the voice information analyzing means and the pointing information analyzing means, and the report output means overlays the output report with a character overlay. A report information input / output system characterized by creating and outputting a combination of a graphic overlay and an image. 12. The report information input / output system according to claim 11, wherein the image analysis means includes a contour extraction means for extracting a contour of the image and a pattern recognition means for recognizing the extracted contour pattern. Characteristic report information input / output system.