JP2006107283A - Document creating apparatus, document creating method, and document creating program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To assist correction of a result of character input or character input itself without changing a holding position of a microphone or a pen. <P>SOLUTION: First, a face picked up image of an operator picked up by an imaging camera 138 is acquired (step S11), a movement pattern of the face is recognized from the face image (step S12), and the recognition pattern is compared and collated with a pattern in a database DB1 (step S13). When a coincident pattern is detected as a result of comparison collation (step S14), the command is read and comparison-collation is performed with a command in a database DB2 (step S15). It is decided if the command exists (step S16), when it is not detected, an error is displayed (step S17), and when it is detected, an edit operation command corresponding to it is read and outputted (step S18). <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a document creation apparatus, a document creation method, and a document creation program that are used in, for example, a medical report creation apparatus, and that enable document input operations to be realized without manual input operations such as a keyboard and a mouse.

  Conventionally, the specialized field of medical practice has been subdivided, and it has been widely performed to request a specialist to interpret a medical image taken by an image diagnostic apparatus such as an X-ray or MRI. At the time of such an interpretation request, the specialist displays the image to be interpreted, summarizes the result into a report (report), and performs a process of passing it to the requester.

  An apparatus for electronically creating a medical report has already been put into practical use. In this apparatus, for example, medical images delivered from an interpretation request source by network distribution or the like are temporarily accumulated, a report sheet is displayed on the screen, and necessary items are input on the sheet. Then, the images to be interpreted are sequentially displayed, and the findings of the images are written in the finding entry column in the sheet, and based on the image findings, the diagnosis result considering the inspection purpose at the interpretation request source is written in the diagnosis entry column. For the image pointed out in the findings, a reduced / enlarged image or a cut-out image is created as necessary and pasted in an image attachment field or the like. The sheet data of the medical report created by the above procedure is transferred to the interpretation request source through a network, for example.

  By the way, in the conventional medical report creation apparatus as described above, various devices such as voice input and pen input are provided in addition to general keyboard input in character input operation at the time of document creation. The number of users who use new devices such as voice input and pen input other than the standard keyboard that has been used in the past is increasing, but when it comes to editing for document editing, switch from microphone / pen to keyboard / mouse. Most users are

  As described above, switching from the microphone / pen to the keyboard increases the time and effort required for doctors to continuously input findings, which is one factor that hinders the widespread use of voice input / pen input. Yes. In addition, voice correction (for example, a command to delete one character (equivalent to pressing the backspace on the keyboard) when you say “Ichimoji Sakujo”, etc.) is also being considered. Not reached.

  Patent Document 1 discloses a human face motion detection method capable of constructing an advanced GUI in order to reduce the inconvenience of using a pointing device such as a mouse while using a keyboard. Has been.

According to this method, the attention direction of the person is extracted from the face image of the person, and the correspondence with the operation command specified in advance is determined by detecting a specific change in the part inside the face, and the attention direction and the determination result The operation command can be used instead of a pointing device. However, the description in this document does not specifically describe an example in which the attention direction and the operation command determination result are used, and cannot simply be used for the above-described medical report creation.
JP 09-305743 A

  As described above, in the document creation apparatus represented by the conventional medical report creation apparatus, the text of the document itself is widely used for voice and pen input, but the editing work uses a keyboard and a mouse. In many cases, processing is performed in a GUI environment.

  SUMMARY OF THE INVENTION The present invention solves the above-described problems, and makes it possible to correct the result of character input or assist the character input itself without changing the microphone and pen, or even in an environment where only the keyboard is used. An object is to provide a document creation method and a document creation program.

  In order to solve the above problems, a document creation apparatus according to the present invention includes a character input device that accepts a character input operation, and a character string generation unit that generates a character string in response to the character input operation of the character input device. An imaging camera for imaging the face of the operator, a motion recognition unit for recognizing a facial motion pattern from a facial image captured by the imaging camera, and correspondence relationships between a plurality of facial motion patterns and commands in advance A command inference unit that retrieves a pattern corresponding to the facial motion pattern recognized by the motion recognition unit from the first database and infers a command corresponding to the pattern; A second database in which a correspondence relationship between the command inferred by the command reasoning unit and the character string editing operation command is registered; A command output unit that reads out and outputs the edit operation command for the character string corresponding to the inference command of the command inference unit with reference to the command source, and the character string generated by the character string generation unit and the command output A document file is created based on an editing operation command output from the section.

  The document creation method according to the present invention includes a character string generation process for generating a character string in response to a character input operation of a character input device, an imaging process for capturing an operator's face with an imaging camera, and the imaging process. A motion recognition process for recognizing a face motion pattern from a face image captured by the image processing method, and a first database in which correspondence relationships between a plurality of face motion patterns and commands are registered in advance. A command inference process for retrieving a pattern corresponding to the recognized facial motion pattern and inferring a command corresponding to the pattern, and a correspondence relationship between the command inferred in advance by the command inference process and the character string editing operation instruction With reference to the second database to be registered, a command output for reading out and outputting the character string editing operation command corresponding to the inference command in the command inference process ; And a process, characterized in that to create the document file based on the editing operation instruction is output string generated by the character string generation step from said instruction output process.

  The document creation program according to the present invention includes a character string generation step of generating a character string in response to a character input operation of a character input device, an imaging step of imaging an operator's face with an imaging camera, and the imaging step A motion recognition step for recognizing a face motion pattern from the face image captured by the step, and a first database in which correspondence relationships between a plurality of face motion patterns and commands are registered in advance. A command inference step for retrieving a pattern corresponding to the recognized facial motion pattern and inferring a command corresponding to the pattern, and a correspondence relationship between the command inferred in advance by the command inference step and the character string editing operation instruction The character string corresponding to the inference command of the command inference step with reference to the registered second database A command output step for reading and outputting an editing operation command, and creating a document file based on the character string generated in the character string generation step and the editing operation command output from the command output process, To do.

  That is, the feature of the present invention is that the operator's face is imaged using an imaging camera, the face motion pattern is detected from the captured face image, and the pattern-command database registered in advance corresponds. A command is inferred, and an edit operation command corresponding to the inference command is output from the inference command-edit operation command / database, thereby enabling editing of a character string only by the movement of a face.

  According to the above configuration, the document creation apparatus and the document creation method that can correct the result of character input or assist the character input without changing the microphone or pen or in an environment where only the keyboard is used. And a document creation program can be provided.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a hospital information system (HIS) including a medical report creation apparatus to which a document creation apparatus according to the present invention is applied and a medical report reproduction apparatus. In FIG. 1, reference numeral 11 denotes a network. In this network 11, a medical information server 12 for sharing medical information including medical images and medical reports, and for example, a radiogram interpretation room, requested for interpretation. A medical report creation device 13 that creates a medical report for an image is connected to a medical report reference device 14 that is installed in each medical office such as internal medicine and surgery, and that can receive the medical report and reproduce it appropriately. Information communication is possible.

  Although not shown in detail, in the present embodiment, the medical information server 12 includes an HIS server, an image file server, and a report system server. The medical report reference device 14 may be a medical report dedicated device, but it is also possible to use, for example, a terminal of an electronic medical record system installed in each clinic.

  The medical report creation apparatus 13 includes a workstation apparatus 131 connected to the network 11 as shown in FIG. The workstation device 131 is connected to a keyboard 132 and a mouse 133 as input devices, and includes a report monitor device 134 and a reference information monitor device 135 as display devices. The reference information monitor device 135 is preferably a multi-display so that a large number of reference information can be monitored at a time. Further, as input devices added by the present invention, a microphone 136, a tablet (pen input) 137, and an imaging camera 138 are provided.

  As shown in FIG. 3, the workstation device 131 receives instruction information corresponding to input operations of the information processing processor 13A, the communication port 13B, the storage unit 13C for storing information for work, the keyboard 132, and the mouse 133. An input interface unit 13D, an output interface unit 13E that outputs monitor display information to the report monitor device 134 and the reference information monitor device 135 are provided. Note that a cache memory is used for the storage unit 13C, but a hard disk device may be used in combination.

  The information processing processor 13A reads contents such as images, past medical reports, history information, and the like for reference in creating findings from the medical information server 12 through the communication port 13B, and stores them in the storage unit 13C. Further, the information processing processor 13A monitors various contents as an application, and displays a content display / editing program A1 for editing such as enlargement, reduction, clipping, and report file creation for creating and editing an electronic file of a medical report. An editing program A2, a link processing program A3, and a network management program A4 are provided, and a program is selected and executed in accordance with power-on and operator instruction input.

  FIG. 4 is a block diagram showing a specific configuration of the document creation function when the present invention is applied to the report file creation / editing program A2 installed in the information processing processor 13A in the medical report creation apparatus 13 having the above configuration. FIG. 5 is a flowchart showing the processing flow.

  In the apparatus of the present invention, an operator sits in front of the report monitor apparatus 134 and creates a document. In the middle of input, rules such as deletion if the operator turns to the left and addition of space if the operator turns to the right are registered in the database, and it is used as an auxiliary function for document input / correction according to the movement of the operator's face. .

  First, the report file creation / editing program A2 of the information processing processor 13A includes a character string creation unit A20, a motion recognition unit A21, a command inference unit A22, and an instruction output unit A23, as shown in FIG.

  The character string creating unit A20 generates a character string in response to a character input operation of the input device such as voice recognition input by the microphone 136 and character input by the pen of the tablet 137. Although the purpose here is not to use an existing input device (keyboard, mouse), it may be used in some cases.

  The motion recognition unit A21 recognizes a facial motion pattern from the operator's face image captured by the imaging camera 138. As a recognition method, for example, a part that is a feature of the face (eyes, nose, mouth, etc.) is extracted, and from the ratio between each part, it is sequentially determined how much the face is directed from the front direction, A method of recognizing a change in the direction in which the face is facing as a motion is conceivable. At this time, the movement is patterned to some extent. For example, it ignores diagonal movements and recognizes only vertical and horizontal movements and patterns of combinations thereof. Further, the movement amount and the duration after the movement may be used as elements of the movement pattern.

  The command reasoning unit A22 includes a database DB1 in which correspondence relationships between a plurality of face motion patterns and commands are registered in advance, and searches the database DB1 for a pattern corresponding to the face motion pattern recognized by the motion recognition unit A21. Infer a command corresponding to the pattern.

  The command output unit A23 includes a database DB2 in which a correspondence relationship between a command inferred by the command reasoning unit A22 and a character string editing operation command is registered in advance, and an inference command of the command reasoning unit A22 is referred to the database DB2. Reads and outputs the editing operation command for the corresponding character string. The commands include, for example, a character string input start command, deletion of the character string, line feed, indentation, backspace, and space insertion.

  In the report file creation / editing program A2 having the above-described configuration, as shown in FIG. 5, first, an image of the operator's face imaged by the imaging camera 138 is acquired (step S11), and a facial motion pattern is obtained from the face image. Recognize (step S12), compare the recognition pattern with the pattern in the database DB1 (step S13), and if a matching pattern is detected as a result of the comparison (step S14), read the corresponding command. Then, it is compared with the command in the database DB2 (step S15). It is determined whether there is a corresponding command (step S16). If it is not detected, an error is displayed (step S17). If it is detected, an editing operation command corresponding to it is read and output (step S18).

  Hereinafter, the relationship between the command inferred from the face movement pattern and the editing operation command will be exemplified.

  (1) In response to an inference command determined to have moved to the right, an instruction to insert a space is output. Adjust the number of characters to insert a space by judging the amount or time facing right.

  (2) In response to the inference command determined to have turned to the left, an instruction to delete one character is output. Determine the amount or time facing left and adjust the number of characters to delete.

  (3) The voice input is started when the face is facing the front, and the voice input is finished when the face is facing the other. In general, you can press the CTRL key on the keyboard to start typing, but this eliminates the need to use the keyboard.

  (4) Insert a line feed when recognizing a pattern of facing down and then facing left.

  As described above, according to the document creation function according to the present invention, for example, when finding information is input by voice recognition and the face is turned to the left while inputting a finding on the finding input screen, the face is turned to the left. The input finding information is deleted in accordance with the action of turning. Deletion starts when the angle information output from the face motion detection means is directed to the left beyond a predetermined value so that the amount to be deleted is linked to the movement of the face. Delete characters. Here, a condition is added so that the findings are not accidentally deleted when the operator stands. In order to prevent accidental deletion, a rule is set such that deletion is performed only when the face is shaken sideways, and deletion is not performed when the face is moved diagonally.

  According to the present invention, when inputting a document using a device other than the keyboard, the device has been forced to be switched from the device to the keyboard. There is no need to change the device, and the document input time can be substantially reduced. In particular, good operability can be obtained by inputting findings while combining voice recognition and facial movement.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

1 is a block diagram showing a configuration of a hospital information system (HIS) including a medical report creation device and a medical report reference device according to the present invention. 1 is a block diagram showing a basic configuration of a medical report creation apparatus and a medical report reference apparatus according to the present invention. 1 is a block diagram showing an embodiment of a medical report creation apparatus according to the present invention. The block diagram which shows the specific structure of the document preparation function at the time of applying this invention to the report file preparation / editing program mounted in the information processing processor of the said medical report preparation apparatus. 5 is a flowchart showing the flow of processing of the document creation function shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Network, 12 ... Medical information server, 13 ... Medical report production apparatus, 14 ... Medical report reference apparatus,
131 ... workstation device, 132 ... keyboard, 133 ... mouse, 134 ... report monitor device, 135 ... reference information monitor device, 136 ... microphone, 137 ... tablet (pen input), 138 ... imaging camera,
13A ... Information processing processor, 13B ... Communication port, 13C ... Storage unit, 13D ... Input interface unit, 13E ... Output interface unit,
A1 ... Contents display / editing program, A2 ... Report file creation / editing program, A3 ... Linking processing program, A4 ... Network management program,
A20 ... Character string creation unit, A21 ... Motion recognition unit, A22 ... Command reasoning unit, A23 ... Command output unit, DB1 ... Pattern-command database, DB2 ... Command-command database.

Claims (11)

A character input device that accepts character input operations;
A character string generation unit that generates a character string in response to a character input operation of the character input device;
An imaging camera for imaging the operator's face;
A motion recognition unit for recognizing a facial motion pattern from a face image captured by the imaging camera;
A first database in which correspondence relations between a plurality of facial motion patterns and commands are registered in advance; a pattern corresponding to the facial motion pattern recognized by the motion recognition unit is searched from the first database; A command inference section for inferring commands corresponding to patterns;
A second database in which a correspondence relationship between the command inferred by the command inference unit and the character string editing operation instruction is registered in advance, and the inference command of the command inference unit is supported by referring to the second database An instruction output unit that reads and outputs the editing operation instruction of the character string,
A document creation apparatus that creates a document file based on a character string generated by the character string generation unit and an editing operation command output from the command output unit.   The document creation apparatus according to claim 1, wherein the movement recognition unit detects a movement direction of the face from a specific direction from the face image, and uses the movement direction as an element of a movement pattern of the face.   The document creation apparatus according to claim 1, wherein the movement recognition unit recognizes a vertical movement, a horizontal movement, or a continuous movement in each direction as the movement pattern of the face, and excludes acceptance of an oblique movement pattern.   The document creation apparatus according to claim 1, wherein the motion recognition unit detects a movement amount of the face from a specific direction from the face image, and uses the movement amount as an element of a face movement pattern.   2. The document creation according to claim 1, wherein the motion recognition unit detects a time when the face is moved in a predetermined direction from a specific direction of the face from the face image, and uses the time as an element of a face motion pattern. apparatus.   The document creating apparatus according to claim 1, wherein the character input device is a voice recognition device or a handwritten character recognition device.   The command output means includes the editing operation command including at least one of a start command of the character string input, a deletion of the character string, a line feed, an inhibit, a back space, and a space insertion. The document creation device according to any one of 3 and 6.   5. The document creation apparatus according to claim 4, wherein the command output means generates and outputs a command for adjusting the number of characters to be deleted or the number of spaces to be inserted as the editing operation command.   6. The document creation apparatus according to claim 5, wherein the command output means generates and outputs a command for adjusting the number of characters to be deleted or the number of spaces to be inserted as the editing operation command. A character string generation process for generating a character string in response to a character input operation of a character input device;
An imaging process of imaging an operator's face with an imaging camera;
A motion recognition process for recognizing a face motion pattern from a face image captured by the imaging process;
A pattern corresponding to the face motion pattern recognized in the motion recognition process is searched by referring to a first database in which correspondence relationships between a plurality of face motion patterns and commands are registered in advance, and corresponding to the pattern. Command inference process to infer commands,
The character string editing operation corresponding to the inference command in the command inference process is performed by referring to a second database in which the correspondence relationship between the command inferred by the command inference process and the character string editing operation instruction is registered in advance. An instruction output process for reading out and outputting instructions;
A document creation method, wherein a document file is created based on a character string generated in the character string generation process and an editing operation command output from the command output process. A document creation program for causing a computer to execute processing for creating and editing a document,
A character string generation step for generating a character string in response to a character input operation of the character input device;
An imaging step of imaging an operator's face with an imaging camera;
A motion recognition step for recognizing a face motion pattern from the face image captured by the imaging step;
A pattern corresponding to the facial motion pattern recognized in the motion recognition step is searched by referring to a first database in which correspondence relationships between a plurality of facial motion patterns and commands are registered in advance, and corresponding to the pattern. A command inference step to infer a command;
The character string editing operation corresponding to the inference command of the command inference step is performed by referring to a second database in which the correspondence relationship between the command inferred by the command inference step and the character string editing operation instruction is registered in advance. An instruction output step for reading out and outputting an instruction;
A document creation program for creating a document file based on the character string generated in the character string generation step and the editing operation command output from the command output process.

Images