JP2013061793A - Input support device, input support method, and input support program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an input support device capable of easily adding a voice input function to an existing application requiring high security.SOLUTION: An information processing device 200 including an input support device of the present invention adds a voice input function to an electronic medical chart unit 203 that is application software for displaying input areas for inputting information on an input screen. The information processing device includes: a screen configuration table storage unit 208 for storing a screen configuration table in which the display position of each input area is associated with first information for specifying reading indicating the input area; and an input information generation unit 210 for obtaining second information indicating a result of voice recognition processing on uttered voice, referring to the screen configuration table to identify the display position of an input area corresponding to the obtained second information, and inputting operation information for selecting the identified display position to the electronic medical chart unit 203.

Description

  The present invention relates to an input support apparatus, an input support method, and an input support program for adding a voice input function to application software such as an electronic medical record.

  Application software (hereinafter simply referred to as “application”) for inputting a large amount of data into a database is usually prepared for each database or for each application. The hospital's electronic medical record is a typical example. With the recent improvement of voice recognition technology, there is an increasing need to incorporate a voice input function into such an existing application.

  Therefore, there is a technique for adding a voice input function to an existing application without modifying the application (see, for example, Patent Document 1).

  The technology described in Patent Document 1 (hereinafter referred to as “conventional technology”) accepts reading settings from a user for user interface components that constitute an input screen of a web application. Then, in the conventional technology, when the speech recognition result of the input speech corresponds to any reading set, the processing according to the component for which the reading is set is executed. For example, when the part is a text input field, the prior art sets a character string input after reading in the input field.

  According to such a conventional technique, the operation can be performed by voice without particularly modifying the existing application.

JP 2007-164732 A

  However, the conventional technology has a problem that it is difficult to apply to an application such as an electronic medical record that requires high security. This is because such an application is usually not open source, and it is difficult to perform processing in parts from outside.

  An object of the present invention is to provide an input support device, an input support method, and an input support program that can easily add a voice input function to an existing application that requires high security.

  An input support apparatus according to an aspect of the present invention is an input support apparatus that adds a voice input function to application software that displays an input area for inputting information on an input screen, and displays the input area. A screen configuration table storage unit that stores a screen configuration table in which the position and first information that identifies the input area are associated with each other; and second information that represents the result of the speech recognition processing for the uttered speech Obtaining, specifying the display position of the input area corresponding to the acquired second information with reference to the screen configuration table, and inputting operation information for selecting the specified display position to the application software An input information generation unit.

  An input support method according to an aspect of the present invention is an input that adds a voice input function to application software that displays an input area for inputting information and label text indicating an attribute of the input area on an input screen. A support method comprising: obtaining a display position of the input area and a display position of the label text; and displaying the input area based on a relative relationship between the display position of the input area and the display position of the label text. A step of generating a screen configuration table in which information specifying the reading of the label text is associated with the position as information specifying the reading indicating the input area, and information indicating the result of the speech recognition processing for the uttered speech The step of acquiring and the input area corresponding to the information indicating the result of the acquired voice recognition process The shown position, has a step of specifying by referring to the screen configuration table, the operation information for selecting the display position specified, the inputting to the application software.

  An input support program according to an aspect of the present invention adds a voice input function to application software that displays an input area for inputting information and a label text for indicating an attribute of the input area on an input screen. An input support program for obtaining a display position of the input area and a display position of the label text in a computer accessible to the application software, a display position of the input area, and a display position of the label text A process of generating a screen configuration table in which information for identifying the reading of the label text is associated with the display position of the input area as information for identifying the reading representing the input area, based on the relative relationship with Processing to acquire information that represents the result of speech recognition processing for spoken speech A process for specifying the display position of the input area corresponding to the acquired information representing the result of the voice recognition process with reference to the screen configuration table, and operation information for selecting the specified display position. The process to input to the software is executed.

  According to the present invention, a voice input function can be easily added to an existing application that requires high security.

The system block diagram which shows the structure of the input assistance system which concerns on one embodiment of this invention FIG. 2 is a block diagram illustrating an example of a configuration of an information processing device according to the present embodiment The figure which shows an example of the content of the table creation rule in this Embodiment The figure which shows an example of the operation conversion rule in this Embodiment FIG. 2 is a block diagram illustrating an example of a configuration of a wireless communication terminal in this embodiment A block diagram showing an example of a configuration of a voice recognition server in the present embodiment The figure which shows an example of the content of the apparatus information in this Embodiment The figure which shows an example of the content of the clerk information in this Embodiment The figure which shows an example of the content of the clerk decision rule in this Embodiment Flowchart illustrating an example of the operation of the information processing apparatus in this embodiment Plan view showing an example of an input screen in the present embodiment Flowchart showing an example of table generation processing in the present embodiment The figure which shows an example of the content of the screen structure table in this Embodiment The figure which shows an example of the content of the information for dictionary in this Embodiment The figure which shows an example of the content of the ID text corresponding | compatible information in this Embodiment The figure which shows an example of a structure of the transmission data from the information processing apparatus to a radio | wireless communication terminal in this Embodiment Flowchart showing an example of operation of the wireless communication terminal in this embodiment The figure which shows an example of a structure of the 1st transmission data from the radio | wireless communication terminal in this Embodiment to a speech recognition server. The figure which shows an example of a structure of the 2nd transmission data from the radio | wireless communication terminal in this Embodiment to a speech recognition server. The flowchart which shows an example of operation | movement of the speech recognition apparatus in this Embodiment. The figure which shows an example of the content of the ID reading corresponding | compatible table in this Embodiment The figure which shows an example of a structure of the transmission data from the speech recognition server in this Embodiment to a calibration terminal. The figure which shows an example of a structure of the transmission data from the calibration terminal in this Embodiment to a speech recognition server. The figure which shows an example of a structure of the transmission data from the speech recognition server in this Embodiment to an information processing apparatus. The flowchart which shows an example of the operation conversion process by the information display apparatus in this Embodiment

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. This embodiment is an example in which the present invention is applied to an input support system that adds a voice input function to an electronic medical chart of a hospital.

  First, the outline of the input support system according to the present invention will be described.

  FIG. 1 is a system configuration diagram showing a configuration of an input support system according to an embodiment of the present invention.

  In FIG. 1, the input support system 100 includes an information processing device 200, a hospital LAN 300 to which the information processing device 200 is connected, and a wireless communication terminal 400. The information processing device 200 and the in-hospital LAN 300 are arranged in a hospital. The wireless communication terminal 400 is carried by a user (physician) of the information processing apparatus 200 and is located in the hospital when the user operates the information processing apparatus 200.

  In-hospital LAN 300, information processing device 200, and wireless communication terminal 400 are actually arranged in a plurality, although not shown.

  The input support system 100 includes a public communication network 500, a voice recognition server 600 connected to the public communication network 500, and first to Nth calibration terminals 700-1 to 700-N. The public communication network 500, the voice recognition server 600, and the first to Nth calibration terminals 700-1 to 700-N are arranged outside the hospital.

  The information processing apparatus 200 is a personal computer on which an electronic medical record application (hereinafter referred to as “electronic medical record”) is mounted. The information input to the electronic medical record is transmitted to an information server (not shown) arranged in the hospital LAN 300, for example. The information processing apparatus 200 implements a voice input operation on the electronic medical chart using the wireless communication terminal 400, the public communication network 500, the voice recognition server 600, and the first to Nth calibration terminals 700-1 to 700-N. .

  In the present embodiment, the electronic medical record is existing application software that displays an input screen using a window whose position and size are variable. On the input screen, a plurality of input areas such as text boxes and a plurality of label texts are arranged. The input area (component) is an image area that accepts at least one of text input and item selection, and is, for example, a text box or a selection button. The label text is text other than text (hereinafter referred to as “input text”) input by the user by voice input or the like displayed on the input screen, and is, for example, a character string displayed as an attribute of the input area. In addition, the electronic medical record in the present embodiment is an application in which reading is not set for each input area and high security is required.

  The in-hospital LAN 300 is a communication network that exchanges various types of highly confidential information such as patient information.

  The wireless communication terminal 400 is, for example, a mobile phone that is carried by a user (physician) of the information processing apparatus 200 and has a wireless LAN function and an Internet communication function. When the wireless communication terminal 400 is located in the hospital, it can be connected to each of the information processing apparatus 200 and the public communication network 500 by wireless communication.

  In addition, the wireless communication terminal 400 inputs a user's uttered voice. The wireless communication terminal 400 uses the public communication network 500, the voice recognition server 600, and the first to Nth calibration terminals 700-1 to 700-N to input voice (hereinafter simply referred to as “terminal input voice”). Get voice recognition result for. Then, the wireless communication terminal 400 transfers the acquired speech recognition result to the information processing apparatus 200.

  The public communication network 500 is a communication network to which an unspecified number of terminals can be connected, such as the Internet.

  The speech recognition server 600 performs highly accurate speech recognition processing on terminal input speech. The voice recognition server 600 obtains a voice recognition result with higher accuracy using the public communication network 500 and the first to Nth calibration terminals 700-1 to 700-N.

  The first to Nth calibration terminals 700-1 to 700-N accept a calibration operation on the result of the speech recognition processing performed by the speech recognition server 600 (hereinafter, referred to as “uncalibrated speech recognition result” as appropriate). Hereinafter, the person who performs the calibration work using the first to Nth calibration terminals 700-1 to 700-N is referred to as “clerk”. The result of the calibration work is appropriately referred to as “calibrated speech recognition result”.

  Such an input support system 100 can obtain a highly accurate speech recognition result by incorporating the calibration work by Clark. Further, by enabling calibration work with a plurality of calibration terminals 700 connected to the public communication network 500, it is possible to increase human resources that can be used, and to obtain more accurate speech recognition results in a shorter time. Can be obtained.

  However, from the viewpoint of ensuring the security of the electronic medical record and the in-hospital LAN 300, the information processing apparatus 200 should not be directly connected to the public communication network 500.

  Thus, as described above, the input support system 100 makes the connection between the information processing apparatus 200 and the public communication network 500 indirect via wireless communication with the wireless communication terminal 400. In addition, the input support system 100 can basically perform only one-way transfer of a voice recognition result by performing voice input with the wireless communication terminal 400, and the information processing apparatus 200 and the public communication network 500 can be used. The frequency of communication with can be greatly reduced.

  Thereby, the input support system 100 can add a highly accurate voice input function to the electronic medical chart while ensuring the security of the electronic medical chart and the in-hospital LAN 300.

  Moreover, the electronic medical record in this Embodiment is an existing application as above-mentioned, and there exists a need to use as it is.

  However, as described above, reading is not set in advance in each input area of the electronic medical record in the present embodiment.

  Therefore, the input support system 100 analyzes the input screen in the information processing apparatus 200, and displays the display position of the input area (hereinafter referred to as “input area position”) and the display position of the label text (hereinafter referred to as “label text position”). Based on the relative relationship, the label text is associated with the input area position. When the information processing apparatus 200 obtains a voice recognition result corresponding to the label text, the input support system 100 selects operation information for selecting the input area position corresponding to the label text (selection operation for the input area position is performed). (Operation information shown) is input to the electronic medical record.

  Thereby, the input support system 100 can add a voice input function to the electronic medical record without modifying the electronic medical record. That is, the input support system 100 can easily add a high-accuracy voice input function to an electronic medical record that is an existing application that requires high security.

  Note that the input support system 100 may acquire the input area position in the information processing apparatus 200 and manually associate the label text with the acquired input area position from the user.

  This is the end of the description of the outline of the input support system 100.

  Next, the configuration of each device will be described.

  FIG. 2 is a block diagram illustrating an example of the configuration of the information processing apparatus 200.

  In FIG. 2, the information processing apparatus 200 includes a LAN communication unit 201, a terminal communication unit 202, an electronic medical record unit 203, an image output unit 204, and an operation input unit 205. The information processing apparatus 200 includes a table creation rule storage unit 206, a screen configuration analysis unit 207, a screen configuration table storage unit 208, an operation conversion rule storage unit 209, and an input information generation unit 210.

  Of these functional units, for example, the table creation rule storage unit 206, the screen configuration analysis unit 207, and the screen configuration table storage unit 208 constitute an application for setting the association between the input area position and the label text. .

  In addition, the operation conversion rule storage unit 209 and the input information generation unit 210 constitute an application that inputs a speech recognition result to the electronic medical record based on the set association.

  The LAN communication unit 201 is a LAN interface, for example, and is connected to the hospital LAN 300 so as to be communicable.

  The terminal communication unit 202 is a wireless LAN interface, for example, and performs wireless communication with the wireless communication terminal 400.

  The electronic medical chart unit 203 is a functional unit that realizes the above-described electronic medical chart. The electronic medical chart unit 203 communicates with the information server of the hospital LAN 300 via the LAN communication unit 201, for example.

  The image output unit 204 is a liquid crystal display (not shown), for example, and displays an input screen generated by the electronic medical chart unit 203.

  The operation input unit 205 is, for example, a mouse and a keyboard (not shown), and accepts a manual input operation on the electronic medical record unit 203 from the user.

  The table creation rule storage unit 206 stores table creation rules in advance. The table creation rule is a rule that defines a condition for associating a label text with an input area position based on a relative relationship between the input area position and the label text position. This relative relationship includes at least the arrangement direction and the distance.

  FIG. 3 is a diagram illustrating an example of the contents of the table creation rule.

  As illustrated in FIG. 3, the table creation rule 810 describes a correspondence target 813 in which a priority order 812 is set for each type 811. The type 811 indicates the type of input form of the input area (hereinafter simply referred to as “type”), and includes “tab”, “text box”, “select button”, and “select menu”. The priority 812 indicates that when there is an association target with a higher priority 812, the association target with a lower priority 812 should be ignored. The association target 813 indicates a label text condition to be associated with the input area.

Here, the symbol D _x represents the x-axis direction of the separation between the input area and the label text, symbol D _y has a represents the y-axis direction of the separation between the input area and the label text To do. The symbol | Δx | represents the difference between the x-axis coordinate of the input area and the x-axis coordinate of the label text, and the symbol | Δy | represents the difference between the y-axis coordinate of the input area and the y-axis coordinate of the label text. Represent.

For example, “label text of D _x <P ₁ and | Δy | <P ₂ located on the left side” is described in association with the priority 812 of “1” of the type 811 of “text box”. This indicates that if there is a label text located on the left side of the text box and satisfying D _x <P ₁ and | Δy | <P ₂ , the label text should be associated with the text box.

Note that, P _1, the user, in the normal sense, in the x-axis direction of the distance and feel information indicating an attribute of the input area of the label text on the left side of the input area, which is the upper limit. Further, P _2, the user, in the normal sense, the difference between the y-axis coordinate and feel information indicating an attribute of the input area of the label text on the left side of the input area, which is the upper limit.

Therefore, the correspondence that associates the label text that is located on the left side of the text box and satisfies D _x <P ₁ and | Δy | <P ₂ with the text box matches the user's feeling when viewing the input screen. . Note that the other associations of the table creation rules 810 are also adapted to the user's sense.

  Therefore, by associating the label text with the input area based on the table creation rule 810, it is possible to set the label text for each input area with the contents suitable for the user's sense.

  The screen configuration analysis unit 207 in FIG. 2 inquires of the electronic medical record unit 203 and acquires the input area position and the label text position. Then, the screen configuration analysis unit 207 generates a screen configuration table from the acquired input area position and label text position based on the table creation rule. The screen configuration table is a table describing the association of the label text with the input area position and the ID for each association. In other words, the ID described in the screen configuration table is information (first information) that specifies the reading representing the input area, and is information (third information) that specifies the reading of the label text.

  In addition, the screen configuration analysis unit 207 transmits a set of ID and label text to the wireless communication terminal 400 via the terminal communication unit 202 for each association described in the screen configuration table.

  Further, when there is a selection menu, the screen configuration analysis unit 207 transmits dictionary information in which the ID of the selection menu and a plurality of menu item texts are combined to the wireless communication terminal via the terminal communication unit 202. The selection menu is an input area that accepts selection of a plurality of menu item texts prepared in advance, and is, for example, a pull-down menu.

  The screen configuration table storage unit 208 stores the screen configuration table generated by the screen configuration analysis unit 207.

  As described above, the screen configuration analysis unit 207 may manually associate the label text with the input area position from the user.

  In this case, the information processing apparatus 200 does not necessarily include the table creation rule storage unit 206. In addition, the merit in this case is that, for example, a reading that is easy for a user to speak (for example, an abbreviation or a foreign language) or a reading that is easy to remember can be set freely.

  The operation conversion rule storage unit 209 stores operation conversion rules in advance. The operation conversion rule is a rule for converting information (second information) representing the result of the speech recognition processing into operation information using the screen configuration table.

  FIG. 4 is a diagram illustrating an example of the contents of the operation conversion rule.

  As shown in FIG. 4, the operation conversion rule 820 describes the content 822 of the operation information in association with the type 821.

  The content of the operation information 822 is operation information to be output regarding an input area (hereinafter referred to as “operation target input area”) corresponding to the label text when the label text corresponding to the result of the speech recognition processing exists in the screen configuration table. The contents of

  For example, the content 822 of the operation information “one click with the acquired absolute coordinates + input text” is described in association with the type 821 of “text box”. This is one-click operation at the absolute coordinate position of the reference point (for example, the upper left corner) of the input area indicated by the result of the speech recognition process, and an operation of continuously inputting the input text included in the result of the speech recognition process. This indicates that the operation information indicating that the information should be output.

  The input information generation unit 210 in FIG. 2 operates the operation information for the display position when the information representing the result of the speech recognition process for the terminal input voice is associated with the display position of any input area in the screen configuration table. Is input to the electronic medical chart unit 203.

  More specifically, the input information generation unit 210 searches the screen configuration table for information representing the result of the speech recognition process sent from the wireless communication terminal 400. Then, when the ID matches the ID described in the screen configuration table, the input information generation unit 210 inputs operation information indicating a selection operation for the display position of the input area corresponding to the ID to the electronic medical chart unit 203. To do.

  In addition, when the information representing the result of the speech recognition process indicates the ID of the selection menu and the input text, the input information generation unit 210 displays operation information indicating a selection operation for the display position of the input area corresponding to the ID. First, an input is made to the electronic medical chart unit 203. Subsequently, the input information generation unit 210 inputs operation information indicating a text input operation to the electronic medical chart unit 203.

  Note that the input information generation unit 210 acquires the operation information by converting the result of the speech recognition process using the operation conversion rule (see FIG. 4).

  FIG. 5 is a block diagram illustrating an exemplary configuration of the wireless communication terminal 400.

  In FIG. 5, the wireless communication terminal 400 includes a network communication unit 401, a device communication unit 402, a voice input unit 403, and a voice recognition management unit 404.

  The network communication unit 401 is a wireless communication interface for performing wireless communication with, for example, a mobile phone network (not shown) connected to the public communication network 500. The network communication unit 401 is communicably connected to each of the voice recognition server 600 and the first to Nth calibration terminals 700-1 to 700-N via the mobile phone network and the public communication network 500.

  The device communication unit 402 is, for example, a wireless LAN interface, and is connected to the information processing device 200 via a communication path different from the public communication network 500 (here, a wireless LAN).

  The voice input unit 403 inputs a user's uttered voice.

  The voice recognition management unit 404 transmits terminal input voice, which is voice input by the voice input unit 403, to the voice recognition server 600 via the network communication unit 401 as a target of voice recognition processing.

  Further, the voice recognition management unit 404 receives the result of the calibration work (calibrated voice recognition result) from the first to Nth calibration terminals 700-1 to 700-N via the network communication unit 401. Then, the voice recognition management unit 404 transmits the received calibration work result as operation information for the electronic medical chart unit 203 to the information processing apparatus 200 via the apparatus communication unit 402.

  In addition, the voice recognition management unit 404 transmits the set of ID and label text received from the information processing apparatus 200 via the apparatus communication unit 402 to the voice recognition server 600 via the network communication unit 401.

  FIG. 6 is a block diagram illustrating an example of the configuration of the voice recognition server 600.

  6, the voice recognition server 600 includes a network communication unit 601, a voice recognition database 602, a reverse recognition processing unit 603, a correspondence table storage unit 604, and a voice recognition processing unit 605. The speech recognition server 600 also includes a device information storage unit 606, a clerk information storage unit 607, a clerk determination rule storage unit 608, and a calibration management unit 609.

  The network communication unit 601 is a LAN interface for performing wired communication with a provider network (not shown) connected to the public communication network 500, for example. The network communication unit 601 is communicably connected to the speech recognition server 600 and the first to Nth calibration terminals 700-1 to 700-N via the provider network and the public communication network 500.

  The speech recognition database 602 stores data such as acoustic models, language models, and dictionaries used for speech recognition processing.

  The reverse recognition processing unit 603 acquires the reading of the label text for each set of ID and label text received from the wireless communication terminal 400 by reverse recognition processing for the label text. And the reverse recognition process part 603 produces | generates the ID reading corresponding | compatible table which matched acquired reading and ID.

  The correspondence table storage unit 604 stores the ID reading correspondence table generated by the reverse recognition processing unit 603.

  The speech recognition processing unit 605 performs speech recognition processing on the terminal input speech received from the wireless communication terminal 400 using the speech recognition database 602.

  The device information storage unit 606 stores device information. The apparatus information includes at least one attribute information of the information processing apparatus 200, an application (in this case, the electronic medical chart unit 203) that is a target of voice input, and a user of the application.

  FIG. 7 is a diagram illustrating an example of the contents of the device information.

  As illustrated in FIG. 7, the device information 830 describes terminal ID 831, device ID 832, field 833, desired level 834, and desired unit price 835 in association with each other.

  The terminal ID 831 is an ID of a wireless communication terminal (including other wireless communication terminals). The device ID is an ID of an information processing device (including other information processing devices) to which a voice recognition function is added using the wireless communication terminal indicated by the terminal ID 831. A field 833 indicates a field of an application that is a target of voice input in the information processing apparatus indicated by the apparatus ID. The desired level 834 indicates the level of accuracy of the calibration of the speech recognition result for the application desired by the user of the information processing apparatus indicated by the apparatus ID (hereinafter simply referred to as “level”). The desired unit price 835 indicates the unit price of voice recognition for the application desired by the user (for example, the amount per voice per minute. Hereinafter, simply referred to as “unit price”).

  For example, a terminal ID 831 “T1” is associated with a device ID 832 “M1”. This indicates that for the terminal input voice received from the wireless communication terminal whose terminal ID is “T1”, the voice recognition result should be transmitted to the information processing apparatus whose apparatus ID is “M1”. . Here, it is assumed that the terminal ID of the wireless communication terminal 400 is “T1”. Further, it is assumed that the device ID of the information processing device 200 is “M1”. Further, a field “833” “medical” is described in association with the terminal ID “T1”. This indicates that the application (in this case, the electronic medical chart unit 203) that is the target of voice input in the corresponding information processing apparatus 200 is an application in the medical field. Further, a desired level 834 of “2 or more” and a desired unit price 835 of “100 yen / minute or less” are described in association with the terminal ID “T1”. This indicates that the desired level of accuracy of the calibration work is 2 or more and the desired unit price is 100 yen or less per minute.

  For example, the device information 830 may be set manually by an operator in advance, or may be created by the calibration management unit 609 inquiring information necessary for the information processing device and the wireless communication terminal.

  The clerk information storage unit 607 in FIG. 6 stores clerk information. The clerk information includes attribute information of the operator of the calibration terminal. In the present embodiment, it is assumed that the clerk information includes information indicating whether or not the calibration work is currently possible at the calibration terminal and information indicating the range of the target of the operator's calibration work.

  FIG. 8 is a diagram illustrating an example of the contents of the clerk information.

  As shown in FIG. 8, the clerk information 840 describes a calibration terminal ID 841, a clerk ID 842, a work status 843, a weak field 844, a strong field 845, a level 846, and a unit price 847.

  The calibration terminal ID 841 is an ID of the calibration terminal 700. The Clark ID 842 is an ID of a Clark who uses the calibration terminal 700 indicated by the calibration terminal ID 841. The work status 843 indicates whether the clerk indicated by the clerk ID 842 is currently in a state where the calibration work can be performed immediately. The weak field 844 indicates a field in which the Clark is not good at calibration work. A specialty field 844 indicates a field in which the Clark is good at calibration work. Level 846 indicates the level of the clerk. The unit price 847 indicates the unit price when the calibration work is performed by the clerk.

  For example, a Clark ID 842 “C1” is described in association with the calibration terminal ID 841 “P1”. This indicates that the uncalibrated speech recognition result transmitted to the calibration terminal ID 841 “P1” is subjected to calibration work by the Clark ID 842 “C1”. Further, a work status 843 “work is possible”, a weak field 844 “literature”, and a good field 845 “medical” are described in association with the Clark ID 842 “C1”. This indicates that the clerk with the clerk ID “C1” is currently capable of immediate proofreading, the weak field is literature, and the strong field is medical. Further, a level 846 of “1” and a unit price 847 of “90 yen / min” are described in association with the Clark ID 842 of “C1”. This indicates that the level of the clerk is 1, and that the unit price when the clerk performs the calibration work is 90 yen per minute.

  The clerk determination rule storage unit 608 in FIG. 6 stores clerk determination rules in advance. The clerk determination rule is a rule that defines the selection condition of the calibration terminal 700 for the result of the speech recognition process based on the relationship between the device information and the clerk information. In the present embodiment, it is assumed that the Clark determination rule at least defines that a calibration terminal that can currently perform calibration work is selected. In addition, the clerk determination rule at least stipulates that when there is an application (in this case, the electronic medical chart unit 203) that is a target of voice input and an operator that includes the user in a good range, the calibration terminal 700 of the operator is selected. It shall be.

  FIG. 9 is a diagram illustrating an example of the contents of the clerk determination rule.

  As shown in FIG. 9, the clerk determination rule 850 describes a condition 852 in which a priority order 851 is set.

  Priority 851 indicates that a clerk that satisfies the higher priority 851 condition should be selected more preferentially. A condition 852 indicates a condition of the Clark to be calibrated.

  For example, a condition 852 that “work is possible” is described in association with the priority order “1”. This indicates that being ready to work is the top priority condition for Clark to be calibrated.

  The calibration management unit 609 in FIG. 6 acquires corresponding device information for each result of speech recognition processing (uncalibrated speech recognition result) by the speech recognition processing unit 605. The calibration management unit 609 selects one or a plurality of calibration terminals 700 from the first to Nth calibration terminals 700-1 to 700-N based on the Clark determination rule. Then, the calibration management unit 609 causes the selected calibration terminal 700 to perform a calibration operation on the result of the speech recognition process. Thereafter, the calibration management unit 609 searches the ID reading correspondence table for the reading indicated by the calibration work result (calibrated speech recognition result) received from the calibration terminal 700. When the reading exists in the ID reading correspondence table, the calibration management unit 609 transmits the ID corresponding to the reading to the wireless communication terminal 400.

  The information processing apparatus 200, the wireless communication terminal 400, and the speech recognition server 600 described above include, for example, a CPU (Central Processing Unit), a storage medium such as a ROM (Read Only Memory) storing a control program, and a RAM (Random Access Memory). ) And the like. In this case, the function of each component described above is realized by the CPU executing the control program.

  Each function described in FIG. 1 is realized by the information processing apparatus 200, the wireless communication terminal 400, and the voice recognition server 600 configured as described above. That is, the input support system 100 includes these devices and the first to N-th calibration terminals 700-1 to 700-N, so that the electronic medical record, which is an existing application that requires high security, is highly effective. Accurate voice input function can be easily added.

  This is the end of the description of the configuration of each device.

  Hereinafter, the operation of each apparatus will be described. However, the description will be given focusing only on the processing related to the addition of the voice recognition function to the electronic medical record.

  FIG. 10 is a flowchart illustrating an example of the operation of the information processing apparatus 200.

  First, in step S1100, the screen configuration analysis unit 207 determines whether the electronic medical record unit 203 has displayed a new input screen.

  The new input screen is not only an input screen whose contents are different from the input screen that has been analyzed (that is, the corresponding screen configuration table has been generated), but at least one of the size and arrangement is different even if the contents are the same. Includes input screens.

  FIG. 11 is a plan view showing an example of the input screen.

  As shown in FIG. 11, the input screen 861 is a window displayed on the desktop screen 862 of the information processing apparatus 200. For example, the position and size of the input screen 861 on the desktop screen 862 can be changed by the operation of the cursor 863 performed by the user via the operation input unit 205.

  In the present embodiment, the input screen 861 has a tab 865 in which a label text 864 of “patient information” is arranged, and a tab 867 in which a label text 866 of “hospital information” is arranged. The input screen 861 has a text box 869 in which a label text 868 of “name” is arranged near the left side. The input screen 861 has a selection button 872 in which a label text 870 “sex” is arranged near the left side and a label text 871 “male” is arranged near the right side. The input screen 861 includes a selection button 874 in which a label text 870 “sex” is arranged in the vicinity of the left side, and a label text 873 of “female” is arranged in the vicinity of the right side. The input screen 861 has a selection menu 876 in which a label text 875 “medical subject” is arranged near the left side. The selection menu 876 has a menu item text group 877 of “Internal medicine”, “Surgery”, “Orthopedics”,.

  As will be described later, the screen configuration analysis unit 207 uses the absolute coordinate system 878 that is the coordinate system of the desktop screen 862 and the relative coordinate system 879 that is the coordinate system of the input screen 861 to analyze the input screen 861 ( Hereinafter, “screen configuration analysis” is performed as appropriate. The absolute coordinate system 878 is composed of an X axis and a Y axis. The relative coordinate system 879 is composed of an x-axis and a y-axis. Note that the absolute coordinate system 878 and the relative coordinate system 879 shown in FIG. 11 are not displayed on the desktop screen 862 and the input screen 861.

  In addition, although what differs only in the position in an absolute coordinate system may be handled as a new input screen, the information processing apparatus 200 which concerns on this Embodiment shall not be handled as a new input screen. However, it is assumed that the information processing apparatus 200 sequentially acquires the position of the input screen (relative coordinate system) in the absolute coordinate system and generates operation information with the contents using the absolute operation coordinate system.

  If the electronic medical chart unit 203 displays a new input screen (S1100: YES in FIG. 10), the screen configuration analysis unit 207 proceeds to step S1200. If the electronic medical chart unit 203 does not display a new input screen (S1100: NO), the image configuration analysis unit 207 proceeds to step S1300.

  In step S1200, the screen configuration analysis unit 207 performs table generation processing in step S1100, and proceeds to step S1300. The table generation process is a process of analyzing the input screen and generating a screen configuration table. Details of the table generation processing will be described later.

  In step S1300, the input information generation unit 210 determines whether a new speech recognition result has been received. When the input information generation unit 210 receives a new speech recognition result (S1300: YES), the input information generation unit 210 proceeds to step S1400. If the input information generation unit 210 has not received a new speech recognition result (S1300: NO), the process proceeds to step S1500.

  In step S1400, the input information generation unit 210 performs an operation conversion process, and proceeds to step S1500. The operation conversion process is a process of converting the result of the voice recognition process for the terminal input voice into operation information and passing it to the electronic medical chart unit 203. Details of the operation conversion process will be described later.

  In step S1500, the input information generation unit 210 determines whether an instruction to end the process is given by a user operation or the like. If the input information generation unit 210 is not instructed to end the process (S1500: NO), the process returns to step S1100. Further, when instructed to end the process (S1500: YES), the input information generation unit 210 ends the series of processes.

  With this operation, the information processing apparatus 200 can generate a screen configuration table corresponding to an input screen each time a new input screen is displayed. Then, each time a new voice recognition result is received, the information processing apparatus 200 can convert the voice recognition result into operation information.

  FIG. 12 is a flowchart illustrating an example of the table generation process (step S1200 in FIG. 10).

  First, in step S <b> 1201, the screen configuration analysis unit 207 acquires screen information of the input screen from the electronic medical record unit 203. The screen information includes the type and display position of each input area constituting the input screen, each label text constituting the input screen, and the display position and size. Screen information can be acquired by passing a corresponding Windows message to the electronic medical chart unit 203 when the input screen is a window in Microsoft Windows (registered trademark).

  In step S1202, the screen configuration analysis unit 207 selects one of the acquired input areas.

  In step S1203, the screen configuration analysis unit 207 associates the label text with the selected input area according to the table creation rule (see FIG. 3).

  For example, when the text box 869 is selected on the input screen 861 illustrated in FIG. 11, the screen configuration analysis unit 207 associates the label text 868 “name” with the text box 869.

  In step S1204, the screen configuration analysis unit 207 registers the association performed in step S1203, the component type, relative coordinates, and the like in the screen configuration table. Such registration in the screen configuration table is performed for all input areas by processing in step S1207 described later.

  FIG. 13 is a diagram showing an example of the contents of the screen configuration table generated from the input screen shown in FIG. 11 based on the table creation rules shown in FIG.

  As shown in FIG. 13, the screen configuration table 880 describes a parent ID 882, a type 883, a relative coordinate 884, a property 885, and a label text 886 for each field ID 881.

  The field ID 881 is an input area ID. The parent ID 882 is the ID of the parent input area that is positioned as the parent of the input area. The type 883 is the type of the input area indicated by the field ID 881. The relative coordinates 884 are the coordinates of the reference point of the input area indicated by the field ID 881 in the relative coordinate system 829 of the input screen 861 (see FIG. 11). A property 885 is a tab index when the input area indicated by the field ID 881 is a tab. The label text 886 is label text 886 associated with the input area indicated by the field ID 881.

For example, a parent ID 882 “1”, a type 883 “text box”, and a relative coordinate 884 “x ₂ , y ₂ ” are described in association with a field ID 881 “2”. This is because the input area whose field ID 881 is “2” is a text box whose parent is the input area whose field ID 881 is “1”, and the relative coordinates of the reference point are (x ₂ , y ₂ ). It shows that. Further, a label text 886 of “name” is described in association with the field ID 881 of “2”. This indicates that the label text 886 “name” is associated with the input area whose field ID 881 is “2”.

  In step S1205 of FIG. 12, the screen configuration analysis unit 207 determines whether or not the type of the input area being selected is a selection menu. If the currently selected input area is a selection menu (S1205: YES), screen configuration analysis unit 207 proceeds to step S1206. If the input area being selected is not a selection menu (S1205: NO), the screen configuration analysis unit 207 proceeds to step S1207.

  In step S1206, the screen configuration analysis unit 207 generates dictionary information from the menu item text of the selection menu, and proceeds to step S1207.

  FIG. 14 is a diagram showing an example of the contents of the dictionary information generated from the input screen shown in FIG.

  As illustrated in FIG. 14, the dictionary information 890 describes a list of menu item text 892 in association with the field ID 891. The menu item text 892 indicates the menu item text to be selected in the input area (selection menu) indicated by the field ID 891.

  Then, in step S1207 in FIG. 12, the screen configuration analysis unit 207 determines whether or not all input areas constituting the input screen have been processed. If there is an unprocessed input area (S1207: NO), the screen configuration analysis unit 207 returns to step S1202. If the screen configuration analysis unit 207 has processed all input areas (S1207: YES), the screen configuration analysis unit 207 proceeds to step S1208.

  In step S1208, the screen configuration analysis unit 207 generates ID text correspondence information from the screen configuration table. The ID text correspondence information is information indicating the correspondence between the field ID and the label text.

  FIG. 15 is a diagram showing an example of the contents of ID text correspondence information generated from the screen configuration table shown in FIG.

  As shown in FIG. 15, ID text correspondence information 900 describes a set of field ID 901 and label text 902 and type 903. A label text 902 indicates a label text associated with the input area indicated by the field ID 901. A type 903 indicates the type of the input area indicated by the field ID 901.

  In step S1209 of FIG. 12, the screen configuration analysis unit 207 transmits the generated ID text correspondence information to the wireless communication terminal 400. Further, when generating the dictionary information, the screen configuration analysis unit 207 also transmits this to the wireless communication terminal 400. Then, the screen configuration analysis unit 207 returns to the process of FIG.

  FIG. 16 is a diagram illustrating an example of a configuration of transmission data from the information processing apparatus 200 to the wireless communication terminal 400.

  As illustrated in FIG. 16, transmission data 910 from the information processing apparatus 200 to the wireless communication terminal 400 includes, for example, ID text correspondence information 911 and further includes dictionary information 912 as appropriate.

  By such table generation processing, the information processing apparatus 200 can determine the association of the label text with the input area position from the input screen of the electronic medical chart unit 203. Then, the information processing apparatus 200 can store the association as screen configuration data and transmit the association to the wireless communication terminal 400 as ID text correspondence information.

  FIG. 17 is a flowchart illustrating an example of the operation of the wireless communication terminal 400.

  First, the speech recognition management unit 404 determines whether or not at least one of ID text correspondence information and dictionary information has been received from the information processing apparatus 200. If the voice recognition management unit 404 receives at least one of ID text correspondence information and dictionary information (S2010: YES), the process proceeds to step S2020. If neither the ID text correspondence information nor the dictionary information is received (S2010: NO), the speech recognition management unit 404 proceeds to step S2030.

  In step S2020, the speech recognition management unit 404 transfers the received ID text information and dictionary information to the speech recognition server 600, and proceeds to step S2030. At this time, the voice recognition management unit 404 assigns the terminal ID of the wireless communication terminal 400 to the information to be transferred as the transmission source ID.

  FIG. 18 is a diagram illustrating an example of a configuration of first transmission data from the wireless communication terminal 400 to the voice recognition server 600.

  As shown in FIG. 18, the first transmission data 920 from the wireless communication terminal 400 to the voice recognition server 600 includes, for example, a terminal ID 921 and ID text correspondence information 922, and further includes dictionary information 923 as appropriate. The terminal ID 921 indicates the terminal ID of the wireless communication terminal 400 that is the transmission source of the first transmission data 920. ID text correspondence information 922 and dictionary information 923 correspond to ID text correspondence information 911 and dictionary information 912 in FIG.

  In step S2030 in FIG. 17, the voice recognition management unit 404 determines whether or not there is a user voice input via the voice input unit 403. If there is a voice input (S2030: YES), the voice recognition management unit 404 proceeds to step S2040. If there is no voice input (S2030: NO), the voice recognition management unit 404 proceeds to step S2050.

  In step S2040, the voice recognition management unit 404 transmits the voice information including the voice data of the terminal input voice to the voice recognition server 600, and proceeds to step S2050. At this time, the voice recognition management unit 404 gives the terminal ID of the wireless communication terminal 400 as the transmission source to the information to be transmitted.

  FIG. 19 is a diagram illustrating an example of a configuration of second transmission data from the wireless communication terminal 400 to the voice recognition server 600.

  As illustrated in FIG. 19, the second transmission data 930 from the wireless communication terminal 400 to the voice recognition server 600 includes, for example, a terminal ID 931 and voice information 932. The terminal ID 931 indicates the terminal ID of the wireless communication terminal 400 that is the transmission source of the second transmission data 930. The voice information 932 is information including voice data of terminal input voice.

  In step S2050 of FIG. 17, the speech recognition management unit 404 determines whether a speech recognition result has been received from the speech recognition server 600. If the speech recognition management unit 404 receives a speech recognition result (S2050: YES), the speech recognition management unit 404 proceeds to step S2060. If the voice recognition management unit 404 has not received a voice recognition result (S2050: NO), the process proceeds to step S2070.

  In step S2060, the voice recognition management unit 404 transfers the received voice recognition result to the information processing apparatus 200, and proceeds to step S2070. Of the ID text correspondence information and the dictionary information, the received information is transferred to the voice recognition server 600, and the process proceeds to step S2030. Details of the transferred data will be described later.

  In step S2070, the voice recognition management unit 404 determines whether an instruction to end the process is given by a user operation or the like. If the voice recognition management unit 404 is not instructed to end the process (S2070: NO), the process returns to step S2010. Further, when instructed to end the process (S2070: YES), the voice recognition management unit 404 ends the series of processes.

  By such an operation, the wireless communication terminal 400 transfers the ID text correspondence information sent from the information processing apparatus 200 to the voice recognition server 600 and transmits the voice information of the terminal input voice to the voice recognition server 600. be able to. In addition, the wireless communication terminal 400 can transfer the voice recognition result sent from the voice recognition server 600 to the information processing apparatus 200.

  FIG. 20 is a flowchart illustrating an example of the operation of the speech recognition apparatus 600.

  First, in step S3010, the reverse recognition processing unit 603 determines whether or not ID text correspondence information (see FIG. 15) has been received from the wireless communication terminal 400. If the ID text correspondence information is not received (S3010: NO), the reverse recognition processing unit 603 proceeds to step S3030 described later. If the ID recognition information is received (S3010: YES), the reverse recognition processing unit 603 proceeds to step S3020. Further, the reverse authentication processing unit 603 reversely converts the label text included in the ID text correspondence information into reading, and generates an ID reading correspondence table.

  FIG. 21 is a diagram showing an example of the contents of the ID reading correspondence table generated from the ID text correspondence information shown in FIG.

  As shown in FIG. 21, the ID reading correspondence table 940 describes the reading 942 in association with the field ID 941. A reading 942 indicates the reading of the label text associated with the input area indicated by the field ID 941.

  For example, the reading 942 “Shimei” is described in association with the field ID 941 “2”. This indicates that the reading of the label text associated with the input area whose field ID 941 is “2” is “Shime”.

  The reverse recognition processing unit 603 constructs or reconstructs a language model from the input area type and label text described in the received ID text correspondence information and the reading obtained by the reverse conversion process, and performs speech recognition. Register in the database 602.

  For example, the reverse recognition processing unit 603 reads the name immediately after the reading “Shimei” because the reading “Shimei” corresponds to a text box and the label text of the group is “name”. The content that is placed is added to the language model. In addition, for example, the reading “Shinryokamo” corresponds to the selection menu. From this, the reverse recognition processing unit 603 determines that the menu item text described in the corresponding dictionary information received immediately after reading “Shinryokamoku” is the language model. Add to

  Note that the speech recognition server 600 manages the ID reading correspondence table 940, the language model, dictionary information, and the like for each wireless communication terminal 400 using the terminal ID.

  In step S3030 of FIG. 20, the reverse recognition processing unit 603 determines whether dictionary information (see FIG. 14) has been received from the wireless communication terminal 400. When the reverse recognition processing unit 603 receives the dictionary information (S3030: YES), the process proceeds to step S3040. If the reverse recognition processing unit 603 has not received a dictionary dictionary (S3030: NO), the process proceeds to step S3050.

  In step S3040, the reverse recognition processing unit 603 registers the menu item text list included in the dictionary information in the speech recognition database 602 as a speech recognition result candidate (dictionary) for the field ID included in the dictionary information. .

  Note that the speech recognition server 600 manages a speech recognition result candidate (dictionary) for each wireless communication terminal 400 using the terminal ID.

  In step S <b> 3050, the voice recognition processing unit 605 determines whether voice information has been received from the wireless communication terminal 400. If the voice recognition processing unit 605 receives voice information (S3050: YES), the process proceeds to step S3060. If the voice recognition processing unit 605 has not received voice information (S3050: NO), the process proceeds to step S3070.

  In step S3060, the speech recognition processing unit 605 performs speech recognition processing on the speech data included in the received speech information using the speech recognition database 6020. As a result of the speech recognition processing, a reading obtained by converting the terminal input speech into a kana and a kana-kanji text obtained by converting the reading into kana-kanji are obtained.

  In step S3080, the calibration management unit 609 determines a clerk to be calibrated according to the clerk determination rule (see FIG. 9) based on the device information (see FIG. 7) and the clerk information (see FIG. 8). Then, the calibration management unit 609 transmits the result of the speech recognition process (uncalibrated speech recognition result) to the calibration terminal 700 corresponding to the determined clerk, and proceeds to step S3070.

  FIG. 22 is a diagram illustrating an example of a configuration of transmission data from the speech recognition server 600 to the calibration terminal 700.

  As shown in FIG. 22, the transmission data 950 from the speech recognition server 600 to the calibration terminal 700 includes a terminal ID 951, speech information 952, and an uncalibrated speech recognition result 953.

  The terminal ID 951 indicates the terminal ID of the wireless communication terminal 400 that is the transmission source of the audio information 952. The uncalibrated speech recognition result 953 is a speech recognition result for the speech information 952.

  The uncalibrated speech recognition result 953 is returned as a calibrated speech recognition result after calibrating in the calibration terminal 700.

  FIG. 23 is a diagram illustrating an example of a configuration of transmission data from the calibration terminal 700 to the speech recognition server 600.

  As shown in FIG. 23, transmission data 960 from the calibration terminal 700 to the speech recognition server 600 includes a terminal ID 961 and a calibrated speech recognition result 962. The terminal ID 961 indicates the terminal ID of the wireless communication terminal 400 that is the transmission source of the voice information 952 that is the basis of the calibrated voice recognition result 962.

  In step S3070 of FIG. 20, the calibration management unit 609 determines whether a calibration result (calibrated speech recognition result) has been received from the calibration terminal 700. When the calibration management unit 609 receives the calibration result (S3070: YES), the calibration management unit 609 proceeds to step S3090. If the calibration management unit 609 has not received the calibration result (S3070: NO), the calibration management unit 609 proceeds to step S3100.

  In step S3090, the calibration management unit 609 acquires a field ID corresponding to the received calibrated voice recognition result in the ID reading correspondence table (see FIG. 21). Then, the calibration management unit 609 transmits the acquired field ID to the wireless communication terminal 400 as a calibrated voice recognition result. In addition, the proofread speech recognition result may be composed of a reading and an input text described in the ID reading correspondence table such as “Shimei, Yamada Hanako” and “Name, Hanako Yamada”. In this case, the calibration management unit 609 transmits the field ID and the input text to the wireless communication terminal 400 as a calibrated voice recognition result. At this time, as the input text, text converted to Kana-Kanji (in the above example, “Yamada Hanako” instead of “Yamada Hanako”) is transmitted. Then, the calibration management unit 609 proceeds to step S3100.

  Transmission data from the voice recognition server 600 to the wireless communication terminal 400 is transferred to the information processing apparatus 200 by the wireless communication terminal 400 as described above.

  FIG. 24 is a diagram illustrating an example of a configuration of transmission data from the voice recognition server 600 to the information processing apparatus 200.

  As shown in FIG. 24, the transmission data 970 from the speech recognition server 600 to the information processing apparatus 200 includes a field ID 971 and further includes an input text 972 as appropriate. The input text 972 is included when the type of the input area indicated by the field ID 971 is, for example, a text box or a selection menu.

  In step S3100 in FIG. 20, the calibration management unit 609 determines whether or not an instruction to end the process is given by an operator operation or the like. When the calibration management unit 609 is not instructed to end the process (S3100: NO), the calibration management unit 609 returns to step S3010. In addition, when instructed to end the process (S3100: YES), the calibration management unit 609 ends the series of processes.

  With such an operation, the voice recognition server 600 can perform voice recognition processing with high accuracy by reflecting the attributes and configuration of the input screen, further select an appropriate clerk, perform calibration work, and further High voice recognition processing can be performed.

  FIG. 25 is a flowchart illustrating an example of the operation conversion process (step S1400 in FIG. 10) performed by the information display apparatus 200.

  First, in step S1401, the input information generation unit 210 acquires the absolute coordinates of the input screen 861 (for example, the absolute coordinates of the origin o of the relative coordinate system 879, see FIG. 11) from the electronic medical chart unit 203. The absolute coordinates of the input screen 861 can be obtained by passing a corresponding Windows message to the electronic medical chart unit 203 when the input screen is a window in Microsoft Windows (registered trademark).

  In step S1402, the input information generation unit 210 acquires the relative coordinates of the input area corresponding to the field ID included in the received proofreading recognition result from the screen configuration table. Then, the relative coordinates of the input area are converted into absolute coordinates based on the absolute coordinates of the input screen. For example, when the relative coordinate system and the absolute coordinate system have the same scale, the input information generation unit 210 adds the value of the relative coordinate of the input area and the value of the absolute coordinate of the input screen, thereby obtaining the input area. Get the absolute coordinates of.

  In step S1403, the input information generation unit 210 generates operation information corresponding to the proofread recognition result according to the operation conversion rule (see FIG. 4). Then, the input information generation unit 210 inputs the generated operation information to the electronic medical record unit 203, and returns to the processing of FIG.

For example, it is assumed that a proofread recognition result including a field ID “2” and an input text “Hanako Yamada” is received. In this case, the input information generation unit 210 first determines from the screen configuration table (see FIG. 13) that the type of the corresponding input area is a text box and the relative coordinates thereof are (x ₂ , y ₂ ). get. As a result, the input information generation unit 210 determines to generate, from the operation conversion rule 820, operation information indicating a one-click operation at the absolute coordinates of the corresponding input area and subsequently an input operation of the input text. Then, the input information generation unit 210 converts the relative coordinates (x ₂ , y ₂ ) into absolute coordinates, generates operation information having the determined contents, and inputs the operation information to the electronic medical chart unit 203.

  Through such an operation conversion process, the information processing apparatus 200 can respond to a voice recognition result without modifying the electronic medical chart unit 203 whose input screen size, position, and configuration are variable. It is possible to perform an operation.

  In addition, through the above-described operation, the information processing apparatus 200 uses an external device such as the wireless communication terminal 400 without modifying the electronic application to be mounted, and in addition to ensuring the security of the electronic application, Voice input function can be added to

  As described above, the input support system 100 according to the present embodiment associates a label text with an input area position based on the relative relationship between the input area position and the label text position, The selection operation was enabled. Thereby, the input support system 100 can easily add a voice input function to the electronic medical chart unit 203. That is, the input support system 100 can easily add a voice input function to an existing application in which reading is not set in the input area.

  Further, since the input support system 100 uses the relative coordinates of the input screen, it is possible to prevent the screen configuration analysis from being performed each time the input screen simply moves, thereby reducing the processing load. .

  The input support system 100 also includes a voice recognition server 600 and a plurality of proofreading terminals 700 arranged in the public communication network 500, and uses these to add a voice recognition function to the electronic medical chart unit 203. The wireless communication terminal 400 is arranged between the terminal 200 and the terminal 200. Thus, the input support system 100 can easily add a voice input function to the electronic medical chart unit 203 without connecting the information processing apparatus 200 to the public communication network 500 so as to be directly communicable. That is, the input support system 100 can easily add a highly accurate voice input function to an application that requires high security.

  In the embodiment described above, the input information generation unit 210 is arranged in the information processing apparatus as a function unit for converting the voice recognition result into operation information. However, the arrangement of the function unit is not limited to this. The functional unit that converts the voice recognition result into the operation information may be disposed in the wireless communication terminal 400, for example. In this case, the screen configuration analysis unit 207 may transmit the generated screen configuration table to the wireless terminal device 400 and store it therein.

  Further, when the configuration and size of each page of the input screen are fixed, the screen configuration analysis unit 207 is not necessary once the screen configuration table is generated for all pages. Therefore, in this case, another function unit (for example, the input information generation unit 210) may delete the screen configuration analysis unit 207 from the information processing apparatus 200.

  Further, the input support system 100 may use text data for reading the label text, coordinates of the input area, and the like as the voice recognition result instead of the ID. However, the screen configuration analysis unit 207 needs to generate a screen configuration table so that the input area can be identified from the speech recognition result.

  Further, the input support system 100 may input operation information emulating a keyboard input signal to an application.

  In this case, for example, the voice recognition management unit 404 of the wireless communication terminal 400 converts the received calibration work result into a keyboard input signal (Hiragana character string), and sends it to the information processing device 200 via the device communication unit 402. Send. This keyboard input signal includes, for example, the contents of the keyboard operation to be performed to input the kana-kanji magic text that is the result of the proofreading operation into the hiragana character string, and the subsequent conversion operation (the input hiragana character string This is a signal indicating that all are selected and converted) / confirming operation (confirming to kana / kanji magic).

  In addition, when the input information generation unit 210 of the information processing apparatus 200 receives this keyboard input signal via the terminal communication unit 202 in a state where any of the input areas is selected, the input information generation unit 210 converts the keyboard input signal into the kana kanji. Convert to text. Then, the input information generation unit 210 inputs operation information indicating an input operation of the kana / kanji text for the display position of the selected input area to the electronic medical record unit 203.

  Further, the input support system 100 may manually accept input area selection and use the speech recognition function only for text input. For example, when the keyboard input signal is realized by a wireless signal of a general wireless keyboard, it is input to the electronic medical chart unit 203 through the same route as a normal wireless keyboard input. This is an input function normally provided in the terminal.

  In addition, the conversion function to the kana / kanji text in the input information generation unit 210 may be combined with the function installed in the information processing apparatus 200. Since the speech recognition dictionary with the speech recognition server 600 has a conversion dictionary (a pair of reading and notation) specialized for the target field, it is imported into the kana-kanji conversion (IME) provided in the information processing apparatus 200. It is desirable to keep it. By doing so, the probability that the kana-kanji conversion will be successful in a single conversion operation increases.

  The screen configuration analysis unit 207 may scan an actually displayed image and acquire (extract) the position and type of each input area and the display position of each label text from the input screen. Such extraction can be performed, for example, by pattern matching or image feature amount extraction.

  Of course, the present invention is applicable to various applications other than electronic medical records.

  The present invention is useful as an input support apparatus, an input support method, and an input support program that can easily add a voice input function to an existing application that requires high security.

DESCRIPTION OF SYMBOLS 100 Input support system 200 Information processing apparatus 201 LAN communication part 202 Terminal communication part 203 Electronic medical chart part 204 Image output part 205 Operation input part 206 Table creation rule storage part 207 Screen structure analysis part 208 Screen structure table storage part 209 Operation conversion rule storage Unit 210 Input information generation unit 300 Hospital LAN
400 wireless communication terminal 401 device communication unit 402, 601 network communication unit 403 voice input unit 404 voice recognition management unit 500 public communication network 600 voice recognition server 602 voice recognition database 603 reverse recognition processing unit 604 correspondence table storage unit 605 voice recognition processing unit 606 Device information storage unit 607 Clark information storage unit 608 Clark determination rule storage unit 609 Calibration management unit 700 Calibration terminal

Claims (11)

An input support device that adds a voice input function to application software that displays an input area for inputting information on an input screen,
A screen configuration table storage unit that stores a screen configuration table in which the display position of the input area and the first information specifying the reading representing the input area are associated with each other;
The second information representing the result of the speech recognition process for the speech voice is acquired, the display position of the input area corresponding to the acquired second information is specified with reference to the screen configuration table, and the specified An operation information for selecting a display position, and an input information generation unit for inputting to the application software,
Input support device. The input screen further displays label text indicating the attributes of the input area,
The display position of the input area and the display position of the label text are acquired, and third information for specifying the reading of the label text is determined based on the relative relationship between the display position of the input area and the display position of the label text. A screen configuration analysis unit that generates the screen configuration table in association with the display position of the input area as the first information specifying the reading representing the input area,
The input support apparatus according to claim 1. The third information for specifying the reading of the label text is information for specifying a result of reverse speech recognition processing for the label text.
The input support apparatus according to claim 2. The screen configuration analysis unit
Queries the application software for the display position of the input area and the display position of the label text.
The input support apparatus according to claim 3. The screen configuration analysis unit
Scanning the input screen, and detecting the input area and the label text from the input screen to obtain the display position of the input area and the display position of the label text,
The input support apparatus according to claim 3. The screen configuration table is:
Using the coordinate system of the input screen, associating the display position of the input area with the first information specifying the reading representing the input area,
The input information generation unit
The operation information for obtaining the position of the input screen on the display screen for displaying the input screen, and selecting the display position of the input area using the coordinate system of the display screen based on the obtained position of the input screen. Enter into the application software,
The input support apparatus according to claim 3. The relative relationship includes at least an arrangement direction and a distance,
The input support apparatus according to claim 3. The input information generation unit
An operation for selecting the display position of the input area when the second information representing the result of the speech recognition process is composed of a combination of the first information specifying the reading representing the input area and the input text. Following the information, input operation information for inputting the input text to the application software.
The input support apparatus according to claim 3. The screen configuration analysis unit
For each reading representing the input area, an ID is assigned as the first information for specifying the reading, and for each ID, a set of the ID and the label text is subjected to the reverse speech recognition processing and the speech recognition processing. To the voice recognition server to perform,
The input information generation unit
Receiving the ID corresponding to the result of the voice recognition process as the second information representing the result of the voice recognition process;
The input support apparatus according to claim 3. An input support method for adding a voice input function to application software that displays an input area for inputting information and a label text indicating an attribute of the input area on an input screen,
Obtaining a display position of the input area and a display position of the label text;
Based on the relative relationship between the display position of the input area and the display position of the label text, the information specifying the reading of the label text with respect to the display position of the input area, the information specifying the reading representing the input area Generating a screen configuration table associated with
Obtaining information representing the result of the speech recognition process for the spoken speech;
Identifying the display position of the input area corresponding to the information representing the acquired result of the voice recognition processing with reference to the screen configuration table;
Inputting operation information for selecting the specified display position into the application software,
Input support method. An input support program for adding a voice input function to application software for displaying an input area for inputting information and a label text for indicating an attribute of the input area on an input screen,
A computer accessible to the application software;
Processing for obtaining the display position of the input area and the display position of the label text;
Based on the relative relationship between the display position of the input area and the display position of the label text, the information specifying the reading of the label text with respect to the display position of the input area, the information specifying the reading representing the input area Processing to generate a screen configuration table associated with
A process of acquiring information representing the result of the voice recognition process for the uttered voice;
Processing for specifying the display position of the input area corresponding to the information representing the acquired result of the speech recognition processing with reference to the screen configuration table;
A process of inputting operation information for selecting the specified display position into the application software, and
Input support program.

Images