JP4694758B2 - Apparatus operating device, program, recording medium, and image forming apparatus - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus operating apparatus, a program, a recording medium, and an image forming apparatus, and more specifically, in an OA apparatus such as a copier, a facsimile apparatus, a printer apparatus, a scanner apparatus, or a multifunction apparatus in which those functions are integrated into one unit. The present invention relates to a device operating device that can be operated without using vision in various operations, a program for executing functions of the device operating device, a computer-readable recording medium storing the program, and an image forming apparatus.
[0002]
[Prior art]
In recent years, office automation equipment such as copiers, facsimile machines, printers, scanners, and multi-function peripherals (hereinafter referred to as MFP (Multi-Function Peripherals)) that integrate these functions are handled by introducing digital technology. Because the functions have become so diverse that it is no longer possible to operate with only the numeric keypad or function keys, a touch panel is provided on the display, characters indicating the operating procedure, pictures indicating the input location, etc. are displayed on the screen. A touch panel input method for performing input by touching an input area set in advance corresponding to is widely spread.
[0003]
[Problems to be solved by the invention]
From the viewpoint of promoting the social advancement of persons with physical disabilities, it is desirable to create an office environment in which persons with disabilities can work in the same way as healthy persons. In particular, in the United States, Article 508 of the Rehabilitation Act was established in 2001. Effective on June 21, the US federal government is obligated to purchase equipment that can be used by people with disabilities in IT equipment procurement, and the state government, related facilities, and private companies are also following up.
[0004]
In the MFP as described above, the operation tends to become more complicated, and it is difficult to use it correctly unless instructions are displayed on a display panel, a touch panel or the like. This tendency results in a decisive barrier, especially for those with visual impairments.
[0005]
For example, when a visually handicapped person operates an MFP, if it is an operation using hard keys, a key button on the liquid crystal touch panel that cannot be perceived by tactile sensation can be used to some extent by remembering the key arrangement. It is absolutely impossible to operate. In this operation, the operation procedure and the position of the touch area must be memorized in advance, but if the operation procedure or the position of the touch area changes, the operation procedure or the touch area position that the user has learned is invalid. Become.
This makes the MFP virtually unusable for those who are visually impaired.
[0006]
In order to solve this problem, an audio guide is provided by the voice synthesizer when it is determined that there is a visual impairment by inserting an ID card or earphone in which information on the visually impaired is recorded into the MFP as described above. There is a device that performs Braille guide using a Braille output device.
[0007]
For example, when it is applied to an automatic transaction device such as an automatic teller machine or an automatic teller machine used in a financial institution, an ID card or an earphone that records information on a visually handicapped person is inserted into the automatic transaction device. When it is determined that there is a visual impairment, it is equipped with a key with braille, a braille output device that outputs arbitrary braille, and a function to output voice guidance from the handset, and operation guidance is performed by voice guidance from the handset, By selecting the transaction type and inputting the amount with a key with Braille, and outputting the input result by a Braille device, even a visually handicapped person can perform the transaction by his own operation.
[0008]
However, the number of users with disabilities who need these measures is very small compared to the number of users with sighted people. Is too expensive.
In addition, if there are automatic transaction apparatuses that are not equipped with devices such as a Braille output device and a handset, and automatic transaction devices that have different positions and usages of the Braille output device and the handset, the user will be confused.
[0009]
Further, in the “composite device” disclosed in Japanese Patent Laid-Open No. 10-117263, when the operator switches to the voice guide mode in an off-hook state, the voice guide may be switched from the speaker to the handset, and the voice guide may leak to the outside. Thus, it becomes possible to use the composite device without causing trouble to people around the composite device.
[0010]
Such a setting method based on voice guidance is easy to understand when the operator is first, but has a disadvantage that the operation time cannot be shortened even after the user has mastered the device. In addition, operations other than the prepared operation procedure cannot be freely performed, and access to functions equivalent to those of sighted persons is not assumed. This is the basis for visually impaired people entering the society.
[0011]
Further, in the “image forming system” disclosed in Japanese Patent Laid-Open No. 2000-322178, when a visually handicapped person having an ID card comes in the vicinity of an image forming instruction device that outputs an image forming command to the image forming device, the ID card When an ID code is read and a personal information file is referred to and it is determined that a person with the ID code has a visual impairment, operation guidance in the image formation instruction apparatus is guided by voice using a voice synthesizer. Thus, even a visually handicapped person can easily perform various operations.
[0012]
In such an image forming system, people with visual impairments are registered in the personal information file. However, it is difficult for people with low vision or presbyopia, such as those who are difficult to see the display contents on the operation panel, or those who are short or wheelchairs. A person who looks up at the operation panel like a person who rides and cannot see the display content is excluded, and it is still difficult to use.
[0013]
The present invention has been made in consideration of the above-described circumstances, and implements a device operating device capable of performing various operations easily and reliably without using vision, and functions of the device operating device. An object of the present invention is to provide a computer-readable recording medium and an image forming apparatus on which the program is recorded.
It is another object of the present invention to provide an apparatus that can reduce the manufacturing cost of the apparatus and that can easily assist a person who can operate the apparatus.
[0014]
[Means for Solving the Problems]
In order to solve the above problem, the device operating device according to claim 1 of the present invention classifies all operation items that can be operated by the device according to function, and further subdivides the classification into final setting items. In a device operating device that operates the device by selecting or setting a value of a hierarchical menu setting item having a multi-level tree structure, A headset, a headset installation unit for suspending the headset connected to the device, a determination unit for determining whether or not the headset is installed in the headset installation unit, and a visually impaired person A mode switching unit for switching between a non-visual mode used by the user and a visual mode used by a sighted person, For each common term included in the operation item on the hierarchical menu, all the paths from the top layer to the bottom layer including the setting item or hierarchical name including the term are extracted, and each newly constructed tree structure An operation support information file registered in advance by associating a node with audio information explaining the hierarchical name or setting item name of the node, and When the determination unit determines that the headset is not installed in the headset installation unit by the mode switching unit, and the operation of the device is shifted to a non-visual mode, A speech recognition unit for recognizing speech of a desired operation item of the visually impaired person, and an operation item corresponding to a speech recognition result recognized by the speech recognition unit is searched from the operation support information file, and By outputting the audio information for the operation item of the search result by voice, the visually impaired person is notified, the desired operation item is selected by the visually impaired person, and the selected operation item is selected as the corresponding operation item of the hierarchical menu. And a help operation unit.
[0018]
Also ,Book Claims of the invention 2 Claims 1 In the apparatus operating device described in the item 1, the determination unit determines whether or not the headset is installed by moving the headset installation unit based on the mass of the headset.
Further, the claims of the present invention 3 Claim 1 Or 2 In the device operating device described in item 1, when the operation of the device is in the non-visual mode, visual operation is not permitted.
Further, the claims of the present invention 4 Claims 1, 2 or 3 The device operating device according to claim 1, further comprising a mode switching key capable of switching between the non-visual mode and the visual mode at any time even when the headset is being used.
Further, the claims of the present invention 5 The program comprises a computer. 4 The program for functioning as each part of the apparatus operating device as described in any one of these.
Further, the claims of the present invention 6 The recording medium of claim 5 The computer-readable recording medium which recorded the program as described in 1 above.
Further, the claims of the present invention 7 The image forming apparatus according to claim 1. 4 The apparatus operating device described in any one of the above is characterized in that image formation is performed after setting items for functions of the image forming apparatus are selected and confirmed.
[0019]
With the above configuration, the manufacturing cost of the device can be reduced, and various operations can be easily and reliably performed without using vision.
In addition, it is possible to provide a device that can be easily operated by a person who can operate the person who cannot perform the operation.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
In the description of the present embodiment, the device operation device according to the present invention will be described as being incorporated into an MFP (Digital Multifunction Peripherals) (hereinafter referred to as the present device). However, the installation target of the device operating device of the present invention is not limited to the MFP, and a similar effect can be expected by incorporating any device having diversified functions.
[0021]
A. MFP incorporating the device operating device of the present invention
FIG. 1 is a block diagram of an MFP incorporating an apparatus operating device according to the present invention.
1, an MFP 100 includes a host connection device 101 for connecting a host and the MFP, a FAX line connection device 102 for connecting to a FAX line, a network connection device 103 for connecting to a network such as a LAN, and the MFP. An operation unit 104 for performing various setting operations, a scanner device 105 that scans an image from a document table and creates image data, a printer engine 106 including an image forming device and a paper supply / discharge device, control of the entire MFP, printer language processing , Facsimile control, scanner control, user recognition processing, sound output control, image data editing processing, control for MFP operations, etc., facsimile received image data, communication record and image data used for each function are saved Storage device 108, sound and sound according to the operation result of the user Speaker for outputting voice, a sound output apparatus 109 of the headphone or the like, consisting of the voice input device 110. Inputs audio operation instruction by the user emitted.
[0022]
In such an MFP 100, when the printer function is used, print data transmitted from the host via the host connection device 101 or print data transmitted from a computer connected via the network connection device 103 is received. The data is temporarily stored in the reception buffer of the control controller 107, developed into image data by the control controller 107 in accordance with printer language processing, sent to the printer engine 106, and printing is executed.
When using the copy function of the MFP 100, printing is executed by sending image data read from the scanner device 105 to the printer engine 106 in accordance with an instruction from the controller 107.
[0023]
If the user is a person who can handle this MFP 100 freely, a copy instruction is given according to the picture information displayed on the operation panel of the operation unit 104, and the image data read by the scanner device 105 is temporarily stored in the storage device. 108, the image data is edited by the control controller 107, the image data is edited, and a plurality of images are overlapped. Then, the image data is printed by the printer engine 106, or connected to the FAX line. 102 can be FAX-transmitted.
If the user is a person who cannot handle the MFP 100 freely, the sound output device displays the operation status in response to an operation instruction given by an input device such as a numeric keypad or the voice input device 110 that can be identified by the user. The sound output to 109 is judged, the operation status is confirmed, and the copy function is operated.
[0024]
Further, image data read by the scanner device 105 of the MFP 100, facsimile image data obtained via the FAX line connection device 102, and image data obtained via the network connection device 103 are converted into the host connection device 101 and the network connection device 103. It can also be used to reprocess the image data sent to the computer via the computer and received by the computer application.
Further, the MFP 100 can obtain image data from an image data server connected to the Internet via the network connection device 103 and print it by the printer engine 106.
[0025]
B. Operation unit and operation panel
FIG. 2 is a front view of the operation unit 104 in the MFP 100 in which the device operation apparatus of the present invention is mounted.
In the center of the operation unit 104, a liquid crystal display unit (LCD) 201 and a touch panel are provided. The liquid crystal display unit 201 displays setting items for functions (FAX, copy, printer, etc.) incorporated in the MFP 100, and the user sets a value in the displayed setting item to thereby display the setting items of the MFP 100. Perform the operation. The function is switched by a function key 206 provided on the left side of the liquid crystal display unit 201.
On the right side of the liquid crystal display unit 201, various keys such as various numeric keys 202, a clear / stop key 203, a start key 204, and a setting confirmation key 205 that are commonly used for each function are arranged.
[0026]
C. Overview of MFP operations
Users of the MFP 100 are classified into the following two types.
(1) Person who cannot operate this MFP freely
Those who are classified into this are screens and touch panels 201 of the liquid crystal display unit 201 of the operation unit 104 such as visually impaired persons, physically handicapped persons, low vision persons, and presbyopic persons (hereinafter referred to as visually impaired persons). A person who cannot see and operate.
Or even if it is a healthy person, the person who selected the operation by voice or sound is also included in this classification.
When a person belonging to this category operates the MFP 100, guidance for operation is performed by voice and sound. This processing mode of the MFP is referred to as “non-visual mode”.
[0027]
(2) Person who can operate this MFP freely
The person classified into this is a person who is not included in the above (1), and is generally a sighted normal person.
When a person belonging to this category operates the MFP 100, the operation is performed by the liquid crystal display unit 201 and the touch panel 201 of the operation unit 104. This processing mode of the MFP is referred to as a “visual mode”.
[0028]
Hereinafter, an outline of the operation of MFP 100 will be described using “operation for copying a document”. Regardless of the above-mentioned user, the user of MFP 100 can use any power source or numeric keypad of MFP. It is assumed that you know whether or not you have a basic function and operation method of the MFP.
[0029]
First, when the MFP 100 is turned on, it is set to operate in the “visual mode”. Therefore, a healthy user gives various instructions as it is through the screen display of the operation unit 104 and the touch panel. The MFP 100 can be operated.
[0030]
On the other hand, a person who wants to operate the MFP in the “non-visual mode” presses the mode switching key, connects a headset including a headphone and a microphone to the MFP 100, and changes from the “visual mode” to the “non-visual mode”. By switching to, the voice notification “entered non-visual mode” is given.
This mode switching key can be switched by continuously pressing a specific key, for example, the “* key” of the numeric keypad for a predetermined time (3 seconds) or more.
Once the “non-visual mode” is set, the instruction from the touch panel is locked, but can be switched at any time by issuing a “mode” switching instruction.
In addition, the connection between the headset and the MFP 100 may be performed using wireless communication or Bluetooth.
[0031]
Next, by depressing the function key 206 “Copy” (see FIG. 2) of the operation unit 104 of the MFP 100, the user is notified by voice that “copy operation is now possible” from the headphones.
Also, in this MFP 100, a single menu having a multi-level hierarchy is provided in which all operation items that can be operated in advance are classified by function for each function such as FAX and copy, and the classification is further subdivided. Create.
For example, the “Copy” function is classified into a hierarchy as shown in FIG. 3 and FIG. 4 (this function classification only shows a part).
[0032]
The user uses a direction movement key installed in the operation unit 104 or a key that can be substituted for this direction movement key (hereinafter, collectively referred to as a direction movement key) to trace a desired operation item in the hierarchy menu. After the desired setting is completed by repeating the operation of selecting and setting the value, the copy of the document is created by pressing the start key 204.
[0033]
As the user operates the hierarchical menu, the explanation of the selected function and the contents of the setting items set so far are fed back to the user by voice, sound or BGM (Back Ground Music). The MFP 100 can be operated without viewing the screen of the liquid crystal display unit 201 of the operation unit 104 or inputting from the touch panel 201.
Here, voice, sound and BGM are set using attributes (sound pitch, sound speed, voice type (male, female, mechanical voice, etc.), sound quality, tone, tone, sound field feeling, etc.) The output is changed according to the type and setting status of the item and the depth and location of the hierarchy.
[0034]
Further, the selection process by the direction movement key in the “non-visual mode” is displayed on the display screen of the liquid crystal display unit 201 of the operation unit 104 in a substantially correlated manner.
Here, when the headphone is turned on and the mode is shifted to the “visual mode” and an input instruction is given from the touch panel 201 of the operation unit 104, the operation result is fed back to the headphone and the mode is switched to the “non-visual mode” again. Since it is possible, it becomes easy for a healthy person to assist visually impaired people.
[0035]
D. Details of operations on this MFP
(1) Structure of hierarchical menu
It is difficult to remember if all the operation functions are made into multiple menus as they are. Therefore, all the operation items that can be operated are classified by function for each function such as FAX function, copy function, printer function, scanner function, and NetFile function handled by the MFP. The classification is further subdivided to create a single menu having a multi-level tree structure and stored in the menu database (DB).
[0036]
For example, if there are more detailed setting items such as “scaling” and “double-sided”, “scaling” and “double-sided” are set as one layer name, and “same size” is further added under those layers. ”,“ Enlarge ”,“ reduced ”, and the like are arranged, and the hierarchical names are subdivided until final setting items such as“ 115% ”and“ 93% ”are arranged.
Hereinafter, the hierarchy name and the setting item are collectively referred to as an operation item.
[0037]
The types of setting items on this hierarchical menu are as follows.
・ Selectable items that can be selected only once in the same hierarchy
For example, “115%” or “122%” under the “enlargement” level
・ Toggle item that can toggle function ON / OFF
For example, “Whether to sort” under the “Finisher” level
・ Numeric items that can be set
For example, the number of copies
・ Operation items that start some operation
For example, creating a destination list registered with the FAX function
[0038]
For example, in the case of the copy function, first, as shown in FIG. 3 and FIG. 4, the hierarchy name of the first hierarchy is set to (number of copies, manuscript, paper, scaling, duplex / aggregation / division, finisher, etc.) Subdivided into the third level.
Also, as shown in FIGS. 3 and 4, numbers correspond to the operation items (hierarchy names and setting items) of each layer, and by specifying these numbers, it is possible to directly perform the operation without tracing each layer. An operation item can be selected. Hereinafter, this number is referred to as a “function number”.
[0039]
(2) Hierarchy menu operation
Next, a method for operating the hierarchical menu as described above will be described.
In order to move and select the operation items arranged on the above-described hierarchical menu, a direction movement key or a key that can be substituted for it is used. In the following description, the numeric keypad 202 is used instead. FIG. 5 shows functions assigned to each key.
[0040]
That is, moving up and down the hierarchy is performed by pressing a key arranged in the vertical direction in the numeric keypad, and adjacent items in the same hierarchy are carried out using keys arranged in the horizontal direction in the numeric keypad.
In FIG. 5, the hierarchy is moved up by pressing the “move up (“ 2 ”)” key, and the hierarchy is moved down by pressing the “move down (“ 8 ”)” key.
Further, the same level is traced to the left by pressing the “move left” (“4”) key, and the same level is traced to the right by pressing the “move right” (“6”) key.
Note that the direction movement key described above may be assigned in reverse.
[0041]
When the user instructs to move to the lower layer of the hierarchical menu, if there are a plurality of lower layer operation items, the user moves to the next operation item by one of the predetermined methods listed below.
[0042]
• Move to the first item below.
If none of them is a default value, for example, it is used when selecting the type of document.
• Move to the item already selected below.
For example, if there is an item that has already been set in the setting items in the lower layer, it moves to the item that has already been set in order to continue the setting after that item or to correct the content of that item.
-Move to the item with the default value in the lower layer.
For example, in the case of density setting, there is a selection item of “light, normal, dark”, but in general, “normal” is set as the default value, so that it moves to the setting item of the default value “normal”. .
[0043]
Further, when the user moves the same hierarchy by operating the horizontal movement key, when the user reaches the end of the hierarchy, the user moves to the opposite end of the hierarchy and performs a cyclic movement. By always moving in a certain direction, it is possible to find an item that the user wants to select. However, there is a high possibility that the number of items in this hierarchy cannot be grasped and the user is lost on the menu.
Alternatively, this cyclic movement may not be performed and no further movement may be performed when the end is reached. In this case, if the desired item is overrun, the move key must be pressed in the opposite direction, but it is possible to grasp how many items are in that hierarchy.
[0044]
A method of using the direction movement key will be described with reference to an example in which the operation is performed so that the copy result is dark because the image of the document is thin (see FIG. 6).
When the operation is started, the current hierarchy name “number of copies” is notified to the user by voice, and the rightward movement key is used to advance to the hierarchy name “original”. At this time, it is informed by voice that the user is at the position of “original”.
Next, in order to select “density” which is a subcategory of this “document”, the hierarchical menu is moved down to the next lower level by using the downward movement key. At this time, it is informed by voice that the user is in the “type” position.
[0045]
Further, the right movement key is pressed to advance to the hierarchical name “density” which is one of the subcategories of “original”. At this time, it is informed by voice that the user is in the “density” position.
Similarly, in order to select “deep” which is a subcategory of this “density”, the hierarchical menu is moved down to the next lower level by using the downward movement key. At this time, it is informed by voice that it is in the “automatic” position.
Further, the right movement key is pressed in order to advance to the hierarchical name “dark” which is one of the subclasses of “density”. At this time, since it is informed by voice that it is in the “dark” position, the enter key is pressed.
As a result, it is possible to achieve the setting “copy the original darkly” requested by the user.
[0046]
Furthermore, in FIG. 5, when the user does not know which level is in the middle of setting, or when he / she wants to select again from the beginning, for example, the “return to top (“ 1 ”)” key is pressed. Thus, it is possible to return from any hierarchy to the top hierarchy (in the example of FIG. 3, “the number of copies” of the first hierarchy).
In this case, setting items that have been selected or set so far can be stored in MFP 100 and moved to items that have been selected and set up to that point by using the move key or the confirmation key.
[0047]
In addition, in FIG. 5, by pressing a “direct jump (“ 0 ”)” key, the user can directly move to a hierarchy or item that the user wants to set.
For example, by pressing a “direct jump” key and inputting a function number from the numeric keypad, it is possible to move to an operation item corresponding to the function number.
For example, when the function number “435” is numerically input using the numeric keypad after the “direct jump” key is pressed, the MFP 100 selects items in the order of “magnification” → “reduction” → “71%”. (See FIGS. 3 and 4).
This function number may not be a setting item at the lowest layer but may be a hierarchy name of an intermediate hierarchy. In this case, since the user moves to a predetermined item in the hierarchy, the user selects an operation item from the item.
[0048]
Further, instead of inputting the function number of the item to be directly set by the “direct jump” key as described above, the user presses the “voice recognition (“ 3 ”)” key, and the user selects “function number”, “ It is also possible to say “hierarchy name” or “setting item name”, recognize it, and move directly to the recognized item.
[0049]
Here, when the user does not remember the “function number”, “hierarchy name”, or “setting item name”, the word representing the function is uttered, and the operation item including the word obtained as a result of the voice recognition is displayed on the hierarchy menu. If a search is performed from the inside and the result is one operation item, the operation item is directly moved to. When there are a plurality of search results, the search results are read out and the user selects them.
For example, if there is “Double-sided / Aggregate / Split” → “Bookbinding” → “One-sided weekly magazine” on the hierarchical menu, if the user utters “Weekly magazine”, it is related to the position of the currently operating hierarchical menu. Without going directly to that weekly magazine.
[0050]
On the other hand, when the user utters and the result of voice recognition is none of the above, the operation support information file is searched using the recognition result as a keyword, and the voice information corresponding to the search result is notified as guidance. Selects a desired setting item according to the guidance.
[0051]
This operation support information is configured as follows.
An operation item including the same keyword on the hierarchical menu is searched, all paths from the uppermost layer to the lowermost layer including the found operation item are taken out, and this is reassembled into a tree structure to form one group.
The tree structure constituting the completed group forms a part of a hierarchical menu including keywords.
[0052]
For example, referring to FIG. 4, functions including the keyword “double-sided” include “original”, “copy”, “aggregate”, “divided”, and “bookbinding” under the “double-sided / aggregated / divided” hierarchy. ”And a setting item including the keyword“ double-sided ”exists in the lower hierarchy. In such a case, the keyword “both sides” has a tree structure as shown in FIG.
A function including the keyword “copy” is below the “scaling” and “double-sided / aggregating / dividing” hierarchies, but is not a setting item at the lowest level. In this case, up to the lowest layer below is taken in as a tree structure, and the tree structure as shown in FIG. 8 is provided.
[0053]
The operation support information configured as described above creates the above-described tree structure information and voice information for each node (hierarchy name or setting item name) of the tree structure using one of the following formats, and an operation support information file: Register with.
・ Recorded voice data
It informs the user by playing the audio data.
・ Data compressed by CELP (code excited linear prediction)
The user is notified by decompressing and reproducing the compressed data.
・ Text data
The user is notified by reading out the text data using speech synthesis.
[0054]
When a user operates this apparatus using this operation support information, the user is made to select a desired operation item by the following procedure.
-Search the operation support information file by a keyword uttered by the user and recognized by voice, and extract a group corresponding to the keyword.
First, read out all audio information corresponding to the extracted group. This is for checking what operation items are in the group.
Read one by one from the second time, and instruct the user to confirm the candidate or go to the next candidate.
-The item determined by the user is determined as the selected operation item.
[0055]
For example, in the case of the operation support information in FIG. 7, when the keyword “both sides” is uttered, it is notified by voice that the following setting items are present.
“Double-sided, consolidated, and split settings”
“1. Set both sides of the manuscript”
“2. Copy on both sides”
“3. Consolidation is set for every 4 sheets on both sides”
“4. Aggregation is set every 8 sheets on both sides”
“5. Splitting is set for both left and right pages”
・ ・ ・ ・ ・ ・
[0056]
In the above, operation support information was selected while reading one by one, but each candidate was given a number as in the above example, and after reading all the numbers, enter the number by entering the numeric keypad or speaking the number. Also good.
[0057]
When the above-described operation support information file has a tree structure for each keyword and is read out, the items on the path of the tree structure are read out in order from the upper hierarchy as one sentence.
Instead of following the path of the tree structure when reading aloud, each group may be developed as one sentence and stored as voice information, and may be read as it is when reading.
[0058]
Also, instead of providing a special operation support information file, search for the operation item name in the menu database or the keyword assigned to the operation item, and find the operation item name found and the upper or lower layer containing the operation item. An item on the route may be extracted, and the operation item name on the extracted route may be read out.
[0059]
Returning to FIG. By pressing the “partial clear (“ 7 ”)” key, it is possible to return the value of the setting item whose default value has been changed to the default value among the setting items below the current hierarchy on the hierarchy menu.
When returning to the setting value, you may inquire for each setting item and return only the items that can be returned to the default value, or ask whether you want to return all the setting items whose default value has been changed. Also good.
For example, if you are currently in the “Finisher” level of the hierarchy menu, if you change the default value in the “sort / stack”, “staple”, and “punch” setting items below that level, change that setting value. Clear and return to default value.
[0060]
Further, in FIG. 5, by depressing a “current position description (“ 9 ”)” key, the operation item on the hierarchical menu is fed back by voice. Examples of the feedback information include an operation item name, a function number, and a set value (number of copies).
For example, if the “current position” key is pressed after the paper is set and fixed on the A4 side, “the paper is set on the A4 side” is fed back by voice.
[0061]
Further, by depressing a “setting confirmation” key 205 (see FIG. 2), the user can collectively confirm the setting items changed from the default values, and can also change the setting values.
[0062]
Further, in FIG. 5, when the hierarchical menu is operated with the above-described keys and setting of a desired setting item (selection or numerical input) is completed, “confirmation (“ # ”)” is performed to confirm the setting item. Press the key. As a result, the MFP 100 displays the contents of the determined setting items on the screen of the liquid crystal display unit 201 of the operation unit 104.
[0063]
In FIG. 5, when the “mode change (“ * ”)” key is pressed, the current mode is changed.
If the current mode is “non-visual mode”, pressing the “mode switching (“ * ”)” key switches to “visual mode”. However, if the headset remains connected, audio, sound, etc. are output to the headset's headphones.
Therefore, in order to return to the “visual mode” completely, the headset also needs to be disconnected.
If the current mode is “visual mode”, the mode can be switched to “non-visual mode” by pressing the “mode switching (“ * ”)” key and connecting the above-described headset to the MFP 100.
[0064]
In addition, by enabling the MFP 100 to recognize a non-contact type IC card and storing the contents set by the above operation with an operation name attached to the IC card, it is possible to operate from the IC card from the next time. By specifying the name and reading the operation content, it is possible to configure so that the same operation does not have to be set again.
[0065]
(3) Operation result operation sound, sound icon and feedback by BGM
As described above, when moving / selecting operation items arranged on the hierarchical menu using the numeric keypad, confirmation of the result of the operation as shown below is performed using operation sounds, sound icons, and BGM (background music). Notify by outputting to the user's headphones.
Here, the sound icon is static information indicating the nature of the object, and is a sound (short single sound) that does not move with different timbres for the purpose of discriminating the type of operation item.
[0066]
The operation sound uses different sounds (chords and very short melody) of the same timbre to give an image of the operation result and to associate the operation with the meaning of the sound in music theory. For example, ascending and descending sounds on the hierarchies, respectively, rising and falling sound forms, 4th degree dissonance sound that makes you feel anxious when it comes to a dead end (failure of movement), and progress that makes you feel stable when selecting and confirming. Use dominant motion.
[0067]
For example, an operation sound for notifying the up and down movement of the hierarchy is prepared, and when the user moves up and down the hierarchy, the operation sound is output to notify that the hierarchy has been moved up and down. This operation sound is different from the above sound icon.
At this time, it is determined whether or not there is an operation item in the lower layer of the movement destination, and the operation sound is output differently depending on the presence or absence of the operation item in the lower layer and notified to the user.
[0068]
If the same level of cyclic movement is not set, when the movement is instructed in the horizontal direction, if there are no more items to move, the user is informed with a “dead end” operation sound.
In addition, when a key such as moving to the top, moving in the horizontal direction, directly moving, or voice recognition is pressed, an operation sound for confirming the selection operation is output so that the user can be surely accepted the operation. I can recognize that.
[0069]
In addition, if the BGM has a slow tone in the upper layer and the BGM has a faster tone as it goes to the lower layer, it is possible to recognize which side of the layer is currently located.
Also, in the case of numerical input such as inputting the number of copies, it can be confirmed that the numerical input is performed by playing BGM until the numerical input is completed.
[0070]
In addition, the user is notified by voice of the operating state of the MFP, for example, “out of paper”, “toner shortage”, “paper jam”, “front cover is open”, and the like. If such a condition cannot be resolved immediately, the condition cannot be grasped if the sound is interrupted.
For example, when the user is away from the apparatus during the copying operation and “paper jam occurs”, the user does not know from this sound alone.
However, if BGM is played after the voice, the user can recognize even in such a state.
[0071]
Also, when the user designates 40 copies, the BGM is run so that the user can recognize how much the copy has been completed. You can know if copying has progressed.
[0072]
In this way, by notifying the user with an operation sound, BGM, or sound icon that indicates the operation image, it is possible to recognize the current state of the user and whether the user has entered it correctly or the operation. Anxiety about whether or not can be completed.
[0073]
(4) Voice feedback of operation results
As described above, when moving / selecting the operation items arranged on the hierarchical menu using the numeric keypad, the user can confirm whether the input and the operation result have been made reliably by outputting from the headphones also by voice. Can be recognized and anxiety can be reduced.
The voice at this time was placed by the user by changing the voice attributes (pitch, voice type, tone quality, tone, tone, speaking speed, sound field feeling) according to the situation. The situation can be grasped intuitively.
[0074]
Further, when a hierarchical menu is reached by a moving operation and an operation item is reached, voice feedback of a predetermined content is performed on the operation item.
For example, when moving to an operation item such as “enlarge” or “114%”, the item name is read out. Thereby, it can be recognized to which operation item the user has moved.
[0075]
When this item name is read out, an already selected item is read out in a higher voice than an item that has not yet been selected. This gives the user a high-activity image and can inform the user that the item is currently selected more quickly and intuitively than with the language expression.
[0076]
For example, in the hierarchical menu of FIG. 3 and FIG. 4, “same size” and “enlarge” in the lower level of “magnification” are to be selected, but which one is currently selected If the user can know whether this is done, it is possible to eliminate the waste of selecting “same size” even though “same size” is selected.
In addition, in an item for switching ON / OFF by selection such as “mixed document size” or the like, it is not known whether it is currently selected (that is, whether it is ON) or not selected (that is, whether it is OFF). I cannot set it to my desired state.
[0077]
Also, when reading the item name, the invalid item is read out with a lower voice than the valid item. This gives the user a low-activity image and can quickly and intuitively tell the user that the item is currently invalid, rather than using linguistic expressions.
Selecting an item may affect other items, resulting in a situation where other items cannot be selected.
[0078]
For example, when “automatic”, “tray 1”, or the like is selected for “paper”, it is not effective to select “paper type of manual paper”. If you can select and move to an item that you can't select, you can confuse the user with a misunderstanding that you can select it.
However, if the item is temporarily deleted from the hierarchical menu or is not moved, the hierarchical menu structure will change dynamically. For users who know the menu structure only auditorily It is difficult to understand why the menu structure has changed, and it causes confusion.
[0079]
When reading out as described above, the upper and lower pitches of the voice need to have a change amount that can be distinguished from the normal pitch, and it is necessary to keep the utterance content in a clearly audible range.
For example, in consideration of a sound range in normal natural speech and a sound range that can ensure clarity by speech synthesis, a change amount of about a half octave is given.
[0080]
In addition, when reading the item name, a reverberation sound is added to the item name having the item of the lower hierarchy and output. This gives the user a lingering image, and can tell more quickly and intuitively that there is an item group that is further sub-classified after that item, rather than by language expression.
For example, there are items on the hierarchical menu that do not have a lower hierarchy such as “same size”, and items that have a sub-category item in a lower hierarchy such as “enlarge”.
Being able to understand this difference without actually moving to a lower hierarchy makes it easier to move the hierarchy menu.
[0081]
In addition, when the “partial clear (“ 7 ”)” key is pressed, it is spoken by voice whether the setting change made to the setting item in the lower layer from the current position on the hierarchical menu may be read out and returned to the default value. Inquire.
At this time, instead of inquiring whether or not to return the set value to the default value one by one, it is inquired whether the setting items in all lower layers can be returned to the default value. You may do it.
[0082]
By pressing the “Description of current position (“ 9 ”)” key to determine which operation item is currently on the hierarchical menu, information such as the operation item name and setting value for the operation item is provided with a predetermined content. Feedback with voice.
For example, after the setting of “paper” is set to “A4 landscape” and the “description of current position” key is pressed, voice feedback “paper is set to A4 landscape” is performed.
[0083]
When the selection of the setting item is completed and a confirmation operation is performed with the “confirmation (“ # ”)” key, the confirmed content is fed back with sound of a predetermined content.
For example, when “200%” is selected / confirmed in “enlargement”, voice feedback is made such that “magnification has been set to 200% enlargement”.
[0084]
When the setting confirmation key is pressed, the setting items that are below the operation items (including the intermediate layer) on the current hierarchy menu and whose settings have been changed from the default values are collectively read out by voice.
At this time, instead of using voice feedback for all the setting items at once, feedback is made for each setting item, and if there is an error in that setting item, the next changed setting is made by operating the movement keys. It is also possible to move to the item and finally press the enter key to make the final decision.
Also, if you want to know all the setting items that have been changed from the default values so far, use the “Return to top (“ 1 ”)” key to move to the top of the hierarchical menu and then press this setting confirmation key Can be confirmed.
[0085]
When the setting of each setting item is completed and the start key is pressed, when the operation of the MFP is completed, voice feedback of a predetermined content for notifying the success or failure is performed.
For example, a result such as “Copying has been completed at a magnification of 122%” or “Copying has failed due to a paper jam” is fed back by voice.
[0086]
Also, during operation or during operation, the voice of the speaker, which is different from the voice during the operation of the operation item described above, continues after the operation sound, for example, “paper is exhausted”, “ Notify the user that "Remaining low", "Paper jam has occurred", "Front cover is open", etc.
[0087]
Providing a knob that changes the sound attributes (pitch, voice type, tone quality, tone, tone color, speech speed, sound field feeling) by the above-mentioned voice feedback, so that the user can easily hear the voice output. Can do.
Further, when the next key operation is performed without waiting for the end of the voice feedback, the user can be frustrated by automatically increasing the speaking speed.
[0088]
By performing such a hierarchical menu structure and sound icons, operation sounds, BGM, and voice guidance, the user can get used to the operation with about 5 minutes of practice when handling the MFP for the first time.
[0089]
The operation result by the user described above is displayed almost synchronously with the screen of the liquid crystal display unit 201 of the MFP 100, and a person who can operate the MFP shifts to the visual mode based on the display and inputs from the touch panel 201. , It is possible to assist a person who cannot operate (for example, a visually impaired person, a weakly-sighted person, a presbyopic person, etc.)
In addition, people with low vision or presbyopic eyes can see the screen and operate in visual mode.
For example, when a user loses his or her operation on the way, when a healthy person once shifts to the visual mode and selects a setting item on the touch panel, the setting is output to the headphones with a sound icon, operation sound and voice. A user such as a visually impaired person can know what is set, and can return to the non-visual mode to continue the operation.
[0090]
(5) Example of operation
Next, an example of operation until the operation is completed after setting “number of copies to 2” using the above-described MFP will be described.
First, when the hierarchical menu is operated to move to “number of copies”, the setting item name “number of copies” is read out from the headphones, followed by a sound icon “(pylon) copies”. At this time, since there is no layer below “number of copies”, no reverberation sound is added.
Here, when the user presses the “confirm (“ # ”)” key, an audio guidance for the number of copies is made following the sound icon “Please input the number of copies with the (Kalan) numeric keypad and press the confirm key” from the headphones. The BGM flows during the numerical input of the number of copies.
Here, the sound of “(pylon)” and “(caran)” indicates an operation sound.
[0091]
When the user presses the “Move Up (“ 2 ”)” key using the numeric keypad, the number input from the headphones is read, and when the user presses the “Confirm (“ # ”)” key, the “( “Charan” has been set to 2 copies.
After this, when the user presses the start key, BGM is played while copying from the headphones, and when the user finishes, there is a voice guidance saying that “two copies have been completed”.
[0092]
E. Functional configuration in MFP
FIG. 9 is a functional configuration diagram of an MFP in which the apparatus operating device of the present invention is incorporated.
The MFP shown in FIG. 9 includes a control unit 300 that performs execution control in response to an instruction from a user, and a support unit 400 that performs speech recognition and speech synthesis processing according to an instruction from the control unit 300.
[0093]
The control unit 300 includes a function control unit 310, an operation control unit 320, an operation input unit 330, a visual input unit 340, a visual display unit 350, a menu control unit 360, a menu database (DB) 365, a voice input unit 370, and a sound output. A unit 380, a help operation unit 385, an operation support information file 386, and a function execution unit 390.
[0094]
The function control unit 310 determines whether to use the visual mode or the non-visual mode, instructs the user to use which function such as a copy function or a facsimile function, and performs initial setting for the operation for the function. And do overall control.
In addition, the function control unit 310 is normally in the visual mode. When the user inserts a headset equipped with headphones and a microphone by pressing a mode switching key, the function control unit 310 controls the non-visual mode. Even if switching is specified, if the headset is not connected, the mode is not changed to the non-visual mode.
[0095]
The operation control unit 320 controls various input operations from the user and operations such as displaying the processing results.
The operation input unit 330 inputs numbers and the like from the numeric keypad.
In the non-visual mode, the operation input unit 330 inputs various operation instructions from a direction moving key for moving the hierarchical structure of the hierarchical menu, various keys for selecting / determining operation items, a numeric keypad, and the like.
In the case of the non-visual mode, the function number determined by the input numeric keypad type and the input numeric keypad type are delivered to the menu control unit 360.
[0096]
(1) Directional movement key
Configure the function number so that it increases by 10 times each time one level is lowered, and increases by 1 every time it moves to the right in the same level, and the current hierarchy menu position is calculated using the type of direction movement key. The operation number can be determined by calculating the function number of the next movement destination from the function number.
For example, when the right movement key is pressed from the initial state (current function number = 1), the function number = 1 + 1 = 2. Here, when the downward movement key is pressed, the function number = 2 × 10 + 1 = 21.
Next, when any one of the upward movement key, the downward movement key, and the rightward movement key is pressed, the function numbers are 2, 211, and 22, respectively.
[0097]
(2) Direct jump
Enter the “direct jump (“ 0 ”)” key to enable the numeric input, the user inputs the function number corresponding to the operation item from the numeric keypad, and finally presses the “confirm (#)” key. .
As a result, the numerical value input from the numeric keypad is set as the function number.
For example, to move to “text / photo”, when “2 → 1 → 2 → #” is pressed, function number = 212 is set.
[0098]
(3) For voice recognition
When the “voice recognition (“ 3 ”)” key is input, the operation input unit 330 activates the voice input unit 370, and the operation item name and word (a word representing a part of the operation item name) input by the user from the microphone. ), Voice recognition is performed from the keyword or function number, and the converted function number is returned.
[0099]
The visual input unit 340 uses a touch panel installed on the screen, and generally performs an input for selecting and setting various operation items displayed on the screen in the visual mode. It is also possible to temporarily switch from the non-visual mode to the visual mode and select an operation item.
The visual display unit 350 displays various functions on the screen in the visual mode. In the non-visual mode, the user selects from the hierarchical menu and displays the set items that have been confirmed.
[0100]
The menu control unit 360 manages the current position on the hierarchical menu using a menu database (DB) 365 that holds the hierarchical menu.
When the type of the input key passed from the operation input unit 330 is accompanied by movement on the menu, it is searched whether the input function number exists in the menu DB 365, and if it exists, the current function number is input. The function number is replaced, the position is moved to the position indicated by the operation item corresponding to the replaced function number, and the sound output of the processing result is performed.
When there is no movement, the function processing and sound output given to the key are performed.
[0101]
For example, when the upward movement key is pressed in the initial state (function number = 1), 1/10 = 0, so that it is not registered in the menu DB 365. In this case, no movement occurs.
Similarly, when the right movement key is pressed, 1 + 1 = 2, and since it is registered in the menu DB 365, it moves to the corresponding item (“document”).
Further, when the left movement key is pressed when the function number = 21, 21-1 = 20, and the corresponding function number is not registered in the menu DB 365, so no movement occurs.
Even when a function number is input by direct jump or voice recognition, if the function number does not exist in the menu DB 365 (for example, “2 → 1 → 6 → #”), no movement occurs.
[0102]
The voice input unit 370 is activated from the operation input unit 330 and inputs the operation item name, function number, word or keyword included in the operation item uttered by the user, and the voice input through the voice recognition unit 410 is input. The voice is recognized, converted into a function number according to the result, and returned to the operation input unit 330.
If a number is input as a result of speech recognition, the recognition result is set as a function number.
When the operation item name is input, the operation item name is converted into the function number by searching the correspondence table or the menu DB 365 stored in the memory in association with the operation item name and the function number.
For example, when “Mojishashin” is uttered, if it is recognized correctly, function number = 212 is set.
[0103]
When a word included in the operation item name is input, an operation item including the word is searched from the menu DB 365, and if the result is one operation item, the searched operation item is displayed. Convert to the corresponding function number. When there are a plurality of search results, the search results are read out and the user selects them.
For example, if there is “Double-sided / Aggregate / Split” → “Bookbinding” → “One-sided weekly magazine” on the hierarchical menu, when the user utters “Weekly magazine”, the function corresponding to the “One-sided weekly magazine” Convert to number.
[0104]
If it is determined that the keyword is not one of the above, the help operation unit 385 is used to select from the operation items related to the keyword, and the function number corresponding to the selected operation item is obtained.
The help operation unit 385 searches the operation support information file 386 with the keyword passed from the voice input unit 370 and extracts a group corresponding to the keyword. The voice information corresponding to the extracted group is read out for the first time in order to confirm what operation items are in the group. From the second time, it is read out one by one, and the user designates the candidate with the confirmation key or the next candidate key.
Or, after reading out all, the sequential number of the candidate to be selected may be input with a numeric keypad or voice.
Alternatively, the desired candidates may be selected by reading one by one from the beginning instead of reading all.
The function number corresponding to the selected item is converted.
[0105]
In the non-visual mode, the sound output unit 380 outputs operation sounds, voice guidance, BGM, and the like that indicate success or failure of various operations using the numeric keypad and function execution in response to instructions from the menu control unit 360 and the operation control unit 320. In addition, voice information related to the keyword uttered by the user in response to an instruction from the help operation unit 385 is output.
[0106]
Function execution unit 390 stores setting values for various setting items passed from operation control unit 320 in the memory of MFP 100 determined according to the function (for example, copy function, facsimile function, etc.). Further, a function is executed according to a setting value stored in the memory of MFP 100 in accordance with an instruction from operation control unit 320.
[0107]
The support unit 400 includes a speech recognition unit 410 and a speech synthesis unit 420.
The voice recognition unit 410 is called by the voice input unit 370, and the user uses a voice recognition dictionary (for example, a copy dictionary) for a function (copy function or facsimile function) currently operated by the user. The result of voice recognition of the uttered voice (operation item name, function number, operation command, etc. of the hierarchical menu) is returned to the voice input unit 370.
[0108]
The voice synthesizing unit 420 is called from the sound output unit 380 and changes the pitch according to an instruction from the menu control unit 360 using the voice synthesis dictionary when voice feedback is made to the user. Or the sound transformed by adding a reverberation sound is returned to the sound output unit 380.
[0109]
When the operation item instructed to the menu control unit 360 is invalid with respect to the voice sent from the voice synthesis unit 420, the pitch is changed to a voice with a lower pitch. If the setting item is selected, the sound is output with a higher pitch.
The upper and lower pitches require a change that is distinguishable from the normal pitch, and it is necessary to keep the utterance content in a clearly audible range, so clarity is ensured in the normal natural speech range and speech synthesis. Considering the possible range, give a change of about a half octave.
[0110]
The reverberation sound is added when there is an operation item below the operation item instructed by the menu control unit 360 for the sound from the speech synthesis unit 420.
For example, it can be realized by adding the audio to the input after passing through the delay device and the attenuator. In this case, the control parameters are the delay time of the delay unit and the attenuation amount of the attenuator, and are experimentally determined in consideration of improving the discrimination from normal speech and ensuring the clarity of the speech.
[0111]
Note that sound field control is performed by pitch control and reverberation sound addition among the sound attributes, but when reading operation items, the sound field feeling should be given by changing the speech speed, voice type, sound quality, etc. It may be.
Speaking speed control can be achieved by waveform expansion / contraction in recording synthesis, and phoneme duration length expansion / contraction in regular synthesis.
In addition, the voice type can be changed by changing the recording speaker in recording synthesis and by changing the segment waveform in rule synthesis.
The sound quality can be controlled by inserting a filter after the output of the speech synthesizer and controlling the cut-off frequency.
[0112]
Here, when voice feedback is performed by the sound output unit 380, the recorded voice may be used instead of voice synthesis from the text data by the voice synthesis unit 420.
[0113]
F. MFP operation
The operation of the MFP configured as described above will be described by taking the copy function operation as an example.
FIG. 10 is a flowchart showing the operation for the copy function operation.
First, when the mode switching key is pressed (YES in step S1), the current setting mode is switched (step S2), and the process returns to step S1 to process the next operation instruction.
When switching from the “visual mode” to the “non-visual mode”, it is necessary to press a mode switching key and connect a headset including headphones and a microphone to the MFP. Further, when the mode is switched to the “non-visual mode”, the input from the touch panel is locked, and the sound of “entering the non-visual mode” is notified from the headphones.
On the other hand, if the current setting mode is “non-visual mode”, the mode is switched to “visual mode”. If the headset remains connected, this “visual mode” is temporary, and the input operated from the touch panel is output to the headphones. Only when the headset is disconnected from the MFP, it switches to the “visual mode”.
[0114]
When the current mode is “visual mode” (YES in step S3), the MFP is operated by the visual input unit 340 using the touch panel 201 (step S4), and the process returns to step S1 to process the next operation instruction. .
[0115]
When the current mode is “non-visual mode”, the following processing is performed.
When a function key (FAX, copy, printer, scanner, net file, etc.) 206 is pressed (YES in step S5), a hierarchical menu corresponding to the pressed function is taken out from the menu DB 365 and initialized (step S6). ), The process returns to step S1 to process the next operation instruction.
Here, assuming that the function key “Copy” (see FIG. 2) is selected, the user is informed by voice that “copy operation is now possible” from the headphones.
[0116]
Next, when the direction movement key (up / down / left / right) is pressed (YES in step S7), the user moves up and down the hierarchy or moves to an adjacent item in the same hierarchy (step S8) to process the next operation instruction. Return to step S1.
The operation input unit 330 manages the location of the hierarchical menu by passing the input key type and the input function number to the menu control unit 360.
[0117]
Further, when moving on the hierarchical menu, the menu control unit 360 calls the sound output unit 380 to play a sound icon, an operation sound and BGM, and also outputs the operation item name as a sound so that the user can reliably operate the menu. Notify that
The voice at this time is a voice obtained by synthesizing the item name by the voice synthesizer 420. If the operation item is invalid, the pitch is lowered, and if the setting item is selected, the pitch is raised and read. . Furthermore, if the operation item has an operation item in the lower layer, a reverberation sound is added and read out.
[0118]
When the menu control unit 360 reaches the end of the hierarchical menu and cannot move any more, the menu control unit 360 calls the sound output unit 380 to output a “dead end” operation sound to notify the user. When a dead end is reached in the vertical direction on the hierarchical menu, a “dead end” operation sound is output to notify the user.
[0119]
Next, when the “return to top (“ 1 ”)” key is pressed (YES in step S9), the operation item in the first menu of the highest hierarchy is moved (step S10), and the next operation instruction is issued. Return to step S1 for processing.
The menu control unit 360 manages the current position so as to be the operation item in the first menu of the highest hierarchy. At the time of the movement, the menu control unit 360 calls the sound output unit 380, outputs a sound icon and the sound of the operation item name of the movement destination, and notifies the user that the operation has been performed correctly.
The voice at this time is synthesized by the voice synthesizer 420 by synthesizing the item name. If this operation item is invalid, the pitch is lowered, and if the setting item is selected, the pitch is raised and read. Furthermore, if the operation item has an operation item in the lower layer, a reverberation sound is added and read out.
[0120]
Next, when the “direct jump (“ 0 ”)” key is pressed (YES in step S11), the process moves to a setting item corresponding to the subsequently input function number (step S12), and the next operation instruction is processed. Therefore, the process returns to step S1.
The operation input unit 330 calls the sound output unit 380 and outputs an operation sound, a voice “Please input a function number”, and BGM. While this BGM is playing, the user inputs a number from the numeric keypad. The input number is read out via the sound output unit 380.
Finally, when the user presses the “determine (“ # ”)” key, the operation input unit 330 stops playing the BGM, and the menu control unit 360 operates the operation item of the operation sound and the function number just input. The name of is output by voice and the operation item is managed as the current position.
[0121]
When the item corresponding to the specified function number is a hierarchical name, the hierarchical name is synthesized by the speech synthesizer 420. If the hierarchical name has an operation item in the lower layer, a reverberant sound is added. If the designated function number is a setting item, the pitch is lowered if the setting item is invalid, and the pitch is increased if the setting item is selected, and the setting item is read out.
For example, following the depression of the key “0”, if “435” is input with the numeric keypad, a high-pitched voice feedback “the scaling is set to 71% reduction” is made.
When the setting item having the lowest function number is specified, the menu control unit 360 holds the value corresponding to the setting item as the setting value of the setting item on the menu DB 365, and the operation control unit 320 executes the function execution. The setting value now specified via the unit 390 is stored in the corresponding setting item in the memory of the MFP 100, and the visual display unit 350 is called to display the determined content on the screen of the operation unit 104.
[0122]
Next, when the “voice recognition (“ 3 ”)” key is pressed (YES in step S13), the input voice is voice-recognized and moved to the operation item corresponding to the recognition result (step S14). The process returns to step S1 to process the next operation instruction.
The operation input unit 330 calls the sound output unit 380 and outputs a voice saying “Please say function number, operation item name, word or keyword” together with the operation sound.
[0123]
Subsequently, the operation input unit 330 calls the voice input unit 370 and obtains the function number designated by the user from the “function number”, “operation item name”, “word”, or “keyword” uttered by the user.
If the recognition result is “numeric value”, the voice input unit 370 sets it as “function number”.
If the recognition result is not “function number”, an operation item including the recognition result (“operation item name” or “word”) is searched from the menu DB 365, and the result is one operation item. If there is, the function number corresponding to the matched operation item is converted. When there are a plurality of search results, the search results are read out and the user selects them.
[0124]
If any of the above is determined to be a keyword, the help operation unit 385 is used to search the operation support information file 386 using this keyword, and the user can select a desired operation item from the group corresponding to the keyword. A name is selected to obtain a function number corresponding to the selected operation item.
[0125]
The function number obtained by the voice input unit 370 is passed to the menu control unit 360, and the operation item for this function number is managed as the current position.
At this time, the menu control unit 360 calls the sound output unit 380, outputs the sound icon and the input operation item name as a sound, and notifies the user that the sound input is correctly recognized.
[0126]
When the item name or function number specified by the user is a hierarchical name, the hierarchical name is synthesized by the speech synthesizer 420. If the hierarchical name has an operation item in the lower layer, a reverberant sound is added. If the specified function number is a setting item, the pitch is lowered if this setting item is invalid, and the setting item is read aloud if the setting item is selected.
When the item name or function number designated by voice designates the lowermost setting item, the menu control unit 360 holds the value corresponding to the setting item as the setting value of the setting item on the menu DB 365, Operation control unit 320 stores the setting value now specified via function execution unit 390 in the corresponding setting item in the memory of MFP 100 and calls visual display unit 350 to display the determined content on the screen of operation unit 104. To display.
[0127]
Next, when the “partial clear (“ 7 ”)” key is pressed (YES in step S15), the setting value of the setting item in the hierarchy below the current position on the hierarchy menu is returned to the default value (step S16). The process returns to step S1 to process the next operation instruction.
The menu control unit 360 calls the sound output unit 380 and voices the operation sound and its command content (a message “clears the current position on the hierarchical menu and the setting items below it”) to the user. Notice.
[0128]
Subsequently, the menu control unit 360 searches for a setting item in the hierarchy below the current position on the hierarchy menu, and when the setting item has been changed from the default value, the setting value of the setting item is read out to the default value. Ask the user whether to return.
When the user indicates that the change may be made (for example, with a confirmation key), the setting value of the setting item is returned to the default value.
Thereafter, a sound icon and a voice message indicating operation completion are output to notify the user that the partial clear has been completed.
Alternatively, instead of inquiring one by one as described above, it may be inquired by voice whether the set values changed from the default values may be returned to the default values all at once.
[0129]
Furthermore, the menu control unit 360 holds the setting item returned to the default value and the default value as the setting value of the setting item on the menu DB 365, and the operation control unit 320 is now designated via the function execution unit 390. The default values of the set items are stored in the corresponding set items in the memory of the MFP 100, and the visual display unit 350 is called to display the determined contents on the screen of the operation unit 104.
[0130]
Next, when the “description of current position (“ 9 ”)” key is pressed (YES in step S17), information indicating which operation item is currently in the hierarchical menu is provided (step S18), and the next operation is performed. Return to step S1 to process the instructions.
The menu control unit 360 calls the sound output unit 380 and notifies the user of the sound icon, the operation item name at the current position, the setting value, and the like by voice.
[0131]
When the current position is a hierarchy name, the hierarchy name is synthesized by the speech synthesizer 420. If the hierarchy name has an operation item in the lower layer, a reverberation sound is added. If the current position is a setting item, the pitch is lowered if the setting item is invalid, and if the setting item is selected, the pitch is increased and the setting item is read out.
[0132]
Next, when the “confirm setting” key is pressed (YES in step S19), the setting item name and setting value are notified by voice of what setting item the user has selected so far (step S19). S20) Return to step S1 to process the next operation instruction.
The menu control unit 360 sequentially moves to a setting item whose setting has been changed from a default value by the user, calls the sound output unit 380, reads out the setting item name and its setting value, and notifies the user. At this time, the menu control unit 360 manages the movement destination as the current position, and can correct the setting item if there is an error.
Further, when the setting item is changed, the menu control unit 360 holds the changed setting item and the setting value as the setting value of the setting item on the menu DB 365, and the operation control unit 320 includes the function execution unit. The setting value of the setting item now specified via 390 is stored in the corresponding setting item in the memory of the MFP 100, and the visual display unit 350 is called to display the determined content on the screen of the operation unit 104.
[0133]
Next, when the “confirm (“ # ”)” key is pressed (YES in step S21), the selection of the setting item at the current position is confirmed (step S22), and the process proceeds to step S1 to process the next operation instruction. Return.
The menu control unit 360 calls the sound output unit 380, outputs the operation sound and the determined operation content by voice, and notifies the user.
Further, when the setting item of the current position is confirmed, the menu control unit 360 holds the confirmed setting item and its setting value as the setting value of the setting item on the menu DB 365. The operation control unit 320 The setting value of the setting item that has been confirmed through the function execution unit 390 is stored in the corresponding setting item in the memory of the MFP 100, and the visual display unit 350 is called to display the determined content on the screen of the operation unit 104. .
[0134]
Finally, when the setting for each setting item is completed and the “start” key is pressed (YES in step S23), the selected function (copy function, facsimile function, etc.) is executed (step S24), and the next Return to step S1 to process the operation instruction.
The operation control unit 320 calls the function execution unit 390 to operate the function selected by the user first. For example, if the designated function is a copy function, a copy operation according to the values set so far is performed. When this operation ends, the operation control unit 320 calls the sound output unit 380 and informs the user of the success or failure by voice.
[0135]
For example, a result such as “Copying has been completed at a magnification of 122%” or “Copying has failed due to a paper jam” is fed back by voice.
In addition, when the user designates a copy of “40 copies”, the operation control unit 320 causes the BGM to flow through the sound output unit 380 so that it can recognize how much copy has been completed. The user can know how much processing has been completed according to the progress of the BGM.
[0136]
Further, the function control unit 310 monitors the state of the apparatus during the operation of the MFP. For example, in the case of “paper out”, “toner shortage”, “paper jam”, “front cover is open”, etc. , Call the sound output unit 380, and after the sound icon, for example, “paper is out”, “toner is low”, “paper jam has occurred”, “front cover is open” Etc. and the BGM are output and notified to the user.
[0137]
G. Other functional configurations of equipment
In the above-described embodiment, the device operation device (the control unit 300 and the support unit 400) of the present invention has been described as being incorporated in the MFP.
Here, as another embodiment, as shown in FIG. 11, the control unit 300 in the device operation device is incorporated into a device such as an MFP, a facsimile device, a printer device, or a scanner device, and the support unit 400 is a general-purpose computer (hereinafter referred to as a PC). ).
In this case, in order to perform communication between the control unit 300 and the support unit 400, the communication unit 500 and the communication unit 600 are connected by a cable, infrared communication, wireless communication, a communication line (including a network), or the like.
[0138]
In such a configuration, when voice recognition processing is performed from the control unit 300, key information indicating that the voice recognition processing is requested via the communication unit 500 and voice data input by the voice input unit 370 are transmitted to the communication unit. To 600.
The communication unit 600 calls the voice recognition unit 410 from this key information, adds the previous key information to the voice recognition result, and sends it back to the communication unit 500.
Communication unit 500 determines that the received data is a voice recognition result from the key information, and returns the received data to voice input unit 370.
[0139]
In addition, when speech synthesis processing is performed from the control unit 300, key information that is a request for speech synthesis processing via the communication unit 500 and text data that the sound output unit 380 wants to convert into speech data are sent to the communication unit 600. Send.
The communication unit 600 calls the speech synthesis unit 420 from this key information, attaches the previous key information to the speech synthesis result, and sends it back to the communication unit 600.
The communication unit 600 determines that the received data is a voice synthesis result from the key information, and returns the received data to the sound output unit 380.
When obtaining voice data, instead of synthesizing voice data from text data, the voice data recorded by the key information of the voice data (what kind of voice data is desired) may be selected. Good.
[0140]
By sharing the functions between the device and the PC in this way, it is not necessary for each device to have speech recognition and speech synthesis processing functions that require a large amount of data such as the support unit 400 and processing time. Maintenance of the dictionary, processing system and PC itself of the support unit 400 can be facilitated.
In the case of such a configuration, the cost balance is better when a plurality of devices are connected to one PC.
[0141]
As still another embodiment, an operation system related to the visual mode and an input system such as a numeric keypad in the non-visual mode of the device operating device may be incorporated into the device, and the non-visual mode processing may be incorporated into a general-purpose computer. .
[0142]
In such a configuration, when the non-visual mode is selected, an instruction by input performed on the device side is transmitted to the PC side, the instruction received on the PC side is processed, and there are setting items and values for the device. Sometimes these values are returned to the device and values are set in the memory.
[0143]
By sharing the functions between the device and the PC in this manner, it is not necessary to have a non-visual mode processing system for each device, and maintenance of the non-visual mode processing system itself can be facilitated.
In the case of such a configuration, the cost balance is better when a plurality of devices are connected to one PC.
[0144]
Furthermore, as another embodiment, mode switching, selection of functions (copy function, facsimile function, etc.) and input when using the non-visual mode are performed by a general-purpose computer, and values are set for the set items. Alternatively, the selected function may be executed on the device side. In this case, infrared communication or wireless communication is used for communication between the device and the PC.
[0145]
Further, the PC may be a portable terminal that can connect a numeric keypad, a start key, a microphone, and headphones, and may be connectable to a device by communication such as Bluetooth.
By assigning functions to the device and the PC in this way, it is possible to customize the non-visual mode processing system for each user having the PC while minimizing changes on the device side.
[0146]
In such a configuration, a user with a PC goes to the vicinity of the device, selects a non-visual mode, selects a function, and operates a hierarchical menu corresponding to the function from the PC. Send value and function execution to device.
In the case of an instruction for selecting a function, the device sets a function of the device (such as a copy function or a facsimile function) and performs initial setting by changing the function on the device side. Also, the setting of items and the execution of functions are processed by the device. These set results are returned to the PC, and the results are notified to the user by voice or the like.
[0147]
H. Non-visual and visual mode determination
In the above-described embodiment, the mode switching key is pressed, and a headset including a headphone and a microphone is inserted into the apparatus, thereby shifting to the non-visual mode. However, it is difficult for visually impaired people to find a headset and insert it into the device.
Therefore, in this embodiment, a configuration will be described in which the non-visual mode can be used even when the headset is inserted into the apparatus.
[0148]
In FIG. 12, the apparatus according to the present embodiment includes a headset installation unit 710 for suspending the headset 720 on the apparatus body, and a determination unit 700 that determines whether or not the headset is hung on the headset installation unit 710. Is provided.
The headset installation unit 710 is in a position indicated by a dotted line and is in contact with the terminal B when the headset 720 is not hung.
When the headset 720 is hung on the headset installation part 710, it moves to the position of the solid line by the mass of the headset itself and comes into contact with the terminal A.
[0149]
The determination unit 700 sends to the function control unit 310 (see FIG. 9) of the control unit 300 of the apparatus main body which terminal (A or B) is in contact with the position of the headset installation unit 710.
That is, when the headset 720 is put on, since the headset installation unit 710 is in contact with the terminal A, the determination unit 700 transmits a signal indicating that the headset 720 is not used.
The function control unit 310 of the control unit 300 that has received this signal determines the use of the visual mode.
On the other hand, when the headset 720 is not hung on the headset installation unit 710, the headset installation unit 710 is in contact with the terminal B, so the determination unit 700 indicates that the headset 720 is in use. Send a signal to show.
The function control unit 310 of the control unit 300 that has received this signal determines the use of the non-visual mode together with the use of the mode switching key.
[0150]
With the above configuration, the headset can be permanently installed in the device, and the user has to find and insert the position of the terminal for connecting the headset to use the device. Can be omitted.
In addition, by providing the headset installation unit in the apparatus main body, the location of the headset is clarified, and it is possible to eliminate the inconvenience that the user (particularly the visually impaired) cannot find the headset and is confused.
By clarifying the location, the user can quickly find the headset and use the device immediately.
[0151]
Further, when the function control unit 310 receives a signal indicating that the headset 720 is not applied to the headset installation unit 710 from the determination unit 700, the input from the visual input unit 340 is performed simultaneously with the transition to the non-visual mode. If the user is locked, even if the user (particularly visually impaired person) unintentionally touches the touch panel, it can be prevented from being set erroneously.
[0152]
Also, the visual input unit 340 can be enabled / disabled by pressing the mode switching key to enable the visual input unit 340 whenever operation assistance is required, or when used by people with low vision or presbyopia. Can be.
[0153]
I. Embodiment as a program
Each function constituting the above-described embodiment is programmed, written in advance on a recording medium such as a CD-ROM, and this CD-ROM is mounted on a computer equipped with a medium driving device such as a CD-ROM drive. Needless to say, the object of the present invention can also be achieved by storing these programs in a memory or storage device of a computer and executing them.
In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the program and the recording medium on which the program is recorded also constitute the present invention.
[0154]
As a recording medium, a semiconductor medium (for example, ROM, nonvolatile memory card, etc.), an optical medium (for example, DVD, MO, MD, CD-R, etc.), a magnetic medium (for example, magnetic tape, flexible disk, etc.) Either may be sufficient.
[0155]
Moreover, not only the functions of the above-described embodiment are realized by executing the loaded program, but the operating system or the like performs part or all of the actual processing based on the instructions of the program, and the processing described above The case where the functions of the above-described embodiments are realized is also included.
[0156]
In addition, a program for realizing the functions of the above-described embodiment is loaded into a memory provided in the function expansion board or the function expansion unit, and the CPU provided in the function expansion board or the function expansion unit is actually installed based on the instructions of the program. A case where part or all of the processing is performed and the functions of the above-described embodiments are realized by the processing is also included.
[0157]
Further, when the above-described program is stored in a storage device such as a magnetic disk of a server computer and distributed in the form of download from a user computer connected via a communication network, the storage device of this server computer is also included in the present invention. Included in the recording medium.
[0158]
【The invention's effect】
As described above, according to the present invention, the manufacturing cost of the apparatus can be reduced, and various operations can be easily and reliably performed without using vision.
In addition, it is possible to provide a device that can be easily operated by a person who can operate the person who cannot perform the operation.
[Brief description of the drawings]
FIG. 1 is a block diagram of an MFP incorporating an apparatus operating device according to the present invention.
FIG. 2 is a front view of an operation unit.
FIG. 3 is a configuration example of a hierarchical menu related to a copy function.
FIG. 4 is a configuration example of a hierarchical menu related to a copy function (continuation of FIG. 3).
FIG. 5 is a functional layout example of a numeric keypad.
FIG. 6 is a diagram for explaining operation by a direction movement key.
FIG. 7 is a configuration example of operation support information for setting items in the lowest layer.
FIG. 8 is a configuration example of operation support information for a setting item of an intermediate hierarchy.
FIG. 9 is a functional configuration diagram of an MFP in which the apparatus operating device of the present invention is incorporated.
FIG. 10 is a flowchart showing an operation for a copy function operation;
FIG. 11 is another configuration diagram of a device incorporating the device operating device of the present invention.
FIG. 12 is another configuration diagram of a device incorporating the device operating device of the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... MFP, 101 ... Host connection device, 102 ... FAX line connection device, 103 ... Network connection device, 104 ... Operation unit, 105 ... Scanner device, 106 ... Printer engine, 107 ... Control controller, 108 ... Storage device, 109 DESCRIPTION OF SYMBOLS ... Sound output device, 110 ... Voice input device, 201 ... Liquid crystal display (LCD) and touch panel, 202 ... Numeric keypad, 203 ... Clear / stop key, 204 ... Start key, 205 ... Setting confirmation key, 206 ... Function key, 300 ... Control unit, 310 ... Function control unit, 320 ... Operation control unit, 330 ... Operation input unit, 340 ... Visual input unit, 350 ... Visual display unit, 360 ... Menu control unit, 365 ... Menu database (DB), 370 ... Voice input unit, 380 ... sound output unit, 385 ... help operation unit, 386 ... operation support information 390 ... Function execution unit, 400 ... Support unit, 410 ... Speech recognition unit, 420 ... Speech synthesis unit, 500,600 ... Communication unit, 700 ... Judgment unit, 710 ... Headset installation unit, 720 ... Headset, A , B ... terminals.

Claims (7)

All operation items that can be operated on the device are classified by function, and the classification is further subdivided into final setting items, and setting items of hierarchical menus having a multi-level tree structure are selected or set values. In the device operating device for operating the device by the above, a headset , a headset installation unit for suspending the headset connected to the device, and whether or not the headset is installed in the headset installation unit For each common term included in the operation item on the hierarchical menu , a determination unit that determines whether, a mode switching unit that switches between a non-visual mode used by a visually impaired person and a visual mode used by a sighted person , Take out all the routes from the top layer to the bottom layer containing the setting item or hierarchy name containing the term, and to each node of the newly built tree structure, In association with audio information describing the hierarchical name or setting item name of the node in advance and the operational support information files registered, by the mode switching unit, the determining that the headset is not installed in the headset installation unit The voice recognition unit that recognizes the voice of the operation item desired by the visually impaired when the operation of the device is shifted to the non-visual mode, and the voice recognized by the voice recognition unit An operation item corresponding to the recognition result is searched from the operation support information file, and voice information for the operation item of the search result is output by voice to notify the visually impaired person and to the visually impaired person a desired operation. A device operation device comprising: a help operation unit that causes an item to be selected and sets the selected operation item as a corresponding operation item of the hierarchical menu. The apparatus operating device according to claim 1 , wherein the determination unit determines whether or not the headset is installed by moving the headset installation unit based on a mass of the headset. 3. The device operating device according to claim 1, wherein when the operation of the device is in a non-visual mode, the visual operation is not permitted. In device operation apparatus according to claim 1, 2 or 3, the device operating device, characterized in that it comprises at any time switchable mode switch key even during use non-visual mode and visual mode the headset. The program for functioning a computer as each part of the apparatus operating device as described in any one of Claims 1 thru | or 4 . A computer-readable recording medium on which the program according to claim 5 is recorded. Using an instrument operating device according to any one of claims 1 to 4, an image forming apparatus for the setting item for the function of the image forming apparatus from the selected-determined to characterized in that to perform the image formation.

Images