JP4070545B2 - Apparatus operating device, program, recording medium, and image forming apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an equipment operating system which enables a users to easily and surely perform various kinds of operations even without using visual sensation. <P>SOLUTION: The equipment operating system with speech guidance forms hierarchical menus by classifying all operative operation items of equipment by functions and by further segmentizing the classifications down to the final set items and operates the equipment by selecting the set items in the hierarchical menus or by setting the values thereof. The system manages the set items defined by inputting the assignment of the set items by speeches from the user and by subjecting the inputted speeches to speech recognition as the positions on the hierarchical menus, and therefore the operation of the equipment without using the visual sensation is made possible. Here, the recognized words subjected to speech recognition are assumed to coincide with the set items of the hierarchical menus and the user is assumed to assign the set items when the user carries out the definition operation. Since the explanation of the determined set items is performed by speech information, the apprehension of erroneous setting by the user can be eliminated. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus operating device, a program, a recording medium, and an image forming apparatus, and more specifically, a copying machine, a facsimile machine, a printer device, a scanner device, and an OA such as a multifunction machine or a computer that integrates these functions. Incorporates a device operating device that can be operated without using vision in various operations on the device, a program for executing the function of the device operating device, a computer-readable recording medium storing the program, and the device operating device The present invention relates to an image forming apparatus.
[0002]
[Prior art]
When operating application software on a personal computer or the like, if there are abundant functions, similar functions are often collected as a hierarchical menu. In the case of such a hierarchical menu, the user needs to make several selections in order to reach the desired function.
In addition, since the menu has a hierarchical structure, there is a problem that it is difficult to intuitively determine which menu is selected to reach the desired function.
[0003]
In order to solve this problem, the “auxiliary operation device for device” in Patent Document 1 indicates the operation mode of the device by voice so that the user is not aware of the hierarchical structure of the menu. To be able to do.
[0004]
[Patent Document 1]
Japanese Patent Laid-Open No. 9-258943
[0005]
[Problems to be solved by the invention]
In the technique of the above-mentioned patent document 1, when voice recognition is performed, the setting according to the recognition result is fixed, so that it is not set correctly when it is set unintentionally or when it is erroneously recognized. Become.
For this reason, there is a problem that the set state must be temporarily returned to the previous state and voice recognition must be activated again.
[0006]
In order to solve this problem, it is conceivable to provide an undo operation for canceling the setting when it is set by mistake.
However, considering the response to the user, it is desirable that the operation is easy. If a new operation called undo is provided, it is necessary to learn the operation for that purpose, which increases the burden on the user.
In addition, if the operation becomes complicated, the willingness to learn will also fade, and it will not be possible to provide devices that are easy to handle for anyone that complies with US accessibility laws (Article 508 of the Rehabilitation Act, Article 255 of the Communications Act, etc.) become.
[0007]
The present invention has been made in consideration of the above-described circumstances, and is a device operating device that allows a user to easily and reliably perform various operations without using vision. It is an object of the present invention to provide a program for executing a function, a computer-readable recording medium storing the program, and an image forming apparatus incorporating the apparatus operating device.
[0008]
[Means for Solving the Problems]
  In order to solve the above-described problem, the device operating device according to claim 1 of the present invention classifies all operation items that can be operated by the device by function, and further subdivides the classification into final setting items. A voice input unit for inputting the designation of the setting item by voice from a user in a device operating device with voice guidance that creates a hierarchical menu and selects or sets a setting item in the hierarchical menu to operate the device. A voice recognition unit that outputs a recognition word obtained by voice recognition of the input voice, and a menu control unit that manages a setting item specified by the voice input unit as a position on the hierarchical menu;, A setting confirmation unit that is managed by the menu control unit and that voice-feeds back a setting item changed from a default value by a user;Be equippedGetIt is characterized by that.
Further, according to a second aspect of the present invention, in the apparatus operating device according to the first aspect, the apparatus further comprises a confirmation means operated by a user, wherein the voice input unit performs voice input to the voice recognition unit. It is assumed that the setting item corresponding to the last recognized word is designated when the setting item corresponding to the recognized recognized word is voice-feedback and the confirming operation is performed by the confirming unit, and the menu control unit Is characterized in that the setting item specified by the voice input unit is explained by voice information.
  Further, the claims of the present invention3Claims2In the device operating device described in the above, the voice input unit is in a state where the user does not perform a confirming operation.The input voice is received by the voice recognition unit.Recognition word recognized by speechVoice feedback of setting items corresponding toContinuationContinueIt is characterized by that.
  Further, the claims of the present invention4Claims3In the device operating device described in the above, the voice input unit is in a state where the user does not perform a confirming operation.The input voice is received by the voice recognition unit.Recognition word recognized by speechVoice feedback of setting items corresponding toContinuationContinueIt is characterized by that.
  Further, the claims of the present invention5Claims3In the device operating device according to claim 1, the voice input unit is, AheadExists in the lower hierarchy of the setting item recognized byVoice feedback of setting items corresponding to the recognized wordsIt is characterized by doing.
  Further, the claims of the present invention6The program comprises a computer.5This is a program for causing a device operating device to function as described in any one of the above.
  Further, the claims of the present invention7The recording medium of claim6A computer-readable recording medium on which the device operation program described in 1 is recorded.
  Further, the claims of the present invention8The image forming apparatus according to claim 1.5The apparatus operating device according to any one of the above is used to select and determine setting items for the function of the image forming apparatus and then form an image.
[0009]
With the above configuration, when users operate the device, it eliminates the anxiety of setting mistakes and can use voice recognition many times with peace of mind. The operation can be performed easily and reliably.
Also, by correctly recognizing pauses inserted between spoken words for voice recognition, users can use voice recognition with peace of mind, making it easy to perform various operations without using vision. And it can be performed reliably.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
In the following description of the embodiment, it is assumed that the device operating device of the present invention is incorporated into an MFP (Multi-Function Peripherals). However, the installation target of the device operating device of the present invention is not limited to the MFP, and the same effect can be expected if the device has diversified functions.
[0011]
A. MFP incorporating the device operating device of the present invention
FIG. 1 is a block diagram of an MFP incorporating an apparatus operating device according to the present invention.
In FIG. 1, an MFP 100 includes a host connection device 101 for connecting a host computer and the MFP, a FAX line connection device 102 for connecting to a FAX line, a network connection device 103 for connecting to a network such as a LAN, and the MFP. An operation unit 104 for performing various setting operations, a scanner device 105 that scans an image from a document table and creates image data, a printer engine 106 having an image forming device and a paper supply / discharge device, control of the entire MFP, printer language Controller 107 for processing, facsimile control, scanner control, user recognition processing, sound output control, image data editing processing, control for MFP operation, etc., facsimile received image data, communication record and image data used in each function The storage device 108 to be saved and the user's operation result Speaker for outputting sound and voice Te, sound output device 109 of the headphone or the like, consisting of the voice input device 110. Inputs audio operation instruction by the user emitted.
[0012]
In such an MFP 100, when the printer function is used, print data transmitted from the host computer via the host connection device 101 or print data transmitted from a computer connected via the network connection device 103 is used. Are temporarily stored in the reception buffer of the control controller 107, and are developed into image data by the control controller 107 in accordance with printer language processing, and are sent to the printer engine 106 for printing.
When using the copy function of the MFP 100, printing is executed by sending the image data read from the scanner device 105 to the printer engine 106 in accordance with an instruction from the controller 107.
[0013]
If the user is a person who can handle this MFP 100 freely, a copy instruction is given according to the picture information displayed on the operation panel of the operation unit 104, and the image data read by the scanner device 105 is temporarily stored in the storage device. 108, the image data is edited by the control controller 107, the image data is edited, and a plurality of images are overlapped. Then, the image data is printed by the printer engine 106, or connected to the FAX line. 102 can be FAX-transmitted.
[0014]
If the user is a person who cannot handle the MFP 100 freely, the sound output device displays the operation status in response to an operation instruction given by an input device such as a numeric keypad or the voice input device 110 that can be identified by the user. Judgment is made based on the sound and sound output to 109, the operation status is confirmed, and the copy function is operated.
[0015]
Further, image data read by the scanner device 105 of the MFP 100, facsimile image data obtained via the FAX line connection device 102, and image data obtained via the network connection device 103 are converted into the host connection device 101 and the network connection device 103. It can also be used to reprocess the image data sent to the computer via the computer and received by the computer application.
Further, the MFP 100 can obtain image data from an image data server connected to the Internet via the network connection device 103 and print it by the printer engine 106.
[0016]
B. Operation unit and operation panel
FIG. 2 is a front view of the operation unit 104 in the MFP 100 in which the device operation apparatus of the present invention is incorporated.
In the center of the operation unit 104, a liquid crystal display unit (LCD) 201 and a touch panel are provided. The liquid crystal display unit 201 displays setting items for functions (FAX, copy, printer, etc.) incorporated in the MFP 100, and the user sets a value in the displayed setting item to thereby display the setting items of the MFP 100. Perform the operation. The function is switched by a function key 206 provided on the left side of the liquid crystal display unit 201.
On the right side of the liquid crystal display unit 201, various keys such as various numeric keys 202, a clear / stop key 203, a start key 204, and a setting confirmation key 205 that are commonly used for each function are arranged.
[0017]
C. Overview of operations using this MFP
Users of the MFP 100 are classified into the following two types.
(1) Person who cannot operate this MFP freely
Those who are classified into this are screens and touch panels 201 of the liquid crystal display unit 201 of the operation unit 104 such as visually impaired persons, physically handicapped persons, low vision persons, and presbyopic persons (hereinafter referred to as visually impaired persons). A person who cannot see and operate.
Or even if it is a healthy person, the person who selected the operation by voice or sound is also included in this classification.
When a person belonging to this category operates the MFP 100, guidance of operation is performed by voice and sound. This processing mode of the MFP is referred to as “voice mode”.
[0018]
(2) Person who can operate this MFP freely
The person classified into this is a person who is not included in the above (1), and is generally a sighted normal person.
When a person belonging to this category operates the MFP 100, the operation is performed by the liquid crystal display unit 201 and the touch panel 201 of the operation unit 104. This processing mode of the MFP is referred to as a “visual mode”.
[0019]
Hereinafter, an outline of the operation of MFP 100 will be described using “operation for copying a document”. Regardless of the above-mentioned user, the user of MFP 100 can use any power source or numeric keypad of MFP. It is assumed that you know whether or not you have a basic function and operation method of the MFP.
[0020]
First, when the MFP 100 is turned on, it is set to operate in the “visual mode”. Therefore, a user who can use the “visual mode” can change various screen displays and touch panels of the operation unit 104 as they are. An instruction can be given and MFP 100 can be operated.
[0021]
On the other hand, a person who wants to operate MFP 100 in the “voice mode” keeps pressing a mode switching key (for example, “* key” on the numeric keypad) for a certain period of time (3 seconds) or more, and a headset equipped with headphones and a microphone to MFP 100. Connect and switch from “Visual Mode” to “Audio Mode”. When switched to “voice mode”, the headphone is notified by voice that “entered voice mode”.
Once the “voice mode” is set, the instruction from the touch panel is locked. However, by switching the “mode”, it is possible to switch at any time even while wearing the headphones.
In addition, the connection between the headset and the MFP 100 may be performed using wireless communication or Bluetooth.
[0022]
Next, by depressing the function key 206 “Copy” (see FIG. 2) of the operation unit 104 of the MFP 100, the user is notified by voice that “copy operation is now possible” from the headphones.
[0023]
Also, in this MFP 100, a single menu having a multi-level hierarchy is provided in which all operation items that can be operated in advance are classified by function for each function such as FAX and copy, and the classification is further subdivided. Create.
For example, the “Copy” function is classified into a hierarchy as shown in FIGS. 3 and 4 (this function classification is only a part).
[0024]
The user uses a direction movement key installed in the operation unit 104 or a key that can be substituted for this direction movement key (hereinafter, collectively referred to as a direction movement key) to trace a hierarchy menu to a desired operation item. The operation of moving and selecting or setting a value is repeated to complete the setting, and then a start key 204 is pressed to create a copy of the document.
In addition, the user can directly input a function to be set using voice recognition instead of the direction movement key as described above.
[0025]
As the user operates the hierarchical menu, the explanation of the selected function and the contents of the setting items set so far are fed back to the user by voice, sound or BGM (Back Ground Music). Even without looking at the screen of the liquid crystal display unit 201 of the operation unit 104 or inputting from the touch panel 201, various operations of the MFP 100 can be easily and reliably performed without using vision.
Here, voice, sound and BGM are set using attributes (sound pitch, sound speed, voice type (male, female, mechanical voice, etc.), sound quality, tone, tone, sound field feeling, etc.) The output is changed according to the type and setting status of the item and the depth and location of the hierarchy.
[0026]
Further, the selection process by the direction movement key or the like in the “voice mode” is displayed on the display screen of the liquid crystal display unit 201 of the operation unit 104 in a substantially correlated manner.
Here, for example, when the user loses his / her operation on the way, when he / she moves to “visual mode” with the headphones on and selects a setting item on the touch panel 201 of the operation unit 104, Since the operation results are fed back to the headphones, after knowing what the user such as the visually impaired has set, the user can return to the "voice mode" and continue the operation. It becomes easy to assist.
[0027]
In addition, since the operation result by the user is displayed almost synchronously with the screen of the liquid crystal display unit 201 of the MFP 100, a weakly sighted person or a presbyopic person can see the screen and operate in the visual mode.
[0028]
D. Details of operations on this MFP
(1) Structure of hierarchical menu
It is difficult to remember if all the operation functions are made into multiple menus as they are, so all the operation items that can be operated are classified according to the function for each function such as FAX function, copy function, printer function, scanner function, NetFile function handled by MFP. The classification is further subdivided to create a single menu having a multi-level tree structure and stored in the menu database (DB).
When the user uses the MFP 100 and selects a desired function from among the functions of the MFP described above using the function key 206, a menu corresponding to the function is read from the menu database and set.
[0029]
If there are more detailed setting items, such as “Variable” and “Double-sided”, the copy function menu uses “Variable” and “Double-sided” as one hierarchical name, and Below the hierarchy name to be selected such as “same size”, “enlarge”, “reduce”, etc., and subdivide until the hierarchy name becomes the final setting item such as “115%”, “93%” . Hereinafter, the hierarchy name and the setting item are collectively referred to as an operation item.
[0030]
The types of setting items on this hierarchical menu are as follows.
・ Selectable items that can be selected only once in the same hierarchy
For example, “115%” or “122%” under the “enlargement” level
・ Toggle item that can toggle function ON / OFF
For example, “Whether to sort” under the “Finisher” level
・ Numeric items that can be set
For example, the number of copies
・ Operation items that start some operation
For example, creating a destination list registered with the FAX function
[0031]
For example, in the case of the copy function, first, as shown in FIG. 3 and FIG. 4, the hierarchy name of the first hierarchy is set to (number of copies, manuscript, paper, scaling, duplex / aggregation / division, finisher, etc.) Subdivided into the third level.
Also, as shown in FIGS. 3 and 4, numbers correspond to the operation items (hierarchy names and setting items) of each layer, and by specifying these numbers, the operation can be directly performed without following the layer. Items can be selected. Hereinafter, this number is referred to as a “function number”.
[0032]
(2) Hierarchy menu operation
Next, a method for operating the above-described hierarchical menu will be described.
In order to move and select the operation items arranged on the hierarchical menu, a direction movement key or a key that can be substituted for it is used. In the following description, the numeric keypad 202 is used instead.
[0033]
FIG. 5 shows functions assigned to each key.
In FIG. 5, the hierarchy is moved up by pressing the “move up (“ 2 ”)” key, and the hierarchy is moved down by pressing the “move down (“ 8 ”)” key.
Further, the same level is traced to the left by pressing the “move left” (“4”) key, and the same level is traced to the right by pressing the “move right” (“6”) key.
Note that the direction movement key described above may be assigned in reverse.
[0034]
When the user instructs to move to the lower layer of the hierarchical menu, if there are a plurality of lower layer operation items, the user moves to the next operation item by one of the predetermined methods listed below.
• Move to the first item below.
• Move to the item already selected below.
-Move to the item with the default value in the lower layer.
[0035]
When the user operates the horizontal movement key to move the same hierarchy and reach the end of the hierarchy, the user moves to the opposite end of the hierarchy and performs a cyclic movement. By always moving in a certain direction, it is possible to find an item that the user wants to select. However, there is a high possibility that the number of items in this hierarchy cannot be grasped and the user is lost on the menu.
Alternatively, this cyclic movement may not be performed and no further movement may be performed when the end is reached. In this case, if the desired item is overrun, the move key must be pressed in the opposite direction, but it is possible to grasp how many items are in the hierarchy.
[0036]
A method of using the direction movement key will be described with reference to an example in which the operation is performed so that the copy result is dark because the image of the document is thin (see FIG. 6).
When the operation is started, the current hierarchy name “number of copies” is notified to the user by voice, and the rightward movement key is used to advance to the hierarchy name “original”. At this time, the user is informed by voice that the user is at the “original” position in the hierarchical menu.
Next, in order to select “density” which is a subcategory of this “document”, the hierarchical menu is moved down to the next lower level by using the downward movement key. At this time, it is informed by voice that the user is in the “type” position.
[0037]
Further, the right movement key is pressed to advance to the hierarchical name “density” which is one of the subcategories of “original”. At this time, it is informed by voice that the user is in the “density” position.
Similarly, in order to select “deep” which is a subcategory of this “density”, the hierarchical menu is moved down to the next lower level by using the downward movement key. At this time, it is informed by voice that it is in the “automatic” position.
Further, the right movement key is pressed in order to advance to the hierarchical name “dark” which is one of the subclasses of “density”. At this time, since it is informed by voice that it is in the “dark” position, the enter key is pressed.
As a result, it is possible to achieve the setting “copy the original darkly” requested by the user.
[0038]
Furthermore, in FIG. 5, when the user does not know which level is in the middle of setting, or when he / she wants to select again from the beginning, for example, the “return to top (“ 1 ”)” key is pressed. Thus, it is possible to return from any hierarchy to the leftmost operation item of the top hierarchy (in the example of FIG. 3, “number of copies” of the first hierarchy).
In this case, setting items that have been selected or set so far can be stored in MFP 100 and moved to the items that have been selected and set by the move key.
[0039]
In addition, in FIG. 5, by pressing a “direct jump (“ 0 ”)” key, the user can directly move to a hierarchy or item that the user wants to set.
For example, by pressing a “direct jump” key and inputting a function number from the numeric keypad, it is possible to move to an operation item corresponding to the function number.
For example, when the function number “435 #” (here, the “#” key indicates the end of the number input) is entered numerically with the numeric keypad after the “direct jump” key is pressed, the MFP 100 displays “scaling”. It is determined that the items are selected in the order of “→ reduction” → “71%” (see FIGS. 3 and 4).
This function number may not be a setting item at the lowest layer but may be a hierarchy name of an intermediate hierarchy. In this case, it moves to the hierarchy name and selects from the operation items in the hierarchy below the hierarchy name.
[0040]
Further, instead of inputting the function number of the item to be set directly by the “direct jump” key as described above, the user presses the “voice recognition (“ 3 ”)” key, and the user selects “function number”, “ It is also possible to say “hierarchical name”, “setting item name” or keyword, and MFP 100 recognizes it and moves directly to the recognized item.
In this voice recognition, the recognition result is fed back by voice. If an error is noticed, the user again speaks and confirms, and then presses the “OK (9)” key to input a desired operation item.
[0041]
Further, the user can utter one or more words, search for an operation item including a speech recognition result from the hierarchical menu, and can directly move to the searched operation item.
For example, if there is “Double-sided / Aggregate / Split” → “Bookbinding” → “One-sided weekly magazine” on the hierarchical menu, when the user utters “Weekly magazine”, it is related to the position of the currently operating hierarchical menu. Without going directly to that weekly magazine.
[0042]
If the user wants to move to a setting item for punching holes in the copied paper, say “Punch” to move to the operation item of the hierarchical menu “Punch”.
Furthermore, if it is desired to move directly to either the “left” or “right” setting item below the “punch” item, it is sufficient to say “punch hiding” and “punch migi”.
If you notice an error after saying “Punch Hidari”, you can say “Punch Migyo” again and press “Confirm (9)”.
[0043]
Further, in FIG. 5, by depressing a “current position explanation (“ 5 ”)” key, the operation item on the hierarchical menu is fed back by voice. Examples of the feedback information include an operation item name, a function number, and a set value (number of copies).
For example, if the “current position” key is pressed after the paper is set and fixed on the A4 side, “the paper is set on the A4 side” is fed back by voice.
[0044]
Further, by depressing a “setting confirmation” key 205 (see FIG. 2), the user can collectively confirm the setting items changed from the default values, and can also change the setting values.
[0045]
Further, in FIG. 5, when the hierarchical menu is operated with the above-described keys and setting of a desired setting item is completed, a “determine (“ 9 ”)” key is pressed to determine the setting item. As a result, the MFP 100 displays the content of the determined setting item on the screen of the liquid crystal display unit 201 of the operation unit 104.
[0046]
In FIG. 5, the “#” key is used as a key for indicating the end of a number when a number is input by a direct jump command or setting of the number of copies.
[0047]
In FIG. 5, when the “mode change (“ * ”)” key is pressed, the current mode is changed.
When the “mode switch (“ * ”)” key is pressed, if the current mode is “audio mode”, the mode is switched to “visual mode”. However, if the headset remains connected, the result of the operation on the touch panel is output to the headphones. Therefore, in order to return to the “visual mode” completely, the headset also needs to be disconnected.
On the other hand, when the “mode switching (“ * ”)” key is pressed and the current mode is “visual mode”, the above-described headset can be switched to “audio mode” by connecting to the MFP 100.
[0048]
(3) Operation result operation sound, sound icon and feedback by BGM
As described above, when moving / selecting operation items arranged on the hierarchical menu using the numeric keypad, confirmation of the result of the operation as shown below is performed using operation sounds, sound icons, and BGM (background music). Output to the user's headphones.
Here, the sound icon is static information indicating the nature of the object, and is a sound (short single sound) that does not move with different timbres for the purpose of discriminating the type of operation item.
[0049]
In addition, the operation sound uses different sounds (chords and very short melody) of the same timbre to generate an image of the operation result, and associates the meaning of the operation with the sound in music theory. For example, ascending and descending sounds on the hierarchies, respectively, the rising and falling sound forms, the 4th degree dissonance sound that makes you feel anxious when it comes to a dead end (failure of movement), and the progress that makes you feel stable when selecting and confirming. Use dominant motion.
[0050]
For example, an operation sound for notifying the up and down movement of the hierarchy is prepared, and when the user moves up and down the hierarchy, the operation sound is output to notify that the hierarchy has been moved up and down. This operation sound is different from the above sound icon.
At this time, it is determined whether or not there is an operation item in the lower layer of the movement destination, and the operation sound is output differently depending on the presence or absence of the operation item in the lower layer and notified to the user.
[0051]
If the same level of cyclic movement is not set, when the movement is instructed in the horizontal direction, if there are no more items to move, the user is informed with a “dead end” operation sound.
In addition, when a key such as moving to the top, moving in the horizontal direction, directly moving, or voice recognition is pressed, an operation sound for confirming the selection operation is output so that the user can be surely accepted the operation. I can confirm that.
[0052]
In addition, if the BGM has a slow tone in the upper layer and the BGM has a faster tone as it goes to the lower layer, it is possible to recognize which side of the layer is currently located. Also, in the case of numerical input such as inputting the number of copies, it can be confirmed that the numerical input is performed by playing BGM until the numerical input is completed.
[0053]
In addition, the user is notified by voice of the operating state of the MFP, for example, “out of paper”, “toner shortage”, “paper jam”, “front cover is open”, and the like. If such a condition cannot be resolved immediately, the condition cannot be grasped if the sound is interrupted.
For example, if the user is away from the MFP while performing a copy operation and “paper jam occurs”, the user cannot recognize this sound alone, but BGM can be played after the sound. In such a state, the user can recognize it.
[0054]
In addition, when the user designates 40 copies, the time required for the entire copy and the time for one BGM to end can be associated with each other so that it can be recognized how much copying has been completed. The user can know how far copying has progressed depending on which side of the BGM he / she heard.
[0055]
In this way, by notifying the user with an operation sound, BGM, or sound icon that gives an image of the operation, it is possible to confirm the user's current state, whether the user has entered it correctly, or operation Anxiety about whether or not can be completed.
[0056]
(4) Voice feedback of operation results
As described above, when moving / selecting the operation items arranged on the hierarchical menu using the numeric keypad, the user can confirm whether the input and the operation result have been made reliably by outputting from the headphones also by voice. Can be recognized and anxiety can be reduced.
The voice at this time was placed by the user by changing the voice attributes (pitch, voice type, tone quality, tone, tone, speaking speed, sound field feeling) according to the situation. The situation can be grasped intuitively.
[0057]
Further, when a hierarchical menu is reached by a moving operation and an operation item is reached, voice feedback of a predetermined content is performed on the operation item.
For example, when moving to an operation item such as “enlarge” or “114%”, the item name is read out. Thereby, it can be recognized which operation item the user has moved to.
[0058]
When this item name is read out, an already selected item is read out in a higher voice than an item that has not yet been selected. This gives the user a high-activity image and can inform the user that the item is currently selected more quickly and intuitively than with the language expression.
[0059]
For example, in the hierarchical menu of FIG. 3 and FIG. 4, “same size” and “enlarge” in the lower level of “magnification” are to be selected, but which one is currently selected If the user can know whether this is done, it is possible to eliminate the waste of selecting “same size” even though “same size” is selected.
In addition, in an item for switching ON / OFF by selection such as “mixed document size” or the like, it is not known whether it is currently selected (that is, whether it is ON) or not selected (that is, whether it is OFF). I cannot set it to my desired state.
[0060]
Also, when reading the item name, the invalid item is read out with a lower voice than the valid item. This gives the user a low-activity image and can quickly and intuitively tell the user that the item is currently invalid, rather than using linguistic expressions.
Selecting an item may affect other items, resulting in a situation where other items cannot be selected.
[0061]
For example, when “automatic”, “tray 1”, or the like is selected for “paper”, it is not effective to select “paper type of manual paper”. If you can select and move to an item that you can't select, you can confuse the user with a misunderstanding that you can select it.
However, if the item is temporarily deleted from the hierarchical menu or is not moved, the hierarchical menu structure will change dynamically. For users who know the menu structure only auditorily It is difficult to understand why the menu structure has changed, and it causes confusion.
[0062]
When reading out as described above, the upper and lower pitches of the voice need to have a change amount that can be distinguished from the normal pitch, and it is necessary to keep the utterance content in a clearly audible range. For example, in consideration of a sound range in normal natural speech and a sound range that can ensure clarity by speech synthesis, a change amount of about a half octave is given.
[0063]
In addition, when reading the item name, a reverberation sound is added to the item name having the item of the lower hierarchy and output. This gives the user a lingering image, and can tell more quickly and intuitively that there is an item group that is further sub-classified after that item, rather than by language expression.
For example, there are items on the hierarchical menu that do not have a lower hierarchy such as “same size”, and items that have a sub-category item in a lower hierarchy such as “enlarge”. Being able to understand this difference without actually moving to a lower hierarchy makes it easier to see the movement of the hierarchy menu.
[0064]
By pressing the “Description of current position (“ 5 ”)” key, information such as the operation item name and setting value for the operation item is provided in order to know which operation item on the hierarchical menu is currently located. Feedback with the contents of
For example, after the setting of “paper” is set to “A4 landscape” and the “description of current position” key is pressed, voice feedback “paper is set to A4 landscape” is performed.
[0065]
Further, when the selection of the setting item is completed and a confirmation operation is performed with the “confirmation (“ 9 ”)” key, the confirmed content is fed back with sound of a predetermined content.
For example, when “200%” is selected / confirmed in “enlargement”, “scaling is set to 200% enlargement” is fed back by voice.
[0066]
When the setting confirmation key is pressed, the setting items that are below the operation items (including the intermediate layer) on the current hierarchy menu and whose settings have been changed from the default values are collectively read out by voice.
At this time, instead of using voice feedback for all the setting items at once, feedback is made for each setting item, and if there is an error in that setting item, the next changed setting is made by operating the movement keys. It is also possible to move to the item and finally press the enter key to make the final decision.
Also, if you want to know all the setting items that have been changed from the default values so far, use the “Return to top (“ 1 ”)” key to move to the top of the hierarchical menu and then press this setting confirmation key Can be confirmed.
[0067]
When the setting of each setting item is completed, the start key is pressed, and when the operation of the MFP is completed, voice feedback of a predetermined content for notifying the success or failure is performed.
For example, a result such as “copying has been completed at 122% magnification” or “copying failed due to paper jam” is fed back by voice.
[0068]
Also, during operation or during operation, the voice of the speaker, which is different from the voice during the operation of the operation item described above, continues after the operation sound, for example, “paper is exhausted”, “ Notify the user that "Remaining low", "Paper jam has occurred", "Front cover is open", etc.
[0069]
Providing a knob that changes the sound attributes (pitch, voice type, tone quality, tone, tone color, speech speed, sound field feeling) by the above-mentioned voice feedback, so that the user can easily hear the voice output. Can do.
Further, when the next key operation is performed without waiting for the end of the voice feedback, the user can be frustrated by automatically increasing the speaking speed.
[0070]
(5) Example of operation
Next, an example of operation until the operation is completed after setting “number of copies to 2” using the above-described MFP will be described.
First, when the hierarchical menu is operated to move to “number of copies”, the setting item name “number of copies” is read out from the headphones, followed by a sound icon “(pylon) copies”. At this time, since there is no layer below “number of copies”, no reverberation sound is added. Here, when the user presses the “confirm (“ 9 ”)” key, an input voice guide for the number of copies is made following the sound icon “Please input the number of copies with the (Kalan) numeric keypad and press the confirm key” from the headphones. The BGM flows during the numerical input of the number of copies.
Here, the sound of “(pylon)” and “(caran)” indicates an operation sound.
[0071]
When the user presses the “2” and “#” keys using the numeric keypad, the number input from the headphones is read. When the user presses the “OK (“ 9 ”)” key, the “(Kalan) You hear “The number of copies is set to 2”.
After this, when the user presses the start key, BGM is played while copying from the headphones, and when the user finishes, there is a voice guidance saying that “two copies have been completed”.
[0072]
E. Functional configuration in MFP
FIG. 7 is a functional configuration diagram of the device operating device of the present invention incorporated in the MFP.
7, the device operating device includes an operation control unit 310, an operation input unit 320, a visual input unit 330, a visual display unit 340, a menu control unit 350, a menu database (DB) 360, a voice input unit 370, and a sound output unit 380. , A function execution unit 390, a speech recognition unit 410, and a speech synthesis unit 420.
[0073]
The operation control unit 310 instructs the user to use which function such as a copy function or a facsimile function, and performs initial setting for the operation and control of the entire apparatus. In the initial state when the MFP is started up, the “visual mode” is set.
[0074]
The operation input unit 320 performs input other than visual input such as a numeric keypad.
In “Visual mode”, numbers etc. are entered from the numeric keypad, but in “Voice mode”, from the direction key for moving the hierarchical structure of the hierarchical menu, various keys for selecting / determining operation items, the numeric keypad, etc. Various operation instructions are input, and the function number determined by the input ten key type and the input ten key type are transferred to the menu control unit 350.
[0075]
(1) In case of “Move Direction” key
Configure the function number so that it increases by 10 times each time one level is lowered, and increases by 1 every time it moves to the right in the same level, and the current hierarchy menu position is calculated using the type of direction movement key. The operation number can be determined by calculating the function number of the next movement destination from the function number.
For example, when the right movement key is pressed from the initial state (current function number = 1), the function number = 1 + 1 = 2. Here, when the downward movement key is pressed, the function number = 2 × 10 + 1 = 21.
Next, when any one of the upward movement key, the downward movement key, and the rightward movement key is pressed, the function numbers are 2, 211, and 22, respectively.
[0076]
(2) In case of “Direct Jump” or “Back to Top” key
Enter the “Direct jump (“ 0 ”)” key to enable numeric input, the user enters the function number corresponding to the operation item from the numeric keypad, and finally press the “Numeric input end (#)” key. Press.
As a result, the numerical value input from the numeric keypad is set as the function number.
For example, to move to “text / photo”, when “2 → 1 → 2 → #” is pressed, function number = 212 is set.
When the “return to top (1)” key is input, function number = 1 is set.
[0077]
(3) For voice recognition
When the “voice recognition (“ 3 ”)” key is input, the operation input unit 320 activates the voice input unit 370, and the operation item name and keyword (a word representing a part of the operation item name) input by the user from the microphone. ) Or the function number is voice-recognized and the converted function number is returned.
[0078]
(4) In case of “Mode change (*)” key
In this case, the mode is switched to return to the operation control unit 310.
When the current mode is “visual mode”, when the user inserts a headset equipped with headphones and a microphone, the mode shifts to “audio mode”. Even if switching is specified here, if the headset is not connected, the “voice mode” is not entered.
On the other hand, when the current mode is the “audio mode”, the process shifts to the “visual mode”. However, if the headset is connected, the temporary “visual mode” is set and the operation result is output to the headphones. However, if the headset is not connected, it switches to the full “visual mode”.
[0079]
(5) For other keys
In the case of the “current position explanation (5)” key, “setting confirmation” key, and “confirmation (9)” key, the type of the numeric keypad is transferred to the menu control unit 350 as it is.
Further, when the function key 206 is pressed, the type is returned to the operation control unit 310 to change various settings.
[0080]
The visual input unit 330 uses a touch panel installed on the screen, and generally performs an input for selecting and setting various operation items displayed on the screen in the “visual mode”. It is also possible to temporarily switch from the “voice mode” to the “visual mode” and select an operation item.
The visual display unit 340 displays various functions on the screen in the “visual mode”. In the “voice mode”, the user selects from the hierarchical menu and displays the set items that are confirmed.
[0081]
The voice input unit 370 is activated from the operation input unit 320 and inputs an operation item name, a function number, a word or a keyword included in the operation item uttered by the user, and the input voice is recognized by the voice recognition unit 410. Then, each function number is converted into a function number according to the result, the operation item name corresponding to the converted function number is voice-feedback, and voice input is repeated until the “Enter (9)” key is pressed. When “(9)” is pressed, the voice input is ended, and the function number at the end is returned to the operation input unit 320.
[0082]
If a number is input as a result of speech recognition, the recognition result is set as a function number. For example, when “2164” is uttered, function number = 2164 (original setting direction) is set (see FIG. 4).
[0083]
When an operation item name is input, the operation item name is converted into a function number by searching the correspondence table or menu DB 360 stored in the memory in association with the operation item name and the function number.
For example, when “Mojishashin” is uttered, it is determined that “original” → “type” → “text / photo” is selected, and function number = 212 is set (see FIG. 4).
[0084]
When “Punch Hidari” is uttered, it is determined that “Punch” → “Left” has been selected, and function number = 631 is set (see FIG. 4). When the user confirms that “punch hiding” was realized that he intended to punch the punch hole to the right, when the user says “punch migi” again, the function number = 632 “punch migi” is sounded. You can correct it with feedback.
[0085]
In this way, no matter how many times speech recognition is performed, since the speech input is not confirmed until the “determine (9)” key is pressed, it is possible to correct the mistake by speaking again. Further, by eliminating the time limit for voice reception, the user can use voice recognition with peace of mind.
[0086]
However, the way of utterance varies depending on the user, and some people utter the word “punch hiding” continuously, inserting a pause (pause) between “punch” and “hidari”. Some people speak.
In this case, since the utterance is interrupted at the pause, it is determined that the utterance has ended, and the voice recognition process is performed. In particular, since it is difficult for a person with a disability in a language function or a motor function to utter a long word continuously, a mechanism that can correctly recognize even if a pause is inserted is required.
[0087]
For this reason, the speech recognition result for the immediately preceding utterance delimited by the pause is converted into a function number (hereinafter referred to as “preceding function number”), and the function number is stored (see FIG. 8). When the voice input unit 370 is activated, this “preceding function number” does not exist, and is set to 0.
Next, it recognizes the voice uttered after the pause, checks whether the recognition result is in the operation item in the lower hierarchy of the “preceding function number”, and provides feedback of the existing operation item and the corresponding function The number is stored as “preceding function number”. On the other hand, if there is no matching operation item, it is determined that there is no connection with the previously recognized recognition result, the current recognition result is discarded, and the previous “preceding function number” is maintained.
Here, when the “preceding function number” is 0, the recognition result is voice-feedback and the function number corresponding to the recognition result is stored in the “preceding function number” as it is.
[0088]
Hereinafter, the recognition process for the utterance divided by the pause and the conversion to the function number are repeated, and when there is an utterance pause in the section longer than the pause section, the repetition is finished, and the last stored "preceding function number" Is a function number for a series of utterances. Therefore, if a mistake is made, it is only necessary to pause the utterance longer than the pause and then utter again correctly.
[0089]
For example, when the utterance of “Punch Hidari” is interrupted in the pause section between “Punch” and “Hidari”, the function number converted to “Punch” is 63. This function number = 63 is stored as “preceding function number”.
Subsequently, when “hidari” is recognized, the operation item name “hidari” is searched on the hierarchical menu in the lower hierarchy of “preceding function number”.
Since the function number = 631 is the matching operation item, this is stored in the “preceding function number”.
In this way, it is possible to recognize a desired item even if uttering with a pose between “punch” and “hidari”, so it can cope with various utterance forms of the user, so there is no restriction on the utterance method, Strengthen support for people with disabilities.
[0090]
On the other hand, if there is no matching operation item, for example, when the previous speech recognition result is “copy” and “hidari” is subsequently recognized by speech, the operation is performed in the hierarchy below the preceding function number = 52 (copy). Since there is no item “hidari”, “preceding function number” = 52 is maintained.
In this way, when a utterance is made with a pause, the previous state is maintained even if the utterance content is incorrect, so that the user can utter the correct content again with peace of mind.
[0091]
The menu control unit 350 manages the current position on the hierarchical menu by a menu database (DB) 360 that holds the hierarchical menu.
When the type of the input key passed from the operation input unit 320 is accompanied by movement on the menu, it is searched whether or not the input function number exists in the menu DB 360, and if it exists, the current function number is input. The function number is replaced, and the position of the operation item corresponding to the replaced function number is moved, and the sound output of the processing result is performed.
In addition, when “explaining the current position” or “confirmation of setting” without moving, voice feedback is performed in accordance with the function given to the key.
In the case of the “confirm” key, the setting item indicated by the current function number is confirmed and stored, and the confirmation is voice-feedback.
For example, when the function number = 631 and the “confirm” key is pressed to confirm the item, a voice feedback “Punch left is set to ON” is made and the setting is confirmed. . Here, if the punch left is already set to ON, the voice feedback “Punch left is set to OFF” is given.
[0092]
For example, when the upward movement key is pressed in the initial state (function number = 1), 1/10 = 0, so that it is not registered in the menu DB 360. In this case, the movement does not occur, and a sound is notified that the movement has failed.
Similarly, when the right movement key is pressed, 1 + 1 = 2, which is registered in the menu DB 360, so that the corresponding item (“original”) is moved to and there is a voice feedback “original”.
[0093]
When the function number is input by direct jump or voice recognition, if the function number exists in the menu DB 360, for example, in the case of “631”, there is a voice feedback of “punch left”.
[0094]
Further, when the left movement key is pressed when the function number is 21, 21-1 = 20, and since the corresponding function number is not registered in the menu DB 360, the movement does not occur and voice feedback indicates that the movement has failed. To do.
Even when a function number is input by direct jump or voice recognition, if the function number does not exist in the menu DB 360 (for example, “2 → 1 → 6 → #”), the movement does not occur and the movement fails. Give voice feedback.
[0095]
In the “voice mode”, the sound output unit 380 outputs operation sounds, voice guidance, BGM, and the like that indicate success or failure of various operations using the numeric keypad and functions in response to instructions from the menu control unit 350 and the operation control unit 310 to the headphones. . For voice, the voice synthesizer 420 is called to receive voice data and output it to the headphones. Alternatively, a recorded voice may be used instead of voice synthesis from text data by the voice synthesis unit 420.
[0096]
Function execution unit 390 stores setting values for various setting items passed from operation control unit 310 in the memory of MFP 100 determined according to the function (for example, copy function, facsimile function, etc.). Further, the MFP function is executed according to the setting value stored in the memory of the MFP 100 according to an instruction from the operation control unit 310.
[0097]
The voice recognition unit 410 is called by the voice input unit 370, and the user uses a voice recognition dictionary (for example, a copy dictionary) for a function (copy function or facsimile function) currently operated by the user. The result of voice recognition of the uttered voice (operation item name, function number, operation command, etc. of the hierarchical menu) is returned to the voice input unit 370.
[0098]
The voice synthesizing unit 420 is called from the sound output unit 380 and changes the pitch according to an instruction from the menu control unit 350 by using the voice synthesis dictionary when voice feedback is performed to the user. Or the sound transformed by adding a reverberation sound is returned to the sound output unit 380.
[0099]
Here, if the operation item instructed to the menu control unit 350 is invalid, the change in the pitch is output with a sound with a lowered pitch.
If the setting item is selected, the sound is output with a higher pitch.
The upper and lower pitches require a change that is distinguishable from the normal pitch, and it is necessary to keep the utterance content in a clearly audible range, so clarity is ensured in the normal natural speech range and speech synthesis. Considering the possible range, give a change of about a half octave.
[0100]
The reverberation sound is added when there is an operation item below the operation item instructed from the menu control unit 350.
For example, it can be realized by adding the audio to the input after passing through the delay device and the attenuator. In this case, the control parameters are the delay time of the delay unit and the attenuation amount of the attenuator, and are experimentally determined in consideration of improving the discrimination from normal speech and ensuring the clarity of the speech.
[0101]
Note that sound field control is performed by pitch control and reverberation sound addition among the sound attributes, but when reading operation items, the sound field feeling should be given by changing the speech speed, voice type, sound quality, etc. It may be.
Speaking speed control can be achieved by waveform expansion / contraction in recording synthesis, and phoneme duration length expansion / contraction in regular synthesis.
In addition, the voice type can be changed by changing the recording speaker in recording synthesis and by changing the segment waveform in rule synthesis.
The sound quality can be controlled by inserting a filter after the output of the speech synthesizer and controlling the cut-off frequency.
[0102]
F. MFP operation
The operation of the MFP configured as described above will be described by taking the copy function operation as an example.
FIG. 8 is a flowchart showing the operation for the copy function operation.
First, when the mode switching key is pressed (YES in step S1), the current setting mode is switched (step S2), and the process returns to step S1 to process the next operation instruction.
When switching from “visual mode” to “audio mode”, a mode switch key must be pressed to connect a headset including headphones and a microphone to the MFP. Further, when the mode is switched to “voice mode”, the input from the touch panel is locked, and a voice notification of “entering voice mode” is given from the headphones.
On the other hand, if the current setting mode is “audio mode”, the mode is switched to “visual mode”. If the headset remains connected, this “visual mode” is temporary, and the input operated from the touch panel is output to the headphones. Only when the headset is disconnected from the MFP, it switches to the “visual mode”.
[0103]
When the current mode is the “visual mode” (YES in step S3), the MFP is operated by the visual input unit 330 using the touch panel 201 (step S4), and the process returns to step S1 to process the next operation instruction. .
[0104]
When the current mode is “voice mode”, the following processing is performed.
When a function key (FAX, copy, printer, scanner, net file, etc.) 206 is pressed (YES in step S5), a hierarchical menu corresponding to the pressed function is taken out from the menu DB 360 and initialized (step S6). ), The process returns to step S1 to process the next operation instruction.
Here, assuming that the function key “Copy” (see FIG. 2) is selected, the user is informed by voice that “copy operation is now possible” from the headphones.
[0105]
Next, when the direction movement key (up / down / left / right) is pressed (YES in step S7), the step moves up and down or moves to an adjacent item in the same layer (step S8), and the step for processing the next operation instruction is performed. Return to S1.
The operation input unit 320 manages the location of the hierarchical menu by passing the input key type and the input function number to the menu control unit 350.
[0106]
Further, when moving on the hierarchical menu, the menu control unit 350 calls the sound output unit 380 to play a sound icon, an operation sound and BGM, and further outputs the operation item name as a sound so that the user can operate the sound reliably. Notify that
The voice at this time is a voice obtained by synthesizing the item name by the voice synthesizer 420. If the operation item is invalid, the pitch is lowered, and if the setting item is selected, the pitch is raised and read. . Furthermore, if the operation item has an operation item in the lower layer, a reverberation sound is added and read out.
[0107]
If the menu control unit 350 reaches the end of the hierarchical menu and cannot move any more, the menu control unit 350 calls the sound output unit 380 to output a “dead end” operation sound to notify the user. When a dead end is reached in the vertical direction on the hierarchical menu, a “dead end” operation sound is output to notify the user.
[0108]
Next, when the “return to top (“ 1 ”)” key is pressed (YES in step S9), the operation item in the leftmost menu of the highest hierarchy is moved (step S10), and the next operation instruction is issued. Return to step S1 for processing.
The menu control unit 350 manages the current position so as to be the operation item in the leftmost menu of the highest hierarchy. At the time of the movement, the menu control unit 350 calls the sound output unit 380, outputs a sound icon and the sound of the operation item name of the movement destination, and notifies the user that the operation has been performed correctly.
The voice at this time is synthesized by the voice synthesizer 420 by synthesizing the item name. If this operation item is invalid, the pitch is lowered, and if the setting item is selected, the pitch is raised and read. Furthermore, if the operation item has an operation item in the lower layer, a reverberation sound is added and read out.
[0109]
Next, when the “direct jump (“ 0 ”)” key is pressed (YES in step S11), the process moves to a setting item corresponding to the subsequently input function number (step S12), and the next operation instruction is processed. Therefore, the process returns to step S1.
The operation input unit 320 calls the sound output unit 380 and outputs an operation sound, a voice “Please input a function number”, and BGM. While this BGM is playing, the user inputs a number from the numeric keypad. The input number is read out via the sound output unit 380.
Finally, when the user presses the “numerical value input end (“ # ”)” key, the operation input unit 320 stops playing the BGM, and the menu control unit 350 sets the operation sound and the function number just input. The name of the operation item is output by voice, and the operation item is managed as the current position.
[0110]
When the item corresponding to the specified function number is a hierarchical name, the hierarchical name is synthesized by the speech synthesizer 420. If the hierarchical name has an operation item in the lower layer, a reverberant sound is added. If the designated function number is a setting item, the pitch is lowered if the setting item is invalid, and the pitch is increased if the setting item is selected, and the setting item is read out.
For example, following the depression of the key “0”, if “435” is input with the numeric keypad, a high-pitched voice feedback “the scaling is set to 71% reduction” is made.
When the setting item having the lowest function number is specified, the menu control unit 350 holds the value corresponding to the setting item as the setting value of the setting item on the menu DB 360, and the operation control unit 310 executes the function execution. The setting value now specified via the unit 390 is stored in the corresponding setting item in the memory of the MFP 100, and the visual display unit 340 is called to display the determined content on the screen of the operation unit 104.
[0111]
Next, when the “voice recognition (“ 3 ”)” key is pressed (YES in step S13), the input voice is voice-recognized and moved to the operation item corresponding to the recognition result (step S14). The process returns to step S1 to process the next operation instruction.
The operation input unit 320 calls the sound output unit 380 and outputs a voice saying “Please say function number, operation item name or keyword” together with the operation sound.
[0112]
Subsequently, the operation input unit 320 calls the voice input unit 370 and obtains the function number designated by the user from the “function number”, “operation item name”, or “keyword” uttered by the user. At this time, since the voice input is not confirmed until the “determine (9)” key is pressed, the previous input can be corrected by speaking again.
In the case of a long operation item name, even when a pose (pause) is inserted and uttered, the front and back of the pause are connected and recognized as one operation item.
If the recognition result is “numerical value”, the voice input unit 370 sets it as “function number”, and when the operation item name is input, the operation item name and the function number are associated with each other and stored in the memory. The correspondence table or menu DB 360 is searched to convert the operation item name into a function number.
[0113]
The function number obtained by the voice input unit 370 is passed to the menu control unit 350, and the operation item for this function number is managed as the current position.
At this time, the menu control unit 350 calls the sound output unit 380, outputs the sound icon and the input operation item name by voice, and notifies the user that the voice input has been correctly recognized.
[0114]
  When the item name or function number specified by the user is a hierarchical name, the hierarchical name is synthesized by the speech synthesizer 420. If the hierarchical name has an operation item in the lower layer, a reverberant sound is added. Also, if the specified function number is a setting item, if this setting item is disabled, the pitch is lowered, and if that setting item is selected, the pitch is raised and the setting item is read out.The
[0115]
Next, when the “description of current position (“ 5 ”)” key is pressed (YES in step S15), information indicating which operation item is currently in the hierarchical menu is provided (step S16), and the next operation is performed. Return to step S1 to process the instructions.
The menu control unit 350 calls the sound output unit 380 and notifies the user of the sound icon, the operation item name at the current position, the setting value, and the like by voice.
[0116]
When the current position is a hierarchy name, the hierarchy name is synthesized by the speech synthesizer 420. If the hierarchy name has an operation item in the lower layer, a reverberation sound is added. If the current position is a setting item, the pitch is lowered if the setting item is invalid, and if the setting item is selected, the pitch is increased and the setting item is read out.
[0117]
Next, when the “confirm setting” key is pressed (YES in step S17), the setting item name and setting value are notified by voice of what setting item the user has selected so far (step). S18) Return to step S1 to process the next operation instruction.
The menu control unit 350 sequentially moves from the default value to the setting item whose setting has been changed, calls the sound output unit 380, reads out the setting item name and its setting value, and notifies the user. At this time, the menu control unit 350 manages the movement destination as the current position, and can correct the setting item if there is an error.
Further, when the setting item is changed, the menu control unit 350 holds the changed setting item and the setting value as the setting value of the setting item on the menu DB 360, and the operation control unit 310 includes the function execution unit. The setting value of the setting item now specified via 390 is stored in the corresponding setting item in the memory of MFP 100, and the visual display unit 340 is called to display the determined content on the screen of the operation unit 104.
[0118]
Next, when the “confirmation (“ 9 ”)” key is pressed (YES in step S19), the selection of the setting item of the current position is confirmed (step S20), and the process proceeds to step S1 to process the next operation instruction. Return.
The menu control unit 350 calls the sound output unit 380, outputs the operation sound and the determined operation content by voice, and notifies the user.
Further, when the setting item of the current position is confirmed, the menu control unit 350 holds the confirmed setting item and its setting value as the setting value of the setting item on the menu DB 360. The operation control unit 310 The setting value of the setting item that has been confirmed through the function execution unit 390 is stored in the corresponding setting item in the memory of the MFP 100, and the visual display unit 340 is called to display the determined content on the screen of the operation unit 104. .
[0119]
Finally, when the setting for each setting item is completed and the “start” key is pressed (YES in step S21), the selected function (copy function, facsimile function, etc.) is executed (step S22), and the next Return to step S1 to process the operation instruction.
The operation control unit 310 calls the function execution unit 390 to operate the function selected by the user first. For example, if the designated function is a copy function, a copy operation according to the values set so far is performed. When this operation ends, the operation control unit 310 calls the sound output unit 380 and informs the user of the success or failure by voice.
[0120]
For example, a result such as “copying has been completed at 122% magnification” or “copying failed due to paper jam” is fed back by voice.
In addition, when the user designates a copy of “40 copies”, the operation control unit 310 plays the BGM through the sound output unit 380 so that it can recognize how much copy has been completed. The user can know how much processing has been completed according to the progress of the BGM.
[0121]
Further, the operation control unit 310 monitors the state of the apparatus during the operation of the MFP. For example, in the case of “out of paper”, “insufficient toner”, “paper jam”, “front cover is open”, etc. , Call the sound output unit 380, and after the sound icon, for example, “paper is out”, “toner is low”, “paper jam has occurred”, “front cover is open” Etc. and the BGM are output and notified to the user.
[0122]
G. Embodiment as a program
The present invention is not limited to the embodiment described above. Each function constituting the above-described embodiment is programmed, written in advance on a recording medium such as a CD-ROM, and the CD-ROM or the like is mounted on a medium driving device such as a CD-ROM drive mounted on a computer. Needless to say, the object of the present invention can also be achieved by storing these programs in a memory or storage device of a computer and executing them.
In this case, the program itself read from the recording medium realizes the functions of the above-described embodiment, and the program and the recording medium on which the program is recorded also constitute the present invention.
[0123]
As a recording medium, a semiconductor medium (for example, ROM, nonvolatile memory card, etc.), an optical medium (for example, DVD, MO, MD, CD-R, etc.), a magnetic medium (for example, magnetic tape, flexible disk, etc.) Either may be sufficient.
[0124]
Moreover, not only the functions of the above-described embodiment are realized by executing the loaded program, but the operating system or the like performs part or all of the actual processing based on the instructions of the program, and the processing described above The case where the functions of the above-described embodiments are realized is also included.
[0125]
In addition, a program for realizing the functions of the above-described embodiment is loaded into a memory provided in the function expansion board or the function expansion unit, and the CPU provided in the function expansion board or the function expansion unit is actually installed based on the instructions of the program. A case where part or all of the processing is performed and the functions of the above-described embodiments are realized by the processing is also included.
[0126]
Further, when the above-described program is stored in a storage device such as a magnetic disk of a server computer and distributed in the form of download from a user computer connected via a communication network, the storage device of this server computer is also included in the present invention. Included in the recording medium.
[0127]
【The invention's effect】
As described above, according to the present invention, when the user operates the device, it is possible to remove the anxiety of erroneous setting and use the voice recognition any number of times without worry. Various operations can be performed easily and reliably.
Also, by correctly recognizing pauses inserted between spoken words for voice recognition, users can use voice recognition with peace of mind, making it easy to perform various operations without using vision. And it can be performed reliably.
[Brief description of the drawings]
FIG. 1 is a block diagram of an MFP incorporating an apparatus operating device according to the present invention.
FIG. 2 is a front view of an operation unit.
FIG. 3 is a configuration example of a hierarchical menu related to a copy function.
FIG. 4 is a configuration example of a hierarchical menu related to a copy function (continuation of FIG. 3).
FIG. 5 is a functional layout example of a numeric keypad.
FIG. 6 is a diagram for explaining operation by a direction movement key.
FIG. 7 is a functional configuration diagram of a device operating device of the present invention incorporated in an MFP.
FIG. 8 is a flowchart showing an operation for a copy function operation;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... MFP, 101 ... Host connection device, 102 ... FAX line connection device, 103 ... Network connection device, 104 ... Operation unit, 105 ... Scanner device, 106 ... Printer engine, 107 ... Control controller, 108 ... Storage device, 109 DESCRIPTION OF SYMBOLS ... Sound output device, 110 ... Voice input device, 201 ... Touch panel and liquid crystal display unit, 202 ... Numeric keypad, 203 ... Stop key, 204 ... Start key, 205 ... Setting confirmation key, 206 ... Function key, 310 ... Operation control unit, 320 ... Operation input unit, 330 ... Visual input unit, 340 ... Visual display unit, 350 ... Menu control unit, 360 ... Menu DB, 370 ... Audio input unit, 380 ... Sound output unit, 390 ... Function execution unit, 410 ... Audio Recognizing unit, 420... Speech synthesis unit.

Claims (8)

Classify all operation items that can be operated on the device by function, subdivide the classification into final setting items, create a hierarchical menu, and select or set the setting item in the hierarchical menu. In a device operating device with voice guidance for operating a device, a voice input unit for inputting designation of the setting item by voice from a user, a voice recognition unit for outputting a recognition word obtained by voice recognition of the input voice, A menu control unit that manages the setting item specified by the voice input unit as a position on the hierarchical menu, and a setting that is managed by the menu control unit and that provides feedback to the setting item changed by the user from a default value device operation and wherein the obtaining Preparations and confirmation means. The apparatus operating device according to claim 1, further comprising a determining unit operated by a user, wherein the voice input unit is a setting item corresponding to a recognized word that is voice-recognized by the voice recognition unit. When the confirmation operation is performed by the confirmation unit, the setting item corresponding to the last recognized word is designated, and the menu control unit is designated by the voice input unit. A device operating device, characterized in that setting items are described by voice information. 3. The device operating device according to claim 2 , wherein the voice input unit is a setting item corresponding to a recognized word that is voice-recognized by the voice recognition unit while the user does not perform a confirming operation. device operation and wherein the continue to Rukoto to voice fed back. In device operation apparatus according to claim 3, wherein the voice input unit that exists in the lower layer of the setting items recognized previously, characterized in that voice feedback setting items corresponding to the recognized word Equipment operation device. In device operation apparatus according to claim 3, wherein the voice input unit, unless present in the lower layer of the setting items setting items corresponding to the recognized word is recognized previously set for the previous recognition word entries An apparatus operating device characterized by maintaining The program for functioning a computer as an apparatus operating device as described in any one of Claims 1 thru | or 5 . The computer-readable recording medium which recorded the apparatus operation program of Claim 6 . Using an instrument operating device according to any one of claims 1 to 5, and selects and determines a setting item for the function image forming apparatus, which comprises an image formed from the image forming apparatus.

Images