JP2021009243A - Image forming device and program for controlling it - Google Patents

Abstract

To suppress a decrease in convenience when an invalid function is instructed by an utterance for an image forming device having a voice recognition function.SOLUTION: An image forming apparatus 100 includes: a voice recognition unit 310 which executes voice recognition processing for a given utterance; a storage unit 320 which holds an operation mode table, a function table, an option table, a panel display table, and the like; a function determination unit 330 which determines a function which a user desires to operate from an utterance, based on a recognized voice result and data stored in the storage unit 320; a valid/invalid determination unit 340 which determines whether the function is valid or invalid; an option control unit 350 which controls the function as one or more options for causing the image forming apparatus 100 to execute a predetermined operation according to the result of a determination; an instruction execution unit 360 which executes a function given by a voice; and a display unit 370 which displays a new option or message according to a result of voice recognition.SELECTED DRAWING: Figure 3

Description

The present disclosure relates to an image forming apparatus, and more specifically to an image forming apparatus having a voice input function.

Speech recognition technology has become more common than the conventional limited world technology due to the speeding up and cost reduction of processors such as CPUs (Central Processing Units) or memory and other hardware, and the spread of AI (Artificial Intelligence) technology. There is. As a result, user operations using voice recognition are rapidly becoming widespread. As for multifunction devices, products that enable voice operation by attaching a voice operation device to the main body of the multifunction device or by linking with an external smart speaker are on the market.

In the voice operation, an interactive interface is adopted for the purpose of compensating for the uncertainty of voice recognition and collecting the settings necessary for the multifunction device to execute the job from the user. During voice operation processing, in order to prevent unintended operations due to erroneous operation of the panel or hard keys, a voice input dedicated screen is displayed on the panel screen, and input from user interfaces other than the panel is temporarily disabled. Will be done.

When the user can instruct the multifunction device with clear job contents, the voice operation switches the panel screen according to the hierarchy, compared with the conventional method of inputting settings to the multifunction device while switching the panel screens one by one. You can read the settings in order without taking any time. Therefore, the user can complete the operation in the shortest time.

If the user forgets the settings, the user cannot narrow down the settings to one, or the user cannot set the settings in the current state of the multifunction device, the multifunction device reads out all the options that can be selected in the interactive interface. , The user can temporarily memorize the information and set it again by voice to make the setting.

Regarding voice input to a multifunction device, for example, Japanese Patent Application Laid-Open No. 2007-58432 (Patent Document 1) discloses a technique for "improving operability by voice input in an image forming apparatus having a voice input / output function". According to the image forming apparatus disclosed in Patent Document 1, "In the voice input mode, it is determined whether or not the operation can be started with respect to the input voice, and if the start is not possible, the missing item is guided by the voice output. By doing so, the operator is made to input voice "(see [Summary]).

Further, Japanese Patent Application Laid-Open No. 2007-102011 (Patent Document 2) states that "the operation is switched depending on the result of voice recognition, and there is a word that matches the dictionary, but if the function is not implemented," the function is implemented. We will disclose an image forming device that provides voice guidance such as "Not available" and presents alternative functions on the UI (see [Summary]).

JP-A-2007-58432 JP-A-2007-102011

In multifunction devices that have become multifunctional and complicated, there are options for setting similar names and options for setting using words that are not familiar to us. In the interactive interface adopted for voice operation, the user needs to temporarily memorize all the options read by the multifunction device until the user gives a voice instruction again. According to the 2016 Japan Ergonomics Society's paper "Relationship between the number of transaction items in voice ATMs and memorability", the number of items that many people can memorize in the short term without mistakes. , If it is an irrelevant item, it is described as "3". It is stated that if there are four or more items, some people will not be able to remember them, or some of the items will be memorized incorrectly, and eventually the correct selection will not be possible.

For example, there are six items for setting the image quality of the manuscript of the multifunction device: text, character photograph, photograph, light text manuscript, copy manuscript, and map, and a general user reads out the options from the multifunction device by the voice guidance function. Even so, there is a problem that the options once memorized overflow from the short-term storage area and the voice guidance is re-listened many times, which significantly reduces the advantage of the voice operation. Therefore, it is necessary to improve the operability by voice input.

The present disclosure has been made to solve the above-mentioned problems, and according to a certain embodiment, a technique for improving operability when operating an image forming apparatus having a voice recognition function by utterance is provided. Will be disclosed.

[Summary of disclosure]
An image forming apparatus according to a certain embodiment includes a control unit that executes each of a plurality of functions in response to a command to the image forming apparatus, and a voice recognition processing unit that recognizes an utterance to the image forming apparatus and outputs a recognition result. A display unit that displays an image, a function determination unit for determining which of the plurality of functions the utterance specifies based on the recognition result, and the determined function. When the option discriminating unit for determining whether the option is a valid option specified as an option for the image forming apparatus or an invalid option not specified as the option, and the determined function is a valid option. It also includes a function setting unit that accepts the setting of the determined function, and a display control unit that displays a valid option as the setting on the display unit when the determined function is an invalid option.

An image forming apparatus according to a certain aspect further includes a dividing unit for dividing the utterance into a plurality of text instructions by identifying the silent section of the utterance. The function discriminating unit determines which function is selected based on the plurality of text instructions. The option determination unit determines whether the determined function is a valid option or an invalid option based on the plurality of text instructions.

In an image forming apparatus according to a certain aspect, the division unit divides the plurality of text instructions into a plurality of words. When the last word acquired by the division is a signed numerical value, the function discriminating unit determines that the function instructed by the utterance is a function related to the concentration setting.

In an image forming apparatus according to a certain aspect, the division unit divides the plurality of text instructions into a plurality of words. When the last word acquired by the division is an unsigned numerical value, the function discriminating unit determines that the function instructed by the utterance is a function related to the setting of the number of copies.

An image forming apparatus according to a certain aspect further includes a memory for storing a function table in which each of the plurality of functions and a keyword are associated with each other. To determine whether any of the above functions is selected, the above function table is searched based on each of the above text instructions, and a keyword matching any of the above text instructions exists in the above function table. In the case, it includes acquiring the function instructed by the above utterance.

In an image forming apparatus that follows a certain aspect, acquiring the above functions means searching the above function table based on the text instructions using all of the above text instructions as keywords, and the keywords matching the text instructions are described above. Includes acquiring the function indicated by the above utterance when present in the function table.

In an image forming apparatus according to a certain aspect, acquiring the above function means dividing the text instruction into a plurality of words, searching the function table using the last word obtained by the division, and the above. This includes acquiring the function when one or more keywords matching the last word are present in the function table.

An image forming apparatus according to a certain aspect further includes an acquisition unit for acquiring an operation mode from each of the above text instructions. The function discrimination unit switches the function discrimination according to the acquired operation mode.

An image forming apparatus according to an aspect further includes a choice table that defines feasible choices for each of the above functions. Determining whether the determined function is a valid option or an invalid option is to search the option table based on the text instruction and the text instruction matches the executable option. This includes determining whether the determined function is a valid option or an invalid option based on whether or not the function is performed.

In an image forming apparatus according to a certain aspect, it is possible to determine whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches the executable option. , Including making the determined option an invalid option when the text instruction does not match the actionable option for each function.

In an image forming apparatus according to a certain aspect, it is possible to determine whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches the executable option. , When the text instruction does not specify any of the feasible options for each of the functions, the above-determined option is determined to be an invalid option.

In an image forming apparatus according to a certain aspect, it is possible to determine whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches the feasible option. When the text instruction matches a plurality of executable options for each function, the determined option is determined to be an invalid option, and one or more options valid for the function are displayed on the display unit. Displaying, accepting a new utterance to the image forming apparatus, determining whether or not an option matching the one or more options is included in the utterance, and one or more of the above options This includes determining that the matching option is a valid option when the matching option is included in the utterance.

An image forming apparatus according to an aspect further includes a counting unit for counting the types of numbers assigned to the valid options. Displaying an effective option for the function on the display unit includes displaying the effective option on the display unit when the number of the above types is 4 or more.

In an image forming apparatus according to a certain aspect, displaying the valid options on the display unit includes assigning the numbers of the valid options as different numbers for each function and option.

In an image forming apparatus according to a certain aspect, displaying the valid options on the display unit includes assigning the same number to a plurality of options when the valid options are numerical values.

According to still other embodiments, a program for controlling the image forming apparatus is provided. The program displays an image on the image forming apparatus, a step of executing each of a plurality of functions in response to a command to the image forming apparatus, a step of recognizing an utterance to the image forming apparatus and outputting a recognition result, and an image. Steps to be performed, a step to determine which of the plurality of functions the utterance specifies based on the recognition result, and the determined function are defined as options for the image forming apparatus. A step of determining whether the option is a valid option or an invalid option that is not defined as the option, and a step of accepting the setting of the determined function when the determined function is a valid option. And, when the determined function is an invalid option, the step of displaying the valid option as the setting on the image forming apparatus is executed.

According to a certain embodiment, the deterioration of the operability due to the sound of the image forming apparatus can be suppressed.
The above and other objectives, features, aspects and advantages of the invention will become apparent from the following detailed description of the invention as understood in connection with the accompanying drawings.

It is a figure which shows the appearance of the image forming apparatus 100. It is a block diagram which shows the structure of the image forming apparatus 100 and the interactive UI (User Interface) device 240. It is a block diagram which shows the functional structure of the image forming apparatus 100 according to a certain embodiment. It is a flowchart which shows a part of the processing executed by the image forming apparatus 100. It is a figure which shows an example of voice data processing received from a user. It is a figure which shows an example of the operation mode table 231 held by the image forming apparatus 100. It is a figure which shows typically an example of the operation mode determination processing of the voice data received from a user. It is a flowchart (the 1) which shows an example of the option discriminating process executed by CPU 221. It is a flowchart (No. 2) which shows an example of the option discriminating process executed by CPU 221. This is an example of the function table 232. It is a figure which conceptually shows an example of the function search process of the voice data received from a user. It is a figure which shows an example of the option table 233. It is a figure which shows typically an example of the choice search process of voice data received from a user. This is the content of the panel display table 234 corresponding to the function "Original image quality / Original setting" of the voice B. This is the content of the panel display table 234 corresponding to the function "paper" of the voice B. This is the content of the panel display table 234 corresponding to the function "number of copies" of the voice B. This is the content of the panel display table 234 corresponding to the function "resolution" of the voice C. This is the content of the panel display table 234 corresponding to the voice C function “save destination”. It is a figure which shows typically an example of the guidance processing of the voice data received from a user. This is an example of a panel display at the time of guidance of the function “original image quality / original setting” when the voice B is given to the image forming apparatus 100 as an operation command. This is an example of the panel display at the time of guidance of the function “save destination” when the voice C is given to the image forming apparatus 100 as an operation command.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are designated by the same reference numerals. Their names and functions are the same. Therefore, the detailed description of them will not be repeated.

The image forming apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing the appearance of the image forming apparatus 100. In a certain aspect, the image forming apparatus 100 is realized as a so-called MFP (Multi Function Peripheral). The MFP has a scanner function, a copy function, a facsimile function, a network function, a box function, and other functions. The image forming apparatus 100 includes an operation panel apparatus 101, a scanner unit 102, a feeder 104, a printer engine 106, a tray 108, and a plurality of paper feeding units 109.

The operation panel device 101 is arranged on the front side of the upper surface of the main body of the image forming device 100 (that is, the side facing the image forming device 100 for operation by the user), and receives an operation on the image forming device 100. As will be described later, the operation panel device 101 includes a plurality of keys for receiving various instructions from the user or input operations such as numbers, characters, symbols, and the like, and a display device. The display device includes a touch panel. The operation panel device 101 displays to the user various information in response to the user operation and / or a menu screen for receiving various operations. In addition, the operation panel device 101 detects the position touch-operated by the user and acquires the input information according to the acquired position.

In a certain aspect, the operation panel device 101 equipped with the touch panel type input device is tilted so that the operation surface is slanted so that the user can easily operate the image forming device 100 while standing in front of the image forming device 100. It is attached to the main body of the image forming apparatus 100. In other aspects, the operation panel device 101 may be removable.

The scanner unit 102 optically reads the document to acquire image data. The feeder 104 is arranged on the upper surface of the main body of the image forming apparatus 100, and sends a document to the scanner unit 102.

The printer engine 106 prints an image on paper or other media based on the acquired image data. The tray 108 is arranged in the central portion of the image forming apparatus 100, and receives the paper which is image-formed by the printer engine 106 and ejected. The plurality of paper feeding units 109 are arranged in the lower part of the main body of the image forming apparatus 100, and supply paper to the printer engine 106.

An example of a detailed configuration for realizing the image forming apparatus 100 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of an image forming apparatus 100 and an interactive UI (User Interface) device 240.

As shown in FIG. 2, the image forming apparatus 100 includes a system controller 201, a memory 207, a microphone 202, a speaker 211, a voice recognition unit 203, a network I / F 204, a printer engine 210, and an output image. It includes a processing unit 209, a storage device 208, an imaging unit 205, an input image processing unit 206, and an operation panel 230. A memory 207, a voice recognition unit 203, a network I / F 204, a printer engine 210, an output image processing unit 209, a storage device 208, an imaging unit 205, an input image processing unit 206, and an operation panel 230 are connected to the system controller 201. Has been done.

The system controller 201 controls the overall operation of the image forming apparatus 100 for various jobs such as a scan job, a copy job, an email transmission job, and a print job. The system controller 201 includes a CPU (Central Processing Unit) 221 and a ROM (Read Only Memory) 222. The CPU 221 executes the control program stored in the ROM 222. The ROM 222 stores various programs for causing the image forming apparatus 100 to execute a predetermined operation, and various fixed data prepared in advance by the manufacturer of the image forming apparatus 100. The system controller 201 reads data from the memory 207 or writes data to the memory 207 by executing a predetermined instruction. The memory 207 is a RAM (Random Access Memory), and temporarily stores data and image data required when the CPU 221 executes a control program.

The memory 207 is used as a working memory of the CPU 221 and holds an operation mode table 231, a function table 232, an option table 233, and a panel display table 234.

In the operation mode table 231, search keywords for searching the copy operation, the scan operation, and other operation modes are preset. In the function table 232, search keywords for searching paper, density, and other functions are preset. In the option table 233, options for each setting screen are preset. The panel display table 234 is extracted from the option table 233 and temporarily stores the options to be presented to the user as guidance.

The network I / F 204 communicates with an external device via the network according to the instruction from the system controller 201. The printer engine 210 performs print processing on paper and other print media based on the print data processed by the output image processing unit 209. For example, in a certain situation, when the image forming apparatus 100 operates as a printer, the printer engine 210 prints an image. In another aspect, when the image forming apparatus 100 operates as a copying machine, the printer engine 210 prints the image read by the imaging unit 205.

The output image processing unit 209 performs conversion processing for converting the format of the image data into print data when printing an image or the like. The storage device 208 is realized by, for example, an HDD (Hard Disk Drive), a flash memory, or the like. The storage device 208 non-volatilely stores various data and the like related to the operation of the image forming device 100. Further, the storage device 208 holds the image data of the screen to be displayed on the operation panel 230 of the image forming device 100. The image pickup unit 205 reads the image of the original and outputs the image data in a predetermined format. The input image processing unit 206 sets the format of the image data output in response to the reading of the image by the imaging unit 205 to TIFF (Tagged Image File Format), JPEG (Joint Photographic Experts Group), PDF (Portable Document Format) and others. Convert to the specified format.

The interactive UI device 240 is connected to the network I / F 204 of the image forming apparatus via the network. The voice recognition unit 203 converts the voice data transmitted from the interactive UI device 240 via the network or the voice data converted by the microphone 202 into text data by voice recognition. The speaker 211 outputs voice data synthesized from text by the system controller 201 to the outside of the image forming apparatus 100. In a certain aspect, the interactive UI device 240 can be realized as a smartphone, a tablet terminal or other information processing terminal having a voice recognition function and a display function.

[Functional configuration]
A functional configuration for realizing the image forming apparatus 100 will be described with reference to FIG. FIG. 3 is a block diagram showing a functional configuration of the image forming apparatus 100 according to a certain embodiment. The image forming apparatus 100 includes a voice recognition unit 310, a storage unit 320, a function determination unit 330, a valid / invalid determination unit 340, an option control unit 350, a command execution unit 360, and a display unit 370.

The voice recognition unit 310 voice-recognizes the utterance to the image forming apparatus 100. Speech recognition is realized using known techniques. The voice recognition unit 310 is realized by a dedicated circuit element that executes voice recognition processing, or by the processor executing a voice recognition program.

The storage unit 320 stores a program and data for causing the image forming apparatus 100 to execute a predetermined process. For example, the storage unit 320 holds an operation mode table 231, a function table 232, an option table 233, a panel display table 234, and the like. In another aspect, the storage unit 320 may store the data generated by the image forming apparatus 100. The storage unit 320 is realized by a hard disk, a flash memory, or the like.

The function discriminating unit 330 desires the user to operate from the voice uttered to the image forming apparatus 100 based on the result of the voice recognized by the voice recognition unit 310 and the data stored in the storage unit 320. Determine the function to be performed. The function determination unit 330 may be realized by the CPU 221 in a certain aspect and may be realized by a circuit element configured to execute the process in another aspect.

The valid / invalid determination unit 340 determines whether the determined function option is valid or invalid. In the present embodiment, the option is valid when the image forming apparatus 100 is instructed to execute the function for the operation mode of the image forming apparatus 100. For example, when the operation mode is the copy mode and an instruction for instructing the density and the number of sheets is given to the image forming apparatus 100, the instruction is accepted by the image forming apparatus 100 as a valid instruction (option). Therefore, in this case, the image forming apparatus 100 operates in response to the command.

On the other hand, the invalid option means that when the image forming apparatus 100 is given an instruction to execute the function for the operation mode of the image forming apparatus 100, the option included in the instruction is invalid. means. For example, in the copy mode, a scanning command such as "scan the original and save it on the server" is given. Therefore, in this case, the image forming apparatus 100 does not operate in response to the command. The valid / invalid determination unit 340 may be realized by the CPU 221 in one aspect and may be realized by a circuit element configured to execute the process in another aspect.

The option control unit 350 controls the function as one or more options for causing the image forming apparatus 100 to execute a predetermined operation according to the result of the determination by the valid / invalid determination unit 340. More specifically, the option control unit 350 determines whether to accept or reject the function given by voice to the image forming apparatus 100 according to the operation mode of the image forming apparatus 100. When the option control unit 350 determines that the function given by voice is valid and should be accepted, the option control unit 350 transmits the result to the instruction execution unit 360. If not, the option control unit 350 transmits the result to the display unit 370. The option control unit 350 may be realized by the CPU 221 in one aspect and may be realized by a circuit element configured to perform the process in another aspect.

The instruction execution unit 360 executes a function given by voice based on the result sent from the option control unit 350. In another aspect, the instruction execution unit 360 may also execute related functions associated with the functions. In this case, the instruction execution unit 360 may ask a question about executing the related function. The instruction execution unit 360 may be realized by the CPU 221 in one aspect and may be realized by a circuit element configured to execute the process in another aspect.

In a certain aspect, the display unit 370 displays an operation screen that is accepted according to the operation mode of the image forming apparatus 100. For example, the display unit 370 may display a function that can be accepted as an effective function. In this case, the user can give a voice command by speaking the function.

In another aspect, the display unit 370 may display a message in response to the output of the option control unit 350. For example, when it is determined that the command given by voice to the image forming apparatus 100 is valid, the display unit 370 may display to that effect. In another aspect, if the command is determined to be invalid, the display unit 370 may display to that effect and further display a message prompting the user to speak another command.

In another aspect, the image forming apparatus 100 may display the advice to the user and the response to the instruction given by the user on the display unit 370. The display unit 370 is realized by a liquid crystal monitor, an organic EL monitor, or the like. In another aspect, the display unit 370 may be detachable from the image forming apparatus 100. In still another aspect, the display unit 370 may be realized by a tablet terminal capable of wirelessly communicating with the image forming apparatus 100.

[Control structure]
The control structure of the image forming apparatus 100 will be described with reference to FIGS. 4 to 7. FIG. 4 is a flowchart showing a part of the processing executed by the image forming apparatus 100. FIG. 5 is a diagram showing an example of voice data processing received from the user. FIG. 6 is a diagram showing an example of the operation mode table 231 held by the image forming apparatus 100. The operation mode table 231 has items of a number, an operation mode, and a keyword, and is used to search the text instruction by the keyword to search the operation mode. FIG. 7 is a diagram schematically showing an example of an operation mode determination process of voice data received from a user.

In a certain aspect, the processing is realized by the CPU or other processor executing an instruction. In other aspects, the process can also be implemented as a combination of circuit elements and other hardware configured to perform a given operation.

In step S410, the CPU 221 sets the panel screen as a screen for accepting voice input (hereinafter, "voice input screen") based on the voice instruction uttered by the user. Further, the CPU 221 changes the setting of the panel screen so that operations other than voice cannot be performed.

In step S420, the CPU 221 sets the operation mode of the image forming apparatus 100 to a predetermined operation mode (for example, copy mode). In other aspects, an operation mode other than the copy mode (for example, scan mode, print mode, facsimile mode, etc.) may be set as the default operation mode.

In step S430, the CPU 221 converts the voice data into text using the result of voice recognition by the voice recognition unit 310, and groups the textualized character data as a plurality of text instructions. At this time, the CPU 221 may delete particles and honorific titles.

Here, referring to FIG. 5, the voice data (voice A, voice B, voice C) given by the user's utterance is converted into text-based voice data. For example, in the case of voice A, the text-converted voice data is "+2 A45 standard stamp is confidential start". Further, the text-converted voice data is divided into a plurality of text instructions with a silent section as a delimiter. For example, in the case of voice A, a plurality of text instructions are "+2, A4, 5, standard stamp is confidential, start".

With reference to FIG. 4 again, in step S440, the CPU 221 searches for the operation mode from each of the plurality of text instructions. For example, the CPU 221 searches for text instructions in numerical order using the keyword “copy” in the operation mode table 231 shown in FIG.

In step S450, the CPU 221 determines whether or not there is an item with a matching keyword. If the operation mode is specified, there will be items with matching keywords. If the operation mode is not specified, there is no matching item. When the CPU 221 determines that there is an item with a matching keyword (YES in step S450), the CPU 221 switches the control to step S460. If not (NO in step S450), the CPU 221 switches control to step S800.

In step S460, the CPU 221 discards (ignores) the text instruction and resets the operation mode. After that, the CPU 221 ends the operation mode setting.

In step S800, the CPU 221 executes the option determination process described later. The CPU 221 determines the options for each function from the next text instruction, and receives the voice emitted from the user.

In step S470, the CPU 221 determines whether or not there is an OK instruction in the voice emitted from the user. When the CPU 221 determines that the OK instruction has been given (YES in step S470), the CPU 221 ends the process (acceptance of the job). If not (NO in step S470), the CPU 221 returns control to step S430. The CPU 221 again determines the options for each function from the text instruction (step S430).

As an example of the operation, first, when the voice A is given, the operation mode of the image forming apparatus 100 is set to the copy mode as the default operation mode (step S420). After that, when the voice B is given, "copy" is searched from the text instruction, and the operation mode remains the copy mode. After that, when the voice C is given, the text-written voice data "scan" is searched. Since "scan" is registered in the operation mode table 231 as a keyword that defines the scan mode (FIG. 6), the operation mode of the image forming apparatus 100 is switched from the copy mode to the scan mode.

[Choice discrimination process]
The control structure of the image forming apparatus 100 will be described in more detail with reference to FIGS. 8 to 12. 8 and 9 are flowcharts showing an example of the option determination process executed by the CPU 221 respectively. In another aspect, the option determination process may be realized by a circuit element or other hardware configured to execute a part or all of the individual processes.

FIG. 10 shows an example of the function table 232. The function table 232 has items of numbers, classifications, functions, and keywords, and is used to search text instructions by keywords to search for functions. The function changes depending on the operation mode, and up to four keywords can be set for one function.

FIG. 11 is a diagram conceptually showing an example of the function search process of voice data received from the user. FIG. 12 is a diagram showing an example of the option table 233. The choice table 233 has numbers, functions, numbers and choice items and is used to search for text instructions by choice. When the choice is a numerical value, the range between the minimum value and the maximum value is specified. The number is a label that serves as a guideline for the correlation between the options. For example, a number in the 100s is assigned to each function and a unique number is assigned to each option. However, when consecutive numerical values are grouped, it is easy for the user to remember the numerical values in the short term. Therefore, in a certain situation, the resolution options are grouped by two and assigned numbers.

With reference to FIG. 8, in step S810, the CPU 221 searches the function table 232 (FIG. 10) for text instructions in numerical order.

In step S815, the CPU 221 determines whether or not an item matching the function table 232 exists. When the CPU 221 determines that a matching item exists (YES in step S815), the control is switched to step S855. If not (NO in step S815), the CPU 221 switches control to step S820.

In step S820, the CPU 221 divides the text instruction into words and sets the final word as a search keyword.

In step S825, the CPU 221 determines whether or not the search keyword is a numerical value. When the CPU 221 determines that the search keyword is a numerical value (YES in step S825), the CPU 221 switches the control to step S830. If not (NO in step S825), the CPU 221 switches control to step S845.

In step S830, the CPU 221 determines whether or not the search keyword is a signed numerical value. When the CPU 221 determines that the search keyword is a signed numerical value (YES in step S830), the CPU 221 switches the control to step S835. If not (NO in step S830), the CPU 221 switches control to step S840.

In step S835, the CPU 221 determines that the function is density. In this case, the functional options include, for example, a setting to darken or lighten the print. The setting for making the print darker and the setting for making the print lighter can be set according to a predetermined numerical value.

In step S840, the CPU 221 determines that the function is the number of copies, and adds a term indicating the number of copies (for example, "parts") to the text instruction.

In step S845, the CPU 221 searches the function table 232. For example, the CPU 221 searches the unit system, the instruction system, and the setting system in the order of the search keyword numbers.

In step S850, the CPU 221 determines whether or not there is an item that matches the function table 232. When the CPU 221 determines that there is a matching item (YES in step S8500), the CPU 221 switches the control to step S855. If not (NO in step S850), the CPU 221 switches control to step S910 (FIG. 9).

In step S855, the CPU 221 determines the function based on the function table 232 and deletes the keyword from the text instruction.

In step S860, the CPU 221 determines whether the text instruction includes a "start" or "start" instruction. When the CPU 221 determines that the text instruction includes a "start" or "start" instruction (YES in step S860), the CPU 221 switches the control to step S865. If not (NO in step S860), the CPU 221 returns control to step S810.

In step S865, the CPU 221 outputs a voice reading aloud the job content received by the image forming apparatus 100 from the speaker 211. After that, the CPU 221 ends the process.

With reference to FIG. 9, in step S910, the CPU 221 determines whether or not the function can be searched. When the CPU 221 determines that the function can be searched (YES in step S910), the control is switched to step S915. If not (NO in step S910), the CPU 221 switches control to step S970.

In step S915, the CPU 221 searches the option table 233 for options within the function from the instructions.

In step S920, the CPU 221 determines whether or not the options match only one function. When the CPU 221 determines that the options match one function (YES in step S920), the CPU 221 switches the control to step S860 (FIG. 8). In this case, since the set value is fixed, the CPU 221 determines whether or not the next text instruction includes the "start" or "start" instruction (step S860). If not (NO in step S920), the CPU 221 switches control to step S925.

In step S925, the CPU 221 determines whether the options match the plurality of functions or do not match at all. When the CPU 221 determines that the options do not match at all (YES in step S925), the CPU 221 switches the control to step S930. On the other hand, when the CPU 221 determines that the options match the plurality of functions (NO in step S925), the CPU 221 switches the control to step S935.

In step S930, the CPU 221 extracts all the options. For example, the CPU 221 copies all the functional choices to the panel display table 234.

In step S935, the CPU 221 copies only the options whose functions match to the panel display table 234.

In step S940, the CPU 221 measures the number of the panel display table 234 (the number of choice types).

In step S945, the CPU 221 determines whether or not the number (the number of choice types) of the panel display table 234 is 4 or more. When the CPU 221 determines whether or not the number (the number of types of choices) of the panel display table 234 is 4 or more (YES in step S945), the CPU 221 switches the control to step S950. If not (NO in step S945), the CPU switches control to step S955.

In step S950, the CPU 221 outputs a voice message "Please specify a number from the options displayed on the panel" from the speaker 211, and then displays the names and options of the functions included in the panel display table 234 on the panel. Display on the screen.

In step S955, the CPU 221 outputs a voice message "Please specify a number from the following options" from the speaker 211, and then continuously outputs the function names and options of the panel display table 234 by voice. ..

In step S960, the CPU 221 newly receives a text instruction from the user and narrows down the options to one. In other aspects, if the numerical specification is valid, the CPU 221 may replace the message "specify by number from choices" with the message "specify from range".

In step S965, the CPU 221 changes the panel screen to a voice input screen and discards the text instruction being processed. Control is transferred to step S860, and the CPU 221 determines whether the next text instruction includes a "start" or "start" instruction (step S860).

FIG. 13 is a diagram schematically showing an example of a voice data option search process received from a user. As shown in FIG. 13, as a result of the CPU 221 searching for the options based on the function, the items that do not match the only option are the voice B function "original image quality / original setting", the function "paper", and the function " The number of copies, the voice C function "resolution", and the function "save destination", and these functions require the user to specify options.

FIG. 14 shows the contents of the panel display table 234 corresponding to the function “Original image quality / Original setting” of the voice B. As shown in FIG. 14, since a plurality of text instructions match the options of the option table 233 of each setting screen, all the options are copied to the panel display table 234, and the contents of the panel display table 234 are the operation panel device 101. It is displayed on (operation panel 230).

FIG. 15 shows the contents of the panel display table 234 corresponding to the function “paper” of the voice B. As shown in FIG. 15, the text instruction does not match the choices in the choice table 233 because it contains no content. Therefore, all the options are copied to the panel display table 234, and the contents of the panel display table 234 are displayed on the operation panel device 101 (operation panel 230).

FIG. 16 shows the contents of the panel display table 234 corresponding to the function “number of copies” of the voice B. As shown in FIG. 16, the text instruction "105 copies" does not match the options in the option table 233. Therefore, all the options are copied to the panel display table 234, and the contents of the panel display table 234 are displayed on the operation panel device 101 (operation panel 230).

FIG. 17 shows the contents of the panel display table 234 corresponding to the function “resolution” of the voice C. As shown in FIG. 17, the text indication "100 dpi" does not match the choices in the choice table 233. Therefore, all the options are copied to the panel display table 234, and the contents of the panel display table 234 are displayed on the operation panel device 101 (operation panel 230).

FIG. 18 shows the contents of the panel display table 234 corresponding to the voice C function “save destination”. As shown in FIG. 18, the text instruction "Toyokawa" matches a plurality of options in the option table 233. Therefore, all the partially matching options are copied to the panel display table 234, and the contents of the panel display table 234 are displayed on the operation panel device 101 (operation panel 230).

FIG. 19 is a diagram schematically showing an example of guidance processing of voice data received from a user. As shown in FIG. 19, the number of choices for the voice B function "original image quality / original setting", the function "paper" and the function "number of copies", and the voice C function "resolution" and the function "save destination" are Each is 4/2/1/4/4, but the numbers (number of types) are 4/2/1/2/4, respectively. The re-entry guidance for the voice B function "paper" and the function "number of copies" and the voice C function "resolution" is voice only. The re-input guidance of the voice B function "original image quality / manuscript setting" and the voice C function "save destination" is used in combination with the voice and the panel screen.

[Display mode]
An example of displaying a message in the image forming apparatus 100 will be described with reference to FIGS. 20 and 21. FIG. 20 is an example of a panel display at the time of guidance of the function “original image quality / original setting” when the voice B is given to the image forming apparatus 100 as an operation command. FIG. 21 is an example of a panel display at the time of guidance of the function “save destination” when the voice C is given to the image forming apparatus 100 as an operation command.

As shown in FIG. 20, the operation panel 230 displays options for the functions “original image quality” and “image setting” identified from the voice B. At this time, the CPU 221 is in a state of accepting an input operation to the operation panel 230. Therefore, when the user touches the operation panel 230 or presses the corresponding number button, the CPU 221 accepts an instruction corresponding to the number.

As shown in FIG. 21, in another aspect, when the image forming apparatus 100 is in the scan mode, the operation panel 230 displays a storage destination that is a candidate for storing the scan data. Even when the image forming apparatus 100 is in the scan mode, when the panel at the time of guidance is displayed on the operation panel 230, the CPU 221 is in a state of accepting an input operation to the operation panel 230. Therefore, when the user touches the operation panel 230 or presses the corresponding number button, the CPU 221 accepts an instruction corresponding to the number.

Some of the technical features disclosed above can be summarized as follows.
(1) The image forming apparatus 100 according to a certain embodiment has a CPU 221 that executes each of a plurality of functions in response to a command to the image forming apparatus 100, and a voice that recognizes an utterance to the image forming apparatus 100 and outputs a recognition result. It includes a recognition unit 203 and an operation panel 230 for displaying an image. As a function discriminating unit, the CPU 221 determines which of the plurality of functions the utterance specifies based on the recognition result. The CPU 221 determines whether the determined function is a valid option defined as an option for the image forming apparatus 100 or an invalid option not defined as the option. The CPU 221 accepts the setting of the determined function as a function setting unit when the determined function is an effective option. Further, as a display control unit, the CPU 221 displays a valid option as a setting on the operation panel 230 when the determined function is an invalid option.

For example, when the operation mode of the image forming apparatus 100 is the copy mode, if an utterance such as "copying 10 copies of a document placed on a feeder on both sides" can be given to the interactive UI device 240, it is specified based on the utterance. The option of the function to be performed is determined to be a valid option. On the other hand, when the operation mode of the image forming apparatus 100 is the copy mode and an utterance such as "scan the original" or "print the original" is given to the image forming apparatus 100, the function specified based on the utterance. The option of is determined to be an invalid option.

(2) In the image forming apparatus 100 according to a certain aspect, the CPU 221 divides the utterance into a plurality of text instructions by identifying the silent section of the utterance as a dividing unit. The CPU 221 determines which function is selected based on a plurality of text instructions as the function determination unit. As an option determination unit, the CPU 221 determines whether the determined function is a valid option or an invalid option based on a plurality of text instructions.

(3) In the image forming apparatus 100 according to a certain aspect, the CPU 221 divides a plurality of text instructions into a plurality of words as a dividing unit. As a function discriminating unit, the CPU 221 determines that the function instructed by the utterance is a function related to the density setting when the last word acquired by the division is a signed numerical value.

(4) In the image forming apparatus 100 according to a certain aspect, the CPU 221 divides a plurality of text instructions into a plurality of words as a dividing unit. The CPU 221 determines that the function instructed by the utterance is the function related to the setting of the number of copies when the last word acquired by the division is an unsigned numerical value as the function discriminating unit.

(5) The image forming apparatus 100 according to a certain aspect further includes a memory 207 for storing a function table 232 in which each of the plurality of functions and a keyword are associated with each other. The CPU 221 searches the function table 232 based on each text instruction in order to determine which function is selected. Further, the CPU 221 acquires the function instructed by the utterance when a keyword matching any of the text instructions exists in the function table 232.

(6) In the image forming apparatus 100 according to a certain aspect, in order to acquire the function, the CPU 221 searches the function table 232 based on the text instruction using all of the text instructions as keywords. The CPU 221 acquires the function instructed by the utterance when the keyword matching the text instruction exists in the function table 232.

(7) In the image forming apparatus 100 according to a certain aspect, the CPU 221 divides a text instruction into a plurality of words in order to acquire a function. The CPU 221 searches the function table 232 using the last word obtained by the division. Further, the CPU 221 acquires the function when one or more keywords matching the last word are present in the function table 232.

(8) In the image forming apparatus 100 according to a certain aspect, the CPU 221 acquires an operation mode from each text instruction as an acquisition unit. As a function discrimination unit, the CPU 221 switches the function discrimination according to the acquired operation mode.

(9) The image forming apparatus 100 according to a certain aspect further includes an option table 233 that defines feasible options for each function. The CPU 221 searches the option table 233 based on the text instruction in order to determine whether the determined function is a valid option or an invalid option. The CPU 221 determines whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches an executable option.

(10) In the image forming apparatus 100 according to a certain aspect, the CPU 221 considers the determined option as an invalid option when the text instruction does not match the executable option for each function.

(11) In the image forming apparatus 100 according to a certain aspect, the CPU 221 determines that the determined option is an invalid option when the text instruction does not specify any of the feasible options for each function.

(12) In the image forming apparatus 100 according to a certain aspect, the CPU 221 determines the determined option as an invalid option when the text instruction matches a plurality of feasible options for each function. The CPU 221 displays one or more options valid for the function on the operation panel 230. The CPU 221 accepts a new utterance to the image forming apparatus 100, and determines whether or not the utterance includes an option that matches one or more options. When the utterance includes an option that matches one or more options, the CPU 221 determines that the matching option is a valid option.

(13) In the image forming apparatus 100 according to a certain aspect, the CPU 221 counts the types of numbers assigned to valid options as a counting unit. In order to display the effective options for the function on the operation panel 230, the CPU 221 causes the operation panel 230 to display the effective options when the number of types is 4 or more.

(14) In the image forming apparatus 100 according to a certain aspect, the CPU 221 assigns a valid option number as a different number for each function and option in order to display the valid option on the operation panel 230.

(15) In the image forming apparatus 100 according to a certain aspect, the CPU 221 assigns the same number to a plurality of options when the valid options are numerical values in order to display the valid options on the operation panel 230.

(16) According to still another embodiment, the program for controlling the image forming apparatus 100 is a step of executing each of a plurality of functions to the CPU 221 included in the image forming apparatus 100 in response to an instruction to the image forming apparatus 100. And, the step of recognizing the utterance to the image forming apparatus 100 and outputting the recognition result, the step of displaying the image, and determining which of the plurality of functions the utterance specifies based on the recognition result. The step of determining whether the determined function is a valid option specified as an option for the image forming apparatus 100 or an invalid option not specified as the option, and the determined function When the selected function is a valid option, the step of accepting the setting of the determined function and the step of displaying the valid option as the setting on the image forming apparatus 100 when the determined function is an invalid option are executed. Let me.

As described above, when the image forming apparatus 100 according to the present embodiment detects an effective option for executing processing according to the operation mode when a command is given by voice, the image forming apparatus 100 selects the option. Enable, otherwise display one or more options corresponding to the mode of operation. The user can instruct the image forming apparatus 100 of the desired option by performing an utterance or an operation for designating the corresponding option from the displayed one or more options.

As a result, the image forming apparatus 100 presents to the user a display prompting the operation as necessary while maintaining the convenience of the operation by the voice, so that the user can form an image even if the result of the voice recognition is insufficient. The device 100 can be used without any trouble. For example, when a complicated operation is performed, the content is displayed on the operation panel 230, so that the user can easily confirm the operation procedure or the setting content of the image forming apparatus 100.

It should be considered that the embodiments disclosed this time are exemplary in all respects and not restrictive. The scope of the present invention is shown by the scope of claims rather than the above description, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims.

The disclosed technical idea is applicable to an image forming apparatus having a voice recognition function.

100 image forming device, 101 operation panel device, 102 scanner unit, 104 feeder, 106, 210 printer engine, 108 tray, 109 paper feed unit, 201 system controller, 202 microphone, 203, 310 voice recognition unit, 205 imaging unit, 206 Input image processing unit, 207 memory, 208 storage device, 209 output image processing unit, 211 speaker, 222 ROM, 230 operation panel, 231 operation mode table, 232 function table, 233 choice table, 234 panel display table, 240 interactive UI Device, 320 storage unit, 330 function discrimination unit, 340 valid / invalid discrimination unit, 350 option control unit, 360 command execution unit, 370 display unit.

Claims (16)

It is an image forming device
A control unit that executes each of a plurality of functions in response to a command to the image forming apparatus,
A voice recognition processing unit that recognizes an utterance to the image forming apparatus and outputs a recognition result,
A display unit that displays images and
A function determining unit for determining which of the plurality of functions the utterance specifies based on the recognition result, and
An option discriminating unit for determining whether the determined function is a valid option defined as an option for the image forming apparatus or an invalid option not defined as the option,
When the determined function is a valid option, a function setting unit that accepts the setting of the determined function, and
An image forming apparatus including a display control unit that displays a valid option as the setting on the display unit when the determined function is an invalid option. Further provided with a dividing section for dividing the utterance into a plurality of text instructions by identifying the silent section of the utterance.
The function discriminating unit determines which function is selected based on the plurality of text instructions, and determines which function is selected.
The image forming apparatus according to claim 1, wherein the option discriminating unit determines whether the discriminated function is an effective option or an invalid option based on the plurality of text instructions. The division unit divides the plurality of text instructions into a plurality of words.
The image forming according to claim 2, wherein the function discriminating unit determines that the function instructed by the utterance is a function related to setting the density when the last word acquired by the division is a signed numerical value. apparatus. The division unit divides the plurality of text instructions into a plurality of words.
The image forming according to claim 2, wherein the function determining unit determines that the function instructed by the utterance is a function related to setting the number of copies when the last word acquired by the division is an unsigned numerical value. apparatus. It also has a memory for storing a function table in which each of the plurality of functions and a keyword are associated with each other.
Determining whether any of the above functions is selected is
Searching the function table based on each of the text instructions,
The image forming apparatus according to claim 2, comprising acquiring the function instructed by the utterance when a keyword matching any of the text instructions is present in the function table. Acquiring the above function
Searching the function table based on the text instruction using all of the text instructions as keywords,
The image forming apparatus according to claim 5, further comprising acquiring the function instructed by the utterance when a keyword matching the text instruction exists in the function table. Acquiring the above function
Dividing the text instruction into multiple words
Searching the function table using the last word obtained by the division,
The image forming apparatus according to claim 5, further comprising acquiring the function when one or more keywords matching the last word are present in the function table. Further provided with an acquisition unit for acquiring an operation mode from each of the text instructions.
The image forming apparatus according to claim 5, wherein the function discrimination unit switches the function discrimination according to the acquired operation mode. Further provided with a choice table that defines the choices that can be performed for each of the above functions.
Determining whether the determined function is a valid option or an invalid option is not possible.
Searching the choice table based on the text instructions
The second aspect of the present invention includes determining whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches the executable option. Image forming device. Determining whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches the executable option means that the text instruction is for each function. 9. The image forming apparatus according to claim 9, wherein the determined option is regarded as an invalid option when it does not match the feasible option. Determining whether the determined function is a valid option or an invalid option based on whether or not the text instruction matches the executable option means that the text instruction is for each function. 9. The image forming apparatus according to claim 9, wherein the determined option is determined to be an invalid option when any of the feasible options is not specified. Determining whether the determined function is a valid option or an invalid option based on whether the text instruction matches the executable option can be determined.
When the text instruction matches a plurality of executable options for each function, the determined option is determined to be an invalid option.
Displaying one or more options effective for the function on the display unit,
Receiving a new utterance to the image forming apparatus and determining whether or not an option matching the one or more options is included in the utterance.
The image forming apparatus according to claim 9, wherein when an option matching the one or more options is included in the utterance, the matching option is determined to be a valid option. Further provided with a counting unit for counting the types of numbers assigned to the valid options.
The image formation according to claim 1, wherein displaying an effective option for the function on the display unit includes displaying the effective option on the display unit when the number of the types is 4 or more. apparatus. The image forming apparatus according to claim 13, wherein displaying the valid options on the display unit includes assigning the numbers of the valid options as different numbers for each function and option. The image forming apparatus according to claim 13, wherein displaying the valid options on the display unit includes assigning the same number to a plurality of options when the valid options are numerical values. A program for controlling an image forming apparatus, the program is applied to the image forming apparatus.
A step of executing each of the plurality of functions in response to a command to the image forming apparatus,
A step of recognizing an utterance to the image forming apparatus and outputting a recognition result,
Steps to display the image and
A step of determining which of the plurality of functions the utterance specifies based on the recognition result, and
A step of determining whether the determined function is a valid option defined as an option for the image forming apparatus or an invalid option not defined as the option, and
When the determined function is a valid option, the step of accepting the setting of the determined function and
A program that executes a step of displaying a valid option as the setting on the image forming apparatus when the determined function is an invalid option.

Images