JP2014203024A - Control device, image forming apparatus, terminal device, control method, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a control device capable of easily improving the precision of an instruction by voice in an image forming apparatus that is operable by voice.SOLUTION: In an image processing system, when an input of a first voice signal representing voice inputted from a microphone connected to an MFP 100 is received (S1), voice recognition based on the first voice signal is performed (S3). When the voice recognition is unsuccessful (S5), an input of a second voice signal representing voice inputted from a microphone connected to a terminal device 300 is received (S7). Voice recognition based on the second voice signal is executed (S9), and the MFP 100 executes image processing associated with a result of the voice recognition (S11).

Description

  The present invention relates to a control device, an image forming device, a terminal device, a control method, and a control program, and more particularly to a control device, an image forming device, a terminal device, a control method, and a control program for an image forming device that can be instructed by voice. .

  There is an operation method in which an instruction is given by voice to an image forming apparatus such as a printer, a facsimile, or an MFP (Multi-Functional Peripheral) which is a complex machine of them. As an example, the image forming apparatus includes a microphone, and voice input through the microphone is recognized using a voice recognition function and converted into a command, and image forming processing is performed.

Japanese Patent Laid-Open No. 2004-226489

  However, depending on the environment in which the image forming apparatus is installed, there is a problem that the accuracy of voice recognition may decrease. For example, if the image forming apparatus is installed in a noisy environment such as a factory or an event venue, or if the image forming apparatus is in the middle of an operation with sound, noise is generated along with the sound for instructing image formation. As a result, the accuracy of voice recognition is reduced.

  Regarding the sound generated by the image forming apparatus itself, Japanese Patent Laying-Open No. 2004-226489 (Patent Document 1) discloses a technique for preventing a decrease in voice recognition rate by interrupting a printing operation when a request for voice recognition work is received. However, since the job processing during printing is interrupted, work efficiency is reduced.

  The present invention has been made in view of such problems, and in an image forming apparatus that can be operated by voice, a control apparatus, an image forming apparatus, and a terminal device that can easily improve the accuracy of voice instructions. It is an object to provide a control method and a control program.

  In order to achieve the above object, according to one aspect of the present invention, the control device is a control device for an image forming apparatus, and the first sound signal representing sound input from a microphone connected to the image forming apparatus. A first voice input means for receiving an input; a communication means for communicating with the terminal apparatus; and a second voice signal that communicates with the terminal apparatus and represents the voice input from a microphone connected to the terminal apparatus. The second voice input means for receiving the input of the voice, the voice recognition means for performing voice recognition based on the voice signal, and the image processing associated with the recognition result by the voice recognition means, Control means for controlling the apparatus to perform image processing; and the control means controls the image forming apparatus using a result of voice recognition based on the first voice signal, or voice based on the second voice signal. And a determination means for determining, for controlling the image forming apparatus using the result of identification. The control means controls the image forming apparatus using the result of the voice recognition based on the first voice signal when the voice recognition based on the first voice signal is successful, and the second voice when the voice recognition based on the first voice signal is not successful. The image forming apparatus is controlled using the result of the speech recognition based on the signal.

  Preferably, the determination unit further determines whether or not the voice recognition based on the first voice signal is appropriate prior to the voice recognition based on the first voice signal by the voice recognition unit. When it is determined that the speech recognition based on the first speech signal is not appropriate, the control unit controls the image forming apparatus using the speech recognition result based on the second speech signal.

  More preferably, the determination unit determines whether or not the speech recognition based on the first voice signal is appropriate by determining whether or not the noise included in the first voice signal is equal to or greater than a predetermined amount.

  Preferably, the determination unit determines whether sound recognition based on the first sound signal is appropriate based on image processing being executed in the image forming apparatus when receiving a sound input with a microphone provided in the image forming apparatus.

  Preferably, when the determination unit determines that the control unit controls the image forming apparatus using the result of the voice recognition based on the second audio signal, the communication unit establishes communication with the terminal device, and the second unit The voice input means receives the input of the second voice signal.

  Preferably, when the determination unit determines that the control unit controls the image forming apparatus using the result of the voice recognition based on the second audio signal, the noise included in the second audio signal is more than a predetermined amount. The communication means transmits a message stored in advance to the terminal device when the noise included in the second audio signal is equal to or greater than a predetermined amount.

  According to another aspect of the present invention, the image forming apparatus communicates with the terminal device and a first sound input means for receiving an input of a first sound signal representing a sound input from a connected microphone. Communication means, a second voice input means for communicating with the terminal device and receiving a second voice signal representing voice inputted from a microphone connected to the terminal device, and voice based on the voice signal A voice recognition unit for performing recognition, an execution unit for specifying the image processing associated with the recognition result of the voice recognition unit, and executing the specified image processing; Determining means for determining whether to perform image processing specified from a result of speech recognition based on a speech signal or to perform image processing specified from a result of speech recognition based on a second speech signal; PrepareThe execution means executes the image processing specified from the result of the voice recognition based on the first voice signal when the voice recognition based on the first voice signal is successful, and the second voice if not successful The image processing specified from the result of speech recognition based on the signal is executed.

  According to still another aspect of the present invention, the terminal device is a terminal device capable of communicating with the image forming apparatus, and is a voice recognition means for performing voice recognition based on a voice signal representing a voice input from a connected microphone. And control means for specifying the image processing associated with the recognition result of the voice recognition means and outputting a control signal to the image forming apparatus to execute the image processing.

  Preferably, the terminal device communicates with the image forming apparatus and is connected to the terminal device and a voice input means for receiving an input of a first voice signal representing a voice inputted from a microphone connected to the image forming apparatus. The control means controls the image forming apparatus using the result of voice recognition based on the first voice signal, or the second voice signal, with the voice signal representing the voice input from the microphone as the second voice signal. And a determination means for determining whether to control the image forming apparatus using the result of voice recognition based on the above. The control means controls the image forming apparatus using the result of the voice recognition based on the first voice signal when the voice recognition based on the first voice signal is successful, and the second voice when the voice recognition based on the first voice signal is not successful. The image forming apparatus is controlled using the result of the speech recognition based on the signal.

  According to still another aspect of the present invention, the control method is a control method for an image forming apparatus using sound, and the first sound signal representing the sound input from the microphone connected to the image forming apparatus is input. An image associated with the result of the speech recognition based on the first speech signal when the step of accepting, the step of performing speech recognition based on the first speech signal, and the speech recognition based on the first speech signal are successful Specifying the processing and causing the image forming apparatus to execute the image processing; and a second representing the voice input from the microphone connected to the terminal device when voice recognition based on the first voice signal is unsuccessful Receiving a voice signal input, performing voice recognition based on the second voice signal, and image processing associated with the result of voice recognition based on the second voice signal. To, and a step of executing image processing on the image forming apparatus.

  According to still another aspect of the present invention, the control program is a program for causing a computer to control the image forming apparatus, and is a first sound signal representing sound input from a microphone connected to the image forming apparatus. When the voice recognition based on the first voice signal is successful, and the voice recognition based on the first voice signal is associated with the result of the voice recognition based on the first voice signal. The image forming apparatus is identified and the image forming apparatus executes the image processing, and when the voice recognition based on the first audio signal is unsuccessful, the terminal apparatus communicates with and is connected to the terminal apparatus. A step of receiving an input of a second voice signal representing a voice inputted from a microphone, a step of performing voice recognition based on the second voice signal, and a second voice signal Based identifies the image processing associated with the result of speech recognition, and a step of executing image processing on the image forming apparatus to the computer.

  According to still another aspect of the present invention, the control program is a program for causing a computer to control the terminal device, and accepts an input of an audio signal representing audio input from a microphone connected to the terminal device. And a step of performing voice recognition based on the voice signal and a step of identifying the image processing associated with the result of the voice recognition and outputting a control signal to the image forming apparatus to perform the image processing. Let

  Preferably, the control program communicates with the image forming apparatus, accepts an input of a first audio signal representing a sound input from a microphone connected to the image forming apparatus, and is input from a microphone connected to the terminal apparatus. The image forming apparatus is controlled by using the result of speech recognition based on the first speech signal, or the result of speech recognition based on the second speech signal is used as the second speech signal. Determining whether to control the image forming apparatus, and causing the computer to execute a control signal and outputting the control signal to the first voice signal when the voice recognition based on the first voice signal is successful. The image forming apparatus is controlled using the result of the speech recognition based on the voice recognition, and the image forming apparatus is controlled using the result of the voice recognition based on the second voice signal when the result is unsuccessful.

  According to the present invention, in an image forming apparatus that can be operated by voice, the accuracy of voice instructions can be easily improved.

It is a figure which shows the specific example of a structure of the image processing system concerning embodiment. FIG. 3 is a diagram illustrating a specific example of a hardware configuration of an MFP (Multi-Functional Peripheral) included in the image processing system. It is a figure which shows the specific example of the hardware constitutions of the terminal device contained in an image processing system. It is a figure showing the operation | movement outline | summary in an image processing system. 2 is a block diagram illustrating a specific example of a functional configuration of an MFP. FIG. 5 is a flowchart showing a specific example of an operation flow in the MFP. 5 is a flowchart showing a specific example of an operation flow in the MFP. It is a flowchart showing the specific example of the process which switches audio | voice input in step S123 of FIG. 12 is a flowchart illustrating a first example of the operation of an MFP according to a modified example. 12 is a flowchart illustrating a first example of the operation of an MFP according to a modified example. 12 is a flowchart showing a second example of the operation of the MFP according to the modification. 12 is a flowchart showing a second example of the operation of the MFP according to the modification.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, these descriptions will not be repeated.

<System configuration>
FIG. 1 is a diagram showing a specific example of the configuration of the image processing system according to the present embodiment.

  Referring to FIG. 1, the image processing system according to the present embodiment includes an MFP (Multi-Function Peripheral) 100 as an example of an image forming apparatus and a terminal device 300 as an example of a terminal apparatus. These are connected by a network such as a LAN (Local Area Network). The image processing system may further include a server 500 as shown.

  The network may be wired or wireless. As an example, as illustrated in FIG. 1, an example in which the MFP 100 and the server 500 are connected to a wired LAN, and the terminal device 300 is connected wirelessly.

  MFP 100 includes a configuration for realizing an image forming function and a voice input device (microphone) as a configuration for realizing a function for receiving voice input (FIG. 2). The image forming apparatus included in the image processing system according to the present embodiment is not limited to the MFP as long as it has at least these functions. Note that the configuration for realizing the function of receiving voice input is not limited to the voice input device, and includes a processing device that processes a voice signal input from the connected voice input device. The same applies to the terminal device 300 described later.

  The terminal device 300 may be a mobile phone or a mobile terminal called a smartphone. The terminal device 300 includes a voice input device (microphone) as a configuration for realizing a function of receiving voice input, and a display device (touch panel) and a voice output device (speaker) as a configuration for realizing a function of outputting information. (FIG. 3). The terminal device included in the image processing system according to the present embodiment may be any device as long as it has at least these functions and can be carried by the user.

<Configuration of MFP>
FIG. 2 is a diagram illustrating a specific example of the hardware configuration of the MFP 100.

  Referring to FIG. 2, MFP 100 includes a CPU (Central Processing Unit) 10 that is an arithmetic device that functions as a control device of MFP 100, and a ROM (Read Only) that stores a program executed by CPU 10 as a memory. (Memory) 11, a RAM (Random Access Memory) 12 for functioning as a work area when the CPU 10 executes a program, an HD (Hard Disk) 16 for storing image data and the like, and a document table (not shown). A scanner 13 for optically reading the original document to obtain image data, a printer 14 for fixing the image data on the printing paper, an operation panel 15 as a display device and an input device, and the network. A network controller 17 for controlling the communication and a microphone 18 Including.

  The operation panel 15 includes a touch panel (not shown) and an operation key group. The touch panel is configured by overlapping a display device such as a liquid crystal display device and a position indicating device such as an optical touch panel or a capacitive touch panel, and displays an operation screen to specify an indicated position on the operation screen. CPU10 displays an operation screen on a touchscreen based on the data for displaying the screen memorize | stored beforehand.

  The specified position (touched position) on the touch panel and the operation signal indicating the pressed key are input to the CPU 10. The CPU 10 specifies the operation content from the pressed key or the displayed operation screen and the designated position, and executes processing based on the operation content.

<Configuration of terminal device>
FIG. 3 is a diagram illustrating a specific example of the hardware configuration of the terminal device 300.

  Referring to FIG. 3, terminal device 300 executes a program with CPU 30 that is an arithmetic device for overall control, ROM 31 for storing a program executed by CPU 30 as a memory, and CPU 30. A RAM 32 for functioning as a work area, a microphone 33, a speaker 34, a touch panel 35 as a display device and an input device, and a network controller 36 for controlling communication via the network.

  When the terminal device 300 has a telephone function such as a mobile phone or a smartphone as described above, the terminal device 300 further includes a configuration for realizing the telephone function in addition to the device configuration of FIG.

<Server configuration>
The server 500 can be realized by a normal computer such as a personal computer. Therefore, the hardware configuration can be the same as the hardware configuration of a normal computer. Therefore, detailed description of the configuration will not be given here.

<Overview of operation>
The user gives a voice instruction when causing the MFP 100 to execute image processing using the image processing system according to the present embodiment. That is, as a first step, the user instructs MFP 100 by voice such as “start copying”. The MFP 100 executes image processing in accordance with the input “start copy” sound.

  When the MFP 100 is installed in a noisy environment such as a factory or an event venue, or when the MFP 100 is performing an operation accompanied by sound, noise may be input together with the sound of “copy start”. . If the “copy start” sound is not properly recognized due to noise, MFP 100 does not execute the image processing instructed by the user.

  Therefore, in the image processing system according to the present embodiment, the terminal device 300, which is a device other than the MFP 100, accepts voice input. The sound received by the MFP 100 is defined as a first sound (first sound signal), and the sound received by the terminal device 300 is defined as a second sound (second sound signal). Specifically, in the image processing system according to the present embodiment, it is assumed that the speech recognition is unsuccessful when the recognition result of the first sound input to MFP 100 is not appropriate as an image processing instruction. At this time, the image processing system accepts the input of the second sound in a state where the second sound can be input. Then, MFP 100 executes image processing according to the second sound.

  In order to perform the above operation, the image processing system according to the present embodiment includes a voice recognition function. As an example, the server 500 includes a voice recognition function. Of course, the MFP 100 or the terminal device 300 may include a voice recognition function instead of the server 500.

  FIG. 4 is a diagram showing an outline of the operation in the image processing system. Referring to FIG. 4, when MFP 100 receives an input of voice (first voice) with microphone 18 (step S1), MFP 100 transmits a voice signal (first voice signal) representing the voice to server 500 (step S1). Step S2). Server 500 executes voice identification based on the first voice signal (step S3), and returns the result of voice recognition to MFP 100 (step S4). In step S3, server 500 converts speech into text as an example, and transmits the text to MFP 100 as a result of speech recognition in step S4.

  MFP 100 performs image processing using the result of voice identification based on the first voice signal (the above text), or the result of voice identification based on the second voice signal representing the voice received by terminal device 300. Is used to determine whether to execute image processing (step S5). MFP 100 previously stores keywords associated with commands for executing the image processing for executable image processing. For example, the MFP 100 stores keywords such as “copy start”, “copy execution”, “copy start”, “copy start”, and the like in association with a command for executing copy processing as image processing. In step S5, the MFP 100 searches for whether the received text matches the stored keyword. If there is a matching keyword, MFP 100 determines that the speech recognition is successful. In this case, MFP 100 determines to perform image processing using the result of voice identification (the text) based on the first voice signal.

  If there is no keyword that matches the received text in step S5, MFP 100 determines that the speech recognition is unsuccessful. In this case, MFP 100 determines to perform image processing using the result of voice identification based on the second voice signal (the voice recognition result received by terminal device 300). Preferably, when MFP 100 determines that voice recognition is unsuccessful, it again accepts input of voice (first voice) with microphone 18. Preferably, MFP 100 uses a result of voice identification based on the second voice signal (a voice recognition result received by terminal device 300) when voice recognition has been unsuccessful a predetermined number of times. The image processing is determined to be executed.

  If it is determined that the image processing is to be executed using the result of the voice identification based on the second voice signal, the MFP 100 automatically establishes communication with the terminal device 300 (step S6). At this time, the MFP 100 may turn off the microphone 18 so that the microphone 18 does not accept voice input, or the microphone 18 processes the voice signal from the microphone 18 while keeping the voice input acceptable. You may make it not.

  Specifically, in step S <b> 6, the MFP 100 starts communication with the terminal device 300 and waits for transmission of an audio signal from the terminal device 300. Communication from the MFP 100 to the terminal device 300 may be to transmit a signal (request) to the terminal device 300 via the network, or when both have a telephone function, the MFP 100 to the terminal device 300. It may be possible to call. Communication from MFP 100 to terminal device 300 preferably includes a message prompting the user of terminal device 300 to input a voice. This message may be voice or text. For example, the MFP 100 may transmit audio data such as “Please give another instruction from the terminal device by voice” to the terminal device 300, and this voice may be output from the speaker 34 of the terminal device 300.

  As described above, establishment of communication between MFP 100 and terminal device 300 includes what is established by request from MFP 100 to terminal device 300 and vice versa. For example, when the MFP 100 executes image processing using the voice identification result based on the second voice signal, the MFP 100 transmits voice data such as “Please call the MFP” to the terminal device 300, Communication may be terminated. Then, communication between MFP 100 and terminal device 300 may be established by calling terminal device 300 to MFP 100.

  When terminal device 300 receives an input of voice (second voice) with microphone 33 (step S7), terminal apparatus 300 transmits a voice signal (second voice signal) representing the voice to server 500 (step S8). Server 500 executes voice identification based on the second voice signal in the same manner as in step S4 (step S9), and returns the voice recognition result (text) to MFP 100 (step S10). The MFP 100 executes image processing using the voice identification result based on the second voice signal received from the server 500 (the voice recognition result received by the terminal device 300) (step S11).

  MFP 100 preferably also determines whether the voice identification is successful or unsuccessful in the same manner as in step S5 with respect to the result of voice identification based on the second voice signal (the result of voice recognition received by terminal device 300). Judging. If the MFP 100 determines that the voice identification based on the second voice signal is unsuccessful, the MFP 100 repeats the process from accepting the voice (second voice) input from the terminal device 300 again.

<Functional configuration>
FIG. 5 is a block diagram showing a specific example of a functional configuration of MFP 100 for performing the above operation. Each function in FIG. 5 is mainly realized by the CPU 10 when the CPU 10 of the MFP 100 reads out and executes a program stored in the ROM 11. However, at least a part may be realized by a hardware configuration shown in FIG. 2 or a hardware configuration such as an electric circuit not shown in FIG.

  Referring to FIG. 5, HD 16 includes a command storage unit 161 that is a storage area for storing a keyword associated with each command for executing image processing.

  Further, referring to FIG. 5, CPU 10 communicates with terminal device 300 via network controller 17 and first audio input unit 101 for receiving an input of a first audio signal representing an audio input from a microphone. Thus, the second voice input unit 102 for receiving the input of the second voice signal representing the voice input from the microphone 33, the voice recognition unit 103 for performing voice recognition based on the voice signal, and the voice recognition unit 103 A control unit 108 for controlling the scanner 13 and the printer 14 to execute the image processing, and a control unit 108 that includes the specifying unit 109 for specifying the image processing associated with the recognition result in Determine whether to control using the result of speech recognition based on the speech signal or to control using the result of speech recognition based on the second speech signal And a communication unit 107 for establishing communication with the terminal device 300 when the determination unit 106 determines to control using the result of speech recognition based on the second audio signal. . When the voice recognition unit 103 succeeds in the voice recognition based on the first voice signal, the control unit 108 performs the image processing using the result of the voice recognition based on the first voice signal so as to execute the image processing. If unsuccessful, control is performed using the result of speech recognition based on the second speech signal.

  As shown in FIG. 4, when server 500 has a voice recognition function and MFP 100 uses the voice recognition function, voice recognition unit 103 transmits the input voice to server 500. Part 104 and recognition result input part 105 for receiving the input of the recognition result from server 500. When MFP 100 has a voice recognition function and uses the voice recognition function, voice recognition unit 103 performs voice recognition processing.

<Operation flow>
6 and 7 are flowcharts showing specific examples of the flow of operations in MFP 100. The operations shown in the flowcharts of FIGS. 6 and 7 are realized by the CPU 10 of the MFP 100 reading out and executing the program stored in the ROM 11 on the RAM 12 and exhibiting the functions shown in FIG.

  Referring to FIG. 6, CPU 10 succeeds in user authentication (YES in step S <b> 101), and enters a mode for controlling MFP 100 using the voice recognition function (YES in step S <b> 103), microphone 18 connected to CPU 10 is set. Turn on to enable voice input from the microphone 18. In addition, a counter for counting the number of times voice recognition is unsuccessful is initialized (step S105).

  CPU10 transmits the 1st audio | voice signal showing the audio | voice input from the microphone 18 to the server 500 (step S107), and receives the recognition result of the audio | voice from the server 500 (step S109). If the speech recognition is successful (YES in step S111), that is, if the text that is the result of speech identification matches any of the keywords stored in association with the command for executing image processing, the CPU 10 The mode for controlling MFP 100 using the voice recognition function is turned OFF, and the counter is reset (step S113). Then, referring to FIG. 7, the CPU 10 executes image processing using the result of voice identification (the above text) based on the first voice signal. That is, the image processing is executed in accordance with the command associated with the text that is the first speech recognition result (step S137).

  Returning to FIG. 6, when the speech recognition based on the first speech signal is unsuccessful (NO in step S111), that is, the text that is the result of speech identification is stored in association with the command for executing the image processing. If none of the keywords matches, the CPU 10 increments the counter for counting the number of times speech recognition was unsuccessful by 1 (step S115). If the number of times of unsuccessful voice recognition has not reached the predetermined number of times defined in advance (NO in step S117), the CPU 10 displays on the operation panel 15 that the voice recognition has failed. The user is notified and prompted to input again (step S119). And CPU10 receives the re-input of an audio | voice and repeats the operation | movement from said step S107.

  When the number of times of unsuccessful voice recognition reaches a predetermined number (YES in step S117), the CPU 10 resets the counter (step S121), and then inputs the voice input from the microphone 18 to the terminal device 300. The process to switch to is performed (step S123). The CPU 10 may turn off the microphone 18 when resetting the counter.

  Referring to FIG. 7, when receiving the input of the second audio signal from terminal device 300 (YES in step S125), CPU 10 transmits the second audio signal to server 500 (step S127). The voice recognition result is received (step S129). If the speech recognition is successful (YES in step S131), that is, if the text that is the result of speech identification matches one of the keywords stored in association with the command for executing image processing, the CPU 10 Image processing is executed using the voice identification result (the above text) based on the second voice signal. That is, the image processing is executed in accordance with the command associated with the text that is the second speech recognition result (step S137). At this time, the CPU 10 may disconnect communication with the terminal device 300 prior to execution of image processing, or may disconnect communication with the terminal device 300 thereafter (step S135).

  If speech recognition based on the second speech signal is unsuccessful (NO in step S131), that is, any keyword stored in association with a command for executing text processing on the text that is the result of speech identification If they do not match, the CPU 10 transmits to the terminal device 300 a message indicating that the voice recognition is unsuccessful and a message prompting re-input of the voice (step S133). Preferably, the CPU 10 transmits, to the terminal device 300, as voice guidance, a message indicating that the voice recognition has been unsuccessful and a message prompting the user to re-input the voice. And CPU10 receives the re-input of the audio | voice from the terminal device 300, and repeats the operation | movement from said step S127.

  If the second audio signal is not input from the terminal device 300 (NO in step S125), the CPU 10 sets the mode for controlling the MFP 100 using the voice recognition function to OFF (step S139), and a series of steps. End the operation.

  Preferably, the CPU 10 captures, from the microphone 33 of the terminal device 300, surrounding sounds before receiving the user's voice input from the terminal device 300 as noise. Then, the CPU 10 compares the noise with a prestored level. When it is determined that the ambient noise is greater than the specified amount, the CPU 10 preferably transmits a previously stored message such as “Please move to a quiet place” to the terminal device 300. To do. Thereby, the accuracy of speech recognition based on the second speech signal can be improved. As will be described later, the noise level may be registered for each login user. This is because, depending on the voice quality of the user, it is easy to recognize voice even if there is some noise, or it is difficult to recognize voice even if there is little noise. Further, the CPU 10 may learn a predetermined level from the determination result of voice identification in step S111.

  FIG. 8 is a flowchart showing a specific example of the process of switching the voice input in step S123. Referring to FIG. 8, CPU 10 includes a case where access information (for example, a telephone number) of terminal device 300 associated with a login user is included in user information such as a telephone directory stored in memory (step). The communication with the terminal device 300 is started by making a call to the telephone number (YES in S201) (step S203). As another example, the CPU 10 may transmit a mail describing its own access information (URL, telephone number, etc.) to the terminal device 300.

  Prior to the start of the communication, the CPU 10 notifies the operation panel 15 of the start of communication such as “communication to Mr. A's terminal device 090-***-***-***”, Buttons such as “OK” and “NG” for instructing permission / inhibition may be displayed to accept the login user's instruction. Then, the CPU 10 may start communication with the terminal device 300 when the “OK” button is pressed on this screen.

  When there is a response from the terminal device 300 to the communication from the MFP 100 (YES in step S207), that is, for example, the user of the terminal device 300 responds to the call to the terminal device 300 or the above When the terminal device 300 accesses the MFP 100 based on the access information described in the mail to the terminal device 300, the CPU 10 transmits a message requesting voice input to the terminal device 300 ( Step S209). In the case of an example in which communication is performed by telephone, the CPU 10 may transmit voice guidance. If the access is to a URL or the like, the CPU 10 may transmit text data. Then, the CPU 10 receives an audio signal from the terminal device 300 according to the request (step S211).

  If the access information of the terminal device 300 associated with the login user is not included in the user information (NO in step S201), the CPU 10 executes a process for acquiring the access information of the terminal device 300 (step S201). S205). In step S <b> 205, for example, the CPU 10 may display “Please enter the phone number of Mr. A's terminal device” on the operation panel 15 to accept direct input of access information. Alternatively, CPU 10 includes access information (for example, a telephone number) of MFP 100 such as “Please call MFP ***-***-***-*** from Mr. A's terminal device” on operation panel 15. A message may be displayed to prompt access to MFP 100. In this case, the CPU 10 may regard the access to itself received within a predetermined time from this display as the access from the terminal device 300 and store the transmission source as the access information of the terminal device 300. At this time, the CPU 10 may accept voice input using access from the terminal device 300 to the MFP 100. However, preferably, the CPU 10 does not start communication using communication from the terminal device 300, but temporarily disconnects the communication and starts communication based on the acquired access information of the terminal device 300. . In this way, charging for communication to the terminal device 300 can be suppressed.

<Effect of Embodiment>
When the MFP 100 according to the present embodiment executes the above-described control, the voice based on the voice signal from the microphone 18 of the MFP 100 due to ambient noise or noise when instructing image processing using the voice recognition function. When the recognition cannot be performed correctly, the voice input from the terminal device 300 such as the user's mobile phone can be used. Thereby, the precision of the instruction | indication of the image process using a speech recognition function can be improved, without impairing a user's convenience.

<Modification>
More preferably, the CPU 10 determines whether or not the voice recognition based on the first voice signal is appropriate prior to the voice recognition based on the first voice signal representing the voice input by the microphone 18. If it is determined that this voice recognition is not appropriate, the CPU 10 performs image processing using the result of voice recognition based on the second sound signal that is a sound signal from the terminal device 300.

  As a first example of the above determination, the CPU 10 makes the above determination based on the image processing being executed when the microphone 18 accepts voice input. For example, when a voice input is received by the microphone 18, a print process is being executed (printing), a finisher process (staple process, etc.) is being executed, or a process involving voice output such as BGM or sound effects is being executed. If a job that generates a loud sound is being executed, there is a high possibility that speech recognition will be unsuccessful. Therefore, the CPU 10 determines that the voice recognition based on the first voice signal is not appropriate when the image input is being performed when the microphone 18 receives the voice input. For this reason, as an example, the CPU 10 stores the type of job accompanied by the above-mentioned process that produces a loud sound in advance, and performs the above determination by determining whether or not the process is applicable.

  9 and 10 are flowcharts illustrating a first example of the operation of MFP 100 according to the modification. The flowcharts of FIGS. 9 and 10 are obtained by adding the determination of step S104 and the process of step S120 after the process of step S103 of the flowcharts of FIGS. That is, referring to FIG. 9, in the first example of the modified example, CPU 10 switches to a mode for controlling MFP 100 using the voice recognition function, and then turns on microphone 18 to accept voice input with microphone 18. Prior to this, it is determined whether or not the job being executed corresponds to the type of job that is stored in advance as described above and involves processing with a loud sound (step S104). When the job being executed corresponds to such a job type (YES in step S104), the CPU 10 performs a process of switching to the input in the terminal device 300 without receiving the first voice input (step S123). ). At this time, preferably, the CPU 10 displays on the operation panel 15 that switching to voice input from the terminal device 300 is performed (step S120). By displaying in this way, the logged-in user can smoothly switch to voice input using the terminal device 300 without performing voice input from the microphone 18.

  As a second example of the above determination, the CPU 10 performs the above determination by determining whether or not the noise included in the first audio signal is greater than or equal to a predetermined amount stored in advance. If the noise representing the surrounding sound is large prior to the voice input from the microphone 18, there is a high possibility that the voice recognition will be unsuccessful. Therefore, the CPU 10 determines that the voice recognition based on the first voice signal is not appropriate when the noise representing the surrounding sound is equal to or higher than a predetermined level before the voice input is received by the microphone 18. For this reason, the CPU 10 stores the above-mentioned predetermined level that becomes a noise threshold value in advance. This predetermined level may be registered for each login user. This is because, depending on the voice quality of the user, it is easy to recognize voice even if there is some noise, or it is difficult to recognize voice even if there is little noise. Further, the CPU 10 may learn a predetermined level from the determination result of voice identification in step S111.

  11 and 12 are flowcharts showing a second example of the operation of MFP 100 according to the modification. The flowcharts of FIGS. 11 and 12 are obtained by adding the process of step S106-1, the determination of step S106-2, and the process of step S120 after the process of step S105 of the flowcharts of FIGS. That is, referring to FIG. 11, in the second example of the modification, when the CPU 10 turns on the microphone 18 in step S105, the microphone 18 uses the surrounding sound before receiving the voice input from the user as noise. (Step S106-1). Then, the CPU 10 compares the noise with a prestored level and determines whether or not noise higher than the predetermined level is generated (step S106-2).

  If the noise is higher than a predetermined level stored in advance (YES in step S106-2), CPU 10 performs a process of switching to the input in terminal device 300 without accepting the first voice input (step S123). At this time, preferably, the CPU 10 displays on the operation panel 15 that switching to voice input from the terminal device 300 is performed (step S120). By displaying in this way, the logged-in user can smoothly switch to voice input using the terminal device 300 without performing voice input from the microphone 18.

  When MFP 100 according to the modification executes the above-described control, it is possible to efficiently instruct image processing using the voice recognition function.

  In the above example, it is assumed that the server 500 performs voice recognition. However, voice recognition may be performed by MFP 100 as described above. Alternatively, terminal device 300 stores in advance a keyword associated with a command for causing MFP 100 to execute image processing, performs voice recognition based on a voice signal representing voice received by microphone 33, and obtains a command from the result. In particular, the command may be transmitted to the MFP 100.

  In addition, the terminal device 300 includes the function of the server 500 described above, and controls the MFP 100 using the voice recognition result based on the first voice signal from the MFP 100 or the voice based on the second voice signal from the microphone 33. It may be determined whether MFP 100 is to be controlled using the recognition result, and MFP 100 may be controlled based on the result. That is, each function illustrated in FIG. 5 may be included in any device included in the image processing system.

  Furthermore, a program for causing the CPU 10 of the MFP 100, the CPU 30 of the terminal device 300, and the like to execute the above-described operation can be provided. Such a program is recorded on a computer-readable recording medium such as a flexible disk attached to the computer, a CD-ROM (Compact Disk-Read Only Memory), a ROM, a RAM, and a memory card, and provided as a program product. You can also. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer. A program can also be provided by downloading via a network.

  A program according to the present invention is a program module that is provided as a part of an operating system (OS) of a computer and that executes necessary processes by calling necessary modules in a predetermined arrangement at a predetermined timing. Also good. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present invention.

  The program according to the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. Such a program incorporated in another program can also be included in the program according to the present invention.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10, 30 CPU, 11, 31 ROM, 12, 32 RAM, 13 Scanner, 14 Printer, 15 Operation panel, 16 HD, 17, 36 Network controller, 18, 33 Microphone, 34 Speaker, 35 Touch panel, 300 Terminal device, 100 MFP, 101 First voice input unit, 102 Second voice input unit, 103 Voice recognition unit, 104 Transmission unit, 105 Recognition result input unit, 106 Judgment unit, 107 Communication unit, 108 Control unit, 109 Identification unit, 161 Command storage Department, 500 servers.

Claims (13)

A control device for an image forming apparatus,
First audio input means for receiving input of a first audio signal representing audio input from a microphone connected to the image forming apparatus;
A communication means for communicating with the terminal device;
A second voice input means for communicating with the terminal device and receiving an input of a second voice signal representing a voice inputted from a microphone connected to the terminal device;
Speech recognition means for performing speech recognition based on a speech signal;
Control means for specifying image processing associated with a recognition result by the voice recognition means and controlling the image forming apparatus to execute the image processing;
The control unit controls the image forming apparatus using a voice recognition result based on the first voice signal, or controls the image forming apparatus using a voice recognition result based on the second voice signal. And a determination means for determining whether to
The control unit controls the image forming apparatus using a result of speech recognition based on the first speech signal when speech recognition based on the first speech signal is successful, and when unsuccessful. A control device that controls the image forming apparatus using a result of voice recognition based on the second voice signal. The determination means further determines whether or not the voice recognition based on the first voice signal is appropriate prior to the voice recognition based on the first voice signal in the voice recognition means,
The control means controls the image forming apparatus using a result of voice recognition based on the second voice signal when it is determined that voice recognition based on the first voice signal is not appropriate. Item 2. The control device according to Item 1.   The determination unit determines whether or not speech recognition based on the first audio signal is appropriate by determining whether or not noise included in the first audio signal is greater than or equal to a predetermined amount. The control device described in 1.   The determination unit determines whether sound recognition based on the first sound signal is appropriate based on image processing being performed in the image forming apparatus when receiving sound input from the microphone included in the image forming apparatus. The control device according to claim 2.   When the determination unit determines that the control unit controls the image forming apparatus using a result of voice recognition based on the second audio signal, the communication unit establishes communication with the terminal device, The control device according to claim 1, wherein the second voice input unit accepts an input of the second voice signal. When the determination unit determines that the control unit controls the image forming apparatus using the result of the voice recognition based on the second audio signal, the noise included in the second audio signal further includes a specified amount. Determine whether or not
The said communication means transmits the message memorize | stored beforehand with respect to the said terminal device, when the noise contained in a said 2nd audio | voice signal is more than the said predetermined amount, The any one of Claims 1-5 The control device described in 1. First voice input means for receiving input of a first voice signal representing voice input from a connected microphone;
A communication means for communicating with the terminal device;
A second voice input means for communicating with the terminal device and receiving an input of a second voice signal representing a voice inputted from a microphone connected to the terminal device;
Speech recognition means for performing speech recognition based on a speech signal;
An execution unit for specifying the image processing associated with the recognition result by the voice recognition unit and executing the specified image processing;
The execution means executes the image processing specified from the result of speech recognition based on the first sound signal, or executes the image processing specified from the result of speech recognition based on the second sound signal. And a determination means for determining whether to
The execution means executes the image processing specified from the result of speech recognition based on the first speech signal when speech recognition based on the first speech signal is successful, and when unsuccessful. An image forming apparatus that executes the image processing specified from a result of voice recognition based on the second voice signal. A terminal device capable of communicating with an image forming apparatus,
Speech recognition means for performing speech recognition based on a speech signal representing speech input from a connected microphone;
A terminal device comprising: control means for specifying image processing associated with a recognition result by the voice recognition means and outputting a control signal to the image forming apparatus to execute the image processing. Audio input means for communicating with the image forming apparatus and receiving an input of a first audio signal representing audio input from a microphone connected to the image forming apparatus;
The control unit uses the result of voice recognition based on the first voice signal as the second voice signal, the voice signal representing the voice input from the microphone connected to the terminal device as the image formation. Determination means for determining whether to control the apparatus or to control the image forming apparatus using a result of voice recognition based on the second voice signal;
The control unit controls the image forming apparatus using a result of speech recognition based on the first speech signal when speech recognition based on the first speech signal is successful, and when unsuccessful. The terminal device according to claim 8, wherein the image forming apparatus is controlled using a result of voice recognition based on the second voice signal. A control method of an image forming apparatus using sound,
Receiving an input of a first audio signal representing audio input from a microphone connected to the image forming apparatus;
Performing speech recognition based on the first speech signal;
When speech recognition based on the first speech signal is successful, the image processing associated with the result of speech recognition based on the first speech signal is identified and the image processing is executed on the image forming apparatus Step to
Receiving voice signal input from a microphone connected to a terminal device when voice recognition based on the first voice signal is unsuccessful;
Performing speech recognition based on the second speech signal;
And a step of specifying image processing associated with a result of speech recognition based on the second audio signal and causing the image forming apparatus to execute the image processing. A program for causing a computer to control an image forming apparatus,
Receiving an input of a first audio signal representing audio input from a microphone connected to the image forming apparatus;
Performing speech recognition based on the first speech signal;
When speech recognition based on the first speech signal is successful, the image processing associated with the result of speech recognition based on the first speech signal is identified and the image processing is executed on the image forming apparatus Step to
A step of communicating with a terminal device when voice recognition based on the first voice signal is unsuccessful and receiving an input of a second voice signal representing a voice input from a microphone connected to the terminal device; ,
Performing speech recognition based on the second speech signal;
A control program for causing the computer to execute the step of specifying the image processing associated with the result of speech recognition based on the second audio signal and causing the image forming apparatus to execute the image processing. A program for causing a computer to control a terminal device,
Receiving an input of an audio signal representing an audio input from a microphone connected to the terminal device;
Performing speech recognition based on the speech signal;
A control program for causing the computer to execute the step of identifying image processing associated with the result of the speech recognition and outputting a control signal to the image forming apparatus to execute the image processing. Communicating with the image forming apparatus and receiving an input of a first audio signal representing an audio input from a microphone connected to the image forming apparatus;
Whether to control the image forming apparatus using a voice recognition result based on the first voice signal, with the voice signal representing the voice inputted from the microphone connected to the terminal device as a second voice signal. And causing the computer to further execute a step of determining whether to control the image forming apparatus using a result of speech recognition based on the second speech signal,
In the step of outputting the control signal, if the speech recognition based on the first speech signal is successful, the image forming apparatus is controlled using the speech recognition result based on the first speech signal, 13. The control program according to claim 12, wherein the image forming apparatus is controlled using a result of speech recognition based on the second speech signal.

Images