JP7139937B2 - Speech processing system, job generation device, job generation method and job generation program - Google Patents

Description

TECHNICAL FIELD The present invention relates to a voice processing system, a job generation device, a job generation method, and a job generation program, and more particularly to a voice processing system that generates a job based on voice, a job generation device that generates a job based on voice, and its voice processing. The present invention relates to a job generation method executed by a system or its job generation device and a job generation program for causing a computer to generate a job based on voice.

2. Description of the Related Art Generally, a multifunction peripheral (hereinafter referred to as "MFP") is installed in an office. The user causes the MFP to execute processes such as printing image data, copying documents, reading documents, storing image data, and transmitting image data. A requester may request another acceptor to perform a process on the MFP. In this case, the acceptor operates the MFP to perform the work. The acceptor must set operating conditions for the MFP in order to have the MFP execute the requested work. If the acceptor misunderstands the content of the work transmitted from the requester, or sets the operating conditions to be set in the MFP incorrectly, the work requested by the requester may not be executed by the MFP. be.

For example, Japanese Unexamined Patent Application Publication No. 2003-114779 discloses means for allowing the user to input a free word for any setting item, means for setting the setting item according to the input free word, and A setting processing apparatus for an image forming apparatus is described, comprising a definition file generating and saving means for generating and saving a definition file according to setting items. However, according to the setting processing device described in Japanese Patent Application Laid-Open No. 2003-114779, although one operator can set a job, one user out of a plurality of users requests the other user for a job. In this case, the other user must re-enter the free word into the setting processing device based on the content of the request, which complicates the operation of setting the job.

JP-A-2003-114779

One of the objects of the present invention is to provide a voice processing system that facilitates job setting operations.

Another object of the present invention is to provide a job generating apparatus that facilitates the operation of setting jobs.

Another object of the present invention is to provide a job generation method that facilitates the operation of setting a job.

Another object of the present invention is to provide a job generation program that facilitates the operation of setting jobs.

According to one aspect of the present invention, a sound processing system includes a plurality of sound collecting devices for collecting sounds, and a job generating device for generating a job to be executed by an image processing device, wherein the plurality of sound collecting devices and the job generation device includes user identification means for identifying a user who has made the utterance, and the job generation device selects the voice collected by the first device among the plurality of voice collection devices and is the first voice by the user identification means. Based on the voice that identifies the user and the voice that is collected by a second device that is different from the first device among a plurality of voice collecting devices and that identifies a second user that is different from the first user by the user identifying means and job generation means for generating a requested job, which the first user has requested the second user to execute, as a job.

According to this aspect, when the job generation device identifies the first user from the voice collected by the first device, based on the voice and the voice uttered by the second user collected by the second device, A requested job that the first user has requested the second user to execute is generated as a job to be executed by the image processing apparatus. Therefore, since the requested job is generated from the conversation of the first user and the second user who are located at separate locations, it is possible to provide a voice processing system that facilitates the operation of setting the job.

Preferably, the job generation device further performs the first user identification information for identifying one of the plurality of pre-registered users from the voice identifying the first user by the user identification means. A voice collecting device that determines a user identified by user identification information detected from a voice identified by one user as a second user, and collects the voice uttered by the second user among one or more other voice collecting devices. to be the second device.

According to this aspect, the user identified by the user identification information detected from the voice identifying the first user is determined to be the second user, and the voice collection device that collects the voice uttered by the second user is the second user. Determined by the device. Therefore, it is possible to easily determine the voice collecting device that collects the voice uttered by the second user who is conversing with the first user.

Preferably, when there are a plurality of voice collecting devices that collect the voice uttered by the second user, the device determining means determines the volume of the voice collected among the plurality of voice collecting devices that collect the voice uttered by the second user. is determined to be the second device.

According to this aspect, since the voice collecting device with the highest volume of the collected voice among the plurality of voice collecting devices collecting the voice uttered by the second user is determined as the pairing device, the accuracy of voice recognition is improved. can increase

Preferably, the job generating means generates the requested job when the voice collected by the second device indicates the contents of the permission.

According to this aspect, it is possible not to generate a job when the second user does not accept the request by the first user.

Preferably, the job generator is any one of a plurality of audio collection devices.

Preferably, the job generation device is an image processing device.

According to still another aspect of the present invention, a job generating apparatus is a job generating apparatus for generating a job to be executed by an image processing apparatus, comprising user identifying means for identifying a user who has uttered a voice; Voice collected by a first device out of the voice collecting devices and the first user is specified by the user specifying means, and voice collected by a second device different from the first device out of the plurality of voice collecting devices and a job generating means for generating a requested job, which the first user has requested the second user to execute, as a job based on the voice by which the second user different from the first user is specified by the user specifying means. Prepare.

According to this aspect, when the first user is identified from the voice collected by the first device, the first user identifies the first user based on the voice and the voice uttered by the second user collected by the second device. 2. The requested job requested to be executed by the user is generated as a job to be executed by the image processing apparatus. Therefore, since the requested job is generated from the conversation between the first user and the second user who are located at separate locations, it is possible to provide a job generating apparatus that facilitates the operation of setting a job.

Preferably, the first user is identified in response to detection of user identification information for identifying one of a plurality of pre-registered users from the voice identifying the first user by the user identifying means. Determining the user identified by the user identification information detected from the voice as the second user, and determining the voice collecting device for collecting the voice uttered by the second user among the plurality of voice collecting devices as the second device. a means;

According to this aspect, the user identified by the user identification information detected from the voice identifying the first user is determined to be the second user, and the voice collection device that collects the voice uttered by the second user is the second user. Determined by the device. Therefore, it is possible to easily determine the voice collecting device that collects the voice uttered by the second user who is conversing with the first user.

Preferably, when there are a plurality of voice collecting devices that collect the voice uttered by the second user, the device determining means determines the volume of the voice collected from among the plurality of voice collecting devices that collect the voice uttered by the second user. Determine the largest sound collection device to be the second device.

According to this aspect, the voice collecting device with the highest volume of the collected voice among the plurality of voice collecting devices collecting the voice uttered by the second user is determined as the second device, so the accuracy of voice recognition is improved. can increase

Preferably, the job generation means generates the requested job when the voice collected by the second device indicates the content of the permission.

According to this aspect, it is possible not to generate a job when the second user does not accept the request by the first user.

According to still another aspect of the present invention, a job generating method is a sound processing system comprising a plurality of sound collecting devices for collecting sound and a job generating device for generating a job to be executed by an image processing device. wherein a user identifying step of identifying a user who has spoken is performed by one of the plurality of voice collecting devices and the job generating device, and the first device among the plurality of voice collecting devices collects the user identification step. and a voice collected by a second device different from the first device among a plurality of voice collection devices, and a voice collected by the first user in the user identification step The job generating device is caused to execute a job generating step of generating as a job the requested job that the first user has requested the second user to execute, based on the voice specifying the second user different from the second user.

According to this aspect, it is possible to provide a job generation method that facilitates the operation of setting a job.

According to still another aspect of the present invention, a job generation method is a job generation method executed by a job generation device that generates a job to be executed by an image processing device, wherein a user who has uttered a voice is specified. a user identifying step, a sound collected by a first device out of the plurality of sound collecting devices, the sound identifying the first user in the user identifying step, and a sound different from the first device out of the plurality of sound collecting devices An image of the requested job that the first user has requested the second user to execute based on the voice collected by the second device and the voice that specifies the second user different from the first user in the user specifying step. and a job generation step for generating a job to be executed by the processing device.

According to this aspect, it is possible to provide a job generation method that facilitates the operation of setting a job.

According to still another aspect of the present invention, the job generation program is a job generation program executed by a computer that controls the job generation device for generating a job to be executed by the image processing device, the job generation program generating a voice. a user identifying step of identifying a user; a sound collected by a first device out of a plurality of sound collecting devices that identifies the first user in the user identifying step; The first user requested the second user to perform based on the voice collected by the second device different from the device and the voice that identifies the second user different from the first user in the user identification step and a job generation step of generating the requested job as a job.

According to this aspect, it is possible to provide a job generation program that facilitates the operation of setting a job.

1 is a diagram showing an overall overview of a speech processing system in one of the first embodiments of the present invention; FIG. 2 is a block diagram showing an example of an overview of the hardware configuration of a smart speaker according to the first embodiment; FIG. 2 is a block diagram showing an overview of the hardware configuration of the MFP according to the first embodiment; FIG. 3 is a block diagram showing an example of functions of a CPU included in the smart speaker according to the first embodiment; FIG. It is a figure which shows an example of a keyword table. 3 is a block diagram showing an example of functions of a CPU included in the MFP according to the first embodiment; FIG. It is a figure which shows an example of a login screen. 4 is a flow chart showing an example of the flow of job generation in the first embodiment; 7 is a flowchart showing an example of the flow of user identification processing; 4 is a flowchart showing an example of the flow of pairing processing; 9 is a flowchart showing an example of the flow of job generation sub-processing; 7 is a flowchart illustrating an example of the flow of job execution processing; 7 is a flowchart showing an example of the flow of execution instruction processing; FIG. 10 is a block diagram showing an example of functions of a CPU included in the smart speaker according to the second embodiment; FIG. FIG. 11 is a block diagram showing an example of functions of a CPU included in the MFP according to the second embodiment; FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are given the same reference numerals. Their names and functions are also the same. A detailed description thereof will therefore not be repeated.

<First embodiment>
FIG. 1 is a diagram showing an overall outline of a speech processing system in one of the first embodiments of the present invention. Referring to FIG. 1 , audio processing system 1 includes MFP (Multi Function Peripheral) 100 , smart speakers 200 , 200 A, 200 B and 200 C, and server 400 .

MFP 100 is an example of an image processing apparatus. Smart speakers 200, 200A, 200B, and 200C are examples of job generation devices. Each of smart speakers 200, 200A, 200B, and 200C generates a job defining a process to be executed by MFP 100, and MFP 100 executes the job. MFP 100 is connected to network 3 . Access points (AP) 9 and 9A are connected to the network 3 . APs 9 and 9A are relay devices having wireless communication functions. Each of smart speakers 200 and 200A is connected to network 3 by communicating with AP9, and each of smart speakers 200B and 200C is connected to network 3 by communicating with AP9A. Therefore, smart speakers 200 , 200 A, 200 B, and 200 C can communicate with each other and with MFP 100 . Network 3 is, for example, a local area network (LAN). In the network 3, the form of connection may be wired or wireless. Network 3 may also be a wide area network (WAN), a public switched telephone network (PSTN), the Internet, or the like. Since smart speakers 200, 200A, 200B, and 200C have the same functions and hardware configurations, smart speaker 200 will be described here as an example unless otherwise specified.

A gateway (G/W) device 7 is connected to the network 3 and the Internet 5 . Gateway device 7 relays network 3 and Internet 5 . Server 400 is connected to Internet 5 . Therefore, each of MFP 100 and smart speakers 200 , 200 A, 200 B, and 200 C can communicate with server 400 via gateway device 7 .

The server 400 has an authentication function for authenticating a user who speaks. For example, the server 400 registers voiceprint information indicating the voiceprints of each of a plurality of pre-registered users. The server 400 identifies users using voiceprints of multiple users. In the present embodiment, smart speakers 200 , 200 A, 200 B, and 200 C transmit voiceprints of sounds collected by each to server 400 . Based on the voiceprint transmitted from each of smart speakers 200, 200A, 200B, and 200C, server 400 identifies the user of the voiceprint, and replies to each of smart speakers 200, 200A, 200B, and 200C.

FIG. 2 is a block diagram showing an example of an overview of the hardware configuration of the smart speaker according to the first embodiment. Referring to FIG. 2, smart speaker 200 includes a central processing unit (CPU) 201 for controlling the entire smart speaker 200, and a ROM (Read Only Memory) 202 for storing programs to be executed by CPU 201. , a RAM (Random Access Memory) 203 used as a work area for the CPU 201, an EPROM (Erasable Programmable ROM) 204 for storing data in a non-volatile manner, a communication unit 205 for connecting the CPU 201 to the network 3, and for displaying information. It includes a display unit 206 , an operation unit 207 that receives user operation input, a microphone 208 , a speaker 209 , and a serial interface 210 .

CPU 201 downloads a program from a computer connected to Internet 5 and stores it in EPROM 204 . Also, when a computer connected to network 3 writes a program into EPROM 204 , the program is stored in EPROM 204 . The CPU 201 loads a program stored in the EPROM 204 into the RAM 203 and executes it.

Microphone 208 collects sound and converts the collected sound into electrical signals. Microphone 208 outputs to CPU 201 audio data converted from audio into electrical signals.

A serial interface 210 is an interface for serial communication with an external device. Here, the serial communication is the USB (Universal Serial Bus) standard. The serial interface 210 can be connected to an external device that can communicate according to the USB standard. The CPU 111 can access external devices via the serial interface 210 . The external device includes storage devices such as a USB memory 211 and a CD drive. Here, a case where the external device is the USB memory 211 will be described as an example. The USB memory 211 includes a semiconductor memory such as EPROM and a serial communication circuit. The CPU 201 loads a program recorded in the USB memory 211 attached to the serial interface 210 into the RAM 203 and executes it.

The medium for storing the program executed by the CPU 201 is not limited to the USB memory 211, and may be an optical disk (CD-ROM (Compact Disk ROM), MO (Magnetic Optical Disc)/MD (Mini Disc)/DVD (Digital Versatile Disc). )), optical card, mask ROM. The programs here include not only programs that can be directly executed by the CPU 201, but also source programs, compressed programs, encrypted programs, and the like.

FIG. 3 is a block diagram outlining the hardware configuration of the MFP according to the first embodiment. Referring to FIG. 3, MFP 100 includes main circuit 110, document reading unit 130 for reading a document, automatic document feeder 120 for conveying a document to document reading unit 130, and document reading unit 130. image forming unit 140 for forming an image on paper or the like based on image data read and output, a paper feeding unit 150 for supplying paper to the image forming unit 140, and processing the paper on which the image is formed. and an operation panel 160 as a user interface.

The post-processing unit 155 executes sorting processing for rearranging and discharging one or more sheets on which images have been formed by the image forming unit 140, punching processing for punching holes, and stapling processing for driving staples.

The main circuit 110 includes a CPU 111, a communication interface (I/F) section 112, a ROM 113, a RAM 114, a hard disk drive (HDD) 115 as a mass storage device, a facsimile section 116, and a CD-ROM 118. and an external storage device 117 . CPU 111 is connected to automatic document feeder 120 , document reading portion 130 , image forming portion 140 , paper feeding portion 150 , post-processing portion 155 and operation panel 160 , and controls MFP 100 as a whole.

ROM 113 stores a program executed by CPU 111 or data necessary for executing the program. The RAM 114 is used as a work area when the CPU 111 executes programs. Further, RAM 114 temporarily stores image data continuously sent from document reading unit 130 .

Communication I/F unit 112 is an interface for connecting MFP 100 to network 3 . CPU 111 communicates with smart speaker 200 via communication I/F unit 112 to transmit and receive data. Also, the communication I/F unit 112 can communicate with a computer connected to the Internet 5 via the network 3 .

Facsimile unit 116 is connected to the public switched telephone network (PSTN) and transmits facsimile data to or receives facsimile data from the PSTN. Facsimile unit 116 stores the received facsimile data in HDD 115 or outputs it to image forming unit 140 . Image forming unit 140 prints the facsimile data received by facsimile unit 116 on paper. Further, facsimile section 116 converts data stored in HDD 115 into facsimile data and transmits the facsimile data to a facsimile machine connected to the PSTN.

A CD-ROM 118 is attached to the external storage device 117 . CPU 111 can access CD-ROM 118 via external storage device 117 . CPU 111 loads a program recorded on CD-ROM 118 attached to external storage device 117 into RAM 114 and executes the program. The medium for storing the program executed by the CPU 111 is not limited to the CD-ROM 118, and may be semiconductor memory such as an optical disk, an IC card, an optical card, a mask ROM, or an EPROM.

Further, the program executed by CPU 111 is not limited to the program recorded on CD-ROM 118, and a program stored in HDD 115 may be loaded into RAM 114 and executed. In this case, another computer connected to network 3 may rewrite the program stored in HDD 115 of MFP 100 or add and write a new program. Furthermore, MFP 100 may download a program from another computer connected to network 3 and store the program in HDD 115 . The programs here include not only programs directly executable by the CPU 111, but also source programs, compressed programs, encrypted programs, and the like.

Operation panel 160 is provided on the upper surface of MFP 100 and includes display portion 161 and operation portion 163 . The display unit 161 is, for example, a liquid crystal display (LCD) or an organic LE (electroluminescence) display, and displays an instruction menu for the user, information about acquired image data, and the like. Operation portion 163 includes a touch panel 165 and a hard key portion 167 . Touch panel 165 is provided on the upper surface or the lower surface of display unit 161 so as to overlap display unit 161 . Hard key portion 167 includes a plurality of hard keys. A hard key is, for example, a contact switch. Touch panel 165 detects a position designated by the user on the display surface of display unit 161 .

In speech processing system 1 according to the present embodiment, smart speakers 200, 200A, 200B, and 200C are arranged at different positions. Therefore, smart speakers 200, 200A, 200B, and 200C each collect voices uttered by different users. Here, when user A and user B talk over the phone, the smart speaker 200 collects the voice uttered by user A, and the smart speaker 200B collects the voice uttered by user B. do.

FIG. 4 is a block diagram showing an example of functions of the CPU 201 included in the smart speaker 200 according to the first embodiment. The functions shown in FIG. 4 may be implemented by hardware, or may be implemented by the CPU 201 of the smart speaker 200 by causing the CPU 201 to execute a program stored in the ROM 202, EPROM 204, or CD-ROM. . Here, a case of causing the CPU 201 included in the smart speaker 200 to execute the job generation program will be described as an example.

Referring to FIG. 4 , CPU 201 included in smart speaker 200 includes voice reception unit 251 , voice recognition unit 253 , user identification unit 255 , job generation unit 257 , job transmission unit 259 , response unit 261 , A caller determination unit 263 , a device determination unit 265 , and a voice information acquisition unit 267 are included. Audio reception unit 251 receives audio data output from microphone 208 . Voice data is input from the microphone 208 only while the user is speaking. Voice accepting portion 251 outputs voice data input from microphone 208 and time information indicating the time when the voice data was input to voice recognition portion 253 and user specifying portion 255 .

Voice recognition unit 253 recognizes the voice specified by the voice data each time voice data and time information are input from voice reception unit 251 . Specifically, the voice recognition unit 253 converts voice data into voice information composed of characters. Techniques for recognizing speech are well known and will not be described again here. Voice recognition unit 253 outputs a set of voice information obtained by converting voice data and time information to job generation unit 257 , response unit 261 and caller determination unit 263 .

User specifying unit 255 specifies the user who has uttered the voice specified by the voice data each time voice data and time information are input from voice accepting unit 251 . Specifically, user identification unit 255 transmits the voiceprint extracted from the voice data to server 400 and requests server 400 to identify the user. Note that the technology for specifying the user who has uttered the voice is well known, so the description will not be repeated here. When server 400 identifies the user of the voice data, server 400 returns the user identification information for identifying the user. The set is output to job generation unit 257 and response unit 261 . Here, the user identification unit 255 identifies user A. FIG. Note that if the smart speaker 200 has an authentication function, the user identification unit 255 may identify the user who uttered the voice of the audio data.

The user identified by user identification unit 255 and the user identified by any of smart speakers 200A, 200B, and 200C may be the same. For example, when the distance between smart speaker 200 and smart speaker 200A is less than a predetermined distance, smart speaker 200 and smart speaker 200A may simultaneously collect voices uttered by one user. In this case, smart speaker 200 and smart speaker 200A each receive an arbitration instruction from server 400 . The arbitration instruction sent by the server 400 to the smart speaker 200 includes the device identification information of the smart speaker 200A, and the arbitration instruction sent by the server 400 to the smart speaker 200A includes the device identification information of the smart speaker 200.

Smart speaker 200 and smart speaker 200A each attempt to identify the user on the device with the loudest collected voice. Specifically, the user identifying unit 255 acquires the volume of the voice collected by the smart speaker 200A from the smart speaker 200A, and compares it with the volume of the voice collected by the own device. The user identifying unit 255 identifies the user if the volume of its own device is higher than the volume obtained from the smart speaker 200A, but does not identify the user if the volume of its own device is lower than the volume obtained from the smart speaker 200A. Further, when the volume of the device itself is the same as the volume acquired from smart speaker 200A, user identification unit 255 identifies the user from either one of smart speaker 200A. For example, when the user identification unit 255 identifies the user, the user identification unit 255 transmits a prohibition signal to the smart speaker 200A to prohibit identification of the user, and the smart speaker 200A prohibits the identification of the user before the device identifies the user. Do not identify the user when receiving a signal.

Caller determination unit 279 determines, as the caller, the user with whom the user who has made the voice has a conversation, based on the voice information input from voice recognition unit 253 . Normally, when a user talks on the phone or the like, he/she utters the name, designation, etc. of the other party of the call in order to confirm the other party. When the name or appellation of a pre-registered user is extracted from the voice information, caller determination unit 279 determines the user with the extracted name or appellation as the caller. Caller determination section 279 outputs the user identification information of the user determined as the caller to device determination section 265 . Here, a case where user A utters user B's name will be described as an example.

The device determination unit 265 determines a device for collecting the caller's voice from among the other smart speakers 200A, 100B, and 200C as the pairing device. The device that collects the voice including the user identification information of the caller is the first device, and the device that collects the voice of the caller is the second device. For example, the device determining unit 265 identifies a device for collecting the voice of the caller by inquiring of the server 400 about the device for collecting the voice of the caller. The query to server 400 includes the caller's user identification information. Server 400 identifies users to smart speakers 200, 200A, 100B, and 200C based on voiceprints received from smart speakers 200, 200A, 100B, and 200C, respectively. Therefore, server 400 can identify a device among smart speakers 200, 200A, 100B, and 200C that collects the voice uttered by the user identified by the user identification information. Device determination unit 265 determines the device identified by server 400 as a pairing device. Device determination section 265 outputs device identification information for identifying a pairing device to audio information acquisition section 267 . Here, since the smart speaker 200B collects the voice of the user B, the device determination unit 265 determines the smart speaker 200B as the pairing device. Device determination unit 265 requests smart speaker 200B, which is a pairing device, to transmit audio information.

The response unit 261 functions when its own device is a pairing device. In other words, response unit 261 functions when communication unit 205 is requested to transmit voice information from any of smart speakers 200A, 200B, and 200C. In response unit 261 , device determination unit 265 receives a set of voice information and time information from speech recognition unit 253 and receives a set of user identification information and time information from user identification unit 255 . When the communication unit 205 is requested to transmit audio information from any of the other smart speakers 200A, 200B, and 200C, the response unit 261 then transmits the time information, the audio information paired with the time information, The user identification information paired with the time information is transmitted to one of the smart speakers 200A, 200B, and 200C that has requested transmission of the audio information.

Note that the device determination unit 265 controls the communication unit 205 to transmit an inquiry command to each of the smart speakers 200A, 100B, and 200C. The device that has received the call may be determined as the pairing device. The inquiry command includes the user identification information of the caller.

The response unit 261 functions when the communication unit 205 receives an inquiry command from any of the other smart speakers 200A, 200B, 200C. In response unit 261 , device determination unit 265 receives a set of voice information and time information from speech recognition unit 253 and receives a set of user identification information and time information from user identification unit 255 . When communication unit 205 receives an inquiry command from any of smart speakers 200A, 200B, and 200C, response unit 261 receives the user identification information included in the inquiry command and the user identification information input from user identification unit 255. match, it responds to the device that sent the inquiry command. When responding to an inquiry command, the response unit 261 then transmits the time information, the voice information paired with the time information, and the user identification information paired with the time information to the device that has sent the inquiry command. Send.

Job generation portion 257 receives a set of voice information and time information from voice recognition portion 253 and receives a set of user identification information and time information from user identification portion 255 . Job generation unit 257 may receive a plurality of sets of voice information and time information and a plurality of sets of user identification information and time information. Job generator 257 uses time information to distinguish between multiple pieces of audio information.

Job generation unit 257 associates the user identification information and the voice information that are paired with the same time information. In other words, job generation unit 257 associates the voice with the user who uttered the voice. Specifically, the job generation unit 257 generates a voice record including voice information converted from voice, user identification information of the user who uttered the voice, and time information indicating the time when the voice was uttered. and adds the voice record to the voice table stored in EPROM 204 .

After the device identification information is input from device determination portion 265, voice information acquisition portion 267 receives time information, voice information, and user identification information from communication portion 205 from the pairing device specified by the device identification information. and add the voice record to the voice table stored in EPROM 204 .

Here, since the pairing device is the smart speaker 200B, the voice table contains the voice records corresponding to the voices collected by the smart speaker 200, which is the own device, and the voice records collected by the smart speaker 200B, which is the pairing device. A voice record corresponding to the voice is registered. The voice record corresponding to the voice collected by smart speaker 200 includes user A's user identification information, and the voice record corresponding to the voice collected by smart speaker 200B includes user B's user identification information.

Job generation unit 257 generates a job that defines processing to be executed by MFP 100 and conditions for executing the processing based on the voice information registered in the voice table. Job generating portion 257 includes process determining portion 271 , normal job generating portion 273 , requested job generating portion 275 , and keyword extracting portion 277 .

The processing determination unit 271 determines setting information based on the audio information. The setting information is information indicating processing to be executed by MFP 100 and conditions for executing the processing. When the audio information includes a file name for identifying data, processing determination unit 271 determines the data specified by the file name included in the audio information as setting information indicating data to be processed.

A keyword extraction unit 277 extracts keywords from the voice information using a keyword table pre-stored in the EPROM 204 . The keyword table associates configuration information with keywords associated with the configuration information. A keyword is one of related information related to setting information.

FIG. 5 is a diagram showing an example of a keyword table. The keyword table includes multiple keyword records that associate keywords with setting information. The setting information defines the process to be executed by MFP 100 or the conditions under which MFP 100 executes the process. The keyword record includes keyword items and setting information items. In the keyword record, a keyword is set in the keyword field, and a process name and/or conditions for executing the process are set in the setting information field. As an example of how the keyword table associates a keyword with a process to be executed by MFP 100, a keyword record including the keyword "scan" associates the keyword "scan" with a scan process that causes MFP 100 to read a document. The keyword table may associate one keyword with a process and conditions for executing the process. For example, for a keyword record containing the keyword "2in1", the number of images to be used as the basis for synthesis must be two as a condition for performing synthesis processing for synthesizing a plurality of images for the keyword "2in1". associate. The keyword table may associate multiple processes with one keyword. For example, a keyword record containing the keyword "copy" associates a copy action with the keyword "copy". Copy processing includes scan processing for causing MFP 100 to read a document and print processing for forming an image on paper.

As an example of how the keyword table associates a keyword with a condition for the MFP 100 to execute processing, the keyword "color" is associated with full color as a condition for print processing to be executed by the MFP 100 . As another example, for the keyword "Taro", the e-mail address "taro@aaa.cpm", which is the destination information assigned to the user as a condition for the MFP 100 to execute the data transmission process, is Associate. As the destination information, facsimile numbers and IP addresses are used in addition to e-mail addresses. Note that the keyword "Taro" indicates the name of a pre-registered user.

Returning to FIG. 4, the keyword extraction unit 277 compares the voice information with a plurality of keywords set in the keyword table. Extract the keyword record that the keyword is set. Keyword extraction portion 277 determines setting information set in the extracted keyword record, and outputs the determined setting information to processing determination portion 271 .

Process determination portion 271 generates a job defining a process to be executed by MFP 100 based on the setting information determined by keyword extraction portion 277 . Specifically, process determining portion 271 generates a job that causes MFP 100 to execute a process determined by the setting information under conditions determined by the setting information.

For example, when the audio information includes the keywords "copy" and "full color", the processing determination unit 271 determines setting information defining scan processing and print processing associated with the keyword "copy" from the keyword table, Determine setting information that defines conditions for executing scan processing and print processing in full color associated with "full color". Processing determination unit 271 determines a job that causes MFP 100 to execute a scan process of reading a document in full color and a print process of forming an image of full-color image data output by the scan process on paper in full color.

Further, when the voice information includes the keywords "send" and "Taro", the process determination unit 271 specifies the scanning process and the data transmission process associated with the keyword "send" from the keyword table, and the keyword "send" from the keyword table. Setting information indicating a destination associated with "Taro" is specified. In this case, the processing determination unit 271 performs scanning processing for reading the document in monochrome, and sends the monochrome image data output after the scanning processing to an electronic mail address registered for the user whose name is Taro. A job is generated that causes MFP 100 to execute a data transmission process for attaching the data to an e-mail and transmitting the data.

Although an example in which the processing determining unit 271 and the keyword extracting unit 277 determine setting information from voice information using a keyword table has been described, deep learning technology using a neural network or the like is employed to obtain voice information and setting information. A model may be generated by learning the relationship between and, and the setting information may be determined from the voice information using the model.

When generating a job, process determination unit 271 determines one or more pieces of user identification information paired with one or more pieces of audio information used to generate the job. For example, if the job is generated based on one or more voice information paired with the user identification of the first user, then the user identification of the first user is determined. If the job is generated based on the one or more voice information paired with the user identification information of the first user and the one or more voice information paired with the user identification information of the second user, the user identification of the first user determining the information and the user identification of the second user; Process determining portion 271 outputs a set of a job and one or more pieces of determined user identification information to requested job generating portion 275 and normal job generating portion 273 . Here, user A who uttered the sound collected by smart speaker 200, which is the device itself, is the first user, and user B who uttered the sound collected by smart speaker 200B, which is the pairing device, is the second user. be.

Requested job generating portion 275 receives a set of a job and one or more pieces of user identification information from process determining portion 271 and generates a requested job. The requested job is a job that the second user causes MFP 100 to execute in order for the second user to execute the work requested by the first user who is the requester to the second user who is the licensor. In other words, the requested job is a job whose execution is instructed to MFP 100 by the second user. Therefore, the requested job is a job input from process determining portion 271 in which an instruction by the second user is set as a start condition for MFP 100 to execute the process determined by the job. The instruction set as the start condition includes an operation input by the second user in order to be authenticated.

Requested job generating portion 275 generates voice information including a requested character string among the one or more voice information input from voice recognition portion 253 when there is a plurality of pieces of user identification information included in each of the one or more voice information. Create a request job if exists. The request character string includes words used when requesting work to others. For example, the request character string includes "please", "please", "shiro", and the like. A request character string may be determined in advance. Alternatively, the CPU 201 may determine the request character string by having the CPU 201 learn conversations between a plurality of users using the MFP 100 .

However, requested job generating portion 275 does not generate a requested job if character information including a permitted character string does not exist in one or more pieces of voice information input from voice recognition portion 253 . The authorization character string includes words used when receiving a request from another person. The permission string includes "understood", "understood", "accept", and the like. The permission character string may be determined in advance. Also, the permission string may be determined by learning conversations between multiple users of MFP 100 .

When there is voice information including the requested character string, requested job generation unit 275 specifies user identification information paired with the voice information as user identification information of the first user. Here, since user A calls user B to request work, user A is determined to be the first user. Further, when voice information including the permission character string exists, requested job generating portion 275 specifies user identification information paired with the voice information as user identification information of the second user. Here, user B is determined to be the second user. Requested job generation portion 275 sets, in the job input from process determination portion 271, acceptance of an instruction from the second user as a start condition for MFP 100 to execute the process defined by the job. Generate a job. Acceptance of the instruction by the second user includes the second user being authenticated. Requested job generating portion 275 outputs a set of the user identification information of the first user and the requested job to job transmitting portion 259 .

When there is voice information including the permission character string, the requested job generation unit 275 determines voice information paired with the time information indicating a time earlier than the time indicated by the time information paired with the voice information. do. Then, requested job generating section 275 extracts character information including the requested character string from the determined voice information. As a result, when there is voice information containing a permission character string, it is sufficient to generate a requested job, so there is no need to extract keywords from all of the voice information, and the load can be reduced as much as possible. In this case, it is preferable to start extracting keywords and generating jobs after confirming the presence of voice information containing the permitted character string.

When a job is input from process determining portion 271 and the requested job is not to be generated, requested job generating portion 275 outputs a normal generation instruction to normal job generating portion 273 . Normal job generation portion 273 receives a set of a job and one or more pieces of user identification information from process determination portion 271 . When a normal generation instruction is input from requested job generation portion 275, normal job generation portion 273 generates a normal job based on the job input from process determination portion 271 and one or more pieces of user identification information. A normal job is a job other than a requested job.

Normal job generation portion 273 determines, as an instruction user, a user whose user identification information is associated by voice information including an instruction character string among the one or more pieces of user identification information input from process determination portion 271 . The instruction string is a word that indicates the content of the processing. For example, the instruction string is "I want to do it", "I want to do it", and the like. In addition, normal job generation portion 273 may determine, as the designated user, the user identified by the user identification information having the largest number of keywords among the one or more pieces of user identification information input from process determination portion 271. . In this case, using a plurality of voice information each including a plurality of keywords used when the processing determination unit 271 generates a job, the keywords corresponding to each user identification information are aggregated, and the number of keywords corresponding to the user identification information is calculated. should be sought. Normal job generation portion 273 generates a normal job by setting an instruction by the instructing user as a start condition for MFP 100 to execute a process determined by the job in the job input from process determination portion 271 . Acceptance of the instruction by the instruction user includes authentication of the instruction user. Normal job generating portion 273 outputs the normal job to job transmitting portion 259 . As a result, job transmitting portion 259 transmits the normal job to MFP 100 . In MFP 100 , when a normal job is received from smart speaker 200 , the normal job is stored in HDD 115 , and when the instructing user operates operation unit 163 , the normal job is set to an executable state.

Job transmission portion 259 transmits a job to MFP 100 . When a set of the requested job and the first user's user identification information is input from requested job generating portion 275 , job transmitting portion 259 transmits the set of the requested job and the first user's user identification information to MFP 100 . Further, when a normal job is input from normal job generating portion 273 , job transmitting portion 259 transmits the normal job to MFP 100 .

Job transmission portion 259 includes an operating user notification portion 281 . When a set of the requested job and the user identification information of the first user is transmitted to MFP 100 , operating user notification portion 281 notifies the second user that the requested job can be executed by MFP 100 . The second user is specified by the user identification information of the second user set as the start condition in the requested job. For example, operating user notification unit 281 controls communication unit 205 to send an e-mail including a message indicating that MFP 100 has an executable requested job to the second user. The e-mail address of the second user is pre-stored. Note that the notification method is not limited to e-mail and may be message transmission. Thus, the second user can be notified that he or she should input an operation for causing MFP 100 to execute the requested job.

FIG. 6 is a block diagram showing an example of functions of CPU 111 of MFP 100 according to the first embodiment. The functions shown in FIG. 6 may be realized by hardware, or are functions realized by CPU 111 provided in MFP 100 by executing a job control program stored in ROM 113, HDD 115, and CD-ROM 118. be. The Job Control Program is part of the Job Generator Program. Referring to FIG. 6 , CPU 111 includes an operating user specifying portion 51 , a setting portion 53 , a job control portion 55 and a client notifying portion 57 .

Job control portion 55 includes a job reception portion 81 , an association portion 83 , and a job execution portion 85 . Job receiving portion 81 controls communication I/F portion 112 and receives a job transmitted from smart speaker 200 . When job receiving portion 81 receives a set of a requested job and the user identification information of the first user from smart speaker 200 , job receiving portion 81 stores the requested job in HDD 115 . In addition, when job receiving portion 81 receives a normal job from smart speaker 200 , job receiving portion 81 stores the normal job in HDD 115 .

Association unit 83 associates the requested job with the first user when communication I/F unit 112 receives a set of the requested job and the first user's user identification information from smart speaker 200 . Specifically, association unit 83 generates a user record including job identification information for identifying the requested job stored in HDD 115 and user identification information of the first user, and stores the user record in HDD 115 .

The job execution unit 85 executes jobs by controlling hardware resources. The hardware resources include communication I/F section 112 , HDD 115 , facsimile section 116 , automatic document feeder 120 , document reading section 130 , image forming section 140 , paper feeding section 150 , post-processing section 155 and operation panel 160 . Jobs include, for example, copy jobs, print jobs, scan jobs, facsimile transmission jobs, and data transmission jobs. Jobs that can be executed by job execution unit 85 are not limited to these, and may include other jobs. The copy job includes scan processing for causing document reading unit 130 to read the document, and print processing for causing image forming unit 140 to form an image of data output by document reading unit 130 reading the document. The print job includes print processing for causing image forming portion 140 to form an image of data stored in HDD 115 and print data externally received by communication I/F portion 112 on paper. The scan job includes scan processing for causing document reading unit 130 to read the document, and output processing for outputting image data output by document reading unit 130 reading the document. The output processing includes data storage processing for storing data in HDD 115 and data transmission processing for transmitting data to communication I/F portion 112 to the outside. The facsimile transmission job includes scanning processing for causing document reading unit 130 to read a document, and facsimile transmission processing for causing document reading unit 130 to read and output data from the document and transmit the data to facsimile unit 116 . The data transmission job includes a data transmission process of controlling communication I/F unit 112 to transmit data stored in HDD 115 or data read and output by document reading unit 130 to another computer.

Operating user identification unit 51 identifies an operating user who operates MFP 100 . For example, when a user operates operation panel 160 and the user inputs user identification information to operation unit 163, the user identified by the user identification information input to operation unit 163 is identified as the operating user. Further, when MFP 100 includes a card reader, operating user identification unit 51 allows the user to cause the card reader to read the user identification information stored in the card when the card reader reads the user identification information stored in the card. accept. The operating user identification unit 51 identifies the user identified by the user identification information read by the card reader as the operating user. The card reader may be a magnetic card reader, or may be a wireless communication device that communicates according to the NFC (Near field communication) standard. When specifying the operating user, operating user specifying portion 51 outputs the user identification information of the operating user to setting portion 53 .

When the user identification information of the operating user is input from operating user specifying portion 51 , setting portion 53 stores in HDD 115 a requested job or a normal job whose start condition is set to an instruction by the user of the user identification information. determine whether there is If such a requested job or normal job is stored in HDD 115, setting unit 53 sets the requested job or normal job to an executable state. In particular, when setting the requested job to an executable state, setting portion 53 displays a setting button for accepting an instruction from the second user on display portion 161 . A setting button is assigned a command for executing a requested job.

FIG. 7 is a diagram showing an example of a login screen. Login screen 500 shown in FIG. 7 is displayed on display portion 161 when the second user operates operation portion 163 and MFP 100 identifies the user. Referring to FIG. 7, login screen 500 includes a plurality of selection buttons 503 and a setting button 501 for selecting various processes. Setting button 501 is displayed when the second user operates operation unit 163 and MFP 100 identifies the second user. Accordingly, when no requested job is stored in HDD 115 , login screen 500 does not include setting button 501 . The setting button 501 includes a character string “work request” and a character string “(Mr. XX)”. The character string "(Mr. XX)" includes the first user name, which is the user identification information of the first user. Therefore, the second user who sees the setting button 501 can know that the setting button 501 is a button for enabling the requested job for performing the work requested by the first user. .

Returning to FIG. 6, instead of displaying setting button 501 on display portion 161, setting portion 53 displays job identification information for identifying one or more jobs stored in HDD 115 in a selectable manner. 161 may be displayed. In this case, when a requested job or a normal job is stored in HDD 115, display portion 161 displays job identification information of the requested job and the normal job in a selectable manner. When the user selects the job identification information of the requested job, the setting unit 53 sets the requested job or the normal job to an executable state in the same manner as when the set button 501 is instructed.

Setting unit 53 changes the setting values of the job when accepting an operation to change the setting values of the job set to be executable. Thereby, the second user can change the setting values erroneously set in the requested job to correct values, and can add and set setting values that have not been set.

Also, a predetermined time may elapse after a requested job whose start condition is set to an instruction by the second user is stored in HDD 115 without being executed. In this case, setting unit 53 does not set the requested job to an executable state even when the user identification information of the second user is input from operating user specifying unit 51 . Specifically, setting portion 53 does not display on display portion 161 setting button 501 for accepting the user's instruction to execute the requested job. If the requested job has not been executed after a predetermined period of time has elapsed, there is a high probability that the requested job has become unnecessary. is. Storage resources can be effectively used by deleting unnecessary data.

Job execution portion 85 included in job control portion 55 executes the requested job when receiving an execution operation input to operation portion 163 by the second user to instruct execution of the requested job. In addition, job executing portion 85 executes a normal job when receiving a normal job execution operation input to operation portion 163 by the instructing user. The designated user is identified by the user identification information associated by the association unit 83 with the job identification information of the normal job.

The requester notification unit 57 notifies the first user that the requested job has been executed in response to the execution of the requested job. The first user is identified by the user identification information associated by the association unit 83 with the job identification information of the requested job. The requester notification unit 57 notifies the first user that the requested job has been executed by a method registered in advance. For example, the requester notification unit 57 sends an e-mail message indicating that the requested job has been executed by the second user to a destination registered in advance for the first user.

FIG. 8 is a flow chart showing an example of the flow of job generation according to the first embodiment. Job generation is a process executed by the CPU 201 of the smart speaker 200 by executing a job control program stored in the ROM 202 and the EPROM 204 . The Job Control Program is part of the Job Generator Program.

Referring to FIG. 8, CPU 201 included in smart speaker 200 determines whether or not voice has been received. Specifically, when the CPU 201 accepts audio data output from the microphone 208, it determines that the audio has been accepted. If CPU 201 receives audio data from microphone 208, CPU 201 advances the process to step S02, but otherwise advances the process to step S07. Here, a case of accepting user A's voice will be described as an example.

In step S02, CPU 201 recognizes the voice and advances the process to step S03. Specifically, CPU 201 recognizes the voice specified by the voice data accepted in step S01, and converts the voice into voice information composed of characters. In step S03, voice information converted from voice is determined, and the process proceeds to step S04.

In step S04, user identification processing is executed, and the processing proceeds to step S05. Details of the user specifying process will be described later, but this is a process of specifying the user who has uttered the voice specified by the voice data. Here, user A is identified. In step S05, CPU 201 generates a voice record and advances the process to step S06. Specifically, CPU 201 generates a voice record including the voice information determined in step S03, the user identification information of the user specified in step S04, and the date and time when the voice data was received. A voice record is information that associates voice information converted from voice, user identification information of the user who uttered the voice, and the date and time when the voice was uttered. In step S06, the voice record is added to the voice table stored in HDD 115, and the process proceeds to step S09.

In step S09, it is determined whether or not the voice information determined in step S03 includes user identification information. If the voice information contains user identification information, the process proceeds to step S10; otherwise, the process proceeds to step S11. Here, since the user A speaks the name of the user B during the call, the name of the user B is extracted from the voice information as the user identification information. In step S10, a pairing process is executed, and the process proceeds to step S11. Although the details of the pairing process will be described later, it is a process of determining, as a pairing device, a device that collects the voice uttered by the user specified by the user identification information extracted from the voice information in step S09. Here, smart speaker 200B is determined as the pairing device.

On the other hand, in step S07, it is determined whether or not there is a pairing device. If step S10 has been performed and a pairing device has been determined, the process proceeds to step S08, otherwise the process returns to step S01. In step S08, voice information, user identification information, and time information are acquired from the pairing device, and the process returns to step S01. Specifically, the CPU 111 requests the smart speaker 200B, which is the pairing device, to transmit the audio information. When smart speaker 200B, which is a pairing device, is requested to send voice information, it returns voice information, user identification information, and time information. Add to the voice table stored in HDD 115 .

When the processes after step S11 are executed, the voice records registered in the voice table are processed. Specifically, in the processing after step S11, the voice record newly registered in the voice table before step S11 is executed is processed. Therefore, all voice records registered in the voice table are processed. In the voice table, step S03 is performed and the voice record including voice information determined from the voice uttered by user A collected by the smart speaker 200, and step S08 is performed and the smart speaker 200A, which is a pairing device, is stored. and a voice record including voice information determined from the voice uttered by user B collected in .

In step S11, it is determined whether or not the voice information contains a permitted character string. There is a high probability that the audio information acquired in step S07, in other words, the audio information determined from the audio collected by the smart speaker 200B, which is the pairing device, contains the permitted character string. The permission character string is a predetermined character string including words used when receiving a request from another person. Specifically, the permission character string includes "understand", "understand", "accept", and the like. If the voice information acquired in step S08 contains the permitted character string, the process proceeds to step S12; otherwise, the process proceeds to step S17.

In step S12, the user of the user identification information paired with the permission character string is determined to be the second user, and the process proceeds to step S13. When the process proceeds to step S13, the voice information to be processed includes the permission character string. Therefore, the second user is the user who utters the permission character string and is the acceptor who accepts the work request from the first user. Here, user B is determined to be the second user.

In step S13, a job generation sub-process is executed, and the process proceeds to step S14. The details of the job generation sub-processing will be described later, but this is a process of generating a job based on one or more voice records contained in the voice table stored in the EPROM 204, and determining the first user who requested the work from the second user. is. Here, user A is determined to be the first user. In step S14, a requested job is generated, and the process proceeds to step S15. A requested job is generated based on the job generated by executing the job generation sub-process and the second user determined in step S12. A requested job is a job for which acceptance of an operation by the second user is set as a condition for starting MFP 100 to execute processing defined by the requested job. Specifically, the requested job is generated by setting the acceptance of the operation by the second user determined in step S12 as a start condition in the job generated in step S13. In the next step S15, CPU 201 controls communication unit 205 to transmit the requested job to MFP 100, and the process proceeds to step S16. In step S16, the existence of the requested job is notified to the second user, and the process ends. Specifically, an email containing a message indicating that executable requested jobs are stored in MFP 100 and having the email address of user B, who is the second user, set as the destination address is generated. will send the email.

On the other hand, in step S17, it is determined whether or not the voice information contains the designated character string. If the voice information contains the designated character string, the process proceeds to step S18; otherwise, the process returns to step S01. In step S18, the user who uttered the instruction character string is determined as the instruction user, and the process proceeds to step S19. The user identified by the user identification information associated with the voice information to be processed is determined as the instruction user who uttered the instruction character string. When the process proceeds to step S18, there is a high probability that the audio information determined in step S03, in other words, the audio information determined from the audio collected by smart speaker 200, includes the permitted character string. In step S19, a job generation sub-process is executed, and the process proceeds to step S16.

In step S20, a normal job is generated, and the process proceeds to step S21. A normal job is a job for which acceptance of an operation by an instructing user is set as a condition for starting MFP 100 to execute processing defined by the normal job. Specifically, a normal job is generated by setting the job generated in step S19 as a start condition to accept an operation by the instruction user determined in step S18. In the next step S21, communication unit 205 is controlled to transmit the normal job to MFP 100, and the process proceeds to step S22. In step S22, the instruction user is notified of the existence of the normal job, and the process ends. Specifically, an e-mail containing a message indicating that executable normal jobs are stored in MFP 100 and having the e-mail address of the designated user set as the destination address is generated, and communication unit 205 generates the e-mail. is sent.

FIG. 9 is a flowchart showing an example of the flow of user identification processing. The user identification process is a process executed in step S04 of FIG. Speech is accepted before the user identification process is executed. Referring to FIG. 9, CPU 201 inquires of server 400 about the user who has uttered the voice (step S31). By extracting a voiceprint from the voice and transmitting the extracted voiceprint to the server 400, the user of the voiceprint is queried. In response to the inquiry, server 400 returns the user identification information of the user identified by the voiceprint, so CPU 201 waits in step S32 until the user identification information is received from server 400 (NO in step S32). , the user identification information is received from server 400 (YES in step S32), the process proceeds to step S33.

Server 400 transmits an arbitration instruction to multiple smart speakers 200, 200A, 200B, and 200C when multiple devices collect the voice of the same user. The arbitration instructions include device identification information for identifying multiple devices that collect the same user's voice. Therefore, CPU 201 determines whether or not an arbitration instruction has been received from server 400 in step S33. If the arbitration instruction is received, the process proceeds to step S34; otherwise, the process proceeds to step S39.

Volumes are compared in step S34. Specifically, the CPU 201 acquires the volume of the sound corresponding to the sound information from all the other devices among the devices specified by the device identification information included in the arbitration instruction. In step S35, the CPU 201 determines whether or not the volume of its own device is the maximum. If the volume of its own device is the maximum, the process proceeds to step S39; otherwise, the process proceeds to step S36. In step S36, it is determined whether or not there is a device with the same volume. If there is a device with the same volume, the process proceeds to step S37; otherwise, the process ends.

In step S37, it is determined whether or not a prohibition signal has been received from any of the other devices. If the inhibit signal is received, the process ends; otherwise, the process proceeds to step S38. In step S38, a prohibition signal is transmitted to all other devices, and the process proceeds to step S39. In step S39, the user identified by the user identification information received in step S32 is specified, and the process returns to the job generation process.

FIG. 10 is a flowchart showing an example of the flow of pairing processing. A pairing process is a process performed in FIG.8 S10. User identification information is extracted from the voice information in step S09 before the pairing process is executed. Referring to FIG. 10, CPU 201 inquires of server 400 about another device that collects the voice uttered by the user identified by the user identification information from the voice information. Specifically, CPU 201 queries other devices by transmitting user identification information to server 400 . In response to the inquiry, if there is a device among smart speakers 200A, 200B, and 200C that collects the voice uttered by the user identified by the user identification information, server 400 returns the device identification information of that device. CPU 201 determines whether device identification information has been received from server 400 in step S42. If the device identification information has been received, the process proceeds to step S43; otherwise, the process returns to the job generation process. In step S43, the device identified by the device identification information received from server 400 is determined as the pairing device, and the process returns to the job generation process.

FIG. 11 is a flowchart showing an example of the flow of job generation sub-processing. The job generation sub-process is a process executed in steps S13 and S19 of FIG. In step S06 before the job generation sub-process is executed, HDD 115 stores a voice record including character information including a permission character string, and a voice record including character information including an instruction character string is stored in HDD 115. There are cases. Hereinafter, an audio record including audio information including a permission character string or an audio record including audio information including an instruction character string is referred to as a record to be processed.

Referring to FIG. 11, CPU 201 reads out the voice record at the time immediately preceding the record to be processed (step S51). Specifically, the CPU 201 includes time information indicating the nearest time earlier than the time indicated by the time information included in the record to be processed among the plurality of voice records included in the voice table stored in the EPROM 204. Read voice record. In the next step S52, audio information contained in the audio record is specified.

Then, it is determined whether or not the voice information contains the requested character string (step S53). The request character string includes words used when requesting work to others. A request character string may be determined in advance. If the voice information contains the requested character string, the process proceeds to step S54; otherwise, the process proceeds to step S55. In step S54, the user who uttered the requested character string is determined as the first user, and the process proceeds to step S55. The user identified by the user identification information included in the voice record read in step S51 is determined as the first user.

In step S55, it is determined whether or not the setting information has been determined from the voice information. When the audio information includes the file name of data to be processed, setting information specifying the data to be processed is determined. If the setting information is determined from the audio information, the process proceeds to step S58; otherwise, the process proceeds to step S56.

In step S56, the voice information is compared with the keyword and the process proceeds to step S57. Specifically, the pronunciation of at least part of the voice information is compared with the pronunciation of at least part of the keywords registered in the keyword table stored in EPROM 204 . In the next step S57, as a result of the comparison, it is determined whether or not there is a keyword including a character string having the same or similar pronunciation as at least part of the voice information. If such a keyword exists, the process proceeds to step S58; otherwise, the process proceeds to step S59.

In step S58, setting information is determined, and the process proceeds to step S59. When the process proceeds from step S55, the setting information determined in step S55 is determined, and when the process proceeds from step S57, the setting information associated with the keyword is set.

In step S59, it is determined whether or not the process is determined based on the setting information determined in step S58. If the process is determined, the process proceeds to step S60; otherwise, the process returns to step S51.

In step S60, under the conditions determined by the setting information determined in step S58, a job is generated for causing MFP 100 to execute the process determined by the setting information determined in step S58, and the process proceeds to step S61. In step S61, it is determined whether or not the job has been completed. A job may define multiple processes, and a job is considered complete when all of the multiple processes have been defined. If the job is completed, the process returns to the job generation process; otherwise, the process returns to step S51.

FIG. 12 is a flowchart illustrating an example of the flow of job execution processing. The job execution process is a function realized by CPU 111 of MFP 100 by executing a job execution program stored in ROM 113 , HDD 115 and CD-ROM 118 . The job executor is part of the job generator. Referring to FIG. 11, CPU 111 of MFP 100 determines whether or not a requested job has been received (step S71). CPU 111 controls communication I/F unit 112 to determine whether the requested job and the user identification information of the first user have been received from smart speaker 200 . If the requested job has been received, the process proceeds to step S72; otherwise, the process proceeds to step S73. In step S72, the requested job and the user identification information of the first user received together with the requested job are stored in HDD 115, and the process proceeds to step S73. User identification information is stored in association with the requested job.

In step S73, it is determined whether or not a normal job has been received. CPU 111 controls communication I/F unit 112 to determine whether or not a normal job has been received from smart speaker 200 . If the normal job has been received, the process proceeds to step S74; otherwise, the process proceeds to step S75. In step S74, the normal job is stored in HDD 115, and the process proceeds to step S75.

In step S75, it is determined whether operation unit 163 has received an operation input by the user. If the operation is accepted, the process proceeds to step S76; otherwise, the process returns to step S71. In step S76, the user who operates operation unit 163 is identified, and the process proceeds to step S77. When the user inputs user identification information to operation unit 163, the user identified by the user identification information input to operation unit 163 is identified as the operating user. Further, an operation by the user to cause the card reader to read the user identification information stored in the card is accepted, and the user identified by the user identification information read by the card reader is specified as the operating user.

In step S77, it is determined whether or not there is a requested job corresponding to the identified user. It is determined whether or not a requested job whose start condition is set to the specified user's instruction is stored in HDD 115 . If the requested job exists, the process proceeds to step S78; otherwise, the process proceeds to step S81.

In step S78, it is determined whether or not the elapsed time since the requested job was received is within a predetermined time. If it is within the predetermined time, the process proceeds to step S79, but if the predetermined time has passed, the process ends. The predetermined time is a predetermined value. If a predetermined time has passed since the requested job was received, there is a high probability that the requested job is no longer needed. Therefore, it is possible to prevent unnecessary requested jobs from being executed.

In step S79, an execution instruction process is executed, and the process proceeds to step S80. Details of the execution instruction process will be described later. In step S80, the first user is notified that the requested job has been executed, and the process ends. The first user is identified using the user identification information stored in HDD 115 in step S72 together with the requested job identified in step S77. For example, an e-mail containing a message indicating that the requested job has been executed by the second user and including the e-mail address of the first user as the destination is generated, and the e-mail is transmitted via communication I/F unit 112. be.

In step S81, it is determined whether or not there is a normal job corresponding to the user identified in step S76. It is determined whether or not a normal job for which the specified user's instruction is set as a start condition is stored in HDD 115 . If the normal job exists, the process proceeds to step S82; otherwise, the process ends. In step S82, it is determined whether or not the elapsed time since the normal job was received is within a predetermined time. If it is within the predetermined time, the process proceeds to step S83; otherwise, the process ends. If a predetermined time has passed since the normal job was received, there is a high probability that the normal job is no longer needed. Therefore, unnecessary normal jobs can be prevented from being executed. In step S83, an execution instruction process is executed, and the process ends. When the execution instruction process is executed, the normal job is targeted for processing, and the normal job is executed.

FIG. 13 is a flowchart illustrating an example of the flow of execution instruction processing. The execution instruction process is a process executed in step S79 or S83 of FIG. The requested job is determined before the execution instruction process is executed. Referring to FIG. 12, CPU 111 determines whether or not there is a requested job. It is determined whether or not there is a requested job corresponding to the user identified as the user operating MFP 100 (step S91). It is determined whether HDD 115 stores a requested job whose start condition is an instruction from the user who operates MFP 100 . If the requested job exists, the process proceeds to step S93; otherwise, the process proceeds to step S94.

In step S92, the setting button 501 is displayed on the display section 161, and the process proceeds to step S93. A setting button 501 is a button for accepting an operation for setting the requested job to an executable state, and is associated with a command for setting the requested job to an executable state. In step S93, it is determined whether or not the setting button 501 has been designated. A standby state is maintained until the setting button 501 is designated (NO in step S93), and if the setting button 501 is designated (YES in step S93), the process proceeds to step S95. Note that if the setting button 501 is not instructed even after the standby time has elapsed since the setting button 501 was displayed, the processing may be terminated. The waiting time is a predetermined time.

On the other hand, in step S94, it is determined whether or not the normal job has been selected. With the job selection screen displaying a list of jobs associated with the user operating MFP 100 displayed, it is determined whether or not a normal job has been selected. The process waits until the normal job is selected, and if the normal job is selected, the process proceeds to step S95. Note that the process may proceed to step S95 when a job different from the normal job is selected. Further, the process may be terminated when the normal job is not selected even after the waiting time has elapsed since the job selection screen was displayed. The waiting time is a predetermined time.

In step S95, a setting screen for setting setting values is displayed, and the process proceeds to step S96. In step S96, the process branches depending on the operation input by the user. If the operation input by the user is the execution operation, the process proceeds to step S95, and if the operation is the setting operation, the process proceeds to step S98. An execution operation is an operation for instructing execution of a job. The setting operation is an operation for setting job setting values. In step S98, the setting value is set according to the setting operation, and the process returns to step S96. In step S97, the requested job or normal job is executed, and the process returns to the job execution process.

As described above, in the speech processing system 1 according to the first embodiment, the speech processing system 1 includes a plurality of smart speakers 200, 200A, 200B, and 200C, and a plurality of smart speakers 200, 200A, 200B, and 200C. For example, the smart speaker 200 identifies the user who uttered the voice collected by the microphone 208, the first user identified voice and other smart speakers 200, 200A, 200B, 200C, e.g., smart speaker 200A. Based on the collected voice uttered by the second user, the job requested by the first user to be executed by the second user is generated as a job for MFP 100 to execute. Therefore, based on the voice collected by the microphone 208 and the voice uttered by the second user collected by the smart speaker 200A, the requested job that the first user has requested the second user to execute is generated. Therefore, since the requested job is generated from the conversation of the first user and the second user who exist at distant locations, the operation for generating the job can be simplified.

Further, each of the smart speakers 200, 200A, 200B, and 200C, for example, the smart speaker 200, further detects the user identification information of any one of a plurality of pre-registered users from the voice identifying the first user. determines the user identified by the detected user identification information as the second user, and collects the voice uttered by the second user among the other smart speakers 200A, 200B, and 200C, for example, The smart speaker 200B is determined as the pairing device. Therefore, it is possible to easily determine the smart speaker 200B that collects the voice uttered by the second user who converses with the first user.

Also, when the smart speakers 200B and 200C collect the voice uttered by the second user, only one of the smart speakers 200B and 200C is determined to be the device for collecting the voice uttered by the second user. Therefore, when the smart speakers 200B and 200C collect the voice uttered by the second user and the smart speaker 200 determines the voice collection device for collecting the voice uttered by the second user, the smart speakers 200B and 200C The one determined in (1) is determined to be the voice collecting device that collects the voice uttered by the second user. Therefore, the accuracy of voice recognition can be improved.

Further, smart speaker 200 generates a request job when the voice collected by smart speaker 200B, which is a pairing device, indicates the content of the permission. Therefore, it is possible to prevent the job from being generated when the second user does not accept the request from the first user.

<Second Embodiment>
MFP 100 in the second embodiment functions as an image processing apparatus as well as a job generation apparatus as does MFP 100 in the first embodiment. The general outline of the speech processing system 1 in the second embodiment is the same as the general outline of the speech processing system 1 in the first embodiment shown in FIG. The hardware configuration of MFP 100 in the second embodiment is the same as the hardware configuration shown in FIG. The hardware configuration of each of smart speakers 200, 200A, 200B, and 200C in the second embodiment is the same as the hardware configuration shown in FIG. Therefore, descriptions thereof will not be repeated.

FIG. 14 is a block diagram showing an example of functions of the CPU 201 included in the smart speaker 200 according to the second embodiment. The functions shown in FIG. 14 are different from the functions shown in FIG. The point is that an information transmission unit 291 is added. Since other functions are the same as those shown in FIG. 4, the description will not be repeated here. Voice information transmitting portion 291 receives a set of voice information and time information from voice recognition portion 253 and receives a set of user identification information and time information from user specifying portion 255 . Voice information transmitting portion 291 transmits to MFP 100 user identification information, voice information, and time information paired with the same time information. Specifically, voice information transmitting portion 291 controls communication portion 205 to transmit user identification information, voice information, and time information to MFP 100 .

FIG. 15 is a block diagram showing an example of functions of CPU 111 of MFP 100 according to the second embodiment. The functions shown in FIG. 15 are different from the functions shown in FIG. The difference is that the job reception section 81 is changed to a job reception section 81A. Since other functions are the same as those shown in FIG. 6, the description will not be repeated here.

The voice information acquisition unit 71 in the second embodiment acquires voice information, user identification information, and time information from any one of smart speakers 200, 200A, 200B, and 200C. Specifically, voice information acquisition unit 71 in the second embodiment controls communication I/F unit 112 so that communication I/F unit 112 can Acquire received audio information, user identification information, and time information. Voice information acquisition unit 71 in the second embodiment stores voice tables corresponding to smart speakers 200, 200A, 200B, and 200C in HDD 115, respectively. Therefore, the voice information acquisition unit 71 in the second embodiment acquires voice information, user identification information, and time information received from any of the smart speakers 200, 200A, 200B, and 200C, for example, the smart speaker 200. Adds the containing voice record to the voice table corresponding to smart speaker 200 stored on HDD 115 .

Further, when device determination unit 265 identifies a device, voice information acquisition unit 71 in the second embodiment performs the Get the voice record from the voice table corresponding to the identified device.

The job generation unit 257, the caller determination unit 263, the device determination unit 265, and the operating user notification unit 281 in the second embodiment are the job generation unit 257 of the CPU 201 included in the smart speaker 200 in the first embodiment, It has the same functions as caller determination unit 263, device determination unit 265, and operating user notification unit 281, respectively. That is, the job generation unit 257 in the second embodiment generates the requested job based on the voice information, user identification information, and time information registered in the voice table corresponding to any one of the smart speakers 200, 200A, 200B, and 200C. and determine the first user. Job generation portion 257 in the second embodiment outputs the requested job and the user identification information of the first user to job control portion 55 . Further, the job generation unit 257 in the second embodiment generates a normal job based on the voice information, user identification information, and time information registered in the voice table corresponding to any one of the smart speakers 200, 200A, 200B, and 200C. , and outputs the normal job to job control unit 55 . Job accepting portion 81A of job control portion 55 in the second embodiment accepts the requested job and the user identification information of the first user when job creating portion 73 creates the requested job. When the job generation unit 73 in generates a normal job, the normal job is accepted.

The operating user notification unit 281 according to the second embodiment has the same function as the operating user notification unit 281 of the CPU 201 included in the smart speaker 200 according to the first embodiment. That is, when a requested job is generated by job generating portion 257 in the second embodiment, operating user notifying portion 281 in the second embodiment notifies the second user that the requested job can be executed by MFP 100. Notice.

In the audio processing system 1 according to the second embodiment, the MFP 100 has part of the functions of the smart speaker 200 of the audio processing system 1 according to the first embodiment. Therefore, the functions of the smart speakers 200, 200A, 200B, and 200C in the second embodiment may be low, and the system configuration can be simplified.

<First modification>
Server 400 in the first modification has some of the functions of MFP 100 in the second embodiment. Server 400 in the first modification functions as a job generation device. That is, server 400 in the first modification functions as an AI (Artificial Intelligence) assistant for smart speakers 200, 200A, 200B, and 200C. An AI assistant provided in the server 400 is registered in advance in each of the smart speakers 200, 200A, 200B, and 200C. Each of the smart speakers 200, 200A, 200B, and 200C includes at least a microphone 208 and a communication unit 205, converts sound collected by the microphone 208 into sound data that is electronic data, and transmits the data to the server 400. . Server 400 recognizes the voice and identifies the user based on the voice data received from each of smart speakers 200 , 200 A, 200 B, and 200 C, and transmits voice information, user identification information, and time information to MFP 100 .

In the audio processing system 1 according to the first modification, the smart speakers 200, 200A, 200B, and 200C may have low functions, and the system configuration can be simplified.

Furthermore, MFP 100 may have the functions of server 400 . In this case, the server 400 becomes unnecessary, so the system configuration can be simplified.

<Second modification>
The MFP 100 according to the second embodiment generates a job in real time each time voice information is received. good. For example, the MFP 100 may generate a job at predetermined time intervals, or may generate a job each time a predetermined number of voice records are added to the voice table.

As described above, in speech processing system 1 according to the second embodiment, any one of smart speakers 200, 200A, 200B, and 200C and MFP 100 identifies the user who has uttered a voice, and MFP 100 identifies smart speaker 200, When the first user is identified from the voice collected by the first device among 200A, 200B, and 200C, the voice and the second user collected by the second device among smart speakers 200, 200A, 200B, and 200C are identified. Based on the uttered voice, a requested job that the first user has requested the second user to execute is generated as a job. Therefore, since the requested job is generated from the conversation of the first user and the second user who exist at distant locations, the operation for generating the job can be simplified.

Further, MFP 100 according to the second embodiment further detects the user identification information of any one of a plurality of pre-registered users from the voice identifying the first user. A user identified by the information is determined as a second user, and a device for collecting voices uttered by the second user, for example, smart speaker 200B, is determined as the second device among smart speakers 200, 200A, 200B, and 200C. Therefore, it is possible to easily determine the smart speaker 200A that collects the voice uttered by the second user who converses with the first user.

Further, in the case where there are a plurality of voice collection devices that collect voice uttered by the second user, for example, smart speakers 200B and 200C, MFP 100 in the second embodiment can The second device is determined as the device with the highest volume of the voice that is heard. Therefore, the accuracy of voice recognition can be improved.

In addition, the MFP 100 according to the second embodiment generates a requested job when the voice collected by the second device indicates the content of the license. can be prevented from generating

It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is indicated by the scope of the claims rather than the above description, and is intended to include all modifications within the meaning and range of equivalents of the scope of the claims.

<Appendix>
(1) User identification information for identifying the user who uttered the voice collected by the voice collecting device and voice obtained by recognizing the voice collected by the voice collecting device from each of the plurality of voice collecting devices further comprising voice information acquisition means for acquiring information,
8. The job generating apparatus according to claim 7, wherein said user specifying means specifies a user based on said user identification information acquired from each of said plurality of voice collecting devices by said voice information acquiring means.

1 voice processing system, 100 MFP, 200, 200A, 200B, 200C smart speaker, 400 server, 3 network, 5 Internet, 7 gateway device, 110 main circuit, 111 CPU, 112 communication I/F section, 113 ROM, 114 RAM , 115 HDD, 116 facsimile section, 117 external storage device, 118, CD-ROM, 120 automatic document feeder, 130 document reading section, 140 image forming section, 150 paper feeding section, 155 post-processing section, 160 operation panel, 161 display unit 163 operation unit 165 touch panel 167 hard key unit 201 CPU 202 ROM 203 RAM 204 EPROM 205 communication unit 206 display unit 207 operation unit 208 microphone 209 speaker 210 serial interface 51 Operating user identification unit 53 setting unit 55 job control unit 57 requester notification unit 71 voice information acquisition unit 73 job generation unit 81 job reception unit 81A job reception unit 83 association unit 85 job execution unit , 251 voice reception unit, 253 voice recognition unit, 255 user identification unit, 257 job generation unit, 259 job transmission unit, 261 response unit, 263 caller determination unit, 265 device determination unit, 267 voice information acquisition unit, 271 process determination 273 normal job generation unit 275 request job generation unit 277 keyword extraction unit 279 caller determination unit 281 operation user notification unit 291 voice information transmission unit.

Claims (13)

a plurality of audio collection devices for collecting audio;
a job generation device that generates a job to be executed by the image processing device;
any one of the plurality of voice collection devices and the job generation device,
Equipped with user identification means for identifying the user who made the utterance,
The job generation device is a voice collected by a first device out of the plurality of voice collection devices and the first user is specified by the user specifying means and the first user out of the plurality of voice collection devices. Said first user requests said second user to execute, based on voice collected by a second device different from said device and for which a second user different from said first user is specified by said user specifying means A voice processing system comprising job generating means for generating the requested job as the job. The job generation device further
A voice identifying the first user in response to detection of user identification information for identifying one of a plurality of pre-registered users from the voice identifying the first user by the user identifying means. The user identified by the user identification information detected from is determined to be the second user, and the voice collection device that collects the voice uttered by the second user among the other one or more voice collection devices is selected as the 2. The audio processing system of claim 1, further comprising device determination means for determining the second device. When there are a plurality of the voice collecting devices that collect the voice uttered by the second user, the device determining means determines the voice collected from among the plurality of voice collecting devices that collect the voice uttered by the second user. 3. The audio processing system of claim 2, wherein the second device is determined as the audio collection device with the highest volume of . 4. The voice processing system according to claim 2, wherein said job generating means generates said requested job when the voice collected by said second device indicates the content of a license. 5. The speech processing system according to any one of claims 1 to 4, wherein said job generation device is one of said plurality of speech collection devices. 5. The voice processing system according to claim 1, wherein said job generation device is said image processing device. A job generation device for generating a job to be executed by an image processing device,
user identification means for identifying the user who has uttered the voice;
Sound collected by a first device out of a plurality of sound collecting devices, the sound identifying the first user by the user identifying means, and a second device out of the plurality of sound collecting devices different from the first device and a voice that identifies a second user different from the first user by the user identifying means, the requested job requested by the second user to be executed by the first user based on the voice collected in and a job generating means for generating a job. The first user is identified in response to detection of user identification information for identifying one of a plurality of pre-registered users from the voice identifying the first user by the user identifying means. The user identified by the user identification information detected from the voice is determined as the second user, and the voice collecting device that collects the voice uttered by the second user among the plurality of voice collecting devices is selected as the second user. 8. The job generating apparatus according to claim 7, further comprising a device determining means for determining the device. When there are a plurality of the voice collecting devices that collect the voice uttered by the second user, the device determining means determines the number of voices to be collected among the plurality of voice collecting devices that collect the voice uttered by the second user. 9. The job generating apparatus according to claim 8, wherein said second device is determined as said voice collecting device having the highest volume. 10. The job generation device according to claim 7, wherein said job generation means generates said requested job when the voice collected by said second device indicates the content of the permission. a plurality of audio collection devices for collecting audio;
A job control method executed in an audio processing system comprising a job generation device for generating a job to be executed by an image processing device,
causing any one of the plurality of voice collection devices and the job generation devices to execute a user identification step of identifying the user who made the utterance;
voice collected by a first device out of the plurality of voice collecting devices, the voice identifying the first user in the user identifying step; The requested job requested by the second user to be executed by the first user is determined based on the voice collected by the device and the voice identifying the second user different from the first user in the user identification step. A job generation method for causing the job generation device to execute a job generation step for generating the job. A job generation method executed by a job generation device for generating a job to be executed by an image processing device, comprising:
a user identification step of identifying the user who uttered the voice;
voice collected by a first device out of a plurality of voice collecting devices, the voice identifying the first user in the user identifying step; A requested job requested by the first user to be executed by the second user based on the voice collected by the device and the voice by which the second user different from the first user is specified in the user specifying step as the job; and a job generating method for causing a job generating device to execute the above. A job generation program executed by a computer that controls a job generation device that generates a job to be executed by an image processing device,
a user identification step of identifying the user who uttered the voice;
voice collected by a first device out of a plurality of voice collecting devices, the voice identifying the first user in the user identifying step; A requested job requested by the first user to be executed by the second user based on the voice collected by the device and the voice by which the second user different from the first user is specified in the user specifying step as the job; and a job generation program for causing the computer to execute a job generation step of generating as the job.

Images