JP2021113899A - Information processing system, information processing method, and program - Google Patents

Abstract

To simplify a voice operation for, when causing an image formation device to execute a plurality of successive jobs by sound, executing the jobs.SOLUTION: An information processing device receives first voice data at a first timing when a voice acquisition device that collects voices to obtain voice data makes transmission, transmits a reading instruction for reading an original converted based on a predetermined reading condition from the first voice data to an image reading device that performs reading of an image on the original at least one or more times, and transmits the reading instruction to the image reading device again in a case where second voice data received at a second timing after the first timing has a content that enables continuation of execution of the reading instruction based on the first voice data.SELECTED DRAWING: Figure 8b

Description

The present invention relates to information processing systems, information processing methods and programs.

Today, AI (Artificial Intelligence) voice assistants that operate devices by voice are known. In addition, smart home appliances that can be operated by voice are also known, and the field of voice operation is expected to grow further in the future.

For example, as an example of voice operation, a system is disclosed in which a server interprets the content uttered to a terminal device and an image forming device connected via a network executes a job based on the interpretation result (a system is disclosed. See Patent Document 1).

However, in the invention disclosed in Patent Document 1, there is no disclosure or suggestion regarding the case where the image forming apparatus wants to execute a plurality of consecutive jobs by voice input, and the conditions for executing the job are set by voice each time. I had to enter and set it.

The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to simplify a voice operation for executing a job when a plurality of consecutive jobs are executed by an image forming apparatus by voice.

In order to solve the above-mentioned problems and achieve the object, the present invention has an audio acquisition device that collects audio to obtain audio data, and an image reader that reads an image at least once on a document. Then, the first audio data is received at the first timing transmitted by the audio acquisition device, and the first audio data is converted into a scanning instruction for scanning the document based on a predetermined scanning condition. An information processing system including an information processing device that transmits the reading command based on the predetermined reading conditions to the image reading device, wherein the information processing device has a second timing after the first timing. When the second audio data received at the timing of 2 has a content that enables the execution of the reading instruction based on the first audio data to be continued, the reading instruction is retransmitted to the image reading device. It is a characteristic information processing system.

According to the embodiment of the present invention, when a plurality of consecutive jobs are executed by the image forming apparatus by voice, it is possible to simplify the voice operation for executing the jobs.

It is a figure which shows an example of the structure of the voice operation system which concerns on this embodiment. It is a figure which shows an example of the hardware composition of a smart speaker. It is a figure which shows an example of the hardware composition of the voice recognition server apparatus. It is a figure which shows an example of the hardware composition of the AI assistant server apparatus. This is an example of various table data. It is a figure which shows an example of the hardware configuration of the MFP. It is a figure which shows an example of the functional block of each device which constitutes a voice operation system. It is a sequence diagram which shows an example of the reading process based on the utterance of a user in 1st Embodiment. It is a sequence diagram which shows an example of the reading process based on the utterance of a user in 1st Embodiment. It is a flowchart which shows an example of information complementation and inquiry processing in 1st Embodiment. It is a flowchart which shows an example of conversion and transmission of the reading instruction in 1st Embodiment. It is a sequence diagram which shows an example of the reading process based on the utterance of the user in 2nd Embodiment. It is a sequence diagram which shows an example of the reading process based on the utterance of the user in 2nd Embodiment. It is a flowchart which shows an example of the execution process of a read instruction in 2nd Embodiment.

Hereinafter, the information processing system, the information processing method, and the voice operation system as an application example of the program will be described.

[Overview of the system]
FIG. 1 is a diagram showing an example of a configuration of a voice operation system according to the present embodiment. As shown in FIG. 1, the voice operation system 1 of the present embodiment includes at least one or more smart speakers 2, a voice recognition server device 3, an AI assistant server device 4, and at least one or more multifunction devices 6 (MFP: Multifunction Peripheral). Hereinafter, it is also simply referred to as MFP 6), and is formed by connecting to each other via a network 7 such as a LAN (Local Area Network).

Here, the voice operation system 1 is an example of an information processing system. The smart speaker 2 is a well-known smart speaker device that can be remotely operated to collect voice with a built-in microphone and obtain voice data. Further, the smart speaker 2 is a device equipped with artificial intelligence that enables viewing of various contents such as music and moving images, weather, news, etc. by voice input, and supports, for example, interactive voice operation. Refers to a speaker that has an AI assistant function. Further, the smart speaker 2 also has a function of remotely controlling various devices such as lighting and home appliances by voice.

The smart speaker 2 receives, for example, a voice operation uttered by a user (also referred to as a user), and is instructed to read a document to the MFP 6 (hereinafter, also referred to as voice information) based on voice data (also referred to as voice information) obtained by the voice operation. It functions as an example of a voice acquisition device in a voice operation system 1 that executes a process (hereinafter, also referred to as a predetermined process) associated with various commands (jobs) such as a read command). As described above, one or more smart speakers 2 may be provided in the voice operation system 1. Further, the smart speaker 2 transmits voice data based on the received voice operation to the voice recognition server device 3 (or cloud service device 5) via the network 7. Further, the smart speaker 2 may have a microphone function, a camera function, and the like for giving feedback to the user in order to supplement the voice data obtained from the voice operation by the user's utterance.

The voice recognition server device 3 has a function of receiving the voice data obtained by the smart speaker 2 and converting it into text data. Further, the AI assistant server device 4 has a function of processing the voice data obtained by the smart speaker 2 in cooperation with the voice recognition server device 3. The voice recognition server device 3 and the AI assistant server device 4 are connected to each other via the network 7, and collectively function as a cloud service device 5. The cloud service device 5 generates and transmits a read instruction to the MFP 6, for example.

At least one or both of the voice recognition server device 3 and the AI assistant server device 4 constituting the cloud service device 5 described above is an example of an information processing device.

The AI assistant server device 4 converts the reading instruction into a read instruction that can be interpreted by the MFP 6 based on the user's intention converted by the voice recognition server device 3. The AI assistant server device 4 transmits the converted read instruction or the like to the MFP 6 via the network 7. Here, the reading command is generated based on, for example, an instruction for the user to read the document given to the smart speaker 2 by voice operation (hereinafter, referred to as a document reading instruction). The document reading instruction is an example of an information processing request.

Further, the AI assistant server device 4 includes a management database 401 (hereinafter referred to as a management DB 401) and a linking database 402 (hereinafter referred to as a linking DB 402) in a storage unit such as the HDD 44. For the management DB 401 and the linking DB 402, for example, a storage unit such as an HDD provided on the network 7 by the cloud service device 5 can be used. In addition, one or both of the management DB 401 and the linking DB 402 may be stored in another server device accessible by the cloud service device 5 via the network 7.

The management DB 401 stores, for example, text data, image data, voice data, and the like as contents (data) provided by the AI assistant server device 4.

The information managed by the management DB 401 can be newly added or changed by, for example, the MFP 6 connected via the network 7. Although the management DB 401 and the MFP 6 are shown as separate bodies in FIG. 1, they may be configured as servers having the same functions. In this case, the management program described later may acquire various information managed by the management DB 401 by transmitting a read command to the MFP 6 to the management DB 401.

On the other hand, the linking DB 402 includes, for example, a device ID for identifying each smart speaker 2 (voice acquisition device) (hereinafter, also simply referred to as a device ID) and an information processing device associated with each smart speaker 2. It is stored in association with the device ID of the MFP6 (MFP_ # 1, MFP_ # 2, etc.). Details of this linking DB 402 will be described later.

In the present embodiment, the management DB 401 and the linking DB 402 are illustrated to be included in the AI assistant server device 4, but they may be provided separately from the AI assistant server device 4, or one of them may be AI. It is included in the assistant server device 4, and the other may be provided separately from the AI assistant server device 4.

Further, in the present embodiment, the two server devices of the voice recognition server device 3 and the AI assistant server device 4 will be described as a cloud service device 5 which is integrated into one. However, each of the voice recognition server device 3 and the AI assistant server device 4 may be further divided into a plurality of server devices.

Further, in the present embodiment, the smart speaker 2 or the MFP 6 may have a part or all of the functions of the cloud service device 5. When the smart speaker 2 or the MFP 6 has all the functions of the cloud service device 5, the voice operation system 1 may not include the cloud service device 5. In such a case, the smart speaker 2 may communicate with the MFP 6 without going through the cloud service device 5, and the voice operation system 1 may construct an input response system 8 in which the smart speaker 2 and the MFP 6 are integrated.

As described above, the case where the cloud service device 5 includes the voice recognition server device 3 and the AI assistant server device 4 has been described, but the AI assistant server device 4 has some or all of the functions of the voice recognition server device 3. The voice recognition server device 3 may have a part or all of the functions of the AI assistant server device 4. That is, the voice recognition server device 3 and the AI assistant server device 4 may complement each other's functions. Further, the cloud service device 5 may be composed of one server or three or more servers.

According to each of the above configurations, in the voice operation system 1, the smart speaker 2 collects the voice related to the reading process uttered by the user, acquires the voice data, and transmits the voice data to the cloud service device 5. The cloud service device 5 generates a read instruction based on the voice data received from the smart speaker 2, and transmits the generated read instruction to the MFP 6 via the network 7. Further, the MFP 6 that has received the read instruction executes the received read instruction. Here, the MFP 6 is an example of an image reading device. The network 7 described above may be composed of either a wired LAN or a wireless LAN.

[Hardware configuration]
Next, the hardware configurations of the smart speaker 2, the voice recognition server device 3, the AI assistant server device 4, and the MFP 6 of the present embodiment will be described in detail with reference to FIGS. 2 to 6.

<Hardware configuration of smart speaker>
FIG. 2 is a diagram showing an example of the hardware configuration of the smart speaker. As shown in FIG. 2, the smart speaker 2 as an example of the voice acquisition device supplies hardware resources including a CPU 21, a RAM 22, a ROM 23, an interface unit (I / F unit) 24, and a communication unit 25 via an internal bus 26. To be connected to each other.

The CPU 21 is a control device that controls the entire smart speaker 2 in an integrated manner.

The RAM 22 has, for example, a function as a work area in which various programs stored in the ROM 23 or the like are downloaded and various processes are executed by the CPU 21.

The ROM 23 stores data constituting various programs including an operation voice processing program. By executing these processing programs, the CPU 21 enables processing by voice operation on the MFP 6. Further, the CPU 21 executes display control of data acquired from the cloud service device 5 on the touch panel 27, voice output control for feedback via the speaker unit 28, image output control, and the like.

A touch panel 27, a speaker unit 28, a microphone unit 29, and an imaging unit (camera unit) 30 are connected to the I / F unit 24.

The communication unit 25 transmits the information obtained by the voice operation by the user to the voice recognition server device 3 via the network 7. Further, when communicating with another device via the network 7, the communication unit 25 can perform communication in either a wired or wireless communication form.

The internal bus 26 is a general-purpose bus that connects the CPU 21, RAM 22, ROM 23, I / F unit 24, and communication unit 25. The type of the internal bus 26 does not matter as long as it is a bus generally used in general-purpose devices such as smart speakers.

In the touch panel 27, for example, a liquid crystal display (LCD: Liquid Crystal Display) and a touch sensor are integrally formed. The touch panel 27 is designated as a desired operation by the user performing a touch operation or the like on a touch key or the like arranged on the liquid crystal display unit.

The speaker unit 28 provides voice feedback to the user to prompt the user to input insufficient information or the like.

The microphone unit 29 acquires voice data given by the voice spoken by the user, for example, in order to cause the MFP 6 to read the document by voice operation. The acquired voice data is transmitted to the voice recognition server device 3 via the communication unit 25, and is converted into text data by the voice recognition server device 3.

The image pickup unit (camera unit) 30 captures a user who uses the smart speaker 2 and other images. The captured image or the like is transmitted to the voice recognition server device 3 via the communication unit 25 as moving image data or still image data (hereinafter, simply referred to as image data).

<Hardware configuration of voice recognition server device>
FIG. 3 is a diagram showing an example of the hardware configuration of the voice recognition server device.
As shown in FIG. 3, the voice recognition server device 3 uses an internal bus for hardware resources including a CPU 31, a RAM 32, a ROM 33, an HDD (Hard Disk Drive) 34, an interface unit (I / F unit) 35, and a communication unit 36. They are interconnected via 37. Further, a display unit 38 and an operation unit 39 are connected to the I / F unit 35.

The HDD 34 stores data constituting the following operation voice conversion program. The operation voice conversion program converts, for example, the voice data received from the smart speaker 2 into text data. Subsequently, the operation voice conversion program determines whether or not the converted text data matches the predefined dictionary information. In determining whether or not the information matches the dictionary information, the operation voice conversion program uses variables such as an intent indicating the user's intention and execution conditions of a predetermined process when the operation voice conversion program matches the dictionary information. Convert to a parameter that indicates. After that, the operation voice conversion program transmits to the AI assistant server device 4 parameters indicating variables such as an intent indicating the user's intention and execution conditions of a predetermined process.

The CPU 31 executes various programs including the above-mentioned operation voice conversion program. That is, the voice recognition server device 3 functions as a device that receives voice data and analyzes the voice. The operation voice conversion program, the operation image conversion program, and the voice assistant program may be executed by one server device or may be executed by different server devices. Further, these programs may be executed by coordinating a plurality of server devices.

The RAM 32 has a function as a work area in which various programs stored in a storage unit such as a ROM 33 are downloaded and various processes are executed by the CPU 31.

The ROM 33 stores data constituting other programs other than the various programs stored in the HDD 34. The CPU 31 may control between the smart speaker 2 and the AI assistant server device 4 by executing various programs stored in the ROM 33.

A display unit 38 and an operation unit 39 are connected to the I / F unit 35.

The communication unit 36 receives the voice data obtained by the voice operation accompanying the user's utterance from the smart speaker 2. Further, when communicating with another device via the network 7, the communication unit 36 can perform communication in either a wired or wireless communication form.

The internal bus 37 is a general-purpose bus that connects the CPU 31, RAM 32, ROM 33, HDD 34, I / F unit 35, and communication unit 36. The type of the internal bus 37 does not matter as long as the voice recognition server device 3 can realize the function as a server device.

The display unit 38 is composed of, for example, a liquid crystal display (LCD), and displays, for example, various states of the voice recognition server device 3.

The operation unit 39 is, for example, a so-called touch panel in which a liquid crystal display unit and a touch sensor are integrally formed. When the operator (user) uses the operation unit 39 to issue an execution command for a desired operation, the operator (user) specifies the desired operation by touching an operation button (software key) or the like displayed on the operation unit 39. ..

Further, the operation voice processing program may be provided by recording a file in an installable format or an executable format on a recording medium readable by a computer device such as a CD-ROM or a flexible disk (FD). Further, it may be recorded and provided on a recording medium readable by a computer device such as a CD-R, a DVD (Digital Versatile Disk), a Blu-ray disc (registered trademark), or a semiconductor memory. Further, it may be provided in the form of being installed via a network such as the Internet, or may be provided by being incorporated in advance in a storage unit such as a ROM of the voice recognition server device 3.

<Hardware configuration of AI assistant server device>
FIG. 4 is a diagram showing an example of the hardware configuration of the AI assistant server device. The AI assistant server device 4 connects hardware resources including a CPU 41, a RAM 42, a ROM 43, an HDD 44, an interface unit (I / F unit) 45, and a communication unit 46 to each other via an internal bus 47. Further, a display unit 48 and an operation unit 49 are connected to the I / F unit 45.

The AI storage unit 40 of the HDD 44 stores dictionary information for interpreting a reading instruction instructed by a user by voice input. This dictionary information includes entity information, action information, and intent information, which will be described later. Further, the HDD 44 stores the following user management table 402a and device management table 402b, respectively. Although predetermined setting values are given to these tables in advance, additions and changes may be made as appropriate. The outline of the user management table 402a and the device management table 402b will be described below.

(Various tables)
FIG. 5 is an example of various tables. In the storage unit such as the HDD 44 of the AI assistant server device 4, the user management table 402a shown in FIG. 5A, the device management table 402b shown in FIG. 5B, and the instruction management table 402c shown in FIG. 5C are used. The configured associating DB 402 is constructed. However, the instruction management table 402c may be stored in the MFP 6. In this case, since the instruction management table 402c is used only by one MFP 6, the instruction management table 402c does not have to include the image reading device name and the device ID of the image reading device. Of these, in the user management table 402a, information including a user name and a user ID is associated and managed for each device ID of the voice acquisition device. Further, in the device management table 402b, the device name of the MFP 6 (image reading device) and the device ID for identifying the image reading device (hereinafter, also simply referred to as the device ID) are used for each voice acquisition device name or device ID of the voice acquisition device. ) And various information including the connection information of the image reader are associated and managed. Further, in the instruction management table 402c, the document size, file format, resolution, color / monochrome, single page / multipage, destination, and continuous processing flag are managed in association with each image reading device name or device ID.

The device ID of the voice acquisition device used in the user management table 402a and the device management table 402b is an example of device identification information for identifying the smart speaker 2 as an example of the voice acquisition device as described above. That is, the device ID is an example of device identification information for identifying the voice acquisition device. Further, instead of or in addition to the device ID of the voice acquisition device, the voice acquisition device name indicating the device name of the voice acquisition device may be managed.

The name of the image reading device managed by the device management table 402b is the name of the device of the image reading device in which the reading command or the like is executed by the utterance to the smart speaker 2 used by the user. The name of the image reading device is given the name of a device such as the above-mentioned MFP 6 or a scanner that operates independently.

On the other hand, the device ID is an example of device identification information for identifying an image reading device. The device ID is information for identifying the MFP 6.

Further, connection information is assigned and stored in the HDD 44 for each image reading device (MFP). Here, the connection information is information necessary for communicating with each MFP, and for example, address information is given.

The device management table 402b is newly additionally registered by associating the user ID of a new unregistered user, the device ID of the voice acquisition device used by the user, and the device ID specified by the user. You may do so.

Further, the continuous processing flag of the instruction management table 402c is for determining whether or not to continue the reading process of the manuscript according to a predetermined utterance content uttered by the user when the manuscript is a book or the like composed of a plurality of pages. It is managed as a flag. This continuous processing flag is managed by setting a value of, for example, "1" when the utterance content such as "next" or "continue" is detected after the user specifies to read the first original. NS. On the other hand, when there is only one document, the user does not make an utterance suggesting reading of the next document, so that the continuous processing flag is managed by setting a value of "0", for example. The continuous processing flag may be given "0" as an initial setting value.

It should be noted that when the reading of the last page of the manuscript consisting of multiple pages is completed according to the utterance content such as "end", "or more" or "this is the last" uttered by the user, or the reading of the manuscript of only one page is completed. When is completed, the instruction in the instruction management table 402c is deleted. However, if the continuous processing flag is "1", it may be changed or set to "0", and if it is "0", that value may be maintained. The setting process related to this continuous processing flag will be described in detail later.

Further, the instruction management table 402c is also newly additionally registered by associating the user ID of the new unregistered user, the device ID of the voice acquisition device used by the user, and the device ID specified by the user, respectively. You may do so. For example, the AI assistant server device 4 confirms whether or not the instruction is included in the instruction management table 402c at the timing of generating the instruction based on the utterance of the user and the timing of transmitting the read instruction to the MFP 6. If it is not included, it can be registered in the instruction management table 402c. At this time, the continuous processing flag is registered as 0 by associating the image reading device name or the device ID of the image reading device with various parameters specified by the user as information for identifying the MFP 6 to which the instruction is transmitted. do.

The CPU 41 converts the interpretation result generated (converted) by the voice recognition server device 3 into data such as a read command for the MFP 6 and transmits the data to the MFP 6 via the network 7. The intention instructed by the user includes, for example, a reading instruction to the MFP 6 and an instruction for various instructions. In this way, the MFP 6 can be operated by the voice data acquired by the smart speaker 2.

The RAM 42 has a function as a work area where various programs stored in a storage unit such as the HDD 44 are downloaded and various processes are executed by the CPU 41, for example.

The ROM 43 stores, for example, data constituting various programs other than the programs stored in the HDD 44.

As described above, the management DB 401 and the linking DB 402 are constructed in the HDD 44. The management DB 401 stores, for example, text data, image data, voice data, and the like indicating the contents provided by the AI assistant server device 4 as the cloud service device 5. Further, in the associating DB 402, for example, the following information is stored on the assumption that a plurality of smart speakers 2 are used. The information is, for example, information in which each device ID that identifies each smart speaker 2 and the device ID of the MFP 6 that executes a reading command or the like by a voice operation on each smart speaker 2 are associated with each other. That is, in the associating DB 402, the device ID of each smart speaker 2 and the device ID of the MFP 6 are associated with each other and stored as a device management table 402b so that the MFP 6 that can be used can be specified by voice operation for each smart speaker 2. Has been done.

A display unit 48 and an operation unit 49 are connected to the I / F unit 45.

The communication unit 46 transmits / receives data to / from the voice recognition server device 3 and the MFP 6 via the network 7. Further, when communicating with another device via the network 7, the communication unit 46 can perform communication in either a wired or wireless communication form.

The internal bus 47 is a general-purpose bus that connects the CPU 41, the RAM 42, the ROM 43, the HDD 44, the I / F unit 45, and the communication unit 46. The type of the internal bus 47 does not matter as long as the AI assistant server device 4 realizes the function of the information processing device.

The display unit 48 is composed of, for example, a liquid crystal display (LCD), and displays, for example, various states of the AI assistant server device 4.

The operation unit 49 is, for example, a so-called touch panel in which a liquid crystal display unit and a touch sensor are integrally formed. When the operator (user) issues an execution command for a desired operation using the operation unit 39, the operator (user) specifies the desired operation by touching an operation button (software key) or the like displayed on the operation unit 49. ..

<Hardware configuration of cloud service device>
As described above, the cloud service device 5 is, for example, a combination of the voice recognition server device 3 and the AI assistant server device 4, and is connected to the smart speaker 2 and the MFP 6 via the network 7, respectively. The hardware configuration that constitutes the cloud service device 5 is as described in the voice recognition server device 3 and the AI assistant server device 4.

<Hardware configuration of MFP>
FIG. 6 is a diagram showing an example of the hardware configuration of the MFP. The MFP 6 includes a controller 600, a short-range wireless communication circuit 620, an engine control unit 630, an operation panel 640, and a network I / F 650.

Of these, the controller 600 controls, for example, the input from the operation panel 640. Further, the controller 600 includes a CPU 601 as a control unit that controls the entire MFP 6, a system memory (MEM-P) 602, a north bridge (NB) 603, a south bridge (SB) 604, and an ASIC (Application Specific Integrated Circuit) 606. It has a local memory (MEM-C) 607 as a storage unit, an HDD controller 608, and an HDD 609 as a storage unit. Further, the NB 603 and the ASIC 606 are connected by an AGP (Accelerated Graphics Port) bus 621.

The NB 603 is a bridge circuit for connecting the CPU 601 to the MEM-P602, SB 604, and ASIC 606. The NB 603 has a memory controller that controls reading and writing to the MEM-P602, a PCI (Peripheral Component Interconnect) master, and an AGP target.

The MEM-P602 includes a ROM 602a, which is a memory for storing programs and data that realizes each function of the controller 600, a RAM 602b, which is used as a storage memory for developing programs and data, a storage memory for document scanning, and a drawing memory for memory printing. Be prepared. The program stored in the RAM 602b is a file in an installable format or an executable format, and is recorded and provided on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD. It may be configured.

The SB 604 is a bridge circuit for connecting the NB 603 to a PCI device and peripheral devices.

The ASIC 606 is an IC (Integrated Circuit) for image processing applications having hardware elements for image processing. Its role is a bridge circuit that connects the AGP bus 621, the PCI bus 622, the HDD controller 608, and the MEM-C607, respectively. Further, the ASIC 606 includes a PCI target and an AGP master, an arbiter (ARB) that mediates the operation and timing of other devices connected to the ASIC 606, a memory controller that controls MEM-C607, and a DMAC (Direct Memory Access Controller) that controls DMA control. ), A PCI unit that transfers data between the scanner unit 631 and the printer unit 632 via the PCI bus 622.

A USB (Universal Serial Bus) interface and an IEEE 1394 (Institute of Electrical and Electronics Engineers 1394) interface may be connected to the ASIC 606.

The MEM-C607 is a local memory used as a copy image buffer and a code buffer.

The HDD 609 is a storage for accumulating image data, accumulating font data used when printing a read original, accumulating forms, and the like. The HDD controller 608 controls reading or writing of data to the HDD 609 according to the control of the CPU 601.

The AGP bus 621 is a bus interface for a graphics accelerator card proposed to speed up graphics processing. The AGP bus 621 can speed up the graphics accelerator card by directly accessing the MEM-P602 with high throughput.

The short-range wireless communication circuit 620 is a circuit for performing short-range wireless communication, and includes an antenna 620a for the short-range wireless communication circuit. The short-range wireless communication circuit 620 is, for example, a wireless communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark).

The engine control unit 630 is composed of a scanner unit 631 and a printer unit 632. The scanner unit 631 and the printer unit 632 include image processing portions such as error diffusion and gamma conversion.

(Configuration of scanner unit)
The operation panel 640 as a part of the operation unit 11 can be mounted on or connected to the MFP 6, and includes a panel display unit 640a and a panel operation unit 640b. In this embodiment, a state in which the MFP 6 can be connected is shown as an example. The panel display unit 640a is provided with a touch panel or the like that displays the current set value, the selection screen, or the like and receives input from the operator. Further, the panel operation unit 640b includes a numeric keypad that accepts input of attribute information (also referred to as various conditions) related to reading the original given in the original size, file format, resolution, and the like, and a start key that accepts a copy start instruction. Specifically, the attribute information related to the reading of the original is given the original size, file format, resolution, color / monochrome, single page / multipage, destination, continuous processing flag, etc., as exemplified in the instruction management table 402c.

The network I / F 650 is an interface for performing data communication using a communication network. The short-range wireless communication circuit 620 and the network I / F 650 are electrically connected to the ASIC 606 via the PCI bus 622.

The MFP 6 can be selected by switching the document box function, the copy function, the printer function, and the facsimile function by the application switching key displayed on the panel display unit 640a or provided in the panel operation unit 640b. That is, the MFP 6 is in the document box mode when the document box function is selected, in the copy mode when the copy function is selected, in the printer mode when the printer function is selected, and in the facsimile mode when the facsimile function is selected.

[Functional configuration]
<Functional configuration of voice operation system>
FIG. 7 is a diagram showing an example of functional blocks of each device constituting the voice operation system. As shown in FIG. 1, the voice operation system 1 is a combination of a smart speaker 2, a voice recognition server device 3, an AI assistant server device 4, and a cloud service device 5 (voice recognition server device 3 and AI assistant server device 4). ) And the MFP 6 are connected via the network 7, respectively.

<Functional configuration of smart speaker>
The smart speaker 2 performs data communication such as voice data, image data, and text data with the voice recognition server device 3 and the AI assistant server device 4 constituting the cloud service device 5.

The CPU 21 of the smart speaker 2 shown in FIG. 2 expands and executes the operation voice processing program stored in the storage unit such as the ROM 23 in the RAM 22, for example, the communication control unit 251 and the acquisition unit 252, and the feedback unit 253. , The storage / reading processing unit 254 (hereinafter, also referred to as the communication control unit 251 to the storage / reading processing unit 254) constitutes a function or a means for functioning.

<Each function configuration of smart speaker>
Next, each functional configuration of the smart speaker 2 will be described. The communication control unit 251 controls communication between the smart speaker 2 and the voice recognition server device 3 or the cloud service device 5 via the network 7, and transmits / receives various data or information. At that time, the communication control unit 251 controls the communication unit 25 of the smart speaker 2 to transmit and receive various data or information. The communication control unit 251 transmits information based on a predetermined operation and instruction performed by the user to the smart speaker 2 acquired by the acquisition unit 252 described below to the voice recognition server device 3 (or cloud service device 5). Send. Further, the communication control unit 251 acquires text data, image data, voice data, etc. from the cloud service device 5 for feedback. Further, the communication control unit 251 also includes a device ID that identifies the smart speaker 2 when transmitting information related to a predetermined operation and instruction performed by the user to the voice recognition server device 3 (or cloud service device 5). And send.

As described above, the communication control unit 251 controls the communication of the smart speaker 2 with another device connected via the network 7 such as a LAN. As a communication method for performing this communication, for example, a communication protocol such as Ethernet (registered trademark) generally used in a LAN is used. The communication control unit has the same function as each communication control unit of the voice recognition server device 3, the AI assistant server device 4, the cloud service device 5, and the MFP 6, which will be described later.

The acquisition unit 252 is an example of the voice data acquisition means. The acquisition unit 252 acquires the instruction voice accompanying the user's voice operation collected through the microphone unit 29. Further, the acquisition unit 252 may acquire an instruction operation including a machine operation such as a tap operation or a physical switch press by the user. That is, the acquisition unit 252 acquires at least one of the information representing the instruction including the instruction voice and the instruction operation. Here, the information representing the above-mentioned instruction corresponds to the instruction information. The user's instruction voice includes, for example, information for converting into a reading command for causing the MFP 6 or the like to read the document and a processing execution command for executing various commands.

By executing the operation voice processing program described above, the acquisition unit 252 acquires the voice data given by the user's utterance and transmits it to the voice recognition server device 3 (or the cloud service device 5). Further, the acquisition unit 252 cooperates with the feedback unit 253 to display the data (voice data, image data, text data, etc.) acquired from the cloud service device 5 on the touch panel 27, or via the speaker unit 28. Notify the user of the voice. The touch panel 27 may be integrally configured with the smart speaker 2 or may be configured separately. When configured separately from the smart speaker 2, the touch panel 27 may be provided with a wireless communication interface necessary for wireless communication or the like performed with the smart speaker 2.

The feedback unit 253 functions so that the smart speaker 2 responds to the user as necessary in the reading of the document and the predetermined processing executed by the MFP 6 based on the voice data given by the user's utterance. By the feedback unit 253, the present embodiment realizes an interactive system with the user. Further, in order to realize the voice operation in this interactive system, the feedback unit 253 provides voice feedback to supplement, for example, insufficient data with respect to the user's instruction voice. Further, the feedback unit 253 may provide the user with the text, voice, or image to be fed back by displaying the touch panel 27 on the screen. The details of the interactive operation and the feedback by the feedback unit 253 will be described later.

The storage / reading processing unit 254 performs processing such as storing various data in the ROM 23 and reading various data such as an operation voice processing program stored in the ROM 23.

In this embodiment, the communication control unit 251 to the storage / reading processing unit 254 may be realized by software. Further, the communication control unit 251 to the storage / reading processing unit 254 may cause another program to execute a part of the processing, or may indirectly execute the processing using the other program. Further, the communication control unit 251 to the storage / reading processing unit 254 may be partially or wholly realized by hardware such as an IC (Integrated Circuit).

<Functional configuration of voice recognition server device>
The voice recognition server device 3 analyzes the voice data received from the smart speaker 2 and converts it into text data. Further, the user's intention is interpreted based on the text data and the pre-registered dictionary information, and the interpretation result is transmitted to the AI assistant server device 4.

The CPU 31 of the voice recognition server device 3 expands and executes an operation voice conversion program or the like stored in a storage unit such as the HDD 34 in the RAM 32 according to the voice data given by the user via the smart speaker 2. By executing this operation voice conversion program, the CPU 31 can use, for example, the communication control unit 351 and the acquisition unit 352, the text conversion unit 353, the interpretation unit 354, the output unit 355, the provision unit 356, and the storage / reading processing unit 357 ( Hereinafter, it constitutes a function or a means for functioning as a communication control unit 351 to a storage / reading processing unit 357).

<Each function configuration of voice recognition server device>
Next, each functional configuration of the voice recognition server device 3 will be described. The communication control unit 351 controls communication with the smart speaker 2 or the AI assistant server device 4 via the network 7, and transmits / receives various data or information. Specifically, the communication control unit 351 controls the communication unit 36 of the voice recognition server device 3 so as to receive the voice data given by the user, transmit the text data to the smart speaker 2, and the like.

The acquisition unit 352 acquires information based on predetermined operations, instructions, and the like transmitted from the smart speaker 2. In addition, the acquisition unit 352 may acquire information based on user operations such as the touch panel, buttons, or switches of the smart speaker 2.

The text conversion unit 353 converts the information acquired by the acquisition unit 352, that is, the voice data into text data.

The interpretation unit 354 interprets the instruction from the user based on the text data converted by the text conversion unit 353. Specifically, the interpretation unit 354 determines whether or not the words included in the text data match the dictionary information based on the dictionary information provided by the voice assistant program. Then, if it matches the dictionary information, the interpretation unit 354 converts it into a parameter indicating a variable such as an intent indicating the user's intention and an execution condition of a predetermined process. The interpretation unit 354 transmits the intent and the parameters to the management program executed by the AI assistant server device 4 via the communication control unit 351. At this time, the interpretation unit 354 also transmits the device ID of the smart speaker 2 together with the intent and the parameter to the management program executed by the AI assistant server device 4 via the communication control unit 351.

The output unit 355 controls the communication unit 36 so as to transmit data such as text data, voice data, and image data to the smart speaker 2.

Further, the CPU 31 functions as the providing unit 356 by executing the voice assistant program stored in the storage unit such as the HDD 34.

The providing unit 356 manages the dictionary information in which the relationship between the text data, the intent, and the parameters stored in the storage unit such as the HDD 34 is defined in advance, and provides the dictionary information to the operation voice conversion program. Further, the providing unit 356 may convert and interpret the utterance content from the user based on the text data converted by the text conversion unit 353. That is, the providing unit 356 may also have the functions of the text conversion unit 353 and the interpretation unit 354. Specifically, the providing unit 356 first acquires text data from the operation voice conversion program, and determines whether or not the words or the like included in the text data match the dictionary information. As a result of the determination, if the information matches the dictionary information, the providing unit 356 converts the text data into an intent and a parameter. After that, the providing unit 356 provides the intent and the parameter to the operation voice conversion program.

The storage / reading processing unit 357 performs reading processing of data constituting various programs such as an operation voice conversion program stored in the ROM 33, for example.

In this embodiment, the communication control unit 351 to the storage / reading processing unit 357 may be realized by software. Further, the communication control unit 351 to the storage / reading processing unit 357 may cause another program to execute a part of the processing, or may indirectly execute the processing using the other program. For example, the voice assistant program may execute a part or all of the functions of the interpretation unit 354 of the operation voice conversion program. Further, the image assistant program may execute a part or all of the functions of the interpretation unit 354 of the operation image conversion program. In these cases, for example, it is determined whether or not the words included in the text data match the dictionary information, and the intent indicating the user's intention and the execution condition of the predetermined process when they match the dictionary information. The conversion to the parameter indicating the variable such as may be executed by the voice assistant application, the image assistant application, or the like. Further, the interpretation unit 354 may acquire the intent and the parameter from the voice assistant program or the like. Further, a part or all of the communication control unit 351 to the storage / reading processing unit 357 may be realized by hardware such as an IC (Integrated Circuit).

Further, in the above-mentioned example, the providing unit 356 is realized by software, but some or all of them may be realized by hardware such as an IC (Integrated Circuit). Similar to other programs.

<Functional configuration of AI assistant server device>
The AI assistant server device 4 acquires, for example, the intent, parameters, device ID of the smart speaker 2 and the like obtained by the operation voice conversion program executed by the voice recognition server device 3, and processes each function described later. conduct.

Further, the AI assistant server device 4 converts the interpretation result received from the voice recognition server device 3 into data such as a read command for the MFP 6. After that, the AI assistant server device 4 transmits the converted data such as the read instruction to the MFP 6. In the MFP 6, a predetermined process is executed according to a read command or the like transmitted from the AI assistant server device 4. In addition to transmitting a read command or the like to the MFP 6, the AI assistant server device 4 transmits, for example, another execution command or the like executed by the MFP 6 to another server device if another server device that manages the MFP 6 exists. May be sent to.

The CPU 41 of the AI assistant server device 4 acquires the management program stored in the storage unit such as the HDD 34 of the voice recognition server device 3 via the network 7, expands it into the RAM 42, and executes it. By executing this management program, the CPU 41 may execute, for example, the communication control unit 451 and the acquisition unit 452, the interpretation result conversion unit 453, the execution determination unit 454, the complementary unit 455, the execution instruction unit 456, the device information acquisition unit 457, and the notification. It constitutes a unit 458, a management unit 459, a search unit 460, and a storage / reading processing unit 461 (hereinafter, also referred to as a communication control unit 451-storage / reading processing unit 461).

<Each function configuration of AI assistant server device>
Next, each functional configuration of the AI assistant server device 4 will be described. The communication control unit 451 controls the communication unit 46 so as to transmit the interpretation result to the user's smart speaker 2, receive the text data related to the voice data given by the user, and the like.

The acquisition unit 452 acquires the intent, the parameter, the device ID of the smart speaker 2, and the like transmitted from the voice recognition server device 3.

The interpretation result conversion unit 453 converts the interpretation result such as the intent and the parameter converted by the operation voice conversion program into a reading command or the like that can be interpreted by the MFP 6. The interpretation result conversion unit 453 is one of the functions of the management program executed by the AI assistant server device 4 (or the cloud service device 5), and has a function of a read instruction conversion means. Further, the scanning instruction that can be interpreted by the MFP 6 is an example of information (processing information) for executing the document scanning process (hereinafter, referred to as scanning process) in the MFP 6.

The execution determination unit 454 compares the state of the MFP 6 indicated by the acquired device information with the document reading instruction and the printing instruction specified by the user, and is based on the document reading instruction and the printing instruction specified by the user. It is determined whether or not each process can be executed by the MFP 6. The contents related to reading and printing of the document specified by the user include, for example, a process of determining whether or not the MFP 6 can be used during the time zone instructed by the user, a process of changing the power state of the MFP 6, and a document for the MFP 6. Reading process and printing process. Further, when it is determined that each process based on the document reading instruction and the printing instruction specified by the user can be executed, the execution determination unit 454 causes the interpretation result conversion unit 453 as an example of an output request to the MFP 6. It is determined to convert to a read command, a print command, or the like. On the other hand, when it is determined that the execution is impossible, the execution determination unit 454 feeds back the response information such as an error message to the smart speaker 2 under the execution of the operation voice conversion program or the like.

The complement unit 455 refers to the device management table 402b and the instruction management table 402c for various execution instructions such as a read instruction and a print instruction converted by the interpretation result conversion unit 453, and provides information necessary for processing in the MFP 6. It has a complementary function. The information required for this processing is, for example, information required for conversion into a read command, a print command, or the like for the MFP 6. The complementary unit 455 is an example of complementary means.

The execution instruction unit 456 instructs the MFP 6 to execute the read instruction, the print instruction, and the like converted by the interpretation result conversion unit 453. Further, the execution instruction unit 456 searches the MFP6 associated with the device ID that identifies the smart speaker 2 used by the user from the association DB 402, and gives the MFP6 a read command and a print command together with an intent and a parameter. Etc. are sent.

The device information acquisition unit 457 is, for example, a connection state indicating whether or not a communication connection with the MFP 6 is established, a power usage state related to power on / off of the MFP 6, and a power supply state to the MFP 6 (normal mode, Acquire device information (energy saving mode, etc.). The device information acquisition unit 457 stores and manages the device information acquired from the MFP 6 in a storage unit such as the HDD 44 in association with the device ID or the like that identifies the MFP 6. The storage destination of this device information may be the device management table 402b, which will be described later, which constructs the linking DB 402. Further, the device information acquisition unit 457 also has a function of supplementing information related to the generation of the read instruction executed by the MFP 6 with reference to the device management table 402b.

The notification unit 458 notifies the operation voice conversion program or the like of text data, voice data, image data, etc. as a response to the document reading instruction, the print instruction, and the like by the user. Further, when the parameters indicating the execution conditions such as the read command and the print command for the MFP 6 are insufficient, the notification unit 458 provides feedback to the smart speaker 2 via the operation voice conversion program or the like. That is, the notification unit 458 prompts the user to input the missing parameters. Here, the notification unit 458 may transmit predetermined parameter information to the smart speaker 2 as information necessary for confirming the missing parameter, or as information necessary for prompting the specification of the parameter. Text data, audio data, image data and the like may be transmitted to the smart speaker 2. By the above-mentioned processing, the user can confirm what kind of information is lacking by voice or the like generated from the smart speaker 2.

The management unit 459 associates the device ID of the smart speaker 2 with the device ID of the MFP 6 based on the information input to the client device connected to the smart speaker 2 or the cloud service device 5, and associates the DB 402 for linking. Register with. That is, in the associating DB 402, the information associated with the device ID of the smart speaker 2 and the device ID of the MFP 6 is stored and managed as the device management table 402b.

The search unit 460 searches and identifies the MFP 6 based on the device ID and the user ID (user ID). The search unit 460 may function as one functional unit together with the management unit 459 described above.

The storage / reading processing unit 461 performs each processing such as reading various data stored in a storage unit such as HDD 44 of the AI assistant server device 4 and writing various data to the storage unit such as HDD 44.

Each function of the communication control unit 451 to the storage / reading processing unit 461 described above is an example, and which functional unit performs what kind of processing may be appropriately changed depending on the software configuration of the voice operation system 1.

In the present embodiment, the communication control unit 451 to the storage / reading processing unit 461 are realized by software, but some or all of them are realized by hardware such as an IC (Integrated Circuit). You may. Further, the function realized by the communication control unit 451 to the storage / reading processing unit 461 is to cause another program stored in the storage unit such as the HDD 34 of the voice recognition server device 3 to execute a part of the processing, or another program. May be used to indirectly execute the process.

(Details of interpretation operation by cloud service device)
Here, the details of the interpretation operation by the cloud service device 5 will be described. As described above, the cloud service device 5 is a device that integrates the voice recognition server device 3 and the AI assistant server device 4, and also functions as one server device. The operation voice conversion program generates intents and parameters based on dictionary information for interpreting various instructions based on the user's utterance. More specifically, the operation voice conversion program determines whether or not the words included in the text data converted from the voice data given by the user's utterance match the dictionary information, and if they match, the dictionary. Generate an interpretation result that includes the intents and parameters defined in the information.

The above-mentioned dictionary information may be in any form as long as it can generate intents and parameters. As an example, dictionary information is configured to include entity information, intent information, and association information. The entity information is information that associates a parameter for the MFP 6 to execute a predetermined process with a natural language. In addition, a plurality of synonyms can be registered in one parameter. The intent information is information indicating a predetermined type of processing as described above. The association information is information that associates the utterance phrase (natural language) and entity information uttered by the user, and the utterance phrase and intent information, respectively. With this association information, the AI assistant server device 4 (or cloud service device 5) can correctly interpret even if the utterance order or nuance of the parameters is slightly changed. Further, the association information may generate a response text (interpretation result) based on the uttered content. The dictionary information shares some functions with the AI assistant service information described above.

Further, synonyms related to parameters as an example of association information are also associated and stored in the entity information. This synonym is associated with, for example, "reading", "reading", "reading", etc. as giving the same command and processing to the MFP 6 for the utterance contents such as "scan" and "scan". ing. By registering such synonyms, for example, when the cloud service device 5 reads a document using the MFP 6, even if it says "scan this at 1000 dpi", it says "read this at 1000 dpi". Even if you speak, you can set it as a parameter that performs the same processing. That is, the cloud service device 5 can be interpreted as the same process.

(Interactive operation)
In the voice operation system 1 of the present embodiment, the interactive operation by the interactive system in which the system responds based on the voice data given by the user's utterance is realized. As described above, this interactive operation is one of the operations executed by the feedback unit 253 of the smart speaker 2. In addition to responding to fixed phrases required for dialogue, the voice operation system 1 responds with two types of responses, "input shortage feedback" and "input confirmation feedback", as specific responses related to reading a manuscript in the MFP 6. do. As a result, the voice operation system 1 realizes an interactive image reading operation system that enables interactive reading processing, printing processing, and the like in the MFP 6.

The “insufficient input feedback” is a response output from the smart speaker 2 when the information necessary for executing the reading of the document in the MFP 6 is not prepared. Further, "insufficient input feedback" is when the input content of the voice data given by the user's utterance cannot be recognized, or the items required for the input content by voice operation (hereinafter referred to as essential parameters) are insufficient. If so, it is output from the smart speaker 2. In other words, for items other than the required parameters (hereinafter, also simply referred to as parameters), it is not necessary to provide insufficient input feedback even if the user does not instruct. On the other hand, the "insufficient input feedback" may include a process of confirming a function required for reading a document in the MFP 6 in addition to the parameters.

In the interactive operation, the feedback unit 253 may change the function and parameters to be confirmed by the user according to the type of the image reading device to which the cloud service device 5 is connected to the communication. In this case, the device information acquisition unit 457 of the AI assistant server device 4 acquires information indicating the type and function of the image reading device at a predetermined timing after the communication with the image reading device is established. After that, the device information acquisition unit 457 may determine the function and parameters that the feedback unit 253 confirms with the user based on the acquired information.

For example, when the image reading device is the MFP 6, the feedback unit 253 can confirm with the user the items (user name, date and time of use, etc.) necessary for reading the original by the MFP 6. Further, the feedback unit 253 may confirm with the user information such as equipment resources used in the MFP 6. Further, the device information acquisition unit 457 may change the essential parameters according to the setting conditions specified by the user. For example, when the reading condition of the document specified by the user is a spread page reading, the device information acquisition unit 457 has a specific condition required for reading the document (for example, reading the document by ADF or reading the document by the platen). Read or etc.) may be set as a required parameter.

The “input confirmation feedback” is a response that is output when the information necessary for executing the reading of the original by the MFP 6 is prepared. That is, "input confirmation feedback" is performed when all the required parameters are instructed. Further, the "input confirmation feedback" is performed to prompt the user to select whether to execute the reading process with the current set value or to change the set value. It should be noted that all the parameters (essential parameters or parameters other than the essential parameters) instructed by the user in order to confirm whether or not the reading process is executed with the current set value by performing the "input confirmation feedback" are performed. Regardless of) can be confirmed to the user.

(Example of information fed back from the AI assistant server device)
In the above description, the feedback unit 253 of the smart speaker 2 has been described as outputting text data and voice data included in the response information. However, the feedback unit 253 forms text data corresponding to the response information based on the text data stored in the storage unit such as the ROM 23 of the smart speaker 2, and the feedback output (at least one of the audio output and the text output). ) May be performed. The specific content of the feedback will be described later.

Next, a specific example of the linking DB 402 will be described with reference to FIG. FIG. 5 is an example of a data table managed by the linking DB 402 used in the voice operation system 1 as an example of the information processing system. For example, in the present embodiment, the name of the image reading device to which the document reading instruction is given from the smart speaker 2 having "ud1001" as the device ID is "MFP_ # 1", and the device ID of "MFP_ # 1" is , "D0001". Although detailed description thereof will be omitted below, the device management table 402b of the linking DB 402 shown in FIG. 5 is associated with the device ID of the voice acquisition device, the image reading device name, and the device ID for each voice acquisition device name. Has been done. That is, in the associating DB 402, the device ID of each smart speaker 2 and the device ID of the MFP 6 are stored in association with each other so that the smart speaker 2 and the MFP 6 can be identified. The types and values of the respective IDs shown in FIG. 5 are examples, and are not limited to the above-mentioned contents.

<Functional configuration of MFP>
Based on the reading instruction transmitted from the cloud service device 5 (or AI assistant server device 4), the CPU 601 of the MFP 6 expands the execution program related to reading the document stored in the storage means such as the HDD 609 into the RAM 602b and executes it. do. By executing this reading command, the CPU 601 can function or function as, for example, a communication control unit 651, a command receiving unit 652, a determination unit 653, a reading execution unit 654, a notification unit 655, and a storage / reading processing unit 656. Configure.

<Each functional configuration of MFP>
Next, each functional configuration of the MFP 6 will be described. The communication control unit 651 communicates with the communication control unit 451 of the AI assistant server device 4 via the network 7. However, direct communication may be performed with the cloud service device 5 (or the AI assistant server device 4).

The instruction receiving unit 652 receives various instructions such as a reading instruction executed by the MFP 6 from the cloud service device 5 (or the AI assistant server device 4). That is, the instruction receiving unit 652 functions as a receiving means for receiving various instructions such as reading instructions from the cloud service device 5 (or AI assistant server device 4).

When the command receiving unit 652 receives the reading command, the determination unit 653 determines that the HDD 64 or the like is based on the information related to the reading command (image reading device name, device ID of the image reading device, user name, user ID, etc.). It searches various information stored in the storage unit, identifies the file to be executed by the read instruction, and generates a read instruction or a predetermined processing request to the cloud service device 5 (or AI assistant server device 4). ..

The reading execution unit 654 executes a reading process in the MFP 6 based on the reading instruction received by the instruction receiving unit 652. Further, for example, when the command receiving unit 652 receives the reading command, the reading execution unit 654 updates the reading status of the document stored in the storage unit such as the HDD 609 based on the above-mentioned various information included in the reading command. do. On the other hand, if the MFP 6 cannot read the document for some reason, it may receive a status signal or the like from the MFP 6 and notify the error to the outside. At that time, the error notification is transmitted directly from the MFP 6 to the smart speaker 2. Further, when the cancellation request related to the reading of the original is acquired in response to the reception of the error notification, the reading execution unit 654 deletes the reading process of the MFP 6 that matches the conditions from the storage unit.

As described above, the reading execution unit 654 executes the reading process or the like based on the contents of the reading command to the MFP 6 and the instruction of the predetermined processing given by the voice operation performed by the user to the smart speaker 2. Responsible for the function of means. In the present embodiment, the reading execution unit 654 has described the reading process in the MFP 6 as an example, but in the case of an image reading device that performs file transmission to an external device, storage in storage, etc. in addition to the reading process. Is capable of output processing such as outputting (transmitting) each file and data received by the image reader in an output format included in a predetermined output request.

The notification unit 655 notifies the smart speaker 2 of the status of the MFP 6. The contents to be notified are, for example, information related to reading the manuscript of the device and other operations, and information related to activation or login of the device. The notification unit 655 may notify the smart speaker 2 of the various information described above when the document reading instruction given by the user is received. On the other hand, the notification unit 655 may notify the smart speaker 2 of the various information described above when the start time of reading the document included in the received reading command is reached. In addition, if there is a duplication in the reading content of the document included in the scanning command, or if a failure occurs in the device before a predetermined time (for example, 10 minutes) before the start time of scanning the document, a notification is sent. The unit 655 may notify the smart speaker 2 via the communication control unit 651 regarding duplication of contents related to the reading process, a failure, or the like by e-mail, image distribution, or the like.

The storage / reading processing unit 656 controls a storage unit such as the HDD 609 to read / write various data.

In the present embodiment, the communication control unit 651 to the storage / reading processing unit 656 are realized by software, but some or all of them are realized by hardware such as an IC (Integrated Circuit). You may.

<Overview of voice operation system processing>
The voice operation system 1 according to the present embodiment includes a smart speaker 2 that collects sound to obtain voice data, an MFP 6 that reads an image of a document at least once, and a voice transmitted by the smart speaker 2. A cloud service device 5 (or an AI assistant server device 4) that receives data, converts the received audio data into a reading command for reading a document based on a predetermined reading condition, and transmits the data to the MFP 6. In the cloud service device 5 (or AI assistant server device 4), the voice data based on the utterance content to the smart speaker 2 continuously given by the user can continuously execute the read command transmitted immediately before. Whether or not, that is, whether or not there is a reading condition for continuously reading the original. When the cloud service device 5 (or the AI assistant server device 4) determines that there is a reading condition for continuously reading the document, the cloud service device 5 (or the AI assistant server device 4) retransmits the reading command inheriting the reading condition to the MFP 6 and the reading command. The MFP 6 that has received the above is configured to continuously read the original based on the retransmitted reading command. The above configuration will be described in detail below.

[Processing or operation of the embodiment]
<First Embodiment>
8a and 8b are sequence diagrams showing an example of a reading process based on a user's utterance in the first embodiment. In the first embodiment, a process of transmitting a document reading request to the MFP 6 from the AI assistant server device 4 and converting it into a reading command when continuously reading the document will be described. Specifically, in the first embodiment, a case where a reading command is transmitted from the AI assistant server device 4 to the MFP_ # 1 by the user's utterance, and the MFP 6 performs the reading of the document and the continuous reading process is illustrated. .. Each process in the sequence diagram is shown below.

In the voice operation system 1 of the present embodiment, a state in which the smart speaker 2 is arranged next to or close to the MFP 6 used by the user will be described as an example. In this state, the user first opens a desired page of the bound document, for example, and places it on the platen of the MFP 6 in an operable state. At this time, the user may put his / her hand on the bound document and hold it on the platen. Then, the user speaks to the smart speaker 2 "scan to Mr. Tanaka at 1000 dpi". Along with this utterance, the acquisition unit 252 of the smart speaker 2 uses, for example, the microphone unit 29 shown in FIG. 3 to transmit voice data based on the utterance voice "scan to Mr. Tanaka at 1000 dpi" uttered by the user. Acquire (step S101).

If the content uttered by the user does not include the resolution, destination, etc. related to reading the document, as in the case of simply "scanning", the complementary unit 455 of the AI assistant server device 4 performs feedback processing described later. Controls the smart speaker 2 to make inquiries such as "How many dpi do you want to scan?" And "Who do you want to send the scanned document to?" By voice. This query contains information to complement the various parameters. That is, the voice operation system 1 in the present embodiment is premised on one question (one-to-one feedback processing) being asked for one content uttered by the user. However, the voice operation system 1 has one question (many-to-one feedback processing) for a plurality of contents uttered by the user, or a large number of questions (one-to-many) for one question uttered by the user. It may be controlled so that feedback processing) is performed.

The content of the utterance for causing the MFP 6 to scan the document is not limited to the above-mentioned content. For example, the utterance content may include content that specifies scan settings indicating various settings for reading the original.

Subsequently, the communication control unit 251 of the smart speaker 2 transmits the acquired voice data to the voice recognition server device 3. The transmission timing for transmitting this voice data is an example of the first timing. Further, the above-mentioned voice data transmitted at the first timing is an example of the first voice data. At this time, the communication control unit 251 also transmits the device ID of the smart speaker 2 to the voice recognition server device 3 (step S102).

The device ID is an example of information for identifying the smart speaker 2 associated with the user, and is as shown in the user management table 402a. The communication control unit 251 uses, for example, in place of or in addition to the device ID, the position information of the smart speaker 2, the user ID that identifies the individual user who uses the smart speaker 2, the user name, or an individual such as an organization to which the user belongs. You may send specific information.

Subsequently, the acquisition unit 352 of the voice recognition server device 3 acquires the voice data transmitted from the smart speaker 2 together with the device ID via the communication control unit 351 and converts it into text (step S103).

The acquisition unit 352 may also have the function of the acquisition unit 252 for acquiring the voice data provided in the smart speaker 2. In that case, the acquisition unit 352 has a function of acquiring, for example, the instruction voice of the user collected through the microphone unit 29 together with the device ID of the smart speaker 2 and the user ID of the user. That is, the functions of step S101 and step S102 described above are also used. Such an acquisition unit 352 may function as an example of the voice data acquisition means, similarly to the acquisition unit 252 of the smart speaker 2. That is, the voice recognition server device 3 may function as an example of a server device provided with voice data acquisition means.

Subsequently, as a specific example of text conversion, the text conversion unit 353 of the voice recognition server device 3 converts the acquired voice data into text. In this text conversion process, for example, information based on a voice operation with the content of "scanning to Mr. Tanaka at 1000 dpi" is converted into text data.

Subsequently, the operation voice conversion program transmits a request for dictionary information to the AI assistant server device 4 via the communication control unit 351 to the voice assistant program executed by the AI assistant server device 4 (step S104). ..

The acquisition unit 452 of the AI assistant server device 4 acquires a request for dictionary information from the voice recognition server device 3 via the communication control unit 451. The AI assistant server device 4 that has acquired the textualized voice data provides the dictionary information to the operation voice conversion program executed by the voice recognition server device 3 in response to the request for the acquired dictionary information (step S105).

Subsequently, the interpretation unit 354 generates an intent and a parameter from the textualized voice data (step S106). As a specific example of the process of step S106, the interpretation unit 354 matches the words included in the text data, the words having a predetermined meaning, and the like with the dictionary information based on the dictionary information acquired from the voice assistant program. Judge whether or not. That is, it interprets the text. When the word contained in the text data and the word having a predetermined meaning match the dictionary information, the interpretation unit 354 has a parameter indicating a variable such as an intent indicating an operation instructed by the user and execution conditions of various processes. Convert to. The processing of the interpretation unit 354 described above may be performed by the providing unit 356.

In the present embodiment, the intent is, for example, information indicating the type of job requested for the MFP 6, that is, information indicating the execution of the reading process requested for the MFP 6. Further, the parameter is, for example, information indicating the setting of the job transmitted by the AI assistant server device 4 to the MFP 6, that is, information indicating various settings such as the resolution related to reading the original and the destination of data in the reading process. be. The intents and parameters to be converted are, for example, "intent: SCAN EXECUTE" (corresponding to "Action" in Table 1). The parameters are, for example, "resolution: 1000 dpi" and "destination: Tanaka". However, the present invention is not limited to the above-mentioned example, and information on other reading-related settings (reading size, color / monochrome, etc.) to be transmitted to the MFP 6 may be included in the parameters.

When sending an email to another device, or when the cloud service device 5 identifies the manuscript to be read, the parameters are the address of the device to which the email is sent, and the external cloud device. It may be information related to a file such as a device ID, a file name of a file to be transmitted, and a network address indicating a storage location of the file.

More specifically, the interpretation unit 354 generates "intent: SCAN EXECUTE" as information related to the intent generated when the reading of the document is executed in the MFP_ # 1 operated by the user. Further, the interpretation unit 354 generates, for example, "image reading device name: MFP_ # 1" as information related to the parameters of the reading process executed in the MFP_# 1. In this way, the interpretation unit 354 indicates, for example, a document reading instruction given by the user, a predetermined processing type (intent), and contents (parameters) related to the predetermined processing, based on the acquired text data. Generate an interpretation result.

Subsequently, the interpretation unit 354 transmits the generated intent, parameters, and device ID of the smart speaker 2 to the management program executed by the AI assistant server device 4 (step S107).

<Information complement processing>
Next, an example of information complement processing executed by the cloud service device 5 (or AI assistant server device 4) will be described.

First, the interpretation result conversion unit 453 of the AI assistant server device 4 converts the data indicating the reading command for the MFP_ # 1 based on the intent, the parameter, the device ID of the smart speaker 2, and the like acquired by the acquisition unit 452. .. At this time, the intent is given "MFP_ # 1" or the like that reads the original and represents the name of the image reading device. Hereinafter, MFP_ # 1 will be described as an example of the image reading device, but any type of image reading device as illustrated in the device management table 402b may be used. Further, the parameters may be of any type as long as they have the contents illustrated in the device management table 402b and the instruction management table 402c.

With the data conversion by the interpretation result conversion unit 453, the search unit 460 uses the device ID of the smart speaker 2 shown in the user management table 402a, the user name of the user, the user ID of the user, and the information managed by the device management table 402b. The MFP_ # 1 provided with the scanner for reading the document is specified based on the above. When specifying the MFP_ # 1, the search unit 460 specifies the image reading device based on various information stored and managed in the device management table 402b. That is, the device ID of the image reading device is collated based on the device ID of the voice acquisition device, and the image reading device is specified. However, it is assumed that the smart speaker 2 is moved from the vicinity of the MFP_ # 1 for some reason and does not match the various information stored and managed in the device management table 402b. In such a case, the search unit 460 acquires at least one of the device ID of the smart speaker 2 and the user ID of the user stored and managed in the user management table 402a, and then the smart speaker 2 and the MFP_ #. It may function to acquire the position information or the like indicating each installation position of 1 and check the validity of the apparatus management table 402b from the mutual positional relationship. If it is determined that the installation positions of the smart speaker 2 and the MFP_ # 1 are deviated from each other by a predetermined value, the search unit 460 determines that the MFP that reads the original is placed near the user for the smart speaker 2. Feedback may be given to the smart speaker 2 so as to convey by voice that it does not exist.

Further, the complement unit 455 refers to the device management table 402b and the instruction management table 402c stored in the associating DB 402 with respect to the voice data given by the user's utterance, and the reading instruction executed by the MFP_ # 1. Complements the information required for conversion (generation) of (step S108). However, if the information necessary for generating the essential parameters related to the read instruction cannot be supplemented even with reference to the device management table 402b and the instruction management table 402c, the complement unit 455 provides feedback to the user via the smart speaker 2. May be controlled to prompt the user to input (acquire) the information necessary for generating the essential parameters. The complement processing related to the complement is performed by the complement unit 455, and the complement unit 455 corresponds to the complement means.

At this time, the management unit 459 may associate the device ID, the user ID, and the information processing device name (MFP_ # 1 or the like) with the linking DB 402 and register them as the user management table 402a and the device management table 402b. can.

FIG. 9 is a flowchart showing an example of information complementation and inquiry processing in the first embodiment.

The acquisition unit 452 of the AI assistant server device 4 acquires an intent, a parameter, a device ID, and the like from the voice recognition server device 3 in the process of step S107 (step S1001).

Subsequently, the interpretation result conversion unit 453 determines whether or not the essential parameters are satisfied from the acquired data such as the intent, the parameter, and the device ID (step S1002). As a method of determining whether or not this essential parameter is satisfied, for example, the interpretation result conversion unit 453 has an intent, a parameter, a device ID, etc. acquired by the user name, the user ID, the information necessary for reading the manuscript, and the like. There is a method of confirming whether or not it is included in the data of. The above determination is realized by the interpretation result conversion unit 453 referring to, for example, the user management table 402a, the device management table 402b, and the instruction management table 402c stored in the linking DB 402.

Subsequently, when it is determined from the acquired data such as the intent, the parameter, and the device ID that the essential parameters are satisfied (Yes in step S1002), the interpretation result conversion unit 453 receives the received intent, the parameter, the device ID, and the like. The data in the above is converted into a read instruction for MFP_ # 1 (MFP6), and this flow is exited (step S1003).

On the other hand, when it is determined from the acquired data such as the intent, the parameter, and the device ID that the required parameters are not satisfied (No in step S1002), the interpretation result conversion unit 453 is stored and managed by the linking DB 402. It is determined whether or not the essential parameters can be satisfied by the information of the various tables (user management table 402a, device management table 402b, and instruction management table 402c) (step S1004).

When it is determined that the essential parameters are satisfied by the information of various tables (Yes in step S1004), the interpretation result conversion unit 453 converts the information into a read instruction for MFP_ # 1 based on the supplemented contents and exits this flow (step). S1005).

On the other hand, when it is determined that the essential parameters are not satisfied by the information in the various tables (No in step S1004), the interpretation result conversion unit 453 asks the user to input the necessary information again for the inquiry of the essential parameters. Feedback processing is performed, and this flow is exited (step S1006). The above is an example of the information complement processing executed by the interpretation result conversion unit 453.

As a method of specifying the user ID described above, for example, there are the following methods. When a user uses the smart speaker 2, he / she speaks his / her name toward the smart speaker 2 toward the microphone unit 29. In response to the input of the name by this utterance, the acquisition unit 352 of the voice recognition server device 3 converts the input name into text. Subsequently, the interpretation result conversion unit 453 of the AI assistant server device 4 collates the user names stored and managed in the user management table 402a described above, and identifies the user ID of the user who has spoken. Instead of inputting the name, the user's e-mail address or the like may be spoken. Further, the image pickup unit (camera unit) 30 of the smart speaker 2 may be used to take a photograph of the user's face or the like, and the photographed image may be collated with the user ID.

Further, as another example, when the smart speaker 2 and the user who uses the smart speaker 2 change, and information regarding reading of a document in MFP_ # 1 is given by a new combination of user ID and device ID, management is performed. Even if the instruction management table 402c is updated by the unit 459 by adding the information to the item of MFP_ # 1 of the instruction management table 402c stored and managed in the storage unit such as the HDD 44 of the AI assistant server device 4. good.

Subsequently, the device information acquisition unit 457 determines whether or not the essential parameters required for reading the original in the MFP_ # 1 are satisfied based on the acquired intents and parameters. This essential parameter is, for example, information for specifying the attribute information related to the reading of the document to be read among the received parameters. That is, arbitrary conditions such as the file format of the file generated after reading the original, the resolution at the time of reading the original, the color / monochrome setting, and the destination at the time of transmitting the file can be set in the essential parameters.

Further, this essential parameter can be stored in advance in a storage unit such as HDD 44 of the AI assistant server device 4 as the instruction management table 402c related to the above-mentioned MFP_ # 1, and can be appropriately set. Further, as for this essential parameter, it is possible to replace the essential parameter with the normal parameter as appropriate according to the combination of the user and the image reading device. That is, it may be defined as an essential parameter under certain conditions, or may be managed as a normal parameter under another condition.

From the above description, the device information acquisition unit 457 has the following features. That is, the device information acquisition unit 457 supplements the information related to the generation of the essential parameters by referring to the device management table 402b and the instruction management table 402c stored in the linking DB 402. Information related to the generation of essential parameters includes information for identifying a manuscript (one-page manuscript, bound manuscript containing multiple pages, etc.) and an image reading device (MFP_ # 1, etc.) in reading the manuscript. Can be mentioned. Specifically, for example, it is information such as "Mr. Tanaka" and "1000 dpi". However, if the information related to parameter generation, the device management table 402b, and the instruction management table 402c cannot be supplemented with the information necessary for generating the essential parameters, the device information acquisition unit 457 complements the essential parameters. As an inquiry for the operation, a request for the missing parameter is transmitted to the smart speaker 2 (step S109).

Further, the acquisition unit 252 of the smart speaker 2 that has received the parameter request from the device information acquisition unit 457 in step S109 transfers the received parameter request to the feedback unit 253. The feedback unit 253 converts the information corresponding to the parameter request into voice, feeds it back to the user via the communication control unit 251 and controls the user to input the information necessary for generating the essential parameters (step). S110). The processes of steps S109 and S110 correspond to the processes of step S1006 of the flowchart described above in FIG. However, when it is determined in FIG. 9 that the processing of step S1006 is unnecessary, the processing of steps S109 and S110 described above is not performed (omitted).

Subsequently, the execution determination unit 454 determines the satisfaction of the essential parameters based on the above-mentioned complement processing. At that time, if the execution determination unit 454 determines that the essential parameters are not satisfied even with the contents complemented by the complement processing, the execution determination unit 454 generates response information for inquiring the essential parameters. Based on the generation of the response information, the notification unit 458 transmits the generated response information to the smart speaker 2 and informs the user by the voice output from the smart speaker 2 or the like.

If the required parameters are not satisfied, the execution determination unit 454 generates response information that specifies the parameters until the required parameters are satisfied, and continues the inquiry to the user via the smart speaker 2. Works for. Regarding the complementation of the information executed in this way and the inquiry processing of the essential parameters, the interpretation result conversion unit 453 and the execution determination unit 454 cooperate with each other to obtain information related to the voice data given by the user's utterance. It may function as an acquisition control unit 462 for complementation.

Further, the essential parameters may be changed based on at least one of the device ID and the user ID acquired from the smart speaker 2 in step S102. Further, it is preferable that the essential parameters include (user name (user name), user ID of the user, etc.) of the user who uses MFP_ # 1. However, if the user does not set the user name, user ID, etc. of the MFP_ # 1 by voice operation or the like, the execution determination unit 454 determines the device ID and user ID acquired from the smart speaker 2 in step S102. Determine if the user can be identified by at least one of them. For example, one smart speaker 2 may be occupied by one user. Therefore, the execution determination unit 454 determines whether the user associated with the device ID of the smart speaker 2 and the user ID of the user is registered in the association DB 402. That is, the execution determination unit 454 has a function of searching for a user based on the device ID and the user ID and identifying the user.

Here, when the execution determination unit 454 can identify the user, the specified user can be set in the parameter as the user of MFP_ # 1. On the other hand, when the user cannot be identified, the execution determination unit 454 may inquire the user to set the user information via the smart speaker 2. That is, in order to generate data indicating a predetermined processing request (reading instruction, etc.), the execution determination unit 454 communicates with the smart speaker 2 via the notification unit 458 and the communication control unit 451 to provide complementary information to the user. You may request the input of.

The parameter may include information related to the user of MFP_ # 1. However, when the user does not set the information related to the user by voice operation, that is, the user name, the user ID, and the like, the execution determination unit 454 performs at least one of the device ID and the user ID acquired from the smart speaker 2 in step S102. It is determined whether or not MFP_ # 1 can be specified by the speaker.

Based on the above determination, the search unit 460 searches for and identifies the MFP_ # 1 used for reading the manuscript. Here, when the MFP _ # 1 can be specified, the search unit 460 sets the MFP _ # 1 as an image reading device used for reading the original as a parameter. On the other hand, when the MFP_ # 1 cannot be specified, the search unit 460 may inquire the user to set the MFP_ # 1 via the smart speaker 2 in cooperation with the notification unit 458.

Even when the user sets the MFP_ # 1 by voice operation, there may be a plurality of image reading devices including the same name as the set MFP_ # 1. Therefore, the execution determination unit 454 may determine whether or not the MFP_ # 1 can be specified by at least one of the device ID and the user ID in addition to the name of the MFP_ # 1 set by the voice operation. That is, the execution determination unit 454 determines whether the MFP_ # 1 associated with the device ID and the user ID is registered in the association DB 402. Following this, the search unit 460 searches for MFP_ # 1 based on the device ID and user ID in addition to the name of MFP_ # 1 set by voice operation, and identifies the target MFP_ # 1 from the search results. do.

Here, Action (action) and Parameter (parameter) as the table data of Table 1 used in the present embodiment will be described with reference to the specific examples shown in Table 1. The interpretation result conversion unit 453 of the AI assistant server device 4 is described in detail below, for example, in order to convert the data indicating the reading command in the MFP_ # 1 based on the interpretation result interpreted by the voice recognition server device 3. The information shown in Table 1 may be stored in a storage unit such as HDD 44 of the AI assistant server device 4 and can be referred to.

The AI assistant server device 4 stores table data including a reading command for the image reading device shown in Table 1 in a storage unit such as the HDD 44. The interpretation result conversion unit 453 of the AI assistant server device 4 stores the information corresponding to Table 1 in a storage unit such as HDD 609 of the MFP 6 in order to convert the interpretation result obtained by the voice recognition server device 3 into a read command. It may be configured so that it can be stored and referred to.

In the case of the example of Table 1, for example, "SCAN EXECUTE", "EMAIL EXECUTE", "STORE EXECUTE" and the like are shown as examples of actions or intents. Further, "1000 DPI", "Tanaka" and "ADDRESS" are shown as examples of parameters. The parameters include all parameters that can be specified as setting values for the reading command to the MFP 6.

In the present embodiment, for example, the interpretation result conversion unit 453 converts the interpretation result of "SCAN EXECUTE" into an instruction indicating "execution of reading a document" for MFP_ # 1. Similarly, the interpretation result conversion unit 453 converts the interpretation result of "EMAIL EXECUTE" into an instruction indicating "transmission of email" to MFP_ # 1. Similarly, the interpretation result conversion unit 453 converts the interpretation result of "STORE EXECUTE" into an instruction indicating "save in storage service" for MFP_ # 1.

That is, the interpretation result conversion unit 453 of the AI assistant server device 4 determines the type of the read instruction for the MFP_ # 1 based on the information included in the action or intent of the interpretation result, and sets the value included in the parameter for the read instruction. Judging as a value, the interpretation result is converted into a reading instruction.

In addition, the execution determination unit 454 stores table data including an execution instruction of a predetermined process different from Table 1 in a storage unit such as HDD 44, and interprets the table data by the interpretation result conversion unit 453. The interpretation result may be fed back to the smart speaker 2.

<Read instruction conversion process>
Next, the conversion to the read instruction executed by the AI assistant server device 4 will be described. The interpretation result conversion unit 453 converts the information supplemented in step S108 into, for example, a reading instruction executed by the MFP _ # 1 and transmits the information to the MFP _ # 1 via the communication control unit 451 (step S111). The read instruction at this time is an example of the first read request. In this case, for example, a reading command corresponding to the instruction content such as "scan this", "scan this to Mr. Tanaka", or "scan this at 1000 dpi" instructed by the user by utterance or the like is issued. , Is transmitted from the cloud service device 5 (or AI assistant server device 4) to the MFP_ # 1 via the communication control unit 451. The communication control unit 451 is an example of communication means.

FIG. 10 is a flowchart showing an example of conversion and transmission of a reading instruction according to the first embodiment.

In FIG. 10, the interpretation result conversion unit 453 and the execution determination unit 454 are voice data given by the user's utterance acquired by the smart speaker 2, a device ID for identifying the smart speaker 2, attribute information related to reading the original, and an MFP 6 Based on the information including the device ID that identifies

First, the interpretation result conversion unit 453 acquires the satisfied essential parameters and converts them into a reading instruction (step S1101). For example, if the utterance contains a content instructing reading such as scanning, it is converted into a reading command. Furthermore, even if the utterance does not include the content that explicitly instructs reading such as "next" or "continue", if the previous instruction is a reading instruction, it is converted into a reading instruction. can do. For example, the execution determination unit 454 specifies the device ID of the image reading device associated with the device ID of the voice acquisition device acquired from the smart speaker 2 from the device management table 402b. When the instruction including the specified device ID is included in the instruction management table 402c but the job type is not explicitly included in the utterance, the execution determination unit 454 can determine that the instruction is a read instruction. ..

Subsequently, the execution determination unit 454 determines whether or not the converted reading command is the first reading of the original. Whether or not the document is read for the first time with respect to the transferred reading command is determined, for example, by checking the value of the continuous processing flag stored and managed in the command management table 402c. Specifically, the execution determination unit 454 determines whether the value of the continuous processing flag is "0" or "1". That is, the execution determination unit 454 specifies the device ID of the image reading device associated with the device ID of the voice acquisition device acquired from the smart speaker 2 from the device management table 402b. Subsequently, the execution determination unit 454 specifies an instruction including the specified device ID from the instruction management table 402c, and confirms the value of the continuous processing flag included in the specified instruction. In this process, the execution determination unit 454 confirms that the value of the continuous process flag is "0". At this time, if the value of the continuous processing flag is "0", the execution determination unit 454 determines that the original is read for the first time, and performs a process for reading the original with the attribute information related to the reading of the original. conduct. On the other hand, if the value of the continuous processing flag is "1", the execution determination unit 454 determines that the document has been read for the second time or later. (Step S1102). Therefore, the value of the continuous processing flag is an example of the scanning conditions for continuing the scanning of the original. The value of the continuous processing flag may be given as an initial value of "0" in the initial state in which a certain document is read. In this way, when converting the voice instruction given by the user into a reading instruction, a continuous processing flag indicating that the original is continuously read (continuously) may be included as a parameter.

When it is determined by checking the value of the continuous processing flag that the document has been read for the first time (Yes in step S1102), that is, when the continuous processing flag is confirmed as "0", the execution determination unit 454 Transmits a scanning command for executing the first document scanning process based on the attribute information related to document scanning described above to the MFP_ # 1 via the communication control unit 451 (step S1103). Further, the execution determination unit 454 changes the value of the continuous processing flag from "0" to "1".

On the other hand, when it is determined by checking the value of the continuous processing flag that it is not the first reading of the original (No in step S1102), that is, when the continuous processing flag is confirmed as "1", the execution determination unit 454 determines whether or not the converted scanning command is the content that terminates the scanning of the original (step S1104). In this step, the execution determination unit 454 searches for, for example, a reading command having a meaning such as “end” or “or more” to end the reading of the document. To give utterances such as "end" and "or more", press the "#" symbol arranged or displayed on the operation unit when executing the copy, print, scan function, etc. in a well-known image forming apparatus or the like. Corresponds to the process of indicating that the final manuscript and the final page. If a request for reading a manuscript is made for the manuscript within a predetermined time from the acquisition of the previous audio data, the execution determination unit 454 may determine that the manuscript is a continuous request for reading the manuscript. good.

When the reading command does not include words such as "end" and "or more" that have the meaning of ending the reading of the original (No in step S1104), the execution determination unit 454 outputs the reading command transmitted immediately before. Retransmission to MFP_ # 1 returns to step S1101, and the process is repeated until the user instructs the end of reading the original (step S1105). In step S1105, the execution determination unit 454 receives voice data when the user's instruction is a read instruction, that is, when the voice recognition server device 3 receives voice data such as "next" and "continue". Even if the parameters generated from the data do not include the required parameters for reading the original, the required parameters for reading the original are satisfied provided that the continuous processing flag is "1". Judge. That is, in the process in step S1105, the execution determination unit 454 retransmits to the MFP_ # 1 together with the reading command transmitted immediately before and the reading conditions such as various parameters transmitted immediately before. The voice data acquisition timing executed from this step S1105 to the next step S1101 is an example of the second timing, which is a timing after the first timing described above. The voice data acquired at the second timing is an example of the second voice data. Further, among the second voice data, voice data having a meaning such as "next" and "continue" that can continuously execute the read instruction executed immediately before is read based on a predetermined reading condition. This is an example of the content that enables the execution of instructions. That is, the voice data such as "next" and "continue" is also an example of the content that inherits the predetermined reading conditions.

However, the execution determination unit 454 does not have to transmit various parameters in the case of continuous reading of the original. That is, the execution determination unit 454 may send only an instruction to execute reading. In this case, the MFP 6 executes reading based on various parameters acquired before that.

When the execution determination unit 454 acquires the same intent within a predetermined time from the reception of the previous intent, it may determine that the original is continuously read. Further, the interpretation result conversion unit 453 can generate "JOB_EXECUTE" as an interpretation result as an intent when the type of job is not explicitly included in the utterance such as "next" or "continue". In this case, the execution determination unit 454 may determine that the continuous reading of the document is performed even when the execution determination unit 454 receives an intent such as "JOB_EXECUTE" that does not specify the job type.

On the other hand, when the scanning command includes words such as "end" and "or more" that have the meaning of ending the reading of the document (Yes in step S1104), the execution determination unit 454 ends the reading of the document. A request is generated, the corresponding read instruction is deleted from the instruction management table 402c, the value of the continuous processing flag is set to "0", and this flow is exited (step S1106). In the process of step S1106, the execution determination unit 454 may only delete the corresponding read instruction, or may delete the corresponding read instruction after setting the value of the continuous processing flag to "0". Further, the execution determination unit 454 may generate an end request when the instruction is not received from the user for a predetermined time or more. A reading instruction including words having the meaning of ending the reading of the original such as "end" and "or more" is an example of the second reading request.

In the present embodiment, when the voice uttered by the user contains content having a meaning irrelevant to the reading of the manuscript, the execution determination unit 454 of the AI assistant server device 4 executes the flowchart of FIG. Previously, the feedback process (step S108 in FIG. 8b) may be executed for the user in cooperation with the interpretation result conversion unit 453.

Further, in the present embodiment, in reading the same document, when the value of the continuous processing flag is "0", it indicates that the document is read for the first time, and when it is "1", the document is read for the second time or later. However, it is not limited to this. For example, in reading the same document, a "1" may indicate that the document has been read for the first time, and a "0" may indicate that the document has been read for the second time or later. Further, the value of the continuous processing flag is not limited to "0" and "1", and may be determined by using other values, character strings, symbols, and the like.

Further, when the original is read from the second time onward, the next original is the original in a state where the user has updated the pages of the original (pages are turned, the direction of the original is changed, etc.). It is assumed that it is placed on the table. Under such a premise, when the execution determination unit 454 confirms the value of the continuous processing flag and determines that the original is not read for the first time (No in step S1102), that is, the continuous processing flag is set to "1". If confirmed, as described in step S1105, the execution determination unit 454 is a reading command for executing the first document scanning process while maintaining the attribute information related to the document scanning that was executed immediately before. The same command as is transmitted to MFP_ # 1 via the communication control unit 451. Further, the execution determination unit 454 maintains the value of the continuous processing flag at "1".

Regarding the value of the continuous processing flag, when reading the same document, "0" indicates that the reading is executed for the first time, and "1" indicates that the reading is executed for the second time or later. However, it is not limited to this. For example, in reading the same document, a "1" may indicate that the reading is executed for the first time, and a "0" may indicate that the reading is executed for the second time or later. Further, the value of the continuous processing flag is not limited to "0" and "1", and may be determined by using other values, character strings, symbols, and the like.

<Manuscript reading and continuous processing>
Returning to the sequence diagram of FIG. 8b, the read instruction executed by MFP_ # 1 will be described. Upon receiving the read instruction from the AI assistant server device 4 in step S111, the MFP_ # 1 receives the read instruction on the network I / F650, generates various control signals corresponding to the contents of the instruction on the CPU 601 and causes the engine control unit 630 to generate various control signals. Send. The various control signals transmitted to the engine control unit 630 are read by the scanner unit 631 by controlling various drive systems for reading the document under the control of the scanning execution unit 654. In this way, the document scanning process and the continuous scanning process for the document are performed based on the received scanning command (step S112). When an instruction or command to end reading the original is received in step S112, the communication control unit 651 of the MFP_ # 1 stores or reads the scan data generated by reading the original in a storage unit such as HDD 609 of its own device. Send to the destination included in the instruction. If the read instruction does not include a destination, the MFP_ # 1 may accept the designation of the destination indicating the transmission destination of the scan data by accepting the operation from the user by the operation unit of the own device.

As described in the process of steps S1102 to S1106 of FIG. 10, the reading execution unit 654 of the MFP_ # 1 performs once with respect to the document according to the content of the reading instruction transmitted from the AI assistant server device 4. The document scanning process is performed for each of the case where the document is read in the above case and the case where the document is continuously read twice or more.

When the predetermined reading process is completed in the reading execution unit 654, the notification unit 655 transmits the end notification for the end request to the AI assistant server device 4 (step S113). The end request related to the reading process may be accepted by the operation unit of MFP_ # 1 according to the operation of the user, and as described above, the user utters, for example, "end" to the smart speaker 2. It may be terminated by doing. Based on the utterance content of "end", the communication control unit 451 of the AI assistant server device 4 acquires an intent instructing the end of the reading process such as "SCAN_END" or "JOB_END" generated by the operation voice conversion program. , Converted to a read command and transmitted to MFP_ # 1. Then, the scanning execution unit 654 of the MFP_ # 1 receives a scanning command, executes scanning of the document, generates a plurality of scan data generated as one file consisting of a plurality of pages, and stores and reads the storage / reading processing unit. It can be stored and stored in a storage means such as HDD 609 via the device. Further, the reading execution unit 654 can cooperate with the communication control unit 651 to transmit one generated file composed of a plurality of pages to an external device by e-mail transmission or the like.

Subsequently, the AI assistant server device 4 that has received the end notification from the MFP_# 1 transmits the presence / absence determination of the continuation command and the utterance request from the communication control unit 451 to the smart speaker 2 (step S114).

Further, the acquisition unit 252 and the feedback unit 253 of the smart speaker 2 that have received the continuation command presence / absence determination and the utterance request provide voice feedback to the user using the MFP_ # 1 and end a series of processes (step). S115).

The flowchart shown in FIG. 10 is an example, and the process executed by the execution determination unit 454 is not limited to the above-mentioned example. For example, the contents of the above-mentioned flowchart may be appropriately changed according to the environment in which the information processing system according to the present embodiment is placed, the use of the system, and the like.

By providing the above-described configuration in the first embodiment, when the user wants to continuously read the document, the user changes the page of the document or the direction of the document with the utterance voice for reading the document. You will be able to continue reading simply by giving a simplified spoken voice without having to speak everything every time.

<Second embodiment>
11a and 11b are sequence diagrams showing an example of a reading process based on a user's utterance in the second embodiment. The difference from the first embodiment is whether or not there is a reading condition for continuing reading of the document based on the reading command transmitted from the AI assistant server device 4 after the MFP 6 receives the reading command. When the MFP 6 determines and there is a reading condition for continuing the reading of the original, the processing of reading the original is performed by taking over the reading condition. Specifically, the case where the MFP_ # 1 receives the reading instruction converted from the document reading instruction given by the user's utterance, interprets and judges by itself, reads the document, and continuously reads the document is illustrated. .. Each process in the sequence diagram is shown below.

<Manuscript reading and continuous processing>
In the sequence diagrams of FIGS. 11a and 11b, steps S101 to S110 are the same as those of FIGS. 8a and 8b, and thus detailed description thereof will be omitted.

Similar to the first embodiment, the interpretation result conversion unit 453 of the AI assistant server device 4 converts the information supplemented in step S108 into, for example, a read instruction executed by the MFP_ # 1, and the communication control unit. It is transmitted to MFP_ # 1 via 451 (step S211). In this case, for example, a reading command corresponding to the instruction content such as "scan this", "scan this to Mr. Tanaka", or "scan this at 1000 dpi" instructed by the user by utterance or the like is issued. , Is transmitted from the cloud service device 5 (or AI assistant server device 4) to the MFP_ # 1 via the communication control unit 451.

Upon receiving the reading command from the AI assistant server device 4 in step S211th, the MFP_ # 1 performs a document scanning process and a continuous scanning process for the document based on the received scanning command (step S212).

Similarly in the second embodiment, the reading execution unit 654 of the MFP_ # 1 finishes reading the document once with respect to the document according to the content of the reading command transmitted from the AI assistant server device 4. The document reading process is performed for each of the case where the document is read and the case where the document is continuously read twice or more.

FIG. 12 is a flowchart showing an example of the execution processing of the reading instruction in the second embodiment. In the flowchart of FIG. 12, the reading command transmitted from the interpretation result conversion unit 453 of the AI assistant server device 4 via the communication control unit 451 in step S111 of FIG. 8b described above is executed by the MFP_ # 1 as a document reading process. It shows an example of being done.

First, the instruction receiving unit 652 of the MFP_ # 1 (MFP6) receives the reading instruction transmitted from the AI assistant server device 4 (step S1201). In the present embodiment, the command receiving unit 652 functions as an example of the reading command receiving means.

Subsequently, the determination unit 653 of the MFP_ # 1 determines whether or not the original is read for the first time in response to the reading command transferred from the instruction receiving unit 652. To determine whether or not the original is read for the first time with respect to the transferred reading instruction, for example, the value of the continuous processing flag stored and managed in the instruction management table 402c stored in the MFP 6 is confirmed. It is done in. In this process, the determination unit 653 confirms that the value of the continuous process flag is "0". At this time, if the value of the continuous processing flag is "0", the determination unit 653 determines that the original is read for the first time, and executes the reading of the original based on the attribute information related to the reading of the original. On the other hand, if the value of the continuous processing flag is "1", the determination unit 653 determines that the original has been read for the second time or later (step S1202). Therefore, the value of the continuous processing flag is an example of the scanning conditions for continuing the scanning of the original. The value of the continuous processing flag may be given as an initial value of "0" in the initial state in which a certain document is read. As described above, the reading instruction converted from the voice instruction given by the user may include a continuous processing flag indicating that the original is continuously read (continuously) as a parameter.

By checking the value of the continuous processing flag, if it is determined that the document has been read for the first time (Yes in step S1202), that is, if the continuous processing flag is confirmed to be "0", the determination unit 653 , The first document scanning process is executed based on the attribute information related to the document scanning described above (step S1203). Further, the determination unit 653 changes the value of the continuous processing flag from "0" to "1".

On the other hand, when it is determined by checking the value of the continuous processing flag that the document is not read for the first time (No in step S1202), that is, when the continuous processing flag is confirmed to be "1", the determination unit 653 Determines whether or not the converted scanning command ends the scanning of the original (step S1204). In this step, the determination unit 653 searches for, for example, a reading command having a meaning such as "end" or "or more" to end the reading of the document. To give utterances such as "end" and "or more", press the "#" symbol arranged or displayed on the operation unit when executing the copy, print, scan function, etc. in a well-known image forming apparatus or the like. Corresponds to the process of indicating that the final manuscript and the final page.

If the scanning instruction does not include words such as "end" and "or more" that have the meaning of ending the reading of the original (No in step S1204), the determination unit 653 re-executes the scanning instruction executed immediately before. It is executed by MFP_ # 1 to return to step S1101 and repeated until the user instructs the end of reading the original (step S1205). In step S1105, the determination unit 653 sets the continuous processing flag to "1" even when the parameters acquired from the voice data such as "next" and "continue" do not include the essential parameters. It is judged that the essential parameters are satisfied on the condition that. When the determination unit 653 acquires the same intent within a predetermined time from the reception of the previous intent, it may determine that the original is continuously read. Further, the determination unit 653 may determine that the original is continuously read even when the intent indicating that the original is being read, such as "JOB_EXECUTE", is received.

On the other hand, when the scanning command includes words such as "end" and "or more" that have the meaning of ending the reading of the original (Yes in step S1204), the scanning execution unit 654 reads the immediately preceding execution. Upon executing the instruction, the determination unit 653 generates an end request related to reading the original and deletes the corresponding reading instruction from the instruction management table 402c, sets the value of the continuous processing flag to "0", and exits this flow (step). S1206). In the process of step S1206, the determination unit 653 may only delete the corresponding read instruction, or may delete the corresponding read instruction after setting the value of the continuous processing flag to "0". A reading instruction including words having the meaning of ending the reading of the original such as "end" and "or more" is an example of the second reading request.

Even in this embodiment, there is no restriction on the relationship between the value of the continuous processing flag and the number of times the document is read in reading the same document. For example, in reading the same document, a "1" may indicate that the document has been read for the first time, and a "0" may indicate that the document has been read for the second time or later. Further, the value of the continuous processing flag is not limited to "0" and "1", and may be determined by using other values, character strings, symbols, and the like.

When the original is read from the second time onward, it is assumed that the next original is placed on the platen with the page of the original updated by the user (page turned). And. Under such a premise, when the determination unit 653 confirms the value of the continuous processing flag and determines that the original is not read for the first time (No in step S1202), that is, the continuous processing flag is confirmed to be "1". If this is done, the scanning execution unit 654, as described in step S1205, provides a scanning command for executing the first document scanning process while maintaining the attribute information related to the document scanning that was executed immediately before. Execute the same instruction. Further, the determination unit 653 maintains the value of the continuous processing flag at "1".

As described above, the continuously read original is described as being placed on the platen with the desired page updated (opened) by the user, but the page of the original is not updated and the next page is not updated. The processing when the reading instruction of is executed by MFP_ # 1 will be described in detail later (processing for duplicate reading).

Further, when the document is read based on the reading command received from the AI assistant server device 4, the MFP_ # 1 sets the operation unit of the own device to indicate the setting related to the reading of the document and the destination of the scan data. The screen to be displayed may be displayed. At the same time, the change of the printing conditions for the scanned document may be accepted, or the file may be transmitted to the external device of the scanned document on condition that the permission of the user is accepted.

Further, the MFP_ # 1 can also generate each page read from the manuscript as one file and transmit the generated file to the external device via the communication control unit 651. In this case, the communication control unit 651 functions as a file transmission means. Although the above-mentioned command receiving unit 652 has been described as functioning as a reading command receiving means for receiving a reading command from the AI assistant server device 4, the communication control unit 651 may also function as a reading command receiving means.

Here, the sequence diagram of FIG. 11b is returned, but since the processing of steps S213 to S215 is the same as that of steps S113 to S115 of FIG. 8b, detailed description thereof will be omitted.

Here, the MFP_ # 1 has the operation unit of the own device, each setting information related to the reading of the document, the file name acquired by the reading process, the storage setting in the internal storage of the own device, the file transmission condition to the external device, and the file transmission condition. A screen or the like indicating that printing may be executed may be displayed. At this time, the printing process of the scanned document may be executed on condition that the permission of the user is accepted.

The MFP_ # 1 may activate the power supply and network settings of the device in response to the reading command transmitted from the cloud service device 5 (or the AI assistant server device 4) to start reading the original. ..

Further, according to the present embodiment, the MFP_ # 1 can immediately execute the job without accepting the operation by the operation unit. As a result, the user can instruct the continuous reading of the original only by voice operation.

The flowchart shown in FIG. 12 is an example, and the process executed by the determination unit 653 is not limited to the above-mentioned example. For example, the contents of the above-mentioned flowchart may be appropriately changed according to the environment in which the information processing system according to the present embodiment is placed, the use of the system, and the like.

(Conditions judged as continuous reading of manuscript)
As described above, in the MFP_ # 1 of the present embodiment, the following can be considered as a case where it is determined that the original is continuously read with respect to the original. One is a case where the parameters included in the voice data given from the user's utterance, such as "next" and "continue", have contents that can be interpreted as meaning to continuously read the manuscript. ..

The other is a case where the voice data related to the reading of the manuscript for the manuscript is acquired within a predetermined time from the acquisition of the voice data related to the previous reading of the manuscript.

The other is a state in which a user logs in to MFP_ # 1, receives a read instruction by the instruction receiving unit 652 (or converts it into a reading instruction by the interpretation result conversion unit 453), and then the user logs in. When the command receiving unit 652 receives the next reading command (or is converted to the next reading command by the interpretation result conversion unit 453), or the command receiving unit 652 receives the reading command and the command receiving unit within a predetermined time. In at least one case when the 652 receives the next reading command, it may be determined that the reading of the original is continuous.

However, the conditions for determining continuous reading of a manuscript are merely examples, and there are no particular restrictions on the conditions for determining continuous reading of a manuscript as long as they do not deviate from the gist of the invention in the present embodiment.

(Processing for duplicate reads)
Each time the MFP_ # 1 reads the original, the MFP_ # 1 may specify the page number at which the reading of the original is completed by using the existing character recognition technique. The MFP_ # 1 can notify the AI assistant server device 4 of the specified page number. The AI assistant server device 4 stores the information notified to the management DB 401, the linking DB 402, etc. in association with the device ID, the device ID, the user ID, and the like. As a result, the AI assistant server device 4 can notify the user of the reading status of the document via the smart speaker 2.

For example, when it is determined that a document having the same page number has been read twice or more, the notification unit 458 of the AI assistant server device 4 performs duplicate reading processing by voice or screen display to the user via the smart speaker 2. You can warn that it has been done. Further, when it is determined that only the document having a predetermined page number is not read even though the document having a continuous page number is read, the notification unit 458 informs the user via the smart speaker 2 by voice or screen. By the display, it is possible to warn that the reading process of the predetermined page has not been performed.

Further, when the user inquires the smart speaker 2 about the reading status of the document by utterance, the AI assistant server device 4 tells the user via the smart speaker 2 the page in which the reading of the document is completed by voice or screen display. You can notify the number. In this case, for example, the interpretation unit 354 that functions by executing the operation voice conversion program generates an intent that inquires about the reading status of the document such as "SCAN_PAGECONFIRM". Further, the execution instruction unit 456, which functions by executing the management program, asks the MFP_# 1 for the read page number of the manuscript, or based on the page number notified from the MFP_# 1, of the manuscript. Check the read status. Then, the notification unit 458 can notify the smart speaker 2 of the page number of which the document has been read via the operation voice conversion program.

In the present embodiment, the voice operation system 1 is based on, for example, the history and usage history of reading the original of the past MFP_ # 1, and the information of each management table for constructing the linking DB 402 of FIGS. 5 and 6. With respect to the voice data accompanying the user's utterance or the like, a configuration may be provided in which machine learning is used to automatically execute related processing in addition to the reading processing in MFP_ # 1.

By providing the above-described configuration in the second embodiment, when the user wants to continuously read the document, the user changes the page of the document or the direction of the document with the utterance voice for reading the document. You will be able to continue reading simply by giving a simplified spoken voice without having to speak everything every time.

[Effect of Embodiment]
As is clear from the above description, the voice operation system 1 according to the present embodiment installs a program including an operation voice processing program, which is a platform application program, on the smart speaker 2, and a cloud service device based on this platform application program. Communicate with 5. When the user performs a voice operation toward the microphone unit 29 provided in the smart speaker 2, the cloud service device 5 analyzes the content of the user's utterance, executes a document reading instruction given by the user, and executes a predetermined process. An image reading device such as an MFP 6 is operated so that each process based on the instruction is performed.

With such a configuration, it becomes possible to continuously read a plurality of documents simply by giving a simplified voice instruction. That is, when a plurality of consecutive jobs are executed by the image forming apparatus by voice, it is possible to simplify the operation for executing the jobs each time.

As a result, it is possible to eliminate the need for an operation by a GUI (Graphical User Interface) such as the touch panel 27. Therefore, even a user who is accustomed to the operation can perform a quicker and simpler input operation. In addition, operation support through dialogue or the like eliminates the need for, for example, complicated network settings, advanced processing settings, or the introduction of new applications. As a result, even an elderly person or a user who is unfamiliar with machine operation can quickly and easily perform the operation desired by the user, and the convenience is improved. Further, when the original must be held by hand when reading the original, it is assumed that the operability to the operation unit or the like is deteriorated. However, according to the voice operation system 1 according to the present embodiment, if the document to be read is placed on the platen and the minimum necessary utterance is made, the document is continuously read, so that the operability is improved. You can expect it.

Further, according to the present embodiment, it is also possible for the cloud service device 5 (or AI assistant server device 4) to determine and process the analysis of the user's intention based on the text data obtained from the user's utterance content. It becomes.

The image reading device is not limited to an image forming device (MFP) as long as it has a communication function and can perform repetitive processing. That is, the image reading device is, for example, an output device such as a PJ (Projector: projector), an IWB (Interactive White Board: a white board having an electronic whiteboard function capable of intercommunication), a digital signage, or a HUD (Head Up Display). Devices, industrial machines, imaging devices, sound collectors, medical devices, network home appliances, automobiles (Connected Cars), notebook PCs (Personal Computers), mobile phones, smartphones, tablet terminals, game machines, PDA (Personal Digital Assistant), digital It may be a camera, a wearable PC, a desktop PC, or the like.

Each function of the above-described embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software like a processor implemented by an electronic circuit, or an ASIC designed to execute each function described above. It shall include devices such as Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays) and conventional circuit modules.

Further, the voice acquisition device is not limited to a smart speaker as long as it is a device having a microphone function, an image pickup function, a speaker function, a display function, an operation function, a communication function, and the like. The voice acquisition device may be, for example, a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet terminal, a game machine, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, a desktop PC, or an earphone type transmitter / receiver. good. The earphone-type transmitter / receiver receives (acquires) the user's own voice spoken while being worn on the user's ear, converts the received voice into voice data, and transmits the received voice to a predetermined server device. , A communication device having a function of receiving (acquiring) feedback results and the like from a predetermined server device.

Similarly, in addition to the above-mentioned MFP, the image reader can be used as long as it can communicate with the server device and the voice acquisition device via a network and can read a document such as a bound book document. It doesn't matter. For example, the image reading device may be an electronic device such as a single scanner.

Finally, the above embodiments are presented as an example and are not intended to limit the scope of the invention. Each of the novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. For example, in the above description of the first embodiment, the voice recognition server device 3 generates text data corresponding to the utterance or the like given by the user, and the AI assistant server device 4 is the user based on the generated text data. Interpreted the intended operation of. However, such a voice recognition function and an interpretation function may be provided on the voice acquisition device side, and the smart speaker 2 may interpret the intended operation from the user's utterance. As a result, the voice recognition server device 3 and the AI assistant server device 4 can be eliminated, and the system configuration can be simplified.

Such embodiments and modifications of the embodiments are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalent scope thereof.

1 Information processing system 2 Smart speaker (an example of voice acquisition device)
3 Voice recognition server device 4 AI assistant server device (an example of information processing device)
6 MFP (an example of an image reader)
252 Acquisition unit (an example of voice data acquisition means)
451 Communication control unit (an example of communication means)
453 Interpretation result conversion unit (example of reading command conversion means)
455 Complementary part (an example of complementary means)
651 Communication control unit (example of file transmission means)
652 Command receiving unit (Example of reading command receiving means)
654 Read execution unit (an example of read control means)

Japanese Unexamined Patent Publication No. 2014-203024

Claims (7)

A voice acquisition device that collects voice and obtains voice data,
An image reader that reads an image at least once on a document,
The first voice data is received at the first timing transmitted by the voice acquisition device, and the first voice data is converted into a reading command for reading the document based on a predetermined reading condition, and the predetermined An information processing device that transmits the reading command to the image reading device based on the reading conditions of
It is an information processing system equipped with
The information processing device
When the second voice data received at the second timing after the first timing is the content that enables the execution of the reading command based on the first voice data to be continued, the reading command An information processing system characterized in that the data is retransmitted to the image reader. The image reader is
The result obtained by executing the reading command transmitted from the information processing device is generated as one file, and the file is stored in the image reading device or transmitted to an external device. The information processing system according to claim 1. An information processing device that is connected to a voice acquisition device that collects voice and obtains voice data.
A receiving means for receiving the first voice data transmitted by the voice acquisition device at the first timing, and
A conversion means for converting the first voice data into a reading command for reading a document based on a predetermined reading condition, and
A transmission means for transmitting the reading command to an image reading device that executes the reading command, and
Have,
The conversion means
When the second voice data received at the second timing after the first timing has the content that enables the execution of the reading command based on the first voice data to be continued, the reading command Convert to
The information processing device is characterized in that the transmitting means retransmits the converted reading command to the image reading device. The information processing apparatus according to claim 3, further comprising a complementary means for supplementing information related to the reading instruction when converting to the reading instruction. The information processing device according to claim 4, wherein the complementing means prompts the voice acquisition device to acquire voice for complementing information related to the reading command. A voice acquisition device that collects voice and obtains voice data,
An image reader that reads the original at least once, and
The first voice data is received at the first timing transmitted by the voice acquisition device, and the first voice data is converted into a reading command for reading the document based on a predetermined reading condition, and the predetermined An information processing device that transmits the reading command to the image reading device based on the reading conditions of
It is an information processing method executed by an information processing system equipped with
The steps performed by the information processing device are
When the second voice data received at the second timing after the first timing has the content that enables the execution of the reading command based on the first voice data to be continued, the reading command An information processing method comprising the step of retransmitting the image to the image reading device. A program comprising causing a computer to execute the information processing method according to claim 6.

Images