JP6229433B2 - Operation guidance server, operation guidance system, image forming apparatus, and program - Google Patents

Description

  The present invention relates to an operation guidance system and related technology.

  In recent years, an image forming apparatus such as an MFP (Multi-Functional Peripheral) has become multifunctional and highly functional, and its operation has become complicated. For this reason, the user may inquire of the support center about the operation method of the image forming apparatus and request a supporter (a person who supports the user at the support center) to guide the operation.

  At that time, the user may receive operation guidance from the supporter based on the display image of the image forming apparatus operated by the user, and the confidential information of the user included in the display image is displayed as it is on the supporter terminal. As a result, the confidential information of the user may be leaked.

  In Patent Document 1, in view of such a problem, the confidential information of the user displayed on the display unit of the image forming apparatus is replaced with a dummy image and displayed on the personal computer of the supporter. It is described that it is prevented from being displayed as it is on the side.

JP 2009-253651 A

  By the way, in operation guidance between a user and a supporter, it is assumed that not only operation guidance using a display image but also operation guidance using voice is performed.

  However, when performing the operation guidance using the display image and the sound, if the above-described conventional technology is used, the confidential information included in the voice uttered by the user is directly transmitted to the terminal of the supporter, and the confidential information of the user is transmitted to the supporter. There is a risk of leakage.

  Therefore, an object of the present invention is to provide a technique capable of avoiding leakage of confidential information included in a user's voice in operation guidance between the user and a supporter.

In order to solve the above problem, the invention of claim 1 is a guidance server in an operation guidance system, which is data of a first display image displayed on an operation unit of an image forming apparatus which is a user's operation target. Image receiving means for receiving one display image data from the image forming apparatus, and when a secret word is included in the first display image data, the secret word in the first display image corresponds to the secret word And a supporter used for operation guidance to the user by a supporter who is a person supporting the user An image transmitting means for transmitting the first composite image data as display data on the supporter terminal to the terminal, and a voice uttered by the user A voice receiving means for receiving user voice data including data from the image forming apparatus, a voice recognition processing on the user speech data, a speech recognition unit for determining whether or not the include the secret word to the user speech data When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is converted into voice data of the alternative word corresponding to the secret word. Voice generating means for generating synthesized user voice data which is data replaced with alternative voice data, and voice transmitting means for sending the synthesized user voice data to the supporter terminal as voice output data at the supporter terminal; It is characterized by providing.

  According to a second aspect of the present invention, in the guidance server according to the first aspect of the invention, the user voice data is divided into a plurality of partial voice data, and the voice generation means is one partial voice data in the user voice data. When the voice recognition process determines that the secret word is included in certain first voice data, the first voice data in which the secret voice data in the first voice data is replaced with the alternative voice data. The synthesized voice data is generated, and the voice transmitting means transmits the first synthesized voice data to the supporter terminal.

  According to a third aspect of the present invention, in the guidance server according to the second aspect of the invention, the first voice data includes a voice uttered by the user during display of the first display image, and the image receiving means. The second display image data, which is the data of the second display image displayed on the operation unit subsequent to the first display image, is converted into the image forming apparatus after the reception of the first display image data is completed. And when the secret word is included in the second display image data, the image generation means replaces the secret word in the second display image data with an alternative word corresponding to the secret word. Second composite image data, which is data of a second composite image, is generated, and the image transmission means permits transmission of the second composite image data when reception of the first audio data is started. The above And permits the transmission of the second synthesized image data in the transmission completion time after the first synthesized speech data generated based on one of the audio data.

  According to a fourth aspect of the present invention, in the guidance server according to the third aspect of the present invention, the image transmission means is configured to receive the first voice data from the reception start time of the first voice data and completion of the transmission of the first synthesized voice data. When the second display image data is received by the image receiving means within a first period that is a period between the time when the output time of the synthesized voice data has elapsed, Transmission of the second composite image data is not permitted until the end point, and transmission of the second composite image data is permitted after the end of the first period.

  A fifth aspect of the present invention is the guidance server according to the first aspect of the present invention, further comprising storage means for storing voice data, wherein the voice generating means is voice data of alternative words corresponding to a plurality of secret words. Generation of a plurality of alternative voice data is started from a predetermined time prior to the generation of the synthesized user voice data, and the generated alternative voice data is stored in the storage means, and the secret word is included in the user voice data When the alternative voice data corresponding to the secret word is not stored in the storage unit, the alternative voice data is generated by a machine voice generation process, and the synthesized user is generated using the generated alternative voice data. Voice data is generated, and the user voice data includes the secret word and the alternative voice data corresponding to the secret word is If already stored in the serial storage means, and generating said composite user speech data using the substitute audio data stored in said storage means.

  According to a sixth aspect of the present invention, in the guidance server according to the fifth aspect of the invention, the voice generating means is responsive to the guidance server receiving a support request signal indicating the occurrence of a request for operation guidance from the user. Then, the generation of the plurality of alternative voice data is started.

  According to a seventh aspect of the present invention, in the guidance server according to the sixth aspect of the invention, the plurality of secret words include a phrase indicating a transmission destination in a destination designation screen in the scan image transmission of the image forming apparatus, and the image forming apparatus. It includes at least one of a phrase indicating a transmission destination included in a destination designation screen in facsimile transmission.

  The invention of claim 8 is the guidance server according to the invention of claim 6, wherein the plurality of secret words are words or phrases indicating file information displayed on an information display screen relating to a file stored in a box of the image forming apparatus. It is characterized by including.

  According to a ninth aspect of the present invention, in the guidance server according to the fifth aspect of the invention, the voice generation unit may include a secret that may be included in a display image in the current operation mode of the image forming apparatus among the plurality of alternative voice data. Alternative voice data corresponding to a word is preferentially generated.

  According to a tenth aspect of the present invention, in the guide server according to the ninth aspect, the current operation mode of the image forming apparatus is any one of a plurality of modes including a scan mode, a facsimile transmission mode, and a box mode. It is characterized by being.

  The invention of claim 11 is the guidance server according to the invention of claim 1, further comprising storage means for storing voice data, wherein the voice generation means receives the first display image data by the image receiving means. Then, generation of a plurality of alternative voice data, which are voice data of alternative words corresponding to the plurality of secret words, is started, the generated alternative voice data is stored in the storage means, and the secret data is stored in the user voice data When the alternative voice data corresponding to the secret word is not stored in the storage means, the alternative voice data is generated by machine voice generation processing, and the generated alternative voice data is used. Generating the synthesized user voice data, and the user voice data includes the secret word and the alternative voice data corresponding to the secret word. There if already stored in said storage means, and generating said composite user speech data using the substitute audio data stored in said storage means.

  According to a twelfth aspect of the present invention, in the guidance server according to any one of the fifth to eleventh aspects, the voice generation unit is configured to perform the plurality of alternative voices according to a priority order based on the frequency of use of the plurality of secret words. It is characterized by generating data.

  According to a thirteenth aspect of the present invention, in the guidance server according to the second aspect of the present invention, the guidance server further comprises storage means for storing the alternative voice data used for generating the first synthesized voice data. When the voice recognition process determines that the secret word is included in the second voice data which is partial voice data of the user voice data different from the first voice data, Using the substitute voice data stored in the storage means, the second synthesized voice data is generated by replacing the secret voice data in the second voice data with the substitute voice data, and the voice sending means The second synthesized voice data is transmitted to the supporter terminal.

  According to a fourteenth aspect of the present invention, in the guidance server according to the first aspect of the invention, the voice receiving means receives supporter voice data including voice data generated by the supporter, and the voice recognition means is the supporter. It is determined whether or not one substitute word corresponding to one or a plurality of secret words is included in the supporter voice data by voice recognition processing on the voice data, and the voice generation means Is included in the supporter voice data, the second alternative voice data that is the voice data of the one alternative word in the supporter voice data is the voice data of the secret word corresponding to the one alternative word. The synthesized supporter voice data replaced with the second secret voice data is generated, and the voice transmitting means converts the synthesized supporter voice data into And transmits the serial image forming apparatus.

  According to a fifteenth aspect of the present invention, in the guidance server according to the fourteenth aspect of the present invention, the guidance server further comprises storage means for storing voice data, and the voice generation means generates the synthesized user voice data based on the user voice data. In this case, the secret voice data extracted from the user voice data is stored in the storage means, and the secret voice data stored in the storage means is used as the second secret voice data. It is characterized by generating data.

  According to a sixteenth aspect of the present invention, in the guidance server according to any one of the second to fourth aspects of the invention, the voice recognizing unit is configured so that the user voice data includes a silent portion of a predetermined time or longer. Of the user voice data, partial voice data divided so as to end at the point in time when the silent state for the predetermined time has elapsed is extracted as the first voice data.

According to a seventeenth aspect of the present invention, in the guidance server according to the sixteenth aspect of the invention, the image receiving means also receives second display image data different from the first display image data from the image forming apparatus, When the second display image data is received by the image receiving means during the voice recognition process of the user voice data, the voice recognition means includes the second display image data of the user voice data. The partial audio data divided so as to end at the reception time of is extracted as the first audio data.

The invention according to claim 18 is the guidance server according to claim 1, wherein the first display image is an image of an information display screen related to a file stored in a box of the image forming apparatus, and the secret word is , the file name, the creator of the file, including the date, and the word indicating at least one of heading file body, the image generation means, the first composite image by replacing an equivalent the secret word in the alternative word When the user voice data includes the secret word, the voice generation unit generates synthesized user voice data in which the secret voice data is replaced with the alternative voice data.

According to a nineteenth aspect of the present invention, in the guidance server according to the eighteenth aspect of the present invention, the secret word includes a word indicating a heading of the file text, and the image generation means replaces the secret word with the substitute word. and generates the first composite image data obtained by converting the portion other than heading the files present statement read avoid image of the first said file present text a synthetic image data .

According to a twentieth aspect of the present invention, there is provided a first built-in image data displayed on an operation unit of an image forming apparatus, which is a user's operation target, on a computer built in a guidance server in the operation guidance system. A step of receiving display image data from the image forming apparatus; and b) an alternative corresponding to the secret word in the first display image when the secret word is included in the first display image data. A step of generating first composite image data which is data of a first composite image replaced with a word; and c) a supporter terminal used for guidance to the user by a supporter who is a person supporting the user. Transmitting the first composite image data as display data at the supporter terminal; and d) the voice uttered by the user Receiving a user voice data including over data from the image forming apparatus, e) by a speech recognition process on the user speech data, determining whether the contains the secret word to the user speech data, f) When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is converted to the voice of the alternative word corresponding to the secret word. Generating synthesized user voice data that is data replaced with alternative voice data that is data; and g) transmitting the synthesized user voice data to the supporter terminal as voice output data at the supporter terminal. It is a program for executing.

According to a twenty-first aspect of the present invention, there is provided an operation guidance system, a supporter used for operation guidance of the image forming apparatus to the user by an image forming apparatus which is a user's operation target and a supporter who is a person supporting the user. A guidance server that mediates between the terminal and the image forming apparatus and the supporter terminal, wherein the guidance server is a first display image data displayed on an operation unit of the image forming apparatus. An image receiving unit that receives display image data from the image forming apparatus, and an alternative that corresponds to the secret word in the first display image when the secret word is included in the first display image data. Image generating means for generating first composite image data which is data of a first composite image replaced with a word, and the supporter terminal with respect to the supporter terminal An image transmitting means for transmitting said first combined image data as the display data, and voice receiving means for receiving user voice data from the image forming apparatus including a sound data emitted by the user, the user voice Voice recognition means for determining whether or not the user voice data includes the secret word by voice recognition processing on the data, and when it is determined that the user voice data includes the secret word, the user voice Voice generating means for generating synthesized user voice data, which is data obtained by replacing secret voice data that is voice data of the secret word in data with substitute voice data that is voice data of the alternative word corresponding to the secret word; The synthesized user voice data is used as voice output data at the supporter terminal. And having a, a sound transmission means for transmitting to the end.

  According to a twenty-second aspect of the present invention, there is provided an image forming apparatus in the operation guidance system, wherein the first display image data is data of a first display image displayed on the operation unit of the image forming apparatus that is a user's operation target. And a first composition in which, when a secret word is included in the first display image data, the secret word in the first display image is replaced with an alternative word corresponding to the secret word. Display on the supporter terminal with respect to a supporter terminal that is used for guidance to the user by an image generation unit that generates first composite image data that is image data and a supporter that is a person supporting the user Image transmission means for transmitting the first composite image data as data for use, and user voice data including voice data uttered by the user A voice recognition unit that determines whether or not the secret word is included in the user voice data by voice recognition processing for the user voice data; and that the secret word is included in the user voice data. If so, the synthesized user voice is data obtained by replacing the secret voice data that is the voice data of the secret word in the user voice data with the substitute voice data that is the voice data of the alternative word corresponding to the secret word. Voice generation means for generating data; and voice transmission means for transmitting the synthesized user voice data to the supporter terminal as voice output data at the supporter terminal.

  According to a twenty-third aspect of the present invention, there is provided a computer built in the image forming apparatus in the operation guidance system, wherein a) data of the first display image displayed on the operation unit of the image forming apparatus which is a user's operation target. A step of obtaining one display image data, and b) when a secret word is included in the first display image data, the secret word in the first display image is replaced with an alternative word corresponding to the secret word. A step of generating first composite image data which is data of the first composite image; and c) a supporter terminal used for guidance to the user by a supporter who is a person supporting the user, Transmitting the first composite image data as display data on a supporter terminal; and d) a user including voice data uttered by the user. Obtaining voice data, e) determining whether or not the secret word is included in the user voice data by voice recognition processing on the user voice data, and f) the secret to the user voice data. When it is determined that a word is included, the confidential voice data that is the voice data of the secret word in the user voice data is replaced with the alternative voice data that is the voice data of the alternative word corresponding to the secret word Generating a synthesized user voice data as data; g) transmitting the synthesized user voice data as voice output data at the supporter terminal to the supporter terminal; and a program for executing the program. And

  According to the first to twenty-third aspects of the present invention, it is possible to avoid leakage of confidential information included in the user's voice in the operation guidance between the user and the supporter.

  In particular, according to the second aspect of the present invention, the secret audio data in the first audio data is replaced with the substitute audio data for the first audio data which is one partial audio data obtained by dividing the user audio data. The first synthesized voice data is generated, and the first synthesized voice data is transmitted to the supporter terminal. Therefore, the first synthesized voice data corresponding to the first voice data is transmitted to the supporter terminal relatively early without waiting for the end of the process for the next portion of the first voice data in the user voice data. obtain. As a result, it is possible to suppress a delay in transmitting audio data to the supporter terminal.

  In particular, according to the third aspect of the invention, when the reception of the first audio data is started, the transmission of the second synthesized image data is not permitted, and after the transmission completion time of the first synthesized audio data. Since the transmission of the second synthesized image data is permitted, the change from the first display image to the second display image in the supporter terminal is performed after the transmission of the first synthesized audio data. Therefore, the voice of the user uttered while viewing the first display image is output on the supporter terminal side during the display of the second display image next to the first display image due to the arrival delay of the voice. This can be suppressed or avoided.

  In particular, according to the fourth aspect of the present invention, the change from the first display image to the second display image in the supporter terminal is performed when the first synthesized speech data is transmitted after the first synthesized speech data is transmitted. This is performed after the time required has elapsed. Therefore, the voice of the user uttered while viewing the first display image is output on the supporter terminal side during the display of the second display image next to the first display image due to the arrival delay of the voice. This can be further suppressed or avoided.

  In particular, according to the fifth aspect of the invention, the generation of the plurality of alternative voice data is started from a predetermined time prior to the generation of the synthesized user voice data, and the generated alternative voice data is stored in the storage means. The If the user voice data includes a secret word and the alternative voice data corresponding to the secret word is already stored in the storage means, the synthesized user voice data using the alternative voice data stored in the storage means Is generated, and the synthesized user voice data is transmitted to the supporter terminal. Therefore, the time required for generating the synthesized voice data is shortened as compared to the case where the generation of the alternative voice data is always started from the time when it is determined that the user voice data includes the secret word. As a result, it is possible to suppress a delay in transmission of the synthesized voice data to the supporter terminal.

  In particular, according to the invention described in claim 13, in the audio processing for the second audio data, the second synthesized audio data is used by using the alternative audio data that has been used and stored for generating the first synthesized audio data. Therefore, it is not necessary to generate the alternative voice data again. Therefore, since the time required for generating the second synthesized voice data is shortened, it is possible to suppress a delay in the transmission of the second synthesized voice data to the supporter terminal.

  In particular, according to the invention described in claim 14, the synthesized supporter voice data obtained by replacing the substitute word included in the supporter voice data issued by the supporter with the second secret voice data corresponding to the substitute word is the user. Therefore, the alternative word included in the supporter voice data is not transmitted to the user. Therefore, it is possible to avoid the user's confusion caused by the alternative word that the user does not know is output as voice in the image forming apparatus.

  In particular, according to the invention described in claim 15, when the synthesized user voice data is generated, the voice data issued in the past by the user is stored as the secret voice data in the storage means, and the generation of the synthesized supporter voice data is performed. At this time, synthesized supporter voice data is generated using the secret voice data already stored in the storage means as the second secret voice data. Therefore, since it is not necessary to generate the second secret audio data again, the time required for generating the synthesis supporter audio data is shortened. As a result, it is possible to suppress a delay in transmission of the synthesized audio data to the image forming apparatus.

It is a figure which shows the structure of an operation guidance system. 2 is a functional block diagram illustrating a schematic configuration of an MFP. FIG. It is a functional block diagram which shows schematic structure of a guidance server. It is a functional block diagram which shows schematic structure of a supporter terminal. It is a figure which shows the outline | summary of operation | movement of an operation guidance system. It is a figure which shows the timing regarding the image data and audio | voice data which concern on 1st Embodiment. It is a figure which shows operation | movement of an operation guidance system. It is a flowchart which shows the image processing etc. of image data. It is a figure which shows a response | compatibility (conversion dictionary) with a secret word and an alternative word. It is a flowchart which shows the audio | voice process etc. of user audio | voice data. It is a figure which shows the audio | voice process regarding the production | generation of synthetic | combination audio | voice data. It is a figure which shows the timing regarding the image data and audio | voice data which concern on 1st Embodiment. It is a figure which shows the timing regarding the image data and audio | voice data which concern on 2nd Embodiment. It is a flowchart which shows the audio | voice process etc. of the audio | voice data which concern on 2nd Embodiment. It is a flowchart which shows the image processing etc. of the image data which concern on 2nd Embodiment. It is a figure which shows the timing regarding the image data and audio | voice data which concern on the modification of 2nd Embodiment. It is a flowchart which shows the audio | voice process etc. of the audio | voice data which concern on the modification of 2nd Embodiment. It is a figure which shows the timing regarding the image data and audio | voice data which concern on 3rd Embodiment. It is a flowchart which shows the audio | voice process etc. of the audio | voice data which concern on 3rd Embodiment. It is a figure which shows a response | compatibility (conversion dictionary) with a secret word and an alternative word. It is a flowchart which shows the audio | voice process etc. of the audio | voice data which concern on 4th Embodiment. It is a figure which shows the timing regarding the image data and audio | voice data which concern on 4th Embodiment. It is a figure which shows operation | movement of the operation guidance system which concerns on 4th Embodiment. It is a figure which shows operation | movement of the operation guidance system which concerns on 4th Embodiment. It is a figure which shows the image data and composite image data which concern on 5th Embodiment. It is a figure which shows the image data and composite image data which concern on 5th Embodiment. It is a figure which shows a response | compatibility (conversion dictionary) with the secret word and alternative word which concern on 5th Embodiment. It is a flowchart which shows the audio | voice processing etc. of the audio | voice data from the supporter which concerns on a modification to a user. It is a figure which shows operation | movement of the operation guidance system which concerns on a modification. It is a figure which shows operation | movement of the operation guidance system which concerns on a modification. It is a figure which shows operation | movement of the operation guidance system which concerns on a modification.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1-1. System overview>
FIG. 1 is a schematic diagram showing the configuration of the operation guidance system 1.

  As shown in FIG. 1, the operation guidance system 1 includes an image forming apparatus 10, a guidance server 50, and a supporter terminal 70. Here, the image forming apparatus 10 is exemplified by an MFP (Multi-Functional Peripheral).

  The elements 10, 50, and 70 in the operation guidance system 1 are connected to each other via a network 108 so as to communicate with each other. The network 108 is configured by a LAN, a WAN, the Internet, or the like. Further, the connection form to the network 108 may be wired connection or wireless connection.

  The operation guidance system 1 is a system that provides operation guidance for an MFP (image forming apparatus) 10. In response to a request from the user 101 of the MFP 10, the supporter 102 (person who supports the user 101) provides operation guidance to the user 101 using the supporter terminal 70.

  Communication between the MFP 10 and the supporter terminal 70 is performed via the guidance server 50 regarding the image data 300 and the user audio data 400 (audio data generated by the user). The guidance server 50 has a function of mediating between the MFP 10 and the supporter terminal 70 with respect to image and sound transmission.

  On the supporter terminal 70, a screen similar to the operation screen of the MFP 10 is displayed based on the image data 300 from the MFP 10 via the guidance server 50. Thereby, the supporter 102 can perform operation guidance for the user 101 while viewing a screen similar to the screen that the user 101 is viewing. The voice of the user 101 input via the microphone 18 of the MFP 10 is transmitted to the supporter terminal 70 via the guidance server 50. Thereby, the supporter 102 can perform operation guidance for the user 101 while listening to the voice of the user 101.

  However, the guidance server 50 performs conversion processing relating to images (specifically, operation guidance screens of the MFP 10). For example, as will be described later, when the confidential information 110 (confidential information) is included in the operation guidance screen (of the MFP 10) transmitted from the MFP 10, the guidance server 50 converts the confidential word 110 into an appropriate alternative word. The image converted into 210 is generated. Then, the guidance server 50 transmits the converted image to the supporter terminal 70 as a display image on the supporter terminal 70. On the supporter terminal 70, the converted image is displayed as an operation guidance screen. According to this, it is possible to avoid leakage of confidential information from the operation guidance image.

  Furthermore, the guidance server 50 also performs conversion processing related to voice. For example, as will be described later, when the confidential voice 110 (confidential information) is included in the user voice transmitted from the MFP 10, the guidance server 50 converts the confidential data 110 into an appropriate alternative word 210. And the converted voice data is transmitted to the supporter terminal 70. The supporter terminal 70 outputs sound based on the converted sound data. According to this, it is possible to avoid leakage of confidential information from the voice of the user 101.

  Hereinafter, such an operation guidance system will be described in detail.

<1-2. Configuration of MFP>
FIG. 2 is a functional block diagram illustrating a schematic configuration of the MFP 10. The MFP 10 is a device (also referred to as a multi-function device) having a scan function, a copy function, a facsimile function, a box storage function, and the like. The MFP has a plurality of operation modes (specifically, a copy mode, a scan mode, a facsimile transmission mode, and a box mode), and the operation of the corresponding function is executed in each mode.

  As shown in the functional block diagram of FIG. 2, the MFP 10 includes an image reading unit 2, a print output unit 3, a communication unit 4, a storage unit 5, an operation unit 6, a controller 9, a speaker 17, a microphone 18, and the like. Various functions are realized by operating these parts in a complex manner. The MFP 10 is also expressed as an image forming apparatus or a print output apparatus.

  The image reading unit 2 is a processing unit that optically reads (that is, scans) a document placed at a predetermined position of the MFP 10 and generates image data of the document (also referred to as a document image or a scanned image). is there. The image reading unit 2 is also referred to as a scanning unit.

  The print output unit 3 is an output unit that prints out an image on various media such as paper based on data related to a print target. The print output unit 3 is also referred to as an image forming unit that forms images on various media.

  The communication unit 4 is a processing unit capable of performing facsimile communication via a public line or the like. Further, the communication unit 4 can perform network communication via the network 108. In this network communication, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) and FTP (File Transfer Protocol) are used. By using the network communication, the MFP 10 can exchange various data with a desired destination. The communication unit 4 includes a transmission unit 4a that transmits various data and a reception unit 4b that receives various data.

  The storage unit 5 includes a storage device such as a hard disk drive (HDD). The storage unit 5 is provided with a plurality of boxes (folders) for each user, and electronic document data (document files) and the like are stored in each box.

  The operation unit 6 includes an operation input unit 6a that receives input to the MFP 10 and a display unit 6b that displays and outputs various types of information. Specifically, the MFP 10 is provided with an operation panel 6c (see FIG. 1). The operation panel (touch screen) 6c has a touch panel 25 on the front side. The touch panel 25 is configured by embedding a piezoelectric sensor or the like in a liquid crystal display panel, and can display various kinds of information and accept an operation input from an operator. The touch panel 25 functions as a part of the operation input unit 6a and also functions as a part of the display unit 6b.

The controller 9 is a control device that is built in the MFP 10 and controls the MFP 10 in an integrated manner. The controller 9 is configured as a computer system including a CPU and various semiconductor memories (RAM and ROM). The controller 9 implements various processing units by executing a predetermined software program (hereinafter also simply referred to as a program) stored in a ROM (for example, EEPROM (registered trademark) ) in the CPU. The program may be installed in the MFP 10 via a portable recording medium such as a USB memory or a network.

  As shown in FIG. 2, the controller 9 implements various processing units including a communication control unit 11, an input control unit 12, a display control unit 13, and a storage control unit 14.

  The communication control unit 11 is a processing unit that controls communication operations with other devices (such as the guidance server 50). For example, the communication control unit 11 receives various commands from the guidance server 50 in cooperation with the communication unit 4 and the like.

  The input control unit 12 is a control unit that controls an operation input operation with respect to the operation input unit 6a. For example, the input control unit 12 controls an operation for receiving an operation input on the operation screen.

  The display control unit 13 is a processing unit that controls the display operation in the display unit 6b. For example, the display control unit 13 displays an operation screen or the like for operating the MFP 10 on the display unit 6b.

  The storage control unit 14 is a processing unit that controls data storage processing related to a storage job.

  The speaker 17 is a device that emits sound based on audio data. The speaker 17 may be built into the MFP 10 or may be attached to the MFP 10 via a terminal.

  The microphone 18 is a device that converts a user's voice or the like into an electrical signal (analog signal). The microphone 18 may be built in the MFP 10 or may be attached to the MFP 10 via a terminal. The electrical signal (analog signal) is converted into digital data (converted into audio data) by the controller 9.

<1-3. Guide server configuration>
FIG. 3 is a functional block diagram illustrating a schematic configuration of the guidance server 50.

  The guidance server 50 is a device that mediates (or relays) the MFP 10 and the supporter terminal 70 with respect to the operation guidance of the MFP 10.

The guidance server 50 is configured as a computer system including a CPU and various semiconductor memories (such as RAM and ROM). In the CPU, the processing control unit 60 implements various processing units by executing predetermined software programs stored in a ROM (for example, EEPROM (registered trademark) ). The program may be installed in the guide server 50 via a portable recording medium such as a USB memory or a network.

  Specifically, the guidance server 50 realizes various processing units including the image processing unit 60a, the audio processing unit 60b, and the communication control unit 67 in accordance with the execution of the program.

  The image processing unit 60a is a processing unit that performs various types of image processing on received image data.

  As illustrated in FIG. 3, the image processing unit 60 a includes an image generation unit 61. The image generation unit 61 performs image composition processing (image generation processing) related to the operation screen.

  The audio processing unit 60b is a processing unit that performs various types of audio processing on received audio data.

  As shown in FIG. 3, the voice processing unit 60 b includes a voice recognition unit 64 and a voice generation unit 65. The voice recognition unit 64 performs voice recognition processing on the received voice data and the like. The voice generation unit 65 processes the received voice data and performs voice synthesis processing (mechanical voice generation processing).

  The communication control unit 67 is a processing unit that controls data transmission / reception operations with a communication partner (for example, the MFP 10) in cooperation with the communication unit 54.

  The storage unit 55 of the guidance server 50 includes a storage device such as a hard disk drive (HDD).

  The guidance server 50 further includes a communication unit 54.

  The communication unit 54 can perform network communication via the network 108. In this network communication, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) and FTP (File Transfer Protocol) are used. By using the network communication, the guidance server 50 can exchange various data with a desired destination. The communication unit 44 includes a transmission unit 54a that transmits various data and a reception unit 54b that receives various data. The transmission unit 54a includes an image transmission unit that transmits image data and an audio transmission unit that transmits audio data. The reception unit 54b receives audio data from an image reception unit that receives image data. And an audio receiving unit.

<1-4. Supporter terminal configuration>
FIG. 4 is a functional block diagram showing a schematic configuration of the supporter terminal 70.

  The supporter terminal 70 is configured as a so-called personal computer. Further, the supporter terminal 70 is an auxiliary device that is operated by the supporter and used for guidance to the user.

  The supporter terminal 70 includes an operation unit 76. The operation unit 76 includes an operation input unit 76a that receives an operation input to the supporter terminal 70, and a display unit 76b that performs display output of various data. Further, the supporter terminal 70 can remotely operate the MFP 10, and a display screen corresponding to the display unit 6 b of the MFP 10 is displayed on the display unit 76 b.

  The supporter terminal 70 includes a CPU, a semiconductor memory, and the like. The supporter terminal 70 implements various processing units by executing predetermined software programs in the CPU. Specifically, as shown in FIG. 4, the supporter terminal 70 realizes various processing units such as a communication control unit 71 and an input control unit 72.

  The communication control unit 71 is a processing unit that controls data transmission / reception operations with a communication destination (for example, the guide server 50) in cooperation with the communication unit 74.

  The input control unit 72 is a control unit that controls an operation input operation with respect to the operation input unit 76a.

  The speaker 77 is a device that emits sound based on voice data from the guidance server 50 or the like. The speaker 77 may be built in the supporter terminal 70 or may be attached to the supporter terminal 70 via a terminal.

  The microphone 78 is a device that converts a user's voice or the like into an electric signal (analog signal). The microphone 78 may be built in the MFP 10 or may be attached to the MFP 10 by external connection. The electrical signal (analog signal) is converted into digital data (converted into audio data) by the controller 9.

<1-5. Operation>
Next, the operation of the operation guidance system 1 according to the first embodiment will be described with reference to FIGS.

  FIG. 5 is a diagram illustrating the operation of the operation guidance system 1. The guidance server 50 receives the image data 300 (display image data 300) from the MFP 10. When the secret word 110 (described later) is included in the image data 301, the guidance server 50 generates the composite image data 350 by image processing (image conversion processing or the like), and the composite image data 350 is stored in the supporter terminal 70. Send to. Further, the guidance server 50 receives the user voice data 400 from the MFP 10. When the confidential voice 110 is included in the user voice data 400, the guidance server 50 generates synthesized voice data 450 (synthesized user voice data 450) by voice processing, and transmits the synthesized voice data 450 to the supporter terminal 70. .

  Thereby, it is possible to avoid the secret word 110 included in the image data 300 (301) and the user voice data 400 from leaking to the supporter 102.

  Image processing and sound processing will be described in more detail with reference to FIG. FIG. 6 is a diagram illustrating timing related to image data and audio data according to the first embodiment.

  The MFP 10 transmits the image data 300 (301) displayed on the touch panel 25 to the guidance server 50. In addition, the MFP 10 transmits user voice data 400 including voice data uttered by the user 101 to the guidance server 50.

  Upon receiving the image data 301 from the MFP 10, the guidance server 50 generates composite image data 350 (351) by image processing (described later) and transmits it to the supporter terminal 70. Further, the guidance server 50 extracts partial voice data 430 (described later) that is a part of the user voice data 400. Then, the guidance server 50 generates synthesized voice data 450 (451) by voice processing (described later), and transmits it to the supporter terminal 70.

  When the supporter terminal 70 receives the composite image data 351 from the guidance server 50, the supporter terminal 70 displays it on the display unit 76b. Further, when the supporter terminal 70 receives the synthesized voice data 451 from the guidance server 50, the supporter terminal 70 outputs (reproduces) the synthesized voice data 451.

  Hereinafter, such image processing and sound processing according to the first embodiment will be described more specifically.

  First, image processing will be described with reference to FIG. FIG. 7 is a diagram illustrating the operation of the guidance server 50 in the first embodiment. In FIG. 7, it is assumed that a user 101 operating the MFP 10 makes an inquiry to the support center regarding the operation method related to the scan function of the MFP 10.

  As illustrated in FIG. 7, the user 101 makes a support request inquiry to the support center while viewing the display image data 301 that is display image data displayed on the touch panel 25 of the MFP 10. Specifically, the user 101 presses a help button (not shown) provided on the operation panel 6c of the MFP 10 while looking at the destination designation screen for scanning image transmission. When the help button is pressed by the user 101, the MFP 10 (specifically, the transmission unit 4 a) transmits a support request signal indicating the generation of operation guidance from the user 101 to the guidance server 50.

  FIG. 8 is a flowchart showing an operation related to image processing after the support request signal is received by the guidance server 50.

  When the reception unit 54b of the guidance server 50 receives the support request signal from the MFP 10, the transmission unit 54a of the guidance server 50 transmits a support request signal to the supporter terminal 70 (step S11).

  Thereafter, when the supporter 102 presses a guidance start button (not shown) of the supporter terminal 70, a signal (start signal) indicating that the operation guidance should be started is transmitted from the supporter terminal 70 to the guidance server 50. The guidance server 50 The operation guidance start signal is transmitted to the MFP 10. As a result, the MFP 10 and the supporter terminal 70 transition to the operation guidance mode.

  Receiving the operation guidance start signal, the MFP 10 (specifically, the transmission unit 4a) transmits the image data 301 being displayed on the touch panel 25 to the guidance server 50, and the guidance server 50 (specifically, the reception unit 54b). ) Receives the image data 301 from the MFP 10 (step S12). Then, the guidance server 50 (specifically, the image generation unit 61) determines whether or not a secret word 110 (described later) is included in the image data 301 (step S13). More specifically, the image generation unit 61 performs character recognition by OCR processing on the image data 301 and determines whether or not the secret word 110 is included.

  When receiving the image data 301, the guide server 50 also receives from the MFP 10 a secret word list 601 (see FIG. 9) that is a list of the secret words 110 included in the image data 301. In the secret word list 601, a destination included in the destination designation screen (image data 301) is extracted and registered as the secret word 110. Then, the guidance server 50 generates a conversion dictionary 651 based on the secret word list 601. In the generated conversion dictionary 651, the secret word 110 and the alternative word 210 corresponding to the secret word 110 are registered.

  Regarding the conversion dictionary 651, specifically, “Hase Real Estate” (111 (110)), “Takahashi Electric” (112 (110)) and “Matsubara Corporation” (113 (110)) are registered as the secret word 110. ing. Furthermore, “ABC” is registered as an alternative word 211 (210) corresponding to the secret word 111 “Hase Real Estate”, and “DEF” is registered as an alternative word 212 (210) corresponding to the secret word 112 “Takahashi Electric”. “GHIJ” is registered as an alternative word 213 (210) corresponding to the secret word 113 “Matsubara Corporation”. Each alternative word 210 is automatically generated by the guidance server 50.

  Here, the secret word 110 is a word (word) indicating information to be kept secret by the user. In the present embodiment, the word to be concealed (more specifically, the transmission destination of the destination designation screen in scan image transmission) included in the image data 301 displayed on the touch panel 25 is the concealment word 110 by the guide server 50. It is determined.

  When it is determined in step S <b> 13 that the secret word 110 is included in the image data 301, the guide server 50 converts the secret word 110 into the substitute word 210 corresponding to the secret word 110 based on the conversion dictionary 651. The composite image data 351 that is the composite image data replaced with is generated (step S14). When the composite image data 351 is generated, the guidance server 50 transmits the composite image data 351 to the supporter terminal 70 as display data on the supporter terminal 70 (step S15). The display unit 76b of the supporter terminal 70 that has received the composite image data 351 displays composite image data 351 instead of the image data 301 (see FIG. 7).

  On the other hand, if it is determined that the secret word 110 is not included in the image data 301, the guidance server 50 does not perform the process of step S14, and the image data 301 is used as it is as the composite image data 351 in step S15. Is done. That is, the image data 301 is transmitted to the supporter terminal 70, and the image data 301 is displayed as it is on the display unit 76b.

  Next, audio processing will be described.

  Upon receiving the operation guidance start signal, the MFP 10 starts transmitting the user voice data 400.

  In FIG. 7, the user 101 utters a voice saying “I want to scan a file and send it to Hase Real Estate” to the MFP 10 while watching the image data 301 being displayed on the touch panel 25 of the MFP 10. is assumed.

  When voice uttered by the user 101 is input to the MFP 10 via the microphone 18, the MFP 10 transmits user voice data 400 that is data of the voice to the guidance server 50. Here, the user voice data 400 is transmitted to the guidance server 50 in real time (as needed).

  The operation after the user voice data 400 is received by the guidance server 50 will be described with reference to the flowchart of FIG. When the guidance server 50 (specifically, the receiving unit 54b) receives the user voice data 400 (step S20), the voice recognition unit 64 determines whether or not the user voice data 400 includes a non-silence portion ( Step S21). If it is determined that a non-silence portion exists, the voice recognition unit 64 next determines whether or not a silence portion of a predetermined time or longer exists in the user voice data 400 (step S22).

  When it is determined that there is a silent part for a predetermined time or longer in the user voice data 400, the voice recognition unit 64 extracts partial voice data 430 that is a part of the voice data of the user voice data 400 ( Step S23). In other words, from the user voice data 400, partial voice data that is segmented so as to end at a point in time when a silent state for a predetermined time has elapsed is extracted as the partial voice data 430.

  Here, the partial audio data 430 is a section (period) sandwiched between the start time of the non-silence part and the time when a silence state for a predetermined time has elapsed from the end time of the non-silence part in the user sound data 400. Voice data. The user voice data 400 includes a silent part and a non-silent part, and the user voice data 400 is divided into a plurality of partial voice data 430 by the silent part.

  In addition, it is preferable that the predetermined time for the presence / absence determination of the silent portion is set to a relatively short period (for example, 0.5 seconds). When the predetermined time of the silent part is set to a relatively short period, the partial voice data 430 is also extracted by the voice recognition unit 64 as data of a relatively short period. As a result, it is possible to suppress a transmission delay of the synthesized voice data 450 corresponding to the partial voice data 430 to the supporter terminal 70 (described later).

  When the partial voice data 430 is extracted in step S23, the guidance server 50 (specifically, the voice generation unit 65) includes the secret word 110 in the partial voice data 430 by voice recognition processing on the partial voice data 430. It is determined whether or not (step S24).

  When the voice recognition unit 64 determines that the secret word 110 is included in the partial voice data 430, the voice generation unit 65 corresponds to the secret word 110 based on the conversion dictionary 651 (see FIG. 9). Alternative voice data 250, which is voice data of alternative word 210, is generated (step S25).

  More specifically, in step S <b> 24, the voice generation unit 65 determines based on the conversion dictionary 651 that the secret word 111 “Hase Real Estate” is included in the partial voice data 430 issued by the user 101. In response to this determination, in step S25, the voice generation unit 65 generates the alternative voice data 250 (251) of the alternative word 211 “ABC” corresponding to the secret word 111 “Hase Real Estate” by the machine voice generation process. . The alternative voice data 250 is voice data (mechanical voice data) artificially generated by imitating a human voice.

  When the substitute voice data 251 is generated in step S25, the voice generation unit 65 replaces the secret voice data 150 (151), which is the voice data of the secret word 110 in the partial voice data 430, with the substitute voice data 251. Synthetic voice data 450 (451), which is data, is generated (step S26). Then, the guidance server 50 transmits the synthesized voice data 450 to the supporter terminal 70 as voice output data at the supporter terminal 70 (step S27). The secret voice data 150 is voice data (recorded voice data) of the voice of the user 101 (voice corresponding to the secret word 110) recorded in the MFP 10.

  On the other hand, when it is determined that the secret word 110 is not included in the partial voice data 430, the voice generation unit 65 does not perform the processes of steps S25 and S26, and uses the partial voice data 430 as the synthesized voice data 450 as it is. Use. That is, the partial audio data 430 is transmitted to the supporter terminal 70, and the partial audio data 430 is output as it is.

  The supporter terminal 70 that has received the synthesized voice data 450 (451) outputs the synthesized voice data 450 (451) instead of the partial voice data 430 via the speaker 77. Specifically, among the voices uttered by the user 101 in the supporter terminal 70, “I want to scan a file” and “I want to send it to” are output by the voice uttered by the user 101 and the secret word 111. “Hase Real Estate” is changed to “ABC” by the alternative voice data 251 and output.

  FIG. 11 is a diagram illustrating audio processing related to generation of the synthesized audio data 451. In FIG. 11, the partial voice data 431 (430) of “I want to scan a file and send it to Hase Real Estate” which is a voice uttered by the user 101 includes the secret word 111 “Hase Real Estate”. In this case, the voice generation unit 65 converts the secret voice data 150 (151) of the secret word 111 “Hase Real Estate” included in the partial voice data 431 into the alternative voice data of the alternative word 211 “ABC” corresponding to the secret word 111. In place of 250 (251), synthesized voice data 451 (450) is generated. In other words, the voice generation unit 65 synthesizes the synthesized voice data 450 (451) by synthesizing the voice data of the partial voice data 430 (431) excluding the secret word 111 and the alternative voice data 250 (251). Generate.

  According to the above operation, when the secret word 110 is included in the display screen of the MFP 10 operated by the user 101, the composite image data 350 (351) in which the secret word 110 is replaced with the alternative word 210 is the supporter terminal. 70, the secret word 110 is not displayed on the display unit 76b of the supporter terminal 70. Therefore, leakage of confidential information included in the display screen of the MFP 10 operated by the user 101 can be avoided.

  In particular, when it is determined that the secret word 110 is included in the partial voice data 430 of the voice uttered by the user 101, the secret voice data 150 (151) in the partial voice data 430 (431) is used as the substitute voice. The synthesized voice data 450 (451) replaced with the data 251 is transmitted to the supporter terminal 70. Therefore, the voice of the secret word 110 uttered by the user 101 is not transmitted to the supporter terminal 70 as it is. As a result, it is possible to avoid leakage of confidential information included in the voice of the user 101.

  Further, if the partial audio data 430 is not extracted from the user audio data 400 of the audio uttered by the user 101, the length of the user audio data 400 (the time of the audio of the user 101) increases, and the supporter terminal 70 The transmission of the synthesized voice data 450 is greatly delayed. On the other hand, in the first embodiment, the user voice data 400 of the voice uttered by the user 101 is divided in a relatively short period, and a plurality of partial voice data 430 is sequentially extracted from the user voice data 400, and the plurality of parts The audio data 430 is sequentially transmitted to the supporter terminal 70. That is, the synthesized voice data 450 corresponding to the partial voice data 430 can be transmitted to the supporter terminal 70 relatively early without waiting for the end of the process for the next part of the partial voice data 430 in the user voice data 400. Therefore, it is possible to suppress a delay in transmission of audio data to the supporter terminal 70.

Second Embodiment
The second embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  In the first embodiment, immediately after the generation of the synthesized image data 351 by the image generating unit 61 is completed, the synthesized image data 350 is transferred from the guidance server 50 to the supporter terminal 70 without considering the transmission timing of the synthesized audio data 451. Sent to. Therefore, a problem as shown in FIG. 12 may occur.

  In FIG. 12, during the utterance by the user 101 (specifically, during generation of the partial audio data 430 (during recording)), the display image on the touch panel 25 of the MFP 10 is an image based on the image data 301 according to the operation of the user 101. To an image based on image data 302 (described later). For example, a situation may be assumed in which the user 101 performs an operation of switching the operation screen in advance while speaking while the user 101 is speaking while viewing an image based on the image data 301.

  In FIG. 12, immediately after the generation of the synthesized image data 352 related to the image data 302 is completed, the synthesized image data 352 is transferred from the guidance server 50 to the supporter terminal without considering the relationship with the transmission timing of the synthesized audio data 451. 70. In accordance with this transmission operation, the display image on the supporter terminal 70 is changed from an image based on the composite image data 351 to an image based on the composite image data 352. Further, after the display image is changed (in other words, during display of an image based on the new synthesized image data 352), the synthesized voice data 451 corresponding to the partial voice data 430 is output from the supporter terminal 70.

  As a result, the synthesized voice data 451 corresponding to the voice uttered while viewing the image based on the original image data 301 should be output during the display of the synthesized image data 351, although it should be output. It is output during display of data 352 (composite image data corresponding to image data 302 next to image data 301). There is a possibility that the supporter 102 may be confused due to such a “deviation” between the image and the sound.

  Note that the image data 302 is image data of a display image displayed on the touch panel 25 of the MFP 10 following the display image based on the display image data 301.

  In the second embodiment, in view of such a problem, a required output time of the synthesized voice data 451 elapses from the completion of the transmission of the synthesized voice data 451 until a predetermined time after the transmission of the synthesized voice data 451 is completed. The mode of stopping transmission of the composite image data 352 is illustrated.

  FIG. 13 is a timing chart showing transmission timings and the like of the image data 300 and the partial audio data 430 according to the second embodiment.

  In the second embodiment, during the utterance by the user 101 (specifically, during the generation of the partial audio data 430), the user 101 performs a switching operation of the operation screen in advance, and the MFP 10 performs the switching operation according to the operation of the user 101. Assume that the display image on the touch panel 25 is switched from an image based on the image data 301 to an image based on the image data 302. Also, the partial audio data 430 is audio data that includes the audio uttered by the user 101 during the display of the image data 301 in the beginning portion thereof. Hereinafter, audio processing and image processing will be sequentially described.

  First, voice processing will be described with reference to FIG. 14 and comparison with FIG. FIG. 14 is a flowchart showing voice processing and the like of the user voice data 400 according to the second embodiment.

  In FIG. 14, step S41 is provided between step S21 and step S22, and step S42 and step S43 are provided after step S27. Specifically, the guidance server 50 changes the stop flag FG to ON (ON) when the voice recognition unit 64 determines that a non-silence portion exists in the user voice data 400 (step S21) ( Step S41).

  Here, the stop flag FG is flag information stored in the storage unit 55, and is controlled by the voice recognition unit 64 or the voice generation unit 65. The stop flag FG is set (changed) to ON (ON) or OFF (OFF). The transmission unit 54a of the guidance server 50 determines whether or not to transmit an image depending on whether the stop flag FG is ON or OFF. If the stop flag FG is ON, the transmission unit 54a does not transmit an image. If the stop flag FG is OFF, the transmission unit 54a transmits an image.

  After the stop flag FG is changed to ON, the guidance server 50 executes steps S22 to S27 as in FIG. Thereby, the synthesized voice data 450 corresponding to the partial voice data 430 is transmitted to the supporter terminal 70.

  When the guidance server 50 transmits the synthesized voice data 450 to the supporter terminal 70, the supporter terminal 70 outputs the synthesized voice data 450. When the required output time of the synthesized voice data 450 elapses after the transmission of the synthesized voice data 450 by the guidance server 50 (step S42), the guidance server 50 changes the stop flag FG to OFF (step S43).

  The required output time (required reproduction time) is the time required for outputting (reproducing) the synthesized voice data 450. The output required time is also expressed as the recording time of the partial sound data 430 (synthesized sound data 450). The output required time may be acquired by the voice recognition unit 64. However, the present invention is not limited to this, and the voice generation unit 65 may calculate the required output time during the generation of the synthesized voice data 450. Alternatively, the supporter terminal 70 transmits a signal to the effect that the output of the synthesized voice data 450 is completed to the guidance server 50 so that the guidance server 50 is notified from the supporter terminal 70 that the required output time has elapsed. Good.

  In this way, the synthesized voice data 450 is transmitted from the guidance server 50 to the supporter terminal 70, and the synthesized voice data 450 is output from the supporter terminal 70. From the detection time of the non-silence portion in the user audio data 400 (start time of the partial audio data 430 (431)) and the completion of transmission of the synthetic audio data 450 (451) corresponding to the partial audio data 430 (431) In the period T1 (see FIG. 13) between the time when the required output time (reproduction required time) of the data 450 (451) has elapsed, the stop flag FG is set to ON. On the other hand, the stop flag FG is set to OFF (OFF) in other periods (for example, a reception period of a silent part (a part determined not to be partial voice data 430) in the user voice data 400).

  Next, image processing will be described with reference to FIG. 15 and a comparison with FIG. FIG. 15 is a flowchart showing image processing and the like according to the second embodiment. Hereinafter, image processing relating to image data 302 next to image data 301 will be described with reference to FIG. It is assumed that the image processing relating to the image data 301 has already been completed by the same operation (see FIG. 8) as in the first embodiment.

  As shown in FIG. 15, each process of step S32-S35 is the same as each process of step S12-S15 in FIG. Since the support request signal transmission / reception operation has already ended before the transmission of the image data 301, the process of step S11 in FIG. 8 is not described in FIG.

  In FIG. 15, step S36 is provided between step S34 and step S35. In step S36, before transmitting the composite image data 352 generated by the image generation unit 61 to the supporter terminal 70, the guidance server 50 (specifically, the transmission unit 54a) is set by the audio processing of FIG. The value of the stop flag FG (whether it is ON or OFF) is recognized.

  If it is recognized that the stop flag FG is OFF, the guidance server 50 permits transmission of the composite image data 352 and transmits the composite image data 352 to the supporter terminal 70 (step S35). On the other hand, when it is recognized that the stop flag FG is ON (specifically, when the guidance server 50 performs the processes of steps S22 to S27 and S41 to S43), the guidance server 50 Transmission of the composite image data 352 is prohibited, and the composite image data 352 is not transmitted to the supporter terminal 70.

  As described above, the stop flag FG is set to ON (ON) in the period T1 (see FIG. 13). Therefore, as shown in FIG. 13, the guidance server 50 that has received the new image data 302 within the above-described period T1 generates composite image data 352 by image processing, but within the period T1, combines it with the supporter terminal 70. The image data 352 is not transmitted.

  Thereafter, when the period T1 ends and the stop flag FG is changed from on (ON) to off (OFF), the guidance server 50 transmits the composite image data 352 to the supporter terminal 70.

  Thus, when the guidance server 50 receives new image data 302 during the period T1 (see FIG. 13), the guidance server 50 (specifically, the transmission unit 54a) ends the period T1. Until the time point, transmission of new composite image data 352 is not permitted, and transmission of composite image data 352 is permitted after the end of the period T1.

  According to the operation as described above, the guidance server 50 does not permit transmission of new synthesized image data 352 when reception of the partial audio data 430 is started, and after the completion of transmission of the synthesized audio data 450 (451). Since the transmission of the synthesized image data 352 is permitted at a predetermined time, the image change from the synthesized image data 351 to the synthesized image data 352 in the display unit 76b of the supporter terminal 70 is performed after the transmission of the synthesized audio data 450 is completed. . Accordingly, the sound of the user 101 uttered while viewing the image data 301 is displayed during the display of the composite image data 352 (an image corresponding to the next image data 302 of the image data 301) due to the delay in the transmission of the sound. It is possible to suppress or avoid output on the supporter terminal side.

  In short, in the supporter terminal 70, it is possible to suppress or avoid a shift between the image display timing based on the synthesized image data 351 and the audio output timing based on the synthesized audio data 451. As a result, it is possible to provide accurate operation guidance to the user 101 without the supporter 102 being confused.

  In particular, the stop flag FG is changed from on to off and the synthesized image data 352 is transmitted after the time required for outputting the synthesized speech data 451 (required playback time) has elapsed since the completion of the transmission of the synthesized speech data 451. Preferably it is allowed. According to this, it is possible to further suppress or avoid the shift between the timing of image display based on the synthesized image data 351 and the timing of outputting audio based on the synthesized audio data 451.

  In the second embodiment or the like, when the user voice data 400 includes a silent part for a predetermined time or longer, the voice recognition unit 64 starts the next silent part from the start point of the silent part of the user voice data 400. The example of extracting the voice data up to the start time of as the partial voice data 430 is exemplified. However, the present invention is not limited to this.

  For example, in the second embodiment (or the first embodiment), the audio data of a part divided so as to end at the time when the synthesized image data 350 is received by the guidance server 50 is changed from the user audio data 400 to the partial audio data 430. May be extracted. In other words, the user voice data 400 may be divided at the time when the operation screen is switched by the user 101, and the partial voice data 430 that is a part of the user voice data 400 may be extracted.

  FIG. 17 is a flowchart showing the operation of such a modification.

  In FIG. 17, in addition to the determination process of step S22, the determination process of step S44 is also performed. If “YES” is determined in either of the determination processes (steps S22 and S44), the process proceeds to step S23, and the partial audio data 430 is extracted. In step S44, it is determined whether new image data has been received.

  For example, even if it is determined in step S22 that there is no silent part longer than a predetermined time, if it is determined in step S44 that new image data 302 has been received, the process proceeds to step S23. In step S <b> 23, the voice recognition unit 64 extracts, as the partial voice data 430, the partial voice data that is divided from the user voice data 400 so that the reception time of the image data 302 of the new display image ends.

  FIG. 16 is a timing chart showing operation timing and the like according to this aspect.

  As shown in FIG. 16, the user voice data 400 of the user 101 that is uttered while viewing an image based on the image data 301 in the MFP 10 is divided when the image data 302 is received by the guidance server 50. The voice recognition unit 64 of the guidance server 50 includes the voice data of the user voice data 400 transmitted from the MFP 10 from the start time of the non-silent part of the user voice data 400 to the time when new image data 302 is received. Extracted as partial audio data 431 (430).

  The guidance server 50 generates synthesized voice data 451 related to the partial voice data 431 and transmits the synthesized voice data 451 to the supporter terminal 70. Then, the synthesized voice data 451 is output from the supporter terminal 70.

  On the other hand, the new image data 302 is changed to composite image data 352 by image processing by the guidance server 50. Then, the guide server 50 transmits the synthesized image data 352 to the supporter terminal 70 after the required output time of the synthesized voice data 451 has elapsed after the transmission of the synthesized voice data 451 is completed. Thereafter, an image based on the composite image data 352 is displayed on the display unit 76 b of the supporter terminal 70.

  According to this, since the voice of the user 101 is divided at the time of change from the image data 301 to the image data 302, the partial voice data 431 having a relatively short period can be extracted. Therefore, it is possible to further suppress the transmission delay of the synthesized voice data 451 to the supporter terminal 70.

  Further, the partial sound data 431 includes only the sound uttered while viewing the image data 301 (the sound uttered while viewing the next image data 302 is not included). Therefore, in the supporter terminal 70, it is possible to further suppress or avoid the deviation between the displayed image (image based on the synthesized image data 351) and the output sound (sound based on the synthesized speech data 451).

<Third Embodiment>
The third embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  In the first embodiment, when the guidance server 50 receives the user voice data 400 of the voice uttered by the user 101, the voice generation unit 65 replaces the substitute voice data 250 (251 corresponding to the secret word 110 included in the voice). ) And the synthesized voice data 450 (451) is generated using the alternative voice data 250. In the first embodiment, the sound generation unit 65 sequentially generates the substitute sound data 250, and the generated substitute sound data 251 is not stored.

  In the third embodiment, a plurality of alternative voice data corresponding to a plurality of secret words 110 at a predetermined time (specifically, when the guide server 50 receives a support request signal) prior to the generation of the synthesized voice data 450. The generation of 250 is started by the voice generation unit 65, and the generated alternative voice data 250 is stored in advance in the storage unit 55 of the guidance server 50. If the user voice data 400 includes the secret word 110 and the alternative voice data 250 corresponding to the secret word 110 is already stored in the storage unit 55, the alternative voice data stored in the storage unit 55 in advance is stored. The synthesized voice data 450 is generated by the voice generation unit 65 using 250.

  FIG. 18 is a diagram illustrating timing related to the operation according to the third embodiment, and FIG. 19 is a flowchart illustrating audio processing on the user audio data 400 of the user 101 according to the third embodiment. This will be specifically described with reference to FIGS. 18 and 19.

  When the user 101 presses a help button (not shown) provided on the operation panel 6c of the MFP 10 for a support request to the support center, the MFP 10 displays a support request signal indicating the occurrence of operation guidance from the user 101. Is transmitted to the guidance server 50.

  When the guidance server 50 receives a support request signal from the MFP 10 (step S11 (FIG. 19)), the guidance server 50 also receives a secret word list 602 (see FIG. 20), which is a list of a plurality of secret words 110 (described later), from the MFP 10.

  In the secret word list 602, the plurality of secret words 110 are registered. Then, the guidance server 50 generates a conversion dictionary 652 (see FIG. 20) based on the secret word list 602. In the generated conversion dictionary 652, the secret word 110 and the alternative words 210 respectively corresponding to the secret word 110 are registered.

  Here, the plurality of concealment words 110 are words (words) indicating a transmission destination in the destination designation screen in the scan image transmission of the MFP 10, words and phrases indicating a transmission destination included in the destination designation screen in the facsimile transmission of the MFP 10, and the MFP 10. And a word / phrase indicating file information displayed on the information display screen relating to the file stored in the box. In other words, the plurality of secret words 110 include a plurality of types of words / phrases to be concealed in each of the plurality of operation modes. However, the secret word 110 does not need to include all of these phrases, and may include a part of these phrases.

  In response to the reception of the support request signal by the guidance server 50, the voice generation unit 65 starts generating the alternative voice data 250 corresponding to the plurality of secret words 110 based on the conversion dictionary 652 (step S51) (FIG. 18). Further, the guidance server 50 sequentially stores the generated substitute voice data 250 in the storage unit 55 of the guidance server 50 (step S52).

  At a predetermined time point accompanying transmission / reception of the operation guidance start signal, the guidance server 50 receives the image data 300 (301) from the MFP 10, performs image processing based on the conversion dictionary 652, and generates composite image data 350 (351). Generate. Then, the guidance server 50 transmits the generated composite image data 350 (351) to the supporter terminal 70 (see FIG. 18).

  During or after the generation of the plurality of alternative voice data 250, the guidance server 50 receives the user voice data 400 (step S20), and executes each process of steps S21 to S24 (see FIG. 19). Thereafter, when it is determined in step S24 that the secret voice 110 is included in the partial voice data 430, the process proceeds to step S53.

  In step S <b> 53, the voice generation unit 65 determines whether the alternative voice data 250 corresponding to the secret word 110 is stored in the storage unit 55 of the guidance server 50.

  When it is determined that the alternative voice data 250 corresponding to the secret word 110 is stored in the storage unit 55, the voice generation unit 65 stores the alternative voice data 250 (251) already stored in the storage unit 55. (Step S54). Then, the voice generation unit 65 generates synthesized voice data 450 (451) using the alternative voice data 251 (step S26), and the guidance server 50 transmits the synthesized voice data 450 to the supporter terminal 70 (step S27).

  For example, when it is determined that the alternative voice data 251 “ABC” (FIG. 20) corresponding to the secret word 111 “Hase Real Estate” is stored in the storage unit 55, the voice generation unit 65 is stored. The alternative voice data 251 “ABC” is acquired from the storage unit 55. Then, the voice generation unit 65 generates synthesized voice data 451 using the alternative voice data 251 “ABC”, and the guidance server 50 transmits the synthesized voice data 451 to the supporter terminal 70. The supporter terminal 70 that has received the synthesized voice data 451 outputs a voice based on the synthesized voice data 451.

  On the other hand, when it is determined in step S54 that the alternative voice data 250 is not stored in the storage unit 55, the voice generation unit 65 generates the alternative voice data 250 corresponding to the secret word 110 by the machine voice generation process. (Step S25). Then, the voice generation unit 65 stores the generated alternative voice data 250 in the storage unit 55 (step S55), and proceeds to step S26. In step S26, synthesized voice data 450 is generated using the alternative voice data 250 generated in step S25.

  According to the operation as described above, when the guidance server 50 receives a support request signal from the user 101, generation of a plurality of alternative voice data 250 is started, and the generated alternative voice data 250 is stored in the storage unit 55 in advance. Is done. When it is determined that the secret voice 110 is included in the user voice data 400 and the alternative voice data 250 is already stored in the storage unit 55, the stored alternative voice data 250 (251) is used. Thus, synthesized voice data 450 (451) is generated. In this case, since the alternative voice data 250 that already exists is used, it is not necessary to newly generate the alternative voice data 250 (251). Therefore, for example, the preparation time of the alternative voice data 251 is shortened compared to when the generation of the alternative voice data 250 (251) is started from the time when it is determined that the user voice data 400 includes the secret word 110, and the synthesis is performed. The time required for generating the audio data 451 is reduced. As a result, it is possible to suppress a delay in transmission of the synthesized voice data 451 to the supporter terminal 70.

  In the third embodiment, a plurality of alternative voice data 250 corresponding to a plurality of secret words 110 that can be displayed in a plurality of operation modes (scan mode, facsimile transmission mode, box mode, etc.) in the MFP 10 are sequentially generated. ing. In the above description, the generation order of the plurality of alternative sound data 250 is not particularly mentioned, but the plurality of alternative sound data 250 may be generated in the priority order as described below.

  For example, among the plurality of secret words 110 that can be displayed in a plurality of operation modes in the MFP 10, the alternative voice data 250 corresponding to the secret word 110 that can be displayed in the current operation mode of the MFP 10 operated by the user 101 has priority. May be generated automatically.

  More specifically, when the current operation mode of the MFP 10 is the scan mode, the sound generation unit 65 includes one or more images that can be displayed in the scan mode (current mode) among the plurality of secret words 110. The secret word 110 included in the (address designation screen 301 or the like) is determined as the priority processing target word. Then, the voice generation unit 65 generates alternative voice data 250 corresponding to the priority processing target word, and stores the generated alternative voice data 250 in the storage unit 55.

  According to this, since the alternative voice data 250 corresponding to the secret word 110 that can be displayed in the current operation mode is preferentially generated, the alternative voice corresponding to the secret word 110 that is likely to be emitted by the user 101. Data 250 can be generated in advance. Therefore, when the synthesized voice data 450 is generated, there is a high possibility that the alternative voice data 250 stored in the storage unit 55 is used.

  Alternatively, a plurality of alternative voice data 250 may be sequentially generated according to the priority order based on the usage frequency of the secret word 110.

  Specifically, when receiving the secret word list 602, the guide server 50 also receives the usage frequency of each secret word 110 from the MFP 10 and generates the conversion dictionary 652 based on the secret word list 602 and the usage frequency. (See FIG. 20). Then, the voice generation unit 65 generates corresponding alternative voice data 250 in order from the secret word 110 having the highest use frequency among the plurality of secret words 110 registered in the conversion dictionary 652, and the generated alternative voice data 250. Is stored in the storage unit 55.

  In FIG. 20, the usage frequency of “Hase Real Estate” in the secret word 111 is 10, the usage frequency of “Takahashi Electric” in the secret word 112 is 20, and the usage frequency of “Matsubara Corporation” in the secret word 113 is 5. It is. In this case, paying attention to the three secret words 112, 111, and 113, the voice generation unit 65 generates alternative voice data 250 corresponding to the secret words 112, 111, and 113 in that order. That is, the alternative voice data 252 251 253 are generated in this order.

  Note that the usage frequency of the secret word 110 described in the conversion dictionary 652 may be the usage frequency of the secret word 110 by a plurality of users who use the MFP 10 (in other words, the usage frequency of the MFP 10). The usage frequency of the secret word 110 by the user 101 (logged-in user) who operates the MFP 10 may be used.

  As described above, a plurality of alternative voice data 250 may be sequentially generated according to the priority (only) based on the usage frequency of the secret word 110. According to this, it is possible to generate in advance the alternative voice data 250 corresponding to the secret word 110 that is likely to be issued by the user 101. Therefore, when the synthesized voice data 450 is generated, there is a high possibility that the alternative voice data 250 stored in the storage unit 55 is used.

  Furthermore, a plurality of alternative audio data 250 may be sequentially generated in accordance with the priority order considering both the current scan mode and the usage frequency.

  Moreover, in the said 3rd Embodiment etc., although the guidance server 50 received the support request signal from the user 101, the aspect in which the production | generation of the some alternative audio | voice data 250 is started is illustrated. However, the present invention is not limited to this.

  For example, when the guidance server 50 receives the image data 300, the generation of the alternative voice data 250 corresponding to the secret word 110 included in the image data 300 may be started.

  Specifically, in response to the reception of the image data 300 (301) by the guidance server 50, the sound generation unit 65 includes a plurality of alternative sound data corresponding to the plurality of secret words 110 included in the image data 300. 250 generation is started. The generated alternative audio data 250 is stored in the storage unit 55. For example, alternative voice data 251, 252, 253 corresponding to the secret words 111, 112, 113 are generated and stored in the storage unit 55 as needed.

  During the generation of the plurality of alternative audio data 250 or after the generation is completed, the same operation as the operation after step S20 in FIG. 19 is executed. Specifically, when the user voice data 400 includes the secret word 110 and the alternative voice data 250 corresponding to the secret word 110 is already stored in the storage unit 55, the user voice data 400 stored in the storage unit 55 Synthetic voice data 450 is generated using alternative voice data 250. On the other hand, when the user voice data 400 includes the secret word 110 and the alternative voice data 250 corresponding to the secret word 110 is not stored in the storage unit 55, the alternative voice data 250 is generated by the machine voice generation process. Then, synthesized user voice data is generated using the generated alternative voice data 250.

  According to this, since the alternative voice data 250 corresponding to the secret word 110 included in the received image data 300 is preferentially generated, it corresponds to the secret word 110 that is likely to be emitted by the user 101. Alternative voice data 250 to be generated can be generated in advance. Therefore, it is possible to improve the possibility that the substitute voice data 250 stored in the storage unit 55 is used when the synthesized voice data 450 is generated.

  In such a modification, a plurality of alternative audio data 250 may be sequentially generated in accordance with the priority order based on the above-described usage frequency. That is, a plurality of alternative sound data 250 corresponding to the plurality of secret words 110 may be generated in accordance with the priority order based on the use frequency of the plurality of secret words included in the received image data 300.

<Fourth embodiment>
The fourth embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  In the first embodiment, when the guidance server 50 receives the partial voice data 430, the alternative voice data 250 (251) corresponding to the secret word 110 included in the partial voice data 430 is converted into the voice generation unit 65 each time. Is generated by Then, the voice generation unit 65 generates synthesized voice data 450 (451) using the alternative voice data 250. In the first embodiment, the sound generation unit 65 sequentially generates the substitute sound data 250, and the generated substitute sound data 251 is not stored.

  In the fourth embodiment, the alternative voice data 250 generated by the voice generation unit 65 (that is, used for generating the synthesized voice data 450) is stored in the storage unit 55 of the guidance server 50. Then, synthesized voice data 450 is generated using the substitute voice data 250 stored in the storage unit 55.

  In the fourth embodiment, it is assumed that the guidance server 50 receives partial voice data 432 (described later) different from the partial voice data 431 among the user voice data 400 emitted by the user 101. When the secret word 110 corresponding to the alternative voice data 250 stored in advance in the storage unit 55 is included in the partial voice data 432, the voice generation unit 65 synthesizes using the stored alternative voice data 250. Audio data 452 is generated.

  Here, the partial voice data 432 is voice data of a part extracted by the voice recognition unit 64 next to the partial voice data 431 in the user voice data 400.

  FIG. 21 is a flowchart showing an operation related to voice processing of the guidance server 50 according to the fourth embodiment.

  The guidance server 50 receives the user voice data 400 (step S21), and the voice recognition unit 64 extracts the partial voice data 430 by executing the processes of steps S21 to S23. Thereafter, the voice recognition unit 64 determines whether or not the secret word 110 is included in the partial voice data 430 by voice recognition processing on the partial voice data 430 (step S24). When it is determined that the secret word 110 is included in the partial voice data 430, the voice generation unit 65 determines whether the alternative voice data 250 corresponding to the secret word 110 is stored in the storage unit 55. Is determined (step S62).

  When it is determined that the alternative voice data 250 corresponding to the secret word 110 is not stored in the storage unit 55, the voice generation unit 65 generates the alternative voice data 250 corresponding to the secret word 110 (step S25). ), And the generated alternative voice data 250 is stored in the storage unit 55 (step S64). Then, the voice generation unit 65 generates synthesized voice data 450 in which the secret voice data 150 in the partial voice data 430 is replaced with the generated alternative voice data 250 (step S26). The synthesized voice data 450 is transmitted to the supporter terminal 70 (step S27).

  On the other hand, when it is determined that the alternative voice data 250 is stored in the storage unit 55, the voice generation unit 65 acquires the stored alternative voice data 250 from the storage unit 55 (step S63). .

  Then, the voice generation unit 65 generates synthesized voice data 450 in which the secret voice data 150 in the partial voice data 430 is replaced with the acquired alternative voice data 250 (step S26). The synthesized voice data 450 is transmitted to the supporter terminal 70 (step S27).

  FIG. 22 is a diagram illustrating timings related to the image data 300 and the partial audio data 431 and 432 according to the fourth embodiment. FIG. 23 is a diagram showing a situation in which the substitute voice data 250 (251) used for generating a certain synthesized voice data 450 (451) is stored in the storage unit 55, and FIG. It is a figure which shows the condition where the alternative voice data 250 (251) already stored is used in the case of the production | generation of the audio | voice data 450 (452).

  With reference to FIGS. 22 to 24 as well, alternative voice data 251 used in generating a certain synthesized voice data 451 is stored in the storage unit 55 in advance, and another synthesized voice is stored using the stored alternative voice data 251. An operation for generating the data 452 will be described.

  First, the guidance server 50 extracts partial voice data 431 from the user voice data 400 (step S23). For example, FIG. 23 shows a situation in which voice data of the part “I want to scan a file and send it to Hase Real Estate” among the voices uttered by the user 101 is extracted as the partial voice data 431. ing.

  Thereafter, the voice recognition unit 64 determines that the secret word 111 “Hase Real Estate” is included in the partial voice data 431 (step S24). At this time, since the alternative voice data 250 corresponding to the secret word 111 is not stored in the storage unit 55, the process proceeds from step S24 to step S62 to step S25. Then, the voice generation unit 65 generates the alternative voice data 251 of the alternative word 211 “ABC” corresponding to the secret word 111 “Hase Real Estate” (step S25), and guides the generated alternative voice data 251 (“ABC”). The data is stored in the storage unit 55 of the server 50 (step S64).

  Then, the voice generation unit 65 generates synthesized voice data 451 using the generated alternative voice data 251 (“ABC”) (step S26), and the guidance server 50 transmits the synthesized voice data 451 to the supporter terminal 70. (Step S27). The supporter terminal 70 outputs a voice based on the received synthesized voice data 451 (“I want to scan a file and send it to ABC”).

  Thereafter, the guidance server 50 extracts partial voice data 432, which is voice data of another part, from the user voice data 400 (step S23). For example, in FIG. 24, the voice data of the portion “Hase Real Estate has been touched, what should I do next?” In the voice uttered by the user 101 is extracted as new partial voice data 432. The status is shown.

  The voice recognition unit 64 determines that the secret word 110 (the secret word 111 “Hase Real Estate”) is included in the partial voice data 432, and determines the alternative word 211 “ABC” corresponding to the secret word 110 (111). Ask. Further, it is determined whether or not the alternative voice data 251 (“ABC”) corresponding to the alternative word 211 “ABC” is already stored in the storage unit 55. At this time, it is determined that the alternative voice data 251 (“ABC”) is already stored in the storage unit 55. In other words, it is determined that the secret voice 110 (“Hase Real Estate”) corresponding to the alternative voice data 251 (“ABC”) already stored in the storage unit 55 is included in the partial voice data 432. .

  Then, the voice generation unit 65 acquires the stored alternative voice data 251 from the storage unit 55 (step S63). Here, the alternative voice data 251 of the alternative word 211 “ABC” corresponding to the secret word 111 “Hase Real Estate” is acquired from the storage unit 55. The voice generation unit 65 generates synthesized voice data 452 using the alternative voice data 251 acquired from the storage unit 55 (step S26). Thereafter, the guidance server 50 transmits the generated synthesized voice data 452 to the supporter terminal 70 (step S27), and the supporter terminal 70 touches the voice based on the received synthesized voice data 452 (“ABC is touched, "What should I do?") Is output.

  According to the operation as described above, the alternative voice data 251 used when generating a certain synthesized voice data 451 is stored in advance and used when the next synthesized voice data 452 is generated. Therefore, it is not necessary to generate the alternative audio data 251 again. Therefore, since the time required for generating the synthesized speech data 452 is shortened, it is possible to suppress a delay in transmitting the synthesized speech data 452 to the supporter terminal 70.

<Fifth Embodiment>
The fifth embodiment is a modification of the first embodiment. Below, it demonstrates centering on difference with 1st Embodiment.

  The first embodiment exemplifies a mode in which the transmission destination included in the image data 300 (301) is determined as the secret word 110 in the scan mode. More specifically, in the secret word list 601 received from the MFP 10, the transmission destination included in the image based on the image data 300 is registered as the secret word 110. A secret word is determined based on such registered contents.

  The fifth embodiment exemplifies a mode in which the word (word) indicating the file name, creator, date, and heading of the file body of the file 550 stored in the box of the MFP 10 is determined as the secret word 110 in the box mode. To do. In the fifth embodiment, operation guidance is performed while viewing the display screen of information relating to the file 550 stored in the box of the MFP 10. Hereinafter, image processing and sound processing in the fifth embodiment will be described in order.

  First, when receiving a support request signal from the user 101, the guidance server 50 also receives the secret word list 603 from the MFP 10 (see FIG. 27). Here, as the secret word 110, the file name (such as “patent”) of the file 550 stored in the box of the MFP 10, the creator (such as “Taro Yamada”), the date (such as “2013/03/11”), and the like Each word (word) indicating the heading (“image forming apparatus”, “invention summary”, etc.) of the file text is registered in the secret word list 603. Then, the guide server 50 generates a conversion dictionary 653 based on the secret word list 603.

  Image processing in the fifth embodiment will be described with reference to FIGS.

  Here, as shown in FIG. 25, it is assumed that image data 303 (300), which is an information display screen related to file 550, is displayed on touch panel 25 of MFP 10. When the guide server 50 receives the image data 303, composite image data 353 (350) is generated by image processing based on the conversion dictionary 653. Then, the composite image data 353 is displayed on the display unit 76b of the supporter terminal 70.

  Specifically, the touch panel 25 of the MFP 10 displays image data 303 having three icons 500 (501 to 503). Below these icons 500 (501 to 503), file names “Patent 1” to “Patent 3” of the corresponding files 550 (551 to 553) are displayed. When the guide server 50 receives the image data 303, the composite image data 353 is generated by image processing based on the conversion dictionary 653, and the composite image data 353 is transmitted to the supporter terminal 70. Then, the composite image data 353 is displayed on the display unit 76b of the supporter terminal 70. In the composite image data 353, the file names “XYZ1” to “XYZ3” (file names expressed using alternative words) of the files 551 to 553 are displayed below the corresponding icons 501 to 503.

  Next, it is assumed that the user 101 presses the icon 501 corresponding to the file 551 “Patent 1”. When the icon 501 corresponding to the file 551 is pressed by the user 101, an image based on the image data 304 is displayed on the touch panel 25 of the MFP 10 (see the left side of FIG. 26). Then, the image data 304 is transmitted from the MFP 10 to the guidance server 50.

  When receiving the image data 304, the guidance server 50 determines whether or not the secret word 110 is included in the image data 304 based on the conversion dictionary 653 (see FIG. 27).

  When it is determined that the secret word 110 is included in the image data 304, the image generation unit 61 generates composite image data 354 in which the secret word 110 is replaced with the alternative word 210.

  Specifically, the image data 304 includes the file name (“patent 1”) of the file 551, the creator (“Taro Yamada”), the secret word 110 of the date (“2013/03/11”), and the file 551. It is determined that the secret word 110 of the headline (“image forming apparatus” and “invention summary”) is included. The image generation unit 61 generates composite image data 354 (see the right side of FIG. 26) in which the secret word 110 is replaced with the corresponding alternative word 210. For example, “patent” that is the secret word 111 (110) in the image data 304 is replaced with “abcd” that is the alternative word 211 (210) when the composite image data 354 is generated.

  Then, the guidance server 50 transmits the generated composite image data 354 to the supporter terminal 70, and the supporter terminal 70 displays the composite image data 354 on the display unit 76b.

  In this embodiment, as described above, the file name, creator, date, and heading of the text of the file 550 included in the image data 300 are determined as the secret word 110. On the other hand, words other than the heading among words (words) included in the text of the file 550 are not determined as the secret word 110.

  However, when a word other than the heading is not determined as the secret word 110 and is displayed as it is on the supporter terminal 70, there is a risk of leakage from a word (word) included in the text of the file 550. In order to avoid such a problem, the image generation unit 61 converts a part other than the heading into a reading avoidance image (an image in which the part cannot be read).

  In addition, there is a high possibility that the body of the file 550 contains a very large number of words, and individual conversion processing (processing for converting each word into an individual alternative word) is performed on all of these words. Performing image processing is not efficient.

  In order to efficiently conceal the large number of words, the interpretation avoidance image is an image generated by image processing that does not involve individual conversion processing (processing for converting each word into individual substitute words). Is preferred. The interpretation avoidance image may be generated by, for example, standard image processing performed uniformly over the entire area other than the heading of the text in the display area of the file text.

  Specifically, among the words included in the body of the file 551, the image of the portion other than the heading is converted into an interpretation avoidance image in which the characters “DUMMY” that cannot be interpreted are enumerated. (See FIG. 26). In this embodiment, an image that repeatedly displays the characters “DUMMY” is used as the interpretation avoidance image. However, the present invention is not limited to this, and other characters such as “*** (asterisk)” are repeated. It may be an image to be displayed. Further, a blank image may be used as the interpretation avoidance image (in other words, an image of a portion other than the heading may be deleted).

  Next, sound processing in the fifth embodiment will be described.

  When the guidance server 50 receives the user voice data 400, the voice recognition unit 64 determines whether or not the secret voice 110 is included in the user voice data 400.

  Here, the secret word 110 is a word (word) indicating the file name, creator, date, and heading of the file text of the file 550 as described above (see FIG. 27).

  When it is determined that the secret word 110 is included in the user voice data 400, the voice generation unit 65 generates the substitute voice data 250 of the substitute word 210 corresponding to the secret word 110, and the substitute voice data 250 is used to generate synthesized speech data 450.

  The guidance server 50 transmits the synthesized voice data 450 to the supporter terminal 70, and the synthesized voice data 450 is output from the supporter terminal 70.

  For example, when it is determined that the confidential voice word 112 (110) “Taro Yamada” is included in the user voice data 400 issued while the user 101 looks at the image data 304, the confidential word 112 “Taro Yamada” Alternative voice data 250 of the corresponding alternative word 212 “abcd” (see FIG. 27) is generated. Thereafter, synthesized voice data 450 is generated using the generated alternative voice data 250, and the synthesized voice data 450 is transmitted to the supporter terminal. The supporter terminal 70 that has received the synthesized voice data 450 outputs a voice based on the synthesized voice data 450.

  According to the operation as described above, the phrase indicating the file name, creator, and date of the file 550 is determined as the secret word 110, and image processing (image conversion processing, etc.) and voice processing (voice conversion) for the secret word 110 are performed. Processing). Accordingly, it is possible to avoid leakage of a secret word (confidential information) included in the display screen in the box mode of the MFP 10. Specifically, regarding the words and phrases that indicate the file name, creator, and date of the file 550, it is possible to prevent confidential information from being leaked through vision and to prevent leakage of confidential information through hearing. It is possible.

  Similarly, since the headline of the text of the file 550 is also determined as the secret word 110, it is possible to prevent confidential information from leaking through hearing and / or vision.

  Further, regarding the part other than the heading of the text of the file 550, the part is converted into the interpretation avoidance image, so that it is possible to prevent leakage of confidential information at least visually.

  In addition, if the conversion process (speech conversion process and / or image conversion process) is performed on a part other than the heading of the body of the file 550, the conversion process for a very large number of words takes a lot of time. On the other hand, in the above aspect, since the conversion process (speech conversion process) related to the voice other than the heading is not performed, the time required for the voice conversion process can be suppressed. Also for the image, the portion other than the heading is not determined as the secret word 110, and the image conversion process to the alternative word 210 is not performed, so that the time required for the image conversion process can be suppressed.

<Modifications>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.

  In each of the above embodiments, the voice transmission process from the MFP 10 to the supporter terminal 70 has been exemplified, but the present invention is not limited to this. For example, the voice transmission process from the supporter terminal 70 to the MFP 10 is performed in the same manner. With reference to the flowchart of FIG. 28, the audio | voice transmission process from the supporter terminal 70 to MFP10 is demonstrated.

  The supporter voice data 410 issued by the supporter 102 is transmitted to the guidance server 50 by the supporter terminal 70. When the guidance server 50 receives the supporter voice data 410 (step S70), the voice recognition unit 64 determines whether or not a non-silent part exists in the supporter voice data 410 (step S71). After that, the voice recognition unit 64 determines whether or not the supporter voice data 410 includes a silent part for a predetermined time or longer (step S72).

  When it is determined that there is a silent part for a predetermined time or longer in the supporter voice data 410, the voice recognition unit 64 extracts the partial voice data 440 that is a part of the supporter voice data 410 (step S73).

  Then, the voice generation unit 65 performs a voice recognition process on the partial voice data 440 to generate an alternative word 210 corresponding to one of the secret words 110 included in the secret word list 601 (see FIG. 9) in the supporter voice data 410. It is determined whether it is included (step S74).

  When it is determined that the alternative word 210 is included in the partial voice data 440, the voice generation unit 65 generates the secret voice data 161 of the secret word 110 corresponding to the substitute word 210 (step S75).

  Then, the voice generation unit 65 generates synthesized voice data 460 (synthesized supporter voice data 460) in which the substitute voice data 261 that is the voice data of the substitute word 210 included in the partial voice data 440 is replaced with the secret voice data 161. (Step S76).

  Thereafter, the guidance server 50 transmits the synthesized voice data 460 to the MFP 10 (step S77), and the synthesized voice data 460 is output in the MFP 10.

  FIG. 29 is a diagram illustrating an example of a voice transmission process from the supporter terminal 70 to the MFP 10. This will be specifically described with reference to FIG.

  In FIG. 29, it is assumed that the supporter 102 utters a voice “Please touch ABC” following the voice transmission process of FIG. 7.

  First, the voice recognition unit 64 recognizes voice data including a voice “Please touch ABC” as partial voice data 441. In addition, the voice recognition unit 64 includes an alternative word corresponding to one of the plurality of secret words 110 (111 to 113) registered in the conversion dictionary 651 (see FIG. 9) in the partial voice data 441 (440). Whether 210 (211 to 213) is included is determined. Specifically, it is determined that the alternative word 211 “ABC” corresponding to the secret word 111 “Hase Real Estate” is included in “Please touch ABC” in the partial audio data 441.

  Then, the voice generation unit 65 generates the secret voice data 161 (160) of the secret word 111 “Hase Real Estate” corresponding to the alternative word 211 “ABC”. The secret voice data 161 (160) is voice data (mechanical voice data) artificially generated by imitating a human voice.

  Thereafter, synthesized voice data 461 (460) is generated by replacing the substitute voice data 261 of the substitute word 211 “ABC” included in the partial voice data 441 with the secret voice data 161 (“Hase Real Estate”). Then, the guidance server 50 transmits the synthesized voice data 461 to the MFP 10.

  The MFP 10 that has received the synthesized voice data 461 outputs the synthesized voice data 461. Specifically, the MFP 10 outputs “Please touch Hase Real Estate.” Which is a voice based on the synthesized voice data 461.

  Here, the user 101 does not know the content of the alternative word 210, and the supporter 102 does not know the content of the secret word 110.

  For this reason, if the supporter voice data 410 generated by the supporter 102 is transmitted to the MFP 10 as it is, the alternative word 210 that the user 101 does not know is transmitted to the user 101, which may cause confusion to the user 101. There is.

  On the other hand, according to the aspect described above, the substitute word 210 included in the supporter voice data 410 issued by the supporter 102 is replaced with the secret voice data 160 to generate the synthesized voice data 460, and the synthesized voice data 460 is transmitted to the user 101. Since it is transmitted, confusion of the user 101 can be avoided.

  Further, since the secret voice data 160 (artificial voice) of the specific secret word 110 (111) is included in the supporter voice data 410 (supporter voice) (the artificial voice is included in the supporter voice), the supporter voice data The user 101 can know that some processing has been performed on the voice portion corresponding to the specific secret word 110 (111) out of 410. In the case where the user 101 knows that conversion processing related to some secret words has been performed, the user 101 has also learned that some conversion processing has been performed on a specific secret word 110 (for example, 111). The user 101 can confirm (guess) that the specific secret word 110 (111) is not transmitted to the supporter 102. In other words, it can be confirmed that confidential information regarding a specific secret word is not leaked.

  Further, in the above aspect, the secret voice data 161 (the voice data of the secret word 110 corresponding to the alternative word 210 included in the supporter voice data 410) is generated one by one in the voice transmission from the supporter 102 side to the user 101 side. Then, conversion processing for the supporter speech (processing for converting the alternative word 210 to the secret word 110 (inverse conversion)) is performed using the confidential speech data 161 (machine speech).

  However, the present invention is not limited to this. For example, the voice data of the user 101 (secret voice data 151) is first stored in the storage unit 55 in advance in voice transmission from the user 101 side to the supporter 102 side (see FIG. 30), and then the user 101 from the supporter 102 side. When the voice transmission to the side is performed, the conversion process for the supporter voice may be performed using the secret voice data 151 already stored in the storage unit 55 (see FIG. 31).

  This will be specifically described with reference to FIGS. 30 and 31. FIG. FIG. 30 is a diagram illustrating a voice transmission process from the user 101 to the supporter 102. FIG. 31 is a diagram illustrating a situation in which voice transmission processing from the supporter 102 to the user 101 is performed.

  In FIG. 30, it is assumed that the user 101 has issued a voice saying “I want to scan a file and send it to Hase Real Estate”.

  The guidance server 50 that has received the user voice data 400 extracts partial voice data 431 (430) from the user voice data 400. Then, based on the conversion dictionary 651 (see FIG. 9), it is determined whether or not the secret word 110 is included in the partial voice data 431.

  Specifically, the voice recognition unit 64 determines that the partial voice data 431 “I want to scan a file and send it to Hase Real Estate” includes the secret word 111 “Hase Real Estate”.

  Then, the voice generation unit 65 extracts the secret voice data 151 of the secret word 111 “Hase Real Estate” issued by the user 101 and stores it in the storage unit 55 of the guidance server 50.

  Thereafter, the voice generation unit 65 generates the synthesized voice data 451 by using the alternative voice data 251 of the alternative word 211 “ABC” corresponding to the secret word 111 “Hase Real Estate”. The generated synthesized voice data 451 is transmitted to the supporter terminal 70 and output from the supporter terminal 70.

  As described above, in this aspect, when the synthesized voice data 450 (451) is generated, the secret voice data 150 (151) output from the user voice data 400 (partial voice data 430 (431)) is stored in the storage unit in advance. Keep it.

  Next, an audio transmission process from the supporter terminal 70 to the MFP 10 will be described with reference to FIG. In FIG. 31, it is assumed that, after the voice transmission process from the user 101 to the supporter 102, the supporter 102 emits a voice “Please touch ABC”.

  The guide server 50 that has received the supporter voice data 410 extracts the partial voice data 441 (440) from the supporter voice data 410. Then, based on the conversion dictionary 651 (see FIG. 9), it is determined whether or not the alternative word 210 corresponding to the stored secret voice data 151 (150) is included in the partial voice data 441.

  Specifically, the alternative word 211 “ABC” corresponding to the stored secret voice data 151 (more specifically, the voice data of the secret word 111 “Hase Real Estate” issued by the user 101) is the partial voice data 441. The voice recognition unit 64 determines that it is included in the voice “Please touch ABC.”

  Then, the voice generation unit 65 replaces the substitute voice data 261 (260) corresponding to the substitute word 211 “ABC” in the partial voice data 441 with the stored secret voice data 151 (150), and the synthesized voice data 461 ( 460). When this synthesized voice data 461 is generated, not the secret voice data 161 (the machine voice data of the secret word 111 “Hase Real Estate” corresponding to the alternative word 211 “ABC”) but the secret voice data 151 (stored in the storage unit 55). Recorded voice data of the user 101). In other words, the secret voice data 150 stored in advance is used as the secret voice data corresponding to the alternative word 211 “ABC” in the partial voice data 441, and the synthesized voice data 461 is generated.

  Thereafter, the guidance server 50 transmits the synthesized voice data 461 (460) to the supporter terminal 70, and the voice of “Please touch Hase Real Estate” is output based on the synthesized voice data 461 at the supporter terminal 70. . The voice “Hase Real Estate” included in the synthesized voice data 461 is output using the voice of the user 101, and the voice “Please touch” contained in the synthesized voice data 461 is used using the voice of the supporter 102. Is output.

  According to such a modified example, the synthesized voice data 460 obtained by replacing the substitute word 210 included in the supporter voice data 410 (partial voice data 441) issued by the supporter 102 with the previously stored secret voice data 150 is voiced. The output data is transmitted to the MFP 10 on the user 101 side. Therefore, it is possible to avoid the confusion of the user 101 due to the substitution word 210 that the user 101 does not know is transmitted to the user 101.

  In addition, voice data issued in the past by the user 101 is stored as the secret voice data 150 in the storage unit 55, and the synthesized voice data 460 is generated using the secret voice data 150 already stored in the storage unit 55. Is done. Accordingly, since it is not necessary to generate the secret voice data 160 that is the voice data of the secret word 111 “Hase Real Estate” corresponding to the one alternative word 211 “ABC”, the time required for generating the synthesized voice data 460 is shortened. Is done. As a result, it is possible to suppress delay in transmission of the synthesized voice data 460 to the MFP 10.

  Further, the secret voice data 150 of the specific secret word 110 (111) issued by the user 101 is included in the supporter voice data 410. Therefore, the user 101 can know that some processing has been performed on the voice portion corresponding to the specific secret word 110 (111) in the supporter voice data 410.

  In the above aspect, the voice (voice related to the secret word 110) uttered by the user 101 (the secret voice data 150) is stored in the storage unit 55, and the secret voice data 150 is transmitted from the supporter 102 side to the user. Although the aspect utilized in the audio | voice transmission process to 101 side is illustrated, it is not limited to this.

  For example, data (substitute voice data 260) of voice (voice related to the substitute word 210) uttered by the supporter 102 is stored in the storage unit 55, and the substitute voice data 260 is voiced from the user 101 side to the supporter 102 side. You may make it utilize in a transmission process.

  Such an aspect will be described with reference to FIG.

  First, as shown in FIG. 31, in the voice transmission process from the supporter 102 side to the user 101 side, when the supporter 102 utters “Please touch ABC”, the guidance server 50 Through similar processing, voice conversion processing is performed, and synthesized voice data 461 (460) is generated. When the synthesized voice data 461 is generated, the substitute voice data 261 that is the recording data of the substitute word 211 “ABC” issued by the supporter 102 is stored in the storage unit 55.

  Thereafter, when voice transmission processing from the user 101 side to the supporter 102 side is performed, the voice generation unit 65 generates synthesized voice data 450 using the stored alternative voice data 260 (261).

  Specifically, for example, when the user 101 utters a voice saying “Do you want to press the Hase Real Estate button?”, Partial voice data 433 (not shown) including the voice is extracted from the user voice data 400. When the voice recognition unit 64 determines that the secret word 111 “Hase Real Estate” is included in the partial voice data 433, synthesized voice data 453 is generated. At this time, the alternative voice data 260 (261) already stored in the storage unit 55 as alternative voice data (substitution voice data) of the alternative word 211 “ABC” corresponding to the secret word 111 “Hase Real Estate” Is used to generate synthesized speech data 453. The generated synthesized voice data 453 is transmitted to the supporter terminal 70 and output from the supporter terminal 70.

  According to such an aspect, particularly, when the synthesized voice data 453 is generated, the alternative voice data 260 stored in advance in the storage unit 55 is used. Therefore, the alternative voice data is generated again by the machine voice generation process. Is not required.

  Moreover, in each said embodiment, although the guidance server 50 performs image processing and an audio | voice process is illustrated, it is not limited to this. For example, the operation of the guide server 50 may be executed by the MFP 10. Specifically, the MFP 10 may perform the same operation as that of the image processing unit 60a and the audio processing unit 60b of the guidance server 50.

1 Operation Guidance System 10 MFP (Image Forming Device)
DESCRIPTION OF SYMBOLS 50 Guide server 70 Supporter terminal 101 User 102 Supporter 110-113 Secret word 210-213 Alternative word 300-304 Image data 350-354 Composite image data 400 User voice data 430-432 Partial voice data 150-152,160 Secret voice data 250 , 251, 260 Alternative voice data 450 to 452, 460, 461 Synthetic voice data 651 to 653 Conversion dictionary

Claims (23)

A guidance server in the operation guidance system,
Image receiving means for receiving, from the image forming apparatus, first display image data which is data of a first display image displayed on an operation unit of the image forming apparatus which is a user's operation target;
When a secret word is included in the first display image data, the first synthesized image data is obtained by replacing the secret word in the first display image with an alternative word corresponding to the secret word. Image generating means for generating composite image data;
Image transmitting means for transmitting the first combined image data as display data on the supporter terminal to a supporter terminal used for operation guidance to the user by a supporter who is a person supporting the user;
Sound receiving means for receiving user sound data including sound data emitted by the user from the image forming apparatus ;
Voice recognition means for determining whether or not the secret word is included in the user voice data by voice recognition processing on the user voice data;
When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is the voice data of the alternative word corresponding to the secret word. Voice generating means for generating synthesized user voice data which is data replaced with certain alternative voice data;
Voice transmitting means for transmitting the synthesized user voice data to the supporter terminal as voice output data at the supporter terminal;
A guidance server comprising: In the guidance server according to claim 1,
The user voice data is divided into a plurality of partial voice data,
When the voice recognition process determines that the secret word is included in the first voice data that is one partial voice data in the user voice data, the voice generation unit is configured to output the first voice data. Generating the first synthesized voice data by replacing the secret voice data in the substitute voice data,
The guidance server characterized in that the voice transmitting means transmits the first synthesized voice data to the supporter terminal. In the guidance server according to claim 2,
The first audio data includes audio uttered by the user during display of the first display image,
The image receiving means receives second display image data, which is data of a second display image displayed on the operation unit subsequent to the first display image, after completion of reception of the first display image data. Received from the image forming apparatus,
The image generation means replaces the secret word in the second display image data with an alternative word corresponding to the secret word when a secret word is included in the second display image data. Generating second composite image data which is image data;
When the reception of the first audio data is started, the image transmission means does not permit transmission of the second synthesized image data, and the first synthesized audio generated based on the first audio data A guide server that permits transmission of the second composite image data after the data transmission is completed. In the guidance server according to claim 3,
The image transmission means is a period between a reception start time of the first audio data and a time when an output required time of the first synthetic audio data has elapsed since the completion of transmission of the first synthetic audio data. If the second display image data is received by the image receiving means within the first period, transmission of the second composite image data is not permitted until the end of the first period and A guide server that permits transmission of the second composite image data after the end of the first period. In the guidance server according to claim 1,
Storage means for storing audio data;
Further comprising
The voice generation means includes
Generation of a plurality of alternative voice data, which are voice data of alternative words corresponding to a plurality of secret words, is started from a predetermined time prior to the generation of the synthesized user voice data, and the generated alternative voice data is stored in the storage means. Store and
When the user voice data includes the secret word and the alternative voice data corresponding to the secret word is not stored in the storage unit, the alternative voice data is generated by a machine voice generation process. Generating the synthesized user voice data using the alternative voice data,
When the user voice data includes the secret word and the alternative voice data corresponding to the secret word is already stored in the storage unit, the alternative voice data stored in the storage unit is used. And generating the synthesized user voice data. In the guidance server according to claim 5,
The voice generation means starts generating the plurality of alternative voice data in response to the guide server receiving a support request signal indicating the generation of an operation guidance request from the user. Information server. In the guidance server according to claim 6,
The plurality of concealment words are at least one of a word indicating a transmission destination in a destination designation screen in scan image transmission of the image forming apparatus and a word indicating a transmission destination included in a destination designation screen in facsimile transmission of the image forming apparatus A guide server characterized by including: In the guidance server according to claim 6,
The guide server, wherein the plurality of secret words include a phrase indicating file information displayed on an information display screen related to a file stored in a box of the image forming apparatus. In the guidance server according to claim 5,
The guidance server preferentially generates alternative voice data corresponding to a secret word that can be displayed in the current operation mode of the image forming apparatus among the plurality of alternative voice data. . In the guidance server according to claim 9,
A guide server characterized in that a current operation mode of the image forming apparatus is one of a plurality of modes including a scan mode, a facsimile transmission mode, and a box mode. In the guidance server according to claim 1,
Storage means for storing audio data;
Further comprising
The voice generation means includes
When the first display image data is received by the image receiving means, generation of a plurality of alternative voice data which are voice data of alternative words corresponding to a plurality of secret words is started, and the generated alternative voice data is Storing in the storage means;
When the user voice data includes the secret word and the alternative voice data corresponding to the secret word is not stored in the storage unit, the alternative voice data is generated by a machine voice generation process. Generating the synthesized user voice data using the alternative voice data,
When the user voice data includes the secret word and the alternative voice data corresponding to the secret word is already stored in the storage unit, the alternative voice data stored in the storage unit is used. And generating the synthesized user voice data. In the guidance server in any one of Claims 5 thru | or 11,
The guidance server, wherein the voice generation means generates the plurality of alternative voice data according to a priority order based on a frequency of use of the plurality of secret words. In the guidance server according to claim 2,
Storage means for storing the substitute voice data used for generating the first synthesized voice data;
Further comprising
The voice generation means determines that the secret word is included in the second voice data, which is partial voice data of the user voice data different from the first voice data, by the voice recognition process. And generating the second synthesized voice data in which the secret voice data in the second voice data is replaced with the substitute voice data using the substitute voice data stored in the storage means,
The guidance server characterized in that the voice transmitting means transmits the second synthesized voice data to the supporter terminal. In the guidance server according to claim 1,
The voice receiving means receives supporter voice data including voice data emitted by the supporter,
The voice recognition means determines whether or not one support word corresponding to one or a plurality of secret words is included in the supporter voice data by voice recognition processing on the supporter voice data,
When the one alternative word is included in the supporter voice data, the voice generation means converts the second alternative voice data, which is the voice data of the one alternative word in the supporter voice data, into the one alternative. Generating synthesized supporter voice data replaced with second secret voice data which is voice data of the secret word corresponding to the word;
The guidance server characterized in that the voice transmitting means transmits the synthesized supporter voice data to the image forming apparatus. In the guidance server according to claim 14,
Storage means for storing audio data;
Further comprising
The voice generation means includes
When generating the synthesized user voice data based on the user voice data, the secret voice data extracted from the user voice data is stored in the storage means,
A guidance server that generates the synthesized supporter voice data by using the secret voice data already stored in the storage means as the second secret voice data. In the guidance server according to any one of claims 2 to 4,
The voice recognition means, when there is a silent part for a predetermined time or more in the user voice data, a part of the user voice data that is classified so as to end when the silent state for the predetermined time has elapsed A guidance server that extracts voice data as the first voice data. The guidance server according to claim 16, wherein
The image receiving means also receives second display image data different from the first display image data from the image forming apparatus;
When the second display image data is received by the image receiving means during the voice recognition process of the user voice data, the voice recognition means includes the second display image data of the user voice data. A guidance server that extracts partial voice data divided so as to end at the reception time of the first voice data. In the guidance server according to claim 1,
The first display image is an image of an information display screen related to a file stored in a box of the image forming apparatus,
The secret word includes a word indicating at least one of a file name, an author, a date, and a file body header of the file ;
Wherein the image generating means generates the first synthesized image data by replacing the person the secret word in the alternative word,
The said voice production | generation means produces | generates the synthetic | combination user audio | voice data which replaced the said confidential audio | voice data with the said alternative audio | speech data, when the said confidential word is contained in the said user audio | voice data. The guide server according to claim 18, wherein
The secret word includes a word indicating a heading of the file body ,
The image generating unit converts the confidential words the portion other than heading the files present statements of the file the statement a first synthesized image data is replaced with the alternative word read avoid image A guide server for generating the first composite image data. In the computer built in the guidance server in the operation guidance system,
a) receiving, from the image forming apparatus, first display image data that is data of a first display image displayed on an operation unit of the image forming apparatus that is an operation target of the user;
b) First composite image data obtained by replacing the secret word in the first display image with an alternative word corresponding to the secret word when a secret word is included in the first display image data. Generating one composite image data;
c) transmitting the first composite image data as display data on the supporter terminal to a supporter terminal used for guidance to the user by a supporter who is a person supporting the user;
d) receiving user audio data including audio data uttered by the user from the image forming apparatus ;
e) determining whether or not the secret word is included in the user voice data by voice recognition processing on the user voice data;
f) When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is converted to the voice of the alternative word corresponding to the secret word. Generating synthesized user voice data that is data replaced with alternative voice data that is data;
g) transmitting the synthesized user voice data as voice output data at the supporter terminal to the supporter terminal;
A program for running An operation guidance system,
An image forming apparatus to be operated by the user;
A supporter terminal used for operation guidance of the image forming apparatus to the user by a supporter who is a person supporting the user;
A guidance server that mediates between the image forming apparatus and the supporter terminal;
With
The guidance server is
Image receiving means for receiving, from the image forming apparatus, first display image data which is data of a first display image displayed on the operation unit of the image forming apparatus;
When a secret word is included in the first display image data, the first synthesized image data is obtained by replacing the secret word in the first display image with an alternative word corresponding to the secret word. Image generating means for generating composite image data;
Image transmitting means for transmitting the first composite image data as display data at the supporter terminal to the supporter terminal;
Sound receiving means for receiving user sound data including sound data emitted by the user from the image forming apparatus ;
Voice recognition means for determining whether or not the secret word is included in the user voice data by voice recognition processing on the user voice data;
When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is the voice data of the alternative word corresponding to the secret word. Voice generating means for generating synthesized user voice data which is data replaced with certain alternative voice data;
Voice transmitting means for transmitting the synthesized user voice data to the supporter terminal as voice output data at the supporter terminal;
An operation guidance system comprising: An image forming apparatus in an operation guidance system,
Image acquisition means for acquiring first display image data which is data of a first display image displayed on the operation unit of the image forming apparatus which is a user's operation target;
When a secret word is included in the first display image data, the first synthesized image data is obtained by replacing the secret word in the first display image with an alternative word corresponding to the secret word. Image generating means for generating composite image data;
Image transmitting means for transmitting the first composite image data as display data on the supporter terminal to a supporter terminal used for guidance to the user by a supporter who is a person supporting the user;
Voice acquisition means for acquiring user voice data including voice data generated by the user;
Voice recognition means for determining whether or not the secret word is included in the user voice data by voice recognition processing on the user voice data;
When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is the voice data of the alternative word corresponding to the secret word. Voice generating means for generating synthesized user voice data which is data replaced with certain alternative voice data;
Voice transmitting means for transmitting the synthesized user voice data to the supporter terminal as voice output data at the supporter terminal;
An image forming apparatus comprising: In the computer built in the image forming apparatus in the operation guidance system,
a) acquiring first display image data which is data of a first display image displayed on the operation unit of the image forming apparatus which is an operation target of the user;
b) First composite image data obtained by replacing the secret word in the first display image with an alternative word corresponding to the secret word when a secret word is included in the first display image data. Generating one composite image data;
c) transmitting the first composite image data as display data on the supporter terminal to a supporter terminal used for guidance to the user by a supporter who is a person supporting the user;
d) obtaining user voice data including voice data uttered by the user;
e) determining whether or not the secret word is included in the user voice data by voice recognition processing on the user voice data;
f) When it is determined that the secret word is included in the user voice data, the secret voice data that is the voice data of the secret word in the user voice data is converted to the voice of the alternative word corresponding to the secret word. Generating synthesized user voice data that is data replaced with alternative voice data that is data;
g) transmitting the synthesized user voice data as voice output data at the supporter terminal to the supporter terminal;
A program for running

Images