JP2016051962A - Image processing device, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To assist input of information to an image processing device by a user using a terminal unit.SOLUTION: An image processing device includes a display control part for displaying one screen selected from a plurality of screens on a display part, an input acquisition part for acquiring input information through the displayed screen, a history storage part for making a memory memorize input history information regarding the acquired input information, and a transmission control part for transmitting the input history information to a terminal unit. The input information includes a first kind of input information input through a first screen, and a second kind of input information input through a second screen. The input history information includes one or more first kind of input information associated with the first screen, and one or more second kind of information associated with the second screen. The transmission control part transmits one or more first kind of input information to the terminal unit among the input history information when the first screen is displayed on the display part, and does not transmit one or more input information to the terminal unit.SELECTED DRAWING: Figure 8

Description

  The present invention relates to input of information via a screen displayed on an image processing apparatus.

  A technique for inputting information such as a setting value to an image processing apparatus such as a multifunction peripheral using a terminal device such as a smartphone is known. For example, Patent Document 1 discloses a technique for displaying the same input screen as the display unit of a multifunction peripheral on a display unit of a smartphone. In this technology, a history of information input via the display unit of the multifunction peripheral is transmitted to the smartphone. The history is used as a dictionary when inputting information via a screen displayed on a smartphone.

JP 2013-251623 A

  However, with the above technique, it may not be easy for the user to input appropriate information to the image processing apparatus. For example, when many types of screens can be displayed on the image processing apparatus, or when a large amount of information needs to be input, it may be difficult for the user to input appropriate information via each screen. there were. For this reason, there is a need for a technique that supports input of information by a user using a terminal device.

  An object of the present invention is to provide a technology that supports input of information into an image processing apparatus by a user using a terminal device.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following application examples.

Application Example 1 An image processing apparatus that is communicably connected to a terminal device,
A display unit;
A display control unit for displaying one screen selected from a plurality of screens including a first screen and a second screen on the display unit;
An input acquisition unit that acquires input information input via a screen displayed on the display unit, wherein the input information includes first type input information input via the first screen; And the second acquisition information input via the second screen, the input acquisition unit,
A history storage unit that stores, in a memory, input history information related to input information acquired by the input acquisition unit, wherein the input history information is associated with the first screen. The history storage unit, including: kind of input information; and one or more pieces of the second kind of input information associated with the second screen;
In order to display the input history information on the display unit of the terminal device, the input control information is transmitted to the terminal device, and the input history is displayed when the first screen is displayed on the display unit. Among the information, one or more types of first type input information are transmitted to the terminal device, and one or more types of second type input information are not transmitted to the terminal device, and the display unit is When the second screen is displayed, one or more of the second type of input information among the input history information is transmitted to the terminal device, and one or more of the first type of input is transmitted. The transmission control unit not transmitting information to the terminal device; and
An image processing apparatus comprising:

  According to the above configuration, when the first screen is displayed, the image processing apparatus transmits one or more first type input information of the input history information to the terminal device, and the second Do not send any kind of input information to the terminal device. In addition, when the second screen is displayed, the image processing apparatus transmits the second type of input information of the input history information to the terminal device, and the first type of input information to the terminal device. Do not send. As a result, according to the screen displayed on the display unit, it is possible to assist the user in inputting information using the terminal device.

Application Example 2 A terminal device that is communicably connected to an image processing device that displays one screen selected from a plurality of screens including a first screen and a second screen,
A display unit;
When the first screen is displayed on the image processing apparatus, first screen identification information indicating the first screen is acquired from the image processing apparatus, and the second screen is acquired by the image processing apparatus. A screen identification information acquisition unit that acquires second screen identification information indicating the second screen from the image processing device when the screen is displayed,
Input history information including one or more types of first type input information that is input via the first screen using the first screen identification information when the first screen identification information is acquired. When the second screen identification information is acquired, one or more second type input information input via the second screen is obtained using the second screen identification information. A history acquisition unit for acquiring input history information including;
When the first screen identification information is acquired, one or more of the first type of input information is displayed on the display unit, and when the second screen identification information is acquired, the one or more of the first type of information is displayed. A display control unit for displaying the second type of input information on the display unit;
A terminal device.

  According to the above configuration, one or more pieces of input information that are input via the screen being displayed on the image processing device among the input history information are displayed on the display unit of the terminal device. As a result, the terminal device can perform input support processing for information input by the user to the image processing device.

  The present invention can be realized in various forms, for example, a system including an image processing device and a terminal device, a control method for the image processing device, and a computer program for realizing these devices or methods. It can be realized in the form of a gram, a recording medium on which the computer program is recorded, or the like.

It is explanatory drawing which shows the system 900 as one Example of this invention. It is a figure which shows an example of UI screen. It is a figure which shows an example of the screen definition table DT. It is a figure which shows an example of screen transition table TT. It is a figure which shows an example of the input history recording table HT. It is a flowchart of the process at the time of screen switching. 6 is a diagram illustrating an example of a screen displayed on the mobile terminal 200. FIG. It is a sequence diagram of the input assistance process of 1st Example. It is a flowchart of a search process. It is a flowchart of a screen transition monitoring process. It is a sequence diagram of the input assistance process of 2nd Example.

A. First embodiment:
A-1. Configuration FIG. 1 is an explanatory diagram showing a system 900 as an embodiment of the present invention. The system 900 includes a multifunction device 100, a portable terminal 200 possessed by a user of the multifunction device 100, and an access point 50.

  The access point 50 realizes a function as a wireless LAN access point by using a communication method in accordance with, for example, the IEEE 802.11a / b / g / n standard. The multi-function device 100 and the portable terminal 200 are connected to be communicable by wireless communication via the access point 50.

  The multifunction device 100 includes a CPU 110 as a controller that controls the entire multifunction device 100, a volatile storage device 120 such as a DRAM, a nonvolatile storage device 130 such as a flash memory, a display unit 140 such as a liquid crystal panel, and a touch panel. The operation unit 150, the reading execution unit 160, the print execution unit 170, and a communication interface 180 for communicating with other devices (for example, the server 300 and the portable terminal 200). The communication interface 180 of the present embodiment is a wireless communication interface that conforms to the above-described IEEE 802.11a / b / g / n standard, for example.

  The volatile storage device 120 is used as a buffer area for temporarily storing various intermediate data generated when the CPU 110 performs processing, for example. The non-volatile storage device 130 stores a computer program 132, a management table group 134 to be described later, and a screen data group 136 to be described later.

  The reading execution unit 160 is an apparatus that generates image data (also referred to as scan data) representing an original by optically reading the original using an image sensor (not shown).

  The print execution unit 170 is an apparatus that prints an image, and is a so-called inkjet printer. However, other types of printers (for example, so-called laser printers) may be employed.

  The CPU 110 executes control processing of the multifunction peripheral 100 by executing the computer program 132 using the volatile storage device 120 and the nonvolatile storage device 130. The CPU 110 can execute an input support process (details will be described later) for supporting the input of information by the user in cooperation with the mobile terminal 200 as a part of the control process. The computer program 132 is provided, for example, in a form stored on a DVD-ROM or the like, or downloaded from a computer such as a server via the network 500.

  The mobile terminal 200 is a multi-function mobile phone such as a smartphone, for example. The mobile terminal 200 includes a CPU 210 as a controller that controls the entire mobile terminal 200, a volatile storage device 220 such as a DRAM, a nonvolatile storage device 230 such as a hard disk drive or a flash memory, and a display unit 240 such as a liquid crystal panel. And an operation unit 250 such as a touch panel and operation keys superimposed on the liquid crystal panel, and a communication interface 280 for performing wireless communication with other devices. The communication interface 280 is, for example, a mobile phone communication method (for example, W-) between an interface that performs wireless communication conforming to the above-described IEEE 802.11a / b / g / n standard and a base station (not shown). And an interface for performing wireless communication complying with (CDMA).

  The volatile storage device 220 is used as a buffer area for temporarily storing various intermediate data generated when the CPU 210 performs processing, for example. The non-volatile storage device 230 stores a computer program 232 different from the control program (not shown) of the mobile terminal 200. The computer program 232 is a program for adding a new function to the mobile terminal 200 (also referred to as “application”). The computer program 232 is provided in a form of being downloaded from a predetermined server, for example, by a provider different from the manufacturer of the mobile terminal 200 (for example, the manufacturer of the multifunction device 100). The CPU 210 can execute input support processing (details will be described later) in cooperation with the multifunction peripheral 100 by executing the computer program 232.

  As indicated by a broken line, the system 900 may include a server 300 connected to the network 500. An example including the server 300 will be described in a second embodiment.

  Next, the screen data group 136 stored in the non-volatile storage device 130 of the multifunction peripheral 100 will be described. The screen data group 136 includes a plurality of screen data representing a plurality of user interface screens (UI screens) displayed on the display unit 140. The UI screen is a screen displayed for inputting predetermined information by the user and notifying the user of information. Information input by the user via the UI screen (also referred to as input information) includes, for example, setting information related to functions such as copying and scanning of the multifunction device 100, function execution instructions for the multifunction device 100, and the like. The CPU 110 of the multi-function peripheral 100 displays one UI screen selected from the plurality of UI screens on the 140. The CPU 110 acquires various types of input information via the UI screen displayed on the display unit 140.

  FIG. 2 is a diagram illustrating an example of a UI screen. The UI screen D1 is a UI screen for performing settings related to electronic mail. The e-mail is used, for example, to realize an Internet fax function for transmitting / receiving scan data by e-mail. The UI screen D1 includes a plurality of options C11 to C13. Options C11 to C13 indicate setting items related to the mail address.

  The UI screen D2 is a UI screen for inputting an SMTP server address, which is one of setting items related to electronic mail. The UI screen D2 includes a field F21 for inputting an SMTP server address and an OK button BT.

  The UI screen D3 is a UI screen for selecting a function used by the user. The UI screen D3 includes a plurality of options C31 to C33 each indicating a function that can be executed by the multifunction peripheral 100. For example, the option C33 indicates a function that uses a cloud service. The cloud service is, for example, a storage service that stores data transmitted from a client (for example, the multifunction device 100 or the portable terminal 200), or a social network service that stores and discloses data transmitted from a client. The functions that use the cloud service include a Scan To Upload function and a Download To Print function, which will be described later.

  The UI screen D4 is a UI screen for inputting selection of the type of cloud service to be used in the function of using the cloud service. The UI screen D4 includes a plurality of options C41 to C43 indicating the type of cloud service. Specifically, the cloud services shown in these options are, for example, “Evernote (registered trademark)”, “PICASA (registered trademark)”, “Dropbox (registered trademark)”, “Facebook (registered trademark)”, etc. This is a known service.

  The UI screen D5 is a UI screen for inputting selection of an account used for logging in to a cloud service to be used. The UI screen D5 includes a plurality of options C51 to C53 indicating a plurality of previously registered accounts.

  The UI screen D6 is a UI screen for inputting a password used to log in to the cloud service to be used. The UI screen D6 includes a field F61 for inputting a password and an OK button BT.

  The UI screen D7 is a UI screen for inputting selection of a function to be executed among a plurality of specific functions that can be executed as a function of using the cloud service. The UI screen D7 includes two options C71 and C72 respectively indicating a Scan To Upload function and a Download To Print function. The Scan To Upload function is a function for generating scan data by reading a document using the reading execution unit 160 and uploading the scan data to the cloud service. The Download To Print function is a function for downloading user image data (for example, scan data) stored in the cloud service from the cloud service and printing using the image data.

  The UI screen D8 is a UI screen for inputting selection of a storage format (file format) of scan data uploaded when the Scan To Upload function is used. The UI screen D8 includes a plurality of options C81 and C82 each indicating a plurality of file format options.

  The UI screen D9 is a UI screen for notifying the user that reading (scanning) of a document using the reading execution unit 160 is being performed. The UI screen D9 includes a message MS9 indicating that a document is being read (scanned).

  When the options included in the UI screens D1, D3 to D5, D7, and D8 are selected by being pressed by the user, input information indicated by the selected options is combined via these UI screens. Is input to the machine 100. In addition, when the OK button BT is pressed in a state where input information is input to the fields F21 and F61 included in the UI screens D2 and D6, the input information in the fields F21 and F61 is input to the multi-function device 100. The

  Next, the management table group 134 stored in the non-volatile storage device 130 of the multifunction peripheral 100 will be described. The management table group 134 includes a screen definition table DT, a screen transition table TT, and an input history recording table HT.

  FIG. 3 is a diagram illustrating an example of the screen definition table DT. In the screen definition table DT, screen IDs that are identification information of a plurality of UI screens including the UI screens D1 to D9 in FIG. 2 are recorded.

  Each UI screen (screen ID) is associated with information indicating whether or not to save the input history information. The input history information is history information related to input information input via the UI screen.

  When it is not useful to refer to past history information when a user inputs specific input information input via a specific UI screen, the UI screen for inputting the specific input information Is associated with information indicating that the input history information is not stored. For example, the UI screen D1 (FIG. 2 (A)) for the user to select a desired setting item and the UI screen D3 (FIG. 2 (C)) for the user to select a function desired to be used are input. It is a screen for inputting the input information according to a user's will. When the user inputs such input information, it is considered that it is less necessary to refer to past history information. For this reason, the UI screens D1 and D3 are associated with information indicating that the input history information is not stored. Further, a UI screen for simply notifying the user of information, for example, a UI screen D9 in FIG. 2I is associated with information indicating that input history information is not stored. In addition, it is considered that it is not desirable in terms of security to leave a history of information with relatively high confidentiality such as a password. For this reason, information indicating that the input history information is not stored is associated with a UI screen for inputting input information with relatively high confidentiality, for example, a UI screen D6 in FIG. .

  On the other hand, when it is useful to refer to past history information when the user inputs specific input information input via a specific UI screen, the specific input information is input. The UI screen is associated with information indicating that the input history information is stored. For example, as specific setting information of items that need to be set in order to use the function of the multifunction peripheral 100, setting information input in the past may be used again. In addition, the user may be able to appropriately determine the setting information to be input by referring to the setting information input in the past. For this reason, UI screens for inputting such specific setting information and relatively low confidential information as input information, for example, UI screens D2, D4, D5, D7, and D8, Information indicating that the input history information is to be saved is associated.

  The plurality of UI screens include a screen classified into a group including two or more UI screens and a screen not classified into a group. The UI screens classified into groups are associated with group IDs that are identification information of the groups. Two or more UI screens belonging to one group are screen groups for inputting input information related to each other. For example, two or more UI screens continuously displayed on the display unit 140 for inputting two or more pieces of input information used as a set of information belong to one group. More specifically, the five UI screens D4 to D8 for inputting a plurality of pieces of input information necessary for executing the function using the cloud service are classified into one group. For this reason, these five UI screens D4 to D8 are associated with “G1” as the same group ID (FIG. 3). A group to which the five UI screens D4 to D8 belong is represented as a group G1. On the other hand, since the UI screens D1 to D3 and D9 are not classified into groups, group IDs are not associated with each other.

  FIG. 4 is a diagram illustrating an example of the screen transition table TT. The screen transition table TT is a table in which transitions of a plurality of UI screens displayed on the display unit 140 are defined. Specifically, in the screen transition table TT, as in the screen definition table DT, IDs of a plurality of UI screens including the UI screens D1 to D9 in FIG. 2 are recorded. Event information and the next screen ID are associated with each UI screen (screen ID). The event information associated with a specific UI screen indicates an event that triggers switching of the screen displayed on the display unit 140 from the specific UI screen to the next UI screen. The next screen ID is an ID of a UI screen displayed on the display unit 140 next to a specific UI screen when an event indicated by the event information occurs. Of the next screen IDs recorded in the screen transition table TT, D10 to D15 are UI screen identification information not shown in FIG. The CPU 110 switches the UI screen displayed on the display unit 140 with reference to the screen transition table TT.

  For example, referring to the screen transition table TT, when the UI screen D1 is displayed on the display unit 140 and the option C11 of the UI screen D1 is selected, the screen displayed on the display unit 140 is the UI screen D1. It can be seen that the screen can be switched to the UI screen D10.

  Further, when the option C33 indicating the cloud service is selected via the UI screen D3, the screen displayed on the display unit 140 is switched to the UI screen D4. Then, it can be seen that the UI screens D4 to D8 belonging to the group G1 are sequentially displayed in succession according to the input information input by the user. Accordingly, an input information group necessary for using the cloud service is input to the multi-function peripheral 100 via the UI screens D4 to D8 belonging to the group G1.

  FIG. 5 is a diagram illustrating an example of the input history recording table HT. The input history recording table HT is a table for recording input history information. Specifically, the input history recording table HT is associated with information indicating that the input history information is stored in the screen definition table DT among a plurality of UI screens including the UI screens D1 to D9 in FIG. A history of input information input via the current UI screen is recorded.

  The input history recording table HT includes a plurality of entries (for example, entries E1 to E6 in FIG. 5) including input history information. Each entry includes a plurality of pieces of information associated with each other, that is, input information input in the past, a screen ID, input date / time information, and a user ID. The screen ID indicates a UI screen for inputting the input information. The input date / time information indicates the date / time when the input information was input. The input date / time information includes, for example, information indicating year, month, day, and time. The user ID is identification information of the user who has input the input information. The multi-function device 100 has a mechanism for accepting a login using a user ID and a password. When the input information is input, the user ID of the user who has logged in to the multifunction peripheral 100 is recorded in the input history recording table HT as the user ID of the user who input the input information.

  For example, referring to the entry E1 of the input history record table HT in FIG. 5, the input information “smtp.abc.com” is input via the UI screen D2 by the user with the user ID “userA” at the input date “T1”. You can see that it was entered.

  Further, in the input history recording table HT, when a UI screen for inputting input information to be recorded belongs to a specific group, the input history is recorded via two or more UI screens belonging to the specific group. An input information group including one or more pieces of input information is recorded together. For this reason, in the input history recording table HT, when a UI screen for inputting input information to be recorded belongs to a specific group, a group ID is recorded instead of the screen ID.

  For example, referring to the entry E3 of the input history record table HT of FIG. 5, the input information group “service 1, account 1, ScanToUpload, PDF” is entered by the user with the user ID “userA” on the input date “T3”. It can be seen that the input was made via a screen group (UI screens D4 to D8) belonging to the group G1.

A-2. System Operation A-2-1. Processing at Screen Switching FIG. 6 is a flowchart of processing at screen switching. The process at the time of screen switching is a process of recording the input history information in the input history recording table HT and switching the UI screen displayed on the display unit 140, and is executed by the CPU 110 of the multifunction peripheral 100. The process at the time of screen switching is started at the timing when an event recorded in the screen transition table TT occurs for the UI screen being displayed on the display unit 140.

  In S110, the CPU 110 determines whether the UI screen being displayed is a UI screen in which input information is to be stored. Specifically, in the screen definition table DT (FIG. 3), when information indicating that the input history information is stored is associated with the UI screen being displayed on the display unit 140, the UI screen being displayed is displayed. Is the UI screen on which the input information should be stored.

  If the UI screen being displayed is not a UI screen for storing input information (S110: NO), the CPU 110 advances the processing to S160 without recording the input history information.

  If the UI screen being displayed is a UI screen for storing input information (S110: YES), the CPU 110 determines in S120 whether the UI screen being displayed is a UI screen belonging to a group. to decide. Specifically, in the screen definition table DT (FIG. 3), when the group ID is associated with the displayed UI screen, the displayed UI screen is a UI screen belonging to the group. To be judged. When the group ID is not associated with the displayed UI screen, it is determined that the displayed UI screen is not a UI screen belonging to the group.

  If the UI screen being displayed is not a UI screen belonging to a group (S120: NO), the CPU 110 records the input information to be saved in the input history recording table HT as input history information in S130. Specifically, the input information to be saved is associated with the ID of the UI screen being displayed, information indicating the current date and time (input date and time information), and the ID of the user who is logged in to the MFP 100, Recorded in the input history recording table HT.

  When the UI screen being displayed is a UI screen belonging to a group (S120: YES), the CPU 110 causes the CPU 110 to display the UI screen being displayed among a plurality of UI screens belonging to the group in S140. Whether the UI screen is the last displayed is determined. Specifically, when the UI screen to be displayed next belongs to the same group as the UI screen being displayed, it is determined that the UI screen being displayed is not the UI screen displayed last. When the UI screen to be displayed next does not belong to the same group as the UI screen being displayed, it is determined that the UI screen being displayed is the UI screen displayed last. The UI screen to be displayed next is specified with reference to the screen transition table TT (FIG. 4). It is specified with reference to the screen definition table DT (FIG. 3) whether the UI screen to be displayed next belongs to the same group as the UI screen being displayed.

  If the UI screen being displayed is not the last UI screen to be displayed (S140: NO), the CPU 110 advances the processing to S160 without recording the input history information. The input information to be saved, which is input via the UI screen being displayed, is held in a state temporarily stored in the buffer area of the volatile storage device 120.

When the UI screen being displayed is the UI screen to be displayed last (S140: YES), the CPU 110, in S150, inputs the input information to be saved input via the UI screen being displayed, Input an input information group in which one or more input information to be saved input via one or more other UI screens belonging to the same group as the UI screen being displayed are arranged in the UI screen display order. Recorded in the input history recording table HT as history information. The input information group includes an input history record table in association with the ID of the group to which the UI screen being displayed belongs, information indicating the current date and time (input date and time information), and the ID of the user who is logged in to the MFP 100 Recorded in HT.
Depending on the input contents of the UI screen, the screen may transition to the next screen without inputting input information. In this case, “blank” indicating that there is no input information is recorded as input history information in the input history recording table HT. When this UI screen belongs to a group, “blank” is recorded at the position where the input history information of the corresponding UI screen is recorded in the input information group recorded as the input history information.

  In S160, the CPU 110 displays the UI screen to be displayed next on the display unit 140 instead of the UI screen being displayed, and ends the process at the time of screen switching. The UI screen to be displayed next is specified with reference to the screen transition table TT (FIG. 4). Every time the UI screen displayed on the display unit 140 is switched by the screen switching process described above, the information to be saved among the input information input via the UI screen being displayed is volatile. It is stored in the storage device 120 as input history information.

A-2-2. Input support processing:
The input support process is a process in which the portable terminal 200 and the multifunction peripheral 100 cooperate to support input of input information by the user. The input support process is started when a start instruction from the user is input to the mobile terminal 200 in a state where the computer program 232 for executing the input support process is activated in the mobile terminal 200.

  FIG. 7 is a diagram illustrating an example of a screen (also referred to as a terminal display screen) displayed on the mobile terminal 200. A terminal display screen TD1 in FIG. 7A is a screen displayed when the computer program 232 is activated. The terminal display screen TD1 includes an information IF related to the multifunction peripheral 100 used by the user and a button SBT for inputting an input support processing start instruction. The MFP 100 used by the user is registered in advance by the user. The button SBT includes a message indicating that when the button SBT is pressed, the input history information of the UI screen being displayed on the multi-function peripheral 100 is displayed. When the user presses the button SBT to input an input support process start instruction, the CPU 210 starts the input support process.

  FIG. 8 is a sequence diagram of the input support process of the first embodiment. In S <b> 2, the CPU 210 of the mobile terminal 200 transmits a request for the ID of the UI screen being displayed to the multifunction device 100.

  When the multifunction device 100 receives the request for the ID of the screen being displayed, the CPU 110 of the multifunction device 100 gives the ID of the UI screen currently displayed on the display unit 140 of the multifunction device 100 to the portable terminal 200 in S4. Send.

  When the mobile terminal 200 receives the ID of the UI screen being displayed, the CPU 210 transmits a request for input history information regarding the UI screen being displayed to the multi-function device 100 in S6. This request includes the received screen ID.

  When the multifunction device 100 receives a request for input history information, the CPU 110 executes a search process for searching the input history record table HT in S8.

  FIG. 9 is a flowchart of the search process. In S200, CPU 110 determines a search key used for the search. In the present embodiment, when the UI screen having the screen ID included in the request transmitted from the mobile terminal 200 in S6 is a UI screen that does not belong to the group, the screen ID is determined as a search key. When the UI screen having the screen ID is a UI screen belonging to a group, the ID of the group is determined as a search key.

  In S210, input history information to be transmitted is extracted by searching the input history recording table HT using a search key. As a result, when the search key is the screen ID, the past input information recorded as the input history information in the input history record table HT is input via the UI screen currently displayed on the display unit 140. Input information is extracted. When the search key is a group ID, among the past input information recorded as input history information in the input history record table HT, a plurality of groups belonging to the UI screen currently displayed on the display unit 140 belong to A group of input information input via the UI screen is extracted.

  For example, when the UI screen being displayed is the UI screen D1 in FIG. 2A or the UI screen D6 in FIG. 2F, the input history information is not extracted. This is because the input information input via the UI screen D1 is not included in the input history record table HT because it is not a history storage target. If the UI screen being displayed is the UI screen D2 in FIG. 2B, the four entries E1, E2, E5, and E6 in FIG. 5 are extracted. In addition, when the UI screen being displayed is the UI screen D5 in FIG. 2E or the UI screen D7 in FIG. 2G, the two entries E3 and E4 in FIG. 5 are extracted.

  In S <b> 10 of FIG. 8, the CPU 110 transmits the input history information extracted by the search process to the mobile terminal 200. The input history information to be transmitted includes all the information included in the extracted entry, that is, the screen ID or group ID, and input date / time information. When there is no input history information extracted by the search process, for example, a notification that there is no input history information is transmitted to the mobile terminal 200 instead of the input history information. Therefore, for example, when the UI screen being displayed is the UI screen D1 in FIG. 2A or the UI screen D6 in FIG. 2F, the CPU 110 does not transmit the input history information, but the input history information. A notification that there is no information is transmitted to the mobile terminal 200. The notification that there is no input history information is, for example, an empty response packet that does not include input history information.

  When the portable terminal 200 receives the input history information, in S12, the CPU 210 determines whether or not a group ID is included in the received input history information. That is, it is determined whether the UI screen being displayed on the display unit 140 is a UI screen belonging to a group. Note that the notification that there is no input history information does not include the group ID. Therefore, when the notification is received, it is determined in this step that the group ID is not included.

  When the received input history information does not include a group ID (S12: NO), in S20, the CPU 210 displays the received input history information on the display unit 240 of the mobile terminal 200.

  A terminal display screen TD2 in FIG. 7B is an example of a screen that is displayed when the UI screen being displayed on the display unit 140 of the multifunction peripheral 100 is the UI screen D2 in FIG. The terminal display screen TD2 includes SMTP server address input history information IH1 input via the UI screen D2. The input history information IH1 includes an SMTP server address as input information input in the past, and input date / time information of the SMTP server address.

  If a notification that there is no input history information is received in S10, CPU 210 displays a screen including a message indicating that there is no input history information instead of input history information in S20. Displayed on the unit 240 (not shown).

  If the received input history information includes a group ID (S12: YES), the CPU 210 transmits a request for input information to the multi-function device 100 in S14. Here, the input information is a plurality of UI screens of a group to which the displayed UI screen belongs, and is input information input by the user via the UI screen displayed before the displayed UI screen. is there. For example, as described above, a plurality of UI screens belonging to the same group are continuously displayed on the display unit 140. For example, when the UI screen being displayed is the UI screen D5 (FIG. 2A) belonging to the group G1, the cloud service input via the UI screen D4 displayed immediately before the UI screen D5 is displayed. The type is entered information. When the UI screen being displayed is the UI screen D7 (FIG. 2A) belonging to the group G1, the cloud input via the UI screens D4 and D5 displayed before the UI screen D7. The service type and account are already entered information.

  When the multifunction peripheral 100 receives a request for input information, the CPU 110 transmits the input information to the portable terminal 200 in S16.

  When mobile terminal 200 receives the input information, CPU 110 executes filter processing in S18. The filter process is a process of extracting an input information group including the input information received in S16 from the input information groups of a plurality of entries included in the input history information received in S10.

  For example, the filtering process will be described by taking as an example a case where the UI screen being displayed is the UI screen D5 in FIG. In this example, in S10, the input information group “service 1, account 1, ScanToUpload, PDF” of entry E3 in FIG. 5 and the input information group “service 2, account 3, ScanToUpload, JPEG” of entry E4 are obtained. Received by the mobile terminal 200. In S16, the type of cloud service input via the UI screen D4 is received by the mobile terminal 200 as input information.

  Here, when the input information received in S16 is “service 1”, the input of the entry E3 including “service 1” out of the input information group of the entry E3 and the input information group of the entry E4. Information groups are extracted by filtering. If the input information received in S16 is “service 2”, the input information group of the entry E4 including “service 2” is extracted by the filtering process.

  Next, the filtering process will be described by taking as an example the case where the UI screen being displayed is the UI screen D4 in FIG. In this example, in S10, the input information group “service 1, account 1, ScanToUpload, PDF” of entry E3 in FIG. 5 and the input information group “service 2, account 3, ScanToUpload, JPEG” of entry E4 are obtained. Received by the mobile terminal 200. In S16, the input completion information is not received. This is because the UI screen D4 is a UI screen that is displayed first in the group, and thus there is no input information. In this case, all input information groups received in S10 are extracted as they are.

  After the filtering process, in S20, the CPU 110 displays one or more input information groups extracted by the filtering process on the display unit 240 of the mobile terminal 200.

  The terminal display screen TD3 in FIG. 7C is an example of a screen that is displayed when the UI screen being displayed on the display unit 140 of the multifunction peripheral 100 is the UI screen D4 in FIG. The terminal display screen TD3 includes input history information IH2 including two input information groups of two entries E3 and E4 in FIG.

  In the terminal display screen TD4 in FIG. 7D, the UI screen being displayed on the display unit 140 of the multi-function device 100 is the UI screen D5 in FIG. 2E, and the input has been input via the UI screen D4. It is an example of a screen displayed when the information is “service 1”. The terminal display screen TD4 includes input history information IH3 including the input information group of the entry E3 including “service 1”. Further, in the terminal display screen TD4, the marking MK indicating that the information has been input is added to “service 1” that is the input information in the input history information IH3.

  The user can select the specific input information by touching the specific input information included in the input history information IH1 to IH3 in the terminal display screens TD2 to TD4 with a finger.

  In S22, it is determined whether selection of specific input information by the user has been accepted. If selection of specific input information is not accepted (S22: NO), CPU 110 waits until selection of specific input information is accepted. When selection of specific input information is accepted (S22: YES), the CPU 110 inputs the specific input information to the multi-function device 100 using the mobile terminal 200 in S24. The input information is transmitted to the multifunction device 100.

  When the multi-function device 100 receives specific input information, in S26, the CPU 110 executes processing for acquiring input information. Specifically, CPU 110 determines whether or not the received specific input information is information of a type to be input via the UI screen being displayed on display unit 140. For example, when the same type of information as the received specific input information has already been input to the multi-function peripheral 100 via the UI screen displayed on the display unit 140, the display unit 140 already has the next UI screen. It is displayed. In this case, the received specific input information may not be the type of information that should be input via the UI screen being displayed on the display unit 140.

  If the received specific input information is not the type of information to be input via the UI screen being displayed on the display unit 140, it is determined that the input of the specific input information is an error.

  When the received specific input information is information of a type to be input via the UI screen being displayed on the display unit 140, the received specific input information is input via the UI screen being displayed. Treated as input information. That is, when the received specific input information is specific setting information, the setting based on the setting information is validated in the multifunction peripheral 100. Further, if the received specific input information is input via the UI screen being displayed, the input of the specific input information corresponds to an event for switching the screen displayed on the display unit 140. When doing so, the above-described processing at the time of screen switching (FIG. 6) is performed. Accordingly, the UI screen displayed on the display unit 140 of the multifunction device 100 can be switched when specific input information is received from the mobile terminal 200.

  In S <b> 28, the CPU 110 transmits a completion notification or an error notification to the mobile terminal 200. The completion notification is a notification indicating that the specific input information transmitted from the mobile terminal 200 in S24 has been input to the multifunction device 100. The error notification is a notification indicating that specific input information has not been input to the multifunction device 100.

  In S30, the CPU 210 executes predetermined post-processing according to the completion notification or the error notification. For example, when a completion notification is received, the input support process is initialized after the user is notified that the input information has been input. When a completion notification is received, the input support process is initialized after the user is notified of the occurrence of an error. When the input support process is initialized, the input support process is executed again from S2 in FIG. This is because when the completion notification or error notification is received, the screen displayed on the display unit 140 may be switched from the UI screen displayed at the start of the input support process to another UI screen. Because it is expensive.

A-2-3. Screen transition monitoring processing:
The UI screen displayed on the display unit 140 of the MFP 100 may be switched at a timing when the mobile terminal 200 cannot recognize it. For example, when the user of the mobile terminal 200 inputs input information via the UI screen being displayed on the display unit 140 without using the mobile terminal 200, the UI screen displayed on the display unit 140 is switched. obtain. In addition, the UI screen displayed on the display unit 140 can be switched when an event such as an error or an event such as reception of a print job or fax data occurs. As a result, there is a possibility that the contents of the terminal display screens TD <b> 2 to TD <b> 4 displayed on the display unit 240 of the mobile terminal 200 and the contents of the UI screen displayed on the display unit 140 of the multi-function device 100 will not match. In order to suppress such inconvenience, in this embodiment, the CPU 210 of the mobile terminal 200 executes a screen transition monitoring process that is a process independent of the input support process in parallel with the input support process described above. .

  FIG. 10 is a flowchart of the screen transition monitoring process. In S <b> 300, the CPU 210 inquires the MFP 100 about the UI screen displayed on the display unit 140 of the MFP 100. Specifically, as in S <b> 2 of FIG. 8, a request for the ID of the UI screen being displayed is transmitted from the portable terminal 200 to the multifunction device 100. The CPU 210 of the multifunction peripheral 100 that has received the request transmits the ID of the UI screen being displayed on the display unit 140 to the portable terminal 200 as a response to the response. As a result, in S <b> 310, the CPU 210 acquires the ID of the UI screen being displayed on the multifunction peripheral 100.

  In S <b> 320, the CPU 210 determines whether the UI screen displayed on the display unit 140 has been switched in the multifunction device 100 based on the screen ID acquired from the multifunction device 100. That is, if the screen ID acquired in S310 of this time is different from the screen ID acquired in S310 of the previous time, it is determined that the UI screen has been switched. If the screen ID acquired in S310 of this time is the same as the screen ID acquired in S310 of the previous time, it is determined that the UI screen has not been switched.

  If it is determined that the UI screen has been switched (S320: YES), the CPU 210 initializes the input support process in S330. As a result, the input support process of FIG. 8 is executed from the beginning.

  When it is determined that the UI screen has not been switched (S320: NO), in S330, the CPU 210 determines whether a predetermined time has elapsed. When the predetermined time has not elapsed (S330: NO), the CPU 210 waits until the predetermined time elapses, and when the predetermined time has elapsed (S330: YES), the CPU 210 returns to S300.

  In the monitoring process described above, it is detected that the UI screen displayed on the multifunction peripheral 100 has been switched (S300 to 320). When it is detected that the UI screen has been switched (S320: YES), the input support process is initialized (S330). Therefore, when the UI screen displayed on the multifunction peripheral 100 is switched, the CPU 210 appropriately displays the screen displayed on the display unit 240 of the mobile terminal 200 (for example, the terminal display screens TD1 to TD4 in FIG. 7). You can switch to

  According to the first embodiment described above, the CPU 110 of the multi-function peripheral 100 receives the first type of input information (for example, the cloud to be used) via the first screen (for example, the UI screen D4 in FIG. 2D). Service type) is acquired from the user, and second type input information (for example, SMTP server address) is acquired from the user via the second screen (for example, UI screen D2 in FIG. 2B). it can. Then, one or more first type input information associated with the first screen and one or more second type associated with the second screen are obtained by the screen switching process of FIG. The input history information including the input information is stored in the memory of the multi-function peripheral 100, that is, the nonvolatile storage device 130 (S130 and S150 in FIG. 6). In the input support process of FIG. 8, the CPU 110 transmits the stored input history information to the mobile terminal 200 in order to display it on the display unit 240 of the mobile terminal 200 (S <b> 10 of FIG. 8).

  Specifically, in response to a request for input history information from the multifunction peripheral 100 (S6 in FIG. 8), a search process (S8 in FIG. 8, FIG. 9) is executed, so that it is being displayed on the display unit 140. Only the input history information associated with the UI screen ID is extracted from the input history recording table HT. Then, the extracted input history information is transmitted to the mobile terminal 200. Therefore, when the first screen is displayed on the display unit 140, one or more first type input information of the input history information is transmitted to the mobile terminal 200, and one or more second type is input. Is not transmitted to the mobile terminal 200. And when the 2nd screen is displayed on the display part 140, one or more 2nd type input information is transmitted to the portable terminal 200 among input history information, and 1 or more 1st type | molds are input. One type of input information is not transmitted to the mobile terminal 200. As a result, according to the UI screen displayed on the display unit 140, only necessary types of input information are displayed on the mobile terminal 200 (FIG. 7), so that the multifunction peripheral 100 uses the mobile terminal 200. It is possible to appropriately support information input by the user. As a result, the convenience for the user of the multifunction device 100 can be improved. For example, since the input history information is automatically recorded in the input history recording table HT, the user does not need to remember the past input history information, and performs an operation for saving the past input history information. No need to do for 100. Further, since the input history information related to the UI screen being displayed is selectively displayed on the display unit 240 of the mobile terminal 200, the user refers to only useful information and easily inputs the input information to the multifunction peripheral 100. be able to.

  Further, according to the first embodiment, the CPU 110 acquires, from the portable terminal 200, input information selected by the user from one or more pieces of input information transmitted to the portable terminal 200 (S24 in FIG. 8). As a result, the user can input the input information via the UI screen displayed on the display unit 140 by operating the operation unit 150 of the multi-function device 100, and the input information can be input via the portable terminal 200. 100 can be entered. Therefore, the convenience for the user of the multifunction device 100 can be improved.

  Further, in the input support process of the first embodiment, when the input information selected by the user is acquired from the portable terminal 200 (S24 in FIG. 8), the CPU 110 executes the process (S26) at the time of acquiring the input information. The In the processing at the time of acquiring the input information, the acquired information is handled as input information input via the UI screen being displayed. As a result, when the input information selected by the user is acquired from the mobile terminal 200, the above-described processing at the time of screen switching (FIG. 6) can be performed. In other words, when the input information selected by the user is acquired from the terminal device during display of the first screen, the CPU 110 changes the UI screen displayed on the display unit 140 from the first screen to the first screen. The screen can be switched to a different screen from the one screen. As a result, the convenience for the user of the multifunction device 100 can be further improved.

  Further, the CPU 110 inputs third type input information (for example, an account for using the cloud service) related to the first type input information (for example, the type of cloud service to be used). A screen (for example, the UI screen D5 in FIG. 2E) can be displayed on the display unit 140. That is, the CPU 110 can acquire the third type of input information that is input via the third screen. Then, in the process at the time of screen switching in FIG. 6, the CPU 110 associates the input information group with the group ID indicating the screen group including the first screen and the third screen (that is, the UI screen group). Store in the non-volatile storage device 130. The input information group associated with the group ID includes first type input information and third type input information. The input information group associated with the group ID is, for example, the input information group of entries E3 and E4 in FIG. In the input support process of FIG. 8, when the first screen is displayed on the display unit 140, the CPU 110 extracts these input information groups from the input history information by the search process (S8 of FIG. 8). Is transmitted to the portable terminal 200 (S10 in FIG. 8). As a result, the multifunction device 100 can cause the mobile terminal 200 to display an input information group including a plurality of input information related to each other on the mobile terminal 200 (FIGS. 7C and 7D). ). For example, in the examples of FIG. 7C and FIG. 7D, an input information group including a plurality of input information necessary for causing the multifunction peripheral 100 to execute the function of using the cloud service is the mobile terminal 200. It is displayed on the display unit 240. Therefore, the multi-function device 100 can appropriately support the input of a plurality of pieces of information related to each other by the user using the mobile terminal 200.

  Note that the mobile terminal 200 operates as follows in accordance with the above-described processing on the multifunction peripheral 100 side. That is, when the first screen (for example, the UI screen D5 in FIG. 2E) is displayed on the multifunction peripheral 100, the CPU 210 of the mobile terminal 200 displays the first type of input information and the third type of input information. Input history information including a plurality of input information groups each including input information (for example, two input information groups of entries E3 and E4 in FIG. 5) is acquired (S10 in FIG. 8), and a plurality of inputs are acquired. A screen including the information group (for example, the terminal display screen TD3 in FIG. 7) is displayed on the display unit 240 (S20 in FIG. 8). After that, when specific first type input information (for example, “service 1”) selected by the user is input to the multi-function device 100, the CPU 210 includes a plurality of input information groups. Input information group including the specific first type input information (for example, the input information group of entry E3 including “service 1”) is displayed on the display unit 240, and does not include the specific first type input information. The group (for example, the input information group of the entry E4 not including “service 1”) is not displayed on the display unit 240. Specifically, when the input information of “service 1” is input to the multi-function device 100 by the user, the screen displayed on the display unit 240 is changed from the terminal display screen TD3 in FIG. The terminal display screen TD4 is switched to (D). As a result, only an appropriate input information group is displayed on the display unit 240 according to the input information that has been input. Accordingly, the mobile terminal 200 can appropriately display an input information group that is highly likely to be input by the user, and thus can more appropriately support the input of a plurality of information by the user.

  Further, in the screen definition table DT of FIG. 3, information indicating whether or not to save the input history information is associated with each UI screen. In other words, in addition to the UI screen where the input history information should be saved, the CPU 110 has a fourth screen where the input history information should not be saved (for example, the UI screen D1 in FIG. 2A or FIG. 2F). The UI screen D6) can be displayed on the display unit 140. Then, in the input support process, when the fourth screen is displayed on the display unit 140, the CPU 110 does not transmit the input history information to the portable terminal 200 in S10 of FIG. As described above, for example, the UI screen D1 in FIG. 2A and the UI screen D6 in FIG. 2F are considered inappropriate for transmitting input history information from the viewpoint of necessity and confidentiality. It is done. According to the first example, transmission of inappropriate input history information to the mobile terminal 200 can be suppressed. As a result, the multi-function device 100 can more appropriately support the input of a plurality of information by the user, and can suppress leakage of highly confidential information, for example.

  In the input support process of the first embodiment, the CPU 210 of the mobile terminal 200 displays a screen ID (screen identification information) indicating the first screen when the first screen is displayed on the multifunction device 100. When the second screen is displayed on the multifunction device 100, the screen ID indicating the second screen is obtained from the multifunction device 100 (S2 and S4 in FIG. 8). Then, when the CPU 210 obtains the screen ID indicating the first screen, the CPU 210 uses the screen ID indicating the first screen to input including the first type input information input via the first screen. When the history information is acquired and the screen ID indicating the first screen is acquired, the second type of input information input via the second screen is included using the screen ID indicating the second screen. Input history information is acquired (S6, S10 in FIG. 8). When the CPU 210 acquires the screen ID indicating the first screen, the CPU 210 displays the first type of input information on the display unit 240, and when the CPU 210 acquires the screen ID indicating the second screen, Is displayed on the display unit 240 (S20 in FIG. 8, FIG. 7B, FIG. 7C). As a result, among the input history information, one or more pieces of input information input via the UI screen being displayed on the multifunction peripheral 100 are displayed on the display unit 240 of the mobile terminal 200. As a result, the mobile terminal 200 can support the input of information to the multifunction peripheral 100 by the user.

  Further, in the input support process of the first embodiment, the CPU 210 transmits specific input information to the multi-function device 100 based on the input information displayed on the display unit 240 based on a user instruction. Specifically, the CPU 210 accepts selection of specific input information from a plurality of pieces of input information displayed on the display unit 240 (S22 in FIG. 8), and receives the specific input information selected by the user from the multifunction peripheral 100. (S24 in FIG. 8). As a result, the user can easily input specific input information to the multifunction peripheral 100 via the portable terminal 200. As a result, the mobile terminal 200 can improve the convenience for the user of the multifunction peripheral 100.

  Furthermore, according to the screen transition monitoring process (FIG. 10) of the first embodiment, the CPU 210 detects that the UI screen displayed on the multifunction peripheral 100 has been switched (S300 to 320 in FIG. 10). Then, when it is detected that the UI screen has been switched (S320 in FIG. 10: YES), the CPU 210 initializes the input support process (S330 in FIG. 10). Therefore, when the UI screen displayed on the multifunction device 100 is switched, the screen displayed on the display unit 240 of the mobile terminal 200 (for example, the terminal display screens TD1 to TD4 in FIG. 7) is automatically switched. It is done. In other words, the screen displayed on the display unit 240 of the mobile terminal 200 when the switching from a specific UI screen to another UI screen is detected for the UI screen displayed on the multifunction peripheral 100 is specified. It is possible to switch from a screen that displays the history of input information input via the UI screen to a screen that displays the history of input information input via the specific UI screen. As a result, the mobile terminal 200 can appropriately support the input of information to the multifunction device 100 by the user according to the UI screen displayed on the display unit 140 of the multifunction device 100.

  In the screen transition monitoring process (FIG. 10), the MFP 100 transmits the ID of the screen being displayed on the display unit 140 to the mobile terminal 200 in response to an inquiry from the mobile terminal 200 (FIG. 10). S310). Accordingly, the mobile terminal 200 can detect switching of the screen displayed on the display unit 140. In other words, when the screen displayed on the display unit 140 is switched, the CPU 210 of the multifunction peripheral 100 transmits information for the mobile terminal 200 to detect the switching of the screen to the mobile terminal 200. . As a result, the mobile terminal 200 can appropriately detect that the screen displayed on the display unit 140 of the multifunction device 100 has been switched. Therefore, the MFP 100 can more appropriately support the input of information by the user according to the screen displayed on the display unit 140 using the mobile terminal 200.

B. Second embodiment:
B-1. Configuration In addition to the system 900 of the first embodiment, the system of the second embodiment includes a server 300 connected to a network 500 such as the Internet, as indicated by a broken line in FIG. In the second embodiment, the access point 50 is connected to the server 300. For this reason, in the system of the second embodiment, the multifunction peripheral 100 and the portable terminal 200 are communicably connected to the server 300 via the access point 50 and the network 500.

  The server 300 includes a CPU 310 as a controller that controls the entire server 300, a volatile storage device 320 such as a DRAM, a non-volatile storage device 330 such as a hard disk drive, and a communication interface 380 for communicating with other devices. And.

  The volatile storage device 320 is used as a buffer area for temporarily storing various intermediate data generated when the CPU 310 performs processing, for example. The nonvolatile storage device 330 stores a computer program 332 and a management table group 334. By executing the computer program 332, the CPU 310 can execute input support processing described later in cooperation with the multifunction peripheral 100 and the portable terminal 200. The computer program 332 is provided, for example, in a form stored in a DVD-ROM or the like, or uploaded from another computer via the network 500.

  The management table group 334 stored in the nonvolatile storage device 330 of the server 300 includes a screen definition table DT (FIG. 3) and an input history recording table HT (FIG. 5). The server 300 can execute input support processing in cooperation with a plurality of image processing apparatuses (for example, a multifunction peripheral and a printer) including the multifunction peripheral 100. For this purpose, the non-volatile storage device 330 stores one input history recording table HT for each image processing device in association with a device ID for identifying the image processing device. The non-volatile storage device 330 stores one screen definition table DT for each model in association with the model ID for identifying the model of the image processing apparatus. The management table group 134 stored in the non-volatile storage device 130 of the multifunction peripheral 100 according to the second embodiment does not include the input history recording table HT, but includes only the screen definition table DT and the screen transition table TT.

B-2. System Operation B-2-1. Processing at Screen Switching The processing at the screen switching in the second embodiment is almost the same as the processing at the screen switching in the first embodiment in FIG. However, in S <b> 130 of FIG. 6, the CPU 110 of the multifunction device 100 transmits the input information to be stored in association with the device ID of the multifunction device 100 to the server 300. 6, the CPU 110 of the multifunction peripheral 100 transmits the input information group to be stored in association with the device ID of the multifunction peripheral 100 to the server 300. When the server 300 receives the input information or the input information group to be saved, the CPU 310 of the server 300 stores the input history recording table HT associated with the device ID of the multifunction device 100 in the nonvolatile storage device 330. The management table group 334 is selected. Then, the CPU 310 records the input information and the input information group to be saved in the selected input history recording table HT. As a result, in the second embodiment, the input history information of the multifunction peripheral 100 is stored in the nonvolatile storage device 330 of the server 300.

B-2-1. Processing at Screen Switching FIG. 11 is a sequence diagram of input support processing according to the second embodiment. In the input support process of the second embodiment, S6b, S8b, and S10b of FIG. 11 are executed instead of S6, S8, and S10 of the input support process (FIG. 8) of the first embodiment. The other steps of the input support process of the second embodiment are the same as the steps with the same symbols in the input support process (FIG. 8) of the first embodiment.

  In S <b> 6 b, the CPU 210 of the mobile terminal 200 transmits a request for input history information for the UI screen being displayed on the multifunction peripheral 100 to the server 300. This request includes the device ID of the multifunction device 100 in addition to the screen ID received from the multifunction device 100. A device ID is stored in advance in the nonvolatile storage device 230 of the portable terminal 200. For example, when the computer program 232 is activated for the first time, for example, the CPU 210 acquires a device ID from the multifunction peripheral 100 in advance and stores it in the nonvolatile storage device 230.

  In S <b> 8 b, the CPU 310 of the server 300 that has received the request for input history information executes the search process of FIG. 9 using the input history recording table HT associated with the device ID of the multifunction device 100. As a result, the input history information to be transmitted to the mobile terminal 200 is extracted from the input history recording table HT associated with the device ID of the multifunction device 100.

  In S <b> 10 b, the CPU 310 transmits the input history information extracted by the search process to the mobile terminal 200.

  As can be seen from the above description, in the input support process of the second embodiment, the CPU 210 of the mobile terminal 200 transmits the ID of the first screen (for example, the UI screen D4 in FIG. 2D) to the server 300. As a result, the input history information including the first type of input information (for example, the type of cloud service to be used) input via the first screen is acquired from the server 300. Further, the CPU 210 transmits the ID of the second screen (for example, the UI screen D2 in FIG. 2B) to the server 300, whereby the second type of input information input via the second screen. Input history information including (for example, SMTP server address) is acquired from the server 300. As a result, the mobile terminal 200 can acquire appropriate input history information from the server 300 using the screen ID acquired from the multifunction device 100. Therefore, it is not necessary to store the input history information in the non-volatile storage device 130 of the multi-function device 100. Therefore, even when the capacity of the non-volatile storage device 130 of the multi-function device 100 is relatively small, the rich input history information Can be saved. Therefore, even when the capacity of the non-volatile storage device 130 of the multifunction peripheral 100 is relatively small, it is possible to sufficiently assist the user in inputting information.

C. Variations:
(1) In the search process of the first embodiment (FIG. 9), the screen ID transmitted from the mobile terminal 200 is used as a search key. Instead, both the screen ID and the user ID may be used as search keys. In this case, as the user ID used as the search key, the ID of the user who has logged in to the multifunction peripheral 100 at the time when the search process is executed is used. In this case, of the input history information recorded in the input history recording table HT, only the input information associated with both the ID of the UI screen being displayed and the ID of the logged-in user is stored in the mobile phone. It is extracted as input history information to be transmitted to the terminal 200.

  In other words, when the first screen (for example, the UI screen D4 in FIG. 2D) is displayed on the display unit 140 in S10 of the input support process in FIG. 8, it is included in the input history information. Among the one or more types of first type input information (for example, the type of cloud service to be used), the first type of input information associated with the first user ID (for example, userA in FIG. 5) is the mobile terminal. 200. And the 1st type input information matched with 2nd user ID (for example, userB of FIG. 5) is not transmitted to the portable terminal 200. FIG. For example, in a specific type of input information (for example, an account), for example, information input by the first user and information input by the second user are different from each other, but input by the first user. The information may be the same over multiple input opportunities. In such a case, it is considered that the input history information of the second user is not very useful for the first user, while the input history information of the second user is useful. Therefore, according to the present modification, useful input history information can be appropriately displayed on the mobile terminal 200 for each user, so that the user can appropriately support the input of information.

(2) In the search process of the first embodiment (FIG. 9), when the screen ID transmitted from the mobile terminal 200 belongs to a group, the group ID for identifying the group is used as a search key. Instead of this, both the group ID and the user designation information may be used as search keys. The user designation information is specific input information designated by the user. The user designation information is, for example, input information, that is, a plurality of UI screens of a group to which the displayed UI screen belongs, and is input by the user via the UI screen displayed before the displayed UI screen. May be input information. Or the input information which the user input into the portable terminal 200 may be sufficient. Since the input information has already been input to the multi-function device 100 when the search process is performed, the CPU 110 of the multi-function device 100 has already acquired the input information when the search process is performed. The input information input to the mobile terminal 200 by the user is transmitted from the mobile terminal 200 to the multifunction device 100 together with the screen ID, for example, in S6 of FIG.

  In this case, in this case, in the search processing of S8, the input history information recorded in the input history recording table HT is associated with the ID of the group to which the UI screen being displayed belongs, and the user Only the input information group including the designation information is extracted as input history information to be transmitted to the mobile terminal 200 in S10. In this case, since it is not necessary to perform the filter process in the portable terminal 200, the processes of S12 to S18 are omitted in FIG. In S20, CPU 110 of portable terminal 200 displays a screen including received input history information as it is on display unit 240.

  In other words, in this modification, the CPU 110 of the multi-function device 100 causes the plurality of input information groups (for example, two entries E3 and E4 in FIG. 5 to be associated with the same group ID in S10 in FIG. 8). Among the input information group), an input information group (for example, the input information group of the entry E3) including the user specified information (for example, “service 1”) is transmitted to the mobile terminal 200, and the input information group not including the user specified information (For example, the input information group of the entry E4) is not transmitted to the mobile terminal 200. As a result, since only the input information group including the user designation information is transmitted to the mobile terminal 200, only the input information group that is more useful to the user is selectively transmitted to the mobile terminal 200. 200 can more appropriately support the input of a plurality of information by the user. In addition, since it is not necessary to perform the filtering process in the mobile terminal 200, it is possible to reduce the processing added by the CPU 210 of the mobile terminal 200.

(3) In the first embodiment, transmission of input information from the portable terminal 200 to the multifunction peripheral 100 (S24 in FIG. 8), that is, input of input information using the portable terminal 200 is performed in units of input information. ing. Instead of this, transmission of input information from the portable terminal 200 to the multifunction peripheral 100 may be performed in units of input information groups belonging to the same group. In this case, when the password is included in the input information group belonging to the same group, it is preferable that the password is not transmitted from the multifunction device 100 to the portable terminal 200 as the input history information. This is because the password is highly confidential information. In this case, password input history information is not displayed on the mobile terminal 200. For this purpose, for example, a plurality of input information groups excluding passwords are displayed as input history information on the mobile terminal 200, and an input information group selected from the plurality of input information groups is transferred from the mobile terminal 200 to the multifunction peripheral. 100. In this case, the password is input via the UI screen displayed on the display unit 140. For example, a case where input information is transmitted from the portable terminal 200 to the multifunction peripheral 100 in units of the input information group of the group G1 according to the embodiment will be described as an example. After the UI screen D4 to be displayed first among the plurality of UI screens belonging to the group G1, the input information group excluding the password is transmitted from the portable terminal 200 to the multi-function device 100. As a result, the CPU 110 determines that the input information group excluding the password has been input to the multi-function peripheral 100, and displays the UI screen displayed on the display unit 140 of the multi-function peripheral 100 from the UI screen D4 in FIG. The screen is switched to the screen D6 in FIG. Then, after the password is input via the UI screen D6, the CPU 110 may switch the UI screen displayed on the display unit 140 from the UI screen D6 to the UI screen D9 in FIG. In other words, when the input information group of the group G1 is transmitted from the mobile terminal 200 to the multifunction peripheral 100, the transmitted input among the UI screens D5 to D8 included in the group G1 to be displayed thereafter. A UI screen D6 for inputting input information (for example, a password) not included in the information group is displayed on the display unit 140, and a UI screen for inputting the input information included in the transmitted input information group D5, D7, and D8 may not be displayed on the display unit 140.

  Alternatively, the CPU 210 of the mobile terminal 200 may generate an input information group including the password by adding a password input by the user to the input information group selected by the user. Then, the input information group modified so as to include the password may be transmitted to the multifunction device 100. In this case, the CPU 110 determines that the input information group including the password has been input to the multi-function peripheral 100, and changes the UI screen displayed on the display unit 140 from the UI screen D4 to the UI screen D9 in FIG. You may switch.

(4) In the first embodiment, input information (S24 in FIG. 8) transmitted from the portable terminal 200 to the multifunction peripheral 100, that is, input information input using the portable terminal 200 is input to the portable terminal 200. It is selected from a plurality of input information displayed as history information. Instead, for example, when the desired input information is not included in the plurality of input information displayed as the input history information on the mobile terminal 200, the specific information of the displayed plurality of input information is not included. The input information may be modifiable by the user. Then, the specific input information corrected by the user may be transmitted from the portable terminal 200 to the multi-function device 100, so that the corrected specific input information may be input to the multi-function device 100.

(5) In the input history recording table HT, when there are a plurality of entries in which duplicate input information (or input information group) is recorded, only the last recorded entry among the plurality of entries is stored. The other entries may be deleted from the input history recording table HT. In this way, since the input history information displayed on the display unit 240 of the mobile terminal 200 does not include a duplicated input history, it is easy to specify input information desired by the user.

(6) In the first embodiment, the screen ID for identifying each UI screen, the group ID for identifying the group to which a plurality of UI screens belong, and the information indicating whether or not to save the input history information are: Although it is separate information, one ID obtained by integrating these pieces of information may be used as identification information for each UI screen. For example, the ID of the UI screen D1 may be “D1G0R0”, and the ID of the UI screen D4 may be “D4G1R1”. In this example, the first two digits of the ID are IDs for identifying the UI screen, the third digit and the fourth digit are group IDs, and the last two digits are whether to save the input history information. It is information which shows. For example, if the third and fourth digits are “G0”, the UI screen to which the ID is assigned indicates that the UI screen does not belong to the group, and the third and fourth digits are “G1”. In some cases, the UI screen to which the ID is assigned indicates that it belongs to the group G1. Further, when the last two digits are “R0”, it means that the input history information is not stored for the UI screen to which the ID is assigned, and when the last two digits are “R0”. This means that the input history information is stored for the UI screen to which the ID is assigned. When such an ID is adopted, the screen definition table DT is not necessary.

(7) The multifunction peripheral 100 of each of the above embodiments may be another type of image processing apparatus, for example, a digital camera, a scanner, or a printer. The server 300 may include a plurality of devices (for example, computers) that can communicate with each other via a network, and the plurality of devices may function as one server.

(8) In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, part or all of the configuration realized by software is replaced with hardware. You may do it.

  As mentioned above, although this invention was demonstrated based on the Example and the modification, Embodiment mentioned above is for making an understanding of this invention easy, and does not limit this invention. The present invention can be changed and improved without departing from the spirit and scope of the claims, and equivalents thereof are included in the present invention.

  50 ... access point, 100 ... multifunction device, 110 ... CPU, 120 ... volatile storage device, 130 ... nonvolatile storage device, 132 ... computer program, 134 ... management Table group 136 ... Screen data group 140 ... Display unit 150 ... Operation unit 160 ... Reading execution unit 170 ... Print execution unit 180 ... Communication interface 200 .. mobile terminal, 210 ... CPU, 220 ... volatile storage device, 230 ... nonvolatile storage device, 232 ... computer program, 240 ... display unit, 250 ... operation unit, 280 ... communication interface, 300 ... server, 310 ... CPU, 320 ... volatile storage device, 330 ... nonvolatile storage device, 332 ... computer program, 334 ... management table Group, 380 ... communication interface, 500 ... network, 900 ... system, T ... screen definition table, TT ... screen transition table, HT ... input history record table

Claims (15)

An image processing apparatus that is communicably connected to a terminal device,
A display unit;
A display control unit for displaying one screen selected from a plurality of screens including a first screen and a second screen on the display unit;
An input acquisition unit that acquires input information input via a screen displayed on the display unit, wherein the input information includes first type input information input via the first screen; And the second acquisition information input via the second screen, the input acquisition unit,
A history storage unit that stores, in a memory, input history information related to input information acquired by the input acquisition unit, wherein the input history information is associated with the first screen. The history storage unit, including: kind of input information; and one or more pieces of the second kind of input information associated with the second screen;
In order to display the input history information on the display unit of the terminal device, the input control information is transmitted to the terminal device, and the input history is displayed when the first screen is displayed on the display unit. Among the information, one or more types of first type input information are transmitted to the terminal device, and one or more types of second type input information are not transmitted to the terminal device, and the display unit is When the second screen is displayed, one or more of the second type of input information among the input history information is transmitted to the terminal device, and one or more of the first type of input is transmitted. The transmission control unit not transmitting information to the terminal device; and
An image processing apparatus comprising: The image reading apparatus according to claim 1,
The image acquisition apparatus, wherein the input acquisition unit further acquires one type of the input information based on the one or more types of the first type of input information already transmitted to the terminal device. The image processing apparatus according to claim 2,
The display control unit, when the first type of input information is acquired from the terminal device during the display of the first screen, the screen displayed on the display unit from the first screen, An image processing apparatus that switches to a screen different from the first screen. The image processing apparatus according to any one of claims 1 to 3,
The plurality of screens further includes a third screen,
The input acquisition unit further acquires the third type of input information related to the first type of input information, which is the third type of input information input via the third screen,
The history generation unit includes one or more pieces of the first type input information and the third type input information associated with a screen group including the first screen and the third screen, respectively. Storing the input history information including the input information group in a memory;
The transmission control unit is an image processing device that transmits one or more input information groups of the input history information to the terminal device when the first screen is displayed on the display unit. The image processing apparatus according to claim 4,
The input history information includes a plurality of the input information groups,
The transmission control unit transmits the input information group including the first type of input information specified by a user among the plurality of input information groups to the terminal device, and specified by the user An image processing device that does not transmit the input information group not including the first type of input information to the terminal device. An image processing apparatus according to any one of claims 1 to 5,
The plurality of screens further includes a fourth screen,
The transmission control unit is an image processing device that does not transmit the input history information when the fourth screen is displayed on the display unit. The image processing apparatus according to claim 1,
The history storage unit stores, in a memory, the input history information including one or more of the first type input information associated with user identification information for identifying a user,
The transmission control unit, when the first screen is displayed on the display unit, of the one or more types of the first type input information included in the input history information, first user identification information Image processing that transmits the first type of input information associated with the terminal device to the terminal device and does not transmit the first type of input information associated with the second user identification information to the terminal device apparatus. The image processing apparatus according to claim 1,
The transmission control unit further includes information for detecting screen switching in the display unit when the screen displayed on the display unit is switched from the first screen to the second screen. An image processing device that transmits to the terminal device. A terminal device communicably connected to an image processing device that displays one screen selected from a plurality of screens including a first screen and a second screen;
A display unit;
When the first screen is displayed on the image processing apparatus, first screen identification information indicating the first screen is acquired from the image processing apparatus, and the second screen is acquired by the image processing apparatus. A screen identification information acquisition unit that acquires second screen identification information indicating the second screen from the image processing device when the screen is displayed,
Input history information including one or more types of first type input information that is input via the first screen using the first screen identification information when the first screen identification information is acquired. When the second screen identification information is acquired, one or more second type input information input via the second screen is obtained using the second screen identification information. A history acquisition unit for acquiring input history information including;
When the first screen identification information is acquired, one or more of the first type of input information is displayed on the display unit, and when the second screen identification information is acquired, the one or more of the first type of information is displayed. A display control unit for displaying the second type of input information on the display unit;
A terminal device. The terminal device according to claim 9,
The terminal device is communicably connected to a server that stores input history information including input information input via each screen in association with each of the plurality of screens displayed on the image processing device. And
The history acquisition unit acquires the input history information including one or more types of input information of the first type from the server by transmitting the first screen identification information to the server, and the second A terminal device that acquires the input history information including one or more of the second type of input information from the server by transmitting screen identification information to the server. The terminal device according to claim 9 or 10, further comprising:
An image including a transmission unit configured to transmit specific first type input information based on one or more of the first type input information displayed on the display unit to the image processing device based on a user instruction; Processing equipment. A terminal device according to any one of claims 9 to 11, further comprising:
A screen that is displayed on the image processing apparatus includes a detection unit that detects that the screen has been switched from the first screen to the second screen;
When the switching from the first screen to the second screen is detected, the display control unit displays one or more pieces of the first type input information on the screen displayed on the display unit. A terminal device that switches from a screen to a screen that displays one or more pieces of the second type of input information. The terminal device according to any one of claims 9 to 12,
The plurality of screens displayed on the image processing apparatus further includes a third screen,
When the first screen identification information is acquired, the history acquisition unit is input via the first type of input information and the third screen using the first screen identification information. Obtaining the input history information including a plurality of input information groups each including third type input information;
The display control unit
When the first screen identification information is acquired, a plurality of the input information groups are displayed on the display unit,
After the plurality of input information groups are displayed on the display unit, when the first type of input information is input to the image processing apparatus by the user, among the plurality of input information groups, An image processing apparatus that displays the input information group including the specific first type input information on the display unit and does not display the input information group that does not include the specific first type input information on the display unit. . A computer program for an image processing apparatus that is communicably connected to a terminal device and includes a display unit,
A display control function for displaying on the display unit one screen selected from a plurality of screens including a first screen and a second screen;
An input acquisition function for acquiring input information input via a screen displayed on the display unit, wherein the input information includes first type input information input via the first screen; Including the second type of input information input via the second screen, and the input acquisition function.
A history storage function for storing input history information related to input information acquired by the input acquisition function in a memory, wherein the input history information is stored in the one or more first screens associated with the first screen. The history storage function, including: seed input information; and one or more second type input information associated with the second screen;
In order to display the input history information on the display unit of the terminal device, a transmission control function for transmitting to the terminal device, wherein the input history is displayed when the first screen is displayed on the display unit. Among the information, one or more types of first type input information are transmitted to the terminal device, and one or more types of second type input information are not transmitted to the terminal device, and the display unit is When the second screen is displayed, one or more of the second type of input information among the input history information is transmitted to the terminal device, and one or more of the first type of input is transmitted. The transmission control function not transmitting information to the terminal device; and
A computer program that causes a computer mounted on the image processing apparatus to realize the above. A computer program for a terminal device that is communicably connected to an image processing device that displays one screen selected from a plurality of screens including a first screen and a second screen, and includes a display unit Because
When the first screen is displayed on the image processing apparatus, first screen identification information indicating the first screen is acquired from the image processing apparatus, and the second screen is acquired by the image processing apparatus. A screen identification information acquisition function for acquiring second screen identification information indicating the second screen from the image processing device when the screen is displayed,
Input history information including one or more types of first type input information that is input via the first screen using the first screen identification information when the first screen identification information is acquired. When the second screen identification information is acquired, one or more second type input information input via the second screen is obtained using the second screen identification information. A history acquisition function to acquire input history information including,
When the first screen identification information is acquired, one or more of the first type of input information is displayed on the display unit, and when the second screen identification information is acquired, the one or more of the first type of information is displayed. A display control function for displaying the second type of input information on the display unit;
A computer program for causing a computer mounted on the terminal device to realize the above.

Images