JP2012104036A - Image processing system, control method of image processing apparatus, portable terminal, information processing apparatus and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system in which an image processing apparatus can be easily operated using a portable terminal device.SOLUTION: By photographing a MFP (Multi Function Peripheral) in a portable terminal having a camera function, a present location and a photography direction are transmitted from the portable terminal to a server, corresponding MFPs 100A and 100B are extracted from information containing respective locations of the MFPs registered in the server, and information on these MFPs is transmitted to the portable terminal. The information contains information on an operation screen of the MFP and display area information specifying an area where the MFP to display the operation screen is present. The portable terminal uses, of the received MFP information, MFP information in which the portable terminal belongs to the range specified by the display area information, to combine the operation screen for operating the MFP to a photographic image and displays a result of the combination.

Description

  The present invention relates to an image processing system, a method for controlling an image processing apparatus, a portable terminal, an information processing apparatus, and a control program, and more particularly to an image processing system for operating an image processing apparatus on a portable terminal, a method for controlling an image processing apparatus, and a portable The present invention relates to a terminal, an information processing apparatus, and a control program.

  Image processing apparatuses such as copiers, printers, and MFPs (Multi Function Peripherals) that are multifunction machines are often used in an office environment, and many users have an opportunity to use them.

  On the other hand, with the recent spread of mobile terminals, each user is carrying a portable device such as a mobile phone having a function of connecting to the Internet, a camera function, and a position detection function, and is accustomed to using it. It is. Then, as disclosed in, for example, Japanese Patent Application Laid-Open No. 2006-351024 (Patent Document 1) and Japanese Patent Application Laid-Open No. 2006-91390 (Patent Document 2), information stored in association with position information in advance is stored. A portable information terminal capable of browsing information such as facility usage status and reservation information on a photographed image by acquiring the information from the stored server and combining and displaying the photographed image and the information. is there.

JP 2006-351024 A JP 2006-91390 A

  With the recent increase in functionality of such image processing apparatuses, operations for the user to issue operation instructions have become complicated. For this reason, there are problems that a user who is not frequently used or a user who uses a plurality of types of devices has difficulty in understanding the usable functions or that the operation is complicated and difficult to use.

  Therefore, in view of the widespread use of portable terminals as described above, it may be desired to use a portable terminal familiar with the use of the image processing apparatus for operation.

  The present invention has been made in view of such problems, and an image processing system, an image processing apparatus control method, a portable terminal, and an information processing apparatus that can easily operate an image processing apparatus using a portable terminal. An object is to provide a device and a control program.

  In order to achieve the above object, according to an aspect of the present invention, an image processing system includes a mobile terminal, an image processing device including processing means, and an information processing device. The portable terminal includes a photographing unit, a first acquisition unit for acquiring position information and orientation information of the portable terminal, a display unit, and an input unit for inputting an instruction for an operation screen displayed on the display unit. Including. The information processing apparatus includes, as information about the image processing apparatus, position information of the image processing apparatus, display information for screen display including choices of functions that are installed and usable, and a position of the mobile terminal that performs effective screen display. Storage means for storing display area information defining the range to which the data belongs. The portable terminal transmits position information and orientation information at the time of photographing with the photographing means to the information processing device, and the information processing device transmits the position information and orientation information of the portable terminal based on the position information and orientation information at the time of photographing with the portable terminal. An image processing device included in a photographed image by the photographing unit is detected and information related to the image processing device is transmitted to the mobile terminal. The mobile terminal includes a display area of the received information related to the image processing device. To display on the display means an operation screen obtained by combining the image taken by the photographing means and the display information using information relating to the image processing device belonging to the range specified in the information, and to accept the function selection by the input means The processing means is further included.

  Preferably, the processing unit of the mobile terminal determines from the position information and the display area information whether or not the mobile terminal belongs to the range specified in the display area information, and determines that the mobile terminal belongs to the range. In such a case, processing for generating an operation screen by combining the captured image and the display information is executed.

  More preferably, the processing unit of the mobile terminal repeats the process of determining and the process of generating the operation screen after displaying the operation screen on the display unit, and updates the operation screen according to the determination result.

  More preferably, the processing unit of the mobile terminal determines that the mobile terminal does not belong to the range defined in the display area information in the determination process performed after the operation screen is displayed on the display unit. In the process of generating the operation screen, the display information displayed on the operation screen is grayed out and combined with the captured image.

  Preferably, the processing unit of the mobile terminal may cause the mobile terminal to move out of the range defined in the display area information from the position information and the display area information after displaying the operation screen on the display unit. A process of determining whether or not there is further executed, and a process of displaying a warning when it is determined that there is a possibility that the portable terminal moves out of the range defined in the display area information .

  Preferably, the image processing system further includes a storage device for storing entry information for each user. The mobile terminal further includes second acquisition means for acquiring the location information of the mobile terminal by acquiring the user's room entry information associated with the mobile terminal from the storage device. It is determined whether or not the mobile terminal belongs to the range defined in the display area information from the position information and the display area information acquired by the second acquisition means.

  Preferably, the display area information is specified by a specified position on the map of the installation location of the image processing apparatus stored in advance.

  According to another aspect of the present invention, a method for controlling an image processing apparatus includes: controlling an image processing apparatus in an image processing system including a portable terminal including an imaging unit and a display unit; an image processing apparatus; and an information processing apparatus. In this method, the information processing apparatus includes, as information related to the image processing apparatus, position information of the image processing apparatus, display information for screen display including choices of functions that are installed and usable, and a mobile that performs screen display. Storing the display area information defining the range to which the position of the terminal belongs, the step of transmitting the position information and orientation information at the time of shooting by the shooting means of the mobile terminal to the information processing device, Based on the position information and orientation information received from the portable terminal and the position information included in the information related to the image processing device, the photographed image by the photographing means of the portable terminal A step of extracting the included image processing device and transmitting information relating to the extracted image processing device to the portable terminal; and the portable terminal including the received information relating to the image processing device as display area information. Using the information related to the image processing apparatus belonging to the specified range, a step of displaying on the display means an operation screen obtained by combining the image captured by the photographing means and the display information, and an instruction for the operation screen on the portable terminal. If accepted, the method includes a step of transmitting a signal representing an instruction to the image processing device, and a step of executing a corresponding image processing based on the signal in the image processing device.

  According to still another aspect of the present invention, the portable terminal is a photographing unit, a first acquisition unit for acquiring position information and orientation information of the portable terminal, a display unit, and an operation displayed on the display unit. An input unit for inputting an instruction to the screen and a processing unit are provided. The processing means includes processing for transmitting position information and orientation information at the time of photographing by the photographing means to the information processing apparatus, and information regarding the image processing apparatus received from the information processing apparatus. Using the information related to the image processing device belonging to the specified range, processing for displaying on the display means an operation screen obtained by combining the captured image and display information on the photographing means, and accepting an instruction for the operation screen on the input means And a process of transmitting a signal representing an instruction to the image processing apparatus.

  Preferably, the mobile terminal acquires the user's entry information associated with the mobile terminal from a storage device for storing the entry information for each user, thereby acquiring the position information of the mobile terminal. The image processing apparatus further includes a second acquisition unit, and the processing unit determines whether the portable terminal belongs to a range defined in the display area information from the position information and the display area information acquired by the second acquisition unit. .

  According to still another aspect of the present invention, the information processing apparatus includes, as information about the image processing apparatus, position information of the image processing apparatus, display information for screen display including options of functions that are installed and usable, When storage means for storing display area information that defines a range to which the position of the mobile terminal that performs screen display belongs, and position information and orientation information at the time of shooting with the shooting means of the mobile terminal from the mobile terminal, Based on the position information and the orientation information, and the position information included in the information related to the image processing device, the image processing device included in the image captured by the imaging means of the mobile terminal is extracted, and the extracted information related to the image processing device is carried And processing means for executing processing to be transmitted to the terminal.

  Preferably, the information processing apparatus further includes a setting unit that stores in advance a map of an installation location of the image processing apparatus, accepts designation of a position on the map, and sets display area information according to the designated position.

  According to still another aspect of the present invention, the control program is a control program for causing a mobile terminal to execute processing, and the mobile terminal includes an imaging unit and a display unit, and includes position information at the time of imaging by the imaging unit and Of the information on the image processing device received from the information processing device and the step of transmitting the orientation information to the information processing device, the information on the image processing device in which the mobile terminal belongs to the range specified in the display area information. A step of displaying on the display means an operation screen obtained by combining the image captured by the photographing means and the display information; and when an instruction to the operation screen is received by the input means, a signal representing the instruction is transmitted to the image processing apparatus. And causing the portable terminal to execute the step of performing.

  According to still another aspect of the present invention, the control program is a control program for causing the information processing apparatus to execute processing, and the information processing apparatus includes, as information about the image processing apparatus, position information of the image processing apparatus, Display information for screen display including choices of functions that are installed and usable, and display area information that defines a range to which the position of the mobile terminal that performs screen display belongs, from the mobile terminal to the imaging means of the mobile terminal An image included in the captured image of the imaging means of the mobile terminal based on the step of receiving the position information and the orientation information at the time of shooting in the camera, and the position information and the orientation information and the position information included in the information related to the image processing device The information processing device includes a step of extracting the processing device and a step of transmitting information regarding the extracted image processing device to the portable terminal. Make.

  According to the present invention, the image processing apparatus can be easily operated using the portable terminal.

It is a figure which shows the specific example of a structure of the image processing system concerning 1st Embodiment. It is a figure for demonstrating the outline | summary of operation | movement in the image processing system concerning 1st Embodiment. 2 is a diagram illustrating a specific example of a hardware configuration of an MFP (Multi Function Peripheral) included in the image processing system according to the embodiment. FIG. 3 is a diagram illustrating a specific example of a configuration of an operation panel of an MFP. FIG. It is a figure which shows the specific example of the hardware constitutions of the portable terminal contained in the image processing system concerning embodiment. It is a figure which shows the specific example of the hardware constitutions of the server contained in the image processing system concerning embodiment. 2 is a block diagram illustrating a specific example of a functional configuration of an MFP. FIG. It is a block diagram which shows the specific example of a function structure of the portable terminal concerning 1st Embodiment. It is a block diagram which shows the specific example of a function structure of a server. 10 is a flowchart illustrating a specific example of an operation in the MFP that performs an MFP information registration operation. It is a figure which shows the specific example of the MFP list memorize | stored in the server concerning 1st Embodiment. FIG. 6 is a sequence diagram showing an operation flow for operating the MFP according to the first embodiment. It is a flowchart showing operation | movement with the portable terminal concerning 1st Embodiment for displaying a tag on a picked-up image. It is a flowchart showing operation | movement with a portable terminal when a box function is selected as a function of operation object. It is a flowchart showing operation | movement with a portable terminal when a scanning function is selected as a function of operation object. It is a block diagram which shows the specific example of the function structure of MFP concerning the modification. FIG. 10 is a sequence diagram illustrating a flow of operations for operating an MFP in an image processing system according to a modified example. It is a figure which shows an example of the screen display of the operation panel of a portable terminal. It is a figure which shows the specific example of a structure of the image processing system concerning 2nd Embodiment. It is a figure for demonstrating an example of the setting method of a displayable area. It is a block diagram which shows the specific example of a function structure of the portable terminal concerning 2nd Embodiment. It is a figure which shows the specific example of the MFP list memorize | stored in the server concerning 2nd Embodiment. FIG. 10 is a sequence diagram illustrating an operation flow for operating an MFP according to a second embodiment. It is a flowchart showing the operation | movement with the portable terminal concerning 2nd Embodiment for displaying a tag on a picked-up image. It is a figure which shows an example of the screen display of the operation panel of a portable terminal. It is a figure which shows an example of the screen display of the operation panel of a portable terminal. It is a figure which shows the specific example of a structure of the image processing system concerning the modification of 2nd Embodiment. It is a figure which shows the specific example of the entrance information memorize | stored in the memory | storage device which is not shown in an entrance management system. It is a flowchart showing operation | movement with the portable terminal concerning the modification of 2nd Embodiment for displaying a tag on a picked-up image.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same.

[First Embodiment]
<System configuration>
FIG. 1 is a diagram illustrating a specific example of the configuration of the image processing system according to the first embodiment.

  Referring to FIG. 1, the image processing system according to the first embodiment includes an MFP (Multi Function Peripheral) 100 as an image processing apparatus, a portable terminal 200, and a server 300.

  The image processing apparatus is not limited to the MFP, and may be another apparatus such as a printer, a facsimile transmitter / receiver, and a copier as long as the apparatus has at least one image processing function. The MFP 100 is an image processing apparatus having a combination of these functions.

  The mobile terminal 200 may be a mobile phone, for example, and may be another device as long as it has at least a camera function, an instruction input function, and a communication function.

The server only needs to be composed of a general personal computer or the like.
MFP 100 and portable terminal 200 can communicate with each other and exchange information. Communication between MFP 100 and portable terminal 200 may be in any communication form as long as it is wireless communication. In the following examples, wireless communication using Bluetooth (registered trademark) is used, but infrared communication, short-range wireless communication, or the like may be used.

  MFP 100 and server 300 can also communicate with each other and exchange information. Communication between the MFP 100 and the server 300 may be any communication form regardless of wired or wireless. Since the image processing system according to the present embodiment is assumed to be used in an office environment, as an example, the MFP 100 and the server 300 are connected to a wired or wireless LAN (Local Area Network), and the network is connected via the network. Communication mode for exchanging information.

  The portable terminal 200 and the server 300 can communicate with each other and exchange information. The communication between the portable terminal 200 and the server 300 may be in any communication form as long as it is wireless communication. In the following examples, wireless communication using a telephone line is assumed. However, infrared communication, short-range wireless communication, or the like may be used, or the portable terminal 200 and the server 300 are connected to a wireless LAN. If there is, a communication form in which information is exchanged via the network is also included.

  In the following description, MFP 100 and portable terminal 200 use Bluetooth (registered trademark) for wireless communication, MFP 100 and server 300 communicate via a LAN, and portable terminal 200 and server 300 use a telephone line. However, the communication modes may be the same, such as communication via a LAN.

  FIG. 1 shows a configuration in which the image processing system includes one MFP, one portable terminal, and one server. However, at least a plurality of MFPs and portable terminals may be included.

<Overview of operation>
The outline of the operation in the image processing system according to the first embodiment is roughly divided into an MFP information registration operation and an operation for operating the MFP.

  Here, “MFP information” refers to information about the MFP included in the image processing system registered in the server 300, and is position information that is information for specifying the position of the MFP 100 and communication that is information for communication. Information and function information that is information for specifying a function that is installed and usable. Specific examples thereof will be described later. The plurality of MFP information stored in the server 300 is also referred to as “MFP list” in the following description.

  The MFP information registration operation is performed prior to the operation for operating the MFP, and the MFP 100 information is registered in the server 300 at a predetermined timing, or the MFP 100 information registered in the server 300 is updated. The operation to update to the information of.

  The operation for operating the MFP refers to an operation for using the portable terminal 200 to perform an operation for causing the MFP 100 to execute image processing using a function provided in the MFP 100. FIG. 2 is a diagram for explaining an outline of operations in the image processing system according to the present embodiment. An operation for operating MFP 100 in the image processing system will be described with reference to FIG.

  Referring to FIG. 2, the user carrying portable terminal 200 takes a picture with pointing portable terminal 200 toward MFP 100 to be operated ((1) in FIG. 2). As a result, a captured image showing the operation target MFP 100 is displayed on the display screen of the portable terminal 200. Further, a function that can be used by MFP 100 is displayed as a tag so as to be overlaid on the captured image ((2) in FIG. 2). The user selects a tag representing a function to be used from among the tags.

  Note that the “tag” here indicates an option displayed on the touch panel, and the function to be operated as shown in (2) of FIG. 2 is described, and the operation is superimposed on the original image. Displayed in the vicinity of an image of the target MFP 100 and indicates an instruction to operate the function when touched.

  For example, when a tag indicating a box function that is a function of processing image data stored in a storage area associated with a user is selected (touched) ("touch" in (2) of FIG. 2), the portable terminal The display screen 200 is switched to a screen for operating the box function of the MFP 100 ((3) in FIG. 2). When image data to be processed is selected on the screen (“touch” in FIG. 2 (3)), the display screen of the mobile terminal 200 is further changed to a screen for selecting a processing method for the image data. It is switched ((4) in FIG. 2). When a processing method is selected on the screen (“touch” in (4) of FIG. 2), image processing based on the operation is executed in MFP 100.

Hereinafter, each apparatus structure for performing these operations will be described.
<Configuration of MFP>
FIG. 3 is a diagram illustrating a specific example of the hardware configuration of the MFP 100.

  Referring to FIG. 3, MFP 100 includes a CPU (Central Processing Unit) 10 that is an arithmetic device for overall control, a ROM (Read Only Memory) 11 for storing a program executed by CPU 10, A RAM (Random Access Memory) 12 for functioning as a work area when the program is executed by the CPU 10, a scanner 13 for optically reading a document placed on a document table (not shown), and obtaining image data; A printer 14 for fixing the image data on the printing paper, an operation panel 15 including a touch panel for displaying information and receiving an operation input to the MFP 100, and a memory 16 for storing the image data , A communication controller 17 for controlling wireless communication using Bluetooth (registered trademark), and a LAN And and a network controller 18 for controlling the communication.

FIG. 4 is a diagram illustrating a specific example of the configuration of the operation panel 15.
Referring to FIG. 4, operation panel 15 includes a touch panel 51 and an operation key group 52. The touch panel 51 is configured by overlapping a display device such as a liquid crystal display device and a position indicating device such as an optical touch panel or a capacitive touch panel, and displays an operation screen to specify an indicated position on the operation screen. The CPU 10 displays an operation screen on the touch panel 51 based on data for displaying a screen stored in advance.

  The operation key group 52 includes a numeric key 521, a start key 522 for instructing start of a job, and a mode selection key 523 for selecting a mode to be activated. Further, each of the mode selection keys 523 includes a display lamp 523A indicating that the LED is selected and has a built-in LED (Light Emitting Diode) or the like.

  The specified position (touched position) on the touch panel 51 and an operation signal indicating the pressed key are input to the CPU 10. The CPU 10 specifies the operation content from the pressed key or the displayed operation screen and the designated position, and executes processing based on the operation content.

<Configuration of mobile terminal>
FIG. 5 is a diagram illustrating a specific example of the hardware configuration of the mobile terminal 200.

  Referring to FIG. 5, mobile terminal 200 includes a CPU 20 that is an arithmetic device for overall control, a ROM 21 for storing a program executed by CPU 20, and a work area when CPU 20 executes the program. RAM 22 for functioning as an electronic device, an electronic compass 23 for detecting the orientation of the portable terminal 200 including a magnetic sensor, a GPS (Global Positioning System) signal or a position signal from a base station, and the portable terminal Communication with a GPS controller 24 for acquiring position information, an operation panel 25 including a touch panel for displaying information and receiving operation inputs to the mobile terminal 200, a camera 26, and a base station (not shown). A first communication controller 27 for controlling communication via a telephone line by performing a A second communication controller 28 for controlling wireless communication using tooth (registered trademark) and a memory 29 for storing telephone book data and the like are included.

  Operation panel 25 may have the same configuration as operation panel 15 of MFP 100. That is, as an example, a touch panel configured by overlapping a display device such as a liquid crystal display device and a position indicating device such as an optical touch panel or a capacitive touch panel, and an operation key are included.

  The CPU 200 displays an operation screen on the touch panel based on data for displaying a screen stored in advance. On the touch panel, an indicated position on the operation screen is specified, and an operation signal indicating the position is input to the CPU 20. An operation signal indicating the pressed key is input to the CPU 20. The CPU 20 specifies the operation content from the pressed key or the displayed operation screen and the indicated position, and executes processing based on the specified operation content.

  In addition, the electronic compass 23 and the GPS controller 24 input a signal indicating the acquired orientation and position information of the mobile terminal 200 to the CPU 20.

<Server configuration>
FIG. 6 is a diagram illustrating a specific example of the hardware configuration of the server 300.

  Referring to FIG. 6, server 300 is configured by a general computer or the like as described above. As an example, CPU 30 that is an arithmetic device for overall control, a program executed by CPU 30, and the like ROM 31 for storing data, RAM 32 for functioning as a work area when CPU 30 executes a program, HD (Hard Disk) 33 for storing MFP information, and the like, and controlling communication via a telephone line A communication controller 34 for performing communication, and a network controller 35 for controlling communication via the LAN.

<Functional configuration>
In the image processing system according to the first embodiment, a functional configuration of each device for realizing the operation as described in the operation outline will be described.

  FIG. 7 is a block diagram illustrating a specific example of a functional configuration of MFP 100. Each function shown in FIG. 7 is a function mainly formed in the CPU 10 when the CPU 10 reads a program stored in the ROM 11 and executes it on the RAM 12. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 7, as a function for realizing the above operation, MFP 100 includes an instruction input unit 101 for receiving an instruction input from operation panel 15 or another device, operation panel 15 or a GPS not shown in FIG. In the case of providing a function, a position information input unit 102 for receiving input of own position information from the function, and an address on a LAN and wireless communication using Bluetooth (registered trademark) are received. An address input unit 103; a location information storage unit 104 for storing location information received by the location information input unit 102; a communication information storage unit 105 for storing the address received by the address input unit 103 as communication information; A function management unit 106 for managing the functions of the MFP 100 and executing the operations of the functions; The detection unit 107 for detecting a change in the position information, communication information, and executable function of the FP 100, and the position information, communication information, and function information of the MFP 100 are transmitted to the server 300 as MFP information, and the server 300 An update unit 108 for registering or updating in the MFP list, and a screen transmission unit for transmitting necessary operation screen data among operation screen data stored in advance in response to a request from the portable terminal 200 described later 109 and an instruction specifying unit 110 for specifying the instruction content from the portable terminal 200.

  The method of specifying the instruction content of the instruction specifying unit 110 is the same as the method of specifying the instruction content when a touch instruction is received along the operation screen on the operation panel 15 provided therein. That is, the instruction specifying unit 110 specifies which position on the operation screen the instruction position is based on the signal specifying the instruction position from the mobile terminal 200 and the transmitted screen data, and is stored in advance. The instruction content corresponding to the specified position is determined with reference to the correspondence between the position on the screen and the instruction content.

  FIG. 8 is a block diagram illustrating a specific example of a functional configuration of the mobile terminal 200. Each function shown in FIG. 8 is a function mainly formed in the CPU 20 when the CPU 20 reads a program stored in the ROM 21 and executes it on the RAM 22. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 8, as a function for realizing the above-described operation, portable terminal 200 includes instruction input unit 201 for receiving an instruction input from operation panel 25, and the portable terminal 200 according to a shooting instruction from operation panel 25. A position acquisition unit 202 for acquiring position information of the terminal 200, a direction acquisition unit 203 for acquiring the direction of the mobile terminal 200 in response to a shooting instruction from the operation panel 25, and a shooting instruction from the operation panel 25 The image acquisition unit 204 for acquiring the image data acquired by executing the shooting by the camera 26 according to the information, and the server 300 together with the position information and the direction information acquired according to the shooting instruction from the operation panel 25 Server request unit 205 for requesting MFP information and acquiring MFP information of MFP 100 to be operated from server 300 in response to the request. An MFP information acquisition unit 206 for generating a screen data by combining selectable tags with the image data based on image data obtained by photographing and MFP information, and screen data. Display processing unit 208 for executing processing to be displayed on operation panel 25, transmission unit 210 for transmitting an operation signal for specifying an instruction position on the display screen based on the screen data to MFP 100, and a response to the signal A screen acquisition unit 209 for acquiring screen data transmitted from the MFP 100.

  Note that the position acquisition unit 202 and the direction acquisition unit 203 acquire position information and direction information in response to a shooting instruction from the operation panel 25, respectively. Therefore, the information includes the shooting position information and the shooting direction, respectively. Information. Therefore, in the following description, the position information and the direction information transmitted from the portable terminal 200 to the server 300 according to the above program are also referred to as shooting position information and shooting direction information, respectively.

  FIG. 9 is a block diagram illustrating a specific example of a functional configuration of the server 300. Each function shown in FIG. 9 is a function mainly formed in the CPU 30 when the CPU 30 reads a program stored in the ROM 31 and executes it on the RAM 32. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 9, as a function for realizing the above-described operation, server 300 receives MFP information input unit 301 for receiving input of MFP information from MFP 100 and the above-described input to the MFP list in a predetermined area of HD 33. An MFP information storage unit 302 for storing or updating MFP information, a portable information input unit 303 for receiving input of a shooting position and a shooting direction together with a request for MFP information from the portable terminal 200, and each position of the MFP list With reference to the information, based on the shooting position and shooting direction of the mobile terminal 200, a search unit 304 for searching for an MFP existing in the captured image of the mobile terminal 200, and the MFP information of the searched MFP 100 is stored in the mobile terminal 200. And a transmission unit 305 for transmitting the message.

<Operation flow for registering MFP information>
FIG. 10 is a flowchart showing a specific example of the operation in MFP 100 that performs the registration operation of MFP information. The operation shown in the flowchart of FIG. 10 is realized by the CPU 10 reading a program stored in the ROM 11 and executing it on the RAM 12.

  Here, registration of MFP information includes “manual registration” performed by user operation and “automatic registration” automatically performed without user operation at a predetermined timing, and either method is adopted. Whether to do this is set in MFP 100 by an administrator or the like in advance. The CPU 10 stores the setting.

  Referring to FIG. 10, when “manual registration” is set (YES in step S <b> 101), in step S <b> 103, CPU 10 executes a predetermined MFP information registration operation in accordance with an operation signal from operation panel 15. . Specifically, CPU 10 accepts input of position information, communication information, and function information as MFP information in accordance with an operation signal from operation panel 15, and stores them. Then, the information is transmitted to the server 300 as MFP information.

  On the other hand, when “automatic registration” is set (NO in step S101), the CPU 10 detects position information, communication information, and function information, respectively, at a predetermined timing. The predetermined timing corresponds to, for example, a timing when the MFP 100 is installed, a timing when network communication is established, a predetermined time interval, and the like. When CPU 10 detects position information, communication information, or function information (YES in step S105), it confirms the presence or absence of a change in comparison with those stored. If it is detected that there has been a change (YES in step S107), in step S109, the CPU 10 executes a predetermined MFP information registration operation. Specifically, CPU 10 transmits the MFP information to server 300 after updating the detected information among the stored position information, communication information, and function information to the latest information. Here, only the changed information of the MFP information may be transmitted, or all the MFP information may be transmitted.

  When server 300 receives MFP information from MFP 100, server 300 updates the MFP list stored in a predetermined area of HD 33. Specifically, if the MFP information of MFP 100 is not registered in the MFP list, the received MFP information is newly registered. If it is already registered, it is updated with the received MFP information.

  By performing this registration operation, the position, address, and functions that are installed and usable in each MFP included in the image processing system are registered in the server 300.

  FIG. 11 is a diagram illustrating a specific example of the MFP list stored in the server 300. Referring to FIG. 11, for each MFP included in the image processing system, the MFP list includes latitude, longitude, and altitude as position information, an IP address that is an address for communicating with server 300 as communication information, and portable terminal 200. A Bluetooth (registered trademark) address, which is an address for communicating with the device, and presence / absence of a box function and a scan function are registered as function information.

  The MFP information is not limited to that shown in FIG. For example, the position information may not include the altitude. Further, the communication information may be an address corresponding to the communication form when performing another form of communication. Moreover, the presence / absence of each of other functions may be included as function information. In addition, in the case of performing user authentication as described later, whether or not a function can be used for each user may be included.

  In the above description, MFP information is registered and updated in the MFP list by transmitting MFP information from MFP 100 to server 300. However, when server 300 has an input device, for example, an administrator of the image processing system For example, the MFP information may be directly input to the server 300. Further, MFP information may be transmitted to server 300 from a communication terminal (not shown) connected to MFP 100. In this case, MFP 100 does not need to be able to communicate with server 300, as long as the communication terminal connected to MFP 100 can communicate with server 300.

<Operation Flow for Operations for MFP>
FIG. 12 is a sequence diagram showing an operation flow for operating the MFP. The left end of FIG. 12 represents the flow of processing in the MFP 100, the center represents the flow of processing in the portable terminal 200, and the right end represents the flow of processing in the server 300. Each operation is realized by the CPU of each device reading a program stored in the ROM and executing it on the RAM.

  Referring to FIG. 12, in the state in which the application for operating the MFP is activated on portable terminal 200 (step S1), when photographing with the camera is performed (step S3), the photographing position and photographing direction on portable terminal 200 Is transmitted to the server 300, and the corresponding MFP information is requested (step S3-1).

  When server 300 accepts the request from portable terminal 200, it refers to each position information in the stored MFP list, and MFP information of MFPs located within a predetermined range from the photographing position of portable terminal 200 in the photographing direction. Is searched (step S5). Then, the corresponding MFP information is transmitted to the portable terminal 200 (step S5-1). In portable terminal 200, a tag for selecting a function specified by the function information included in the received MFP information is combined with the image data from camera 26 by the application, and the combined image is displayed on operation panel 25. (Step S7).

  When a tag is selected (touched) on an image displayed on operation panel 25 of portable terminal 200 (step S9), a screen request is made to MFP 100 together with information for specifying a function corresponding to the selected tag. (Step S9-1).

  The MFP 100 transmits screen data for operating the processing by the requested function to the requesting mobile terminal 200 (step S11). Here, screen data stored in advance for display on its own operation panel 15 may be transmitted to the mobile terminal 200, or corresponding screen data stored for the mobile terminal 200 may be transmitted. In addition, when information identifying itself is also transmitted from the requesting mobile terminal 200, resolution conversion is performed on screen data stored in advance for display on the operation panel 15 based on the information of the mobile terminal 200. Etc. may be transmitted.

  Here, the screen data itself is assumed to be transmitted from the MFP 100 to the mobile terminal 200. However, if the above-mentioned program running on the mobile terminal 200 has a function of generating an operation screen, it is necessary for the operation screen. Only the information may be transmitted from the MFP 100 to the portable terminal 200, and the portable terminal 200 may generate screen data based on the information.

  The portable terminal 200 displays an operation screen based on the transmitted screen data. When a touch operation is performed along the screen (step S13), information specifying the instructed position is transmitted as an operation signal to MFP 100 (step S13-1). As an example of the operation signal transmitted here, an operation signal specified by a relative position with respect to a displayed screen at a position indicated on the operation panel 25 can be cited. As another example, when an application executed on the mobile terminal 200 has a function of analyzing screen data from the MFP 100, the operation content corresponding to the position designated on the operation panel 25 is specified, and the content is specified. An operation signal may be transmitted to MFP 100.

  MFP 100 specifies the operation content based on the screen data transmitted to portable terminal 200 and the instructed position represented by the received operation signal (step S15). At this time, if further operation is required before the selected function is executed, screen data for the next operation screen is transmitted according to the program for executing the function concerned. In this case, the operations in steps S11 to S15 are repeated until the function is executed.

  When all operations necessary for executing the function are specified, MFP 100 executes image processing according to the function in accordance with the operation (step S17).

  If the program running on the mobile terminal 200 has a function of generating a control signal, a control signal for the MFP 100 is generated according to the screen data and the position designated on the operation panel 25, and In step S13-1, a control signal may be transmitted instead of the operation signal. In this case, MFP 100 executes image processing in accordance with a control signal from portable terminal 200.

  Hereinafter, the operation of the mobile terminal 200 will be described in detail with reference to flowcharts. 13, FIG. 14, and FIG. 15 are flowcharts showing specific examples of operations in portable terminal 200 that perform operations for operating the MFP. FIG. 13 is a flowchart showing an operation for displaying a tag superimposed on a photographed image, FIG. 14 is a flowchart showing an operation when a box function is selected as a function to be operated, and FIG. 6 is a flowchart showing an operation when a scan function is selected as a target function. The operations shown in the flowcharts of FIGS. 13, 14, and 15 are realized by the CPU 20 reading a program corresponding to the MFP operation application stored in the ROM 21 and executing it on the RAM 22.

  Referring to FIG. 13, when CPU 20 is executing an application for operating the MFP (YES in step S201) and shooting is being performed with camera 26 (YES in step S203), in step S205, CPU 20 Then, information indicating the photographing position and the photographing direction is transmitted to server 300, and transmission of MFP information of the corresponding MFP is requested.

  If the response is received from server 300 (YES in step S207) and the response is MFP information (NO in step S209), CPU 20 selects a tag for selecting the function specified by the function information of the MFP information. A process for combining and displaying the image data obtained by photographing is displayed.

  Specifically, when the function information indicates that the box function can be used (YES in step S211), in step S213, the CPU 20 performs shooting based on the MFP position information included in the MFP information. The display position of the MFP in the obtained image data is calculated, and a tag for instructing the start of the operation for the box function is synthesized and displayed at or near that position. In the screen example of (2) in FIG. 2, image data of a tag described as “with BOX” is stored in advance, and the screen data is generated by synthesizing the image data at the calculated position. 25.

  If the function information indicates that the scan function can be used (YES in step S215), the CPU 20 obtained in step S217 is photographed based on the MFP position information included in the MFP information. The display position of the MFP in the image data is calculated, and a tag for instructing the start of the operation for the scan function is synthesized and displayed at that position or in the vicinity thereof. In the screen example of (2) of FIG. 2, image data of a tag described as “scan” is stored in advance, and the screen data is generated by synthesizing the image data at the calculated position. To display.

  Next, referring to FIG. 14, when a tag representing the box function is selected from the displayed tags (YES in step S301), in step S303, CPU 20 determines the Bluetooth of MFP 100 from the communication information of the MFP information. The (registered trademark) address is read, and communication using the MFP 100 and Bluetooth (registered trademark) is started. In step S305, screen data for operating the processing by the box function is transmitted to MFP 100.

  When the corresponding screen data is received from MFP 100, CPU 20 causes operation panel 25 to display an operation screen based on the screen data in step S307. When an operation based on the operation screen, that is, a touch operation on the operation screen is received (YES in step S309 and NO in step S311), in step S313, the CPU 20 generates a signal for specifying the designated position on the operation screen. An operation signal is transmitted to MFP 100.

  Thereafter, the operation returns to step S305 and is repeated until the subsequent operation is completed (YES in step S311). That is, every time a touch operation on the operation screen is received, a signal for specifying a designated position on the operation screen is transmitted as an operation signal to MFP 100, and the next screen data is transmitted from MFP 100. Thereby, as shown in (3) and (4) of FIG. 2, the operation screen for inputting the operation in the box function is changed, and the operation necessary for the processing by the box function is changed to the operation of the portable terminal 200. It will be received by the panel 25.

  Even when the tag representing the scan function is selected from the displayed tags, the CPU 20 performs the same operation as that shown in FIG. Specifically, referring to FIG. 15, when a tag representing a scan function is selected from the displayed tags (YES in step S401), in step S403, CPU 20 determines MFP 100 from the communication information of the MFP information. The Bluetooth (registered trademark) address is read out, and communication using the MFP 100 and Bluetooth (registered trademark) is started. In step S405, screen data for operating the processing by the scan function is transmitted to MFP 100.

  When the corresponding screen data is received from MFP 100, CPU 20 causes operation panel 25 to display an operation screen based on the screen data in step S407. When an operation based on the operation screen, that is, a touch operation on the operation screen is accepted (YES in step S409 and NO in step S411), in step S413, the CPU 20 generates a signal for specifying the designated position on the operation screen. An operation signal is transmitted to MFP 100.

  Thereafter, the operation returns to step S405 and is repeated until the subsequent operation is completed (YES in step S411). That is, every time a touch operation on the operation screen is received, a signal for specifying a designated position on the operation screen is transmitted as an operation signal to MFP 100, and the next screen data is transmitted from MFP 100. As a result, the operation screen for inputting the operation by the scan function is changed as in (3) and (4) of FIG. 2, and the operation panel 25 of the portable terminal 200 receives the operation necessary for the processing by the scan function. It will be.

<Effect of the first embodiment>
By performing the above operation in the image processing system according to the first embodiment, the operation of MFP 100 using portable terminal 200 as described in the operation overview becomes possible.

  In the above example, an example in which the box function and the scan function are provided as functions of the MFP 100 is described. However, with the recent increase in functionality of the MFP, more functions are installed and usable. In that case, the operation screen displayed on the operation panel 15 of the MFP 100 displays many options for selecting a function to be operated from among these functions, or displays many options for operations in the functions. In some cases, the operation screen is difficult for an unfamiliar user to operate. In addition, functions installed in MFPs and usable functions may vary from MFP to MFP, so it is impossible for an unfamiliar user to immediately determine whether or not a desired function can be used in the target MFP. In some cases.

  In such a case, in the image processing system according to the first embodiment, a function that can be used in the MFP is a tag by starting a dedicated application on the portable terminal 200 that is used to the user and photographing the MFP. Displayed. Further, by selecting the tag, an operation screen of the function to be operated is displayed on operation panel 25 of portable terminal 200, and it is possible to operate MFP 100 by performing operation input on the screen. .

  As a result, the user can recognize the usable functions at a glance, and can easily operate the functions. Further, even when MFP 100 is at a distant position, the operation can be performed with portable terminal 200 that is carried without moving to that position.

<Modification>
In MFP 100, user authentication may be performed when using the function. User authentication may be performed similarly in MFP 100 according to a second embodiment described later. This case will be described as a modified example.

  FIG. 16 is a block diagram illustrating a specific example of a functional configuration of the MFP 100 according to the modification. Each function shown in FIG. 16 is also a function mainly formed in the CPU 10 when the CPU 10 reads a program stored in the ROM 11 and executes it on the RAM 12. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 16, MFP 100 according to the modified example as a function for realizing the above operation includes login information input unit 111 and a user for receiving input of login information in addition to the functions shown in FIG. 7. And an authentication unit 112 for performing authentication.

  In the modification, the function management unit 106 stores functions that are installed and usable, and stores user information that is permitted to be used for each function or user information that is information of a function that is permitted to be used for each user. .

  FIG. 17 is a sequence diagram showing an operation flow for operating the MFP in the image processing system according to the modification. The operation flow shown in FIG. 17 is different from the operation flow shown in FIG. 12 in that the operation for user authentication shown in step S10 is further performed.

  Specifically, in the image processing system according to the modification, a function is selected by the portable terminal 200 in step S9, and a screen request is issued together with information specifying the function corresponding to the tag selected in step S9-1. When done for MFP 100, MFP 100 determines whether or not the selected function requires user authentication. When user authentication is required, the operation for user authentication in step S10 is performed.

  That is, if user authentication is required, in step S <b> 10-1, MFP 100 transmits screen data for inputting login information used for user authentication to portable terminal 200 in place of the requested screen data. In step S10-2, when input of login information is accepted along the screen, in step S10-3, the MFP 100 refers to the stored user information and determines whether the selected function can be used. When it is determined that the function can be used, that is, when the user authentication for the function is successful, the screen data requested in step S11 is transmitted to the mobile terminal 200.

  By doing in this way, operability can be improved, ensuring security also about the function which requires user authentication, such as a box function, for example.

  In this modification, the input of login information is accepted, but the MFP 100 stores the correspondence relationship between the Bluetooth (registered trademark) address of the mobile terminal 200 and the user information, and the mobile phone in step S9-1. When screen data is requested from terminal 200 to MFP 100, user authentication may be performed based on the address. Alternatively, instead of such user authentication, authentication using a pass key (PIN code) in communication using Bluetooth (registered trademark) between the MFP 100 and the portable terminal 200 may be performed.

[Second Embodiment]
The image processing system according to the present embodiment is assumed to be used in an office environment. In a normal office, a plurality of MFPs may be installed at a certain interval, or a plurality of MFPs may be installed near a neighboring room across a wall.

  In the image processing system according to the first embodiment, MFP information about an MFP in the shooting range is sent from the server 300 to the portable terminal 200 based on the shooting position and shooting direction of the portable terminal 200. Therefore, when there are a plurality of MFPs within the shooting range, all the MFP information is transmitted from the server 300.

  In addition, the image processing system according to the present embodiment enables a user who is not frequently using the MFP or a user who uses a plurality of types of MFPs to operate the MFP with ease of use. For this reason, in the first embodiment, only “tags” as function options are displayed superimposed on the captured image. However, in order to further improve the convenience for the user, in addition to “tags” as options, A simple description such as the name and function of the MFP may also be displayed.

  In the second embodiment to be described later, explanations are displayed in addition to function options, and these are collectively referred to as “tags”.

  In such a case, depending on the environment, the display of the mobile terminal 200 may be as shown in FIG. 18, for example. That is, as shown in FIG. 18, in addition to MFP 100A installed in the room where the user who took the image with portable terminal 200 is located, MFP 100B installed in the adjacent room across the wall is also in the shooting range. A “tag” is displayed near the image in the image. Further, as described above, each simple explanation is also displayed.

  Since the device used for the operation is the portable terminal 200, it is assumed that the screen is small. For this reason, when displayed as shown in FIG. 18, a large amount of information is displayed on the captured image, which may be difficult to understand or difficult to operate for some users.

  Therefore, as a second embodiment, an embodiment in which screen display is easier to see and operate will be described.

<System configuration>
FIG. 19 is a diagram illustrating a specific example of the configuration of the image processing system according to the second embodiment. The image processing system according to the second embodiment can be the same as the configuration of the image processing system according to the first embodiment shown in FIG.

  However, FIG. 19 shows a configuration slightly different from the configuration shown in FIG. 1 as a modification of the system configuration. That is, referring to FIG. 19, in the image processing system according to the second embodiment, MFP 100 and server 300 are connected to a network such as a wired or wireless LAN. The mobile terminal 200 can also be connected to the network, and does not directly communicate with the MFP 100 and the server 300, but communicates with the MFP 100 and the server 300 via the network.

  Since the system configuration of FIG. 19 is shown as a modification of the system configuration according to the first embodiment, the image processing system according to the first embodiment can also employ this system configuration.

<Overview of operation>
In the image processing system according to the second embodiment, the “MFP information” includes the portable terminal 200 that can display the tag of the MFP in addition to the position information, communication information, and function information for each MFP. Information for specifying an area is stored in the server 300. In the following description, the area is referred to as “displayable area”, and information for specifying the displayable area for each MFP is referred to as “display area information”.

  The displayable area may be set by MFP 100 or may be set by server 300. If set in server 300, server 300 includes an input device such as an operation panel in addition to the configuration shown in FIG.

  FIG. 20 is a diagram for explaining an example of a displayable area setting method. When the displayable area is set by MFP 100, FIG. 20 shows a setting screen displayed on operation panel 15 of MFP 100. When the displayable area is set by server 300, FIG. It is a figure showing the setting screen displayed on the operation panel.

  Using FIG. 20, the displayable area is set as follows. That is, the setting device stores in advance a plan view of the vicinity of the MFP 100 having a simple sketch. As shown in FIG. 20, a plan view of the vicinity of the MFP 100 is displayed on the operation panel of the setting device, and the positions of the two points P1 and P2 that specify the displayable area are displayed on the display. Accept input. In the example of FIG. 20, an example in which a drag operation from the point P1 to the point P2 is received is shown. These two points may be specified by other methods. By specifying these two points, the setting device specifies a rectangular range having these two points as diagonals, and stores the rectangular range as a displayable area of the MFP 100.

  In the image processing system according to the second embodiment, the tag of MFP 100 is displayed over the captured image only when mobile terminal 200 is located in the displayable area. When portable terminal 200 is outside the displayable area of MFP 100, no tag is displayed even if MFP 100 is within the shooting range of portable terminal 200. If the portable terminal 200 moves from the displayable area of the MFP 100 to the outside of the MFP 100 while the tag for the MFP 100 is displayed, the tag display disappears with the movement.

<Functional configuration>
In particular, the functional configuration of the mobile terminal 200 will be described as the functional configuration of the apparatus for realizing the operation described in the operation outline in the image processing system according to the second embodiment.

  FIG. 21 is a block diagram illustrating a specific example of a functional configuration of the mobile terminal 200 according to the second embodiment. The functional configuration of the mobile terminal 200 according to the second embodiment is substantially the same as the functional configuration of the mobile terminal 200 according to the first embodiment shown in FIG. 8, but the configuration of the generation unit 207 is one. Different parts.

  Referring to FIG. 21, in the second embodiment, generation unit 207 of portable terminal 200 includes a determination unit 207A.

  FIG. 22 is a diagram illustrating a specific example of the MFP list stored in the server 300 according to the second embodiment. Referring to FIG. 22, in the second embodiment, server 300 includes display area information as MFP information in addition to position information. In FIG. 22, information to be displayed such as options for selecting a function and description of the MFP is shown as “display screen information”.

  When the displayable area is defined by a rectangle specified by two designated points as shown in FIG. 20, the display area information is designated by two designated points P1 and P2 as shown in FIG. Contains information. The information of the points P1 and P2 may be position information (longitude, latitude, and altitude) of each point, and the portable terminal 200 also stores a plan view of the vicinity where the MFP 100 similar to the server 300 is installed. In this case, the coordinates on the plan view may be used.

  The generation unit 207 of the portable terminal 200 stores the display area information in the received MFP information as area information 207B.

  Determination unit 207A acquires the position of portable terminal 200 from position acquisition unit 202, and determines whether portable terminal 200 is located within the displayable area represented by area information 207B of MFP 100. If it is determined that the image is located, the generation unit 207 combines the tag of the MFP 100 with the image data obtained by shooting to generate screen data. If it is determined that it is located outside the displayable area, screen data is generated without combining the tag of MFP 100 with image data obtained by shooting.

<Operation Flow for Operations for MFP>
FIG. 23 is a sequence diagram showing an operation flow for operating the MFP according to the second embodiment. Also in FIG. 23, as in FIG. 12, the left end represents the flow of processing in the MFP 100, the center represents the flow of processing in the portable terminal 200, and the right end represents the flow of processing in the server 300. Each operation is realized by the CPU of each device reading a program stored in the ROM and executing it on the RAM.

  Referring to FIG. 23, in the second embodiment, the flow in the first embodiment is different from the MFP information transmitted from server 300 to portable terminal 200 (step S5-1 '). Further, the processing for combining the images using the MFP information and displaying the tag is different (step S7 '). That is, in step S5-1 ', MFP information including position information, display screen information, and display area information is transmitted from server 300 to portable terminal 200 in response to a request. In step S7 ′, the portable terminal 200 determines whether display is possible based on the display area information of the MFP 100, and then uses the MFP information about the MFP 100 that can be displayed, out of the received MFP information. Image composition processing for displaying tags is performed.

  FIG. 24 is a flowchart showing the operation for displaying the tag superimposed on the photographed image in the portable terminal 200 in step S7 '. The operation shown in the flowchart of FIG. 24 is realized by the CPU 20 reading a program corresponding to the MFP operation application stored in the ROM 21 and executing it on the RAM 22.

  Referring to FIG. 24, when receiving the MTP information related to the MFP requested from server 300, CPU 20 determines that based on the current position of portable terminal 200 and the display area information included in each of the received MFP information. It is determined whether or not portable terminal 200 is within the displayable area set for each MFP (step S501).

  As described above, when the display area information is the position information of the two points forming the diagonal of the rectangular area that is the displayable area, the CPU 20 compares the position information with the position information obtained by the GPS controller 24. It can be determined whether or not it is within the displayable area.

  When the display area information is a coordinate on the plan view stored in advance, the CPU 20 converts the position information obtained by the GPS controller 24 into the coordinate on the plan view, and can be displayed as the display area information. By comparing the area and the coordinates indicating the position of the mobile terminal 200, it can be determined whether or not the area is within the displayable area.

  If it is determined that mobile terminal 200 is within the displayable area (YES in step S501), CPU 20 includes a display screen included in MFP information for MFP 100 determined as such in the received MFP information. A process for combining and displaying the display data obtained by the information with the image data obtained by photographing is executed.

  With this processing, a screen as shown in FIG. 25 is displayed on the operation panel 25 of the portable terminal 200. Specifically, referring to FIG. 25, even when two MFPs, MFP 100A and MFP 100B, exist in the shooting range from portable terminal 200, portable terminal 200 is set for MFP 100B. When not in the displayable area but in the displayable area set for MFP 100A, the tag of MFP 100B is displayed on the captured image, and only the tag of MFP 100A is displayed.

  If it is determined that mobile terminal 200 is not within the displayable area set for MFP 100 (NO in step S501), step S503 is not performed, and at predetermined time intervals. Based on the position information of the mobile terminal 200, the determination as to whether or not the mobile terminal 200 is within the displayable area set for the MFP 100 is repeated. As a result, when it is determined that the area is within the displayable area, step S503 is performed.

  After displaying the screen of FIG. 25, the CPU 20 displays the displayable area in which the mobile terminal 200 is set for the MFP 100 based on the position information of the mobile terminal 200 at a predetermined time interval. The determination whether or not it is within is repeated (step S509).

  Preferably, for the MFP, in addition to the displayable area, a belt-shaped warning area is set from the outer edge of the displayable area to the inside of the predetermined width. The warning area may be automatically set by the server 300 or a device that has set the displayable area with respect to the set displayable area. Therefore, it may be set.

  The CPU 20 further determines whether or not the mobile terminal 200 is in the warning area (step S505). This determination method can be the same as the determination method in step S501. If it is determined that the mobile terminal 200 is within the warning area (YES in step S505), the CPU 20 performs a process for notifying that effect, that is, the possibility of going out of the displayable area (step). S507). As an example of the notification in step S507, a warning may be displayed on the operation panel 25. Specifically, as shown in FIG. 26, a warning message stored in advance on the display screen of FIG.

  If no warning area is set, steps S505 and S507 need not be performed.

  If it is determined in step S509 that the mobile terminal 200 is within the displayable area set for the MFP 100 after the display screen as shown in FIG. 25 is displayed (YES in step S509). CPU20 repeats the process after said step S503. Thereby, the display screen is updated according to the position of the mobile terminal 200.

  On the other hand, when it is determined that mobile terminal 200 is not within the displayable area set for MFP 100 (NO in step S501), CPU 20 deletes the tag display of MFP 100 from the display screen (step S511). ). In step S511, CPU 20 executes processing for displaying the image data obtained by shooting at this time without combining the display data of MFP 100. As a result, the tag of MFP 100 is not displayed on the display screen.

  As another process of step S511, the tag of MFP 100 for which the tag has already been displayed in the previous process may be grayed out and displayed. FIG. 25 shows an example in that case.

<Effects of Second Embodiment>
By performing the above operation in the image processing system according to the second embodiment, even when there are a plurality of MFPs in the shooting range from portable terminal 200, the display set for those MFPs is displayed. When the portable terminal 200 does not exist in the possible area, the tag of the MFP is not combined with the captured image and displayed. Therefore, the tags of each of the plurality of MFPs as shown in FIG. 18 are displayed, and it is possible to prevent the tags from becoming difficult to see or to operate. This can further improve the convenience for the user when the image processing system has a plurality of MFPs installed in a predetermined range, particularly in an office environment.

<Modification>
In the above description of the second embodiment, the mobile terminal 200 determines whether or not the mobile terminal 200 is within the displayable area using the position information obtained by the GPS controller 24. . However, the determination is not limited to this method. Another example will be described as a modification.

  FIG. 27 is a diagram illustrating a specific example of the configuration of the image processing system according to the modification of the second embodiment. The image processing system according to the modification includes an entrance management system 400 for managing entrance of each user in addition to the configuration of the image processing system according to the second embodiment shown in FIG.

  In an environment in which the image processing system is installed, a reader for reading a user ID from an ID card or a portable terminal carried by the user is installed for each preset area. The entrance management system 400 is electrically connected to the reader, and receives information on the user who has entered and exited the area from the reader.

  The room entry management system 400 is configured by a general computer or the like, and includes a communication controller for communicating with the reader, a network controller for connecting to a network to which the MFP 100 or the like is connected, and a memory for storing room entry information. And a processing device for performing processing for updating the entry information of the storage device based on information from the reader.

  FIG. 28 is a diagram showing a specific example of room entry information stored in a storage device (not shown) of the room entry management system 400. Referring to FIG. 28, in room entry management system 400, as an example, the time of entry of each user into each area is stored as room entry information. The room entry information may be updated every day or may be updated every certain time. Further, the exit time may be stored together with the entrance time.

  In the modification, the mobile terminal 200 stores the association with the user ID in advance. And the position of the portable terminal 200 at that time is specified by making an inquiry to the room management system 400 using the user ID.

  FIG. 29 is a flowchart showing an operation for displaying a tag superimposed on a photographed image in the mobile terminal 200 according to the modification of the second embodiment. The operation shown in the flowchart of FIG. 29 is also realized by the CPU 20 reading out a program corresponding to the MFP operation application stored in the ROM 21 and executing it on the RAM 22.

  Referring to FIG. 29, in the modified example, prior to the operation according to the second embodiment shown in FIG. 24, the CPU 20 enters the room using the user ID associated with the mobile terminal 200. The management system 400 is inquired about room entry information (step S601). In step S <b> 501, the CPU 20 identifies the area where the room has been newly entered in the room entry information as the position of the mobile terminal 200, and compares the area with the displayable area set for the MFP 100. It is determined whether or not the portable terminal 200 is within the displayable area (step S501).

  If the displayable area set for each MFP and the area managed by the room entry management system 400 are different, the portable terminal 200 stores these correspondences in advance and obtains them from the room entry management system 400. The area managed by the room entry management system 400 in which the portable terminal 200 specified by the entered room entry information exists is compared with the displayable area to determine whether or not the displayable area belongs.

Subsequent operations are the same as those in the second embodiment.
With such a configuration, the above determination in the second embodiment can be facilitated.

  Further, it is possible to provide a program for causing the MFP 100 to execute the above-described operation, a program for causing the portable terminal 200 to execute the above-described operation, and a program for causing the server 300 to execute the above-described operation. Such a program is stored in a computer-readable recording medium such as a flexible disk attached to the computer, a CD-ROM (Compact Disk-Read Only Memory), a ROM (Read Only Memory), a RAM (Random Access Memory), and a memory card. And can be provided as a program product. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer. A program can also be provided by downloading via a network.

  The program according to the present invention is a program module that is provided as a part of a computer operating system (OS) and calls necessary modules in a predetermined arrangement at a predetermined timing to execute processing. Also good. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present invention.

  The program according to the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. Such a program incorporated in another program can also be included in the program according to the present invention.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  11, 21, 31 ROM, 12, 22, 32 RAM, 13 Scanner, 14 Printer, 15, 25 Operation panel, 16, 29 Memory, 17, 34 Communication controller, 18, 35 Network controller, 23 Electronic compass, 24 Controller, 26 camera, 27 first communication controller, 28 second communication controller, 51 touch panel, 52 operation key group, 100, 100A, 100B MFP, 101, 201 instruction input unit, 102 position information input unit, 103 address input unit, 104 position Information storage unit, 105 Communication information storage unit, 106 Function management unit, 107 Detection unit, 108 Update unit, 109 Screen transmission unit, 110 Instruction identification unit, 111 Login information input unit, 112 Authentication unit, 200 Mobile terminal, 202 Location acquisition Part 203 Acquisition unit, 204 image acquisition unit, 205 server request unit, 206 information acquisition unit, 207 generation unit, 207A determination unit, 207B area information, 208 display processing unit, 209 screen acquisition unit, 210,305 transmission unit, 300 server, 301 Information input unit 302 Information storage unit 303 Portable information input unit 304 Search unit 521 Numeric keypad 522 Start key 523 Mode selection key 523A Display lamp.

Claims (14)

A portable terminal, an image processing device including processing means, and an information processing device;
The portable terminal is
Photographing means;
First acquisition means for acquiring position information and orientation information of the mobile terminal;
Display means;
Input means for inputting instructions to the operation screen displayed on the display means,
The information processing apparatus includes, as information relating to the image processing apparatus, position information of the image processing apparatus, display information for screen display including choices of functions that are installed and usable, and a mobile that performs effective screen display. Storage means for storing display area information that defines a range to which the terminal position belongs,
The portable terminal transmits position information and orientation information at the time of photographing with the photographing means to the information processing device,
The information processing device detects an image processing device included in a photographed image by the photographing unit of the portable terminal based on position information and orientation information at the time of photographing with the portable terminal, and information related to the image processing device To the mobile device,
The mobile terminal uses the information related to the image processing apparatus belonging to the range specified in the display area information, and the captured image by the imaging unit, among the information related to the received image processing apparatus, An image processing system further comprising processing means for causing the display means to display an operation screen combined with the display information and accepting selection of the function by the input means. The processing means of the mobile terminal is:
A process of determining whether or not the mobile terminal belongs to a range defined in the display area information from the position information and the display area information;
The image processing system according to claim 1, wherein when it is determined that the portable terminal belongs to the range, a process of generating the operation screen by combining the captured image and the display information is executed.   The processing unit of the portable terminal repeats the process of determining and the process of generating the operation screen after displaying the operation screen on the display unit, and updates the operation screen according to the result of the determination The image processing system according to claim 2.   When the processing means of the mobile terminal determines that the mobile terminal does not belong to the range defined in the display area information in the determination processing executed after the operation screen is displayed on the display means The image processing system according to claim 3, wherein, in the process of generating the operation screen, the display information displayed on the operation screen is grayed out and combined with the captured image.   The processing means of the portable terminal moves the portable terminal out of the range defined in the display area information from the position information and the display area information after displaying the operation screen on the display means. Further processing is performed to determine whether or not there is a possibility of being performed, and a warning is displayed when it is determined that the mobile terminal may move out of the range specified in the display area information The image processing system according to claim 1, wherein the processing is executed. A storage device for storing entry information for each user;
The portable terminal is
Further including second acquisition means for acquiring position information of the mobile terminal by acquiring the room entry information of the user associated with the mobile terminal from the storage device;
The processing means determines whether or not the mobile terminal belongs to a range defined in the display area information from the position information and the display area information acquired by the second acquisition means. The image processing system in any one of -5.   The said display area information receives designation | designated of a position on the map of the installation place of the said image processing apparatus memorize | stored beforehand, and is prescribed | regulated by the said designated position. Image processing system. A control method for the image processing apparatus in an image processing system including a mobile terminal including an imaging unit and a display unit, an image processing apparatus, and an information processing apparatus,
The information processing apparatus includes, as information relating to the image processing apparatus, position information of the image processing apparatus, display information for screen display including options of functions that are installed and usable, and a position of the mobile terminal that performs the screen display And display area information that defines the range to which
The portable terminal transmitting position information and orientation information at the time of photographing by the photographing means of the portable terminal to the information processing apparatus;
In the information processing device, based on the position information and orientation information received from the portable terminal and the position information included in the information related to the image processing device, the information is included in a photographed image by the photographing unit of the portable terminal. Extracting an image processing device and transmitting information about the extracted image processing device to the mobile terminal;
In the portable terminal, out of the information related to the received image processing apparatus, the portable terminal uses information related to the image processing apparatus belonging to the range defined in the display area information, Displaying on the display means an operation screen obtained by combining the display information;
In the portable terminal, when receiving an instruction for the operation screen, transmitting a signal representing the instruction to the image processing device;
A control method for an image processing apparatus, comprising: executing corresponding image processing based on the signal in the image processing apparatus. Photographing means;
First acquisition means for acquiring position information and orientation information of the mobile terminal;
Display means;
An input unit for inputting an instruction to the operation screen displayed on the display unit;
Processing means,
The processing means includes
Processing for transmitting position information and orientation information at the time of photographing by the photographing means to the information processing device;
Of the information related to the image processing device received from the information processing device, the mobile terminal uses information related to the image processing device belonging to the range specified in the display area information, and A process for causing the display means to display an operation screen obtained by combining the display information;
A portable terminal that executes a process of transmitting a signal representing the instruction to the image processing apparatus when an instruction to the operation screen is received by the input means. Second acquisition means for acquiring the location information of the mobile terminal by acquiring the user's entrance information associated with the mobile terminal from a storage device for storing the entrance information for each user Prepared,
The processing means determines whether or not the mobile terminal belongs to a range defined in the display area information from the position information and the display area information acquired by the second acquisition means. The portable terminal as described in. As information about the image processing device, position information of the image processing device, display information for screen display including options of functions that are installed and usable, and a range to which the position of the mobile terminal that performs the screen display belongs are specified. Storage means for storing display area information;
When receiving the position information and the orientation information at the time of photographing with the photographing means of the portable terminal from the portable terminal, based on the position information and the orientation information and the position information included in the information related to the image processing device, the portable terminal An information processing apparatus comprising: processing means for executing processing for extracting an image processing device included in a photographed image by the photographing means and transmitting information relating to the extracted image processing device to the portable terminal .   12. The image processing apparatus according to claim 11, further comprising a setting unit that stores in advance a map of an installation location of the image processing apparatus, accepts designation of a position on the map, and sets the display area information according to the designated position. The information processing apparatus described. A control program for causing a mobile terminal to execute processing,
The portable terminal includes photographing means and display means,
Transmitting position information and orientation information at the time of photographing by the photographing means to the information processing device;
Of the information related to the image processing device received from the information processing device, the mobile terminal uses information related to the image processing device belonging to the range specified in the display area information, and Displaying the operation screen combined with the display information on the display means;
A control program for causing the portable terminal to execute a step of transmitting a signal representing the instruction to the image processing apparatus when an instruction to the operation screen is received by the input means. A control program for causing an information processing apparatus to execute processing,
The information processing apparatus includes, as information relating to the image processing apparatus, position information of the image processing apparatus, display information for screen display including options of functions that are installed and usable, and a position of the mobile terminal that performs the screen display And display area information that defines the range to which
Receiving position information and orientation information at the time of photographing with the photographing means of the portable terminal from the portable terminal;
Extracting an image processing device included in a photographed image of the photographing means of the mobile terminal based on the positional information and the orientation information and the positional information included in the information related to the image processing device;
The control program which makes the said information processing apparatus perform the step which transmits the information regarding the extracted said image processing apparatus with respect to the said portable terminal.

Images