JP2013046379A - Image processing system, image processing apparatus, and display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system capable of properly assisting the maintenance of an image processing apparatus.SOLUTION: A CPU 10 of an MFP 100 comprises: a tag data generator 103 for identifying a position in terms of a maintenance procedure on the basis of an instruction inputted from a portable terminal and generating tag data according to the maintenance procedure from tag information composed to an image taken by the portable terminal; and a screen data generator 105 for generating screen data by switching a transmission-target content from the tag data to the screen data to be consecutively reproduced along the maintenance procedure, in accordance with the positional relation between the position/orientation of the portable terminal and a maintenance-target component.

Description

  The present invention relates to an image processing system, an image processing apparatus, and a display method, and more particularly to an image processing system, an image processing apparatus, and a display method that process information related to maintenance of the image processing apparatus.

  In office environments and the like, image forming apparatuses such as copiers, printers, and MFPs (Multi-Functional Peripherals) that are multifunctional devices of these are widely used.

  As a method for facilitating maintenance of these image forming apparatuses, for example, a method of assisting camera repair disclosed in Japanese Patent Laid-Open No. 2002-176571 (Patent Document 1) can be applied. Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique in which an animation image for maintenance of a camera is reproduced on the display device when the HDD in the camera is taken out and connected to another display device in the repair mode. When this technology is applied, for example, it may be possible to display an animation image for assisting maintenance of the image forming apparatus on the portable terminal by communicating the portable terminal carried by the service person with the image forming apparatus.

JP 2002-176571 A

  By applying the method disclosed in Patent Document 1 as described above, the serviceman can perform maintenance with reference to the displayed animation image or the like, but the maintenance skill of the serviceman is low, for example. In this case, it may be difficult to perform maintenance according to the displayed image. In other words, since this method does not assist according to the actual maintenance, the actual maintenance can be dissociated from the displayed content. Therefore, there arises a problem that proper maintenance may not be performed.

  The present invention has been made in view of such problems, and an object thereof is to provide an image processing system, an image processing apparatus, and a display method capable of appropriately assisting maintenance.

  In order to achieve the above object, according to an aspect of the present invention, an image processing system includes a portable terminal including an imaging unit, a display unit, and an input unit, an image processing device, a portable terminal, and an image processing device. And a server capable of communication. The image processing apparatus includes a first storage unit for storing position information of the image processing apparatus, a second storage unit for storing the position of a maintenance target component of the image processing apparatus, and the image processing apparatus. Generating means for generating content related to maintenance at the terminal, transmitting means for transmitting the content to the mobile terminal, and control means for controlling transmission of the content to the mobile terminal. The content includes image data representing the content of maintenance for each maintenance procedure and information for specifying the maintenance target component, and includes the first content synthesized with the captured image on the portable terminal, Second content that is continuously played back along the procedure. The control means specifies a position on the maintenance procedure based on an instruction input from the portable terminal, transmits the first content corresponding to the position to the image processing apparatus, the position and orientation of the portable terminal, and maintenance. A process of switching the content to be transmitted from the first content to the second content and transmitting the second content to the image processing device according to the positional relationship with the target component is executed.

  Preferably, in the process of transmitting the second content to the image processing apparatus, the control unit determines the first content based on the position on the maintenance procedure specified in the process of transmitting the first content to the image processing apparatus. Processing for specifying the playback start point of the second content switched from the content is performed, and the second content is transmitted to the image processing apparatus from the playback start point.

  Preferably, the server according to the first storage means for storing the position of the image processing device, the acquisition means for acquiring the position and orientation of the mobile terminal, and the position and orientation at the time of shooting with the mobile terminal The image processing apparatus included in the shooting range of the portable terminal is specified as a content transmission target, and processing means for performing processing for requesting the image processing apparatus to transmit the content is provided.

  Preferably, the portable terminal includes position information acquisition means for acquiring its own position information.

  Preferably, the image processing apparatus includes position information acquisition means for acquiring its own position information.

  Preferably, the control unit switches the first content related to maintenance to the second content and transmits the second content to the portable terminal when the maintenance target part is not included in the photographing range of the portable terminal.

  Preferably, the control unit transmits, to the portable terminal, screen data that prompts the user to change the position prior to transmission of the first content when the imaging target range of the portable terminal does not include a maintenance target component.

  According to another aspect of the present invention, the image processing apparatus is an image processing apparatus capable of communicating with a portable terminal including a photographing unit, a display unit, and an input unit, for storing position information of the image processing apparatus. First storage means, second storage means for storing the position of a maintenance target part of the image processing apparatus, generation means for generating content related to maintenance in the image processing apparatus, and portable content The transmission means for transmitting to a terminal and the control means for controlling transmission to the portable terminal of a content are included. The content includes image data representing the content of maintenance for each maintenance procedure and information for specifying the maintenance target component, and includes the first content synthesized with the captured image on the portable terminal, Second content that is continuously played back along the procedure. The control unit specifies a position on a maintenance procedure to be transmitted based on an instruction input from the portable terminal, transmits a first content corresponding to the position to the image processing apparatus, and a position of the portable terminal In accordance with the positional relationship between the orientation and the part to be maintained, a process of switching the content to be transmitted from the first content to the second content and transmitting the second content to the image processing apparatus is executed.

  According to still another aspect of the present invention, the display method is a method for displaying content related to maintenance of the image processing apparatus on a portable terminal including a photographing unit and a display unit, and the content is a content of maintenance for each maintenance procedure. Image data to be represented and information for specifying the part to be maintained, the first content to be combined with the photographed image on the portable terminal, and the second to be continuously reproduced along the maintenance procedure And transmitting the position and orientation at the time of shooting on the portable terminal from the portable terminal to the image processing apparatus existing in the shooting range of the portable terminal, and inputting instructions from the portable terminal in the image processing apparatus Based on the maintenance procedure, and sends the first content according to the position to the image processing apparatus. In the portable terminal, in which the first content is combined with the captured image on the portable terminal and displayed, and in the image processing apparatus, the positional relationship between the position and orientation of the portable terminal and the maintenance target component Accordingly, the step of switching the content to be transmitted from the first content to the second content and transmitting the second content to the image processing device, and the display of the first content combined with the photographed image on the portable terminal And starting the reproduction of the second content.

  According to still another aspect of the present invention, the display program is a program for causing the image processing apparatus to execute a process of displaying content related to maintenance of the image processing apparatus on a portable terminal including a photographing unit and a display unit. , Including image data representing the content of maintenance for each maintenance procedure and information for specifying the part to be maintained, the first content to be combined with the captured image on the portable terminal, and the maintenance procedure And the second content that is played back continuously, and when the position and orientation at the time of shooting on the mobile terminal are received from the mobile terminal, the position on the maintenance procedure is specified based on the instruction input from the mobile terminal A step of transmitting the first content corresponding to the position to the image processing apparatus, and a portable terminal The step of switching the content to be transmitted from the first content to the second content and transmitting the second content to the image processing device in accordance with the positional relationship between the position and orientation and the part to be maintained. To run.

  According to this invention, since the display according to the maintenance work is made, the maintenance work can be assisted appropriately.

It is a figure which shows the specific example of a structure of the image processing system concerning embodiment. 2 is a diagram illustrating a specific example of a hardware configuration of an MFP included in an image processing system. It is a figure which shows the specific example of the hardware constitutions of the portable terminal contained in an image processing system. It is a figure which shows the specific example of the hardware constitutions of the server contained in an image processing system. It is a figure showing the flow of information in an image processing system. It is a figure showing the operation | movement outline | summary in an image processing system. 2 is a block diagram illustrating a specific example of a functional configuration of an MFP. FIG. It is a block diagram which shows the specific example of a function structure of a portable terminal. It is a block diagram which shows the specific example of a function structure of a server. It is a flowchart showing the specific example of the flow of operation | movement in a server. It is a flowchart showing the specific example of the flow of operation | movement with a portable terminal. 3 is a flowchart showing an operation flow in the MFP. It is a figure which shows the specific example of a display screen. It is a figure which shows the specific example of a display screen. It is a figure which shows the specific example of a display screen. It is a figure which shows the specific example of a display screen.

  Embodiments of the present invention will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same.

<System configuration>
FIG. 1 is a diagram showing a specific example of the configuration of the image processing system according to the present embodiment.

  Referring to FIG. 1, the image processing system according to the present embodiment includes an MFP (Multi-Functional Peripheral) 100A, a portable terminal 200, and a server 300 as image processing apparatuses.

  The image processing apparatus is not limited to the MFP, and may be other apparatuses such as a printer, a facsimile transceiver, and a copier. The MFP 100 is an image processing apparatus having a combination of these functions.

  The mobile terminal 200 may be a mobile phone, for example, and may be another device as long as it has at least a camera function, an instruction input function, and a communication function.

  The server 300 may be configured by a general personal computer or the like.

  In the following description, as shown in FIG. 1, server 300 is included separately from MFP 100, but server 300 may be included in MFP 100 and MFP 100 may function as a server.

  MFP 100 and server 300 can communicate with each other and exchange information. Communication between MFP 100 and server 300 may be in any communication form, whether wired or wireless. Since the image processing system according to the present embodiment is assumed to be used in an office environment, as an example, MFP 100 and server 300 are connected to a wired or wireless LAN (Local Area Network), and the network is connected via the network. Communication mode for exchanging information.

  The portable terminal 200 and the server 300 can also communicate with each other and exchange information. The communication between the portable terminal 200 and the server 300 may be any communication form as long as it is wireless communication. In the following examples, wireless communication using a telephone line is assumed. However, infrared communication, short-range wireless communication, or the like may be used, or the portable terminal 200 and the server 300 are connected to a wireless LAN. If there is, a communication form in which information is exchanged via the network is also exemplified.

  In the following description, MFP 100 and server 300 communicate with each other via LAN, and portable terminal 200 and server 300 perform wireless communication using a telephone line, that is, these are different communication forms. However, the same communication form may be used, for example, communication via a LAN.

<Device configuration>
FIG. 2 is a diagram illustrating a specific example of the hardware configuration of the MFP 100.

  Referring to FIG. 2, MFP 100 includes a CPU (Central Processing Unit) 10 that is an arithmetic device for overall control, a ROM (Read Only Memory) 11 for storing a program executed by CPU 10, and the like. A RAM (Random Access Memory) 12 for functioning as a work area when the CPU 10 executes the program, a scanner 13 for optically reading a document placed on a document table (not shown), and obtaining image data; A printer 14 for fixing image data on printing paper, an operation panel 15 including a touch panel for displaying information and receiving operation inputs to the MFP 100, and a memory 16 for storing image data A network controller 17 for controlling communication via the LAN and the state of the MFP 100 The and a sensor 18 for detecting.

  The operation panel 15 includes a touch panel (not shown) and an operation key group. The touch panel is configured by overlapping a display device such as a liquid crystal display device and a position indicating device such as an optical touch panel or a capacitive touch panel, and displays an operation screen to specify an indicated position on the operation screen. CPU10 displays an operation screen on a touchscreen based on the data for displaying the screen memorize | stored beforehand.

  As the sensor 18, for example, a sensor for detecting the remaining amount of toner, a sensor for detecting jamming (paper jam), temperature abnormality, operation abnormality, etc. of each part, or the presence / absence of mounting of each component is detected. For example, a sensor for this purpose.

  The memory 16 further stores component information that is information for each component that is a maintenance target, maintenance information that is information related to maintenance, and information (position information) that indicates the position where the MFP 100 is installed. . When the MFP 100 has a position detection function (not shown), the position where the MFP 100 is installed may be acquired by the function or may be input from the operation panel 15. The position for each part may be input in advance, or may be detected and registered by the sensor 18 when the part is mounted. Here, the position where MFP 100 is installed may be registered as an absolute position such as latitude and longitude as an example, and the position of each part is defined by, for example, one corner of operation panel 15 as the origin. The position may be registered with coordinates from the origin. By specifying the absolute position of the origin, the absolute position for each part is also specified.

  Maintenance information includes tag information used to generate tag data, which is data for displaying tags, which will be described later, and screen data for continuously displaying the maintenance contents in accordance with the maintenance procedure with videos, animations, etc. And image information used to generate.

  The tag information indicates information indicating the maintenance procedure, the maintenance content in the procedure, and the parts to be subjected to the maintenance content for each maintenance. The maintenance content is an image displayed as a tag, which will be described later, and is displayed to assist maintenance. As will be described later, an image (tag) representing the content of the captured image of the MFP 100 is displayed. It is added and displayed.

FIG. 3 is a diagram illustrating a specific example of the hardware configuration of the mobile terminal 200.
Referring to FIG. 3, portable terminal 200 includes a CPU 20 that is an arithmetic device for overall control, a ROM 21 for storing a program executed by CPU 20, and a work area when CPU 20 executes the program. RAM 22 for functioning as an electronic device, an electronic compass 23 for detecting the orientation of the portable terminal 200 including a magnetic sensor, a GPS (Global Positioning System) signal or a position signal from a base station, and the portable terminal A GPS controller 24 for acquiring the position information, an operation panel 25 including a touch panel for displaying information and receiving an operation input to the portable terminal 200, a camera 26, and a base station (not shown). A controller 27 for controlling communication via a telephone line, and a telephone directory And a memory 28 for storing the data.

  Operation panel 25 may have the same configuration as operation panel 15 of MFP 100. That is, as an example, a touch panel configured by overlapping a display device such as a liquid crystal display device and a position indicating device such as an optical touch panel or a capacitive touch panel, and an operation key are included.

  The CPU 200 displays an operation screen on the touch panel based on data for displaying a screen stored in advance. On the touch panel, an indicated position on the operation screen is specified, and an operation signal indicating the position is input to the CPU 20. An operation signal indicating the pressed key is input to the CPU 20. The CPU 20 specifies the operation content from the pressed key or the displayed operation screen and the indicated position, and executes processing based on the specified operation content.

  In addition, the electronic compass 23 and the GPS controller 24 input the acquired signal indicating the orientation and position information of the mobile terminal 200 to the CPU 20.

FIG. 4 is a diagram illustrating a specific example of the hardware configuration of the server 300.
Referring to FIG. 4, server 300 is configured by a general computer or the like as described above. As an example, CPU 30 that is an arithmetic device for overall control, a program executed by CPU 30, and the like ROM 31 for storing data, RAM 32 for functioning as a work area when CPU 30 executes a program, HD (Hard Disk) 33 for storing MFP information, and the like, and controlling communication via a telephone line A communication controller 34 for performing communication, and a network controller 35 for controlling communication via the LAN.

  The HD 33 stores location information of the MFP 100 as information related to the MFP 100. This position information may be stored by being input in advance, or may be transmitted from MFP 100 and stored.

<Overview of operation>
FIG. 5 is a diagram showing a flow of information in the image processing system according to the present embodiment.

  Referring to FIG. 5, portable terminal 200 transmits information representing the position and orientation when imaged with camera 26 to server 300. Also, an instruction to update the display and an instruction to switch the display from a tag to a maintenance screen such as animation are transmitted to the server 300. Server 300 requests display data from MFP 100 in response to a request from portable terminal 200.

  The MFP 100 generates display data in response to a request from the server 300 and transmits the display data to the server 300. The server 300 transfers the received display data to the mobile terminal 200.

  In the portable terminal 200, when the display data is data for displaying a tag, the tag, which is an image represented by the display data, is combined with the captured image and displayed. If the display data is data for displaying a maintenance screen such as an animation, the maintenance screen represented by the display data is displayed.

FIG. 6 is a diagram showing an outline of the operation in the image processing system according to the present embodiment.
Referring to FIG. 6, when a maintenance application is activated on the portable terminal (step S11) and the camera 26 is photographed in accordance with the photographing instruction (step S12), information indicating the position and orientation at the time of photographing is transmitted to the server. Thus, tag data is requested from the server (step S13).

  The server stores information representing the received position and orientation of the mobile terminal (step S31). Then, from the stored MFP position information, an MFP existing in the shooting range of the mobile terminal is specified (step S32), and information indicating the position and orientation of the mobile terminal is transmitted to the MFP. The corresponding tag data is requested (step S33).

  The MFP reads the tag information from the maintenance information related to the target maintenance, and generates tag data (step S51). Here, the first tag data is generated in order according to the display order of the tags to be displayed in order according to the maintenance procedure, and transmitted to the server (step S52). The server transfers the received tag data to the mobile terminal (step S34).

  The portable terminal displays the tag based on the tag data superimposed on the captured image (step S14). On the screen on which the tag is displayed, for example, a “Next” button or the like for updating to display the tag for the next maintenance procedure is displayed. Then, when an update instruction is accepted by the button or the like (step S15), the operation returns from the step S13. Every time there is an instruction to update the tag display, the operation from step S13 is repeated, and the tags are sequentially displayed on the captured image in accordance with the update instruction according to the maintenance procedure.

  Here, it is assumed that the portable terminal is provided with means for instructing switching of the display from the tag display to a maintenance screen such as animation. This means may be, for example, a switching button provided on the screen on which the tag is displayed, or may be a key (not shown).

  When receiving an instruction to switch the display from the tag display to the maintenance screen such as animation (step S16), the mobile terminal requests screen data for displaying the maintenance screen from the server (step S17). The server transfers the request to the MFP (step S35).

  Upon receiving this request, the MFP executes a switching process for switching from transmission of tag data to transmission of screen data (step S53). Specifically, referring to the maintenance procedure indicated in the tag data last transmitted to the server, the corresponding portion of the maintenance screen that continuously displays the maintenance content is displayed at the reproduction start position. As specified. Then, screen data is generated based on the image information so that reproduction is started from that position, and transmitted to the server (step S54). The server transfers the received screen data to the mobile terminal (step S36).

  In the portable terminal, the received screen data is reproduced (step S18). As a result, the maintenance contents from the procedure following the maintenance procedure represented by the tag displayed at the end are sequentially reproduced in sequence with the screen data.

<Functional configuration>
FIG. 7 is a block diagram showing a specific example of a functional configuration of MFP 100 for realizing the above-described operation. Each function shown in FIG. 7 is a function mainly formed in the CPU 10 when the CPU 10 reads a program stored in the ROM 11 and executes it on the RAM 12. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 7, in memory 16, a component information storage unit 161 for storing component information that is information for each component, and a maintenance information storage unit 162 for storing maintenance information that is information related to maintenance, Is included.

  Further, referring to FIG. 7, MFP 100 receives an input unit 101 for receiving a data request from the server via network controller 17, and when the request is a request for tag data, it is transmitted together with the request. A determination unit 102 for determining whether or not the position and orientation of the mobile terminal are directed toward a maintenance target component, and a tag for generating tag data by referring to tag information according to the determination A data generation unit 103; a specifying unit 104 for specifying a reproduction start position when the request is a request for maintenance screen data; and a screen for generating screen data from the reproduction start position with reference to image information In order to transmit data to the server via the data generation unit 105 and the network controller 17 And a transmission unit 106.

  FIG. 8 is a block diagram illustrating a specific example of a functional configuration of the mobile terminal 200 for realizing the above-described operation. Each function shown in FIG. 8 is a function mainly formed in the CPU 20 when the CPU 20 reads a program stored in the ROM 21 and executes it on the RAM 22. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 8, as a function for realizing the above operation, portable terminal 200 includes an instruction input unit 201 for receiving an instruction input from operation panel 25, and a GPS controller according to a shooting instruction from operation panel 25. 24, the position acquisition unit 202 for acquiring the position of the mobile terminal 200 from the signal obtained from the signal 24, and the orientation of the mobile terminal 200 from the signal obtained from the electronic compass 23 in response to a shooting instruction from the operation panel 25. An orientation acquisition unit 203 for acquisition, an image acquisition unit 204 for acquiring image data captured by executing shooting by the camera 26 in response to a shooting instruction from the operation panel 25, and shooting from the operation panel 25 Information for transmitting the position and orientation acquired in response to the instruction to the server 300 via the controller 27 And a tag data input unit 206 for accepting input of tag data transmitted from the server 300 via the controller 27, and a tag based on the tag data is synthesized with image data obtained by photographing. A generation unit 207 for generating display data, a screen data input unit 209 for receiving input of screen data transmitted from the server 300 via the controller 27, and a screen based on the display data or screen data. And a display processing unit 208 for executing processing to be displayed on the operation panel 25. Further, the transmission unit 205 transmits a request according to an instruction to update the tag from the operation panel 25 or an instruction to switch the display of the tag to the maintenance screen via the controller 27 to the server. Note that the position acquisition unit 202 and the orientation acquisition unit 203 may acquire the position and orientation of the mobile terminal when the instruction is given, and the transmission unit 205 may transmit the request together with the request.

  FIG. 9 is a block diagram illustrating a specific example of a functional configuration of the server 300. Each function shown in FIG. 9 is a function mainly formed in the CPU 30 when the CPU 30 reads a program stored in the ROM 31 and executes it on the RAM 32. However, at least a part of the functions may be formed by the hardware configuration shown in FIG.

  Referring to FIG. 9, HD 33 stores MFP information storage unit 331 that is a storage area for storing the position of the MFP, and the position and orientation of mobile terminal 200 transmitted from mobile terminal 200. A mobile terminal information storage unit 332 that is a storage area is included. In addition, a storage area for storing tag data and screen data transmitted from the MFP may be included.

  Further, referring to FIG. 9, as a function for realizing the above-described operation, server 300 receives a request for display data together with a position and a direction at the time of shooting from portable terminal 200 via communication controller 34. Section 301, storage section 302 for storing the position and orientation as the position and orientation of portable terminal 200 in portable terminal information storage section 332, the position and orientation of portable terminal 200, and the stored MFP position , A specifying unit 306 for specifying an MFP within the shooting range of the portable terminal 200 as a display data request destination, and a second input for receiving input of tag data or screen data from the MFP 100 via the network controller 35 Unit 303 and its display data via communication controller 34 A first transmitting unit 304 for transmitting to the terminal 200 and a first unit for transferring the position and orientation from the portable terminal 200 to the MFP 100 specified via the network controller 35 and requesting display data. 2 transmission unit 305.

<Operation flow>
FIG. 10 is a flowchart showing a specific example of the operation flow in server 300. The operation shown in the flowchart of FIG. 10 is realized by the CPU 30 reading out a program stored in the ROM 31 and executing it on the RAM 32 to exhibit each function shown in FIG.

  Referring to FIG. 10, when CPU 30 of server 300 receives a request for display data from portable terminal 200 (YES in step S101), CPU 30 stores the position and orientation of the portable terminal included in the request (step S103). . Here, even if the tag data is requested and the update is requested or the screen data for assist screen display is requested, the position at the time of request from the portable terminal 200 and The direction shall be transmitted.

  The CPU 30 compares the position and orientation of the mobile terminal with the stored MFP position, and identifies the MFP existing in the shooting range of the mobile terminal. As a result, when an MFP existing within the shooting range is specified (YES in step S105), the CPU 30 requests display data from the MFP (step S107).

  When the display data transmitted from MFP 100 in response to the request is received (YES in step S109), CPU 30 transfers the display data to portable terminal 200 in step S111.

  In FIG. 10, the subsequent operation is skipped when the MFP 100 does not exist in the shooting range of the mobile terminal 200. As another example, an MFP closest to the shooting range may be extracted and transmitted to the MFP. In this case, screen data may be requested.

  Alternatively, when the update of the tag is instructed and the MFP 100 does not exist in the shooting range of the mobile terminal 200, the screen data may be requested to the MFP 100 instead of the request for the tag data. . By doing in this way, even when the shooting range of the portable terminal 200 is out of the MFP 100, the portable terminal 200 performs a display for assisting the maintenance related to the nearest MFP 100. Further, when the MFP 100 is no longer included in the photographed image, an assist screen such as animation is displayed instead of the tag.

  FIG. 11 is a flowchart showing a specific example of the flow of operations in portable terminal 200. The operation shown in the flowchart of FIG. 11 is realized by the CPU 20 reading a program stored in the ROM 21 and executing it on the RAM 22 to exhibit each function shown in FIG.

  Referring to FIG. 11, when CPU 20 of mobile terminal 200 receives a shooting instruction from operation panel 25 in a state where an assisting application is activated (YES in step S301), it executes a shooting operation in step S303, and In step S305, the tag data is requested by transmitting the current position and orientation to the server 300.

  When the tag data transmitted in response to the request is received (YES in step S307), in step S309, the CPU 20 combines the tag represented by the received tag data with the photographed image obtained in the photographing in step S303. Is displayed on the operation panel 25.

  When an instruction to update the tag is received (YES in step S311), the CPU 20 returns the operation to step S305, and transmits the position and orientation of the mobile terminal 200 at that time to the server 300, whereby tag data Request an update.

  When an instruction to switch from the tag display to the assist screen display is received (YES in step S313), the CPU 20 requests screen data from the server 300 in step S315.

  When the screen data transmitted in response to the request is received (YES in step S317), in step S319, the CPU 20 reproduces the received screen data and displays an assist screen such as a moving image or animation on the operation panel 25.

  FIG. 12 is a flowchart showing an operation flow in MFP 100. The operation shown in the flowchart of FIG. 12 is realized by the CPU 10 reading out a program stored in the ROM 11 and executing it on the RAM 12 to exhibit each function shown in FIG.

  Referring to FIG. 12, when CPU 10 of MFP 100 receives a request for tag data from server 300 (YES in step S501), it compares the position and orientation of portable terminal 200 included in the request with the position of the part to be maintained. Then, it is determined whether or not the portable terminal 200 exists within the maintenance work range. As a result, if it is determined that mobile terminal 200 is present within the work range and is facing the work direction (YES in step S503), in step S505, CPU 10 determines the tag data for the maintenance based on the tag information. And transmit it to the server 300. At this time, if the request in step S501 is to update the tag data, the tag data corresponding to the next maintenance procedure of the previously transmitted tag data is created and transmitted.

  On the other hand, when the mobile terminal 200 does not exist within the work range, or when it is determined that the mobile terminal 200 exists but does not face the work direction (NO in step S503), these positions are compared in step S503. Thus, the moving direction is specified, display data for displaying a message for urging movement in that direction is created and transmitted to the server 300. This is created by a method such as substituting the moving direction specified in the format stored in advance.

  Alternatively, in this case, the screen data of the tag data transmitted earlier may be transmitted to the server 300 instead of the tag data. At this time, the CPU 10 refers to the position on the maintenance procedure indicated in the tag data last transmitted to the server 300, and reproduces the corresponding portion of the maintenance screen that continuously displays the maintenance contents. Identifies as the starting position. Then, screen data is generated based on the image information so that reproduction is started from that position, and is transmitted to the server 300.

  When the screen data request is received from the server 300 (YES in step S509), in step S511, the CPU 10 specifies the position in the next maintenance procedure based on the tag data last transmitted in step S505, and stores the image information. A position corresponding to the position on the maintenance procedure is specified as the reproduction start position. In step S 513, the CPU 10 creates screen data from which playback is started, and transmits the screen data to the server 300.

<Effect of Embodiment>
When such an operation is performed in the image forming system according to the present embodiment, the tag is displayed on the captured image on the mobile terminal in accordance with the maintenance progress of the service person who carries the mobile terminal. It will be. Therefore, the service person can perform maintenance while referring to the tag according to his / her progress, and can perform maintenance efficiently.

  For example, when the service person carrying the portable terminal is not properly at the work position, as shown in FIG. 13, a tag for guiding to the work position is displayed on the captured image.

  When the maintenance work is started, a tag representing the maintenance content in the maintenance procedure as shown in FIG. 14 is displayed superimposed on the photographed image. The serviceman can perform the maintenance procedure by referring to the tag and performing the maintenance work at the location indicated by the tag.

  Then, when the maintenance contents are executed, the maintenance contents of the next maintenance procedure shown in FIG. 15 are displayed by instructing to update the tag, for example, by pressing the “Next” button shown in FIG. The represented tag is displayed superimposed on the photographed image. In the same way as in the screen of FIG. 15, when the tag is instructed to be updated by pressing the “Next” button or the like, the tag representing the maintenance content of the next maintenance procedure shown in FIG. Is done. In this way, the service person can update the tag according to his / her maintenance progress, and can efficiently perform maintenance.

  Also, when it is difficult to perform maintenance while shooting with a mobile terminal, such as when using both hands, for example, pressing the “animation switching” button shown in FIG. 15 or FIG. By instructing switching, the screen is switched from the display in which the tag is combined with the photographed image to the assist screen for animation or the like. At this time, if the maintenance is already performed with the tag according to the maintenance procedure, the assist screen is reproduced in order from the subsequent maintenance procedure, so that the maintenance can be performed efficiently.

  Furthermore, it is possible to provide a program for causing the MFP 100, the portable terminal 200, and the server 300 to execute the above-described operation. Such a program is stored in a computer-readable recording medium such as a flexible disk attached to the computer, a CD-ROM (Compact Disk-Read Only Memory), a ROM (Read Only Memory), a RAM (Random Access Memory), and a memory card. And can be provided as a program product. Alternatively, the program can be provided by being recorded on a recording medium such as a hard disk built in the computer. A program can also be provided by downloading via a network.

  The program according to the present invention is a program module that is provided as a part of a computer operating system (OS) and calls necessary modules in a predetermined arrangement at a predetermined timing to execute processing. Also good. In that case, the program itself does not include the module, and the process is executed in cooperation with the OS. A program that does not include such a module can also be included in the program according to the present invention.

  The program according to the present invention may be provided by being incorporated in a part of another program. Even in this case, the program itself does not include the module included in the other program, and the process is executed in cooperation with the other program. Such a program incorporated in another program can also be included in the program according to the present invention.

  The provided program product is installed in a program storage unit such as a hard disk and executed. The program product includes the program itself and a recording medium on which the program is recorded.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  10, 20, 30 CPU, 11, 21, 31 ROM, 12, 22, 32 RAM, 13 Scanner, 14 Printer, 15, 25 Operation panel, 16, 28 Memory, 17, 35 Network controller, 18 Sensor, 23 Electronic compass , 24 GPS controller, 26 camera, 27 controller, 34 communication controller, 100 MFP, 101 input unit, 102 determination unit, 103 tag data generation unit, 104,306 identification unit, 105 screen data generation unit, 106,205 transmission unit, 161 parts information storage unit 162 maintenance information storage unit 200 portable terminal 201 instruction input unit 202 position acquisition unit 203 orientation acquisition unit 204 image acquisition unit 206 tag data input unit 207 generation unit 208 table Processing unit, 209 Screen data input unit, 300 server, 301 first input unit, 302 storage unit, 303 second input unit, 304 first transmission unit, 305 second transmission unit, 331 MFP information storage unit, 332 portable terminal information Memory part.

Claims (10)

A portable terminal including a photographing unit, a display unit, and an input unit;
An image processing device;
A server capable of mutual communication with the portable terminal and the image processing apparatus,
The image processing apparatus includes:
First storage means for storing position information of the image processing apparatus;
Second storage means for storing a position of a maintenance target component of the image processing apparatus;
Generating means for generating content related to maintenance in the image processing apparatus;
Transmitting means for transmitting the content to the mobile terminal;
Control means for controlling transmission of the content to the mobile terminal,
The content includes image data representing details of maintenance for each maintenance procedure and information for specifying a part to be maintained, and the first content combined with a captured image on the portable terminal; And second content continuously played in accordance with the maintenance procedure,
The control means includes
A process of identifying a position on a maintenance procedure based on an instruction input from the portable terminal, and transmitting a first content corresponding to the position to the image processing apparatus;
The content to be transmitted is switched from the first content to the second content in accordance with the positional relationship between the position and orientation of the portable terminal and the maintenance target component, and the second content is transferred to the image processing apparatus. And an image processing system for executing processing to be transmitted to.   In the process of transmitting the second content to the image processing apparatus, the control means, based on the position on the maintenance procedure specified in the process of transmitting the first content to the image processing apparatus, 2. The image processing according to claim 1, wherein processing for specifying a playback start point of the second content switched from the first content is performed, and the second content is transmitted to the image processing device from the playback start point. 3. system. The server
First storage means for storing the position of the image processing device;
Obtaining means for obtaining the position and orientation of the mobile terminal;
According to the position and orientation at the time of shooting with the mobile terminal, the image processing device included in the shooting range of the mobile terminal is identified as the transmission target of the content, and the transmission of the content to the image processing device The image processing system according to claim 1, further comprising: processing means for performing processing that requests   The image processing system according to claim 1, wherein the portable terminal includes position information acquisition means for acquiring own position information.   The image processing system according to claim 1, wherein the image processing apparatus includes position information acquisition means for acquiring own position information.   The control means switches the first content related to the maintenance to the second content and transmits it to the mobile terminal when the maintenance target part is not included in the shooting range of the mobile terminal. The image processing system according to claim 1.   The control means transmits screen data for prompting a change of position to the portable terminal prior to transmission of the first content when the imaging target range of the portable terminal does not include the maintenance target part. The image processing system according to claim 1. An image processing apparatus capable of communicating with a portable terminal including an imaging unit, a display unit, and an input unit,
First storage means for storing position information of the image processing apparatus;
Second storage means for storing a position of a maintenance target component of the image processing apparatus;
Generating means for generating content related to maintenance in the image processing apparatus;
Transmitting means for transmitting the content to the mobile terminal;
Control means for controlling transmission of the content to the mobile terminal,
The content includes image data representing details of maintenance for each maintenance procedure and information for specifying a part to be maintained, and the first content combined with a captured image on the portable terminal; And second content continuously played in accordance with the maintenance procedure,
The control means includes
A process of identifying a position on a maintenance procedure to be transmitted based on an instruction input from the portable terminal, and transmitting a first content corresponding to the position to the image processing apparatus;
The content to be transmitted is switched from the first content to the second content in accordance with the positional relationship between the position and orientation of the portable terminal and the maintenance target component, and the second content is transferred to the image processing apparatus. An image processing apparatus that executes processing to be transmitted to A method for displaying content related to maintenance of an image processing apparatus on a portable terminal including a photographing unit and a display unit,
The content includes image data representing details of maintenance for each maintenance procedure and information for specifying the maintenance target component, and the first content to be combined with the captured image on the mobile terminal; Second content continuously played along the maintenance procedure,
Transmitting the position and orientation at the time of photographing with the portable terminal from the portable terminal to the image processing existing in the photographing range of the portable terminal;
Identifying a position on a maintenance procedure based on an instruction input from the mobile terminal in the image processing apparatus, and transmitting first content corresponding to the position to the image processing apparatus;
In the portable terminal, the step of combining the first content with a photographed image of the portable terminal and displaying it,
In the image processing apparatus, the content to be transmitted is switched from the first content to the second content according to the positional relationship between the position and orientation of the mobile terminal and the maintenance target component, and the second content is switched. Transmitting content to the image processing device;
And switching the display of the first content combined with the photographed image and starting the reproduction of the second content in the portable terminal. A program for causing the image processing apparatus to execute processing for displaying content related to maintenance of the image processing apparatus on a portable terminal including a photographing unit and a display unit,
The content includes image data representing details of maintenance for each maintenance procedure and information for specifying the maintenance target component, and the first content to be combined with the captured image on the mobile terminal; Second content continuously played along the maintenance procedure,
When the position and orientation at the time of shooting with the mobile terminal are received from the mobile terminal, the position on the maintenance procedure is specified based on the instruction input from the mobile terminal, and the first content corresponding to the position is Transmitting to the image processing device;
The content to be transmitted is switched from the first content to the second content in accordance with the positional relationship between the position and orientation of the portable terminal and the maintenance target component, and the second content is transferred to the image processing apparatus. A display program causing the image processing apparatus to execute the step of transmitting to the image processing apparatus.

Images