JP2019208174A - Image processing system and server device - Google Patents

Abstract

To make it possible to provide a user with a help content for an image processing apparatus in a mode in which a burden on a producer of a content for the image processing apparatus is minimized.SOLUTION: An MFP 200, when a help button 2072 is operated, creates a screen capture of an area 2071 and transmits an image of the screen capture (capture image) to a server device 100. When receiving the capture image, the server device 100 analyzes the capture image, specifies a screen displayed on the MFP 200 on the basis of a result of the analysis, and specifies a help content corresponding to the specified screen. The server device 100 then transmits the specified help content to the MFP 200. When receiving the help content, the MFP 200 displays the help content in the area 2071 as shown at the lower right in a Figure 2.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to an image processing system, and more particularly, to an image processing system in which a server device and an image processing device cooperate to process image data.

  2. Description of the Related Art Conventionally, various techniques for using an image processing apparatus or an image of its operation screen for operation of an image processing apparatus such as an MFP (Multi-Functional Peripheral) have been proposed. For example, Japanese Patent Laying-Open No. 2013-025638 (Patent Document 1) discloses a case where an AR (Augmented Reality) marker is attached to an image processing apparatus, the image processing apparatus is photographed with a mobile terminal, and the AR marker is detected in an imaging screen. A technique for displaying an operation panel (AR operation panel) corresponding to the AR marker on the screen of the mobile terminal is proposed.

  Japanese Patent Laid-Open No. 2010-003017 (Patent Document 2) is a screen that is determined to be usable from the viewpoint of copyright or the like in a technique for obtaining a screen capture of an operation screen used for an operation manual of an information processing apparatus. It is proposed to obtain a screen capture on the condition.

  Japanese Patent Laying-Open No. 2006-045635 (Patent Document 3) discloses an image forming apparatus that displays a model name and a code for identifying the displayed operation screen together with the operation screen. When the user captures the identification code with the mobile terminal, the mobile terminal specifies the model name and operation screen of the image forming apparatus. The mobile terminal further selects help information of the specified model, extracts items that can be set on the operation screen from the help information, and displays them.

JP 2013-025638 A JP 2010-003017 A JP, 2006-045635, A

  The technique described in Patent Literature 3 can provide help information corresponding to the model of the image forming apparatus to a user who does not know the model of the image forming apparatus in front of him. However, in order to realize the technique described in Patent Document 3, the operation screen of the image processing apparatus (image forming apparatus) needs to be configured to include an identification code, which is a burden on the content producer of the image processing apparatus. Result. In addition, a burden on the content creator arises in that it is necessary to prepare all help information when releasing the content.

  The present disclosure has been conceived in view of the actual situation, and the purpose of the present disclosure is to provide help content of the image processing apparatus to the user in a manner that minimizes the burden on the content producer of the image processing apparatus. Is to do.

  According to an aspect of the present disclosure, an image processing system including an image processing device and a server device capable of communicating with the image processing device is provided. An image processing system includes a screen transmission unit that transmits an image of an operation screen of an image processing device to a server device, a help content specifying unit that specifies help content corresponding to the operation screen transmitted by the screen transmission unit, and a help content. Help content providing means for providing to the user of the image processing apparatus.

  The image processing device includes an information processing unit that captures an operation screen of the image processing device, and a communication unit that transmits the operation screen captured by the information processing unit to the server device. You may be comprised by the communication part.

  The help content providing means transmits the help content to the image processing device, transmits link information to the help content to the image processing device, or redirects the communication unit to the device that stores the help content. The content may be provided to the user of the image processing apparatus.

  The image processing system may further include a portable terminal that can communicate with the server device. The portable terminal may include an imaging unit and a communication unit that transmits an image captured by the imaging unit to the server device, and the screen transmission unit may include an imaging unit and a communication unit.

  The help content providing means transmits the help content to at least one of the mobile terminal and the image processing device, redirects the communication unit to the device storing the help content, transmits link information to the help content, or The help content may be provided to the user of the image processing device by displaying link information to the help content on the display device.

  Help content specifying means classifies the operation screen into at least one of a plurality of predetermined types based on the image of the operation screen, and specifies help content corresponding to the classified at least one type May be.

  The plurality of types are an operation screen representing the same source as the image processing apparatus and includes a first type including the original operation screen, a second type including an operation screen in which the original operation screen is changed, A third type including an operation screen representing a source different from the image processing apparatus may be included.

  The help content specifying unit may select the help content to be provided to the user of the image processing apparatus according to whether or not the image processing apparatus is in a frozen state.

  The help content specifying unit may specify the image processing device based on the location information of the mobile terminal that is the transmission source of the information on the image processing device in the frozen state or the address on the network of the image processing device.

  According to another aspect of the present disclosure, a server device is provided that includes a communication device that receives an operation screen of an image processing device, a processor, and a storage device that stores a program executed by the processor. The program causes the processor to execute a step of specifying help content corresponding to the operation screen and a step of providing the help content to the user of the image processing apparatus.

  The step of specifying the help content classifies the operation screen into at least one of a plurality of predetermined types based on the image of the operation screen, and the help content corresponding to the classified at least one type May be included.

  The plurality of types are an operation screen representing the same source as the image processing apparatus and includes a first type including the original operation screen, a second type including an operation screen in which the original operation screen is changed, A third type including an operation screen representing a source different from the image processing apparatus may be included.

  According to the present disclosure, the server device specifies help content corresponding to the operation screen of the image processing device, and provides the help content to the user of the image processing device. Accordingly, the content producer of the image processing apparatus can add the help content of the content to the server device after the content is produced. The producer does not have to prepare all the help contents at the time of producing contents of the image processing apparatus. Therefore, the burden on the producer can be reduced.

It is a figure which shows an example of a structure of an image processing system. It is a figure which shows typically an example of the provision aspect of the help content in an image processing system. It is a figure which shows typically the other example of the provision aspect of the help content in an image processing system. 2 is a diagram illustrating an example of a hardware configuration of a server device 100. FIG. 2 is a diagram illustrating an example of a hardware configuration of an MFP 200. FIG. 3 is a diagram illustrating an example of a hardware configuration of a mobile terminal 300. FIG. It is a figure for demonstrating an example of the flow of a process for a help content being provided in the aspect shown by FIG. It is a figure for demonstrating an example of the flow of a process for a help content being provided in the aspect shown by FIG. It is an example of the flowchart of the subroutine of step S20. It is a figure which shows an example of the registered image. It is a figure which shows the other example of the registered image. 6 is a diagram illustrating an example of a screen (image) included in an image transmitted from the MFP 200 or the mobile terminal 300. FIG. It is a figure which shows an example of the screen produced | generated by the registered main body. FIG. 5 is a diagram schematically illustrating an example of a help content provision mode when the MFP 200 is in a frozen state. It is a figure which shows an example of the process for implement | achieving the provision aspect of the help content shown by FIG. 10 is a diagram schematically illustrating another example of a help content provision mode when the MFP 200 is in a frozen state. FIG. It is a figure which shows an example of the process for implement | achieving the provision aspect of the help content shown by FIG.

  An embodiment of an image processing system will be described below with reference to the drawings. In the following description, the same parts and components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, these descriptions will not be repeated.

[1. Configuration of image processing system]
FIG. 1 is a diagram illustrating an example of a configuration of an image processing system. The image processing system in FIG. 1 includes a server device 100, an MFP 200, and a mobile terminal 300. These are configured to be able to communicate with each other. The image processing system may be configured by the server device 100 and the MFP 200 without the portable terminal 300. The server apparatus 100 provides help content as information explaining how the MFP 200 is handled.

(An example of how help content is provided)
FIG. 2 is a diagram schematically illustrating an example of a help content provision mode in the image processing system. In the state shown in the lower left of FIG. 2, the user of the MFP 200 needs information that explains the handling of the MFP 200. The MFP 200 includes an operation panel 207 that displays the status of the MFP 200 and the like. Operation panel 207 displays an area 2071 for displaying content for providing various services in MFP 200, a help button 2072, and a start button 2073. The help button 2072 and the start button 2073 may be software buttons displayed on the display, or may be hardware buttons. The user operates a help button 2072 to acquire the above information. The help button 2072 and the start button 2073 may be displayed constantly when displayed as software buttons, or may be displayed only when a given content is displayed.

  When the help button 2072 is operated, the MFP 200 generates a screen capture of the area 2071 and transmits an image of the screen capture (hereinafter also referred to as “capture image”) to the server apparatus 100 (step (1 in FIG. 2) )).

  Upon receiving the captured image, the server apparatus 100 analyzes the captured image (step (2) in FIG. 2), and identifies the screen displayed on the MFP 200 based on the analysis result (step (3 in FIG. 2). )), The help content corresponding to the specified screen is specified (step (4) in FIG. 2). Then, the server apparatus 100 transmits the specified help content to the MFP 200 (step (5) in FIG. 2).

  Upon receiving the help content, the MFP 200 displays the help content in the area 2071 as shown in the lower right of FIG.

(Other examples of how help content is provided)
FIG. 3 is a diagram schematically illustrating another example of a help content providing mode in the image processing system. In the example of FIG. 3, when information about the MFP 200 in the state shown in the lower left is required, the user obtains help content for the MFP 200 in the mobile terminal 300 (hereinafter, also referred to as “help application”). Is activated (step (1A) in FIG. 3).

  The help application requests the user to capture the area 2071. When the user captures the area 2071 (step (1B) in FIG. 3), the help application transmits the captured image to the server device 100 (step (1C) in FIG. 3).

  When the image is acquired, the server apparatus 100 analyzes the image to identify the screen and identifies the help content corresponding to the identified screen (steps (2) to ( 4)). The server device 100 transmits the identified help content to the mobile terminal 300 (step (5) in FIG. 3). The portable terminal 300 displays the received help content on the display 304 (step (6) in FIG. 3). Thus, the user can operate MFP 200 with reference to the help content.

  In the example shown in FIGS. 2 and 3, transmission of help content to the MFP 200 or the portable terminal 300 is an example of providing help content. Another example of providing help content is to redirect to a storage device that stores the help content. When the help content is provided in this way, the server device 100 redirects the MFP 200 or the portable terminal 300 to the storage device that stores the help content specified in the step (4).

  Still another example of the help content is to transmit a link to a storage device that stores the help content to the MFP 200 or the mobile terminal 300. The link may be transmitted as a character string representing an IP (Internet Protocol) address, or may be transmitted as information such as a QR code (registered trademark) representing the address. When the link is received, the MFP 200 or the portable terminal 300 displays the help content as shown in FIG. 2 or FIG. 3 by connecting to the address indicated by the link.

[2. Hardware configuration]
Hereinafter, an example of the hardware configuration of each of the server apparatus 100, the MFP 200, and the mobile terminal 300 will be described.

(Server apparatus 100)
FIG. 4 is a diagram illustrating an example of a hardware configuration of the server apparatus 100.

  The server device 100 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a storage device 103, and a communication device 106 as main components. The CPU 101, the RAM 102, and the communication device 106 are connected to each other via an internal bus.

  The CPU 101 is an example of an arithmetic device that executes processing for controlling the overall operation of the server device 100. The RAM 102 functions as a work area when the CPU 101 executes processing. The storage device 103 stores a program (server program) executed by the CPU 101 and given data. The storage device 103 is, for example, a hard disk. Communication device 106 is implemented by a device (for example, a network card) for communicating with MFP 200 and portable terminal 300.

(MFP 200)
FIG. 5 is a diagram illustrating an example of a hardware configuration of the MFP 200.

  The MFP 200 includes, as main components, a CPU 201, a ROM 202, a RAM 203, an NV-RAM 204, a storage device 205, a communication device 206, an operation panel 207, a scanner unit 208, a printer unit 209, a facsimile ( FAX) unit 210. The CPU 201, the ROM 202, the RAM 203, the NV-RAM 204, the storage device 205, the communication device 206, the operation panel 207, the scanner unit 208, the printer unit 209, and the FAX unit 210 are connected to each other via an internal bus. Has been.

  The CPU 201 is an example of an arithmetic device that executes processing for controlling the overall operation of the MFP 200. The ROM 202 stores various data including a program executed by the CPU 201. The RAM 203 functions as a work area when the CPU 201 executes a program. The NV-RAM 204 is a non-volatile memory, and holds setting values for the operation of the MFP 200.

  Storage device 205 stores programs and data (including document data such as image data) used for the operation of MFP 200. Document data may be input to the MFP 200 via a network, or may be generated by reading an image with the scanner unit 208. The storage device 205 is, for example, an EEPROM (Electronically Erasable Programmable Read-Only Memory), CD-ROM, DVD-ROM, USB memory, memory card, FD, hard disk, SSD, magnetic tape, cassette tape, MO, MD, IC card. This is realized by a medium for storing data in a nonvolatile manner, such as an optical card (excluding a memory card), an optical card, a mask ROM, and an EPROM. The storage device 205 may be a subject of commercial transactions independent of the MFP 200 as a recording medium that can be attached to and detached from the MFP 200.

  The communication device 206 is a device for communicating with the server device 100 and the mobile terminal 300, and is realized by a network card, for example.

  The operation panel 207 includes a touch sensor (input unit) and a display (display unit). Operation panel 207 displays the state of MFP 200. Further, the operation panel 207 accepts input of various types of information (such as copy image quality or copy paper setting value, scan transmission destination (destination registration)).

  A scanner unit 208 scans a set original and generates image data of the original. A known method can be employed as a method for generating image data in the scanner unit 208.

  The printer unit 209 receives, for example, image data read by the scanner unit 208, print data transmitted from an external information processing apparatus such as the portable terminal 100, and FAX data received by the FAX unit 210 by electrophotography. Convert to data for printing. The printer unit 209 further prints an image such as a document based on the converted data. A well-known technique can be adopted as an image forming mode such as an electrophotographic system.

  The FAX unit 210 transfers the image data read by the scanner unit 208 to an external device via a public line. A known technique can be employed as the transfer method.

  The CPU 201 can detect the state of elements (scanner unit 208, printer unit 209, etc.) in the MFP 200 by executing a predetermined program. For example, the CPU 201 detects that the scanner unit 208 and / or the printer unit 209 is executing a job, that a paper jam has occurred in the printer unit 209, and the like.

(Mobile terminal 300)
FIG. 6 is a diagram illustrating an example of a hardware configuration of the mobile terminal 300.

  The mobile terminal 300 includes a CPU 301, a RAM 302, a storage device 303, a display 304, a touch sensor 305, a communication device 306, and a camera 307 as main components. The CPU 301, RAM 302, storage device 303, display 304, touch sensor 305, communication device 306, and camera 307 are connected to each other via an internal bus.

  The CPU 301 is an example of an arithmetic device that executes processing for controlling the overall operation of the mobile terminal 300. The RAM 302 functions as a work area when the CPU 301 executes processing. The storage device 303 stores various data including various programs such as an OS, a Web browser, and a help application executed by the CPU 301 and data used for executing these programs. The storage device 303 is, for example, an EEPROM.

  The display 304 is a display device for displaying an image generated by a program such as a web browser. The touch sensor 305 is an example of an input device for inputting information such as setting values to the mobile terminal 300 on a web browser. The communication device 306 is a device for communicating with the server device 100 and the MFP 200, and is realized by a wireless communication circuit, for example.

  Although the portable terminal 300 is comprised by the smart phone, for example, if it is a terminal which has an imaging function and a communication function, it does not need to be portable.

[3. Process flow]
Hereinafter, the flow of processing for realizing the help content provision mode will be described with reference to FIGS. 2 and 3.

(Providing help content in FIG. 2)
FIG. 7 is a diagram for explaining an example of the flow of processing for providing help content in the manner shown in FIG.

  FIG. 7 includes a flowchart of processing on the MFP 200 side (left side in FIG. 7) and a flowchart of processing on the server device 100 side (right side in FIG. 7). With reference to FIG. 7, the flow of processing will be described.

  In step S10, CPU 201 of MFP 200 determines whether help button 2072 has been operated or not. CPU 201 stops control at step S10 until it is determined that help button 2072 has been operated (NO at step S10), and if it is determined that help button 2072 has been operated (YES at step S10), control proceeds to step S12. .

  In step S <b> 12, the CPU 201 generates a screen capture of the operation panel 207 display. As shown in FIG. 2, the screen capture target may be only the area 2071 for displaying content, or the entire display including software buttons such as the help button 2072.

  In step S <b> 14, the CPU 201 transmits the screen capture image generated in step S <b> 12 to the server apparatus 100.

  When the CPU 101 of the server apparatus 100 receives an image from the MFP 200, in step S20, the CPU 101 executes control for providing help content. Details of the control in step S20 will be described later with reference to FIG.

  When help content is provided from server apparatus 100, CPU 201 of MFP 200 displays the help content in step S16.

  When the server apparatus 100 transmits help content to the MFP 200 as providing help content, the CPU 201 of the MFP 200 displays the received help content on the display of the operation panel 207. When the server apparatus 100 redirects the MFP 200 to a storage device that stores help content as provision of help content, the CPU 201 reads the help content stored in the storage device and displays the help content on the operation panel 207. When providing the help content, when the server apparatus 100 transmits the address of the storage device that stores the help content to the MFP 200, the CPU 201 uses the address to connect to the storage device, and the help content stored in the storage device. And the help content is displayed on the operation panel 207.

(Providing help content in FIG. 3)
FIG. 8 is a diagram for explaining an example of the flow of processing for providing help content in the manner shown in FIG.

  FIG. 8 includes a flowchart of processing on the mobile terminal 300 side (left side in FIG. 8) and a flowchart of processing on the server device 100 side (right side in FIG. 8). For example, the CPU 301 of the mobile terminal 300 executes processing according to the flowchart of FIG. 8 as a function of the help application. With reference to FIG. 8, the flow of processing will be described.

  In step S <b> 32, CPU 301 acquires an image on operation panel 207 of MFP 200. The CPU 301 may output a message instructing to shoot the operation panel 207 to the display 304. The user may shoot the operation panel 207 using the camera 307 according to the message. In one example, the CPU 301 acquires an image on the operation panel 207 as an image captured by the camera 307.

  In step S34, the CPU 301 transmits the image (captured image) acquired in step S32 to the server apparatus 100 together with a help request.

  When the CPU 101 of the server apparatus 100 receives an image from the mobile terminal 300, it executes control for providing help content in step S20. Details of the control in step S20 will be described later with reference to FIG.

  When the help content is provided from the server device 100, the CPU 301 of the mobile terminal 300 displays the help content in step S36.

  When the server device 100 transmits help content to the mobile terminal 300 as providing help content, the CPU 301 of the mobile terminal 300 displays the received help content on the display 304. When the server device 100 redirects the portable terminal 300 to a storage device that stores the help content as provision of help content, the CPU 301 reads the help content stored in the storage device and displays the help content on the display 304. When providing the help content, when the server device 100 transmits the address of the storage device that stores the help content to the mobile terminal 300, the CPU 301 uses the address to connect to the storage device and stores the address in the storage device. The help content is read and the help content is displayed on the display 304.

  For providing the help content, the server device 100 may display the address of the storage device that stores the help content as a QR code or the like on a given display. In this case, when the user captures the QR code with the camera 307, the CPU 301 acquires the address. Thereafter, the CPU 301 accesses the address, acquires help content, and displays the help content on the display 304.

(Providing help content by the server device 100)
FIG. 9 is an example of a flowchart of a subroutine of step S20.

  Referring to FIG. 9, in step S <b> 200, CPU 101 of server device 100 determines whether an image transmitted from MFP 200 or portable terminal 300 includes an image of a screen registered as the screen of MFP 200. If CPU 101 determines that the transmitted image includes a registered screen image (YES in step S200), control proceeds to step S202; otherwise (NO in step S200), control proceeds to step S208. To proceed.

  The determination of whether or not the transmitted image includes a registered screen image is realized, for example, by determining whether or not the former includes a feature that constitutes the latter. More specifically, the feature may be an icon. In one example, the image registered as the screen A includes a plurality of icons, and the CPU 101 transmits an image transmitted when the transmitted image includes images of a given number or more of the plurality of icons. Is determined to include the image of screen A. Otherwise, it is determined that the transmitted image does not include the image of screen A.

  In step S200, determining that the transmitted image includes the image of screen A corresponds to specifying screen A as the screen corresponding to the transmitted image.

  In step S202, CPU 101 determines in step S200 whether there is help content on the screen identified as the screen corresponding to the transmitted image. In one example, the storage device 103 of the server device 100 stores help content for each screen in association with each screen of content displayed on the MFP 200. However, the help content need not be associated with every screen of the content.

  When a producer of content for the MFP 200 generates help content for the content for the MFP 200, the producer may register the help content in the server device 100. The producer further associates each help content with a corresponding screen (feature) in order to clarify at which stage of the content for the content for the MFP 200 each help content includes the help information. You may register with.

  The help content is provided by operating the help button 2072 or photographing by the mobile terminal 300 even if the access code is not given to the content for the MFP 200 itself. As a result, the creator can generate the help content after releasing the content for the MFP 200.

  The CPU 101 determines whether or not help content associated with the screen specified as described above is stored in the storage device 103, and determines that such help content is stored (in step S202). YES), control proceeds to step S204; otherwise (NO in step S202), control proceeds to step S206.

  In step S204, the CPU 101 extracts help content associated with the screen specified as described above.

  In step S <b> 206, CPU 101 extracts the top page of content for help of MFP 200. The help content may correspond to the model of the MFP 200.

  CPU 101 may specify the model of MFP 200 based on, for example, information used for communication with MFP 200 (for example, the MAC address of MFP 200) and management information in the image processing system. This management information associates the model name of each MFP in the image processing system with the MAC address of each MFP. Further, the CPU 201 of the MFP 200 may transmit the model name of the MFP 200 to the server apparatus 100 together with the captured image in step S14.

  For example, the CPU 101 may specify the model of the MFP 200 based on the position information of the mobile terminal 300 and the management information in the image processing system. This management information associates the arrangement position of each MFP in the image processing system with the model name of each MFP. The portable terminal 300 may acquire the position information of the portable terminal 300 using the GPS function, and may transmit the position information together with the captured image in step S34. The CPU 101 may specify the model of the MFP 200 by acquiring the model name of the MFP disposed at the position closest to the position information of the mobile terminal 300.

  In step S208, CPU 101 determines whether an image transmitted from MFP 200 or portable terminal 300 includes an image of a customized screen of the original screen (registered screen). An example of an image including a customized screen image is an image including a logo of a manufacturer that manufactures (or sells) MFP 200. Another example is an image that contains more than a given number (or percentage) of registered screen features. A specific example of the customized image will be described with reference to FIGS.

  FIG. 10 is a diagram illustrating an example of a registered image. FIG. 10 shows a screen 1000 which is an example of a copy setting screen.

  FIG. 11 is a diagram illustrating another example of a registered image. FIG. 11 shows a screen 1100 that is an example of a scan setting screen.

  FIG. 12 is a diagram illustrating an example of a screen (image) included in an image transmitted from the MFP 200 or the mobile terminal 300.

  Screen 1200 includes an element 1201 (represented by “*” in FIG. 12 for ease of explanation) representing the logo of the manufacturer of MFP 200. In one example, the CPU 101 determines that the transmitted image includes a customized screen by detecting an element 1201 whose shape matches the manufacturer's logo.

  In another example, the CPU 101 determines that the transmitted image includes a customized image because the screen 1200 includes a predetermined number or more of features included in the screen 1000 of FIG. An example of the feature included in the screen 1000 is a character string. More specifically, the screen 1000 includes six character strings “color”, “paper”, “magnification”, “original / output”, “single side”, and “double side”. The CPU 101 detects four character strings “paper”, “color”, “single side”, and “double side” by character recognition on the image of the screen 1200. When the feature of the screen 1000 is the above six character strings, the image on the screen 1200 includes four character strings among the six character strings. For example, as an example of customized image conditions, it is set to include four or more of the six character strings. In this case, the CPU 101 determines that the screen 1200 is a screen generated by customizing the screen 1000. That is, the CPU 101 determines that the transmitted image (image including the screen 1200) includes the customized screen image of the original screen (screen 1000).

  Note that the screen 1200 does not include features common to the screen 1100 of FIG. 11, and thus is not determined to be a customization screen of the screen 1100 of FIG. 11. More specifically, the screen 1100 includes character strings “Scan”, “folder”, and “format”, but the screen 1200 does not include these character strings.

  Returning to FIG. 9, when CPU 101 determines that the image transmitted from MFP 200 or portable terminal 300 includes the original screen (registered screen) including a customized screen image (YES in step S208). Control proceeds to step S210; otherwise (NO in step S208), control proceeds to step S212.

  In step S210, CPU 101 extracts help content associated with the original screen for the screen included in the image transmitted from MFP 200 or portable terminal 300. For example, when an image including the screen 1200 of FIG. 12 is transmitted, the CPU 101 extracts help content associated with the screen 1000 that is the original screen of the screen 1200.

  In step S <b> 212, CPU 101 determines whether an image transmitted from MFP 200 or portable terminal 300 includes a screen generated by a subject registered in MFP 200 (for example, a so-called “third vendor”). FIG. 13 is a diagram illustrating an example of a screen generated by a registered subject. In one example, when the image includes a feature associated with the registered subject, the CPU 101 determines that the image includes a screen generated by the subject.

  Based on the screen 1300 in FIG. 13 and the element 1301 representing the logo of the main body (represented by “*” in FIG. 13 for ease of explanation), the CPU 101 displays an image including the screen 1300. Then, it is determined that the image includes a screen generated by the subject.

  Another example of the feature is that an element corresponding to the logo is arranged at a specific position on the screen. Yet another example is that a software key such as a start key is arranged at a specific position on the screen.

  Returning to FIG. 9, when CPU 101 determines that the transmitted image includes a screen generated by the registered subject (YES in step S212), control proceeds to step S214, otherwise (step S212). NO), the control proceeds to step S216.

  In step S214, the CPU 101 extracts the content of the main help page that generated the screen included in the transmitted image. The extracted content is, for example, help content prepared by a so-called third vendor. The content may be stored in the storage device 103. The content may be stored in a server managed by the third vendor. The CPU 101 may acquire the help content by accessing the server of the third vendor.

  In step S216, CPU 101 extracts the top page of the help content of MFP 200, as in step S206. The help content may correspond to the model of the MFP 200.

  In step S218, CPU 101 transmits the information extracted in steps S206, S210, S214, and S216 to MFP 200 (FIG. 7) or portable terminal 300 (FIG. 8). Thereafter, the CPU 101 returns the process to FIG. 7 or FIG.

  According to the processing described above, server apparatus 100 provides MFP 200 or portable terminal 300 with information (help content) for handling MFP 200. Providing means making it available to the user. In this sense, an example of provision is to send the help content itself. Another example is sending help content link information (text information or code representing an address). Yet another example is displaying help content link information on a given display.

  In the processing described above, the screen included in the image transmitted from the MFP 200 or the portable terminal 300 is an original screen (first type) by the manufacturer of the MFP 200, or a screen generated by customizing the original screen. It is determined whether the screen is a (second type) or a screen (third type) generated by a registered subject (so-called third vendor), and help content corresponding to the determined type is extracted. In addition, the processing extracts help content according to such a case even when the screen included in the transmitted image does not belong to any of the above types.

[4. Providing help content to frozen MFPs (1)]
When the MFP 200 is in the frozen state, the server apparatus 100 may select the help content in a different manner from the case where the MFP 200 is not in the frozen state. FIG. 14 is a diagram schematically illustrating an example of an aspect of providing help content when the MFP 200 is in a frozen state. Note that the freeze state of the MFP 200 means a state in which the MFP 200 is in a so-called solidified state, so that even if the user operates the MFP 200, the MFP 200 does not respond to the operation.

  In FIG. 14, compared with FIG. 3, the mobile terminal 300 transmits freeze information to the server apparatus 100 in addition to transmission of the captured image to the server apparatus 100 (process (1C)) (process (1D)). . The freeze information is information indicating that the MFP 200 is in a freeze state. In one example, the help application of the mobile terminal 300 displays a freeze button on the display 304. A user who recognizes that the MFP 200 is in a frozen state operates the freeze button in addition to photographing the operation panel 207 of the MFP 200 with the camera 307 as an operation related to a captured image to the server apparatus 100. Thereby, in addition to the captured image, freeze information is transmitted to the server apparatus 100. When the server apparatus 100 receives the freeze information, the server apparatus 100 provides help content corresponding to the freeze state to the user.

  FIG. 15 is a diagram illustrating an example of a process for realizing the help content provision mode illustrated in FIG. 14. FIG. 15 shows a flowchart (left side) on the mobile terminal 300 side and a flowchart (right side) on the server device 100 side.

  In the process of FIG. 15, compared to FIG. 8, the CPU 301 of the mobile terminal 300 determines whether or not the freeze button has been operated in step S <b> 35 </ b> A. The determination of whether or not the freeze button has been operated is an example of whether or not information indicating that the MFP 200 is in a frozen state has been input to the mobile terminal 300. If CPU 301 determines that the freeze button has been operated (YES in step S35A), control proceeds to step S35B; otherwise (NO in step S35A), control proceeds to step S36.

  In step S35B, CPU 301 transmits freeze information to server device 100.

  In step S26, CPU 101 of server device 100 determines whether MFP 200 is in a frozen state or not. For example, if freeze information associated with a captured image is transmitted from the MFP 200, the CPU 101 determines that the MFP 200 is in a frozen state. If the freeze information associated with the captured image is not transmitted, the CPU 200 freezes. Judge that it is not in a state. If CPU 301 determines that MFP 200 is in a frozen state (YES in step S26), control proceeds to step S28; otherwise (NO in step S26), control proceeds to step S20.

  In step S <b> 28, CPU 101 transmits help content for freezing to portable terminal 300. An example of the help content for freezing is help content that describes an operation for canceling the freeze state. Another example is information necessary to call a service person.

  According to the processing described with reference to FIG. 15, when MFP 200 is in a frozen state, the help content for freezing is provided to the user of portable terminal 300 (the user of MFP 200).

  Note that the CPU 101 of the server apparatus 100 may determine the help content to be provided to the user according to the type of screen displayed on the MFP 200 even when the MFP 200 is in a frozen state. For example, in step S <b> 28, the CPU 101 identifies the type of screen included in the image transmitted from the mobile terminal 300, and determines whether the screen type corresponds to the frozen state that is resolved by the user's operation. It is specified whether to cope with a freeze state that requires repair of the MFP 200. In the former case, the CPU 101 may provide content representing an operation method, and in the latter case, information (phone number or the like) for calling a service person may be provided as freeze help content.

[5. Providing help content to a frozen MFP (2)]
Whether the MFP 200 is in a frozen state or not may be confirmed by the server device 100, not the user of the mobile terminal 300. FIG. 16 is a diagram schematically illustrating another example of a help content providing mode when the MFP 200 is in a frozen state.

  In the example of FIG. 16, when compared with FIG. 3, when the server apparatus 100 receives a captured image from the mobile terminal 300, the server apparatus 100 executes the process (2A) and the process (2B).

  In step (2A), server device 100 identifies MFP 200 that has photographed operation panel 207.

  For example, the server apparatus 100 specifies the MFP 200 using the position of the mobile terminal 300. More specifically, the server apparatus 100 stores MFP arrangement information in the image processing system, specifies the position of the mobile terminal 300 that has transmitted the captured image, and designates the MFP installed near the position as the MFP 200. Identify. In one example, the mobile terminal 300 acquires position information of the mobile terminal 300 from GPS (Global Positioning System), and the server device 100 specifies the position of the mobile terminal 300 based on the position information.

  In another example, server device 100 identifies MFP 200 using information in the captured image. When the captured image includes information identifying the MFP 200 such as the serial number of the MFP 200, the server apparatus 100 acquires the information by character recognition of the captured image and identifies the MFP 200.

  In step (2B), server device 100 confirms the state of MFP 200 identified in step (2A). For example, the server apparatus 100 communicates with the MFP 200 and confirms the state of the MFP 200 remotely.

  If the MFP 200 is in a frozen state, the server apparatus 100 provides help content for freeze, and otherwise provides help content as described with reference to FIG.

  FIG. 17 is a diagram illustrating an example of a process for realizing the help content providing mode illustrated in FIG. 16. FIG. 17 shows a flowchart (left side) on the mobile terminal 300 side and a flowchart (right side) on the server device 100 side.

  The process of FIG. 17 further includes step S22 and the like as compared with FIG. When CPU 101 of server apparatus 100 receives the captured image from portable terminal 300, it identifies MFP 200 in step S22 (step (2A) in FIG. 16).

  In step S24, the CPU 101 confirms the state of the MFP 200 specified in step S22 (step (2B) in FIG. 16).

  In step S26, CPU 101 determines whether MFP 200 is in a frozen state or not. In one example, CPU 101 determines that MFP 200 is in a frozen state if communication with MFP 200 is attempted but no response is received from MFP 200. If CPU 301 determines that MFP 200 is in a frozen state (YES in step S26), control proceeds to step S28; otherwise (NO in step S26), control proceeds to step S20. In step S28, CPU 101 provides freeze help content in the same manner as described in FIG.

[6. Expansion of help contents]
In the image processing system, server apparatus 100 is requested to provide help content together with an image including the screen of MFP 200. The server apparatus 100 can make such a request into a database and use it for various aspects. In one example, if CPU 101 determines that there is no help content for a given screen more than a predetermined number of times (NO in step S202), CPU 101 may notify that it is necessary to generate help content for that screen.

  In another example, the CPU 101 counts the number of times help content is requested for each screen. A large number of requests for help content on a screen means that the screen is difficult for the user to handle. Therefore, the CPU 101 may output information requesting improvement of processing related to the screen in the content of the MFP 200.

  Each embodiment disclosed this time must be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims. In addition, the invention described in the embodiment and each modification is intended to be carried out independently or in combination as much as possible.

  100 server apparatus, 101, 201, 301, 200 MFP, 300 portable terminal, 304 display, 305 touch sensor, 307 camera, 2071 area.

Claims (12)

An image processing system comprising an image processing device and a server device capable of communicating with the image processing device,
Screen transmission means for transmitting an image of an operation screen of the image processing apparatus to the server apparatus;
Help content specifying means for specifying help content corresponding to the operation screen transmitted by the screen transmitting means;
An image processing system comprising: help content providing means for providing the help content to a user of the image processing apparatus. The image processing apparatus includes:
An information processing unit for capturing an operation screen of the image processing apparatus;
A communication unit that transmits the operation screen captured by the information processing unit to the server device,
The image processing system according to claim 1, wherein the screen transmission unit includes the information processing unit and the communication unit.   The help content providing means transmits the help content to the image processing device, transmits link information to the help content to the image processing device, or stores the help content in the device that stores the help content. The image processing system according to claim 2, wherein the help content is provided to a user of the image processing apparatus by redirecting the image. A portable terminal capable of communicating with the server device;
The portable terminal is
An imaging unit;
A communication unit that transmits an image captured by the imaging unit to the server device,
The image processing system according to claim 1, wherein the screen transmission unit includes the imaging unit and the communication unit.   The help content providing means transmits the help content to at least one of the portable terminal and the image processing device, causes the device storing the help content to redirect the communication unit, and links to the help content The image processing system according to claim 4, wherein the help content is provided to a user of the image processing device by transmitting information or displaying link information to the help content on a display device.   The help content specifying means classifies the operation screen into at least one of a plurality of predetermined types based on the image of the operation screen, and the help corresponding to the classified at least one type The image processing system according to claim 1, wherein content is specified.   The plurality of types include an operation screen that represents the same source as the image processing apparatus and includes an original operation screen, and a second type that includes an operation screen in which the original operation screen is changed. The image processing system according to claim 6, comprising: a type and a third type including an operation screen representing a source different from the image processing apparatus.   The said help content specific | specification means selects the help content provided to the user of the said image processing apparatus according to whether the said image processing apparatus is in a frozen state, The any one of Claims 1-7 The image processing system described.   The help content specifying means specifies the image processing device based on location information of a mobile terminal that is a transmission source of information on the image processing device in a frozen state, or an address on the network of the image processing device. The image processing system according to claim 8. A communication device for receiving an operation screen of the image processing device;
A processor;
A storage device for storing a program executed by the processor,
The program is stored in the processor.
Identifying help content corresponding to the operation screen;
Providing the help content to a user of the image processing apparatus.   The step of identifying the help content classifies the operation screen into at least one of a plurality of predetermined types based on the image of the operation screen, and corresponds to the classified at least one type The server device according to claim 10, comprising identifying help content to be processed.   The plurality of types include an operation screen that represents the same source as the image processing apparatus and includes an original operation screen, and a second type that includes an operation screen in which the original operation screen is changed. The server apparatus according to claim 11, comprising a type and a third type including an operation screen representing a source different from the image processing apparatus.