JP2010219879A - Help information providing system, server apparatus, target device, mobile communication terminal, help information providing method, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a help information providing system, server apparatus, target device, mobile communication terminal, help information providing method, program and recording medium in which accurate help information corresponding to a status of the target device can be acquired. <P>SOLUTION: A target device, a server apparatus providing help information and a mobile communication terminal are connected over a network, the target device transmits a device ID and a status code to the server apparatus, and the mobile communication terminal extracts an image of a marker attached to the target device from captured image data captured by imaging the target device, acquires the device ID and a plane code of the target device form the marker image and transmits them to the server apparatus. Based on the device ID and the status code transmitted from the target device and the device ID and the plane code transmitted from the mobile communication terminal, the server apparatus transmits specific help information corresponding to the status of the target device to the mobile communication terminal, and the mobile communication terminal produces and displays help information providing image data by combined the transmitted help information with the captured image data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a help information providing system, a server device, a target device, a mobile communication terminal, a help information providing method, a program, and a recording medium, and more particularly, a help information providing system capable of obtaining accurate help information according to the state of the target device, The present invention relates to a server device, a target device, a mobile communication terminal, a help information providing method, a program, and a recording medium.

  2. Description of the Related Art In recent years, devices with advanced functions and compounding have become widespread, such as composite image forming apparatuses (MFPs: multifunction printers). Such a highly functional apparatus has a complicated operation method, and it is necessary to refer to a product manual (paper medium) or help text displayed on a display unit provided in the apparatus. The following technologies have been proposed for the help function.

  Patent Document 1 discloses a server device that stores operation method information for each image processing device and downloads operation-rich information to a requesting image processing device based on an operation manual request code. This can cope with an increase in manual information.

  Further, in Patent Document 2, in an environment where a plurality of the same models are introduced in a system in which help data is placed on a network in advance (stored in a help server) and accessed and downloaded as necessary, Since the same help data may be downloaded from individual image output devices, help information can be manipulated for other image output devices on the network to avoid wasting network resources. An image output apparatus system is disclosed. That is, if there is manual data cached on a nearby network (another image output device), it is used.

  However, the above techniques have the following problems.

  As described above, Patent Literature 1 and Patent Literature 2 disclose means for downloading and displaying manual data from a server on an image forming apparatus. If this means is used, it is possible to display help information corresponding to the operation status of the apparatus. However, in a device with a complicated structure such as an image forming device, when some trouble occurs in the internal mechanism, etc., it is necessary to operate a part in a location away from the panel, such as inside the device or on the back side. There is. At this time, since the user cannot operate the component while referring to the display unit (display panel) provided in the image forming apparatus, there is a possibility that the operation is incorrect or extra time is spent on the operation.

  Also, if the help displayed on the device cannot provide sufficient information to the user, such as when the display panel is simplified or omitted because the display panel is small, the user can use the manual book attached to the device or the printed details. You may need to browse help. This makes the function that can display help information meaningless.

  This invention is made | formed in view of such a condition, and it aims at obtaining the exact help information according to the state of object apparatus.

  In the help information providing system according to the present invention, a target device, a server device that provides help information for assisting operation of the target device, and a mobile communication terminal that receives the help information are connected to a network. In the help information providing system, the target device includes a device ID uniquely assigned to each target device, and a surface code indicating a location where a marker for detecting the position / tilt of the device is attached. A marker that represents the device, and a transmission unit that transmits the device ID of the device and a status code indicating the state of the device to the server device, and the mobile communication terminal includes an imaging unit that captures the target device, The marker information is acquired by extracting the marker image from the captured image data and acquiring the device ID and the surface code of the target device from the extracted marker image. Means, and transmitting means for transmitting the acquired device ID and surface code to the server device, wherein the server device receives the device ID and status code transmitted from the target device, and the portable communication terminal. The mobile communication terminal further includes a transmission unit that transmits unique help information corresponding to the state of the target device to the mobile communication terminal based on the transmitted device ID and the surface code, and the mobile communication terminal further includes the server device. And generating the help information providing image data by combining the help information transmitted from the image and the captured image data, and display means for displaying the generated help information providing image data. .

  The server device according to the present invention provides help information in which a target device, a mobile communication terminal, and a server device that provides help information for assisting operation of the target device to the mobile communication terminal are connected to a network. The server device of the system, the unique device corresponding to the state of the target device based on the device ID and the status code transmitted from the target device and the device ID and the surface code transmitted from the mobile communication terminal Transmission means for transmitting the help information to the portable communication terminal.

  The target device according to the present invention provides help information in which a target device, a mobile communication terminal, and a server device that provides help information for assisting operation of the target device to the mobile communication terminal are connected to a network. A surface code indicating the target device of the system, wherein the target device has its own device ID uniquely assigned to each target device, and a position / tilt detection marker attached to the target device. And a transmission means for transmitting the device ID of itself and a status code indicating the status of the device to the server device.

  The mobile communication terminal according to the present invention is a help information in which a target device, a mobile communication terminal, and a server device that provides help information for assisting the operation of the target device to the mobile communication terminal are connected to a network. In the mobile communication terminal of the providing system, an imaging unit that images the target device, an image of the marker is extracted from the captured image data, and a device ID and a surface code of the target device are extracted from the extracted marker image Marker information acquisition means for acquiring information, transmission means for transmitting the acquired device ID and surface code to the server device, help information transmitted from the server device and the captured image data, and help A generating unit that generates information providing image data; and a display unit that displays the generated help information providing image data. That.

  In the help information providing method according to the present invention, a target device, a server device that provides help information for assisting the operation of the target device, and a mobile communication terminal that receives the help information are connected to a network. A help information providing method in a help information providing system, wherein the target device transmits its device ID and a status code indicating its status to the server device, and the portable communication terminal An image of a marker for position / inclination detection of the device affixed to the target device is extracted from captured image data obtained by capturing the device, and the device of the target device uniquely assigned to each target device from the extracted marker image A step of acquiring an ID and a surface code indicating a location where the marker is affixed; and the mobile communication terminal acquires the acquired device I And the surface code to the server device, the server device based on the device ID and status code transmitted from the target device, and the device ID and surface code transmitted from the mobile communication terminal. A step of transmitting unique help information corresponding to the state of the target device to the mobile communication terminal, and the mobile communication terminal combines the help information transmitted from the server device and the captured image data, Generating help information providing image data and displaying it on a display unit.

  In the help information providing program according to the present invention, a target device, a server device that provides help information for assisting the operation of the target device, and a mobile communication terminal that receives the help information are connected to a network. A help information providing program in a help information providing system, in which the target device transmits its own device ID and a status code indicating its own state to the server device, and the portable communication terminal is the target An image of a marker for position / inclination detection of the device affixed to the target device is extracted from captured image data obtained by capturing the device, and the device of the target device uniquely assigned to each target device from the extracted marker image A process of acquiring an ID and a surface code indicating a location where the marker is affixed; and Based on the processing for transmitting the ID and the surface code to the server device, the device ID and the status code transmitted from the target device, and the device ID and the surface code transmitted from the mobile communication terminal. The process of transmitting unique help information corresponding to the state of the target device to the mobile communication terminal, and the mobile communication terminal combines the help information transmitted from the server device and the captured image data. A process of generating help information providing image data and displaying it on a display unit is executed by a computer.

  The recording medium according to the present invention is a computer-readable recording medium for recording the processing of the help information providing program according to the present invention.

  According to the present invention, accurate help information corresponding to the state of the target device can be obtained.

1 is an overall configuration diagram of a help information providing system according to an embodiment of the present invention. It is a block diagram of the mobile telephone 200 which concerns on embodiment of this invention. It is a figure for demonstrating the data which the help server 300 which concerns on embodiment of this invention memorize | stores. It is a time chart which shows the operation | movement process of the help information provision system which concerns on embodiment of this invention. It is a figure for demonstrating the process in step S7 of FIG. It is a figure which shows an example of the help information provision image data displayed on the liquid crystal display 207 which concerns on embodiment of this invention.

  Embodiments of the present invention will be described below in detail with reference to the drawings. The embodiments described below are preferred embodiments of the present invention, and thus various technically preferable limitations are given. However, the scope of the present invention is particularly limited in the following description. As long as there is no description of the effect, it is not restricted to these aspects.

  In the embodiments described below, a description will be given using an MFP (hereinafter referred to as a digital multifunction peripheral) as an example of an image forming apparatus as a target device, a mobile phone as a mobile communication terminal, and a help server as a server device. These are merely examples, and the target device may be an office device such as a printer or a facsimile, or an experimental device in addition to the digital multi-function peripheral. The portable terminal may be a digital camera.

(Constitution)
A help information providing system according to an embodiment of the present invention will be described. FIG. 1 shows the overall configuration of the system. The digital multifunction peripheral 100 and the help server 300 can communicate via a network such as the Internet. Similarly, the mobile phone 200 can be connected to the network and can communicate with the help server 300.

  The configuration of the digital multi-function peripheral 100 is generally known. A characteristic configuration of the digital multifunction peripheral 100 according to the present embodiment includes a network interface that is a communication unit that connects to a network and transmits and receives data, and further includes an apparatus ID (IDentification) that uniquely identifies itself. Remember. In addition, it has a function of determining a status code indicating its own status (such as a failure). The digital multifunction peripheral 100 is also characterized in that a marker 101 for detecting the position / tilt of the device is attached.

  The marker 101 is printed on a device that can be attached to the surface of the device, such as a sticker, and is attached to the digital multifunction peripheral 100. The marker 101 represents a device ID of the digital multi-function peripheral 100 and a surface code indicating a location where the marker 101 is attached. For example, the marker 101 is affixed to the side surface or rear surface of the digital multifunction peripheral 100, the inside of the apparatus, or the like, and is affixed so that the surface and the marker can be accommodated simultaneously by the camera 201 (imaging means) included in the mobile phone 200. ing.

  The marker 101 may be a figure that can detect the tilt and distance from the camera when it is housed in the camera. As a specific example of the marker, a QR (Quick Response) code which is an example of a two-dimensional code can be considered.

  The configuration of the mobile phone 200 is shown in FIG. A mobile phone 200 according to this embodiment includes a digital multifunction peripheral (target device) and a camera 201 that is an imaging unit for acquiring a marker image, a marker information acquisition unit 202, an operation input unit 203, a wireless network interface 204, help. A data acquisition unit 205, a display image generation unit 206, and a liquid crystal display 207 are provided.

  The marker information acquisition unit 202 detects a marker image from the image data acquired by the camera 201, and extracts the position and inclination of the marker and embedded information.

  The operation input unit 203 accepts user key operations and the like.

  The wireless network interface 204 is a means for communicating with a network.

  The help data acquisition unit 205 acquires help data from the wireless network interface 204 and the help server 300 connected via the network.

  The display image generation unit 206 superimposes (synthesizes) image data acquired from the camera 201 and help data (for example, drawing of a help object) acquired from the help server 300 to generate help information providing image data.

  The liquid crystal display 207 is an example of a display unit that is a means for displaying display data.

  Using the mobile phone 200 having the above-described configuration, the user captures the location where the help information is to be provided (the entire target device, the inside of the target device, the back side, etc.) and receives the help information. Note that when the target device is imaged, the marker 101 attached to the location is also imaged together.

  That is, by displaying the help information superimposed on the image data captured by the mobile phone 200, the user can be provided with accurate help information.

  A technique called AR (Augumented Reality) can be used as a method of superimposing other image data (explanatory image or text) on image data obtained by capturing a real scene. As a technique related to the AR display in the portable terminal, for example, a technique described in Patent Document 3 can be applied. Patent Document 3 discloses a method of displaying an explanation image or text superimposed on an image acquired from a camera on a mobile terminal (mainly a mobile phone with a camera). The data to be superimposed is obtained from the digital code in the image. A technique using a two-dimensional matrix code (a kind of barcode) as a marker in the marker method AR described in Non-Patent Document 1 is also applicable.

  The help server 300 is an information processing apparatus including at least a CPU (Central Processing Unit) 330, a storage unit 310 that is a storage unit such as a memory or an HDD (Hard Disk Drive), and a network interface 320 that is a communication unit.

  Data stored on the storage means by the help server 300 will be described with reference to FIG. FIG. 3 is a diagram for explaining data stored in the help server 300 according to the present embodiment.

  The help server 300 includes a state table 301 that manages states for each device ID, a help table 302 that manages association of help data, and a help data table 303 that stores help data.

  The status table 301 is a set of three sets of information: a device ID, a model code indicating the model of the device corresponding to the device ID, and a status code indicating the status of the device.

  The help table 302 is a set of four information sets of model code, status code, surface code, and help code.

  The help data table 303 is a set of sets of help codes and help data bodies. The help code is an identifier. The help data body is a help object that is actually rendered data.

  The help object has relative coordinates from the corresponding marker, and draws the help object at the coordinates calculated from the marker image obtained from the camera of the portable terminal.

  The help object may be descriptive text. The help object may be image data such as JPEG. The help object is a 3D object, that is, data composed of vertex coordinates, color information, and the like, and may be constructed and rendered as a 3D object on the mobile phone 200 that has received the help object. The help object may be a combination of the types of data listed above.

(Operation processing)
Next, an operation process of the help information providing system according to the present embodiment will be described. FIG. 4 is a time chart showing an operation process of the help information providing system according to the present embodiment.

  When the digital multi-function peripheral 100 generates some kind of failure (step S1), the digital multi-function peripheral 100 displays the occurrence of the failure on its operation panel (display unit) (step S2), and the state corresponding to the type of failure The code and the unique device ID held by itself are transmitted to the help server 300 via the network (step S3). The display of the occurrence of the failure (step S2) is not limited to the display notification, and any notification means (email notification, voice notification, etc.) may be used.

  When the help server 300 receives the device ID and the status code from the digital multi-function peripheral 100, the help server 300 rewrites the status code of the corresponding device ID on the status table 301 with the latest status code received from the digital multi-function peripheral 100 (step S4). .

  On the other hand, the user knows that a failure has occurred in the digital multifunction peripheral 100 due to the occurrence of the failure displayed on the display unit of the digital multifunction peripheral 100. The user directs the camera 201 of the mobile phone 200 so that the digital multifunction peripheral 100 and the marker 101 attached to the digital multifunction peripheral 100 are placed in the camera video, and captures an image (step S5). The marker information acquisition unit 202 extracts a marker image from the captured image data, and reads the device ID and surface code represented by the marker 101. The help data acquisition unit 205 transmits the read device ID and surface code to the help server 300 via the wireless network interface 204 and the network (step S6).

  Here, the mobile phone 200 holds in advance addresses indicating the location of the help server on the network (for example, URL (Uniform Resource Locators) of the server).

  The mobile phone 200 is preferably in a state where communication with the help server 300 is possible at any time while attempting to acquire an image of the marker 101 using the camera 201. For example, when transmitting from the mobile phone 200 to the help server 300, each time the device ID and the surface code based on the marker 101 that can be detected from the captured image data acquired by the camera 201 change, the information is stored in the help server. To 300. When the mobile phone 200 receives data from the help server 300, a polling method that tries to acquire help data at a fixed interval or a communication connection is always established, and the mobile phone 200 is actively transmitted from the help server 300. Help data can always be obtained using a push method that receives incoming data.

  When the help server 300 receives the device ID and the surface code from the mobile phone 200 and requests help data, the help server 300 extracts the model code and the state code associated with the device ID from the state table 301. Next, the help code corresponding to the three combinations of the model code and the status code extracted from the status table 301 and the surface code transmitted from the mobile phone 200 is extracted from the help table 302. Help data corresponding to the extracted help code is extracted from the help data table 303 (step S7). FIG. 5 is a diagram for explaining the processing in step S7.

  The mobile phone 200 transmits the retrieved help data as a response to the mobile phone 200 (step S8).

  The help data acquisition unit 205 included in the mobile phone 200 passes the help data received from the help server 300 to the display image generation unit 206. On the other hand, the marker information acquisition unit 202 passes the position and inclination of the marker 101 in the captured image data to the display image generation unit 206.

  The display image generation unit 206 calculates the position, size, and inclination of displaying the help object sent as help data from the position and inclination of the marker 101, and superimposes the help object on the liquid crystal display 207 with the captured image data. The help information providing image data which is the composite image data generated in this way is displayed (step S9). As a drawing processing method to be superimposed, the methods described in Non-Patent Document 1 and Patent Document 3 described above can be used.

  FIG. 6 is a diagram illustrating an example in which composite image data generated by superimposing a help object on captured image data on the liquid crystal display 207 is displayed as help information providing image data. Here, an example is shown in which an object to be colored on the operation target part, an object to which the part is pointed and an explanation text is added, and an object indicating the direction in which the part is operated are displayed.

  As described above, in the help information providing system according to the present embodiment, the target device is imaged by the camera included in the mobile phone, the help data corresponding to the state of the target device is received from the help server on the network, and the mobile phone Generates and displays a composite image in which the received help data is superimposed on the captured image data of the target device, so that the user can obtain help information according to the status of the target device through a mobile phone in an easily understandable and accurate manner. be able to. In addition, it can be expected that the time required for solving the problem related to the operation of the device is shortened.

  Furthermore, the help information providing system according to the present embodiment can cope with changes in the digital multifunction peripheral 100 (step S10). That is, when the user performs an operation according to the help information providing image and the state of the digital multifunction peripheral 100 changes, the digital multifunction peripheral 100 sends a new status code and device ID corresponding to the changed state to the help server 300. Send. The help server 300 rewrites the status code and transmits help data corresponding to the changed state to the mobile phone 200. When the mobile phone 200 receives new help data, the mobile phone 200 newly generates and displays a help information providing image based on the new help data. As described above, the help information providing image is updated according to a change in the state of the target device, and thus the user can receive an operation instruction to interact with the help server 300.

  Generation processing of help information providing image data displayed on the liquid crystal display 207 of the mobile phone 200 (composition processing of help data and captured image data) is performed as needed according to changes in the help data. Therefore, the help object is displayed as if it exists on the real scene.

  As described above, in the help information providing system according to the present embodiment, when the user operates the target device (digital multifunction peripheral 100), the mobile phone 200 receives help data corresponding to the changed state and displays the help data. In order to change the information providing image data, it can be used as an interactive help system using the mobile phone 200 as an interface. Therefore, when the target device is operated, information regarding the operation location and the operation procedure according to the state of the target device can be visually and easily shown. The user does not need to perform an operation for displaying the next data on the mobile phone 200, and an effect of improving user convenience can be expected.

  When the digital multifunction peripheral 100 transmits the changed state code to the help server 300 for the change in the state, the camera 201 of the mobile phone 200 continues to image the target device (or operates according to the help information providing image). When the target device is imaged every time (the image is taken periodically), the captured image data acquired by the camera 201 also changes every moment in response to a change in the sight caused by a user operation on the device. The change of the marker 101 may be acquired, and the changed information may be transmitted to the help server 300.

[Other Embodiments]
Next, if the communication between the digital multifunction peripheral 100 and the help server 300 cannot be established for some reason, or if the digital multifunction peripheral 100 does not have a means for transmitting data to the help server 300 in the first place, the state on the help server 300 Because the code cannot be updated, help information appropriate to the situation cannot be provided. Therefore, it is conceivable to transmit a status code from the mobile phone 200 to the help server 300.

  In the present embodiment, when a failure occurs, the digital multi-function peripheral 100 displays a device ID and a status code on the display unit. The user inputs these pieces of information by operating keys on the operation input unit 203 of the mobile phone 200. The mobile phone 200 can update the status code in the status table 301 held by the help server 300 by transmitting the input device ID and status code to the help server 300.

  After transmitting the device ID and the like to the help server 300, the mobile phone 200 images the digital multifunction peripheral 100 and the marker 101 with the camera 201, performs the processing shown in the above embodiment (steps S5 to S9 in FIG. 4), You can receive help information according to your status.

  Thereby, even when the target device cannot transmit the state information for some reason, it can be replaced by using the mobile phone 200, and an improvement in fault tolerance can be expected.

  Even if the target device cannot transmit the status information for some reason, when the status of the digital multifunction peripheral 100 changes, a new status code corresponding to the changed status is provided from the mobile phone 200 each time. If it transmits to the server 300, the interactive help display as mentioned above as step S10 of FIG. 4 can be utilized.

  In addition to inputting the device ID and the status code displayed on the display unit of the digital multi-function peripheral 100 from the operation input unit 203 of the mobile phone 200, the digital multi-function peripheral 100 generates a QR code representing the device ID and the status code. You may show on a display part.

  In this case, the mobile phone 200 transmits the device ID and the status code read from the QR code image data captured by the camera 201 to the help server 300. These functions are realized by the marker information acquisition unit 202 and the help data acquisition unit 205.

  In this way, the QR code displayed on the display unit of the digital multifunction peripheral 100 is photographed by the camera 201 of the mobile phone 200, and the device ID and the status code included in the QR code are read and automatically transmitted to the help server 300. By providing a function for updating the status code, transmission of the status code and the like from the mobile phone 200 can be performed with few operations. As a result, it is possible to expect an effect of reducing the degree of decrease in convenience at the time of failure.

  The help data provided by the help server is composed of non-image data such as vertex information, color information, and character information, and is constructed as image data by a display image generation unit in a mobile phone and displayed. As a result, the amount of data transmitted from the help server to the mobile phone can be reduced. Thereby, the effect of improving the response speed from data transmission to display and lowering the cost for communication can be expected.

  In addition to the device ID, the marker 101 includes information (surface code) for identifying a surface such as a side surface, a rear surface, and a surface inside the device, in addition to the device ID, and corresponds to each surface of the target device. A marker is affixed.

  Thereby, different help can be displayed depending on the position where the user points the mobile phone toward the target device (location to be imaged), and the help display can be applied to a wide range of the target device as much as possible. The effect of improving user convenience can be expected.

  In the above embodiment, a digital multifunction peripheral is used as an example of the target device. That is, it is possible to support a target device, such as a digital multifunction device, which is complicated to some extent and for which help information is important for operation. As a result, it is possible to expect an effect that allows the user to quickly remove the failure at the time of the failure.

  In the above-described embodiment, a mobile phone is used as an example of a mobile communication terminal, and an image is captured by the camera 201 included in the mobile phone. However, a terminal having an imaging function and a communication function, such as a digital camera, can also be used. It is. Thus, by using a familiar device as the mobile communication terminal, the user does not need to prepare a special device. As a result, the effect of shortening the time can be further increased, and the effect of reducing the physical and psychological barrier to use can be expected.

  Each of the above-described embodiments is a preferred embodiment of the present invention, and the scope of the present invention is not limited to only the above-described embodiments, and those skilled in the art do not deviate from the gist of the present invention. It is possible to construct a form in which various modifications are made by modifying or substituting the embodiment.

  For example, the control operation in the above-described embodiment can be executed by hardware, software, or a combined configuration of both.

  When executing processing by software, whether the program is read from the ROM storing the program recording the processing sequence to the memory (RAM) in the computer incorporated in the dedicated hardware and executed. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

  For example, the program can be recorded in advance on a hard disk or a ROM (Read Only Memory) as a recording medium. Alternatively, the program is stored on a removable recording medium such as a magnetic disk such as a floppy (registered trademark) disk, an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk such as an MO (Magneto optical) disk. It is possible to store (record) temporarily or permanently.

  Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the removable recording medium as described above, or is wirelessly transferred from the download site to the computer, or is wired to the computer via a network such as a LAN (Local Area Network) or the Internet. However, the computer can receive the transferred program and install it on a recording medium such as a built-in hard disk.

  In addition to being executed in time series in accordance with the processing operations described in the above embodiment, the processing capability of the apparatus that executes the processing, or a configuration to execute in parallel or individually as necessary Is also possible.

  In addition, the system described in the above embodiment can be configured to have a logical set configuration of a plurality of devices or to mix the functions of each device.

DESCRIPTION OF SYMBOLS 100 Digital multifunction device 101 Marker 200 Mobile phone 201 Camera 202 Marker information acquisition part 203 Operation input part 204 Wireless network interface 205 Help data acquisition part 206 Display image generation part 207 Liquid crystal display 300 Help server 301 Status table 302 Help table 303 Help data table 310 Storage Unit 320 Network Interface 330 CPU

JP 2001-31462 A Japanese Patent No. 4154285 JP 2008-193640 A

Augmented Reality Construction Method Using Two-dimensional Matrix Code, Junichi Kakimoto, WISS96

Claims (15)

A target information, a server device that provides help information for assisting operation of the target device, and a mobile communication terminal that receives the help information are connected to a network, and a help information providing system,
The target device is
A marker representing its own device ID uniquely assigned to each target device and a surface code indicating a location where a marker for detecting the position / tilt of the device is affixed is affixed,
A transmission means for transmitting the device ID and a status code indicating the status of the device to the server device;
The mobile communication terminal is
Imaging means for imaging the target device;
Marker information acquisition means for extracting an image of the marker from the captured image data and acquiring the device ID and the surface code of the target device from the extracted marker image;
Transmitting means for transmitting the acquired device ID and surface code to the server device,
The server device
Based on the device ID and the status code transmitted from the target device and the device ID and the surface code transmitted from the mobile communication terminal, unique help information corresponding to the status of the target device is obtained from the mobile communication terminal. A transmission means for transmitting to
Further, the mobile communication terminal is
Generating means for synthesizing the help information transmitted from the server device and the captured image data to generate help information providing image data;
A help information providing system comprising: display means for displaying the generated help information providing image data. When the status code is changed, the transmission means included in the server device transmits new help data to the mobile communication terminal,
The generation unit and the display unit included in the mobile communication terminal generate and display the help information providing image data and update each time help information is newly transmitted from the server device. The help information providing system according to 1.   The transmission unit included in the target device transmits a new status code corresponding to the changed state and a device ID to the server device when the state of the device changes. Help information providing system. The target device includes a state information display unit that displays a new state code corresponding to the changed state and a device ID when the state of the device changes.
The help information providing system according to claim 2, wherein the transmission unit included in the mobile communication terminal transmits the new status code and the device ID input by the user to the server device.   5. The help information providing system according to claim 4, wherein the status information display means included in the target device displays the new status code and the device ID as a QR code.   The said help information is comprised from non-image data, such as vertex information, color information, and character information, and is constructed | assembled as image data by the production | generation means with which the said mobile communication terminal is provided. Help information providing system according to item 1.   The help information providing system according to claim 1, wherein the target device is a digital multi-function peripheral.   8. The help information providing system according to claim 1, wherein the mobile communication terminal is a mobile phone.   8. The help information providing system according to claim 1, wherein the portable communication terminal is a digital camera. A target device, a mobile communication terminal, and a server device that provides help information for assisting operation of the target device to the mobile communication terminal, the server device of the help information providing system connected to a network, ,
Based on the device ID and status code transmitted from the target device and the device ID and surface code transmitted from the mobile communication terminal, unique help information corresponding to the status of the target device is obtained from the mobile communication terminal. The server apparatus characterized by including the transmission means to transmit to. A target device, a mobile communication terminal, and a server device that provides help information for assisting operation of the target device to the mobile communication terminal are the target devices of a help information providing system connected to a network. ,
The target device is affixed with a marker representing its own device ID uniquely assigned to each target device, and a surface code indicating a location where a marker for device position / tilt detection is affixed,
A target device comprising: a transmission unit configured to transmit the device ID of itself and a status code indicating the state of the device to the server device. The target device, the mobile communication terminal, and the server device that provides help information for assisting the operation of the target device to the mobile communication terminal are the mobile communication terminals of the help information providing system connected to the network. And
Imaging means for imaging the target device;
Marker information acquisition means for extracting an image of the marker from the captured image data and acquiring the device ID and the surface code of the target device from the extracted marker image;
Transmitting means for transmitting the acquired device ID and surface code to the server device;
Generating means for synthesizing the help information transmitted from the server device and the captured image data to generate help information providing image data;
A mobile communication terminal comprising: display means for displaying the generated help information providing image data. A help information providing method in a help information providing system in which a target device, a server device that provides help information for assisting operation of the target device, and a mobile communication terminal that receives the help information are connected to a network. There,
The target device transmits its own device ID and a status code indicating its own status to the server device;
The mobile communication terminal extracts an image of a marker for position / inclination detection of the device affixed to the target device from captured image data obtained by capturing the target device, and is unique for each target device from the extracted marker image. Obtaining the device ID of the target device assigned to the surface code indicating the location where the marker is affixed;
The portable communication terminal transmitting the acquired device ID and surface code to the server device;
Based on the device ID and the status code transmitted from the target device and the device ID and the surface code transmitted from the portable communication terminal, the server device obtains unique help information corresponding to the status of the target device. Transmitting to the mobile communication terminal;
The portable communication terminal includes the step of combining the help information transmitted from the server device and the captured image data, generating help information providing image data, and displaying the help information providing image data on a display unit. Information provision method. A help information providing program in a help information providing system in which a target device, a server device that provides help information for assisting operation of the target device, and a mobile communication terminal that receives the help information are connected to a network. There,
A process in which the target device transmits its device ID and a status code indicating its own status to the server device;
The mobile communication terminal extracts an image of a marker for position / inclination detection of the device affixed to the target device from captured image data obtained by capturing the target device, and is unique for each target device from the extracted marker image. A process of acquiring the device ID of the target device assigned to and the surface code indicating the location where the marker is attached;
A process in which the mobile communication terminal transmits the acquired device ID and surface code to the server device;
Based on the device ID and the status code transmitted from the target device and the device ID and the surface code transmitted from the portable communication terminal, the server device obtains unique help information corresponding to the status of the target device. , Processing to transmit to the mobile communication terminal;
The portable communication terminal causes the computer to execute a process of combining the help information transmitted from the server device and the captured image data to generate help information providing image data and displaying the help information providing image data. Help information providing program.   A computer-readable recording medium for recording the processing of the help information providing program according to claim 14.

Images