JP2013205425A - Image forming apparatus, mobile terminal device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To instruct a user to appropriately remove a jam in accordance with a jam state when the jam is generated.SOLUTION: An image forming apparatus 1 at time of jam generation predicts a jam location in a conveyance path, executes a photographing instruction with respect to a mobile terminal device 8, and determines a jam state by analyzing a photographed image acquired from the mobile terminal device 8. The image forming apparatus 1 generates an instruction image for instructing a jam release operation on the basis of the determination result, and instructs a user to perform the release operation according to the jam state by displaying the instruction image overlapping the photographed image displayed by the mobile terminal device 8.

Description

  The present invention relates to an image forming apparatus, a portable terminal device, and a program, and more particularly to a technique for guiding a jam clearing operation when a jam occurs in a conveyance path for conveying a sheet.

  In general, an image forming apparatus such as an MFP (Multifunction Peripheral), when a jam (paper jam) occurs in a conveyance path for conveying a sheet, generally provides a user with operation guidance for releasing the jam. Is. However, jams that occur in the image forming apparatus occur at various locations in the conveyance path. For this reason, it is common that the work performed by the user to release the jam varies depending on the location where the jam occurs.

  2. Description of the Related Art Conventionally, an image forming apparatus as disclosed in, for example, Patent Document 1 is known in order to be able to accurately identify a location where a jam occurs. In this conventional image forming apparatus, a plurality of cameras are installed at intervals in the conveyance path, and when the occurrence of a jam is detected, a plurality of images taken by each of the plurality of cameras are displayed. When one image is selected by the user from the plurality of images, it is specified that a jam has occurred at the installation position of the camera corresponding to the selected one image. The image forming apparatus guides the user to the release operation by acquiring the release information of the jam at the camera installation position.

JP 2006-305820 A

  However, in the conventional image forming apparatus, it is necessary to install a plurality of cameras in the conveyance path, and it is also necessary to install a lighting fixture in order to perform photographing with the camera satisfactorily. Therefore, there is a problem that the peripheral configuration of the conveyance path is increased in size and complexity, and the cost of parts of the image forming apparatus is increased. Further, in the conventional image forming apparatus, the user himself / herself has to select an image on which the paper causing the jam is taken from a plurality of images, and there is a problem that the working efficiency is poor.

  Further, when a jam occurs in the image forming apparatus, in order for the user to remove the paper that has caused the jam, an appropriate operation must be performed according to the state of the paper. In other words, even if a jam occurs at the same location in the transport path, if the state of the paper is different, the work content for removing the paper will be different. For example, if the paper is wound around a roller, the roll must be released by rotating the roller in the opposite direction, whereas the paper is not wound around the roller. Therefore, it is not necessary to perform such a roller rotation operation. However, since the conventional image forming apparatus simply acquires release information according to the installation position of the camera that has captured the image on which the paper is captured, an appropriate release to be performed by the user according to the state of the paper Cannot guide the operation.

  For this reason, conventionally, it is a fact that various release operations to be performed by the user according to the state of the sheet must be described in, for example, an instruction manual. However, if various release operations corresponding to the state of the paper are described in the instruction manual or the like for each position where a jam can occur in the transport path, the number of pages becomes enormous. In particular, in order to explain the canceling operation in an easy-to-understand manner, the number of pages becomes even larger when illustrations are used frequently. For this reason, after the user visually confirms the state of the paper, it takes time to find an appropriate release operation method corresponding to the state of the paper, and the jam cannot be efficiently removed.

  Even when many illustrations are used, the angle viewed by the user may differ from the illustration, and the paper state visually confirmed by the user may not always match the paper state shown in the illustration. For this reason, it is difficult for the user to know which of the various release operations should be performed, and there may be a case where an appropriate release operation method cannot be found.

  In addition, various release operations that should be performed by the user according to the state of the paper are not described in the instruction manual, but are stored in advance as an image in the image forming apparatus, for example, based on the selection operation by the user. Even when an image corresponding to one release operation is displayed, there is a problem similar to the above.

  Accordingly, the present invention has been made to solve the above-described problem, and when a jam occurs, an image that can guide the user to an operation that can appropriately release the jam according to the jam state. An object of the present invention is to provide a forming device, a portable terminal device, and a program.

  In order to achieve the above object, an invention according to claim 1 is an image forming apparatus that has a conveyance path for conveying a sheet and detects occurrence of a jam in the conveyance path. A detection sensor for detecting, a communication means for communicating with a portable terminal device having an imaging function, and a jam occurrence position in the conveyance path based on a position of the paper detected by the detection sensor when a jam occurs in the conveyance path. Predicting and photographing guidance means for photographing the predicted location to the portable terminal device, and obtaining a photographed image photographed by the portable terminal device based on the photographing guidance and performing image analysis Based on the determination result in the image analysis means, a guide image for guiding the jam release operation is generated. And, by displaying superimposed on the captured image displayed in the portable terminal device, and the release operation guide means for guiding the release operation in accordance with a jammed state, a structure which is characterized in that it comprises.

  According to a second aspect of the present invention, in the image forming apparatus according to the first aspect, an open / close door that can be manually operated by a user and that opens the transport path, and the open / close door opens the transport path. An opening sensor that detects the opening of the opening and closing door when the opening sensor does not detect the opening of the conveyance path after the shooting guide means performs shooting guidance. An opening / closing door operation guide image for guiding the operation is generated and displayed superimposed on the captured image displayed on the mobile terminal device, so that the transport path is opened and the predicted location can be captured. It is the structure characterized by guiding opening operation to do.

  According to a third aspect of the present invention, in the image forming apparatus according to the second aspect, when the release operation guiding means guides the opening operation, and the paper that is a jam factor is not detected by the image analyzing means, A shooting guide image for changing the shooting position or shooting direction by the mobile terminal device is generated and displayed superimposed on the shot image displayed on the mobile terminal device, so that the shooting position or shooting direction by the mobile terminal device is displayed. It is the structure characterized by guiding.

  According to a fourth aspect of the present invention, in the image forming apparatus according to any one of the first to third aspects, the image analysis unit identifies a position and a state of the paper that is a jam factor by analyzing the captured image, The release operation guide means determines a paper extraction operation according to the position and state of the paper specified by the image analysis means, and generates a jam release operation guide image for performing the extraction operation. The configuration is as follows.

  According to a fifth aspect of the present invention, in the image forming apparatus according to the fourth aspect, the extraction operation includes an operation of a movable part that is operable in the conveyance path and that can be operated by a user. is there.

  According to a sixth aspect of the present invention, in the image forming apparatus according to the fourth or fifth aspect, the image analysis unit is configured to perform the portable operation based on a position of a reference point image included in a photographed image photographed by the portable terminal device. By identifying the shooting position of the terminal device and correcting the paper image included in the shot image shot by the mobile terminal device based on the shooting position of the mobile terminal device and matching it with a predetermined jam pattern image It is the structure characterized by specifying the state of.

  According to a seventh aspect of the present invention, in the image forming apparatus according to any one of the first to sixth aspects, the image analysis unit is based on a position of a reference point image included in a photographed image photographed by the portable terminal device. The shooting position of the mobile terminal device is specified, and the release operation guide means deforms the guide image according to the shooting position of the mobile terminal device when generating the guide image. .

  According to an eighth aspect of the present invention, in the image forming apparatus according to any one of the first to seventh aspects, after the release operation guide means guides the release operation, the image analysis means no longer detects a sheet. When the recovery operation guide image for guiding the recovery operation after the jam release is generated and displayed superimposed on the captured image displayed on the mobile terminal device, the recovery operation after the jam release is guided It is the structure characterized by these.

  According to a ninth aspect of the present invention, in the image forming apparatus according to any one of the first to eighth aspects, the portable terminal device is an operation panel that can be attached to and detached from the image forming apparatus. It is.

  According to a tenth aspect of the present invention, there is provided a portable terminal device capable of communicating with an image forming apparatus, an imaging unit that captures an image, a display unit that displays a captured image captured by the imaging unit, and the image forming apparatus. The image pickup device is driven to start shooting based on a shooting guide received from the image forming apparatus when a jam occurs in a conveyance path for carrying the paper through the camera. The image capturing control means for transmitting the image to the image forming apparatus and the release operation guide image for guiding the jam release operation from the image forming apparatus are overlaid on the captured image displayed on the display means. And a display control means for displaying the release operation guidance image together.

  According to an eleventh aspect of the present invention, there is provided a portable terminal device capable of communicating with an image forming apparatus, an imaging unit that captures an image, a display unit that displays a captured image captured by the imaging unit, and the image forming apparatus. A shooting control unit that starts shooting by driving the imaging unit based on a shooting guide received from the image forming apparatus when a jam occurs on a conveyance path for carrying a sheet through the sheet, and the imaging unit The image analysis means for detecting the paper that is the cause of the jam in the image forming apparatus and determining the jam state by acquiring the photographed image captured by the image processing apparatus, and the jam determined by the image analysis means Based on the state, a release operation guide image for guiding the jam release operation is generated and displayed on the photographed image displayed on the display means. By the release operation guide means for guiding the release operation in accordance with a jammed state, a structure which is characterized in that it comprises.

  According to a twelfth aspect of the present invention, there is provided a program that is executed in a portable terminal device that includes an imaging unit that captures an image and a display unit that displays a captured image captured by the imaging unit. The portable terminal device drives the imaging means based on a shooting guide received from the image forming apparatus when a jam occurs in a conveyance path for carrying paper through the image forming apparatus. Shooting control means for starting shooting and transmitting a shot image obtained by the imaging means to the image forming apparatus, and receiving a release operation guide image for guiding a jam release operation from the image forming apparatus. And a display control means for displaying the release operation guidance image superimposed on the photographed image displayed on the display means. It is configured to.

  According to a thirteenth aspect of the present invention, there is provided a program that is executed in a portable terminal device that includes an imaging unit that captures an image and a display unit that displays a captured image captured by the imaging unit. The portable terminal device drives the imaging means based on a shooting guide received from the image forming apparatus when a jam occurs in a conveyance path for carrying paper through the image forming apparatus. An image analysis unit that detects a jam as a jam factor in the image forming apparatus by acquiring a captured image captured by the image capturing unit and performing image analysis. And a release operation guidance image for guiding the jam release operation based on the jam state determined by the image analysis means By displaying superimposed on the captured image displayed in the display unit, a configuration which is characterized in that function as release operation guidance means for guiding the release operation according to the jammed state.

  According to the present invention, when a jam occurs in the conveyance path of the image forming apparatus, it is possible to guide the user to an operation that can appropriately release the jam according to the jam state.

1 is a diagram illustrating an example of an external configuration of an image forming apparatus. FIG. 3 is a diagram illustrating an example of a paper transport mechanism provided inside the image forming apparatus. FIG. 4 is a diagram illustrating an example of an apparatus state when a jam release operation is performed in the image forming apparatus. It is a block diagram which shows an example of the control mechanism of an image forming apparatus and a portable terminal device. It is a figure which shows an example of cancellation | release operation guidance information. It is a figure explaining the method of specifying the imaging | photography position of a portable terminal device. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 10 is a diagram illustrating an example of various guide images displayed on the mobile terminal device in accordance with the result of image analysis when a jam occurs in the image forming apparatus. FIG. 3 is a diagram illustrating an outline of a processing sequence performed in cooperation between the apparatus main body of the image forming apparatus and the mobile terminal device in the first embodiment. 6 is a flowchart illustrating an example of a processing procedure performed by the image forming apparatus when a jam occurs. It is a flowchart which shows an example of the detailed process sequence of an image analysis process. It is a flowchart which shows an example of the detailed process sequence of cancellation | release operation guidance processing. 10 is a flowchart illustrating an example of a detailed processing procedure of image generation processing for generating a release operation guide image based on a paper position and a paper state. It is a block diagram which shows an example of the control mechanism of the image forming apparatus in 2nd Embodiment, and a portable terminal device. It is a figure which shows the outline | summary of the processing sequence which the apparatus main body of an image forming apparatus and a portable terminal device cooperate in 2nd Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings. In the embodiments described below, members that are common to each other are denoted by the same reference numerals, and redundant descriptions thereof are omitted.

(First embodiment)
FIG. 1 is a diagram illustrating an example of an external configuration of an image forming apparatus 1 according to the present embodiment. The image forming apparatus 1 is an apparatus called an MFP, for example, and includes a plurality of functions such as a print function, a scan function, and a copy function.

  The image forming apparatus 1 includes a scanner unit 2 that reads an original and generates image data on an upper portion of the apparatus main body, and an original that automatically takes the original set by the user to a document reading position of the scanner unit 2. And a transport unit 3. The scanner unit 2 and the document transport unit 3 operate when a scan function or a copy function is selected, and start a document reading operation based on a job execution instruction from a user.

  Further, the image forming apparatus 1 includes a printer unit 4 that performs print output by forming an image on a sheet based on image data at the center of the apparatus main body. A plurality of paper feed cassettes 5 are provided below the printer unit 4 so as to feed the paper serving as a printing medium to the printer unit 4 one by one. The printer unit 4 and the paper feed cassette 5 operate when a print function or a copy function is selected, and sheets are fed one by one from the paper feed cassette 5 to a conveyance path provided inside the apparatus body. In step 4, an image is formed on the sheet. The paper on which image formation has been performed in the printer unit 4 is further transported through the transport path, and is discharged to a paper discharge unit 6 provided at the top of the printer unit 4.

  Further, an operation panel 7 serving as a user interface of a user who uses the image forming apparatus 1 is provided on the front side of the image forming apparatus 1. The operation panel 7 displays various operation screens for the user who uses the image forming apparatus 1 and accepts operation input by the user while the various operation screens are displayed. Therefore, the user can select a function to be operated in the image forming apparatus 1, perform detailed job settings, or issue a job execution instruction by operating the operation panel 7. .

  As shown in FIG. 1, the operation panel 7 can be attached to and detached from the apparatus main body of the image forming apparatus 1, and the user can remove the operation panel 7 from the apparatus main body and use it as necessary. . That is, when the operation panel 7 is detached from the apparatus main body of the image forming apparatus 1, the operation panel 7 functions as a portable terminal apparatus 8 similar to a general tablet terminal apparatus. Further, as will be described later, an imaging unit 53 such as a camera capable of taking an image is provided on the back side of the operation panel 7. Therefore, when the operation panel 7 functions as the mobile terminal device 8, the image on the back side can be taken by the imaging unit 53. Further, when the operation panel 7 functions as the mobile terminal device 8, the operation panel 7 performs wireless communication with the apparatus main body of the image forming apparatus 1.

  In the image forming apparatus 1 as described above, a paper transport mechanism for transporting paper as a printing medium is incorporated inside the printer unit 4 and the paper feed cassette 5, and at the time of printing output by the paper transport function. A conveyance path through which the sheet is conveyed is defined. An open / close door 9 is provided on the side surface of the image forming apparatus 1 to open the conveyance path when a jam (paper jam) occurs in the conveyance path. The open / close door 9 is opened manually by a user when a jam occurs. When no jam occurs, the closed state is obtained as shown in FIG. The image forming apparatus 1 can execute a job with the open / close door 9 closed.

  Also, three types of reference markers M 1, M 2, and M 3 are provided on the side surface of the image forming apparatus 1 at three locations around the open / close door 9. These reference markers M 1, M 2, and M 3 are provided with different colors, patterns, symbols, and the like, and are provided at predetermined reference positions around the open / close door 9.

  FIG. 2 is a diagram illustrating an example of the paper transport mechanism 10 provided inside the image forming apparatus 1. The paper transport mechanism 10 transports the paper fed from the paper feed cassette 5 in the direction indicated by the arrow F1 in the figure and discharges it to the paper discharge unit 6. A registration roller 11, a transfer belt 12, a secondary transfer roller 13, a fixing unit 17, and a discharge roller 18 are provided on the conveyance path indicated by the arrow F 1. The fixing unit 17 includes a fixing heater 14, a fixing roller 15, and a pressure roller 16, and the sheet passes between the fixing roller 15 and the pressure roller 16. Further, a detection sensor 20 that detects a sheet is provided at a predetermined position before the registration roller 11, and a detection sensor 21 that detects a sheet is also provided at a predetermined position behind the fixing unit 17.

  In the case of single-sided printing, the paper fed from the paper feed cassette 5 passes through the conveyance path indicated by the arrow F1 in the figure and is discharged to the paper discharge unit 6. That is, when the leading edge of the fed paper is detected by the detection sensor 20, the paper is temporarily stopped so as to abut the registration roller 11. Then, the image (toner image) formed on the transfer belt 12 is conveyed from the registration roller 11 toward the secondary transfer roller 13 in time for reaching the secondary transfer roller 13. When the sheet passes between the transfer belt 12 and the secondary transfer roller 13, an image is transferred to one side of the sheet, and image formation is performed. When the sheet that has passed through the transfer position passes between the fixing roller 15 and the pressure roller 16 of the fixing unit 17, the sheet is subjected to a heat and pressure process to fix the image transferred on the entire surface of the sheet. Then, the paper is discharged to the paper discharge unit 6 by the discharge roller 18.

  Further, the paper transport mechanism 10 has a transport path for inverting the paper on which an image is formed on one side in the case of duplex printing. That is, when the trailing edge of the sheet on which the image is formed on one side is detected by the detection sensor 21 and the trailing edge of the sheet is held by the discharge roller 18, the discharge roller 18 is driven reversely, and the sheet is moved to the arrow in the figure. The sheet is conveyed in the direction indicated by F2 and guided again to the registration roller 11. A plurality of transport rollers 19 are provided in the transport path indicated by the arrow F2. Therefore, in the case of double-sided printing, the sheet is reversed in the conveyance path indicated by the arrow F2, so that image formation can be performed on the other side different from the surface on which image formation has been performed first. .

  The sheet conveying mechanism 10 having the conveying path as described above is a member in which a portion surrounded by a broken line in the drawing is incorporated in the open / close door 9. That is, one of the pair of registration rollers 11, the secondary transfer roller 13, and a plurality of conveyance rollers 19 for reversing and conveying the paper are attached to the open / close door 9. As for the fixing unit 17, the fixing heater 14, the fixing roller 15, and the pressure roller 16 are integrally attached to the apparatus main body side. Therefore, when the open / close door 9 is opened on the side surface of the image forming apparatus 1, the conveyance path indicated by the arrow F <b> 1 described above can be opened except for a portion that passes through the fixing unit 17.

  The image forming apparatus 1 according to the present embodiment prompts the user to remove the operation panel 7 from the apparatus body and use it as the portable terminal device 8 when a jam occurs in the conveyance path as described above. The image forming apparatus 1 is configured to perform various kinds of operation guidance for releasing the jam in the mobile terminal device 8 when the user is holding the mobile terminal device 8.

  FIG. 3 is a diagram illustrating an example of an apparatus state when a jam release operation is performed in the image forming apparatus 1. When a jam occurs in the image forming apparatus 1, the user holds the portable terminal device 8 as shown in FIG. 3 and starts photographing the image forming apparatus 1 using the portable terminal device 8. At this time, the captured image is displayed on the mobile terminal device 8. The image forming apparatus 1 uses an augmented reality (AR) technique on an image displayed on the mobile terminal device 8 to display an image related to an operation to be performed by the user in real time. In this way, the user is guided to the optimum release operation. Note that augmented reality is a technique in which additional information generated by a computer is superimposed on an actual image obtained by shooting.

  That is, when a jam occurs, the image forming apparatus 1 according to the present embodiment causes the user to carry the portable terminal device 8 and photograph the state of the conveyance path where the jam has occurred. And the state of jam is discriminate | determined by image-analyzing the picked-up image obtained by the portable terminal device 8. FIG. The image forming apparatus 1 is configured to generate a guide image for guiding a jam release operation to be performed by the user based on the result of the image analysis, and to display the guide image on the mobile terminal device 8. Hereinafter, the image forming apparatus 1 will be described in more detail.

  FIG. 4 is a block diagram illustrating an example of a control mechanism of the image forming apparatus 1 and the mobile terminal device 8. As shown in FIG. 4, the image forming apparatus 1 includes a controller 30, a ROM 31, the above-described paper transport mechanism 10, a storage unit 32, and a communication unit 33.

  The controller 30 includes a CPU and a memory (not shown), and controls the operation of each unit in the image forming apparatus 1 by reading and executing a program 34 stored in advance in the ROM 31. In particular, when a jam occurs in the image forming apparatus 1, the controller 30 functions as a shooting guide unit 41, an image analysis unit 42, and a release operation guide unit 43. The image analysis unit 42 further functions as a shooting position specifying unit 45, a paper detection unit 46, a paper image correction unit 47, and a jam state determination unit 48. Details of processing performed by each of these units will be described in detail later.

  The paper transport mechanism 10 includes detection sensors 20 and 21 that detect paper in the transport path, an open sensor 22 that detects that the open / close door 9 is opened, and a jam detector 35 that detects a jam in the transport path. It has. For example, the jam detection unit 35 detects the occurrence of a jam when the above-described various rollers stop abnormally during the conveyance of a sheet.

  The storage unit 32 is a nonvolatile raw storage unit such as NVRAM or a hard disk device. The storage unit 32 stores reference point information 36 and release operation guidance information 37 in advance. The reference point information 36 is information indicating the relative positions of the three reference markers M1, M2, and M3 provided on the side surface of the apparatus main body of the image forming apparatus 1. The release operation guidance information 37 is information that is referred to when various guide images for guiding the user to perform a jam release operation are generated.

  FIG. 5 is a diagram illustrating an example of the release operation guide information 37. As shown in FIG. 5, the release operation guidance information 37 is information in which the device state, the jam state, and the guidance information are associated with each other.

  The device state is a state of the device detected by various sensors such as an open / closed state of the door 9. Therefore, the device state of the release operation guidance information 37 is a device state that can be detected by the image forming apparatus 1 and is registered in advance as a device state related to a jam release operation. FIG. 5 shows an example in which only the open / close state of the door 9 is registered in the apparatus state, but the apparatus state detected by other sensors may be further registered. .

  The jam state is a jam state determined by the image analysis unit 42. In other words, in this jam state, all of a plurality of types of jam states that can be detected by the image analysis unit 42 are registered. In this embodiment, since the position and state of the paper that caused the jam can be detected as the jam state, the release operation guide information 37 individually registers the jam state according to the paper position and the paper state. Has been. It should be noted that the jam state is not detected when the image analysis unit 42 cannot detect the paper causing the jam from the photographed image taken by the mobile terminal device 8, and therefore, the jam state shown in FIG. Such an undetected state is also registered as a jam state.

  The guidance information is information for performing operation guidance for the user. This guidance information includes guidance text, reference image, and display position information. The guidance text is information for displaying in text the operation for guiding the user. The operation image is an image such as an arrow registered in advance to indicate to the user the operation direction of the open / close door 9 and other operation levers, or to indicate the direction of the paper removal operation causing the jam. is there. The display position is information defining a display position for displaying the guidance text and the operation image. Such guidance information is registered in advance according to the type of the device state and the jam state.

  Returning to FIG. 4, the communication unit 33 of the image forming apparatus 1 performs wireless communication with the portable terminal device 8 when a jam occurs.

  On the other hand, the mobile terminal device 8 includes a controller 50, an imaging unit 53, a display unit 54, a communication unit 55, and a memory 56.

  The controller 50 includes a CPU and a memory (not shown). The controller 50 reads and executes a program 57 stored in advance in the memory 56, thereby controlling the operation of each unit in the mobile terminal device 8. In particular, when a jam occurs in the image forming apparatus 1 and shooting guidance is received from the image forming apparatus 1, the controller 50 functions as a shooting control unit 51 and a display control unit 52.

  The imaging unit 53 is provided on the back side of the mobile terminal device 8 as described above, and can capture an image on the back side of the mobile terminal device 8. When the imaging unit 53 starts imaging based on an instruction from the controller 50, the imaging unit 53 performs imaging of a plurality of frames per second and outputs sequentially obtained captured images to the controller 50.

  The display unit 54 is configured by a color liquid crystal display, for example, and displays various images to a user who operates the mobile terminal device 8. In particular, when the imaging unit 53 is performing a shooting operation, an image shot by the imaging unit 53 can be displayed as a live view image.

  The communication unit 55 performs wireless communication with the image forming apparatus 1. The memory 56 is a non-volatile memory such as NVRAM, and stores the above-described program 57 in advance. For example, the program 57 is configured as a dedicated program for guiding a user to cancel a jam in cooperation with the apparatus main body of the image forming apparatus 1 when a jam occurs.

  Next, the operation of each part in the above configuration will be described. In the image forming apparatus 1, when the jam detection unit 35 of the paper transport mechanism 10 detects the occurrence of a jam, first, the photographing guide unit 41 of the controller 30 functions. The photographing guide unit 41 reads the output signals of the detection sensors 20 and 21 that detect the paper, and predicts where a jam has occurred in the conveyance path. For example, when the detection sensor 20 installed at a position in front of the registration roller 11 detects a sheet, the photographing guide unit 41 causes a jam at a relatively lower position in the conveyance path shown in FIG. Predict. On the other hand, when the detection sensor 21 provided at the rear position of the fixing unit 17 detects the sheet, the photographing guide unit 41 causes a jam at a position relatively above the conveyance path shown in FIG. Predict that

  When the shooting guide unit 41 predicts a jam occurrence position as described above, the shooting guide unit 41 determines a shooting target region for shooting the predicted position. For example, in the image forming apparatus 1 of the present embodiment, since the opening / closing door 9 for opening the conveyance path is provided on the right side surface of the apparatus main body, the imaging target region is determined as the right side surface of the apparatus main body. In addition, when there are a plurality of opening / closing doors 9 for opening the conveyance path, the photographing guide unit 41 is provided with an opening / closing door for opening a predicted position in the conveyance path from among the plurality of opening / closing doors 9. Is determined as an imaging target area. Then, the photographing guide unit 41 transmits the photographing target region determined as described above to the mobile terminal device 8 and performs photographing guidance for the user who operates the mobile terminal device 8.

  When the mobile terminal device 8 receives the shooting guidance from the image forming apparatus 1, the controller 50 activates the program 57. Thereby, the controller 50 functions as the imaging control unit 51 and the display control unit 52. Then, the shooting control unit 51 drives the imaging unit 53 to start a shooting operation by the imaging unit 53.

  Further, the display control unit 52 displays the shooting target area determined by the shooting guide unit 41 on the display unit 54 and prompts the user to take a picture of the shooting target area. That is, the display control unit 52 urges the user to take an image of a peripheral area where the opening / closing door 9 that opens the predicted portion is provided as an imaging target area. Further, the display control unit 52 urges the user to capture the reference markers M1, M2, and M3 provided at three locations around the open / close door 9 within the field of view of the imaging unit 53.

  On the other hand, the imaging control unit 51 drives the imaging unit 53 to start the imaging operation, and then sequentially inputs captured images captured by the imaging unit 53, and outputs the captured images to the display control unit 52. The display control unit 52 displays the captured image captured by the imaging unit 53 on the display unit 54. In addition, the shooting control unit 51 transmits a shot image obtained from the imaging unit 53 to the image forming apparatus 1 via the communication unit 55.

  When the image forming apparatus 1 obtains a photographed image photographed by the mobile terminal device 8, the photographing position specifying unit 45, the paper detection unit 46, the paper image correction unit 47, and the jam state determination unit 48 sequentially function in the image analysis unit 42. Perform image analysis of captured images.

  The shooting position specifying unit 45 specifies the relative shooting position of the mobile terminal device 8 based on the shot image obtained in a state where the mobile terminal device 8 is shooting the shooting target region. FIG. 6 is a diagram illustrating a method for specifying the shooting position of the mobile terminal device 8. For example, as illustrated in FIG. 6A, when the captured image G10 in which three reference markers M1, M2, and M3 are captured is acquired from the mobile terminal device 8, the captured position specifying unit 45 displays the captured image. The position of the reference point image corresponding to each of the reference markers M1, M2, and M3 in G10 is specified. Then, the shooting position specifying unit 45 reads the reference point information 36 from the storage unit 32 and performs a calculation based on the reference point information 36 and the position of the reference point image, thereby specifying the shooting position of the mobile terminal device 8. . That is, the positions of the three reference markers M1, M2, and M3 are known on the side surface of the image forming apparatus 1, and the distance between any two of the three reference markers M1, M2, and M3 is also known. It is. Furthermore, the internal angle of the triangle formed by these three reference markers M1, M2, and M3 is also known. Therefore, based on the position where the reference point images corresponding to each of the reference markers M1, M2, and M3 are captured in the captured image G10, calculation is performed by applying the trigonometric method, whereby the shooting position of the mobile terminal device 8 is determined. Can be identified. Then, the shooting position specifying unit 45 specifies the shooting position of the mobile terminal device 8 in the XYZ three-dimensional coordinate system virtually defined on the side surface of the image forming apparatus 1 as shown in FIG. 6B. . When the shooting position is specified in this way, the shooting direction in which the mobile terminal device 8 is shooting the shooting target area is also specified.

  If the reference point images corresponding to the three reference markers M1, M2, and M3 cannot be extracted from the captured image obtained from the mobile terminal device 8, the imaging position specifying unit 45 The shooting position cannot be specified.

  The paper detection unit 46 is a processing unit that extracts a paper image that causes a jam from a captured image. For example, the paper detection unit 46 extracts, as a paper image, an image area in which the saturation is lower than a predetermined threshold and the brightness is higher than the predetermined threshold. Note that when there is no image area that meets such conditions in the captured image, no paper image is extracted.

  The paper image correcting unit 47 corrects the paper image extracted by the paper detecting unit 46 based on the shooting position of the mobile terminal device 8 specified by the shooting position specifying unit 45, so that the shape of the paper image is previously obtained. Correction is made to the shape of the paper image extracted when the image is taken from the set photographing reference position and photographing reference direction. For example, the paper image is enlarged when the mobile terminal device 8 is farther from the image forming apparatus 1 than the photographing reference position, and the paper image is reduced when the mobile terminal device 8 is approaching the image forming apparatus 1. . Further, when the mobile terminal device 8 is located higher than the shooting reference position and the depression angle of the mobile terminal device 8 is larger than the reference shooting direction, the keystone correction is performed so that the reduction ratio of the upper part of the paper image is larger than that of the lower part. I do. By such correction, the shape of the paper image included in the photographed image becomes the same shape as the shape of the paper image when photographed from a predetermined photographing reference position and photographing reference direction. Note that, when the shooting position is not specified by the shooting position specifying unit 45, or when the paper image is not extracted by the paper detection unit 46, the above correction is not performed.

  The jam state determination unit 48 determines the state of a jam occurring on the conveyance path based on the processing result. The jam state determination unit 48 determines the position of the paper in the conveyance path when the shooting position of the mobile terminal device 8 is specified by the shooting position specifying unit 45 and the paper image is extracted by the paper detection unit 46. The position is specified, and the state of the paper is specified based on the paper image corrected by the paper image correction unit 47.

  For example, when the photographed image photographed by the mobile terminal device 8 includes three reference point images, and the photographing position of the portable terminal device 8 can be specified, the registration roller 11 and the secondary transfer described above are included in the photographed image. It is possible to specify in which position the roller 13 and the fixing unit 17 are reflected. Therefore, the jam state determination unit 48 specifies the position from which the paper image extracted by the paper detection unit 46 is extracted with respect to the registration roller 11, the secondary transfer roller 13, and the fixing unit 17. The position of the paper in the road is specified.

  Further, the jam state determination unit 48 specifies the sheet state by matching the shape of the sheet image corrected by the sheet image correction unit 47 with a plurality of types of jam pattern images registered in advance. The jam pattern image registered in advance is an image in which various sheet states that may occur when a jam occurs in the image forming apparatus 1 are previously captured from the above-described shooting reference position and shooting reference direction for each occurrence position. is there. Therefore, the jam state determination unit 48 matches the shape of the paper image corrected by the paper image correction unit 47 with a plurality of types of jam pattern images registered in advance, so that the paper is wound around various rollers. It is possible to specify the state of the paper such as whether or not it is, or whether or not it is in the accordion state.

  On the other hand, when the shooting position of the mobile terminal device 8 is not specified by the shooting position specifying unit 45, or when the paper image is not extracted by the paper detection unit 46, the jam state determination unit 48 is as described above. It is impossible to specify the position of the paper and the state of the paper that caused the jam. Therefore, in such a case, the jam state determination unit 48 determines the jam state by regarding the paper position and the paper state as undetected states. If the shooting position of the mobile terminal device 8 is not specified, it is determined that the reference point image has not been extracted.

  When the image analysis is performed as described above, the release operation guide unit 43 functions next. The release operation guide unit 43 generates various guide images for guiding the jam release operation based on the image analysis result in the image analysis unit 42, and superimposes the guide images on the captured image displayed on the mobile terminal device 8. It is a processing unit that guides the release operation according to the jam state to the user by displaying them together. The release operation guide unit 43 reads the guide information corresponding to the jam state from the release operation guide information 37 based on the device state detected by the various sensors and the analysis result in the image analysis unit 42, and generates a guide image. That is, the release operation guide unit 43 reads the guide text from the release operation guide information 37 to generate a guide image, and also reads the operation image for indicating the operation direction to the user to generate the guide image.

  When the release operation guide unit 43 reads out an operation image for indicating the operation direction to the user and generates a guide image, the release operation guide unit 43 displays the operation image registered in the release operation guide information 37 as the shooting position of the mobile terminal device 8. A guide image is generated by deforming the operation image according to the above. Thereby, a guide image such as an arrow is generated as an image indicating an appropriate operation direction in accordance with the viewpoint of the mobile terminal device 8.

  Then, the release operation guide unit 43 transmits the guide image generated as described above to the mobile terminal device 8. At this time, the release operation guide unit 43 instructs the display position of each guide image to the mobile terminal device 8 based on the display position registered in the release operation guide information 37.

  The mobile terminal device 8 acquires the guide image transmitted by the release operation guide unit 43 of the image forming apparatus 1 as described above. Then, the display control unit 52 displays the guidance image on the display position instructed from the release operation guidance unit 43 by superimposing it on the captured image displayed on the display unit 54.

  7 to 14 are diagrams showing examples of various guide images displayed on the mobile terminal device 8 according to the result of image analysis when a jam occurs in the image forming apparatus 1. First, FIG. 7 illustrates a case where the mobile terminal device 8 has captured a captured image G10 as illustrated in FIG. That is, in the shooting screen G10 in FIG. 6A, the open / close door 9 is in a closed state, and thus it is not possible to extract a paper image that causes a jam in the image forming apparatus 1. At this time, the release operation guide unit 43 detects that the open / close door 9 is closed by the output of the open sensor 22, so that the open / close door 9 is closed as a device state and the jam state is not detected. Know that there is. Therefore, the release operation guide unit 43 refers to the release operation guide information 37 and, based on the guide information corresponding to the grasped state, the door operation guide image for guiding the user to open the door 9. G11 and G12 are generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 7 is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 7, a guidance image G11 indicating the opening direction of the opening / closing door 9 and a guidance image of a text prompting the user to open the opening / closing door 9 with respect to the captured image G10 captured by the imaging unit 53. G12 is superimposed and displayed. Therefore, the user can grasp how to open and close the open / close door 9 by such a display screen.

  Next, when the captured image captured by the mobile terminal device 8 is the captured image G20 illustrated in FIG. 8A, the display unit 54 of the mobile terminal device 8 displays the captured image G20 as illustrated in FIG. In addition, the captured image G20 and the guide images G21 and G22 are displayed in a superimposed state. That is, in the photographed image G20 of FIG. 8A, the open / close door 9 is opened, and the image does not include the paper that causes the jam. Therefore, the release operation guide unit 43 refers to the release operation guide information 37, generates shooting guide images G <b> 21 and G <b> 22 for changing the shooting position or shooting direction of the mobile terminal device 8, and transmits the shooting guide images G <b> 21 and G <b> 22 to the mobile terminal device 8. . As a result, an image as shown in FIG. 8B is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 8B, the guide image G21 indicating that the shooting position is moved in the horizontal direction with respect to the shot image G20 shot by the imaging unit 53, and the sheet with the shooting position changed to the user. The text guidance image G22 that prompts the user to search for the text is superimposed and displayed. Therefore, the user can grasp from this display screen that the portable terminal device 8 should be moved to copy the paper that caused the jam.

  Next, when the captured image captured by the mobile terminal device 8 is the captured image G30 illustrated in FIG. 9A, the display unit 54 of the mobile terminal device 8 displays the captured image G30 as illustrated in FIG. In addition, the captured image G30 and the guide images G31 and G32 are displayed in a superimposed state. That is, in the photographed image G30 in FIG. 9A, the open / close door 9 opens the conveyance path, and the paper image GS that causes a jam is shown. In such a state, the image analysis unit 42 determines the paper position and the paper state. For example, in the photographed image G30 shown in FIG. 9A, the sheet is bent into an accordion state at a position before the leading end of the sheet enters the fixing unit 17. Therefore, the release operation guide unit 43 refers to the release operation guide information 37, reads the guide information corresponding to such a paper position and paper state, and releases the release operation guide image G31 for appropriately removing the paper from the conveyance path. , G32 is generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 9B is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 9B, for the captured image G30 captured by the imaging unit 53, a guide image G31 indicating the paper pulling direction and a text guide image G32 that prompts the user to remove the paper. Are superimposed and displayed. Therefore, the user can appropriately remove the paper that caused the jam by using such a display screen.

  Next, when the photographed image photographed by the portable terminal device 8 is the photographed image G40 shown in FIG. 10A, the display unit 54 of the portable terminal device 8 displays the photographed image G40 as shown in FIG. In addition, the captured image G40 and the guide images G41 and G42 are displayed in a superimposed state. That is, also in the photographed image G40 in FIG. 10A, the paper image GS that causes a jam is shown. Accordingly, the paper position and the paper state are determined by the image analysis unit 42. For example, in the photographed image G40 shown in FIG. 10A, the leading end of the sheet is wound around the transfer belt 12 before the sheet enters the fixing unit 17. Therefore, the release operation guide unit 43 refers to the release operation guide information 37, reads the guide information corresponding to such a paper position and paper state, and releases the paper properly wound around the transfer belt 12. Operation guide images G41 and G42 are generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 10B is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 10B, for the captured image G40 captured by the imaging unit 53, a guide image G41 indicating the paper pulling direction and a text guide image G42 that prompts the user to remove the paper. Are superimposed and displayed. Therefore, the user can appropriately remove the paper that has caused the jam from the transfer belt 12 by such a display screen.

  Next, when the captured image captured by the mobile terminal device 8 is the captured image G50 illustrated in FIG. 11A, the display unit 54 of the mobile terminal device 8 displays the captured image G50 as illustrated in FIG. In addition, the captured image G50 and the guide images G51 and G52 are displayed in a superimposed state. That is, the sheet image GS that causes a jam is also captured in the captured image G50 of FIG. 11A. In the example of FIG. 11A, after the sheet enters the fixing unit 17, the rear end of the sheet is the accordion. It is bent into a shape and jammed. Therefore, the release operation guide unit 43 refers to the release operation guide information 37, reads the guide information corresponding to the paper position and paper state, and uses the release operation guide image G51 for appropriately removing the paper from the fixing unit 17. , G52 are generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 11B is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 11B, for the captured image G50 captured by the imaging unit 53, a guide image G51 indicating the paper pulling direction and a text guide image G52 that prompts the user to remove the paper. Are superimposed and displayed. Therefore, the user can appropriately remove the paper that caused the jam from the fixing unit 17 by using such a display screen.

  Next, when the photographed image photographed by the portable terminal device 8 is the photographed image G60 shown in FIG. 12A, the display unit 54 of the portable terminal device 8 displays the photographed image G60 as shown in FIG. In addition, the captured image G60 and the guide images G61 and G62 are displayed in a superimposed state. That is, the paper image GS that causes a jam is also shown in the photographed image G60 of FIG. 12A, and in the example of FIG. 12A, the paper is wound around the secondary transfer roller 13. . Therefore, the release operation guide unit 43 refers to the release operation guide information 37, reads the guide information corresponding to the paper position and paper state, and the release operation guide for appropriately removing the paper from the secondary transfer roller 13. The images G61 and G62 are generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 12B is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 12B, an operation of rotating the secondary transfer roller 13 to release the wrapping state around the secondary transfer roller 13 with respect to the captured image G60 captured by the imaging unit 53. A guidance image G61 indicating the direction in which the movable part such as a lever is rotated and a text guidance image G62 prompting the user to perform such a rotation operation are displayed in a superimposed manner. Therefore, the user can appropriately remove the paper that has caused the jam from the secondary transfer roller 13 by such a display screen.

  Next, when the captured image captured by the mobile terminal device 8 is the captured image G70 illustrated in FIG. 13A, the display unit 54 of the mobile terminal device 8 displays the captured image G70 as illustrated in FIG. In addition, the captured image G70 and the guide images G71 and G72 are displayed in a superimposed state. That is, the paper image GS that causes a jam is also shown in the captured image G70 of FIG. 13A. In the example of FIG. 13A, the front end of the paper has entered the fixing unit 17, and the rear end of the paper. Is engaged with the registration roller 11. Therefore, the release operation guide unit 43 refers to the release operation guide information 37, reads the guide information corresponding to such a paper position and paper state, and releases the operation guide for properly removing such paper from the conveyance path. The images G71 and G72 are generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 13B is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 13B, the guide image G71 indicating the direction in which the sheet in a state of being bitten at both the front end and the rear end is appropriately removed from the captured image G70 captured by the imaging unit 53. The text guidance image G72 prompting the user to perform such a removal operation is displayed in a superimposed manner. Therefore, the user can appropriately remove the paper that has caused the jam from the conveyance path by using such a display screen.

  After guiding the user to remove the paper as described above, if the image analysis unit 42 analyzes the captured image acquired from the mobile terminal device 8 and no paper image is detected, it causes a jam. It is possible to grasp that the removed paper has been removed. Therefore, after the release operation guide unit 43 guides the user to the release operation for removing the paper, as shown in FIG. 14, when the photographed image G80 acquired from the mobile terminal device 8 no longer includes the paper image, Recovery operation guide images G <b> 81 and G <b> 82 for guiding a recovery operation after the jam release such as an operation of closing the opening / closing door 9 are generated and transmitted to the mobile terminal device 8. As a result, an image as shown in FIG. 14 is displayed on the display unit 54 of the mobile terminal device 8. That is, in the display state shown in FIG. 14, a guidance image G81 indicating the closing direction of the opening / closing door 9 and a guidance image of a text prompting the user to close the opening / closing door 9 with respect to the captured image G80 captured by the imaging unit 53. G82 is superimposed and displayed. Therefore, the user can appropriately perform the recovery operation using such a display screen.

  Next, FIG. 15 is a diagram illustrating an outline of a processing sequence performed in cooperation between the apparatus main body of the image forming apparatus 1 and the mobile terminal apparatus 8 when a jam occurs. As shown in FIG. 15, when the image forming apparatus 1 detects the occurrence of a jam (process P1), the image forming apparatus 1 predicts the location where the jam has occurred, and generates a shooting guide D1 for allowing the user to take a picture of the predicted location. It transmits to the terminal device 8 (process P2). When the portable terminal device 8 obtains the photographing guide D1 from the image forming device 1, the portable terminal device 8 starts the program 57 (process P3) and starts photographing by the imaging unit 53 (process P4). The portable terminal device 8 transmits the captured image D2 obtained by the imaging unit 53 to the image forming apparatus 1 (process P5) and displays it on the display unit 54 (process P6). Thereby, the image forming apparatus 1 performs image analysis (process P7), generates a guide image D3 based on the image analysis result, and transmits it to the mobile terminal device 8 (process P8). The portable terminal device 8 guides the user to perform an operation for releasing the jam by superimposing and displaying the guide image D3 received from the image forming apparatus 1 on the captured image D2 captured by the imaging unit 53 (process P9). ). Thereafter, the processes P5 to P9 are repeatedly executed until the recovery operation by the user is completed. Therefore, each time the captured image D2 obtained by capturing with the mobile terminal device 8 changes, a guide image D3 suitable for the changed captured image D2 is displayed in a superimposed manner. When the jam is released by the user's operation and the restoration operation is completed, the image forming apparatus 1 transmits an end instruction D4 to the mobile terminal device 8 (process P10). Based on this end instruction D4, the mobile terminal device 8 ends the activation state of the program 57 and returns to the original state (process P11).

  As described above, the image forming apparatus 1 and the mobile terminal device 8 perform processing in cooperation with each other, so that when a jam occurs, the user can be guided to an appropriate release operation according to the jam occurrence state. Even in the process of user operation, it is possible to guide the appropriate operation each time.

  Next, a processing procedure performed in the image forming apparatus 1 when a jam occurs in the present embodiment will be described in detail. 16 to 19 are flowcharts illustrating an example of a processing procedure performed in the image forming apparatus 1 when a jam occurs. First, as shown in FIG. 16, when detecting the occurrence of a jam, the image forming apparatus 1 predicts the occurrence location (step S10), and determines an imaging target region by the mobile terminal device 8 (step S11). For example, in the present embodiment, the side surface on which the opening / closing door 9 of the apparatus main body is provided is determined as the imaging target region. Then, the image forming apparatus 1 transmits a shooting guide to the mobile terminal device 8 (step S12).

  When the image forming apparatus 1 transmits a shooting guide and then receives a shot image from the mobile terminal device 8 (YES in step S13), the image forming apparatus 1 performs an image analysis process (step S14), and subsequently performs a release operation guidance process (step S14). Step S15). Through these steps S14 and S15, operation guidance corresponding to the photographed image is given to the user. Then, the image forming apparatus 1 determines whether or not the jam has been eliminated (step S16). If the jam has not been eliminated (NO in step S16), the process returns to step S13. On the other hand, if the jam is resolved by the user's operation (YES in step S16), it is determined whether or not the recovery operation by the user is finished (step S17). If the restoration operation has not yet been performed (NO in step S17), the image forming apparatus 1 performs a restoration operation guidance process for causing the user to perform the restoration operation (step S18). As shown in FIG. 14, this process is a process for generating a guide image for causing the user to close the open / close door 9 and transmitting it to the mobile terminal device 8. When the recovery operation by the user ends (YES in step S17), an end instruction is transmitted to the mobile terminal device 8, and all the processes are ended (step S19).

  FIG. 17 is a flowchart illustrating an example of a detailed processing procedure of the image analysis process (step S14). When starting this process, the image forming apparatus 1 performs a process of extracting a reference point image from the captured image received from the portable terminal device 8 (step S20). Then, it is determined whether or not all the three reference point images have been extracted (step S21). If the three reference point images have been normally extracted, the shooting position by the mobile terminal device 8 is specified (step S22). .

  Next, the image forming apparatus 1 performs a process of extracting a paper image from the captured image received from the mobile terminal device 8 (step S23). It is determined whether or not the paper image has been extracted (step S24). If the paper image has been extracted, the position of the paper is specified (step S25). Then, the image forming apparatus 1 corrects the paper image based on the photographing position of the mobile terminal device 8 (step S26), and specifies the paper state based on the corrected paper image (step S27).

  On the other hand, when the three reference point images cannot be normally extracted in step S21, the subsequent processing (steps S22 to S27) is skipped. Even when the paper image cannot be extracted in step S24, the subsequent processing (steps S25 to S27) is skipped. With the above processing, the image analysis processing (step S14) ends.

  FIG. 18 is a flowchart illustrating an example of a detailed processing procedure of the release operation guidance process (step S15). When starting this process, the image forming apparatus 1 reads the release operation guide information 37 (step S30). Then, it is determined whether or not the shooting position of the mobile terminal device 8 is specified by the image forming process (step S14) (step S31). If the shooting position is not specified (NO in step S31), the image forming apparatus 1 is determined. Generates a photographing guide image so as to photograph the three reference markers M1, M2, and M3 (step S32), and transmits the guidance image to the mobile terminal device 8 (step S39).

  On the other hand, if the shooting position is specified (YES in step S31), the image forming apparatus 1 next determines whether or not a paper image has been extracted by the image analysis process (step S14) (step S33). As a result, if a paper image has been extracted (YES in step S33), the image forming apparatus 1 generates an image for canceling operation for appropriately removing the paper based on the paper position and the paper state. Is performed (step S37). On the other hand, when the sheet image has not been extracted (NO in step S33), the image forming apparatus 1 reads the output of the opening sensor 22, and determines whether the open / close door 9 has already been opened as the apparatus state. (Step S34). When the open / close door 9 is open (YES in step S34), the image forming apparatus 1 generates a shooting guidance image that guides the user to take a picture while changing the shooting position or the shooting direction so that the sheet is captured (step S35). ). When the open / close door 9 is closed (NO in step S34), a shooting guidance image for guiding the user to open the open / close door 9 for shooting is generated (step S36). Then, the image forming apparatus 1 deforms the guide image generated in any of steps S37, S35, and S36 according to the shooting position of the mobile terminal device 8, thereby changing the operation direction corresponding to the viewpoint of the mobile terminal device 8. (Step S38). Then, the image forming apparatus 1 guides an operation to be performed by the user by transmitting the deformed guide image to the mobile terminal device 8 (step S39).

  FIG. 19 is a flowchart illustrating an example of a detailed processing procedure of image generation processing (step S37) for generating a release operation guide image based on the paper position and the paper state. When starting this process, the image forming apparatus 1 determines whether or not the leading end of the sheet has entered the fixing unit 17 (step S40). If the leading edge of the sheet has entered the fixing unit 17 (YES in step S40), the image forming apparatus 1 further determines whether the trailing edge of the sheet has passed through the registration roller 11 (step S41), and the trailing edge of the sheet. If it has passed through the registration roller 11 (YES in step S41), based on the release operation guide information 37, a release operation guide image for pulling the rear end of the sheet downward is generated (step S42). On the other hand, when the trailing edge of the sheet does not pass through the registration roller 11 (NO in step S41), the image forming apparatus 1 guides the releasing operation to pull out both side edges of the sheet to the front based on the releasing operation guide information 37. An image is generated (step S43).

  On the other hand, if the leading end of the sheet has not entered the fixing unit 17 (NO in step S40), the image forming apparatus 1 further determines whether the trailing end of the sheet has passed through the registration roller 11 (step S44). If the trailing edge has not passed through the registration roller 11 (NO in step S44), a release operation guide image is generated based on the release operation guide information 37 (step S48). If the trailing edge of the sheet has passed through the registration roller 11 (YES in step S44), the image forming apparatus 1 further determines whether or not the paper is wound around the roller (step S45). Based on the release operation guide information 37, a release operation guide image for removing the paper by rotating the roller is generated based on the release operation guide information 37 (step S46). On the other hand, if it is not wound around the roller (NO in step S45), the image forming apparatus 1 generates a release operation guide image for removing the sheet from the conveyance path as it is based on the release operation guide information 37 (step S47). .

  As described above, in the image forming apparatus 1 according to the present embodiment, when a jam occurs in the transport path for transporting the paper, the jam is based on the position of the paper detected by the detection sensors 20 and 21 provided in the transport path. Is predicted, and shooting guidance for shooting the predicted portion is given to the mobile terminal device 8. Thereby, in the portable terminal device 8, the photographing operation by the imaging unit 53 is started. The image forming apparatus 1 determines the jam state by acquiring the photographed image and performing image analysis, and generates a guide image for guiding the jam release operation based on the determination result. The image forming apparatus 1 transmits the guidance image to the mobile terminal device 8 and displays the guidance image superimposed on the captured image displayed on the mobile terminal device, thereby guiding the user to the release operation according to the jam state. It is.

  Therefore, the image forming apparatus 1 of the present embodiment can guide an appropriate release operation to the user when a jam occurs without providing a plurality of cameras in the conveyance path. Therefore, the peripheral configuration of the conveyance path is not increased in size and complexity, and the problem that the cost of parts increases is also solved.

  Further, the image forming apparatus 1 having the above-described configuration can guide the release operation suitable for the jammed state only by photographing the situation where the jam is occurring using the mobile terminal device 8. In particular, since the mode of guiding the release operation is a mode of displaying the guidance image superimposed on the captured image displayed on the mobile terminal device 8, the user is viewing the display screen of the mobile terminal device 8. It is convenient that you can know the method of jam release operation. Further, the image forming apparatus 1 is configured to analyze a captured image captured by the mobile terminal device 8 and guide the user to an appropriate release operation according to the jam state. Therefore, it is not necessary for the user to determine the jam condition by himself or to find an appropriate release operation himself from the instruction manual. There is an advantage that the work can be started.

  Further, the image forming apparatus 1 of the present embodiment specifies the shooting position of the mobile terminal device 8 based on the position of the reference point image included in the captured image captured by the mobile terminal device 8, and guides the user to the release operation. When generating a guide image for this purpose, the guide image is deformed according to the specified shooting position. Therefore, the guide image displayed on the mobile terminal device 8 so as to be superimposed on the captured image is displayed as an image corresponding to the shooting viewpoint of the mobile terminal device 8. Therefore, if the user looks at the display screen of the mobile terminal device 8, the user can grasp an appropriate operation according to the jam state.

(Second Embodiment)
Next, a second embodiment will be described. The first embodiment described above exemplifies a mode in which, when a jam occurs in the image forming apparatus 1, an image analysis is performed on a captured image obtained by photographing with the mobile terminal device 8 on the apparatus main body side of the image forming apparatus 1. did. On the other hand, in this embodiment, after starting image photography in the portable terminal device 8, the image analysis is performed in the portable terminal device 8, and the form which guides the cancellation | release operation of jam to a user is demonstrated.

  FIG. 20 is a block diagram illustrating an example of a control mechanism of the image forming apparatus 1 and the mobile terminal device 8 according to the second embodiment. In the present embodiment, when the image forming apparatus 1 detects the occurrence of a jam, the controller 30 of the image forming apparatus 1 functions as the shooting guide unit 41 as in the first embodiment, and the mobile terminal device 8 is Send shooting guide.

  On the other hand, the controller 50 of the portable terminal device 8 functions as the imaging control unit 51, the image analysis unit 42, and the release operation guidance unit 43 by starting the program 57 described above when receiving the imaging guidance from the image forming apparatus 1. . Specific operations of the image analysis unit 42 and the release operation guide unit 43 that function in the controller 50 of the portable terminal device 8 are the same as those that function on the apparatus main body side of the image forming apparatus 1 in the first embodiment. . Therefore, in the present embodiment, when the mobile terminal device 8 drives the imaging unit 53 to start a shooting operation, image analysis is performed in the mobile terminal device 8 to determine the jam state. Then, a guide image for guiding the release operation suitable for the jam state is generated in the mobile terminal device 8, and the guide image is displayed in a state of being superimposed on the photographed image displayed on the display unit 54. .

  However, in the present embodiment, in order to perform image analysis in the mobile terminal device 8, the reference point information 36 and a plurality of types of jam pattern images described above are stored in advance in the memory 56 of the mobile terminal device 8. In the present embodiment, the above-described release operation guidance information 37 is also stored in advance in the memory 56 of the mobile terminal device 8 in order to guide the release operation suitable for the jam state in the mobile terminal device 8. Other configurations are the same as those described in the first embodiment.

  FIG. 21 is a diagram illustrating an outline of a processing sequence performed in cooperation between the apparatus main body of the image forming apparatus 1 and the mobile terminal apparatus 8 when a jam occurs in the second embodiment. As shown in FIG. 21, when the occurrence of a jam is detected (process P20), the image forming apparatus 1 predicts a location where a jam has occurred and generates a shooting guide D1 for allowing the user to take a picture of the predicted location. It transmits to the terminal device 8 (process P21). When the portable terminal device 8 obtains the photographing guide D1 from the image forming apparatus 1, the portable terminal device 8 starts the program 57 (process P22) and starts photographing by the imaging unit 53 (process P23). And the portable terminal device 8 will display the picked-up image on the display part 54 (process P25), if the picked-up image acquired by the imaging part 53 is acquired (process P24). Further, when the mobile terminal device 8 acquires the captured image, the mobile terminal device 8 performs image analysis of the captured image (process P26), and generates a guide image for guiding the user to cancel the jam based on the image analysis result (step P26). Process P27). Then, the portable terminal device 8 displays the guidance image on the display unit 54 so as to be superimposed on the captured image captured by the imaging unit 53, thereby guiding the user to perform an operation for releasing the jam (process P28).

  Thereafter, the processes P24 to P28 are repeatedly executed until the recovery operation by the user is completed. For this reason, whenever a captured image obtained by capturing with the mobile terminal device 8 changes, a guide image suitable for the changed captured image is displayed in a superimposed manner. When the jam is released by the user's operation and the restoration work is completed, the image forming apparatus 1 detects this and transmits an end instruction D4 to the portable terminal device 8 (process P29). Based on this end instruction D4, the mobile terminal device 8 ends the activation state of the program 57 and returns to the original state (process P30).

  As described above, in the present embodiment, since all the processing for the captured image taken by the mobile terminal device 8 is performed inside the mobile terminal device 8, the image forming apparatus 1 and the mobile terminal device 8 mutually transmit and receive image data. There is no need to do it. Therefore, compared with the first embodiment, there is an advantage that the processing efficiency is further improved. Other operational effects are the same as those described in the first embodiment.

(Modification)
As mentioned above, although preferred embodiment regarding this invention was described, this invention is not restricted to the thing of the content mentioned above, A various modification is applicable. Hereinafter, some modified examples will be described.

  For example, in the above description, an embodiment in which the operation panel 7 of the image forming apparatus 1 is removable from the apparatus main body and the portable terminal device 8 is configured by removing the operation panel 7 from the image forming apparatus 1 has been illustrated. However, the mobile terminal device 8 is not necessarily limited to being configured by being removed from the main body of the image forming apparatus 1. That is, the mobile terminal device 8 may be configured by a general tablet terminal device or a smartphone. In this case, the above-described program 57 may be installed in advance in the mobile terminal device 8 such as a tablet terminal device or a smartphone as a dedicated application for performing processing in cooperation with the image forming apparatus 1.

  Further, in the above-described embodiment, the form in which the image forming apparatus 1 guides the release operation when a jam occurs during print output is illustrated. However, in the image forming apparatus 1, a jam may also occur when the document transport unit 3 transports the document set by the user one by one to the document reading position of the scanner unit 2. Therefore, even when a jam occurs in the conveyance path where the document conveyance unit 3 conveys the document, an appropriate release operation may be guided to the user by the above-described method.

  Moreover, in the said embodiment, if the reference | standard point image of reference | standard marker M1, M2, M3 is contained in the picked-up image image | photographed using the portable terminal device 8, it will carry on the basis of the position where these reference point images are included. The case where the shooting position of the terminal device 8 is specified and the shooting direction of the mobile terminal device 8 is specified based on the shooting position is illustrated. However, the present invention is not limited to this. For example, when the mobile terminal device 8 is provided with three-axis acceleration sensors that are orthogonal to each other, the orientation of the mobile terminal device 8 can be detected by the acceleration sensors. You may comprise so that an imaging | photography direction may be specified based on the attitude | position of the portable terminal device 8 detected in that way.

  In the above-described embodiment, the case where the image forming apparatus 1 is an apparatus having a plurality of functions such as a print function, a scan function, and a copy function is illustrated. However, it is not limited to this. That is, the present invention can be applied to one having a conveyance path for automatically conveying paper, and the image forming apparatus 1 may be, for example, a printer having only a print function.

DESCRIPTION OF SYMBOLS 1 Image forming apparatus 8 Portable terminal device 20,21 Detection sensor 22 Opening sensor 33,55 Communication part (communication means)
41 Shooting guide (shooting guide means)
42 Image analysis unit (image analysis means)
43 Release operation guide section (release operation guide means)
51 Shooting control unit (shooting control means)
52 Display Control Unit (Display Control Unit)
53 Imaging unit (imaging means)
54 Display section (display means)
57 programs

Claims (13)

An image forming apparatus having a conveyance path for conveying a sheet and detecting occurrence of a jam in the conveyance path,
A detection sensor for detecting paper in the transport path;
Communication means for communicating with a portable terminal device having an imaging function;
When a jam occurs in the transport path, a location where a jam occurs in the transport path is predicted based on the position of the paper detected by the detection sensor, and shooting guidance for capturing the predicted location is provided to the mobile terminal device. Shooting guide means to perform;
Image analysis means for determining a jam state by acquiring a captured image captured by the mobile terminal device based on the shooting guide and performing image analysis;
Based on the determination result in the image analysis means, a guide image for guiding a jam release operation is generated and displayed superimposed on a photographed image displayed on the mobile terminal device, thereby canceling according to the jam state. Release operation guidance means for guiding the operation;
An image forming apparatus comprising: An openable / closable door that can be manually operated by a user and opens the conveyance path;
An opening sensor for detecting that the opening / closing door opens the conveyance path;
Further comprising
The release operation guide means is an open / close door operation guide image for guiding the operation of the open / close door when the opening sensor does not detect the opening of the transport path after the shooting guide means performs the shooting guide. Is generated and displayed on the mobile terminal device so as to be superimposed on the captured image, thereby opening the transport path and guiding the opening operation to make the predicted location ready to be captured. The image forming apparatus according to claim 1.   The release operation guide means generates a shooting guide image for changing a shooting position or a shooting direction by the mobile terminal device when the image analysis means does not detect a paper that causes a jam after guiding the opening operation. The image forming apparatus according to claim 2, wherein a shooting position or a shooting direction of the mobile terminal device is guided by being superimposed on a captured image displayed on the mobile terminal device. The image analysis means identifies the position and state of the paper that is the cause of the jam by analyzing the captured image,
The release operation guide means determines a paper extraction operation according to the position and state of the paper specified by the image analysis means, and generates a jam release operation guide image for performing the extraction operation. The image forming apparatus according to claim 1.   5. The image forming apparatus according to claim 4, wherein the extraction operation includes an operation of a movable portion provided in the conveyance path and operable by a user.   The image analysis means specifies a shooting position of the mobile terminal device based on a position of a reference point image included in a shot image shot by the mobile terminal device, and is included in the shot image shot by the mobile terminal device 6. The image forming apparatus according to claim 4, wherein the sheet image is specified by correcting a sheet image to be corrected based on a shooting position of the portable terminal device and matching with a predetermined jam pattern image. . The image analysis means specifies a shooting position of the mobile terminal device based on a position of a reference point image included in a shot image shot by the mobile terminal device,
The image forming apparatus according to claim 1, wherein the release operation guide unit deforms the guide image according to a shooting position of the mobile terminal device when generating the guide image.   The release operation guide means generates a recovery operation guide image for guiding the recovery operation after the jam release when the image analysis means no longer detects paper after guiding the release operation, and the portable terminal 8. The image forming apparatus according to claim 1, wherein the image forming apparatus guides a recovery operation after the jam release by displaying the captured image displayed on the apparatus in a superimposed manner. 9.   The image forming apparatus according to claim 1, wherein the portable terminal device is an operation panel that can be attached to and detached from the image forming apparatus. A portable terminal device capable of communicating with an image forming apparatus,
Imaging means for taking an image;
Display means for displaying a photographed image photographed by the imaging means;
The image pickup device is driven to start shooting based on a shooting guide received from the image forming device when a jam occurs in a conveyance path for carrying paper through the image forming device. Photographing control means for transmitting a photographed image obtained by the method to the image forming apparatus;
Display control means for displaying the release operation guide image superimposed on the photographed image displayed on the display means in response to receiving the release operation guide image for guiding the jam release operation from the image forming apparatus; ,
A portable terminal device comprising: A portable terminal device capable of communicating with an image forming apparatus,
Imaging means for taking an image;
Display means for displaying a photographed image photographed by the imaging means;
A shooting control unit configured to drive the imaging unit to start shooting based on a shooting guide received from the image forming device when a jam occurs in a conveyance path for carrying a sheet in the image forming device; ,
Image analysis means for detecting a paper that is a jam factor in the image forming apparatus and determining a jam state by acquiring a photographed image photographed by the imaging means and performing image analysis;
Based on the jam state determined by the image analysis means, a release operation guide image for guiding the release operation of the jam is generated and displayed on the captured image displayed on the display means. Release operation guidance means for guiding the release operation according to
A portable terminal device comprising: A program that is executed in a portable terminal device that includes an imaging unit that captures an image, and a display unit that displays a captured image captured by the imaging unit, and is capable of communicating with the image forming apparatus, the portable terminal device ,
The image pickup device is driven to start shooting based on a shooting guide received from the image forming device when a jam occurs in a conveyance path for carrying paper through the image forming device. Photographing control means for transmitting a photographed image obtained by the method to the image forming apparatus, and
Display control means for displaying the release operation guide image superimposed on the photographed image displayed on the display means in response to receiving the release operation guide image for guiding the jam release operation from the image forming apparatus,
A program characterized by functioning as A program that is executed in a portable terminal device that includes an imaging unit that captures an image, and a display unit that displays a captured image captured by the imaging unit, and is capable of communicating with the image forming apparatus, the portable terminal device ,
A shooting control unit that starts shooting by driving the imaging unit based on a shooting guide received from the image forming device when a jam occurs in a conveyance path for carrying a sheet through the image forming device;
Image analysis means for detecting a paper that is a jam factor in the image forming apparatus and determining a jam state by acquiring a captured image taken by the imaging means and performing image analysis; and
Based on the jam state determined by the image analysis means, a release operation guide image for guiding the release operation of the jam is generated and displayed on the captured image displayed on the display means. Release operation guide means for guiding the release operation according to
A program characterized by functioning as

Images