JP6252735B2 - Image processing program - Google Patents

Description

  The present invention relates to an image processing program, and more particularly, to an image processing program in which an image captured by an information terminal is combined with an image such as a plurality of contents stored in a storage unit of the information terminal and displayed.

  Conventionally, a target device such as a multifunction device, a server device that provides help information corresponding to the target device, and a mobile communication terminal including an imaging unit such as an imaging camera are connected to a network, and the target device is a device. The ID and status code can be transmitted to the server device, and the mobile communication terminal captures the two-dimensional barcode attached to the front surface of the target device and transmits the barcode information to the server device. There is known a technique of obtaining unique help information corresponding to the state of the image, synthesizing the help information and captured image data, and displaying AR (Augmented Reality).

JP 2010-219879 A

  However, in the above-described conventional image processing program, the server device can display only the status code of the target device. For example, detailed description content corresponding to the portion of the target device is prepared. There wasn't. For this reason, for example, various information such as functions, operation methods, warnings, and the like related to a plurality of parts of one target device cannot be easily known for each part, and there is a problem that the convenience is inferior.

An object of the present invention, if they have photo-Zuto illustrating an appearance of a real or label producing device of the label producing apparatus as a printing apparatus, functions related to multiple sites of one label producing apparatus, method of operation, a warning such as An object of the present invention is to provide an image processing program that can easily know various information for each part and improve convenience.

In order to achieve the above object, the present invention captures at least a part of an appearance or an appearance image of a label producing apparatus as a printing apparatus having at least one characteristic shape portion and a plurality of content target portions, which are determined in advance. The feature shape portion is added to the captured image data generated by the imaging means for the computing means of the information terminal having imaging means for generating corresponding captured image data and display means for performing desired display. A feature determination procedure for determining whether or not the feature shape portion is included, and a plurality of contents related to each of the plurality of content target portions triggered by determination that the feature shape portion is included in the captured image data in the feature determination procedure The content or content-related information is stored in association with each content target part and stored in the plurality of contents or contents. Among the related information, the content acquisition procedure for acquiring at least the corresponding content target part included in the captured image data, and the content for the content target part of the captured image data generated by the imaging unit An image generation procedure for generating post-addition image data by adding the content or content-related information corresponding to the content target part acquired by the acquisition means, and the post-addition image data generated by the image generation procedure A display control procedure to be displayed on the display means , wherein the image generation procedure is performed on the content target part as the content related information with respect to the content target part of the captured image data generated by the imaging means. A tap button that includes the name and can be tapped by the operator. In addition to generating the added post-addition image data, the positional relationship between the content or content-related information and the content target portion corresponding to the content or the content-related information regardless of temporal changes in the captured image data generated by the imaging unit However, the post-addition image data is generated so as to be kept substantially constant, and the content target portion is a print button that is provided in the label producing apparatus and for an operator to instruct the production of a printed matter, In the content acquisition procedure, when the post-addition image data is displayed on the display unit and the tap button corresponding to the print button is tapped by an operator, the moving image data stored in the storage unit The display control means is capable of interrupting a moving picture from which a label is drawn based on the moving picture data. That displays on.

The information terminal on which the image processing program of the present invention is executed includes an imaging unit, a display unit, and a storage unit. Then, by imaging the label producing apparatus as a printing apparatus by the image pickup means, content-related information (hereinafter appropriately associated with predetermined content or the content on the captured image of the label producing apparatus, these are collectively " A new image to which “content etc.” is added is generated and displayed on the display means.

That is, the appearance or appearance image of the label producing apparatus is provided with a plurality of predetermined content target parts, each of which is associated with at least one characteristic shape portion and the content. In addition, the plurality of contents and the like are stored in a manner uniquely associated with each of the plurality of content target parts.

When the image processing program of the present invention is executed by the arithmetic means, first, in the characteristic determination procedure, whether or not the characteristic shape portion is included in the captured image data generated by the imaging of the imaging means. Is determined. When the characteristic shape part is included and the above determination is satisfied, the content acquisition procedure is executed in response to this. That is, in the present invention, the subsequent processes are sequentially performed by using the image of the characteristic shape portion of the label producing apparatus as the printing apparatus as a trigger.

  In the content acquisition procedure, among the plurality of contents stored in the storage unit, the corresponding content target part is included in the captured image data (in other words, the content target part exists within the imaging visual field range of the imaging unit) At least). Thereafter, in the image generation procedure, new post-addition image data is generated by adding the acquired content or the like to the corresponding content target portion in the captured image data. Thereby, new image data in which content corresponding to the content target portion within the imaging visual field range is added is generated and displayed on the display means in the subsequent display control procedure.

As described above, in the present invention, the content of the content corresponding to each part of the content target part included in the picked-up image can be obtained only by capturing the appearance or appearance image of the label producing apparatus as the printing apparatus. Etc. can be displayed together. Thus, the operator, if they have photo-Zuto illustrating an appearance of a real or label manufacturing apparatus of the label manufacturing apparatus, a plurality of sites (especially without reading the details of the instruction manual) one label producing device Various functions such as functions, operation methods, and warnings can be easily known for each part, and convenience can be improved.

According to the present invention, if they have photo-Zuto illustrating an appearance of a real or label producing device of the label producing apparatus as a printing apparatus, functions related to multiple sites of one label producing apparatus, method of operation, a warning such as Various information can be easily known for each part, and convenience can be improved.

It is a front view of an example of a printing apparatus (actual machine) applied to the image processing program of the present invention. It is a front view of an example of the simple instruction manual corresponding to the printing apparatus shown in FIG. In the image processing program of this invention, the example of a display of a touchscreen part at the time of imaging the printing apparatus (real machine) in an information terminal is shown, and it is explanatory drawing which changes from an actual machine display state to AR display state. In the image processing program of this invention, the example of a display of a touchscreen part at the time of imaging the printing apparatus (real machine) in an information terminal is shown, and it is explanatory drawing of AR expansion display state. It is explanatory drawing which shows the example of a display of a touchscreen part at the time of imaging the printing apparatus (power button operation part of an instruction manual) in an information terminal in the image processing program of this invention. It is explanatory drawing of the AR display state at the time of imaging the printing apparatus (power button operation part of an instruction manual) in the information terminal in the image processing program of the present invention. It is explanatory drawing of the other AR display state at the time of imaging the printing apparatus (power button operation part of an instruction manual) in the information terminal in the image processing program of the present invention. It is explanatory drawing which shows the example of a display of a touchscreen part at the time of imaging the printing apparatus (print button operation part of an instruction manual) in an information terminal in the image processing program of this invention. It is explanatory drawing of the AR display state at the time of imaging the printing apparatus (print button operation part of an instruction manual) in the information terminal in the image processing program of the present invention. It is explanatory drawing which shows the example of a moving image display at the time of imaging the printing device (print button operation part of an instruction manual) in an information terminal, and operating a print button in a touch panel part in the image processing program of the present invention. It is explanatory drawing which shows the example of a display of a touchscreen part at the time of imaging the printing apparatus (label cut button operation part of an instruction manual) in an information terminal in the image processing program of this invention. It is explanatory drawing which shows AR display state at the time of imaging the printer (the tape cut button operation part of an instruction manual) in an information terminal in the image processing program of this invention. It is a flowchart of the main routine in the image processing program of this invention. It is a flowchart of the language switching routine in the image processing program of this invention. It is a flowchart of a moving image reproduction routine in the image processing program of the present invention. It is a flowchart of the audio | voice reproduction routine in the image processing program of this invention.

First, referring to FIG. 1, a label producing apparatus as a printing apparatus which is an imaging target of the present invention will be described. A label production apparatus (hereinafter also simply referred to as “actual machine”) 10 includes a power button 11, a print button 12, an operation key unit 13 such as various numeric keys and alphabetic keys, and a display unit that displays input characters and the like when characters are input. 14 and a label cut lever 15 for cutting the label when label printing based on the inputted characters is executed. The label producing apparatus 10 includes, for example, a USB connection unit for connection with a personal computer (not shown), a power supply connection unit, and the like, and is always supplied with power from a replaceable battery.

  Further, for example, in the package or instruction manual at the time of sale, the label producing apparatus 10 enables the user to recognize information related to the operation or the like by the simplified instruction manual 20 as shown in FIG. This simple instruction manual 20 may be an instruction manual or a user manual regarding the minimum necessary operation during normal label printing. For example, power-on (upper left in FIG. 2) operation, character input operation (upper right in FIG. 2) ), A print execution operation (lower left in FIG. 2), a label cut operation (lower right in FIG. 2), and the like are printed. Here, in the instruction manual, for example, in order to recognize an intuitive operation, a picture or a photograph imitating all or part of the label manufacturing apparatus 10 and the operating hand (or finger) is printed. Such information may not be printed. Thereby, it is possible to contribute to simplification of information entered in the instruction manual and reduction of the number of pages.

  As shown in FIG. 3, for example, a smartphone is used for the information terminal 30. The information terminal 30 includes, for example, a speaker unit 31 and a microphone unit 32 for calling on the front side, and a touch panel unit 33 such as a touch panel type liquid crystal display. Note that a camera unit 34 as an imaging unit is provided on the back side of the information terminal 30. Further, in the information terminal 30, a communication unit (not shown) for calling or connecting to the Internet, a ROM (not shown) storing programs for executing various functions, a temporary information recording A known information processing unit such as a RAM (not shown) and a CPU (not shown) for executing a program stored in the ROM is provided.

In the ROM of the information terminal 30, the feature shape portion of the label producing device 10, the content or the content related information, and the moving image information related to the operation behavior of the label producing device 10 when the above-described operation target part is operated are stored for each operation target. It is stored in association with the part. The camera unit 34 of the information terminal 30 displays the captured image on the touch panel unit 33.

  Thereby, the label manufacturing apparatus 10 and the simple instruction manual 20 can capture the appearance of the label manufacturing apparatus 10 by imaging the label manufacturing apparatus 10 with the camera unit 34 of the information terminal 30, as shown in FIG. The various buttons 11, 12, 13, or the size, position, shape, and the like of the display unit 14 and the label cut lever 15 are identified as the characteristic shape unit, and after specifying the model, the various buttons 11, 12, 13, or display The description of the operation target portion of the unit 14 and the label cut lever 15 can be displayed on the touch panel unit 33 as AR. The AR display technology itself uses a known technology.

  Specifically, as shown in FIG. 4, when the image of the label producing apparatus 10 captured by the camera unit 34 of the information terminal 30 is displayed on the touch panel unit 33, the imaging target and the feature shape stored in the ROM When a part is compared with a CPU and an imaging target is included in a feature shape part stored in a ROM, as a label producing apparatus 10, a demo pattern (for example, displaying text directly on the display appearance of the touch panel part 33 (for example, The content 35 indicating “Please press the power button” or the like is displayed.

Further, when the image of the label producing apparatus 10 captured by the camera unit 34 of the information terminal 30 is displayed on the touch panel unit 33, the imaging target and the feature shape part stored in the ROM are compared by the CPU, and the imaging target Is included in the feature shape portion stored in the ROM, the label producing apparatus 10 has a “button name” enclosed in a rectangular frame or the like in addition to the leader drawn from the corresponding operation target portion on the display appearance. Etc. and content related information 36 waiting for tap is displayed.

  The tap can use the content 35, the content related information 36, the various buttons 11, 12, 13 displayed on the touch panel unit 33.

  At this time, the zoom function of the camera unit 34 or the display size of the content displayed on the touch panel unit 33 and the display size of the content related information 36 are changed by moving the information terminal 30 closer to or away from the label producing apparatus 10. can do.

  In addition, when the simplified instruction manual 20 is used, in addition to the print image of the label producing apparatus 10, a hand, a finger or the like indicating the operation state is recognized as the feature shape portion. Thereby, in the information terminal 30, as shown in FIG. 5, the state itself imaged with the camera part 34 is recognized as a feature shape part.

That is, in the ROM of the information terminal 30, the moving image related to the behavior of the label producing apparatus 10 when the characteristic shape part of the simple instruction manual 20, the content 35 or the content related information 36, and the operation target part described above are operated. Information is stored in association with each operation target part.

  Specifically, as shown in FIG. 5, when an image captured by the camera unit 34 of the information terminal 30 is displayed on the touch panel unit 33, the imaging target and the feature shape portion stored in the ROM are compared by the CPU. Then, when the imaging target is included in the feature shape portion stored in the ROM, as shown in FIG. 6, a picture or a photograph imitating the label producing apparatus 10 (hereinafter, also simply referred to as “appearance image”) A content 35 indicating a demo pattern for displaying a sentence (for example, “please press the power button”) directly on the display appearance of the touch panel unit 33 is displayed.

Further, when an image captured by the camera unit 34 of the information terminal 30 is displayed on the touch panel unit 33, the imaging target is compared with the feature shape portion stored in the ROM, and the imaging target is stored in the ROM. When it is included in the feature shape part, the appearance image is displayed in addition to the leader line drawn from the corresponding operation target part on the display appearance, and “button name” enclosed in a rectangular frame etc. The content related information 36 to be displayed is displayed.

Here, on the touch panel unit 33, for example, a Japanese display button 33a and an English display button 33b are displayed. As a result, the CPU displays the content 35, the content related information 36, etc. in Japanese at all times (or initially). Further, when the CPU performs a tap operation on the English display button 33b in the Japanese display state, the CPU switches to the display of the content 35 and the content related information 36 in English. In addition, when the label production apparatus 10 is an English-speaking apparatus (shape or nameplate), when the native language of the current position acquired using the GPS function of the information terminal 30 is English-speaking, the language of the information terminal 30 When the setting is English, the information terminal 30 performs initial setting or automatic switching to display English as switching auxiliary information.

  Further, as shown in FIG. 7, the content related information 36 includes text (for example, “please press the power button”) once in a portion corresponding to the display unit 14 in the appearance image displayed on the touch panel unit 33. The content 35 indicating a demo pattern by scrolling is displayed.

As shown in FIG. 8, when the content related information 36 displays a part corresponding to the print button 12 in the appearance image displayed on the touch panel unit 33, as shown in FIG. As shown in FIG. 10, printing is executed as shown in FIG. 10 when the user taps the “print button” as the content information 36, “press the print button” as the content related information 36, or the appearance image imitating the print button 12. Thus, the moving image information 37 simulating the label drawn out is displayed.

Furthermore, as shown in FIG. 11, the content related information 36 is displayed when a portion corresponding to the label cut lever 15 is displayed in the appearance image displayed on the touch panel unit 33, as shown in FIG. 12. 35 is displayed. In this content 35, for example, a speaker mark 35a is displayed. When the speaker mark 35 a is tapped, the same sound as the content 35 is output from the speaker unit 31.

  As described above, the appearance of the label producing apparatus 10 including the characteristic shape portion and the operation target portion (the power button 11, the print button 12, the operation key portion 13, the display portion 14, the label cut lever 15 and the like) determined in advance ( A camera unit 34 that captures at least a part of an appearance image such as an actual machine) or the simplified instruction manual 20 and generates corresponding captured image data; a touch panel unit 33 having a desired operation function and a desired display function; The information terminal 30 having a CPU includes a CPU that performs various processes based on a program.

  Hereinafter, a main processing routine by the CPU of the information terminal 30 will be described with reference to FIG.

  First, the CPU of the information terminal 30 activates the AR mode by a program stored in the ROM (step S1).

  Next, the CPU of the information terminal 30 executes the splash screen process that is in the process of starting the AR mode including the process for setting the touch panel unit 33 to full screen display (step S2).

  The CPU of the information terminal 30 acquires data related to the feature shape portion stored in the ROM, and temporarily stores it in, for example, the RAM (step S3). The reason why the data related to the feature shape portion is temporarily stored in the RAM is to secure the processing speed, and when it is not necessary, the data related to the feature shape portion stored in the ROM is directly used.

  The CPU of the information terminal 30 activates the camera unit 34 (step S4), generates captured image data from the data captured by the camera unit 34 (real time), and displays it on the touch panel unit 33 (step S5).

  The CPU of the information terminal 30 compares the captured image data with the data related to the feature shape portion temporarily stored in the RAM, and determines whether or not the feature shape portion is included in the captured image data (step S6).

  In the comparison determination in step S6, when the feature shape portion is not included in the captured image data, the captured image data is generated from the data captured in real time by the camera unit 34 and displayed on the touch panel unit 33. Meanwhile, it is continuously determined whether or not the feature shape portion is included in the captured image data.

  Here, when the feature shape portion is included in the captured image data, the CPU of the information terminal 30 acquires the content 35 or the content related information 36 of the operation target part included in the corresponding feature shape portion (step). 7).

Here, the CPU of the information terminal 30 displays on the touch panel unit 33 an AR based on an image based on the data captured by the camera unit 34 and an animated image created from the video file of the content 35 or the content related information 36. .

  In this AR display state, the CPU of the information terminal 30 enters a standby state. In addition, when the information terminal 30 is moved, the routine after step S5 described above is displayed in real time.

  Next, the language switching mode will be described with reference to FIG. This language switching mode is an interrupt mode, and can be interrupted when a Japanese display button 33a and an English display button 33b are displayed in the routine after step S9, for example, as shown in FIG. It has become.

  When the CPU of the information terminal 30 recognizes that the language has been switched, that is, that the English display button 33b has been tapped (step S11), the Japanese display of the content 35 or the content related information 36 in the current display state is switched to the English display ( Step S12) performs substantially the same image processing (3D display) as in Step S9 (Step S13), and enters a standby state.

  Next, the moving image playback mode will be described with reference to FIG. In this moving image generation mode, it is also an interrupt mode, and in the routine after step S9, for example, as shown in FIG. Interruption is possible when the print button 12 on the appearance image is tapped.

  When the CPU of the information terminal 30 is displaying the content 35 or the print button 12 that is a moving image target and is tapped as an operation target portion (step S21), the moving image data is acquired from the RAM (or ROM). Then (step S22), for example, as shown in FIG. 10, display of a moving image is started as if the label at the time of actual printing is drawn (step S23).

  The CPU of the information terminal 30 monitors, for example, whether or not an operation target portion such as the moving image stop operation, for example, the content 35 or the print button 12 as the moving image target, is tapped again during the moving image reproduction (step S24). When tapped, the reproduction of the moving image is stopped (step S25).

  The CPU of the information terminal 30 monitors, for example, whether or not a moving image stop operation, for example, a content 35 that is a moving image target or an operation target portion such as the print button 12 is continuously double-tapped during moving image reproduction (step S26). In the case of double tapping, the moving image information for full screen display of only the corresponding part is acquired from the RAM (step S27), and for example, full screen display (for example, two-dot chain line shown in FIG. 10) The reproduction of the moving image is continued by switching to the range surrounded by 33A (step S28).

  Then, the CPU of the information terminal 30 substantially waits for step S23 or step S28 until the moving image reproduction ends. When the moving image reproduction ends, the CPU of the information terminal 30 stands by for step S10. Further, in the full screen display in step S28, when there is a tap again during the reproduction of the moving image, it is possible to return to the original moving image display size.

Next, the audio output mode will be described with reference to FIG. This audio output mode is an interrupt mode, and can be interrupted when the speaker mark 35a is displayed in the routine after step S9, for example, as shown in FIG.

When the CPU of the information terminal 30 recognizes the voice output, that is, that the speaker mark 35a has been tapped (step S31), as shown in FIG. Corresponding sound is output from the speaker unit 31 (step S32), and a standby state is entered.

  By the way, in the above embodiment, for example, when an image corresponding to the print button 12 is displayed on the touch panel unit 33, a message such as “Please press the print button” is displayed as the content 35 and can be tapped. However, the message contents can display more detailed operation contents.

  For example, in the case of the USB connection unit described above, characters and images of a connectable device (for example, a printer or a personal computer other than label production such as a printer or a multi-function device) are input. Numerical values such as standards (for example, 100V to 240V) and output (18V) can be displayed.

  Furthermore, in the said embodiment, although the outside of the label production apparatus 10 was made into object, the inside of the state which removed the cover can be made into object. For example, various information such as battery / label cartridge replacement method information, product information such as the type and number of the battery or label cartridge, and danger information (high temperature part) for the thermal head can be displayed as the content 35.

  As described above, the CPU determines whether or not the feature shape portion is included in the captured image data generated by the camera unit 34 (step S6), and it is determined that the feature shape portion is included in the captured image data. In response to this, the content 35 or the content related information 36 related to the operation target part and the moving image information 37 related to the operation behavior of the label producing apparatus 10 when the operation target part is operated are stored in association with the operation target part. The content 35 or the content related information 36 in which the corresponding operation target part is included in the captured image data is acquired from the ROM (step S7), and the operation target part related to the content 35 or the content related information 36 on the touch panel unit 33 is displayed. The tap determination procedure (step by step) for determining whether or not the tap operation by the operator has been performed on the displayed area. And a moving image acquisition procedure for acquiring moving image information 37 corresponding to the tap operation stored in the ROM when it is determined that the tap operation has been performed on the display area in the tap determination procedure (step S22). And the 1st display control procedure (step S23) which controls the touch panel part 33 so that the moving image information 37 acquired by the moving image acquisition procedure may be displayed.

  As described above, the information terminal 30 on which the image processing program of the present embodiment is executed includes the camera unit 34, the touch panel unit 33, and the ROM. Then, by capturing an image of the label producing apparatus 10 or the simplified instruction manual 20 with the camera unit 34, the content related to the predetermined content 35 or the content 35 with respect to the captured image of the label producing apparatus 10 or the simplified instruction manual 20. A new image (AR) to which the information 36 is added is generated and displayed on the touch panel unit 33.

  That is, the appearance of the label manufacturing apparatus 10 or the appearance image of the simplified instruction manual 20 includes the appearance shape of the label manufacturing apparatus 10, various buttons 11, 12, 13 or the sizes and positions of the display unit 14 and the label cut lever 15. , And the like, and various operation buttons 11, 12, 13 associated with the contents 35, 36 and predetermined operation target parts such as the label cut lever 15 are provided.

  The ROM of the information terminal 30 stores contents 35, 36 and moving image information 37 related to the operation behavior of the label producing apparatus 10 when the operation target part is operated, for example, when the operation target part is operated. The information representing how the entire label producing apparatus 10 including the operation target part operates is stored in a manner uniquely associated with the operation target part.

  When the image processing program of the present embodiment is executed by the CPU of the information terminal 30, first, in the feature determination procedure in step S6, the feature shape portion is included in the captured image data generated by the imaging of the camera portion 34. Whether or not is included is determined. When the feature shape portion is included and the determination is satisfied, the content acquisition procedure of step S7 is executed in response to this.

  That is, in the present embodiment, the subsequent processes are sequentially performed with the image of the characteristic shape portion of the label producing apparatus 10 as a trigger.

  That is, in the content acquisition procedure of step S7, the corresponding operation target part is included in the captured image data among the plurality of contents 35 and 36 stored in the ROM of the information terminal 30, in other words, the imaging of the camera unit 34. At least those in which the operation target region exists within the visual field range are acquired.

  Then, when the operator taps the display area of the operation target part of the touch panel unit 33, it is determined that the tap operation has been performed in the tap determination procedure in step S21. Then, moving image information corresponding to the tap operation stored in the ROM of the information terminal 30 as described above is acquired in the moving image acquisition procedure in step S22, and the moving image information is acquired in the first display control procedure in step S23. Displayed on the touch panel unit 33.

  As described above, in the present embodiment, by capturing an appearance image of the label producing apparatus 10 or an appearance image of the simplified instruction manual 20, the content corresponding to the operation target portion included in the captured image is displayed. The contents 35 and 36 can be displayed together. Furthermore, by performing a tap operation on the operation target part, it is possible to display and simulate an operation behavior similar to that of the actual label producing apparatus 10 on the touch panel unit 33.

  As a result, the operator can prepare the label without reading the details of the instruction manual, as long as the actual label manufacturing apparatus 10 or a photograph / drawing showing the appearance of the label manufacturing apparatus 10 (simple instruction manual 20) is available. Various information such as the function, operation method, and warning of the operation target part of the apparatus 10 and the behavior of the operation when the operation target part is operated can be easily and easily known. As a result, the convenience for the operator can be improved.

  Further, content 35 or content related information 36 corresponding to the operation target part acquired by the CPU of the information terminal 30 is added to the operation target part of the captured image data generated by the camera unit 34, and the post-addition image data A second display control procedure (step S28) in step S28 for controlling the touch panel unit 33 to display the image generation procedure in step S8 for generating the image data and the post-addition image data generated in the image generation procedure in step S8; In the tap determination procedure of step S26, the tap operation by the operator is performed on the display area displaying the operation target portion related to the content 35 or the content related information 36 on the touch panel unit 33 displaying the post-addition image data. The first display control procedure in step S23 is the same as the image in step S8. Second display in place of the displayed additional image data at a control procedure is generated in the generation procedure step S28, it controls the touch panel unit 33 to display the video information obtained by the video acquisition procedure of step S22.

  In the content acquisition procedure of step S7, the corresponding operation target part is included in the captured image data among the plurality of contents 35, 36 stored in the ROM of the information terminal 30, in other words, the imaging field range of the camera unit 34. At least those in which the operation target part exists are acquired.

  Thereafter, in the image generation procedure in step S8, new post-addition image data is generated by adding the acquired contents 35, 36 to the corresponding operation target part in the captured image data. As a result, new image data in which the contents 35 and 36 corresponding to the operation target region within the imaging visual field range are added is generated and displayed on the touch panel unit 33 in the second display control procedure in step S28. The

  Then, when the operator taps the display area of the operation target part of the touch panel unit 33 displaying the operation target part with contents 35 and 36, it is determined that the tap operation has been performed in the tap determination procedure of step S21. Is done. Then, moving image information corresponding to the tap operation stored in the ROM of the information terminal 30 as described above is acquired in the moving image acquisition procedure in step S22, and the moving image information is acquired in the first display control procedure in step S23. Displayed on the touch panel unit 33.

Furthermore, the moving image acquisition procedure in step S22 is performed by the label producing apparatus 10 when the operation target part related to the tap operation is the print button 12 (an image corresponding to the print button 12 for the operator to instruct the production of the printed matter). Acquire moving image information including the behavior of printed matter production and printed matter discharge.

Accordingly, the operator can simulate the printed matter production behavior and the printed matter discharge behavior on the touch panel unit 33 without actually operating the label producing apparatus 10. As a result, the convenience for the operator can be improved with certainty.

  In addition, for example, it can be used as a substitute for demonstration equipment, such as checking the printing speed by performing a printing simulation using a photograph printed on the package at the store. It becomes.

  In addition, the CPU of the information terminal 30 according to the present embodiment includes a feature determination procedure in step S6 for determining whether or not the captured image data generated by the camera unit 34 includes a feature shape portion, and a feature in step S6. When it is determined in the determination procedure that the captured image data includes a feature shape portion, a plurality of contents 35 or content related information 36 related to a plurality of content target parts are stored in association with each content target part. The content acquisition procedure of step S7 for acquiring at least the content 35 corresponding to the content target portion included in the captured image data among the plurality of content 35 or content related information 36 from the ROM of the information terminal 30, and the camera unit 34 By the CPU of the information terminal 30 for the content target part of the captured image data The image generation procedure in step S8 for generating post-addition image data by adding the content 35 or content-related information 36 corresponding to the acquired content target part, and the post-addition image data generated in the image generation procedure in step S8 Is displayed on the touch panel unit 33, and the display control procedure of step S9 is executed.

  The information terminal 30 on which the image processing program of the present embodiment is executed includes a camera unit 34, a touch panel unit 33, and a ROM. Then, by capturing the appearance of the label producing apparatus 10 or the appearance image of the simple instruction manual 20 with the camera unit 34, the content related information relating to the predetermined content 35 or the content 35 with respect to the captured image of the label producing apparatus 10. A new image to which 36 is added is generated and displayed on the touch panel unit 33.

  That is, the appearance of the label manufacturing apparatus 10 or the appearance image of the simple instruction manual 20 includes at least one characteristic shape portion and a plurality of predetermined content target portions to which the contents 35 and 36 are respectively associated. ing. In addition, the plurality of contents 35 and 36 are stored in the ROM of the information terminal 30 in a manner uniquely associated with each of the plurality of content target parts.

  When the image processing program of the present embodiment is executed by the CPU of the information terminal 30, first, in the feature determination procedure in step S6, the feature shape portion is included in the captured image data generated by the imaging of the camera portion 34. Whether or not is included is determined.

  When the determination that the feature shape portion is included is satisfied, the content acquisition procedure of step S7 is executed with this as a trigger. That is, in the present embodiment, the subsequent processes are sequentially performed with the image of the characteristic shape portion of the label producing apparatus 10 as a trigger.

  In the content acquisition procedure of step S7, the corresponding content target part is included in the captured image data among the plurality of contents 35 and 36 stored in the ROM of the information terminal 30, in other words, the imaging field range of the camera unit 34. At least the content target part is acquired.

  Thereafter, in the image generation procedure of step S8, new post-addition image data is generated by adding the acquired contents 35, 36 to the corresponding content target part in the captured image data.

  As a result, new image data in which the contents 35 and 36 corresponding thereto are added to the content target portion within the imaging visual field range is generated and displayed on the touch panel unit 33 in the subsequent display control procedure of step S9.

  As described above, in the present embodiment, only the appearance of the label production apparatus 10 or the appearance image of the simplified instruction manual 20 is captured, and each content target part included in the captured image corresponds to the part. The contents 35 and 36 of the contents to be displayed can be displayed together.

  As a result, the operator has only one photograph of the label production apparatus 10 as long as the actual label production apparatus 10 and a photograph / drawing showing the appearance of the label production apparatus 10 are present (especially without reading the details of the instruction manual). Various information such as functions, operation methods, and warnings related to a plurality of parts can be easily known for each part, and convenience can be improved.

  The image generation procedure in step S8 includes the name of the content target part as the content related information 36 for the content target part of the captured image data generated by the camera unit 34, and the operator can perform a tap operation. Generated image data (contents 35, 36 and image of the print button 12) with the tap button added is generated.

  Thereby, a tap button that also serves as a name display of the content target part is displayed in the imaging field of view, and the content itself is displayed in detail when the tap button is further tapped. It can be.

  Therefore, this is particularly effective when it is difficult to display the full text of the content 35 on the content target part in the imaging field of view, such as when the content 35 is relatively long.

  Furthermore, the image generation procedure of step S8 is the content 35 or the content related information 36 regardless of the temporal change of the captured image data generated by the camera unit 34 (for example, when the imaging region is shifted to the left and right and the field of view changes). And the post-addition image data are generated in real time so that the positional relationship between the image data and the corresponding content target portion is maintained substantially constant.

  As a result, the field of view taken by the camera unit 34 changes, and even if the content target part moves within the field of view, the contents 35 and 36 of the content target part move together in synchronization. As a result, the sense of unity between the content target portion displayed on the touch panel unit 33 and the contents 35 and 36 can be maintained.

  In addition, the content acquisition procedure in step S7 is the operation method (for example, label) of the operated part when the content target part is an operated part (such as the print button 12 or the label cut lever 15) operated by the operator. In the case of the cut lever 15) or when the content 35 or the content related information 36 including the function (for example, the print button 12) is acquired, and the content target part is a connection unit (USB, power connector, etc.) with other devices The content 35 or the content related information 36 including connectable device information is acquired, and the content 35 or the content related including the caution warning when the content target part is a cautionary part (cutter or thermal head). Information 36 is acquired.

  As a result, when the operation field (such as various operation levers, opening / closing covers, lids, knobs, etc.) is included in the imaging field, the operation method and function of the operation area can be added and displayed. Part (USB, power supply connector, etc.) can be displayed with the name of the device that can be connected to the connection part, etc. When included, a warning indicating that the part needs attention can be added and displayed. As a result, convenience can be improved with certainty.

  In the present embodiment, the appearance (actual machine) of the label producing apparatus 10 having a predetermined characteristic shape part and content target part (operation lever, key SW, connector, dangerous part) or a simple instruction manual 20 are determined. A camera unit 34 that captures at least a part of an appearance image (a picture or a photograph of a device printed in an instruction manual) and generates corresponding captured image data, a touch panel unit 33 that performs desired display, and an operator's request The feature determination procedure of step S6 for determining whether or not the feature image portion is included in the captured image data generated by the camera unit 34 with respect to the CPU of the information terminal 30 having the operation means for performing the operation When the feature determination procedure in step S6 determines that the feature shape portion is included in the captured image data, the content target portion prepared for each of a plurality of types of languages is involved. Content 35 in which the corresponding content target part is included in the captured image data from the ROM of the information terminal 30 that stores the content 35 or the content related information 36 in association with the content target part (operation button, key SW, connector, dangerous part). Alternatively, the content acquisition procedure in step S7 for acquiring the content related information 36 and the content 35 corresponding to the content target part acquired by the CPU of the information terminal 30 with respect to the content target part of the captured image data generated by the camera unit 34. Alternatively, the content generation information 36 is added to generate the post-addition image data in step S8, and the content 35 or content-related information added to generate the post-addition image data in the image generation procedure in step S8. Selectively switch 36 language types And language switching procedure in step S11, the display control procedure of step S9 for displaying the additional image data generated by the image generation procedure in step S8 on the touch panel unit 33, is executed.

  The information terminal 30 on which the image processing program of the present embodiment is executed includes a camera unit 34, a touch panel unit 33, and a ROM. Then, by capturing the appearance of the label producing apparatus 10 or the appearance image of the simple instruction manual 20 with the camera unit 34, the content related information relating to the predetermined content 35 or the content 35 with respect to the captured image of the label producing apparatus 10. A new image to which 36 is added is generated and displayed on the touch panel unit 33.

  In other words, the appearance of the label producing apparatus 10 or the appearance image of the simple instruction manual 20 includes a predetermined content target portion in which the characteristic shape portion and the contents 35 and 36 are associated with each other. The contents 35 and 36 are prepared for each of a plurality of types of languages, and are stored in a manner uniquely associated with the content target part.

  When the image processing program of the present embodiment is executed by the CPU of the information terminal 30, first, in the feature determination procedure in step S6, the feature shape portion is included in the captured image data generated by the imaging of the camera portion 34. Whether or not is included is determined.

  When the feature shape portion is included and the determination is satisfied, the content acquisition procedure of step S7 is executed in response to this. That is, in the present embodiment, the subsequent processes are sequentially performed with the image of the characteristic shape portion of the label producing apparatus 10 as a trigger.

  In the content acquisition procedure of step S7, the corresponding content target part is included in the captured image data among the plurality of contents 35 and 36 stored in the ROM of the information terminal 30, in other words, the imaging field range of the camera unit 34. At least the content target part is acquired.

  Thereafter, in the image generation procedure of step S8, new post-addition image data is generated by adding the acquired contents 35, 36 to the corresponding content target part in the captured image data.

  As a result, new image data in which the contents 35 and 36 corresponding thereto are added to the content target portion within the imaging visual field range is generated and displayed on the touch panel unit 33 in the subsequent display control procedure of step S9.

  At this time, the language types of the contents 35 and 36 added for generating post-addition image data are selectively switched by the language switching procedure in step S11. Therefore, the contents 35 and 36 in the new image data displayed on the touch panel unit 33 can be selectively switched appropriately to any one of a plurality of types of languages.

  As described above, in the present embodiment, only the appearance of the label producing apparatus 10 or the appearance of the simplified instruction manual 20 is captured, and the content corresponding to the content target portion included in the captured image is displayed. The contents 35 and 36 can be displayed together.

  As a result, the operator can identify the content target portion of the label producing apparatus 10 without reading the details of the instruction manual, as long as there is an actual photograph of the label producing apparatus 10 or a photograph / drawing showing the appearance of the label producing apparatus 10. Various information such as functions, operation methods, and warnings can be easily known.

  At that time, by displaying the contents 35 and 36 in a language appropriately selected from a plurality of types of languages, for example, the display can be performed in a language that is most easily understood by the operator. Can be improved.

  In addition, the language switching procedure in step S11 can selectively switch the language type of the content 35 or the content related information 36 in accordance with the switching operation of the operator on the operation means.

  Thereby, since the contents 35 and 36 can be displayed in a language selected by the operator's intention, the convenience for the operator can be improved with certainty.

  Further, the CPU of the information terminal 30 is caused to further execute switching auxiliary information (language setting information; nameplate image data; GPS position information) for switching in the language switching procedure in step S11. The language switching procedure in step S11 is: The language type of the content 35 or the content related information 36 is selectively switched according to the content of the switching auxiliary information.

  In the present embodiment, for example, language setting information in the information terminal 30, the language of characters (for example, a logo, a nameplate, etc.) included in the appearance of the label production device 10 of the captured image data or the appearance of the simplified instruction manual 20 Based on the switching auxiliary information such as information and position information acquired by the information terminal 30 (for example, acquired by the GPS function), the language type of the contents 35 and 36 is automatically switched in the language switching procedure in step S11. To be done. Thereby, since the operator can display the contents 35 and 36 in an appropriate language without having to manually select a language, the convenience of the operator can be improved with certainty.

  The language switching procedure in step S11 includes language setting information in the information terminal 30 as switching auxiliary information, the appearance of the captured image data label producing apparatus 10 or the characters (ex. The language type of the content 35 or the content related information 36 is selectively switched according to at least one of the language information of the nameplate) and the position information acquired by the information terminal 30.

  This makes it possible to display the contents 35 and 36 in an appropriate language without the need for the operator to select a language manually, thereby improving the convenience for the operator.

  Further, the ROM of the information terminal 30 stores the content 35 or the content related information 36 for each specification (ex. US specification, JP specification) of the label producing apparatus 10 having a plurality of specifications respectively corresponding to a plurality of destinations. The content acquisition procedure in step S7 is stored in association with the part, and the appearance or simplified handling of the label producing apparatus 10 for the captured image data of the content 35 or the content related information 36 stored in the ROM of the information terminal 30 is used. The specification (contents 35 displayed in English) corresponding to the language information of the characters (for example, nameplates for English-speaking products) included in the appearance image of the manual 20 is acquired, and the language switching procedure in step S11 is a switching assist. According to the language setting information in the information terminal 30 as information (Japanese information used because the information terminal 30 is used by Japanese) Language types of content 35 or content-related information 36 selectively switches.

  In the present embodiment, a plurality of specifications (for example, for the United States, for Japan, etc.) corresponding to a plurality of destinations are prepared for the label producing apparatus 10. The ROM of the information terminal 30 stores the contents 35 and 36 in association with the content target parts for each specification.

  Then, in the content acquisition procedure of step S7, based on the characters of the appearance of the label producing apparatus 10 (or the appearance image of the simplified instruction manual 20) included in the captured image data, the contents 35, etc. relating to the specification corresponding to the language type, etc. 36 is obtained.

  On the other hand, in the language switching procedure in step S11, the language types of the acquired contents 35 and 36 are switched so as to correspond to the language setting of the information terminal 30.

  Thereby, for example, a Japanese operator who lives in the United States and uses the label production apparatus 10 for the United States uses the information terminal 30 (language setting is Japanese) that he / she owns to take an image. Thus, the contents 35 and 36 of the label producing apparatus 10 for the US specification can be displayed in Japanese from the beginning. By displaying such fine contents 35 and 36, the convenience of the operator can be improved more reliably.

  The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit and technical idea of the present invention.

  Furthermore, the printing apparatus of the present invention is not limited to the label producing apparatus 10 described above, and can be applied to other various printers and multifunction machines. Similarly, the information terminal 30 is not limited to the above-described smartphone, and is not particularly limited as long as it has an imaging function such as another mobile phone terminal or various tablet terminals.

  In addition, the flowcharts shown in FIGS. 13, 14, 15, and 16 do not limit the present invention to the procedures shown in the above-described flow, but add / delete procedures within a range that does not depart from the spirit and technical idea of the invention. Alternatively, the order may be changed.

  In addition, although not illustrated one by one, the present invention is implemented with various modifications within a range not departing from the gist thereof.

10 Label production equipment (printing equipment)
11 Power button (feature shape part / operation target part)
12 Print button (feature shape part / operation target part)
13 Operation key part (feature shape part / operation target part)
14 Display part (feature shape part / operation target part)
15 Label cut lever (feature shape / operation target part)
20 Simple Instruction Manual 30 Information Terminal 33 Touch Panel Unit 34 Camera Unit (Imaging Means)
35 Content 36 Content related information

Claims (2)

An imaging unit that captures at least a part of an appearance or an appearance image of a label producing apparatus as a printing apparatus that includes at least one characteristic shape portion and a plurality of content target portions, and generates corresponding captured image data. And a display means for performing a desired display, with respect to the computing means of the information terminal,
A feature determination procedure for determining whether or not the feature shape portion is included in the captured image data generated by the imaging means;
When it is determined that the feature shape portion is included in the captured image data in the feature determination procedure, a plurality of contents or content-related information respectively associated with the plurality of content target parts are associated with each content target part. A content acquisition procedure for acquiring at least one of the plurality of contents or content-related information from the stored storage means, wherein the corresponding content target part is included in the captured image data;
The content or content related information corresponding to the content target part acquired by the content acquisition unit is added to the content target part of the captured image data generated by the imaging unit, and the post-addition image data is obtained. An image generation procedure to be generated;
A display control procedure for displaying the post-addition image data generated in the image generation procedure on a display means;
An image processing program for causing execution,
The image generation procedure includes:
The addition with a tap button added to the content target part of the captured image data generated by the imaging means, which includes the name of the content target part as the content-related information and can be tapped by an operator Generate post-image data,
Regardless of the temporal change of the captured image data generated by the imaging means, the positional relationship between the content or content-related information and the content target portion corresponding to the content or the content-related portion is maintained substantially constant. Generate image data after addition,
The content target part is:
The label producing apparatus includes a print button for an operator to instruct the production of a printed matter,
The content acquisition procedure includes:
When the tap button corresponding to the print button is tapped by an operator while the post-addition image data is displayed on the display means, the moving image data stored in the storage means is acquired,
The display control means includes
An image processing program characterized in that, based on the moving image data, a moving image from which a label is drawn is displayed on the display means so as to be interruptible . In the image processing program according to claim 1 ,
The content acquisition procedure includes:
When the content target part is an operated part operated by an operator, the content or the content related information including the operation method or function of the operated part is acquired,
When the content target part is a connection unit with another device, the content including the connectable device information or the content related information is acquired,
An image processing program that acquires the content or the content-related information including a caution warning when the content target portion is a caution portion.

Images