JP6986237B2 - Virtual label display processing program - Google Patents

Description

The present invention relates to a virtual label display processing program executed by an operation terminal that operates a label creation device.

A technique for creating a label to be attached to an object to be attached by operating a label creating device from an operation terminal is well known. In this kind of technology, a print image of a label created on a display monitor of an operation terminal such as a personal computer (PC) is displayed. When the operator operates and inputs characters, symbols, etc. while looking at the image display, the corresponding print data is generated by the operation terminal and transmitted to the label creation device. As a result, in the label making apparatus, printing corresponding to the print data is performed on the conveyed tape (label tape) by the printing means (thermal head), and the label is created. It is also known that a printed image of a bulletin board suitable for the place where the label is to be posted can be easily created based on the environmental information at the place where the label is to be posted (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2015-082237

In the above-mentioned prior art, the operator determines the suitability for the environment in which the label is to be attached when the label is actually created. Therefore, print data may be created in advance when there are a large number of objects to be pasted, such as equipment in the workplace or personal belongings brought to school. Since the print data is created before where to paste it, it was not possible to judge the compatibility between the created label and the place to paste it. In this way, the technical content of "the place where the label is attached is already decided and the print content (template) suitable for that place is determined" is disclosed, but the opposite, "the label (including the print content) is already determined". It was fixed and could not meet the simulation needs of deciding where to put the label that fits the label.

An object of the present invention is to provide a virtual label display processing program capable of finding a suitable place for attaching a label by using print data for a label created in advance and improving convenience for a user. There is something in it.

In order to achieve the above object, the virtual label display processing program of the present invention includes an operation unit, an image pickup unit, a display unit, and a calculation unit for operating a label creation device that creates a stickable label. The first step of generating virtual image data for displaying the virtual image corresponding to the image image of the label on the display unit for the calculation unit of the mobile terminal, and outputting the virtual image data to the display unit. A second screen is displayed on the display unit, which includes the virtual image of the label at a predetermined first position on the display unit and further includes a first button for instructing the start of shooting by the imaging unit. When the step and the operation for the first button on the first screen are performed, the real image data of the desired field of view taken by the image pickup unit is acquired, and the real image data and the virtual image data are combined. A second screen in which the composite image data is output to the display unit and the virtual image of the label is superimposed on the image of the field of view so that the virtual image of the label is included in the second position on the display unit. Execute the third step to be displayed.

The virtual label display processing program of the present invention is used to find a suitable place for attaching a label to be created by a user. That is, when the virtual label display processing program is executed in the arithmetic unit of the mobile terminal, the virtual image data of the label is first generated (first step). The virtual image data may, for example, reflect the result of the user performing an editing operation via the operation unit on an appropriate editing screen, or may be print data of a label in a pre-completed state. Then, the virtual image data is output, and the first screen is displayed, which includes the virtual image of the label at a predetermined first position on the display unit and further includes the first button instructing the start of shooting by the imaging unit (second step). ).

Then, when the user operates the first button on the first screen, the real image data of the desired field of view taken by the image pickup unit is acquired, and the composite image data obtained by synthesizing the real image data and the virtual image data is obtained. A second screen is displayed, which outputs and superimposes the virtual image of the label on the image of the field of view so that the virtual image of the label is included in the second position on the display unit (third step). As a result, the virtual image of the label is displayed in a state of being overlapped with the image of the desired field of view, so that the user can see the virtual appearance when the label is attached to the object to be pasted in the field of view image. The aspect can be easily imagined visually.

By performing the above processing, the user moves the mobile terminal while displaying the virtual image of the label to be created, or moves it by holding it in his hand to image various places. By turning around, you can find a suitable place to put a label. As a result, the convenience for the user can be improved.

According to the present invention, it is possible to visually simulate the mode when the label is attached to the object in various states with different visual fields.

The appearance of the operation terminal for executing the virtual label display processing program is shown, (a) is a front view of the operation terminal, and (b) is a rear view of the operation terminal. It is a front view which shows the appearance of the label making apparatus which is the operation target of the operation terminal. It is a functional block diagram which shows the functional structure of an operation terminal and a label making apparatus. This is a flow showing a control procedure related to label creation executed by the control circuit of the label creation device. It is a front view which shows an example of the appearance shape of the created label. It is explanatory drawing which shows the layout screen of the print data of a label. It is explanatory drawing which shows the virtual label display screen. An example of a composite display of a virtual label image and a virtual label image is shown, (a) is an explanatory diagram of an image pickup state, and (b) is a pasting position setting in which the captured real object image and a virtual label image are compositely displayed. It is explanatory drawing of the search) screen. It is a flowchart which shows the control procedure executed by the control circuit of an operation terminal. It is a front view which shows the appearance of the other label making apparatus which is the operation target of the operation terminal.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

In FIGS. 1A and 1B, the mobile terminal 10 includes a device main body 11 and an image pickup unit 12 having a camera function as an image pickup means.

The apparatus main body 11 is composed of, for example, a touch panel type color liquid crystal display, and has a display unit 13 as a display means for displaying various information and various messages and allowing an operator to perform various operations, an operation button that can be operated by the operator, and the like. The operation unit 14 of the above is provided.

The image pickup unit 12 is provided on the upper part of the apparatus main body 11 (the upper part in FIGS. 1A and 1B), and is on the opposite side of the surface of the apparatus main body 11 where the display surface of the display unit 13 is provided. That is, a camera lens 12A is provided on the back surface. For example, when the operator displays an appropriate icon or the like on the display unit 13 by the operation of the operation unit 14 and presses the icon with his / her finger to select the icon, the touch panel function activates the image pickup unit 12 to shoot an object (movie shooting or video shooting). It is possible to select still image shooting).

In FIG. 2, the label making device 20 has a device main body 21. The device main body 21 is, for example, a display unit 22 composed of a liquid crystal display and displaying various information and various messages, an operation unit 23 having a plurality of button switches and the like capable of various operations by the operator, and inside the device main body 21. It is provided with a lever 24 for driving the arranged cutter 31 (see FIG. 3 described later).

The operation unit 23 has various function keys 23a including input keys for inputting characters, symbols, numbers, etc., and a movement key that functions as a movement key for the cursor position when inputting characters and a transition key when selecting a mode. 23b, etc. are included. The operation unit 23 can have a plurality of functions with one key.

In FIG. 3, when the mobile terminal 10 uses the control circuit unit 15, the storage unit 16 for storing various information, and the camera function in addition to the image pickup unit 12, the display unit 13, and the operation unit 14 described above. It has a distance measuring unit 17 for distance measurement and the like that can be used for focusing and the like, and a communication control unit 18. Of the display units 13, the touch panel portion that accepts the operation function by the touch panel method and the operation unit 14 having various keys function as the operation means according to each claim.

The control circuit unit 15 includes a CPU (calculation unit), RAM, and ROM (not shown). The control circuit unit 15 executes various programs stored in advance in the ROM while using the temporary storage function of the RAM by the arithmetic function of the CPU. The various programs include the virtual label display processing program of the present invention. Further, the control circuit unit 15 is connected to the label creating device 20 by wireless communication via the communication control unit 18, and can transmit and receive various information and various data to and from the label creating device 20.

The storage unit 16 is composed of a non-volatile memory such as, for example, an Electrically Erasable Programmable Read-Only Memory (EEPROM), which can add and erase stored contents. In this storage unit 16, in addition to the virtual label display processing program according to the present invention executed by the control circuit unit 15 (CPU), the label L is created and the compatibility is determined using the camera function (see the flow described later). Various programs for this are stored.

The communication control unit 18 controls information communication performed with the label creating device 20 connected via radio.

In FIG. 3, the label creating device 20 includes a control circuit unit 25, a storage unit 26 for storing various information, and a communication control unit 27, in addition to the display unit 22, the operation unit 23, the lever 24, and the cutter 31 described above. , A cartridge holder 32, a transport device 33 as a transport means, and a print head 34 as a printing means.

The control circuit unit 25 includes a CPU, RAM, and ROM (not shown). The control circuit unit 25 executes various programs stored in advance in the ROM while using the temporary storage function of the RAM. As a result, the control circuit unit 25 controls the entire label making device 20. Further, the control circuit unit 25 is connected to the mobile terminal 10 via the communication control unit 27, and can send and receive various data to and from the mobile terminal 10.

The storage unit 26 is composed of a non-volatile memory such as, for example, an Electrically Erasable Programmable Read-Only Memory (EEPROM), which can add and erase stored contents. The storage unit 26 can store print data transmitted from the mobile terminal 10.

The communication control unit 27 controls information communication performed with the mobile terminal 10 connected via wireless as described above.

The cartridge holder 32 is configured so that the cartridge 36 capable of supplying the printed tape 35 can be attached and detached. The cartridge 36 includes a tape roll 37 (which is originally spiral but is simplified and shown as concentric circles) in which the tape to be printed 35 is wound around a reel member (not shown). The cartridge holder 32 acquires the tape color information (tape color and tape width) of the set cartridge 36 and transmits it to the control circuit unit 25.

The transport device 33 is provided so as to face the print head 34. The transport device 33 is controlled by the control circuit unit 25 to transport the printed tape 35 supplied from the cartridge 36 mounted on the cartridge holder 32, that is, unwound from the tape roll 37 of the cartridge 36.

The print head 34 is controlled by the control circuit unit 25 to perform desired printing on the printed tape 35 that is fed out from the tape roll 37 and conveyed by the conveying device 33.

The cutter 31 cuts the printed tape 35, which has been printed by the print head 34, to a predetermined length by operating the lever 24 by the operator, and has one label L with printing (FIG. 5 described later, etc.). See).

The procedure for creating the label L by the label creating device 20 will be described with reference to FIG.

In FIG. 4, the process shown in this flow is started by the mobile terminal 10 outputting a label creation instruction to the label creation device 20 (“START” position). First, in step S1, the control circuit unit 25 outputs a control signal to the transfer device 33, and feeds the printed tape 35 from the tape roll 37 of the cartridge 36 mounted on the cartridge holder 32, that is, conveys the printed tape 35. To start.

Then, the process proceeds to step S2, and the control circuit unit 25 determines whether or not the printed tape 35 has been conveyed by a predetermined amount. The predetermined amount is, for example, a transport distance (so-called front margin transport amount) such that the tip of a print area (not shown) of the tape to be printed 35 reaches a position substantially opposite to the print head 34. The determination in step S2 is not satisfied until the predetermined amount is conveyed (step S2: No), the loop waits, and when the predetermined amount is conveyed, the determination in step S2 is satisfied (step S2: Yes), and the process proceeds to step S3.

In step S3, the control circuit unit 25 outputs a control signal to the print head 34, and starts printing the print data stored in the storage unit 26 on the print area of the printed tape 35 conveyed by the transfer device 33. Let me.

After that, in step S4, the control circuit unit 25 determines whether or not all the printing of the print data to the print area of the printed tape 35 by the print head 34 started in step S3 is completed. The determination in step S4 is not satisfied until all printing is completed (step S4: No), the loop waits, and when all printing is completed, the determination in step S4 is satisfied (step S4: Yes), and the process proceeds to step S5.

In step S5, the control circuit unit 25 further conveys the printed tape 35 by a predetermined amount (for example, a transport distance such that the entire print area exceeds the cutter 31 by a predetermined length, that is, a transport amount for the back margin). Determine if it has been done. The determination in step S5 is not satisfied until the predetermined amount is conveyed (step S5: No), the loop waits, and when the predetermined amount is conveyed, the determination in step S5 is satisfied (step S5: Yes), and the process proceeds to step S6.

In step S6, the control circuit unit 25 outputs a control signal to the transport device 33, and stops the feeding of the printed tape 35 from the tape roll 37 started in step S2, that is, the feeding of the printed tape 35.

Then, in step S7, the control circuit unit 25 outputs a display signal to the display unit 22 and causes the display unit 22 to display that the tape to be printed 35 can be cut by operating the lever 24. After that, the process shown in this flow ends. When the operator operates the lever 24 according to the above display, the cutter 31 operates and the printed tape 35 is cut. By cutting with the cutter 31, the portion printed by the print head 34 is separated from the printed tape 35 to form one label L.

As shown in FIG. 5, the label L created as described above is a rectangular label having a width Lw and a length Lg. In this example, a print RR (character string of "UENO-TECH-10450") corresponding to the print data is printed on the surface of the label L.

In the above basic configuration, the feature of the present embodiment is that the mobile terminal 10 has a virtual label equivalent to the label L before the label creating device 20 actually creates the label L as shown in FIG. By displaying the figure (hereinafter, appropriately referred to as "virtual label") together with the image of the background object taken by the imaging unit 12, the suitability of determining whether or not it is preferable to actually attach the label L to the background object is determined. Is made to be a mobile terminal 10. The details will be described below in order.

<Transition of screen display status>
First, the "layout" screen shown in FIG. 6 is displayed on the screen of the display unit 13 by an appropriate operation input by the operator via the display unit 13 and the operation unit 14. On the screen of the display unit 13 on this "layout" screen, the characters "layout" are displayed in the work content display field 13a, and the print data R such as input characters is displayed in the print data entry field 13b imitating the print tape. A "label creation" button and a "virtual paste" button that are displayed and switched by the touch panel function are displayed in the operation button field 13c.

Then, in the display state of this "layout" screen, as shown in FIG. 6, the operator traces with a finger in the print data entry field 13b (or conversely, in the margin) using the touch panel function to write. Enter the desired text character (character string of "UENO-TECH-10450 ..." in the illustrated example. The cursor C is displayed at the end) and set the print data R. ..

At this time, the length of the label L is set by the control circuit unit 15 according to the content (for example, the number of characters) of the print data R. The width of the label L is fixedly determined in advance, or is set by the same operator operation as described above in an entry field (or another screen, window, etc.) (not shown). In this example, the length dimension of the label L is 35 [mm] and the width dimension is 9.5 [mm], which is indicated on the screen of the display unit 13 such as near the margin of the print data entry field 13b. It is displayed. The contents of the print data R of the label L set by such an editing operation and the length dimension and the width dimension of the label L are stored in the storage unit 16.

After the above editing operation is completed, the operator presses (touches) the "label creation" button in the operation button field 13c, so that the print data R generated as described above is transmitted via the communication control unit 18. Is transmitted to the label making device 20. As a result, the flow shown in FIG. 4 is executed, and the label creating device 20 creates the label L on which the print RR corresponding to the print data R is printed as shown in FIG.

On the other hand, after the above editing operation is completed, the operator presses the "virtual paste" button in the operation button field 13c. As a result, the screen of the display unit 13 is switched to the "virtual label display" screen shown in FIG. 7. On the screen of the display unit 13 on this "virtual label display" screen, the characters "virtual label display" are displayed in the work content display field 13a, the virtual label LI is displayed, and the operation button field 13c is displayed by the touch panel function. The video icon to be switched is displayed. The video icon is pressed (touched) at the start / end to shoot a moving image. At this time, instead of this video icon, a camera icon for shooting a still image may be displayed, or a video icon and a camera icon may be displayed.

In the switched "virtual label display" screen, the virtual label LI reflecting the editing operation on the "layout" screen of FIG. 6, that is, the image image of the label L actually created is displayed on the screen of the display unit 13. It is displayed in a predetermined size in the center. At this time, the display position of the virtual label LI is fixed on this "virtual label display" screen. Then, the operator uses the camera function of the mobile terminal 10 to attach the actual label L, that is, the object to be attached, that is, the object to be determined for conformity (for example, described later). The electric kettle P) is placed at a position within the field of view of the image pickup unit 12, and the video icon in the operation button field 13c is pressed. As a result, the image pickup unit 12 starts shooting a moving image of the object to be pasted.

FIG. 8 shows an example of the display state of the display unit 13 after the moving image shooting is started in this way. For example, as shown in FIG. 8A, when the image pickup unit 12 takes a picture of the object to be pasted (for example, the electric kettle P), the real image taken (hereinafter, as shown in FIG. 8B) is taken. As appropriate, the "paste position setting (search)" screen is displayed so that the "object image PI") is superimposed on the virtual label LI as the background of the virtual label LI. During shooting, a "REC" mark is displayed in the vicinity of the video icon in the operation button column 13c to notify that recording is in progress. If it can be recognized that recording is in progress, the notification mode is arbitrary.

Then, when the operator holds the mobile terminal 10 by hand and changes the positional relationship with the pasted object, the position and size of the object image PI displayed on the screen of the display unit 13 also change in real time. , The virtual label LI continues to maintain a fixed display at a predetermined size in the center. As a result, the control circuit unit 15 of the mobile terminal 10 considers the relative positional relationship between the object image PI and the virtual label LI, and "preferably attaches the created label L to this place of the object. (Conspicuous / inconspicuous) ”can be replaced with the virtual label LI on the object image PI by virtual pasting to determine the compatibility.

At this time, on the screen of the display unit 13, a plurality of color selection icons AC in which the tape color and the character color are paired are displayed so that the character color and the tape color of the generated virtual label LI can be changed. Is displayed. The color selection icon AC is a pair of a preset character color and a tape color. The color selection icon AC allows the operator to visually select the color. For example, the tape color on the left side and the character color on the right side are displayed on the display unit 13 in a list format. The application can include icon information for displaying the color selection icon AC on the display unit 13. At this time, the mobile terminal 10 may be acquired from the label creating device 20 as a part of the update information of the application, for example. Further, for example, the mobile terminal 10 acquires the current tape color information (tape color and tape width) after having the label making device 20 identify the type of the cartridge 36 set in the cartridge holder 32 by the cartridge holder 32. You may. As a result, the mobile terminal 10 causes the operator to make the icon display corresponding to the tape color currently worn among the display of the color selection icon AC of the display unit 13 different from other icon displays (for example, to enlarge). Can be recognized by. As described above, the control circuit unit 15 of the mobile terminal 10 targets the label L, for example, based on the compatibility of the tape color and the character color of the virtual label LI with the color of the captured object image PI and the like. The suitability of whether or not it is preferable to attach it to an object is determined, and at least when a determination result of that it is preferable is obtained, a notification to that effect is given (details will be described later).

In such a basic configuration, when the label L to be printed is completed as the print data R, and the contents such as characters and figures to be printed and the appearance such as the color and size of the tape to be printed 35 are determined, the operator can perform the operation. , It is most suitable for attaching the label L by moving while shooting with a mobile terminal 10 such as a smartphone with a camera and displaying a virtual image (virtual label LI) of the label L on the shot background image in real time. It enables simulations to search for and rotate parts. As a result, when the control circuit unit 15 determines that the compatibility is high (or determines whether the compatibility is high or low) by comparing the color information of the background image that changes from moment to moment with the color information of the label, a message to that effect is obtained. By notifying the operator by displaying M, voice, or vibration, it is possible to find the most suitable place to attach the label, and it is possible to improve the convenience for the user. be.

The control circuit unit 15 stores, for example, sample data indicating compatibility of color coordination in the storage unit 16 by a table method or the like, and uses the stored sample data to generate a virtual label LI and an object image PI. The suitability of can be determined. The hue of the object can be recognized by a two-dimensional color image sensor (CCD) provided in the image pickup unit 12.

<Control content>
The control content executed by the control circuit unit 15 (specifically, the CPU) in order to execute the display content (simulation) as described above will be described with reference to the flow chart of FIG.

In FIG. 9, first, in step S11, the control circuit unit 15 outputs a display signal to the display unit 13, displays a “layout” screen as a screen for operation editing (see FIG. 6), and then labels L. Operation by the operator to create or edit the print data R is acquired according to the input content. After that, the process proceeds to step S12 in response to the operation of the above-mentioned "virtual paste" button. When the operator does not create or edit the label L, that is, when the print data R that has already been created is used, it is sufficient to acquire the print data R from the storage unit 16. Further, when the print RR or the like of the label L is fixedly determined (using a template or the like), or when the creation / editing is performed using another device, the creation / editing in this step S11 is performed. The routine of the control circuit unit 15 related to the operation is omitted, and for example, the print data R including the current tape color information is acquired. It should be noted that this step S11 corresponds to the editing acceptance procedure described in each claim.

In step S12, the control circuit unit 15 displays the virtual label LI corresponding to the image image of the label L on the screen of the display unit 13 based on the print data R including the current tape color information acquired in step S11. The virtual image data of is generated and the process proceeds to step S13. The generated virtual image data is output to the display unit 13, and the virtual label LI is displayed on the screen of the display unit 13 (see FIG. 7). Note that this step S12 corresponds to the virtual image generation procedure described in each claim.

In step S13, the control circuit unit 15 acquires real image data (including image data of the object image PI) of a desired field of view including an object (electric kettle P) photographed by the operator in the image pickup unit 12. This step S13 constitutes the actual image acquisition procedure described in each claim. After that, the process proceeds to step S14.

In step S14, the control circuit unit 15 synthesizes the virtual image data generated in step S12 and the real image data acquired in step S13 to generate composite image data. This step S14 constitutes the composite image generation procedure described in each claim. After that, the process proceeds to step S15.

In step S15, the control circuit unit 15 outputs the composite image data generated in step S14 to the display unit 13, and displays the virtual label LI and the object image PI on the screen of the display unit 13 in an superimposed manner (FIG. FIG. 8). It should be noted that this step S15 corresponds to the superimposed image generation procedure and the composite image output procedure described in each claim.

Then, in step S16, the control circuit unit 15 accepts the mode setting when the control circuit unit 15 determines the suitability by the operation of the operator's mobile terminal 10, and also accepts the mode setting according to the accepted mode. It should be at least a part of the conditions for determining sex. It should be noted that this step S16 corresponds to the suitability selection procedure described in each claim.

For example, when the control circuit unit 15 has a known image recognition processing function, the actual image data acquired in step S13 is analyzed to include contours, edges, uneven portions, corner portions, black portions, and the like. At least two points having characteristics different from those of the site are extracted and the feature points are set. In this example, two points of the handle portion of the electric kettle P of the object image PI are set as feature points (see FIG. 8). As with the above, the operator may arbitrarily perform this feature point at a desired position by performing an appropriate operation on the screen of the display unit 13.

Then, in step S17, the control circuit unit 15 receives the virtual image data of the virtual label LI generated in step S12 and the actual image data acquired in step S13 based on the conditions corresponding to the mode set in step S16. Based on the above, it is determined whether or not the appearance of the object to be attached and the appearance of the label L satisfy the desired compatibility. When the control circuit unit 15 determines that the desired compatibility is satisfied, the control circuit unit 15 proceeds to step S18. On the other hand, if the control circuit unit 15 does not determine that the desired compatibility is satisfied, the control circuit unit 15 repeats this routine until it determines that the compatibility is satisfied. However, the control circuit unit 15 ends this routine when the operator performs an operation for forcibly ending the conformity determination, or when it is not determined that the conformity determination is satisfied more than a predetermined number of times. In the case of moving image shooting, whether or not the desired suitability is satisfied is determined by using a predetermined determination start operation of the operator as a trigger. At this time, in the case of moving image shooting, the suitability may always be determined in real time. Here, the control circuit unit 15 extracts the color information of the actual image data and the color information including the label of the print data, and the compatibility of these two color information (the label L is clearly visible on the background wall or object). Whether it is such a combination in the case of the mode you want to stand up, or whether it is such a combination in the case of the mode where you want to bury the label L in the background wall or object) is calculated or compared every moment by the CPU. Judgment is possible with. Note that this step S17 corresponds to the determination procedure described in each claim.

Then, the process proceeds to step S18, and the control circuit unit 15 notifies the operator that the desired compatibility is satisfied, and ends this routine. As the notification method, a message M to that effect may be displayed on the display screen of the display unit 13, or the eyes may be moved by voice. Specifically, when the judgment result related to suitability exceeds a certain threshold value, it is regarded as "the best place to put this label is here!", And at that point, "This" is immediately displayed on the smartphone screen. It is displayed as "is the best threshold for your label", the label virtual image is blinking, it is notified by voice, or it is notified by vibration. Note that this step S18 corresponds to the notification procedure described in each claim. Even if it is not determined in step S17 that the compatibility is satisfied, it is possible to sequentially notify a message or the like indicating that the label L is not suitable here.

This allows the operator to recognize the best place to put the label. At this time, the determination of the compatibility between the virtual label LI and the object can include a color change due to light and dark as the color of the background, that is, the object to be attached. Further, as the color of the label L, in addition to the color of the tape to be printed 35, the color of the ink printed on the label L may be taken into consideration. As the color information of the determination method, RGB values are used for calculation. At this time, it is possible to determine the suitability not only by the mere hue but also by the hue according to the time zone, the season, and the like. As a result, for example, the label of the cooking content when serving in the morning buffet of the hotel and the label when serving the same cooked food at night will change the color of the favorable impression. Therefore, such a condition may be included in the determination criteria.

Further, in determining the suitability, the purpose of use of the label L is, for example, when the label is to be clearly conspicuous with respect to the background, and conversely, when the label is to be embedded in the background so as to be as inconspicuous as possible. Can be assumed. For example, if the content of the print is a character such as "Be careful of forgetting the umbrella!", If the color of the tape is blue, by searching for an object that matches the blue, an image that suits not only the location but also the purpose. It is possible to determine the suitability including.

Here, when it is desired to make the label stand out clearly with respect to the background, whether or not the background color (RGB value) and the tape color (RGB value) are equal to or higher than a predetermined threshold value set to be considerably large. The target rigidity can be determined by the above. On the contrary, when it is desired to bury the label L in the background so as to be as inconspicuous as possible, the background color (RGB value) and the tape color (RGB value) are set within a predetermined range (for example, 0). It can be determined by whether or not the threshold value is equal to or higher than a predetermined threshold value close to a predetermined value and is larger than the threshold value but considerably smaller than the threshold value). It should be noted that the determination may be made based on the combination of color coordination described above. Therefore, such determination of suitability can be applied not only to the created print data R in general, that is, to the technique of "searching for a background that matches the label L" but also to the technique of "searching for a label that matches the background".

Further, the relative size of the object to be attached and the label L can also be a factor in determining the conspicuous / inconspicuous compatibility. That is, even if the labels L have the same size, they are conspicuous when they are attached to a narrow place, and they are less conspicuous when they are attached to a wide place. Therefore, it is possible to include such an area ratio between the object and the label L in the determination condition.

The distance measuring unit 17 measures the distance between the mobile terminal 10 and the object by, for example, a known distance measuring method such as an optical sensor, and the actual external dimensions, uneven shape, etc. of the object are measured by the angle of view. Get the size of. As a result, the control circuit unit 15 may display the virtual label LI by scaling according to the acquired external dimensions of the object and the like.

As described above, the mobile terminal 10 acquires the virtual image generation procedure (step S12) for generating the virtual image data and the real image data of the desired field including the pasted object taken by the imaging unit 12. An image acquisition procedure (step S13) and a composite image generation procedure (step S14) for generating composite image data by synthesizing the real image data acquired in the real image acquisition procedure and the virtual image data generated in the virtual image generation procedure. , A composite image output procedure (step S15) in which the composite image data generated in the composite image generation procedure is output to the display unit 13 and the virtual label LI is superimposed and displayed on the image in the desired field of view on the display unit 13. Based on the actual image data acquired in the actual image acquisition procedure and the virtual image data generated in the virtual image generation procedure, it is determined whether or not the appearance of the object to be pasted and the appearance of the label L satisfy the desired compatibility. A determination procedure (step S17), a notification procedure (step S18) for performing a predetermined conformity notification (display on the display unit 13; voice notification; vibration notification) when the determination procedure determines that conformity is satisfied. Is to execute.

Therefore, the virtual label display processing program is used to find the most suitable place for attaching the label L to be created by the operator. That is, when the virtual label display processing program is executed by the control circuit unit 15 of the mobile terminal 10, virtual image data is first generated (virtual image generation procedure). This data may reflect, for example, the result of an editing operation performed by the operator via the touch panel of the display unit 13 on an appropriate editing screen, or may be print data of the label L in a pre-completed state. .. Then, when the operator photographs a desired field of view including the object to be pasted by the imaging unit 12 of the mobile terminal 10, actual image data corresponding to the captured result is acquired (actual image acquisition procedure). As a result, the virtual image data and the real image data are combined (composite image generation procedure), and the combined image data is output to the display unit 13 (composite image output procedure). As a result, the display unit 13 displays the virtual label LI in a state of being overlapped with the image of the desired visual field, so that the operator can attach the label L to the object to be attached in the visual field image. The virtual appearance of the above can be easily visualized.

Here, in the mobile terminal 10, it is possible to determine the desired compatibility between the appearance of such an object to be attached and the appearance of the label L attached to the appearance. As an example of compatibility, whether or not the label L appears to stand out clearly when the label L is attached to the object to be attached (clarity as a clear feeling), or conversely, the label L is attached. It is conceivable whether or not the label L appears to be buried moderately when attached to the object (the degree of burial as an inconspicuous feeling). Based on the real image data and the virtual image data, it is determined in the determination procedure whether or not the desired suitability is satisfied. Then, when the determination is satisfied, the operator is notified of the conformity to the effect that the conformity is satisfied in the notification procedure.

By performing the above processing, the operator moves the mobile terminal 10 or holds it in his / her hand to capture images of various places while the virtual label LI to be created is displayed on the display unit 13. By doing so, it is possible to find the most suitable place for attaching the label L. As a result, the convenience for the operator can be improved.

At this time, the display position of the virtual label LI on the display unit 13 is substantially fixed regardless of the fluctuation of the visual field. By generating the composite image data, the operator moves the mobile terminal 10 or holds it in his / her hand. Even when the field of view displayed on the display unit 13 changes from moment to moment, the virtual label LI can be displayed at the same position in the display unit 13 without changing.

As shown by the three black dots in FIG. 8B, if the position correlation of the triangle is set between the object image PI and the virtual label LI, a certain fixed correspondence can be maintained. It will be possible. As a result, even if the field of view captured by the image pickup unit 12 fluctuates and the object image PI displayed on the display unit 13 moves or changes in size, the virtual label LI is moved or the size is changed so as to follow it. Can be changed. As a result, even though the label L has not yet been attached to the object to be attached (in this example, the electric kettle P) in the real world, the label L is attached to the object as if it were attached to the display unit 13. It is possible to virtually realize a state as if it were. This makes it possible to visually simulate the mode when the label L is attached before the object is actually attached. Further, since the virtual label LI can be simulated in various states in which the field of view is changed, the operator can obtain a feeling closer to the actual pasted state, and the convenience can be improved.

Further, by determining the suitability based on the color information of the predetermined range (pasting portion) including the object to be pasted, which is narrower than the desired field of view in the real image data, and the color information of the virtual image data. For example, by using RGB values as color information, suitability can be determined by numerical arithmetic processing.

As a desired suitability, at least, by determining whether or not the clarity (clearness) of the label L when the label L is attached to the object to be attached is equal to or higher than a predetermined threshold value, the operator You can find the best place to look the most clearly when you put the label L that you plan to create.

On the contrary, as a desired suitability, at least, by determining whether or not the degree of burial (inconspicuous feeling) of the label L when the label L is attached to the object to be attached is within a predetermined range. It is possible to find the optimum place to be buried properly when the operator attaches the label L to be created.

In this way, the control circuit unit 15 is further executed with the compatibility selection procedure (step S16) for accepting the selection input of the desired compatibility content via the touch panel of the display unit 13, and the appearance of the object to be pasted. By determining whether the appearance of the label L and the appearance of the label L satisfy the desired compatibility received in the compatibility selection procedure, the operator can see, for example, the most clearly when the label L to be created is attached. You can select and enter whether to search for a place or to search for a place to be buried properly when the label L to be created is attached.

The control circuit unit 15 synthesizes and superimposes the virtual image data in a mode in which the appearance size of the label L is scaled and the real image data based on the appearance of the pasted object based on the real image data. By generating data, the label L is scaled (enlarged or reduced) in order to satisfy the desired suitability (want to be clearly conspicuous or moderately buried), for example, depending on the color and shape of the object to be pasted. ) Is better, and the operator can visually recognize that fact.

Moreover, the control circuit unit 15 is further made to execute the edit acceptance procedure (step S11) for accepting the edit operation for the print content of the label L, and the virtual image data reflecting the edit operation accepted in the edit acceptance procedure is generated. As a result, the virtual label LI can be displayed in the display unit 13 in a form that reflects the result of the editing operation performed by the operator for printing on the label L.

Further, in the present embodiment, in particular, on the layout screen (see FIG. 6) displayed on the display unit 13, the operator can optionally set the mode of the label L such as the print RR by performing an editing operation. .. As a result, the virtual label LI corresponding to the label L for which the operator desires to set the print RR or the like is displayed in a state of being virtually attached to the object (before actually creating the label L). Can be made to.

In the above, the print data R for the label L created in advance is obtained by taking a picture with the mobile terminal 10 with respect to the object to be attached (in the example, the electric kettle P) while taking an image with the image pickup unit 12. An example has been described in which a virtual label LI based on the above is generated and a suitable place for attaching the virtual label LI is searched for (see FIG. 8).

However, the present invention is not limited to this, and for example, the print data R created in advance is not limited to one. That is, even if the contents to be printed are exactly the same, the print data R for a plurality of types of labels L may be created by changing the size and color of the label L, the color of characters, and the like. In such a case, when the operator shoots the place with the mobile terminal 10 when the place to which the label L is to be attached has already been decided, the above-mentioned is described on the side of the shooting background image displayed on the screen of the display unit 13. Instead of the color selection icon AC, the virtual label LI based on the print data R already created is displayed in a list. Then, when one of them is selected, the selected virtual label LI is superimposed and displayed in real time with the object image PI as the background. Selecting another tape replaces the selected virtual label LI. By repeating this, the control circuit unit 15 may determine the compatibility with the virtual label LI of the tape color most suitable for sticking to the place.

Specifically, an example of which color of the label L already created fits after the operator first determines the place to attach the label L will be described.

In this case, the mobile terminal 10 photographs a place (wall, object, device, etc.) to which the label L is attached, and displays the photographed image on the screen of the display unit 13.

After that, when the operator appropriately operates the mobile terminal 10, virtual label LIs for a plurality of types of labels L previously stored in the storage unit 16 are displayed in a list on the screen of the display unit 13 (objects). It does not overlap with the image PI and is displayed in a separate frame, but it may be displayed in an overlapping manner).

Then, the operator selects a sample of any one label L listed on the screen of the display unit 13, so that the virtual label LI corresponding to the control circuit unit 15 is displayed and switched to the center on the screen. As a result, the overlapping virtual display as if the label L is attached to the object image PI is performed in real time, and at the same time, the suitability can be determined.

The arrows shown in the block diagram shown in FIG. 3 indicate an example of signal flow, and do not limit the signal flow direction.

Further, the routines in the flow charts shown in FIGS. 4 and 9 do not limit the present invention to the above routines, and add / delete or change the order of procedures within a range that does not deviate from the purpose of the invention and the technical idea. You may. For example, the condition setting (conformity selection procedure) for determining legality shown in step S16 may be accepted at the same time as or immediately after the creation / editing of the label L in step S11. That is, it may be performed at any timing as long as it is a routine before the suitability is determined by the control circuit unit 15 and after the print data R is acquired.

In addition to the above-mentioned above, the methods according to the above-described embodiment and each modification may be appropriately combined and used. Also, the mobile terminal 10 is. As long as it has a camera function, a display function, and a communication function and is excellent in portability, it is not limited to the smartphone described above, and may be, for example, a tablet terminal, a digital camera, a digital video camera, or the like. Further, as shown in FIG. 10, since the touch panel of the display unit 13 of a smartphone, a digital camera, a digital video camera, etc. can be used as the operation unit 14, the display unit 22 and the operation unit 23 of the label making device 20 are abolished. (Or omitted).

In addition, although not illustrated one by one, the present invention is carried out with various modifications within a range not deviating from the gist thereof.

10 Operation terminal 12 Imaging unit (imaging means)
13 Display unit (display means / operation means)
15 Control circuit unit (calculation means)
20 Label making device

Claims (5)

For the calculation unit of a mobile terminal including an operation unit, an image pickup unit, a display unit, and a calculation unit for operating a label creation device that creates a label that can be attached.
The first step of generating virtual image data for displaying a virtual image corresponding to the image of the label on the display unit, and
A first button that outputs the virtual image data to the display unit, includes the virtual image of the label at a predetermined first position on the display unit, and further includes a first button instructing the start of shooting by the imaging unit. The second step of displaying the screen on the display unit,
When the operation to the first button of the first screen is performed, while an actual image of the desired field of view captured by the imaging unit, wherein the said label such that the virtual image is included in the second position the virtual image formed by superimposing, a third step of displaying a second screen,
To execute ,
The second screen provides an option in which the color of the tape constituting the virtual image can be selected from a plurality of colors and an option in which the color of characters formed as the virtual image can be selected from a plurality of colors. Including the color change part including
On the second screen, the color changing portion is displayed at a position where it is superimposed on the real image of the desired field of view and is not superimposed on at least a part of the virtual image.
Virtual label display processing program. The color changing unit is a plurality of color changing sets in which the color options of the tape and the color options of the characters can be collectively selected by the combination of the color of the tape and the color of the characters. Virtual label display processing program described in. On the second screen,
The display position of the virtual image, the is maintained in the second position regardless of variations in the visual field,請Motomeko 1 or claim 2 virtual label display processing program according on the display unit. It said first position and said second position is the same position on the display unit, the virtual label display processing program according to any one of請Motomeko 1 to claim 3. Further to the arithmetic unit,
A fourth step of displaying a third screen on the display unit, which includes a third button for shifting to the first screen, accepts an editing operation for the print content of the label, and displays print data corresponding to the editing operation, is executed. Let me
And in the third button the first screen switching is displayed by the operation of the print data is displayed in the first position of the display unit as the virtual image, either請Motomeko 1 to claim 4 The virtual label display processing program described in item 1.

Images