JP3236488U - A mobile terminal device that generates an AR image and a stamp used for it. - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mobile terminal device capable of reliably reading a marker from an imprint and generating an AR image of a person or the like of the imprint, and a stamp used thereof. SOLUTION: A portable terminal device 10 for generating an AR image of the present invention has an imprint imaging means for imaging an imprint including a name 20 and a marker 23 arranged around the name 20, and an image captured by the imaging means. The imprint image acquisition means for extracting the imprint image from, the marker extraction means for extracting the markers contained in the imprint from the acquired imprint image, and the generated image associated with the extracted marker are selected from the generated image database. The generated image control means for generating and the generated image display means 15 are provided. A plurality of generated images 30 associated with the marker can be used, and can be selected and displayed based on the imaging conditions. [Selection diagram] Fig. 1

Description

The present invention relates to a mobile terminal device that generates an AR image on a screen by reading an imprint and a stamp used thereof.

AR (Augmented Reality) is translated as augmented reality, and is a technology that enables visually augmented reality by superimposing digital information based on information from the real world. In order to utilize this technology, an index called an AR marker that links the real world with digital information is required. In the present specification, it is simply abbreviated as a marker.

Patent Document 1 describes a technique in which an imprint displaying an icon such as a character or a logo that is easy for human eyes to understand is used as a marker, and an AR image is displayed on the screen when the imprint is read by a mobile terminal. However, there is a problem that it is difficult to make an image that is easy for the human eye to understand and a marker that can be reliably read by the imaging means, and it is not easy to manufacture the marker.

Patent Document 2 describes a technique in which an image that is easy for the human eye to understand is used as the first layer, a marker that is invisible to the human eye but can be reliably read by the imaging means is used as the second layer, and the marker is read using infrared rays. Are listed. However, Patent Document 2 has a problem that a special reading means is required. The image of Patent Document 2 is, of course, not an imprint.

It should be noted that a general seal displays the surname of a person, and if there are multiple people of the same identity or if the surname is common, it may be difficult to understand just by looking at the imprint. It has not been proposed to use AR to make up for the problem of this stamp.

Japanese Patent No. 5468152 Japanese Unexamined Patent Publication No. 2008-146108

Therefore, an object of the present invention is to solve the above-mentioned conventional problems, and to provide a portable terminal device capable of reliably reading a marker from an imprint and generating an AR image of a person or the like of the imprint, and a stamp used thereof. That is.

The portable terminal device for generating an AR image of the present invention, which has been made to solve the above-mentioned problems, includes an imprint imaging means for imaging an imprint including a name and a marker arranged around the name, and an image pickup means using the image pickup means. An imprint image acquisition means for extracting an imprint image from the acquired image, a marker extraction means for extracting a marker included in the imprint from the acquired imprint image, and a generated image associated with the extracted marker from the generated image database. It is characterized by comprising a generated image control means for selecting and generating and a generated image display means.

It should be noted that the number of generated images associated with the marker can be a plurality of, and the generated images can be selected based on the imaging conditions.

Further, the stamp of the present invention made to solve the above-mentioned problems recognizes the marker on the periphery of the name included in the imprint, selects and generates the generated image associated with the recognized marker, and displays the display unit. It is a stamp used for a mobile terminal device to be displayed on the screen, and is characterized by having a marker engraved on the periphery of the name.

According to the present invention, since the marker is arranged around the name of the imprint, different images can be generated and displayed on the screen according to the marker even if the names are the same. Further, the marker can be reliably read by the imaging means.

It is explanatory drawing of 1st Embodiment of this invention. It is a perspective view which shows the seal and the imprint. It is a block diagram which shows the structure of a mobile terminal apparatus. It is explanatory drawing of the processing procedure of the imprint image acquisition means. It is explanatory drawing of the processing procedure of a marker extraction means. It is a flow figure which shows the procedure of 1st Embodiment. It is explanatory drawing of the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the 2nd Embodiment. It is a flow chart which shows the procedure of 2nd Embodiment.

An embodiment of the present invention will be described below.
FIG. 1 is an explanatory diagram of a first embodiment of the present invention. In this embodiment, when the imprint 20 of the name is imaged by the mobile terminal device 10, the generated image 30 including additional information about the person with the name is displayed on the screen of the mobile terminal device 10.

As shown in FIG. 2, the stamp 21 has a circular shape, and the marker 23 is arranged around the peripheral portion of the name 22. In this embodiment, a ring-shaped marker 23 in which 1/8 arcs are connected is arranged on the outer peripheral portion of the name 22. Further, the imprint 20 includes a protrusion shape 24 indicating the upper part. The name is not necessarily limited to the Chinese character notation, and may be a notation using an alphabet.

In this embodiment, the marker 23 is continuously formed on the entire circumference of the peripheral portion of the name 22. However, it does not necessarily have to be continuous, and the marker 23 may be partially formed on the peripheral portion of the name 22, or the marker 23 may be further reduced to be concentrated on a part of the peripheral portion of the name 22. In either case, it is preferable that the marker 23 and the name 22 are separated.

A stamp is a hard stamp made of ivory, plastic, metal, wood, etc., or a stamp made of non-porous rubber or elastomer, and the stamp is attached to a holder member. It can be interpreted in a broad sense, such as a rubber stamp made of rubber, a stamp made of porous rubber, or an elastomer, and then impregnated with ink, and the stamp is assembled to a holder member.

The marker 23 of this embodiment can assign a numerical value from 0 to 9 to each segment of the arc of 1/8 and code a numerical value of 10 to the 8th power. The numerical value is represented by the area ratio of the colored portion in each segment. For example, if the area ratio of the colored portion is 30%, it is "3", and if it is 50%, it is "5". However, it is also possible to assign 0 or 1 to each segment, in which case a numerical value of 2 to the 8th power can be coded. By further increasing the number of segments, the number that can be coded can be further increased. However, since the imprint 21 formed by the stamp 21 has a limit in size and sharpness, if the number of segments is increased too much, a reading error may occur, and about 8 to 12 is a preferable range.

FIG. 3 is a block diagram showing the configuration of the mobile terminal device 10. The portable terminal device 10 of the first embodiment includes imprint imaging means 11, imprint image acquisition means 12, marker extraction means 13, generated image control means 14, and generated image display means 15. Each means will be described below.

First, the user acquires a dedicated application program by downloading, and further stores it in the program area of the storage unit in the mobile terminal. (Not shown.) Then, the user performs an operation for starting the application program. By activating this application program, it becomes possible to generate and output an effect image by AR or the like based on the image recognition of the marker arranged around the name by the mobile terminal device 10.

The imprint image pickup means 11 is a camera mounted on the mobile terminal device 10, and images the imprint 20 of the stamp 21 as shown in FIG. At this time, the image is not always taken from directly above the imprint 20, and the distance between the imprint 20 and the imprint imaging means 11 is not constant. Therefore, the imprint image acquisition means 12 is used from the images obtained by the imprint imaging means 11. For example, the imprint image is taken out by the following procedure.

First, color masking is applied to the image. Since the imprint 20 is usually reddish, which is the color of red meat, masking is performed so that only the reddish portion is highlighted. Next, the image is converted to grayscale, and a threshold value is set to obtain a binarized image.

At this stage, a large number of figures may be detected in the image, so the figure with the largest area is detected as the imprint figure. The recognition accuracy can be improved by utilizing whether or not the detected figure has a peripheral portion. From the angle between the imprint image pickup means 11 and the imprint 20, it is considered that the detected imprint figure often has an elliptical shape. Therefore, the imprint figure is cut out as an circumscribing rectangle as shown in FIG. Furthermore, it is transformed into a square according to the long side of the cut out image, and the imprint figure is approximated to a circle.

In this way, the figure that approximates the circle is image-processed, and the outline of the imprint figure is extracted again. Then, the minimum circumscribed circle and the circumscribed rectangle are created for the detected imprint. As described above, since the imprint 20 has a protrusion shape 24 indicating the upper part, the minimum circumscribed circle includes the protrusion shape 24, and the center of the minimum circumscribed circle is biased toward the protrusion shape 24. On the other hand, since the circumscribed rectangle embeds the protrusion shape 24 in the rectangle as shown in FIG. 4, the center of the circumscribed rectangle coincides with the center of the circle of the imprint figure. By calculating the vector from the center of the circumscribed rectangle to the center of the minimum circumscribed circle, the inclination of the imprint figure can be known, and the inclination of the imprint figure can be corrected.

Next, the marker extraction means 13 reads the value encoded by the marker 23 from the imprint figure. First, as shown in FIG. 5, the part of the name 20 in the imprint figure is masked, and a rectangle including a 1/8 arc is cut out. Then, the pixel of the cut out portion is read out, and the numerical value is specified by the ratio of the binarized pixel. This rectangle contains a part other than the marker 23, but the reading error can be eliminated by preparing and comparing the number of pixels in the part where the data does not exist and the sample of the number of pixels for each value in advance. Can be done.

In addition to using the ratio of the number of pixels in this way, it is also possible to adopt a method of extracting the outline of a 1/8 arc from the cut out image and comparing the circumference of the portion showing the value with the 1/8 arc. In this way, the value encoded by the marker 23 can be reliably read.

Finally, the name 22 included in the imprint 20 is converted into text. This operation can be performed using the API (Application Programming Interface) provided by Google Cloud Platform. This software can also recognize vertical characters. The above procedure is shown in FIG.

Next, the generated image control means 14 accesses the generated image database 16 and selects the generated image associated with the value read from the marker 23. The generated image database 16 is stored in an external server capable of communicating with the mobile terminal device 10. In the first embodiment, the generated image 30 is additional information about a person with a name displayed on the imprint 20, for example, as shown in FIG. 1, a surname, a photograph, an affiliation, and a telephone number. Such a generated image 30 is displayed on the generated image display means 15 of the mobile terminal device 10 together with the imprint 20. In FIG. 1, the generated image 30 is displayed as if it springs out from the imprint 20, and even if the imprint 20 has the same name, different generated images 30 can be displayed on the screen according to the marker 23.

It is preferable that the contents of the generated image database 16 be rewritable so that the person can respond to changes in the affiliation of the person.

Next, a second embodiment of the present invention will be described. As shown in FIG. 7, the second embodiment displays a character on the mobile terminal device 10, but there are a plurality of generated images associated with the marker, and the generated image is selected based on the imaging conditions. It is the one that was made. In addition, also in the second embodiment, the name and the imprint in which the marker 23 is arranged around the name are used.

Also in this second embodiment, after the dedicated application is started, the imprint is imaged by the imprint image pickup means 11 of the mobile terminal device 10, the imprint image acquisition means 12 takes out the imprint image, and the marker extraction means 13 is a marker from the imprint figure. Reading the value encoded by is the same as in the first embodiment. However, as shown in FIG. 8, a memory 17 is built in the mobile terminal device 10, and in addition to an imprint image and marker identification information, an imaging condition storage means 18 is stored therein. The image pickup condition storage means 18 stores the image capture date and time and the number of times the imprint was imaged.

The generated image database 16 of the server stores images to be generated according to the marker, the date and time of imaging, and the number of times of shooting. When the user captures a marker with the mobile terminal device 10, the generated image control means 14 requests the generated image database 16 to generate a generated image that matches conditions such as the identified marker, the recognized imaging date and time, and the number of times of shooting. The generated image database 16 transmits a generated image that matches the request, and causes the generated image display means 15 to display the generated image. As a result, the user can see different generated images depending on the imaging time and the number of times of shooting, and can obtain the feeling of organically communicating with the appearing character.

For example, if the imaging date and time is in the morning of August 1st and the number of imaging times is the first, along with the image of the character with a tense expression, "Good morning. Nice to meet you. I, Shachihata. Thank you for your continued support. When I displayed the words "." And then took a second image on the afternoon of August 15, I found an image of a slightly relaxed facial expression and "Hello. It's still hot days. When the third image was taken on the afternoon of September 15th, with the image of a relaxed smile, hello. Let's go to the moon viewing with me. "Is displayed. In this way, it is possible to obtain a timely dialogue according to the imaging date and time, and an image of tone and facial expression according to the number of imaging times. FIG. 9 shows the procedure of the second embodiment.

It is also possible to include the position information in the imaging conditions, specify the imaging location by using the GPS receiving function mounted on the mobile terminal device 10, and make the generated image different depending on the imaging location.

If the generated image database 16 can be pre-registered and rewritten, the imprint provider can provide more timely information. Furthermore, if a template for pre-registration is created, pre-registration can be performed more easily.

It is also possible to create a large number of stamps with the same name with different code values and display different characters according to the captured imprints.

As described above, according to the present invention, it is possible to reliably read a marker from an imprint and generate an AR image of a person or the like of the imprint on the screen of the mobile terminal 10.

10 Portable terminal device 11 Imprint image acquisition means 12 Imprint image acquisition means 13 Marker extraction means 14 Generated image control means 15 Generated image display means 16 Generated image database 17 Imaging condition recognition means 20 Imprint 21 Stamp 22 Name 23 Marker 24 Protrusion shape 30 Generated image

Claims (4)

An imprint imaging means for imaging an imprint including a name and a marker arranged around the name, and
An imprint image acquisition means for extracting an imprint image from an image captured by this imaging means, and an imprint image acquisition means.
A marker extraction means for extracting markers included in the imprint from the acquired imprint image, and
The generated image control means that selects the generated image associated with the extracted marker from the generated image database and generates it,
A mobile terminal device for generating an AR image, which comprises a generation image display means. The portable terminal device for generating an AR image according to claim 1, wherein a plurality of generated images associated with the marker are generated. The portable terminal device for generating an AR image according to claim 2, wherein the generated image is selected based on an imaging condition. It is a stamp used for a mobile terminal device that recognizes the marker around the circumference of the name included in the imprint, selects the generated image associated with the recognized marker, generates it, and displays it on the display unit. A stamp characterized by a marker engraved on the circumference.

Images