JP7473758B1 - Dot Pattern - Google Patents

Abstract

[Problem] The technical problem is to enable a dot pattern to be read with visible light by a simple printing method and simple processing on any reading device such as a smartphone. [Solution] A dot pattern in which dots are arranged according to a predetermined rule, the dots being the reference for encoding with the dot pattern and including a plurality of reference dots including dots whose orientation can be specified, and information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots, a plurality of dot patterns are formed on a medium, and at least a portion of an image other than dots is formed at a predetermined position on the medium superimposed or adjacent to the dot pattern, at least one dot pattern includes an undecodable area where at least a portion of the dots cannot be decoded, and dots corresponding to the undecodable area are complemented by a dot pattern other than the dot pattern including the undecodable area. [Selected Figure] Figure 1

Description

The present invention relates to a dot pattern that can be read in the visible light range.

Dot codes, a type of two-dimensional code, have been known for some time. The dot pattern that encodes the dot code can be printed superimposed on graphics such as pictures and characters, making it an extremely excellent two-dimensional code that can store a large amount of information on a printed matter without impairing the appearance of the printed matter. To make the dot pattern less visible to the human eye, very fine dots are printed with infrared absorbing ink (carbon ink, stealth ink, etc.), and even if the dot pattern is printed superimposed on the graphics, only the dots can be read using a dedicated reading device equipped with an infrared camera. Since the dots and graphics are superimposed, they cannot be read in the normal visible light range. Currently, audio pens and Bluetooth (registered trademark) pens are particularly used. When the dot pattern is touched with the audio pen to read the dot code, a sound corresponding to the dot code is instantly output. Furthermore, with the spread of smartphones and tablets, Bluetooth pens have appeared, and when a dot code is read, content such as a video is output to a wirelessly connected smartphone or tablet.

However, because both require a dedicated reading device, there is an increasing need to read dot codes using the cameras on smartphones, the world's most powerful content platform. However, smartphones are equipped with infrared blocking filters to prevent voyeurism, and it is not possible to capture only the dot pattern in the visible light range where it is printed overlaid with graphics.

Here, a method for reading dot patterns has been proposed that enables reading dot codes in the visible light range by performing color separation processing (for example, Patent Document 1).

International Publication No. WO2006/040832

However, in color separation processing, only one or two of the colors cyan (C), magenta (M), and yellow (Y) can be used to distinguish black dots, so that black (ink or composite black, a mixed color) and dark colors do not appear. This often means that the original color of the graphic cannot be expressed. Furthermore, if the color is too dark, it may not be possible to distinguish it from a dot, so color separation requires very complex and delicate processing. Furthermore, to actually achieve this processing, it is necessary to accurately focus on the fine dot pattern, and there are many constraints such as the performance of the smartphone camera and the printing condition.

The present invention was made in consideration of these problems, and its technical objective is to enable dot patterns to be read with visible light using a simple printing method and simple processing on any reading device, such as a smartphone.

(1) In order to solve the above problem, the dot pattern of the present invention is a dot pattern in which dots are arranged according to a predetermined rule, and the dots serve as a reference for encoding with the dot pattern and include a plurality of reference dots including dots whose orientation can be specified, and information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots, and the dot pattern is formed in a plurality of dot patterns on a medium, and at least a portion of an image other than the dots is formed at a predetermined position on the medium, superimposed on or adjacent to the dot pattern, and at least one of the dot patterns includes an undecodable area in which at least some of the dots cannot be decoded, and dots corresponding to the undecodable area are complemented by a dot pattern other than the dot pattern including the undecodable area.

(2) Furthermore, at least a portion of the dot pattern may not be printed in the unusable area, or at least a portion of the dot pattern may be printed superimposed on the image.

(3) The dot pattern of the present invention is a dot pattern in which dots are arranged according to a predetermined rule, and the dots serve as a reference for encoding with the dot pattern and include a plurality of reference dots including dots whose orientation can be specified, and information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots, and the dot pattern is formed in a plurality of dot patterns on a medium, and at least a portion of an image other than the dots is formed at a predetermined position on the medium, superimposed on or adjacent to the dot pattern, and the image is a focusing image on which a dot code reading device that reads the dot pattern can be focused.

(4) The dot pattern of the present invention is a dot pattern in which dots are arranged according to a predetermined rule, and the dots serve as a reference for encoding with the dot pattern and include a plurality of reference dots including dots whose orientation can be specified, and information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots, and the dot pattern is formed in a plurality of dot patterns on a medium, and at least a portion of an image other than the dots is formed in a predetermined position on the medium superimposed or adjacent to the dot pattern, and the image includes a specific image that can be recognized by an information processing device built into or connected to a dot code reading device that reads the dot pattern, and at least a portion of the dot code encoded with the dot pattern specifies at least one of the specified information and one or more information processes associated with the specific image.

(5) Furthermore, a plurality of dot-like protrusions may be provided superimposed on, adjacent to, or close to the image.

(6) Furthermore, at least a portion of the information dots may be composed of multiple dots of different densities, and the dots of at least one of the densities may be dummy dots that do not define information or information dots that define true information.

(7) Furthermore, the focus image may include at least a portion of a specific image, the specific image may be recognized by the information processing device built into or connected to the dot code reading device, and at least a portion of the dot code encoded by the dot pattern may identify at least one of the one or more predetermined pieces of information and one or more pieces of information processing associated with the specific image.

(8) Furthermore, at least a portion of the dot code encoded in the dot pattern may be associated with at least one of predetermined information and information processing.

(9) Furthermore, at least one of the predetermined information and information processing associated with at least a portion of the dot code may be formed by extension processing in a predetermined positional relationship with the medium surface on which the dot code is formed.

(10) Furthermore, at least one of the one or more pieces of specified information and one or more pieces of information processing associated with the specific image may be formed by performing an extension process in a specified positional relationship with the specific image.

(11) Furthermore, the predetermined positional relationship may be formed by performing an extension process based on at least the coordinate values included in the dot code.

(12) Furthermore, the coordinate values may correspond to coordinate values indicating the medium surface and may be able to identify a specific position on the medium surface.

(13) Furthermore, the predetermined positional relationship may be formed by obtaining the inclination angle and/or rotation angle between the shooting direction of the dot code reading device and the specific image from the orientation and deformation state of the dot pattern photographed by the dot code reading device, and performing an extension process on the predetermined positional relationship.

(14) Furthermore, the dot may be at least one of a size and a shape that the dot code reader cannot focus on within a predetermined time, and the focusing image may be one that the dot code reader can focus on within the predetermined time.

(15) Furthermore, the medium may be either a print medium or a display, and the dot pattern and the image may be printed on the print medium or displayed on the display.

(16) Furthermore, the dots may be formed superimposed on a predetermined background, and the colors of the dots and the color of the predetermined background may be set so that the arrangement of the dots can be obtained by distinguishing them from the predetermined background.

(17) Furthermore, the dot pattern may be formed on the medium in at least two adjacent patterns on the top, bottom, left and right.

(18) Furthermore, in the case where at least a portion of the reference dot cannot be recognized in the unrecognizable area and the dot cannot be decoded, a reference dot may be virtually placed at the position of the unrecognizable reference dot based on other recognizable reference dots, and the dot may be decoded.

(19) Furthermore, the medium may have a writing area formed nearby.

(20) The dot pattern generating device of the present invention includes a means for forming a plurality of dot patterns according to any one of (1) to (4) on a medium.

(21) The dot code reader of the present invention includes a means for capturing an image of a dot pattern described in any one of (1) to (4) and a means for decoding the dot pattern into a dot code.

(22) The dot pattern generation program of the present invention is a dot pattern generation program that causes a computer to form multiple dot patterns according to any one of (1) to (4) on a medium.

(23) The dot pattern reading program of the present invention is a dot pattern reading program that causes a computer to photograph a dot pattern according to any one of claims 1 to 4 using a dot code reading device and decode the dot pattern into a dot code.

(24) Furthermore, it is possible to access a table in which the dot code and information specifying the memory destination where information or an operation instruction is stored are registered in correspondence with each other, and the memory destination may be obtained from the table based on the decoded dot code, and the information may be output or the operation instruction may be executed.

(25) Furthermore, information or operation instructions to be registered in a table in association with the decoded dot code may be accepted, the accepted information or operation instructions may be stored in a predetermined storage destination, and the dot code and information specifying the storage destination may be registered in the table.

(26) The dot pattern generating method of the present invention is a dot pattern generating method for forming a plurality of dot patterns according to any one of (1) to (4) on a medium.

(27) The dot pattern reading method of the present invention is a dot pattern reading method that captures a dot pattern described in any one of (1) to (4) and the image, and decodes the dot pattern into a dot code.

(28) The dot pattern reading system of the present invention is a dot pattern reading system comprising: an information processing device including a reading means for photographing a dot pattern described in any one of (1) to (4) and decoding the dot pattern into a dot code; an output means for outputting at least one of predetermined information and the result of information processing; and a server for receiving the dot code and transmitting the predetermined information and information processing associated with at least a portion of the dot code to the information processing device.

(29) The dot pattern reading method of the present invention is a dot pattern reading method in which an information processing device photographs a dot pattern described in any one of (1) to (4), decodes the dot pattern into a dot code, and outputs at least one of predetermined information and the result of information processing, and a server receives the dot code and transmits the predetermined information and information processing associated with at least a portion of the dot code to the information processing device.

This two-dimensional code uses a simple printed graphic to focus, and the dot code is decoded using only the fine dot pattern photographed in the visible light range. This means that the dot code can be easily read by photographing it with a smartphone camera, making it inexpensive to use and capable of providing a variety of content.

1 is a diagram illustrating an information processing system using a two-dimensional code (dot pattern) according to a first embodiment of the present invention. 1A and 1B are external views of a reading device, in which FIG. 1A shows the front side of the device and FIG. FIG. 1 is a hardware configuration diagram of a smartphone. 1A to 1C are diagrams illustrating a medium on which a dot pattern according to the present invention is printed. FIG. 13 is a diagram for explaining how information dots are arranged. FIG. 13 is a diagram showing an example of code allocation of information dots. FIG. 13 is a diagram showing a general-purpose example of a dot pattern of “GRID0”. FIG. 13 is a diagram showing a modified example of the dot pattern of “GRID0”. FIG. 13 is a diagram showing an example of dot code concatenation. This shows a general-purpose example of a "GRID5" dot pattern. 13 is a flowchart showing a procedure for analyzing and decoding a dot pattern. FIG. 13 is a diagram showing a search start point set in a binarized photographed image of a GRID1 dot pattern. FIG. 13 is a diagram for explaining a procedure for searching for at least six dots closest to a search point. FIG. 1 is a diagram for explaining a procedure for searching for a reference dot; FIG. 13 is a diagram for explaining the procedure for searching for a reference dot (2). FIG. 11 is a diagram (3) for explaining the procedure for searching for a reference dot. 11A and 11B are diagrams illustrating a procedure for searching for a dot that is shifted from a virtual point by a predetermined interval on a grid line. FIG. 11 is a diagram (1) for explaining a procedure for acquiring numerical information defined in an information dot. FIG. 13 is a diagram (2) for explaining the procedure for acquiring numerical information defined in an information dot. 13 is a flowchart illustrating generation of a dotted icon. 11 is a flowchart showing a process from reading a dotted icon to outputting information. 10 is a flowchart showing details of a dot pattern analysis process. FIG. 13 is a diagram showing a dot icon in which a dot pattern and an icon image are printed in an overlapping manner. FIG. 13 is a diagram showing the result of binarizing dots from a captured image. 13A and 13B are diagrams illustrating a procedure for dividing a binarized dot image into nine regions and setting the center point of each region as a search start point (reference dot search start point). 11A to 11C are diagrams illustrating a procedure for acquiring numerical information of information dots that can be recognized from a binary image. 11A and 11B are diagrams illustrating a procedure for determining the arrangement of information dots by setting virtual reference dots. 11A to 11C are diagrams illustrating a procedure for acquiring numerical information by combining information dots in a plurality of regions. FIG. 11 is an explanatory diagram showing an example of the configuration of an augmented reality system using a specific image according to a second embodiment of the present invention. 1 is a table showing the correspondence between dot codes and displayed screens. 13 is an example of a screen that appears on a smartphone display after photographing an icon. 1 is a flowchart (1) showing the process of augmented reality according to the present invention. 11 is a flowchart (2) showing the process of augmented reality according to the present embodiment. 10A and 10B are diagrams illustrating the correspondence between coordinate values defined in a dot pattern and coordinate values indicating a medium surface. FIG. 1 is an explanatory diagram of the application of the present invention to a wine list. FIG. 13 is an explanatory diagram of the case where the present invention is used in a product catalog. FIG. 13 is an explanatory diagram of the case where the present invention is used in an emergency exit. FIG. 13 is an explanatory diagram showing the case where the present invention is used for explanation in an aquarium. FIG. 13 is an explanatory diagram of a case where a dot icon is a sticker showing a number. FIG. 40 is an explanatory diagram showing an example of use of FIG. 39 . FIG. 13 is a diagram showing an example of a notebook on which dot icons are printed. FIG. 1 is an explanatory diagram of the case where the present invention is used for authenticity determination and traceability. 11A to 11C are diagrams illustrating a technique for further improving the accuracy of authenticity determination in a dot pattern. FIG. 13 is an explanatory diagram of the case where the present invention is used for product promotion. FIG. 13 is an explanatory diagram of a case where dot icons are set on a tourist information map. FIG. 13 is an explanatory diagram of a case where a convex portion is provided in a dot icon.

The following describes an embodiment of the present invention with reference to the drawings.

First Embodiment
1 to 28, a first embodiment of the present invention will be described.

The first embodiment is characterized by the formation of a focusing image for photographing a dot pattern.

Figure 1 is a diagram illustrating an information processing system using a two-dimensional code (dot pattern) according to a first embodiment of the present invention. This image may be used as an icon indicating a specific meaning.

The information processing system shown in FIG. 1 includes a medium 10 on which an icon 1 having a dot pattern formed thereon (hereinafter referred to as a "dotted icon" or "dot icon") is printed, and a reading device 20 that captures the dot pattern.

Figure 2 shows the external appearance of the reading device, where (a) shows the front side of the device and (b) shows the back side of the device.

In the present invention, a commercially available smartphone, tablet, mobile phone, etc. can be used as the reading device. A dedicated reading device may also be used. Here, the structure when the reading device is a smartphone is described.

In the following description of the embodiment, the reading device is described as a "smartphone," but the present invention can of course also use reading devices other than smartphones, such as tablets and mobile phones.

As shown in FIG. 2A, the smartphone 20 has a display 220 on the front of a vertically long rectangular housing 210, and a plurality of operation buttons 230 on both sides. The housing 210 is made of materials such as resin and metal. The display 220 is vertically long and rectangular, and is a so-called touch panel display that combines a display such as a liquid crystal panel with a touch position input device such as a touch pad. The display 220 displays icons and the like required for operating the smartphone 20. The user can operate the smartphone 20 by touching the display 220 or by operating buttons and the like provided on the housing. In addition, images, videos, web pages, and the like are displayed on the display when the user operates the display. The operation buttons 230 accept user operation inputs such as power ON/OFF and volume. The operation buttons 230 may be provided on the front of the housing 210. In addition, the physical operation buttons 230 may not be provided, and operation inputs similar to those of the operation buttons 230 may be accepted on the display.

As shown in FIG. 2(b), the smartphone 20 has an imaging unit 240 on the back side. The imaging unit 240 captures an image of a subject in response to a predetermined operation by the user. In this embodiment, the imaging unit 240 captures an image of a dot pattern.

Figure 3 is a hardware configuration diagram of the smartphone 20.

As shown in the figure, the smartphone 20 contains components necessary to function as a smartphone, such as a display 220, an antenna 2010, a transmission/reception unit 2011, an information processing unit 2015, a memory unit 2016, a speaker 2017, a microphone 2013, an imaging unit 2018, and a power source 2014.

The transmitting/receiving unit 2011 transmits the signal (hereinafter, data) received by the antenna 2010 to the information processing unit 2015. The transmitting/receiving unit 2011 also transmits the data received from the information processing unit 2015 via the antenna 2010.

The information processing unit 2015 is a processor such as a CPU, and controls the entire reading device in response to instruction signals input from the operation buttons and the display. The information processing unit 2015 also stores data received by the transmission/reception unit 2011 via the antenna 2010 in the storage unit 2016. Furthermore, if the data received by the transmission/reception unit 2011 is audio data, the information processing unit 2015 outputs it to the speaker 2017 via the antenna 2010. The storage unit 2016 stores programs such as applications and operating systems executed by the information processing unit 2015, in addition to the data received by the transmission/reception unit 2011. The information processing unit 2015 executes a process of analyzing and decoding the dot pattern captured by the imaging unit 2018 based on the program stored in the storage unit 2016. The process for the dot pattern will be described later.

In the above-mentioned smartphone, the imaging unit 2018, the speaker 2017, and the display 220 are all integrated, but the present invention is not limited to this. For example, the reading device may have only the imaging unit 2018 and be connected to a personal computer having the speaker 2017 and the display 220 by wire or wirelessly. Furthermore, the imaging unit 2018, the speaker 2017, and the display 220 may all be separate and connected by wire or wirelessly. In addition, it may be a wearable computer with a camera built into a watch or glasses, or it may be a combination of a smartphone that is wirelessly connected to cameras mounted on various devices.

Figure 4 is a diagram illustrating a medium 10 on which a dot pattern according to the present invention is printed.

In this embodiment, an English language teaching material (book) is used as the medium 10. An icon is printed in the upper left corner of each page of the medium 10. The icon is composed of a picture and an outer frame. In FIG. 1(a), an icon with a speaker picture and an icon with a microphone picture are printed. In the area where the icons are printed, a two-dimensional code, which is the dot pattern of the present invention, is printed to form a dot icon 1. The word "Play" is written under the speaker picture icon, and the word "Record" is written under the microphone picture icon.

The icon may also serve as a focusing image. Since the dot pattern is very fine, if an image is taken with only the dot pattern printed or displayed, it is difficult to focus accurately on the dots, and the dots are photographed in a blurred state. Therefore, by using the icon as a focusing image as shown in the figure, the icon is automatically focused on, and at the same time, the dots forming the dot pattern are also focused on, so that the dot pattern is clearly photographed and accurately decoded into a dot code. In order to make it easier to focus, it is desirable for an icon suitable as a focusing image to be an image with strong edges in which the color and density change compared to the surroundings, regardless of whether it is the outside or inside of the icon, in other words, an image with clear contrast. The surroundings of the icon may be dark, the icon may be light, and the edges may be strong. Of course, the above-mentioned edges may exist inside the image forming the icon. When a user takes a picture of an icon with a smartphone, the depth of field varies depending on the model of smartphone. In such cases, the focus may not be achieved at a short distance, or the icon image may be captured so small relative to the capture area that the dots captured at the same time cannot be accurately recognized. In such cases, text or graphics may be displayed on the display of the user's smartphone to allow the user to manipulate the position of the smartphone from the print medium surface so that the distance between the smartphone and the medium is appropriate. In addition, a focused image may be selected from multiple captured images during autofocus operation using continuous shooting or video. However, since video generally has a lower resolution than still images (including continuous shooting), when using a video, the resolution must be sufficient to recognize the captured dots. Note that if the area in which the dots are formed is small, the display of the shooting guide (frame) should be small, and if the area is large, the shooting guide (frame) should be widened to capture within the depth of field in focus.

Furthermore, icons have the role of informing users of what information they are accessing. For example, in Fig. 4(a), the speaker icon informs users that audio will be played, and the microphone icon informs users that audio will be recorded. When a user takes a picture of the speaker icon with a smartphone, the text on the photographed page, such as "Hello!" or "Good Morning!", is played from the smartphone's speaker. When a user takes a picture of the microphone icon, audio recording is possible. This allows users to easily take a picture of the speaker icon, listen to a model English voice, take a picture of the microphone icon, record their own voice, and have their pronunciation and accent checked and coached through audio output and graphic display, thereby improving the learning effect. Fig. 4(b) shows multilingual icons formed on pamphlets, posters, and signage. When "JP" is read, the content is output to the smartphone in Japanese, when "EN" is read, and when "CN" is read, the voice, text, and graphics of the content to be provided to the user are output in Japanese, when "EN" is read, and when "CN" is read, in Chinese. Naturally, the different dot codes formed around each icon correspond to the language. In addition to teaching materials, it can be used in various scenes such as inbound tourism. If Braille is placed around or inside the icon and the dot pattern is read, it can also be used for visually impaired people. Figure 4 (c) is an icon for controlling audio and video. As in Figures 4 (a) and (b), it can be drawn with lines or letters, or a filled icon can be placed in the center with a dot pattern around it. Figures 4 (a) to (c) can have or not have a border as long as the icon part functions as a focus image. Furthermore, Figure 4 (d) is an example of the icon part of Figure 4 (c) being white or a light-colored background without black mixed in, with a dot pattern arranged so that the dots can be distinguished, and the area around the icon being a dark-colored focus image.

In order for the information processing unit 2015 to extract the dot pattern from the read image and decode it into a dot code, it is necessary to distinguish between the background and the dot pattern. Therefore, it is preferable to set the dot color and background color so that the background and dots can be distinguished.

For example, a case where the dots are set to black or nearly black will be described. In this case, if the background is also black or nearly black, it will be difficult to distinguish the dots. Therefore, it is desirable not to use black or nearly black ink (K ink) for the background. Even when printing using C (cyan), M (magenta), and Y (yellow) inks, the overlapping parts of the three colors C, M, and Y will be a color close to black. Therefore, it is desirable to print so that the three colors C, M, and Y do not overlap, or to print with one or two of C, M, and Y. When printing with one or two of C, M, and Y, if the dot area ratio of each color is below 50%, it is often possible to distinguish the dots from the background. If the dot area ratio is below 30%, it will be even easier to distinguish. Note that Y has a higher dot recognition rate than C and M, even if they have the same dot area ratio.

Dots may be set in a color other than black. In this case, the color should be specific enough to be distinguished from the background.

The outer frame or image of the icon may or may not have a dot pattern printed thereon. If the dot pattern is not printed thereon, the aesthetic look of the icon image is better maintained. Also, only the outer frame or the image of the icon may be printed.

In FIG. 4, paper is described as the medium, but the present invention is not limited to this. For example, printing can be done on any printable medium, such as cloth, screen, metal foil, etc. Furthermore, the dot pattern can be formed by engraving or holes other than printing. In this case, it is possible to form the dot pattern on metal, plastic, etc. other than paper. The dot pattern can also be displayed on a display, or the dot pattern can be projected onto a screen using a projector.

When displaying a dot pattern on a display, just as when forming it on a medium, it is preferable to set the dot color and background color so that the dots can be distinguished from the background. Any color scheme can be used for the graphic area, but it is preferable to set the contrast with the background as high as possible.

Generally, on a display, colors are expressed using three colors: R (red), G (green), and B (blue). In full color, each RGB color is handled with a value between 0 and 255 (8 bits). If a dot is to be black, the dot part is set to approximately R=0, G=0, B=0. For the background color, at least one of the RGB colors is set to 6 bits or more. 7 bits or more is even better. The brighter the background, the easier it is to distinguish between the dot and the background. Note that dots can also be identified by setting the color of the dot to a color that can be distinguished from the colors used for the icon, outer frame, and background color. This method can also be used in printing.

The medium 10 may be two-dimensional or three-dimensional. Examples of three-dimensional media include globes and product boxes, but are of course not limited to these, and dot patterns can be formed on any three-dimensional medium.

[Dot pattern description]
Next, an example of the above-mentioned dot code (dot pattern) will be described below with reference to FIGS.

Here, a "dot pattern" refers to an information code encoded using an algorithm for arranging multiple dots. Details will be given later, but the dots include at least information dots that define numerical information, and reference dots that serve as the basis for defining the information dots.

The numerical information (code) obtained by reading the above dot pattern is called a dot code, and is collectively referred to as a dot code. The same applies hereafter.

As for the encoding algorithm for the information code using the dot pattern, well-known algorithms such as GridMark's Grid Onput (registered trademark), Anoto's Anoto pattern, and Sonix's OID can be used.

Gridmark's Grid Onput (registered trademark) dot pattern is covered by numerous patents, including patent numbers 3706385, 3858051, 4142683, and 5331989. Details of these patents are described in the respective patent publications, but an overview is provided below.

The dot pattern encoding algorithm is provided as a dot pattern generation program executable by a computer. There is no restriction on how the program is provided as long as it is executable by a computer. For example, it may be stored on a recording medium such as a CD-ROM, or it may be downloaded to a computer via a network. Furthermore, the dot pattern encoding algorithm is the same whether it is read using visible light or infrared light, so there are no particular limitations.

Any other dot pattern can be used as long as it is either invisible or, if visible, is simply recognized as a pattern.

By defining coordinate values, the dot pattern can encode different information codes depending on the reading position. Furthermore, the dot pattern has a reference orientation for encoding and decoding information codes, and by reading this orientation, the rotation angle of the reading device relative to the dot pattern can be obtained. On the other hand, when the reading device is tilted relative to the dot pattern forming medium, the direction and the degree to which the reading device has been tilted can also be obtained from the change in brightness of the captured image.

(How to arrange information dots in FIG. 5)
The information dots are arranged as shown in FIGS.

Note that the arrangement of information dots is not limited to the examples shown in Figures 5(a) to (c).

Figure 5 (a) shows information dots placed at positions offset from the imaginary point, i.e., at a predetermined distance from the imaginary point and at an angle relative to a predetermined direction. Specifically, as shown in (a-1) of the same figure, information dots are placed above, below, left, right, and diagonally to the imaginary point. Note that the "angle" includes 0 degrees.

In FIG. 1(a-1), dots are arranged in eight directions with a constant distance from the virtual point, but in the dot pattern of the present invention, dots may be arranged in eight directions with the distance between the dots changed. This makes it possible to represent 16 different types of numerical information.

Also, multiple information dots may be placed based on one virtual point. Furthermore, as shown in FIG. 1(a-2), it is also possible to define information by whether or not to place an information dot at a virtual point. In this case, in addition to the dot placement method in (a-1), information may be defined by whether or not to place an information dot at a virtual point. This makes it possible to increase the amount of information. Also, instead of placing information dots at positions shifted from the virtual point as shown in (a-1), it is also possible to define information by whether or not to place an information dot at a virtual point as shown in (a-2).

In Fig. 5(b), information dots are arranged in a total of four virtual areas of two rows and two columns, but since placing information dots near the boundaries can cause misrecognition, Fig. 5(c) shows an example in which adjacent virtual areas are arranged at a fixed interval.

In addition, in FIGS. 5B to 5C, it is possible to increase the amount of information by arranging a plurality of information dots in the virtual area, or by not arranging any information dots.
(Code assignment of information dots in FIG. 6)
The code allocation for the information dots is as shown in FIGS.

In other words, as shown in FIG. 6(a), all of the data may be assigned to a "code value" that means information other than position information, such as a company code or voice information, or as shown in FIG. 6(b), the data may be assigned to two data areas, "X coordinate value" and "Y coordinate value", that mean position information as one code format, or as shown in FIG. 6(c), the data areas may be assigned to "code value", "X coordinate value", and "Y coordinate value". When assigning coordinate values to a rectangular area, the data areas for "X coordinate value" and "Y coordinate value" may be different in order to reduce the amount of data. Furthermore, although not shown, a "Z coordinate value" may be further assigned to define the height in the position coordinates. Note that when "X coordinate value" and "Y coordinate value" are assigned, the coordinate values are incremented by a predetermined amount in the + direction of the X and Y coordinates due to position information, so all dot patterns will no longer be the same.

(First example ("GRID0"), Figures 7 to 9)
7 to 9 are diagrams illustrating a first example of a dot pattern.

The applicant has tentatively named the first example of the dot pattern "GRID0."

The feature of "GRID0" is that it uses reference dots placed at offset positions, making it possible to recognize at least one of the range and direction of the dot pattern.

As shown in Figures 7 to 9, "GRID0" has the following configuration:

(1) Information Dots Information dots are used to store information.

The arrangement of the information dots is as shown in Figures 5(a) to (c), and the allocation of codes to the information dots is as shown in Figures 6(a) to (c).

(2) Reference Dots The reference dots are arranged at a plurality of preset positions and are used to specify the positions of virtual points or virtual areas, which will be described later.

In addition, at least one of the reference dots is positioned at a position that is shifted from the positions of the other reference dots. This reference dot is sometimes called a "directional reference dot."

Although not shown, the reference dots placed in the shifted position may be placed in addition to the reference dots placed in the original position.

The reference dot placed at a shifted position specifies the reference direction of the reference dot and the information dot relative to the virtual point, or the reference dot and the information dot placed in the virtual area. By determining this reference direction, it becomes possible to provide and read information in the direction of the information dot relative to the virtual point. It is also possible to specify the range of a dot pattern that defines one piece of data with multiple information dots. This makes it possible to read the range of the dot pattern and decode the data, even if the dot pattern is arranged vertically and horizontally.

(3) Virtual Point or Virtual Region A virtual point or virtual region is specified by the arrangement of reference dots.

Figure 7 shows general examples of the dot pattern for "GRID0", where (a) shows an example where the reference dots are arranged in a roughly plus character shape, and (b) shows an example where the number of information dots arranged is increased.

Figure 8 shows modified dot patterns for "GRID0", where (a) shows an example where the reference dots are arranged in a roughly square shape, (b) shows an example where the reference dots are arranged in a roughly L shape, and (c) shows an example where the reference dots are arranged in a parallelogram.

In FIG. 1A, the reference dots are arranged in the horizontal direction (first direction) and vertical direction (second direction). The center point of the lattice area surrounded by the four reference dots is set as a virtual point, and the information dots are arranged based on at least one of the distance and direction from the virtual point.

Here, the reference dots placed at the four corners of a dot pattern are shifted to indicate the direction and range of the dot pattern. The reference dots, including the shifted reference dots, are placed at equal intervals in the horizontal direction.

In FIG. 1B, the reference dots are arranged in the horizontal direction (first direction) and vertical direction (second direction). The intersection of a vertical line virtually set from the reference dot in the horizontal direction and a horizontal line virtually set from the reference dot in the vertical direction is set as a virtual point, and the information dots are arranged based on at least one of the distance and direction from the virtual point.

Here, one of the vertically aligned dots is shifted to the right to indicate the orientation of the dot pattern. The horizontally aligned reference dots are equally spaced horizontally, and the vertically aligned reference dots are equally spaced vertically.

The angle at which the first direction and the second direction intersect is not limited to 90 degrees as shown in the figure, but may be another specified angle such as 45 degrees or 60 degrees.

Figure 3C shows an example where the angle at which the first direction and second direction intersect is other than 90 degrees. Reference dots are placed horizontally (first direction) and diagonally (second direction). The center point of the lattice area surrounded by the four reference dots is set as a virtual point, and information dots are placed at least one distance and direction from the virtual point.

Figure 9 shows an example of connecting multiple dot patterns, such as the dot pattern in Figure 8(a), in which reference dots are arranged in a roughly square shape, adjacent to each other so that some of the reference dots are common. The condition for connecting is that the dots at both the top and bottom and/or left and right ends of a dot pattern must be in the same position. Note that connecting only vertically or horizontally is also possible. Although not shown, the dot patterns in Figures 8(b) and (c) can also be connected or joined vertically and/or horizontally.

Furthermore, although not shown in the figure, there is also a method in which a dot pattern in which reference dots are arranged in a roughly rectangular shape is arranged vertically and horizontally at a specified interval, i.e., two or more dots are arranged independently vertically and horizontally.

The dot patterns in Figures 8(a) and (c) and Figure 9 are also tentatively called "GRID1."

(Second Example ("GRID5"))
The second example of the dot pattern is tentatively named "GRID5" by the applicant.

"GRID5" replaces the reference dots placed at shifted positions in "GRID0" in that it allows the range and direction of a dot pattern to be recognized by the "way the reference dots are placed." In order to recognize the direction of a dot pattern using the "way the reference dots are placed," the placement of the reference dots must be asymmetrical around an axis, so that no matter how much it is rotated (except 360°) around any point, it will not become the same as the placement before rotation. Furthermore, it is necessary to be able to recognize the range and direction of a dot pattern even if multiple dot patterns are arranged vertically and/or horizontally and connected or linked together.

In addition, "GRID5" uses pattern recognition to recognize the direction of the dot pattern. In other words, the shape of the dot pattern formed by the reference dots is stored in a storage means. Then, the direction of the dot pattern can be determined by comparing the image of the read dot pattern with the shape stored in the storage means.

Figure 10 shows general-purpose examples of "GRID5" dot patterns, where (a) shows an example where the reference dots are arranged in a roughly house shape that is asymmetric in the vertical direction, (b) shows an example where the reference dots are arranged in a roughly cross shape that is asymmetric in the vertical direction, and (c) shows an example where the reference dots are arranged in a roughly isosceles triangle that is asymmetric in the vertical direction.

In "GRID5," the reference dots can be arranged in any way, as long as the arrangement of dots allows for pattern recognition.

(Dot Pattern Analysis)
Next, the procedure for analyzing and decode the read dot pattern tentatively named "GRID1" (hereinafter referred to as "GRID1 dot pattern") among the above-mentioned dot patterns will be described with reference to Figures 11 to 19.

When the imaging unit 2018 captures the dot pattern, the dot pattern is analyzed by the information processing unit 2015 inside the smartphone.

Figure 11 is a flowchart explaining the analysis of dot patterns.

First, the information processing unit 2015 binarizes the dot pattern by image processing (step S00). A dot can be expressed by the number of pixels, and in this process, only images in which the number of pixels falls within the minimum to maximum number are determined as dots. The minimum number is preferably about 2 to 4, and the maximum number is preferably about 15 to 30. The minimum and maximum numbers of pixels determined as dots are highly dependent on the resolution of the camera used to take the image and the area captured at that resolution, so it is desirable to increase or decrease them appropriately depending on the resolution and area captured by the camera. In other words, if the resolution of the camera is high, the minimum and maximum numbers of pixels are increased, and even if the area captured is narrowed by close-up or enlargement, it is desirable to increase the minimum and maximum numbers of pixels in the same way. The area captured by the camera is based on the distance from the subject and the zoom function of the camera. In addition, when capturing an icon 1 with a dot, it is desirable to provide a guide (frame) for capturing the icon 1 with a dot on the display of the smartphone. If the size of the dotted icon 1 changes, the guide (frame) or the dotted icon 1 may be enlarged or reduced so that the dotted icon 1 fits within the guide (frame). Although it is desirable to perform the analysis on the original image (photographed image), image processing may be performed to eliminate missing dots by removing motion blur, increasing contrast, or making the edges of dots clear.

Next, a search start point is set (step S01). FIG. 12 is a diagram showing a search start point set in a binarized captured image of a GRID1 dot pattern. Horizontal and vertical lines do not exist in the actual captured image. The center point of the captured image is set as the search start point (reference dot search start point). Note that the search start point may be a point other than the center point. Also, the search start point is not limited to a fixed point, and one or more search points may be set arbitrarily.

Next, as shown in FIG. 13, at least six dots closest to the search starting point are searched for (step S02). Six dots are selected first because the dot pattern arrangement rules dictate that at least one reference dot is included. In FIG. 13, the dots are numbered 1 through 6 in order of proximity to the search starting point. Below, the dot numbered 1 will be called "dot 1," the dot numbered 2 "dot 2," and so on.

Next, the positional relationship of each dot to other dots is determined (step S03).

Specifically, as shown in FIG. 14, first, the dot (dot 1) closest to the search point among the six dots found is used as a reference, and the distance (interval (1)) from dot 1 to the closest dot is found (FIG. 14(a)), and a search is performed to see if there is a dot at the same distance (interval (2)) in the opposite direction (FIG. 14(b)). In FIG. 14, there is no dot at a position spaced apart by interval (2) from dot 1, and it is not possible to search for dots spaced apart at equal intervals. In that case, a dummy dot, which is a virtual dot, is placed at that position. Next, as shown in FIG. 14(c), the dot closest to dot 1 and the dummy dot are set as the ends, and a search is performed to see if there is a dot at the same interval (interval (3), (4)) on the extension of both ends as the same distance from these ends to the reference dot. In FIG. 14, there is no dot at a position spaced apart by interval (3) or (4) from the ends. Therefore, since three or more dots are not lined up at equal intervals on the line generated by intervals (1) to (4) in FIG. 14, it is determined that there is no reference dot in this arrangement.

Note that dummy dots are placed only once, when the interval (2) is calculated, but in other embodiments, dummy dots may be placed when the intervals (3) and (4) are calculated. With regard to the placement of dummy dots, in the GRID1 dot pattern, the reference dots are shifted to place the directional reference dots, so there will always be a position on the line of reference dots where no reference dot is placed.

Figure 15 is a diagram for explaining the same processing as that explained in Figures 14 (a) to (c) for the second closest dot from dot 1. If there is no reference dot in the arrangement formed by dot 1 and the dot closest to dot 1, the same search processing is performed for the second closest dot from dot 1 as was performed for the dot closest to dot 1. In Figure 15, five dots are not lined up in a straight line at equal intervals on the line generated by intervals (1) to (4). Therefore, the arrangement formed by dot 1 and the second closest dot from dot 1 is not an arrangement in which a reference dot is placed.

For dot 1, the same process is performed from dot 1 to the sixth closest dot. In this embodiment, since dot 1 is a key dot, even if the search process is performed from dot 1 to the sixth closest dot, the five dots will not be lined up in a straight line with equal intervals. In that case, the search process for dot 1 is terminated.

In this way, if the above process fails to find equally spaced dots on a straight line (this may include a predetermined number of imaginary points), in other words, if the closest dot of the at least six closest dots to the above-mentioned search start point is not the reference dot, the same operation is performed using the second closest dot to the search point as the reference dot to decode the dot code. If equally spaced dots on a straight line still cannot be found, the same operation can be repeated up to six dots, starting with the dots closest to the search point as the reference.

Figure 16 shows the results of a search process for dot 6, which is a rare example where dots 1 to 5 were not the reference dots. As shown in Figure 16 (a), for dot 6, a line is searched for that is generated by intervals (1) to (4), in which five dots, including a dummy dot, are lined up in a straight line at roughly equal intervals. As a result, the equally spaced dots excluding dot 6 and the dummy dots are recognized as reference dots (step S04).

Although not shown in the figure, if equally spaced dots are not found on one of the extensions of both ends, a search can be made on the opposite side for dots that are equally spaced (spacing (4)') from the dot at the end and the dot one position inward.

Next, as shown in Figure (b), a search is made to see if there are any equally spaced dots on straight lines perpendicular to each of the five dots found. If these straight lines are drawn virtually, they will form grid lines, with reference dots located at the intersections (grid points) of the grid lines, and dots are searched for in a grid pattern. If equally spaced dots cannot be found, a virtual point is placed at that position, and a search is made for equally spaced dots from the virtual point. As this process continues, grid lines are drawn horizontally and vertically in a grid pattern at equal intervals from the five dots initially found, and the reference dot is searched for.

Next, a dot that is shifted a specified distance on the grid line from the virtual point is searched for (step S05).

As shown in FIG. 17, a dot exists at a position shifted by a certain distance from the virtual point. This shifted dot is a direction reference dot, which indicates the arrangement direction of the dot pattern and is an extremely important dot that defines the arrangement relationship of each information dot (information dot index: ID) and the numerical information of the information dot. The direction reference dot is arranged at every four lattice points vertically and horizontally. Specifically, as shown in FIG. 17, an area (area composed of 4×4 lattices) composed of 5×5 reference dots surrounded by a direction reference dot (a dot surrounded by a circle in the figure) that recognizes the direction and range of the dot pattern is judged to be one dot pattern, and the direction and range of the dot pattern are judged. The arrangement of the direction reference dots may be determined arbitrarily, and the arrangement interval of the direction reference dots may be changed depending on the size (amount of information) of the dot pattern.

Next, the information dot is searched for. The center of the four reference dots (one of which may be the virtual point mentioned above) on the lattice point is set as a virtual point, and a search is made to see if a dot exists at a specified distance from the virtual point in any of the four directions, up, down, left, right, and diagonally, relative to the arrangement direction of the dot pattern. The numerical information defined for the information dot can be obtained based on the position of the searched dot (step S06).

Specific examples are shown in Figures 18 and 19. Figure 18(a) is a diagram showing the correspondence between the arrangement of information dots and numerical information, Figure 18(b) is a diagram showing one dot pattern when area A in Figure 17 is photographed, and Figure 18(c) is a diagram showing the ID and numerical information assigned to each dot. In this diagram, one information dot defines 3-bit numerical information, and is decoded into a 48-bit dot code of 101101000111001110011100100001110011111010101001 by arranging them in the order of ID ₁₆ to ID ₁ .

Figure 19(a) is a diagram showing IDs and numerical information when area B in Figure 17 is photographed, and Figure 19(b) is a diagram showing IDs and numerical information when area C in Figure 17 is photographed. In Figures 18(b) and 18(c), an area with four direction reference dots at the four corners is detected as one dot pattern. However, the detection of one dot pattern is not limited to this. Figure 19(a) is an example of detecting an area having 16 information dots centered around one direction reference dot as one dot pattern, and Figure 19(b) is an example of detecting an area having two direction reference dots at the left and right ends as one dot pattern. In these cases, the IDs are not arranged in the order of ID1 to ID16. However, if an area where 16 information dots are arranged is searched for based on the direction reference dot, one dot pattern including all of ID1 to ID16 can be detected. Then, from this dot pattern, the same dot code as in Figure 18 is decoded.

In this way, in the dot pattern of the present invention, the range and direction of the dot pattern can be recognized by the direction reference dot. Therefore, if a certain number of information dots, including the key dot, can be obtained, the dot code can be accurately decoded regardless of the arrangement of those information dots.

The search for reference dots, direction reference dots, and information dots is performed by determining whether the dot exists within a bounding box, which is a circle or square of a specified size centered on the placement position of each dot. This is because the central coordinate value of the dot will shift due to printing and binarization precision, and distortion that occurs when the dot pattern is photographed in a state that is not parallel to the medium surface. In other words, the presence or absence of a dot is searched for using a bounding box of a specified size so that the search can be performed even if the central coordinate value of the dot is slightly shifted.

[Creating dotted icon 1]
Next, the generation of the dotted icon 1 will be described with reference to FIG.

First, dot pattern data is created (step S10). As explained in the above [Explanation of dot pattern], a dot code containing numerical information is encoded using the dot pattern encoding algorithm to create the dot pattern data.

Next, image data for the icon is created (step S11).

Next, data for dotted icon 1 is created (step S12). The dot pattern data and the icon image data are superimposed to create data for dotted icon 1. At this time, in the area where the dot pattern and the icon image overlap, they may be left as they are, or processing may be performed to remove the dots in order to make the icon image clearer.

The processing of steps S10 to S12 is performed on a personal computer, tablet terminal, smartphone, etc.

Next, the dotted icon 1 is output (step S13). Specifically, the dotted icon 1 is printed using a printer, or the data of the dotted icon 1 is displayed on a user's display. Note that the dotted icon 1 may be printed using various commercial printing machines, or may be engraved on a medium.

In FIG. 20, the dot pattern data is created before the icon data is created, but the present invention is not limited to this. The icon data may be created before the dot pattern data is created. Also, both may be created at the same time.

[About the dot pattern analysis of dotted icon 1]
The dotted icon 1 of the present invention may be realized using any of the dot patterns described above. However, as described below, it is preferable to use the GRID1 dot pattern because the direction and range of the dot pattern are recognized using the direction reference dot and the numerical information of different areas is synthesized.

The procedure for analyzing the dot pattern printed in the icon area (around and near the icon) will be described using Figures 21 to 23. Note that the configuration, technology, and procedure that have already been described will not be described.

Figure 21 is a flowchart showing the process from reading dot icon 1 to outputting information.

First, the user starts an application on the smartphone (step S20). The user downloads an application for reading and decoding the dot code onto his or her smartphone. The user then starts the application when reading the dot pattern.

Next, the user reads the dot pattern printed on the icon with a smartphone (step S21). As shown in FIG. 23, the dot pattern and an icon image (an image of a wine glass in this figure) are printed superimposed on the icon. The user photographs the dot pattern by focusing on the icon image manually or using the smartphone's autofocus function. Note that if a thick frame is provided around the icon as in FIGS. 1 and 4, the focus may be set on the thick frame.

At least one of the dots constituting the dot pattern may be of such a size or shape that the smartphone cannot adjust the focus within a predetermined time. On the other hand, the focus image must be of such a size and color that the smartphone can adjust the focus within a predetermined time. When the focus image is focused, the dot pattern formed on the same surface as the focus image if the medium is two-dimensional, or on the same surface or approximately the same surface as the focus image if the medium is three-dimensional, also comes into focus at the same time. The predetermined time is preferably within 0.4 seconds. Since it is said that humans feel a "pause" in about 0.5 seconds, adding about 0.1 seconds, which is the time required to decode the dot code, display the information corresponding to the dot code, and process the information, the time required from focusing to processing the information is about 0.5 seconds. Note that it may take 1 to 2 seconds if the image displayed during the waiting time is changed so that the user does not feel a "pause."

More specifically, in a dot pattern, the size of the dots and the spacing between the dots can be determined by various values depending on the purpose of the dot pattern, the area on which the dot pattern can be printed, etc., but when reading with a dedicated infrared camera, the size of the dots is about 0.04 mm (600 dpi, 1 pixel), the spacing between reference dots is about 0.5 mm (600 dpi, 12 pixels), and the size of one dot pattern is about 2 mm x 2 mm = ⁴ mm2. In contrast, in the present invention, as described above, a smartphone or the like is brought close to the image to take a picture. The resolution of cameras such as smartphones is increasing year by year, and even such fine dots can be accurately recognized and decoded into a dot code. However, with older generations of smartphones and low-cost smartphones, the resolution of the camera is low and the dots are often too fine to be recognized. Therefore, it is desirable to use a dot pattern in which the dot size is 1.5 to 2.0 times larger than in the above case. That is, by setting the size of each dot to a diameter of about 0.06 mm (400 dpi, 1 pixel) to 0.08 mm (600 dpi, 2×2 pixels or 300 dpi, 1 pixel), it becomes possible to recognize the dot pattern even if the image is slightly out of focus or blurred due to camera shake when enlarged. The interval between the reference dots may be 0.5 mm or more, taking into consideration the visual effect. Furthermore, when the dot pattern is printed not on paper but on a poster, signboard, box, etc. and photographed from a distance with a smartphone, the size and interval of the dots must be appropriately large. In other words, if the dots in the photographed image are too small, they cannot be recognized as dots, so the size and interval of the dots must be such that the image of the dot pattern photographed with a smartphone can be decoded into a dot code.

In the present invention, the dot pattern may be formed from a material that absorbs infrared rays, and the dot pattern may be photographed with a dedicated infrared camera. In the future, it is possible that smartphones will be able to read light other than visible light, such as infrared rays and ultraviolet rays. In that case, the dot pattern may be formed from a material that absorbs wavelengths other than visible light. The material that absorbs wavelengths other than visible light may be transparent. By using a transparent material, it is possible to make the dot pattern less visible, even if the dots are relatively large.

Next, the information processing unit 2015 of the smartphone analyzes the dot pattern (step S22). When the user photographs the icon, the area shown by the rectangle in FIG. 23 is read, and the photographed image is recorded in the memory unit 2016 inside the smartphone. In the analysis by the information processing unit 2015, a binarized image is generated from the recorded photographed image, and the central coordinate value of each dot is calculated.

Next, the information processing unit 2015 performs pattern analysis based on the coordinate values of the centers of the dots that make up the dot pattern, obtains the numerical information defined in the information dots, and decodes the dot code (step S23).

Next, information corresponding to the dot code is output or an operation instruction is given (step S24). For example, information may be output by outputting sound from the smartphone's speaker or displaying content such as images or text on the display. For example, any operation may be given as an operation instruction, such as enabling audio recording from a microphone or enabling a GPS function or wireless function. Information corresponding to the dot code may be output or an operation instruction may be assigned to the dot code read by the smartphone by selecting information or an operation instruction stored in the smartphone or cloud by touch operation or the like. Alternatively, information or an operation instruction stored in the smartphone or cloud may be selected by touch operation or the like and assigned to a predetermined dot code (dot icon 1) that has been registered in advance. The above assignment may be performed by starting a dedicated app, or the function may be built into the dot icon 1 reader.

Note that a browser may be used instead of an application. In that case, the captured image may be sent to a server, where it may be binarized and decoded into a dot code.

Figure 22 is a flowchart showing the details of the dot pattern analysis process in step S22.

The dot pattern is analyzed by the information processing unit 2015 inside the smartphone.

First, the information processing unit 2015 binarizes the dot pattern through image processing (step S30).

Now, let us refer to FIG. 24. FIG. 24 is a diagram showing the result of binarizing dots from a photographed image. The part where the icon image is printed becomes a void area (unusable area) where dots are not recognized. The icon image is deleted when it is judged as white because it is smaller than the threshold value during binarization, or judged as black because it is larger than the threshold value. If it is judged as black, it is recognized as a huge mass of dots, so it can be determined that it is not a dot and can be deleted. In other words, this is the case when the number of pixels that make up the image exceeds the maximum predetermined number of pixels that make up a dot. It is also possible to use image processing or AI (Artificial Intelligence) to determine whether it is an icon image and delete it. Note that the border of the wine glass is drawn to clearly show the void area, and does not exist in the actual binarized image.

Next, the reading area is divided (step S31), and a search start point is set (step S32). Specifically, as shown in FIG. 25, the binarized dot image is divided into nine areas, and the center point of each area is set as the search start point (reference dot search start point). In other words, search start points 1 to 9 are set. In the binarized image of this embodiment, there are areas where dots are missing and dots cannot be searched for. For this reason, the image is divided into multiple areas and many search start points are set.

The number and shapes of the divided areas are not limited to those shown in FIG. 25. Furthermore, the search points are not limited to fixed points, and one or more search points may be set by searching areas where no graphics are superimposed. Furthermore, in this embodiment, the number of information dots is 16, but the search points may be increased as appropriate until all 16 information dots are searched.

Next, at least six dots closest to each search start point are searched for, and the reference dot and the shifted dot are searched for (step S33). Of the nine search start points, first, for search start point 1, the reference dot and the shifted dot are searched for and grid lines are drawn using the method described in steps S03 to S05 of FIG. 11. In this embodiment, there are areas where dots are missing, but even if one dot pattern forming one dot code is partially cut out, the algorithm is configured to be able to fill in the missing information with information dots in other areas. As described above, originally, 5 x 5 = 25 reference dots including at least one directional reference dot are required, but if 2 x 2 = 4 reference dots including the directional reference dot (which may include a virtual point set when searching for the directional reference dot) can be searched for, at least one information dot can be searched for, and the arrangement relationship (information dot index: ID) and numerical information of those information dots can be obtained by the directional reference dot.

When there are no more grid lines that can be drawn from search start point 1, the same process is carried out for search start point 2 and onwards. When all non-missing reference dots and shifted dots have been found, the search process ends.

Next, we will explain the algorithm according to the present invention that can supplement missing information with information dots from other areas even if a dot pattern that forms a dot code is partially cut out. That is, as shown in Figure 26, numerical information of information dots that can be recognized from the binary image is obtained (step S34). Information cannot be obtained from areas where dots are missing. In other words, numerical information of information dots can only be obtained from at least one or more directional reference dots and from locations where neither reference dots nor information dots are missing. Dots indicated by numbers in circles are information dots from which numerical information can be obtained.

Next, as shown in FIG. 27, a virtual reference dot is set (step S35) to determine the arrangement of the information dots that could not be recognized above. In order to decode the dot code, it is necessary to recognize the arrangement of the information dots and obtain numerical information based on the orientation of the dot pattern and the four reference dots surrounding the information dot. In the area where the reference dots are missing, the reference dots are arranged at equal intervals in a lattice pattern, and at least one directional reference dot is arranged, so a virtual reference dot is set as a virtual reference dot based on the positional relationship of the reference dots already searched. In FIG. 27, the x marks are virtual reference dots. The virtual reference dots allow the numerical information of the information dots that could not be obtained in step S34 to be obtained. The information dots surrounded by thick circles are information dots for which numerical information can be obtained by setting the virtual reference dots. The virtual reference dots can be set by performing linear interpolation using the increment value (differential coefficient) of the distance between the searched reference dots, or by determining a natural coordinate system from the deformed lattice and arranging the information dots in the natural coordinate system. Of course, they may be set by other methods.

Next, the numerical information is synthesized (steps S36-37). Since the binarized image has void areas, there are areas where information dots are missing in one dot pattern. Therefore, numerical information of the information dots is obtained from multiple areas (step S36). In FIG. 27, the numerical information of the information dots (9), (12), (13), and (14) cannot be obtained in the lower left dot pattern, and the numerical information of the information dots (1), (2), (4), and (5) cannot be obtained in the lower right dot pattern. Therefore, as shown in FIG. 28, the information dots (1) to (8) in the lower left dot pattern and the information dots (9) to (16) in the lower right dot pattern (dots surrounded by dotted lines) are synthesized to obtain all numerical information of (1) to (16) (step S37). Note that this numerical information is obtained from the arrangement of the information dots relative to the direction indicated by the direction reference dot. In other words, unless at least one direction reference dot is arranged, the numerical information itself cannot be obtained, and the information dot number cannot be specified. The number of the information dot is determined from the arrangement of the direction reference dots. In FIG. 28, numbers are omitted for information dots that are not used in the synthesis.

Finally, the dot pattern is decoded into a dot code based on the synthesized numerical information (step S38). The procedure for decoding the dot code is as described in FIG. 18.

In this embodiment, information dots are searched for for all four dot patterns contained in the icon, but the present invention is not limited to this, and the search may be stopped once the numerical information of all information dots (1) to (16) has been obtained. As described above, in FIG. 28, the numerical information of all information dots can be obtained using only the lower left dot pattern and the lower right dot pattern, and the numerical information is synthesized using only two dot patterns. In such a case, it is possible to search only the lower left dot pattern and the lower right dot pattern.

However, by searching all dot patterns, it is possible to check whether information dots with the same number have the same numerical information, providing a powerful error checking function.

Second Embodiment
Next, a second embodiment of the present invention will be described with reference to Figures 29 to 34. Note that configurations, techniques, and procedures that are the same as those in the first embodiment will be omitted.

The second embodiment is characterized in that the dot icon 1 is a specific image, and the specific image is used to realize Augmented Reality (AR). Augmented reality is a technology that displays a virtual world overlaid on the real world, and for example, outputs content that adds digital information to real-world images.

Figure 29 is an explanatory diagram showing an example configuration of an augmented reality system using a specific image according to the second embodiment of the present invention.

The system shown in FIG. 29 includes a medium 30 on which an icon with a dot pattern is printed, and a reading device 20 (smartphone) that captures the dot pattern.

In this embodiment, the medium 30 is a chocolate product package. Although not shown, a QR code (registered trademark) that allows an augmented reality program to be downloaded may be printed on the product package. If the user does not own the augmented reality program, the user photographs the QR code with a smartphone and downloads the augmented reality program. Of course, the method of downloading the augmented reality program is not limited to this.

A rabbit icon is printed approximately in the center of the product package. A dot pattern is printed over the rabbit icon to form dot icon 1. The user photographs dot icon 1 with an augmented reality program running on their smartphone.

Figure 30 is a table showing the correspondence between dot codes and the screens that are displayed, and Figure 31 is an example of a screen that is displayed on the smartphone display after photographing an icon.

When an augmented reality program is downloaded, a table like that shown in Figure 30 is also downloaded at the same time and stored in the smartphone's memory. For example, if the dot code is "10101," the corresponding content is "video of a rabbit jumping," and if the dot code is "10110," the corresponding content is "video of a rabbit running." The table may register addresses where the "video of a rabbit jumping" or "video of a rabbit running" is stored, or it may be a command to move the 3DCG in that way.

The table may be stored on a server instead of on the smartphone. Also, the correspondence between the dot code and the content may be established by other methods than the table.

When a user photographs an icon with the smartphone camera while the augmented reality program is running, the smartphone reads the dot pattern printed and superimposed on the icon, analyzes the dot pattern according to the procedure explained in Figures 22 to 28, and obtains numerical information (dot code).The smartphone then refers to the table in Figure 30 and displays a 3DCG image associated with the dot code on the display, superimposed on the actual image (the chocolate product packaging) captured by the camera.

Figure 31 (a) shows a 3DCG image of a rabbit hopping displayed on a display, in which the dot code defined in the read dot pattern is "10101." Figure 31 (b) shows a 3DCG image of a rabbit running displayed on a display, in which the dot code defined in the read dot pattern is "10110."

The content formed by augmented reality processing (augmented processing) is not limited to 3DCG, but may be a combination of two-dimensional images, text, audio, and multiple contents. Furthermore, regardless of the term augmented reality, it includes any processing that adds, deletes, emphasizes, or attenuates information from an icon.

In addition, the entire dot code defined by one dot pattern may be used as a code value that identifies the content, or only a portion of the dot code may be used as a code value that identifies the content.

Conventional AR involves reading an AR marker and outputting content that corresponds to the AR marker. However, this method only allows one piece of content to be output for one type of AR marker. In contrast, the present invention makes it possible to output multiple different pieces of content using the same specific image by printing different dot patterns on each of the same specific images (corresponding to AR markers).

Figure 32 is a flowchart showing the augmented reality processing according to the present invention.

First, the user launches an application program for augmented reality (step S70). Then, the user uses the smartphone camera to read the icon image (dot icon 1) on which the dot pattern is printed (step S71).

Next, image recognition processing is performed (step S72). The smartphone's information processing unit 2015 performs image recognition processing on the scanned image to detect an icon image (step S73).

Next, the icon image is analyzed (step S74). The information processing unit 2015 analyzes the icon image to recognize the icon pattern. Then, the position and orientation of the icon in real space are detected (step S75).

The information processing unit 2015 further detects a dot pattern by image recognition processing (step S76). Then, the information processing unit 2015 analyzes the dot pattern (step S77) and decodes the dot pattern into a dot code (step S78). The specific processing for analyzing and decoding the dot pattern is as shown in Figures 22 to 28.

The icon image detection and dot pattern detection may be performed simultaneously, or one of the processes may be performed first.

Next, the information processing unit 2015 performs an augmentation process based on the position and orientation of the icon detected in step S75 and the dot code decoded in step S78 (step S79). First, the information processing unit 2015 refers to the table described in FIG. 30 to recognize the process corresponding to the dot code. Then, the content (augmented reality image) to be synthesized in the real image (real space) is controlled according to the position and orientation of the icon detected in step S75, i.e., to have a predetermined positional relationship with the icon. For example, when the dot code is 10101 in FIG. 30, the position and inclination of the 3DCG of the rabbit jumping are controlled according to the position and orientation of the rabbit pattern. Note that instead of matching the position and orientation of the icon, the augmented reality image may be synthesized according to the position of the medium surface on which the icon is printed, i.e., to have a predetermined positional relationship with the medium surface.

Next, the information processing unit 2015 displays the augmented reality image generated by the augmentation process on the display (step S80). Then, this process ends (step S81).

In general, in the extension process, an AR marker is photographed with a camera, and the AR marker is recognized to output content associated with the AR marker. However, in order to recognize an AR marker, many feature points are required, and it may be difficult to accurately recognize the shape and image of the AR marker. In contrast, in the present invention, content associated with an icon is recognized using a dot pattern. This makes it possible to output content more accurately and easily than when an AR marker is used. Furthermore, as described above, by forming different dot patterns and defining different code values even for the same icon, it is possible to output different content corresponding to each code value.

By defining coordinate values for the dot pattern, an augmented reality image may be synthesized so as to have a predetermined positional relationship with the medium surface, corresponding to position information based on the coordinate values defined in the dot pattern on the medium surface printed without using an AR marker. In this case, the coordinate values defined in the dot pattern, or the coordinate values and code values, act as the AR marker.

<Modification of the second embodiment>
Next, a modification of the second embodiment will be described.

This embodiment is characterized in that the dot pattern is further defined with coordinate values.

Figure 33 is a flowchart showing the augmented reality processing in this embodiment.

First, the user launches an application program for augmented reality (step S90). Then, the user uses the smartphone camera to read the icon image (dot icon 1) on which the dot pattern is printed (step S91).

Next, image recognition processing is performed (step S92). The information processing unit 2015 of the smartphone detects the dot pattern (step S93). The smartphone then analyzes the dot pattern (step S94) and decodes the dot pattern into a dot code (step S95). The specific processing for analyzing and decoding the dot pattern is as shown in Figures 22 to 28.

Next, the information processing unit 2015 detects the deformation state of the dot pattern (step S96). From the relationship between the detected dots and the tilt of the camera, the orientation and deformation state of the dot pattern are detected. Then, the information processing unit 2015 performs an extension process based on the deformation state of the dot pattern and the dot code (step S97). First, the information processing unit 2015 refers to the table described in FIG. 30 to recognize the content corresponding to the dot code. In addition to the content, the dot code also has a defined coordinate value. From this coordinate value, the arrangement position of the icon on the medium is recognized. Then, from the orientation and deformation state of the dot pattern detected in step S96, the tilt angle and/or rotation angle between the camera's shooting direction and the icon is obtained, and for example, when the dot code is 10101 in FIG. 30, the tilt, size, etc. of the 3DCG of the rabbit jumping is adjusted.

The position recognized as the starting point of the expansion process may be the center of the dot icon 1, or a predetermined position within the dot icon 1. Also, instead of recognizing the position on the medium where the dot icon 1 is placed, a predetermined position on the surface of the medium may be recognized.

Next, the information processing unit 2015 displays the augmented reality image generated by the augmentation process on the display (step S98). Then, this process ends (step S99).

As mentioned above, many feature points are required to recognize an AR marker. Therefore, complex calculations are required not only to determine the content drawn on the AR marker, but also to determine the position of the AR marker.

As in this embodiment, by defining coordinate values using a dot pattern, it becomes possible to perform the expansion process without requiring complex calculations.

The coordinate values defined in the dot pattern can be made to correspond to coordinate values indicating the medium surface. Specifically, this is as shown in FIG. 34. That is, the coordinate values (x1, y1) to (x2, y2) of the position where the icon is placed are specified using a coordinate system with the bottom left coordinate value of the medium surface as the origin (0, 0).

The coordinate system indicating the medium surface and the coordinate system of the dot code may be different. For example, the dot code defines a Cartesian coordinate system, but the medium surface may be a circular coordinate system or a three-dimensional spherical coordinate system. Moreover, it is also possible to indicate position information of the medium surface regardless of the position where the dot pattern is formed.

In this way, by performing augmented processing using icons printed with dot patterns, it becomes possible to perform processing in a more expansive, faster, and simpler manner than conventional augmented reality, such as using AR markers.

In the embodiment, the dot code is associated with content, but the present invention is not limited to this, and the dot code may be associated with information processing, operation instructions, etc.

<Specific uses>
35 to 46 are diagrams for explaining various applications of the present invention.

Figure 35 is an explanatory diagram of the application of the present invention to a wine list.

Fig. 1A shows a list of red wines. This list may be placed on a restaurant table as a menu, or distributed to customers as a liquor store flyer. A red wine icon (wine glass mark) is drawn on the left side of the list, and the name and price of the red wine are written on the right side. A dot pattern is printed over the wine glass mark. Fig. 1B shows the screen of a smartphone when taking a picture. When a dot icon reader (which may be an application program for augmented reality) that reads the dot code printed over the dot icon 1 is launched and a page for taking pictures is opened, a rectangular frame is displayed from the top to the center of the screen, and an instruction is displayed at the bottom of the screen saying "Please take a picture so that the mark fits within the frame." The user takes a picture of one of the three wine glass marks so that the desired wine glass mark fits within the frame. For example, in Fig. 1B, the top wine glass mark is photographed. The smartphone reads and analyzes the dot pattern printed over the wine glass mark using the method described in Figs. 22 to 28. When the analysis of the dot pattern is completed, the screen of Fig. 1C is displayed. The screen displays information about Cabernet Sauvignon, such as its characteristics and recommended dishes.

Figure 36 is an explanatory diagram of the use of this invention in a product catalog.

Fig. 1(a) is a part of a catalog. In this page, autumn vegetables are listed, and a check mark is drawn in the upper left corner of each autumn vegetable. A dot pattern is printed over the check mark. Fig. 1(b) is the smartphone screen when taking a photo. When the Dot Icon 1 Reader (which may be an application program for augmented reality) is launched and the page for taking photos is opened, a rectangular frame is displayed from the top to the center of the screen, and an instruction is displayed at the bottom of the screen saying "Please take a photo so that the mark fits within the frame." The user takes a photo so that the check mark drawn in the upper left corner of the target vegetable fits within the frame. In Fig. 1(b), the check mark in the upper left corner of a pumpkin is photographed. The smartphone reads and analyzes the dot pattern printed over the check mark using the method described in Figs. 22 to 28. When the analysis of the dot pattern is completed, the screen in Fig. 1(c) is displayed. The screen displays a recipe for cooking using pumpkin. Note that instead of the check mark, a dot code corresponding to each autumn vegetable may be printed over the dot icon 1.

Figure 37 is an explanatory diagram of the use of this invention in an emergency exit.

(a) in the figure is a pictogram of an emergency exit installed on a wall or the like. A dot pattern is printed over the pictogram. (b) in the figure is the screen of a smartphone when taking a picture. When an application program for augmented reality is started and a page for taking pictures is opened, a rectangular frame is displayed from the top to the center of the screen, along with an instruction to "take a picture so that the mark fits within the frame." The user takes a picture so that the pictogram of the emergency exit fits within the frame. The smartphone recognizes the pictogram using the method described in Figs. 32 and 33, and reads and analyzes the dot pattern printed over the pictogram using the method described in Figs. 22 to 28. When the recognition of the pictogram and the analysis of the dot pattern are completed, the image in (c) in the figure is displayed. The image of the corridor is a photographed image of the place where the user is actually located. The dot pattern defines position information (coordinate values) indicating the position of the emergency exit, and based on this position information, a photographed image of the corridor where the user is located is displayed. Then, an arrow and the words "Emergency exit 20m" and "15m left" are displayed over the image of the corridor. The user can evacuate to a safe place by following the arrow and the words.

Generally, all pictogram images indicating emergency exits within a building are the same. In conventional augmented reality, which recognizes and calls up images, one pictogram can only call up one piece of the same information. Therefore, as shown in the figure, by reading and analyzing the dot pattern printed over the pictogram, it is possible to display different information corresponding to each installation location, even for the same pictogram.

In addition, location information may be obtained by GPS instead of or in addition to obtaining it from the dot pattern.

Figure 38 is an explanatory diagram of how the present invention can be used to explain things at an aquarium.

(a) in the figure shows the aquarium. The icon of the professor is installed on the wall under the tank. A dot pattern is printed over the icon. (b) in the figure shows the smartphone screen when taking a picture. When an application program for augmented reality is started and a page for taking pictures is opened, a rectangular frame is displayed from the top to the center of the screen, and an instruction is displayed at the bottom of the screen saying "Please take a picture so that the mark fits within the frame." The user takes a picture so that the icon of the professor fits within the frame. The smartphone recognizes the icon by the method described in Figs. 32 and 33, and reads and analyzes the dot pattern printed over the icon by the method described in Figs. 22 to 28. When the recognition of the icon and the analysis of the dot pattern are completed, the screen shown in (c) in the figure is displayed. The screen of the sea turtle is an image of the aquarium that is displayed when the smartphone is held over the actual position of the user. Images of the sea turtles and explanatory text are displayed in a list or scrolled display, superimposed on the image of the actual aquarium. When the user holds their smartphone over a creature that matches the image displayed, an action such as 'ping pong' may be output and displayed.

In facilities such as aquariums and zoos, explanations of the creatures kept and the exhibits are often posted on the walls. However, when there are a lot of visitors, there are times when a person does not have time to carefully read the explanations. In this embodiment, the user can obtain information about the creatures in the aquarium using their smartphone, making it possible to read the explanations without having to worry about the flow of visitors.

Furthermore, in conventional augmented reality that recognizes and calls up images, one icon can only call up one piece of the same information. By reading and analyzing the dot pattern printed over the icon as shown in the figure, it is possible to display different information corresponding to each location, even if the icon is the same.

In the embodiment of FIG. 37, a gyro sensor provided in the smartphone may be utilized. For example, in FIG. 37, when the user points the smartphone in various directions after reading the pictogram of an emergency exit, the arrow and text displayed will vary depending on the direction. Also, in FIG. 38, when the user points the smartphone at a creature in an aquarium after reading the icon of the professor, the creature may be image-recognized and a description of the creature may be displayed.

Figures 39 and 40 are explanatory diagrams of the use of this invention in a seal.

Figure 39 shows dot icon 1 in the form of a sticker showing a number. In this figure, ten icons numbered 1 to 10 are printed on a medium, and each icon is a sticker that can be peeled off individually. A different dot pattern is printed over each icon.

Figure 40 is an explanatory diagram showing an example of use of Figure 39. A plurality of people own the same sticker. One of the people (here, user A) associates content such as an image or video with the dot code. The association is performed, for example, by downloading a dedicated application to a smartphone. When user A starts the application and takes a picture of one of the dot icons 1 printed on the sticker, the dot pattern printed superimposed on the icon is read. Next, a screen prompting the user to select the content to be associated is displayed on the display, and user A selects the content to be associated with the icon by following the instructions on the screen. When the selection is completed, a table that associates the dot code with the content is generated and transmitted to the cloud. Note that the dot code and the content may be transmitted to the cloud, and the table may be created in the cloud. After that, when other users (here, users B to D) take a picture of the dot icon 1 with the same number as the dot icon 1 associated by user A with their own smartphone, the content associated by user A is displayed on the display of users B to D or output from the speaker. Note that the content may be an image (photo), video, music, etc. It is possible to associate not only a single piece of content (e.g., one photo) with the dot icon 1, but also multiple pieces of content (e.g., multiple photos) together. In this case, the dot icon 1 acts as a folder. Also, when multiple pieces of content are associated with the dot icon 1, a folder may be automatically created on the smartphone. Even if the association between the dot icon 1 and the multiple pieces of content is released, the folder created on the smartphone will be maintained.

When sending and receiving content between friends or acquaintances, email or messaging apps (such as LINE) are often used. However, when the size of an image or video is large, problems arise, such as not being able to send or receive the file, or even if it is possible, the time and effort required. There are services that transfer large files, but there are problems in that the procedures required for uploading and downloading files are complicated, and the download deadline is short and often expires. With the present invention, content can be sent and received with simple operations without worrying about the file size or download deadline.

The stickers are preferably used by sticking them on a diary, notebook, etc. As the number of dot icons 1 associated with content increases, it becomes difficult to know which content is associated with which dot icon 1. Therefore, by sticking a dot icon 1 on a diary or notebook and writing the content details (such as "drinking party" or "visiting a museum") in the empty space next to it, the user can easily access the content they want to view.

The present invention can also be used for social networking services (SNS), in addition to personal content exchanges between friends and acquaintances.

In addition, the association may be not only content, but also various operational instructions.

In this embodiment, the dot icon 1 is a sticker, but the dot icon 1 can be printed on any medium, such as a diary, notebook, calendar, or sticky note.

Figure 41 is an example of a notebook with dot icon 1 printed on it. Next to dot icon 1, there is a memo section and a section for entering dates. For example, at construction sites, it is common to take photos of buildings under construction. However, it is difficult to immediately collect photos taken while working on-site into a folder and give the folder a name. With the notebook of the present invention, photos taken can be immediately collected and associated with dot icon 1, and the content of the photo and the date it was taken can be entered on the spot. This makes it possible to carry out work efficiently.

Figure 42 is an explanatory diagram of how the present invention can be used for authenticity determination and traceability.

Figure (a) shows the outer box and the cosmetic product itself. A dot icon 1 is printed on the outer box. A user (a person who purchases the cosmetic product) takes a picture of the dot icon 1 with their smartphone. If the cosmetic product is genuine, the word "Genuine" is displayed on the display, as shown in Figure (b). Note that the dot icon 1 may also be provided on the cosmetic product itself.

With many products now manufactured in Japan and exported overseas, the damage caused by counterfeit goods has become a serious problem, and technology for determining authenticity is becoming increasingly important. Dot patterns are highly confidential and are already being used to determine the authenticity of products, but because they require a dedicated reading device, they have mostly been used by businesses such as product sellers. With this invention, dot patterns can be read with a smartphone, making it possible for ordinary consumers to determine the authenticity of products as well.

Dot icon 1 may also be used for product traceability at logistics centers.

Here, we explain a technique that further improves the accuracy of authenticity determination in dot patterns.

Figure 43 is an enlarged view of a partial area of a dot pattern that further improves the accuracy of authenticity determination. In the GRID1 dot pattern, information dots are placed based on a virtual point to define information, but in the present invention, two dots are placed from the virtual point. The two dots have different densities, one is printed with the same density as the reference dot, and the other is printed in a lighter color than the reference dot. The light-colored dot is light enough to be read by a smartphone.

Of the two dots, one is a true information dot that defines numerical information, and the other is a dummy dot that does not define information (different from the dummy dots in Figures 14 to 16). Both the dark and light colored dots can be true information dots and dummy dots. The allocation of information dots and dummy dots can differ depending on the virtual point. For example, in the figure, at the virtual point in the upper center and the virtual point in the lower left, the dark dots are dummy dots (dots marked "false") and the light colored dots are information dots (dots marked "true"), but at the other virtual points, the dark dots are information dots and the light colored dots are dummy dots. This allocation is stored in advance in a storage means.

When a dot pattern is photographed with a smartphone, it is decoded into a dot code according to the stored allocation. If the allocation of information dots and dummy dots differs from that stored in advance, it will not be decoded. In this case, a message or audio may be displayed to inform the user that the product is not genuine.

In this way, by providing two dots, a genuine information dot and a dummy dot, it is possible to determine whether the product is genuine or a counterfeit even if the dot pattern is copied exactly as it is, thereby further enhancing the secrecy of the dot pattern.

Figure 44 is an explanatory diagram of the present invention used for product promotion. In Figure 44 (a), a dot icon 1 is printed on a beverage container. When a user (a person who purchases a beverage) takes a picture of the dot icon 1 with their smartphone, a special movie exclusive to the purchaser is displayed on the user's smartphone, as shown in Figure 44 (b). Of course, the special gift given to a user who takes a picture of the dot icon 1 is not limited to a movie. For example, it may be a link to a special website, a coupon, or a stamp for a stamp rally. The dot icon 1 can be set to match the package, so it does not spoil the design or aesthetics of the package. Therefore, it can be used in a wide range of situations where design is important. The services provided may be changed by changing the dot pattern printed on the dot icon 1.

If the beverage container is a transparent plastic bottle, it is preferable to print the dot icon 1 on top of a solid light color. Since there is a possibility that something other than the dot icon 1 may be reflected in a transparent container, this is done to prevent reflections and ensure that the dot pattern can be read accurately.

Figure 45 is an explanatory diagram of the case where dot icon 1 is set on a tourist guide map. Figure 45 (a) shows a tourist guide map distributed in cities, towns, villages, specific regions, etc. On the map, the locations of major facilities in the region, such as hotels, shrines, and museums, are shown as icons. These icons are dot icons 1 with a dot pattern (not shown) superimposed and printed on them. When a user takes a picture of dot icon 1 with their smartphone, information related to the icon is output to the user's smartphone. For example, when a user takes a picture of dot icon 1 of a shrine, an introduction to the shrine is displayed on the user's smartphone (Figure 45 (b)). This makes it possible to provide more detailed information on a general map.

The dot icon 1 may be used not only on maps distributed to each user, but also on guide signs installed in train stations, on roads, etc. When a user photographs a dot icon 1 that displays a facility or the like listed on a guide sign, information about the facility and directions from the guide sign to the facility are displayed on the user's smartphone.

Figure 46 is an explanatory diagram of a case where a dot icon 1 is provided with a raised portion. Figure (a) is an example of a printed matter (medium) on which a dot icon 1 with a raised portion is printed. In addition to the dot icon 1 with a raised portion, the printed matter also has text information and braille corresponding to the text information printed on it. The printed matter in this figure is a guide to major facilities such as tourist spots, and the text information is the name of the facility. The braille in this figure is an image and differs from the actual braille.

Figure (b) is an enlarged view of a dot icon 1 with protrusions. A number of protrusions are arranged on the dot icon 1, and these are arranged to form Braille. In this figure, Braille corresponding to the letters "DI" representing the dot icon are arranged, but this is not limiting. Also, instead of Braille, it may be a specific symbol or line. The protrusions are arranged superimposed on the icon image. This is because the dot pattern of the area where the image is originally printed is not analyzed, and therefore superimposing the protrusions on the icon image does not affect the analysis of the dot pattern.

Figure (c) is an enlarged view showing another example of a dot icon 1 with a protrusion. In this figure, the protrusion is placed along the outline of the dot icon 1. In this way, if the protrusion is placed along the outline of the dot icon 1, the size and range of the dot icon 1 as well as the position of the dot icon 1 can be recognized by touch. Note that in this figure, the protrusion is placed along the entire outline of the dot icon 1, but this is not limited, and protrusions may be placed only on part of the outline, such as the four corners of the dot icon 1. Furthermore, the Braille in Figure (b) may be placed superimposed on the icon image, and the protrusion may be placed along the outline of the dot icon 1.

By providing a convex portion on the dot icon 1, it becomes easier for visually impaired people to image the dot icon 1.

Note that the above uses are merely examples, and the screens displayed on the smartphone and examples of use are of course not limited to those described above.

Although the present invention has been described above, the present invention is not limited to the above-mentioned embodiment. Each of the configurations and processes described above can be implemented alone, and configurations described in different embodiments or for different purposes can also be freely combined and implemented.

For example, an icon may have both the function of serving as a focus image in embodiment 1 and the function of serving as an augmented reality marker in embodiment 2.

The two-dimensional code of the present invention can be used for a wide variety of purposes, including language learning materials, tourist guides, various advertisements, authenticity verification, and many more.

1 Dot Icon 1, Dot Icon 1
10, 30 Medium 20 Reading device (smartphone)
210 Housing 220 Display 230 Operation button 240 Imaging unit 2010 Antenna 2011 Transmitting/receiving unit 2013 Microphone 2015 Information processing unit 2016 Storage unit 2017 Speaker 2018 Imaging unit

Claims (33)

A dot pattern in which dots are arranged according to a predetermined rule,
The dots are
A plurality of reference dots serving as a reference for encoding with the dot pattern and including dots whose orientation can be specified;
information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots;
a plurality of the dot patterns are formed on the medium, and at least a part of an image other than the dots is formed at a predetermined position on the medium, superimposed on or adjacent to the dot patterns;
At least one of the dot patterns includes an undecodable area, and dots corresponding to the undecodable area are complemented by a dot pattern other than the dot pattern including the undecodable area.
Dot pattern. In the unusable area, at least a part of the dot pattern is not printed, or at least a part of the dot pattern is printed in a superimposed manner on the image.
The dot pattern according to claim 1 . A dot pattern in which dots are arranged according to a predetermined rule,
The dots are
A plurality of reference dots serving as a reference for encoding with the dot pattern, the reference dots including dots whose orientation can be specified;
information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots,
a plurality of the dot patterns are formed on the medium, and at least a part of an image other than the dots is formed at a predetermined position on the medium, superimposed on or adjacent to the dot patterns;
The image is a focusing image on which a dot code reading device that reads the dot pattern can be focused. A dot pattern in which dots are arranged according to a predetermined rule,
The dots are
A plurality of reference dots serving as a reference for encoding with the dot pattern, the reference dots including dots whose orientation can be specified;
information dots arranged in a plurality of arrangement areas specified based on the arrangement of the reference dots,
a plurality of the dot patterns are formed on the medium, and at least a part of an image other than the dots is formed at a predetermined position on the medium, superimposed on or adjacent to the dot patterns;
the image includes a specific image that can be recognized by an information processing device that is built into or connected to a dot code reading device that reads the dot pattern,
At least a portion of the dot code encoded in the dot pattern identifies at least one of one or more predetermined information and one or more information processes associated with the specific image;
At least one of the one or more pieces of predetermined information and the one or more pieces of information processing associated with the specific image is formed by performing an extension process in a predetermined positional relationship with the specific image.
Dot pattern. The dot pattern according to claim 4 , wherein the predetermined positional relationship is formed by performing an expansion process based on at least coordinate values included in the dot code. A plurality of dot-like convex portions are provided on, adjacent to, or close to the image.
The dot pattern according to any one of claims 1 to 4. At least a portion of the information dots is made up of a plurality of dots having different densities,
The dots of at least one density are dummy dots that do not define information or information dots that define true information;
The dot pattern according to any one of claims 1 to 4. At least one of the dot patterns includes an undecodable area, and dots corresponding to the undecodable area are complemented by a dot pattern other than the dot pattern including the undecodable area.
The dot pattern according to claim 3 or 4. In the unusable area, at least a part of the dot pattern is not printed, or at least a part of the dot pattern is printed in a superimposed manner on the image.
The dot pattern according to claim 8 . The dot pattern according to claim 1 or claim 4, wherein the image is a focusing image on which a dot code reading device that reads the dot pattern can be focused. the image includes a specific image that can be recognized by an information processing device that is built into or connected to a dot code reading device that reads the dot pattern,
The dot pattern of claim 1 or 3, wherein at least a portion of a dot code encoded in the dot pattern specifies at least one of one or more predetermined pieces of information and one or more information processes associated with the particular image. the focusing image includes at least a part of a specific image, and the specific image is recognized by an information processing device built into or connected to the dot code reading device;
At least a portion of the dot code encoded in the dot pattern identifies at least one of one or more predetermined information and one or more information processes associated with the specific image.
The dot pattern according to claim 3 . The dot pattern according to any one of claims 1 to 4, wherein at least a portion of the dot code encoded in the dot pattern is associated with at least one of predetermined information and information processing. The dot pattern according to claim 12, wherein at least one of the predetermined information and information processing associated with at least a portion of the dot code is formed by an expansion process in a predetermined positional relationship with the medium surface on which the dot code is formed. The dot pattern according to claim 14 , wherein the predetermined positional relationship is formed by performing an expansion process based on at least coordinate values included in the dot code. The dot pattern according to claim 15, wherein the coordinate values correspond to coordinate values indicating the medium surface and can identify a specific position on the medium surface. The dot pattern according to claim 4, wherein the predetermined positional relationship is formed by obtaining an inclination angle and/or rotation angle between the shooting direction of the dot code reading device and the specific image from the orientation and deformation state of the dot pattern photographed by the dot code reading device, and performing an expansion process in the predetermined positional relationship. The dot has at least one of a size and a shape that the dot code reader cannot focus on within a predetermined time.
The focusing image is capable of being focused on by the dot code reading device within the predetermined time.
The dot pattern according to claim 3 . The medium is either a print medium or a display;
The dot pattern according to any one of claims 1 to 4, wherein the dot pattern and the image are printed on the printing medium or displayed on the display. The dots are formed so as to be superimposed on a predetermined background,
The dot pattern according to any one of claims 1 to 4, wherein a color of the dots and a color of the predetermined background are set so that the arrangement of the dots can be obtained by distinguishing it from the predetermined background. The dot pattern according to any one of claims 1 to 4, wherein the dot pattern is formed on the medium with at least two adjacent dots in the vertical and horizontal directions. When at least a part of the reference dot cannot be recognized in the unrecognizable area and the dot cannot be decoded,
2. The dot pattern according to claim 1, wherein a reference dot is virtually placed at the position of the unrecognizable reference dot based on other recognizable reference dots, and the dot is decoded. The medium has a writing area formed adjacent thereto.
The dot pattern according to any one of claims 1 to 4. A means for forming a plurality of dot patterns according to any one of claims 1 to 4 on a medium,
Dot pattern generator. A dot pattern according to any one of claims 1 to 4 and a means for photographing the image;
means for decoding the dot pattern into a dot code;
A dot code reading device comprising: On the computer,
A plurality of dot patterns according to any one of claims 1 to 4 are formed on a medium.
A dot pattern generation program. On the computer,
Photographing the dot pattern according to any one of claims 1 to 4 with a dot code reader;
Decoding the dot pattern into a dot code;
Dot pattern reading program. It is possible to access a table in which the dot code and information specifying a storage destination in which information or an operation instruction is stored are registered in correspondence with each other,
obtaining the storage destination from the table based on the decoded dot code;
outputting the information or executing the operation instruction;
28. The dot pattern reading program according to claim 27. receiving information or an operation instruction to be registered in a table in association with the decoded dot code;
storing the received information or operation instruction in a predetermined storage location;
registering the dot code and information specifying the storage destination in the table;
28. The dot pattern reading program according to claim 27. A plurality of dot patterns according to any one of claims 1 to 4 are formed on a medium.
Dot pattern generation method. Photographing the dot pattern according to any one of claims 1 to 4 and the image,
decoding the dot pattern into a dot code;
Dot pattern reading method. A reading means for photographing the dot pattern according to any one of claims 1 to 4 and decoding the dot pattern into a dot code;
An information processing device including an output unit that outputs at least one of predetermined information and a result of information processing;
a server that receives the dot code and transmits the predetermined information and information processing associated with at least a portion of the dot code to the information processing device;
A dot pattern reading system comprising: an information processing device photographs the dot pattern according to any one of claims 1 to 4, decodes the dot pattern into a dot code, and outputs at least one of predetermined information and a result of information processing;
a server receives the dot code and transmits the predetermined information and information processing associated with at least a portion of the dot code to the information processing device;
Dot pattern reading method.

Images