JP6245914B2 - Identification code reading system and method - Google Patents

Description

  The present invention relates to an identification code reading system and method.

  Conventionally, an identification code such as a one-dimensional code (for example, a barcode) or a two-dimensional code (for example, a QR code (registered trademark)) that can be read as an image is photographed, and information corresponding to the identification code in the photographed image ( That is, an identification code reading system that acquires overlay information) is known. Patent Documents 1 to 3 propose that the acquired information (for example, the name of an article to which an identification code is attached) is superimposed on a captured image and displayed on the screen of the display unit. According to this system, the information acquired in this way is superimposed on the captured image and displayed on the screen of the display unit. Therefore, information about the article with the identification code in the captured image (for example, the article name) Can be grasped on the screen of the display unit.

JP 2012-215989 A JP 2012-181705 A JP 2011-55250 A

On the other hand, there is a variety of advantages if a database in which information on a plurality of articles with identification codes attached is stored in association with each other and information on other articles related to the photographed identification code is obtained. Therefore, it is conceivable to provide the above database in a conventional identification code reading system.
However, the proposed conventional identification code reading system only acquires and displays the information of the first article to which the identification code in the photographed image is attached, and information on the other related second articles. There is no description about displaying. Therefore, even when the first article and the second article associated therewith exist in the photographed image, it is not displayed that the two displayed articles are related. Further, the presence of the related article and the information on the related article are not displayed. Therefore, the proposed conventional identification code reading system has a problem that even if the above-described database is provided, the information in the database cannot be used effectively in the screen display.

  For example, when the first article with the identification code in the photographed image is a storage box and the other article associated therewith is a notebook stored in the storage box, the notebook is stored in the storage box. It is difficult to grasp on the screen of the display unit that it will be. In addition, when a storage box and a note taken out from the storage box are displayed in the image, it is difficult to grasp on the screen of the display section that the notebook is an article to be stored in the storage box. is there.

  An object of the present invention is to enable reading as an image associated with first information relating to the first article when the first article with the first identification code readable as an image is displayed. Another object is to provide an identification code reading system and method capable of easily grasping the second information related to the second article with the second identification code on the screen of the display unit.

  In the identification code reading system according to the present invention, the first information related to the first article to which the first identification code that can be read as an image is attached is the second information to which the second identification code that can be read as an image is attached. A database to be stored in association with the second information relating to the article, imaging means for imaging the first article with the first identification code, and an identification code for recognizing the first identification code in the captured image Recognizing means, information extracting means for accessing the database and extracting from the database the second information associated with the first information relating to the first article with the recognized first identification code in the image; And display means for displaying the second information extracted by the information extraction means superimposed on the image photographed by the photographing means.

  Preferably, when extracting the second information from the database, the information extracting means extracts from the database the first information related to the first article with the recognized first identification code in the image. Alternatively, the first information included in the first identification code is extracted, and the display unit displays the first information and the second information extracted by the information extraction unit superimposed on the image captured by the imaging unit. To do.

Preferably, the database stores the first information and the second information in a hierarchical structure.
Preferably, the database stores an image indicating the appearance of the second article with the second identification code. An image showing the appearance of the second article can be a captured image of the second article, an icon representing the second article, or the like.

  Preferably, the information extraction unit extracts an image showing the appearance of the second article together with the second information, and the display unit uses the image showing the appearance of the second article extracted by the information extraction unit as the second item. In addition to the above information, the image is superimposed and displayed on the image photographed by the photographing means.

  An identification code reading method according to the present invention includes a photographing step of photographing a first article with a first identification code that can be read as an image with a camera, and the computer recognizes the first identification code in the photographed image. An identification code recognition step, and a second identification code to which the computer can read first information relating to the first article with the first identification code that can be read as an image. A second database associated with the first information relating to the first article to which the recognized first identification code is attached in the image; The extracted information extraction step extracted from the information and the second information extracted in the information extraction step are superimposed on the image photographed in the photographing step and displayed on the display. Characterized in that it comprises a display step.

  According to the present invention, the first information related to the first article attached with the first identification code that can be read as an image is related to the second article attached with the second identification code that can be read as an image. Associated with the second information. Therefore, it is possible to easily grasp on the screen information related to the second article associated with the first information related to the first article attached with the first identification code in the photographed image. In particular, when the database stores an image indicating the appearance of the second article, the appearance of the second article associated with the information related to the first article with the first identification code in the captured image. Can be displayed on the screen, the second information relating to the second article associated with the first information relating to the first article assigned with the first identification code in the photographed image is displayed on the screen. This can be grasped more easily.

1 is a block diagram of an embodiment of an identification code reading system according to the present invention. FIG. It is a figure which shows an example of the hierarchical structure of the information stored in the database of FIG. It is a figure which shows an example of the identification code used with the identification code reading system shown in FIG. It is a flowchart of operation | movement of the identification code reading system of FIG. It is a figure which shows the example of a display of the articles | goods on the screen of the display of FIG. It is a figure which shows the example of a display of the overlay information of the articles | goods on the screen of the display of FIG. It is a figure which shows the example of a display of the overlay information of the other articles | goods on the screen of the display of FIG. It is a figure which shows the other example of the hierarchical structure of the information stored in the database of FIG. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the storage box shown in FIG. 8, and corresponding overlay information. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the storage box shown in FIG. 8, and corresponding overlay information. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the storage box shown in FIG. 8, and corresponding overlay information. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the binder shown in FIG. 8, and corresponding overlay information. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the document shown in FIG. 8, and corresponding overlay information. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the document shown in FIG. 8, and corresponding overlay information. It is a figure which shows the example of a display on the screen of the display of FIG. 1 of the document shown in FIG. 8, and corresponding overlay information.

Embodiments of an identification code reading system and method according to the present invention will be described in detail with reference to the drawings.
FIG. 1 is a block diagram of an embodiment of an identification code reading system according to the present invention. The identification code reading system shown in FIG. 1 includes a terminal 1, a network (communication path) 2, and a server 3.

  The terminal 1 is realized as a smart phone with a camera function, a personal computer (PC) to which a camera is connected, and the like. A camera (photographing unit) 1A as a photographing unit and a computer (identification code recognizing unit) as an identification code recognizing unit. 1B, a display 1C as display means, and a communication interface 1D as communication means.

  The camera 1A takes an image of an article with an identification code that can be read as an image. For this purpose, the camera 1A includes a lens 1a, an image sensor 1b, and an analog / digital converter (AD) 1c, and outputs digital image data of a photographed identification code to the computer 1B.

  The computer 1B is realized by the same configuration as a normal computer and controls the operation of the terminal 1. In the present embodiment, the computer 1B recognizes the identification code in the image captured by the camera 1A, converts the recognized identification code into identification code information, and converts the converted identification code information to the communication interface 1D and the network 2. To the server 3. Note that the computer 1B also extracts information included in the photographed identification code.

  The display 1C displays the overlay information provided from the server 3 via the network 2 and the communication interface 1D as described later, superimposed on the image captured by the camera 1A.

  The server 3 includes a database 3A, a computer (information extraction unit) 3B as information extraction means, and a communication interface 3C. The database 3A stores the first information related to the first article attached with the first identification code that can be read as an image, and the second information related to the second article attached with the second identification code that can be read as an image. The information is stored in association with the information. In the present embodiment, the database 3A stores the first information of the first article and the second information of the second article in a hierarchical structure. An example of the hierarchical structure of information stored in the database 3A is shown in FIG. In the example shown in FIG. 2, the database 3A stores the identification code information in association with the article name.

  For example, a case where the camera 1A captures a storage box B with an identification code will be described. The database 3A relates to a storage box B as a first article with a first identification code that can be read as an image. The first information is stored in association with the second information related to the books A and B as the second article with the second identification code that can be read as an image.

  Further, the database 3A includes the first information related to the book A as the first article to which the first identification code that can be read as an image is attached and the second information that is read as an image to which the second information is attached. The second information related to the storage box B as the second article is stored in association with the second information.

  Further, the database 3A includes the first information on the binder A as the first article with the first identification code readable as an image, and the second information with the second identification code readable as an image. The second information relating to the storage box D as the second article and the third information relating to the documents A and B as the third article with the third identification code readable as an image are stored in association with each other.

  The computer 3B controls the operation of the server 3. In the present embodiment, the computer 3B receives the identification code information as the first identification code information from the computer 1B via the communication interface 1D, the network 2, and the communication interface 3C. Then, the computer 3B accesses the database 3A, and is attached with the first information related to the first article with the first identification code information and the second identification code information associated with the first information. Second information relating to the second article is extracted. The first information includes overlay information that is display information related to the first article, and the second information includes overlay information that is display information related to the second article.

  In the present embodiment, the computer 1B determines the number of identification codes extracted from the photographed image when raster scanning the photographed image in the screen of the display 1C. Recognize multiple identification codes. When the computer 1B recognizes a single identification code existing in the screen of the display 1C, the display 1C displays the overlay information corresponding to the single identification code extracted by the computer 1B as a single identification code in the screen of the display 1C. Display in association with. On the other hand, when the computer 1B recognizes a plurality of identification codes existing in the screen of the display 1C, the display 1C displays overlay information corresponding to the plurality of identification codes extracted by the computer 1B. Are displayed in association with the identification code.

  FIG. 3 is a diagram showing an example of an identification code used in the identification code reading system shown in FIG. The identification code 11 shown in FIG. 3 is a two-dimensional code having a size of 35 × 35 cells, divided into 9 × 9 blocks, and having one cell of bright separation space 23 between adjacent blocks. Therefore, the separation space is a lattice-like pattern with 8 × 8 columns and a cell interval of 1 cell width. The first position detection pattern 12A is in the lower right 3 × 3 block and the 3 × 3 separation space, and the second position detection pattern 12D is in the upper right 2 × 2 block and the 2 × 2 separation space. The third position detection pattern 12B is located in the 3 (horizontal) × 2 (vertical) block and the 3 × 2 separation space portion, and the lower left 2 (horizontal) × 3 (vertical) block and the 2 × 3 separation space portion. Are provided with fourth position detection patterns 12C, respectively. Therefore, a continuous pattern having a size larger than that of the first to fourth position detection patterns does not appear in the blocks other than the first to fourth position detection patterns of the identification code 11.

  The minimum unit of the code portion of the identification code 11 is a cell. The identification code 11 usually takes a square or rectangular shape. Although other shapes are possible, most of the two-dimensional codes are quadrangular, and the two-dimensional code of the first embodiment is also square. However, the shape is not limited to this, and may be rectangular or other shapes.

  In general, the two-dimensional code represents data in light and dark cells, and the identification code 11 also represents data in light and dark cells. Furthermore, although the identification code 11 is a 35 × 35 cell square, it can be formed into a code having a larger size or a shape such as a rectangle whose vertical and horizontal sizes are changed. A square two-dimensional code of 35 × 35 cells is referred to as version 2 × 2. Here, the minimum version is 1 × 1, and the cell size at this time is 27 × 27. The embedding of the pattern is possible when both the vertical and horizontal versions are 2 or more. The reason for not embedding a picture when the version is 1 × 1 is that even though the amount of data is small, if the pattern is embedded, the amount of data is further reduced, and even if it is recognized, almost no data can be obtained. The above restrictions were added. The identification code 11 can be extended in the vertical direction and the horizontal direction, respectively. When the horizontal version is 2 and the vertical version is 3, it is written as version 2 × 3. As the version increases by 1, the cell size increases by 8 and the number of blocks increases by 2. Therefore, in version 3 × 4, the cell size is 43 × 51. The version can be selected from 1 to 20, but is not limited to this.

  In the first position detection pattern 12A, the dark cell portion includes a square frame having a width of 1 cell of 8 × 8 cells and a 2 × 2 square at the center of the square frame. In the second position detection pattern 12D, the dark cell portion has a square frame of 1 × 4 cells of 4 × 4 cells. The third position detection pattern 12B is a horizontally long rectangular frame in which the dark cell portion has a rectangular cell with a width of 1 cell of 8 × 4 cells. The fourth position detection pattern 12C is a vertically long rectangular frame in which a dark cell portion has a rectangular cell with a width of 1 × 4 × 8 cells.

  In the identification code 11, four different position detection patterns are arranged at the four corners of the square two-dimensional code. Note that the position detection patterns are different when the shape, the size, and the arrangement direction on the two-dimensional code are different, and also include the case where the line width ratios are different. However, since it is affected by shooting conditions and the like, the difference in color is not targeted, but a light and dark binary image is targeted.

  In the identification code 11, the position detection pattern is arranged near each vertex of a square or a rectangle. This is because the influence of dirt can be dispersed by arranging them as far as possible rather than arranging them at one place. However, the position of the position detection pattern is not limited to the four corners. Like the identification code 11, not only a square but also a rectangular shape is used as a position detection pattern, so that the shape, the inclination, and the size can be easily distinguished, so that it can be recognized at high speed. In addition, the hollowed rectangle is easy to distinguish from other reflections, and there is little waste of data.

  The reason for arranging the four position detection patterns is that in the two-dimensional projective transformation used to calculate the mapping of coordinates with high accuracy in the analysis, the cell of the code and the coordinates of the captured image corresponding to the cell are calculated. This is because it is desirable to give four sets of coordinates as parameters. Even if there are only two or three position detection patterns, it is possible to perform projective transformation after calculating four points by interpolating the coordinates, but shooting from the lick, distortion, blurring, etc. In the identification code 11, four position detection patterns are arranged because it is difficult to obtain accurate coordinates and greatly affects the accuracy of data extraction.

  Also, the reason why the position detection pattern is different is that even if there is a position detection pattern that could not be recognized, the position detection pattern that could not be recognized can be reliably identified, and the coordinates of the position detection pattern that could not be recognized can be predicted. This is because the position detection pattern can be corrected. When all the same position detection patterns are used, it is not easy to specify the position detection patterns that could not be recognized as much as when different position detection patterns are used, and correction of the position detection patterns becomes difficult.

  56 blocks other than the four position detection patterns of the identification code 11 are data portions, and include an actual data block, an error correction code block, a version information block, a format information block, and a design embedding information block.

  As described above, the data portion includes 3 × 3 cells as one block and embeds data in units of blocks. The data section includes version information blocks 15 and 16, a format information block 17, and pattern embedding information blocks 18A, 18B, 19A and 19B, and the remaining blocks are actual data blocks 22. In the identification code 11, a part of the actual data block 22 is used as an error correction code and a position correction pattern.

  Therefore, the remaining blocks 22 excluding the position detection pattern, version information block, format information block, picture embedding information block, block provided with the position correction pattern, and error correction code block are actual data blocks. If the pattern is embedded, the pattern embedded block indicated by the pattern embedded information block is also excluded from the actual data block. The actual data is recorded in order such that the blocks belonging to the actual data block are laid out in order from the upper left to the right, and laid from the left one block below when the laying is completed.

  A block has 3 × 3 cells, and each cell represents one bit of light and dark, so one block has a maximum of 9-bit information (9-bit representation), but here is resistant to blurring and blurring of the image Therefore, the data amount per block can be set smaller than the 9-bit data amount. In the identification code 11, the block size is 3 × 3 cells, but the block size is not limited to this, and the minimum 1 × 1 cell size and the vertical and horizontal cell sizes are different. Block sizes such as are also possible.

  The version information block records information related to the size of the identification code 11, and there are two types of versions, the vertical version information block 15 and the horizontal version information block 16, and corresponding version information is recorded. The vertical version information block 15 and the horizontal version information block 16 disperse the risk of missing due to dirt or the like by disposing two blocks each storing the same information at two locations apart from each other. As version information, a block representing a numerical value of the version number (vertical and horizontal 1 to 20) is arranged.

  The format information block 17 records format information composed of a pattern embedding flag (1 bit) and data block type data (2 bits), and arranges three blocks storing the same information at three positions apart. The pattern embedding flag indicates whether or not a pattern is embedded, that is, whether or not a pattern embedding area is provided. When the pattern embedding is present, a pattern embedding information block is provided. The data block type data indicates the type of the data block. The data block type is the data capacity per block, and any of 6, 7, and 9 bits / block is selected. The amount of data decreases in the order of 9 bits> 7 bits> 6 bits, but the readability of the data is in the order of 6 bits> 7 bits> 9 bits. Take into account and choose. This will be described later.

  The design to be embedded is embedded in units of blocks, and when a plurality of adjacent blocks are design embedding regions, a separation space between them is also included in the design embedding region. The external shape of the pattern to be embedded is a rectangle, and the image of the pattern is not limited to a binary image in cell units, and any image may be used. Also, there may be a plurality of rectangular blocks in which the pattern is embedded. For example, the pattern may be a multi-value image having a higher resolution than the cell, a color image, or, of course, a binary image in units of cells. In the first embodiment, as a limitation of embedding the pattern, it is assumed that the pattern is not embedded in the upper code block 2 rows, the left block 2 columns, the lower block 3 rows, and the right block 3 columns. In this range, according to the specification of the two-dimensional code, a pattern embedding information block having image information is arranged in a portion outside the pattern embedding range.

  The pattern embedding information block records information on the size and block position of the area in which the pattern is embedded in the vertical and horizontal directions, and there are two types of vertical embedding information blocks 18A and 18B and horizontal embedding information blocks 19A and 19B. Corresponding version information is recorded. The vertical embedding information blocks 18A and 18B record the design embedding height (3 bits or 6 bits) and the design embedding offset height (3 bits or 6 bits). The horizontal embedding information blocks 19A and 19B designate a picture embedding width (3 bits or 6 bits) and a picture embedding offset width (3 bits or 6 bits). Whether to use 3 bits or 6 bits is related to version information as will be described later. If the version is 2 to 4, 3 bits are required, and if the version is 5 to 20, 6 bits are required. In the case of 2 to 4, the vertical embedding information block 18B and the horizontal embedding information block 19B are unnecessary.

  The pattern embedding information block exists only when the pattern embedding flag is set in the format information block. The identification code 11 assumes that the number of embedding patterns is only one, and if it is assumed that a plurality of embedding patterns are embedded in a plurality of areas, the bit of the pattern embedding flag in the format information block The number is increased to indicate the number of pattern embedding areas, and the above-mentioned pattern embedding information blocks are provided for the number of patterns. The design size and design embedding start position are specified in units of blocks. The block coordinates are expressed as (n, m) with the block at the upper left corner as the origin, the number of horizontal blocks is blW, and the number of vertical blocks is blH. In the first embodiment, the pattern embedding area is limited to a rectangular area consisting of upper left (2, 2) and lower right ((blW-4, blH-4). The maximum number of horizontal blocks of the pattern is blW-. Five and the maximum number of vertical blocks is blH-5.

  The position of the picture is embedded in the code as block embedding information blocks with the block coordinates that are the starting point (upper left) and the block size so that the position of the picture can be recognized during code analysis. . Similar to the version information block, it is handled separately in the horizontal and vertical directions. In the horizontal direction, the amount of data is divided when blW-5 is less than 8 (horizontal version 2 to 4) and in other cases (horizontal version 5 to 20), and the vertical length is the same. The values of blW and blH can be obtained by analyzing the horizontal and vertical versions.

  When blW is less than 13, the horizontal block size is a minimum of 1 and a maximum of 8 blocks, 8 can be represented by 3 bits, and similarly there are 7 horizontal offset blocks, 2 to 8, Since it can be represented by 3 bits, the total is 6 bits. Therefore, a block is represented by 6 bits, and these can be represented by one block. The same applies to the vertical. When blW is 13 or more, the horizontal block size is represented by 6 bits, and similarly, the number of horizontal offset blocks is represented by 6 bits. Therefore, it is expressed using 2 blocks. The same applies to the vertical.

  The horizontal embedding information blocks 19A and 19B are outside the upper left and lower right horizontal version information blocks (upper and lower sides), and the vertical embedding information blocks 18A and 18B are outside the lower left and lower right vertical version information blocks. (Left side and right side)

  When the pattern is not embedded, that is, when the pattern embedding flag of the format information block indicates “None”, the code embedding information block and the pattern embedding area become unnecessary, and the code becomes efficient.

  The position correction pattern is a target pattern used for the coordinate correction of the actual data block, the error correction code block, and the pattern embedding information block when the pattern is embedded. Although the approximate coordinates in the identification code 11 can be acquired from the position detection pattern, an error occurs in the coordinates of the data portion due to paper warping, bending, lens distortion, and displacement at the time of position detection pattern acquisition. Provide. Unlike the position detection pattern, the position correction pattern does not need to be distinguished from other noises. If the position detection pattern can be calculated, the position of the position correction pattern can be easily determined. Therefore, it is desirable that the position correction pattern has a shape that facilitates fine coordinate correction.

  In the identification code 11, the position correction pattern arranges blocks (position correction blocks) composed of predetermined cell combinations at regular intervals instead of actual data. A block in which the center cell of the block is dark and the surrounding area is bright is arranged if the block coordinates (7n, 7m) (n and m are integers of 0 or more) are data portions. However, when the position detection block, the version information block, the format information block, the design embedding information block and the design should exist at the coordinates, they are not arranged. Here, with respect to the block coordinates, the upper left coordinate reference coordinate of the block is (0, 0), and the block on the right is expressed as (1, 0). The block for providing the position correction pattern is not limited to the above.

  In the actual data, a message and a segment combining the header (message type (message encoding) and message size) that qualifies the message are arranged by the number of messages. As a special segment, a segment that does not include a message and contains only a termination flag is prepared. When the capacity of actual data is excessive, a segment of the termination flag is arranged, and then padding is arranged. The actual data is divided into blocks according to the data capacity per block indicated by the data block type data. Similarly, the error correction code is also divided into blocks. The computer 1B recognizes the identification code and also extracts information included in the actual data.

  When a Reed-Solomon code is used as the error correction code, error correction is performed in units of words, so it is desirable to make one word into one block. When one word spans a plurality of blocks, even when one block is contaminated, all words related to the block are subjected to error correction, and the correction efficiency is deteriorated. Dirt that causes corrections and color jumps due to spotlights are often concentrated in one place, but by making a block, the data to be corrected can be gathered in one place at the same time, enabling efficient correction. And increase the possibility of recognizing the code.

  FIG. 4 is a flowchart of the operation of the identification code reading system of FIG. Here, a case will be described in which the identification codes shown in FIG. 3 attached to the storage box A, storage box B, notebook A, and binder A shown in FIG.

  First, the camera 1A photographs an article (step S1). Next, the display 1C displays an image including an article photographed by the camera 1A on the screen (step S2). FIG. 5 is a diagram illustrating a display example of articles on the display screen of FIG. In FIG. 5, articles 31, 32, 33, and 34 correspond to storage box A, storage box B, notebook A, and binder A shown in FIG. 2, respectively, and printed identification codes 31A, 32A, 33A, and 34A are the articles. 31, 32, 33, and 34, respectively.

  The identification codes 31A, 32A, 33A and 34A are each photographed by the camera 1A and read by the computer 1B.

  Next, the computer 1B recognizes the identification codes 31A, 32A, 33A, and 34A in the image photographed by the camera 1A, converts the recognized identification codes 31A, 32A, 33A, and 34A into identification code information, and converts them. The identification code information is transmitted to the server 3 (step S3).

  Next, the computer 3B receives the identification code information from the computer 1B (step S4). Next, the computer 3B accesses the database 3A, and overlay information of the articles 31, 32, 33, 34 corresponding to the identification code information and an overlay of the second article associated with the articles 31, 32, 33, 34. Extract information.

  In this case, the computer 3B extracts overlay information indicating that the article 31 is the storage box A as the overlay information of the article 31, but there is overlay information of the second article associated with the overlay information of the article 31. Therefore, the overlay information of the second article associated with the overlay information of the article 31 is not extracted.

  Further, the computer 3B extracts the overlay information indicating that the article 32 is the storage box B as the overlay information of the article 32, and stores the storage box as the overlay information of the second article associated with the overlay information of the article 32. Overlay information indicating that the articles in the lower hierarchy of B are the book A and the book B is extracted.

  Further, the computer 3B extracts the overlay information indicating that the article 33 is the note A as the overlay information of the article 33, and the note 3A of the notebook A as the overlay information of the second article associated with the overlay information of the article 33. Overlay information indicating that the higher-level article is the storage box C is extracted.

  Furthermore, the computer 3B extracts the overlay information indicating that the article 34 is the binder A as the overlay information of the article 34, and uses the binder A as the overlay information of the second article associated with the overlay information of the article 34. Overlay information indicating that the higher-order item is the storage box D and overlay information indicating that the lower-order item of the binder A is the document A and the document B are extracted.

  Next, the computer 3B transmits the overlay information of the articles 31, 32, 33, and 34 to the computer 1B (step S6). Next, the computer 1B receives the overlay information of the articles 31, 32, 33, and 34 from the computer 3B (step S7).

  Next, the display 1C superimposes the overlay information of the articles 31, 32, 33, and 34 on the articles 31, 32, 33, and 34 photographed by the camera 1A and displays them on the screen (step S8).

  FIG. 6 is a diagram illustrating a display example of article overlay information on the display screen of FIG. 1. In FIG. 6, the display 1 </ b> C displays overlay information 31 </ b> B indicating that the article 31 is the storage box A on the screen while being superimposed on the article 31. Further, the display 1 </ b> C displays overlay information 32 </ b> B indicating that the article 32 is the storage box B superimposed on the article 32 on the screen. Further, the display 1 </ b> C displays overlay information 33 </ b> B indicating that the article 33 is a note A on the screen while being superimposed on the article 33. Furthermore, the display 1 </ b> C displays overlay information 34 </ b> B indicating that the article 34 is the binder A superimposed on the article 34 on the screen.

  Next, the computer 3B transmits the overlay information of the second article associated with the overlay information of the articles 31, 32, 33, and 34 to the computer 1B (step S9). Next, the computer 1B receives the overlay information of the second item associated with the overlay information of the items 31, 32, 33, and 34 from the computer 3B (step S10).

  Next, the display 1C has a screen in which the overlay information of the second article associated with the overlay information of the articles 31, 32, 33, and 34 is superimposed on the articles 31, 32, 33, and 34 photographed by the camera 1A, respectively. (Step S11), and the processing flow ends.

  FIG. 7 is a diagram showing a display example of overlay information of another article on the display screen of FIG. In FIG. 7, the display 1 </ b> C displays overlay information 32 </ b> D indicating that the low-order item of the storage box B is the book A and the book B as the overlay information of the second item associated with the overlay information of the item 32. And superimposed on the article 32 and displayed on the screen. Further, the display 1 </ b> C displays overlay information 33 </ b> C indicating that the higher-order article of the notebook A is the storage box C on the screen while being superimposed on the article 33. Further, the display 1 </ b> C displays overlay information 34 </ b> C indicating that the higher-order item of the binder A is the storage box D and overlay information 34 </ b> D indicating that the lower-order item of the binder A is the document A and the document B. And superimposed on the article 34 and displayed on the screen.

  According to the present embodiment, the identification code is attached to the first information of the articles 31, 32, 33, 34 as the first article with the identification codes 31A, 32A, 33A, 34A readable as images. Identification associated with the information of the articles 31, 32, 33, 34 to which the identification codes 31A, 32A, 33A, 34A in the captured image are attached. The second information about the second article to which the code is attached can be easily grasped on the screen of the display 1C.

  Therefore, the presence / absence of an article in the upper hierarchy and the lower hierarchy of the articles 31, 32, 33, 34, the type of at least one of the articles in the upper hierarchy and the articles in the lower hierarchy of the articles 31, 32, 33, 34, It can be easily grasped on the screen of the display 1C.

  FIG. 8 is a diagram showing another example of the hierarchical structure of information stored in the database of FIG. In the example shown in FIG. 8, the database 3A stores the identification code together with the appearance name, which is an image showing the article name, storage start date, article size, and article appearance. That is, the database 3A stores an appearance image of the second article with the identification code. The appearance image of the second article can be a photograph of the second article, an icon representing the second article, or the like.

  9A, 9B, and 9C are diagrams showing display examples on the screen of the display of FIG. 1 of the storage box shown in FIG. 8 and the corresponding overlay information. In FIG. 9A, the camera 1A images the article 41 with the identification code 41A, the display 1C has overlay information 41B indicating that the article 41 is the storage box A, and the higher-order article of the storage box A exists. In this example, overlay information 41C as an alternative image indicating that the item is not to be displayed and overlay information 41D indicating an appearance image of the binder A that is an item in the lower hierarchy of the storage box A are displayed on the screen while being superimposed on the item 41. In the display example of FIG. 9A, the overlay information 41C is displayed in characters.

  In FIG. 9B, the camera 1A images the article 41 with the identification code 41A, the display 1C has overlay information 41B indicating that the article 41 is the storage box A, and the higher-order article of the storage box A exists. In this example, overlay information 41C ′ as an alternative image indicating that the item is not to be displayed and overlay information 41D indicating the appearance image of the binder A that is an item in the lower hierarchy of the storage box A are displayed on the screen while being superimposed on the item 41. In the display example of FIG. 9B, the overlay information 41C 'is displayed with a symbol indicating that there is no higher-order item.

  In FIG. 9C, the article 41 with the identification code 41A is photographed, and the display 1C has the overlay information 41B indicating that the article 41 is the storage box A and the appearance of the binder A, which is an article in the lower hierarchy of the storage box A. A case where overlay information 41D indicating an image is superimposed on the article 41 and displayed on the screen is shown. That is, in the example shown in FIG. 9C, the display 1C does not display the substitute image indicating that the high-order item of the storage box A does not exist, thereby indicating that the high-order item of the storage box A does not exist. .

  10 is a diagram showing a display example of the binder shown in FIG. 8 and the corresponding overlay information on the display screen of FIG. In FIG. 10, the camera 1 </ b> A takes an image of the article 42 with the identification code 42 </ b> A, and the display 1 </ b> C has overlay information 42 </ b> B indicating that the article 42 is binder A, and a storage box that is an article in a higher layer of the binder A. The overlay information 42C indicating the appearance image of A and the overlay information 42D indicating the appearance images of the documents A and B, which are articles in the lower layers of the binder A, are superimposed on the article 42 and displayed on the screen.

  11A, 11B, and 11C are diagrams showing display examples of the document shown in FIG. 8 and the corresponding overlay information on the display screen of FIG. In FIG. 11A, the camera 1A images the article 43 with the identification code 43A, and the display 1C has overlay information 43B indicating that the article 43 is the document A, and binder A, which is an article in a higher hierarchy of the document A. In this example, overlay information 43C indicating the appearance image of the document and overlay information 43D as an alternative image indicating that there is no lower-order item of the document A are displayed on the screen while being superimposed on the item 43. In the display example of FIG. 11A, the overlay information 43D is displayed in characters.

  In FIG. 11B, the camera 1A photographs the article 43 with the identification code 43A, and the display 1C has overlay information 43B indicating that the article 43 is the document A, and binder A, which is an article in a higher hierarchy of the document A. The overlay information 43C indicating the appearance image of the document and the overlay information 43D ′ as an alternative image indicating that the article of the lower hierarchy of the document A does not exist are superimposed on the article 33 and displayed on the screen. In the display example of FIG. 11B, the overlay information 41D 'is displayed with a symbol indicating that there is no higher-order item.

  In FIG. 11C, the camera 1A takes an image of the article 43 with the identification code 43A, and the display 1C displays overlay information 43B indicating that the article 43 is the document A and binder A which is an article in a higher hierarchy of the document A. A case is shown in which overlay information 43 </ b> C indicating the external appearance image is superimposed on the article 43 and displayed on the screen. In other words, in the example illustrated in FIG. 11C, the display 1C does not display the substitute image indicating that the lower-order item of the document A does not exist, thereby indicating that the lower-order item of the document A does not exist.

  According to the examples shown in FIGS. 9A, 9B, 9C, 10, 11A, 11B, and 11C, the database 3A stores the appearance image of the second article to which the identification code is attached. An appearance image of the second article to which the identification code associated with the first information related to the first article to which the identification code is attached can be displayed on the screen of the display 1C. Therefore, the second information related to the second article attached with the identification code associated with the first information related to the first article attached with the identification code in the captured image is further displayed on the screen of the display 1C. Easy to grasp. Therefore, for example, it becomes easy to search for the second article after grasping the appearance of the second article related to the displayed first article. In addition, when a plurality of articles in a hierarchical relationship are displayed on the screen, it becomes clear that these articles are in a hierarchical relationship by recognizing the identification code attached to them. As the overlay information, a display indicating that there is a hierarchical relationship may be performed.

  The present invention is not limited to the above-described embodiment, and many changes and modifications can be made. For example, instead of the identification code shown in FIG. 3, a one-dimensional code (for example, a barcode) or another two-dimensional code (for example, a QR code (registered trademark)) can be used to recognize a single identification code. The present invention can be applied to both the case of performing and the case of recognizing a plurality of identification codes.

  Further, the database can be provided in the terminal instead of the server. Also, instead of sequentially displaying the article, the overlay information of the article and the overlay information of the other article related to the article, the overlay information of the article, the article overlay information and the other article related to the article is displayed at the same time. You can also.

  Further, as overlay information, it is possible to display the presence / absence of an article in a higher hierarchy, the presence / absence of an article in a lower hierarchy, a list of articles, the number of articles in a higher hierarchy, and the number of articles in a lower hierarchy. In addition, overlay information of other articles related to the article can be nested and displayed in the overlay information of the article. In addition, the overlay information of the article and the overlay information of other articles related to the article can be displayed as a tab. Further, the overlay code of the article is included in the identification code attached to the article, and the overlay information of the other article associated with the overlay information of the article is extracted from the database, and the overlay information of the article is attached to the article. It can also be extracted from the identification code.

1 Terminal 1A Camera (shooting unit)
1a lens 1B computer (identification code recognition part)
1b Image sensor 1C Display 1c Analog to digital converter (AD)
1D, 3C communication interface 2 network (communication path)
3 server 3A database 3B computer (information extraction part)
11, 31A, 32A, 33A, 34A, 41A, 42A, 43A Identification code 12A First position detection pattern 12B Third position detection pattern 12C Fourth position detection pattern 12D Second position detection pattern 15 Version information block (vertical)
16 Version information block (horizontal)
17 Format information block 18A, 18B Pattern embedding information block (vertical)
19A, 19B Pattern embedding information block (horizontal)
31, 32, 33, 34, 41, 42, 43 Goods 31B, 32B, 32D, 33B, 33C, 34B, 34C, 34D, 41B, 41C, 41C ', 41D, 42B, 42C, 42D, 43B, 43C, 43C ', 43D, 43D' Overlay information

Claims (8)

Associating the first information relating to the first article attached with the first identification code readable as an image with the second information relating to the second article attached with the second identification code readable as an image A database that stores an image indicating an appearance of the second article to which the second identification code is attached ;
Photographing means for photographing the first article to which the first identification code is attached;
An identification code recognition means for recognizing the first identification code in the photographed image;
Information extracting means for accessing the database and extracting the second information associated with the first information relating to the first article with the recognized first identification code in the image from the database; ,
Display means for displaying the second information extracted by the information extraction means superimposed on an image taken by the photographing means;
An identification code reading system comprising: When extracting the second information from the database, the information extraction unit extracts, from the database, the first information relating to the first article with the recognized first identification code in the image. ,
The identification code reading system according to claim 1, wherein the display unit displays the first information and the second information extracted by the information extraction unit so as to be superimposed on an image captured by the imaging unit. The information extraction means extracts the first information included in the first identification code,
The identification code reading system according to claim 1, wherein the display unit displays the first information and the second information extracted by the information extraction unit so as to be superimposed on an image captured by the imaging unit.   The identification code reading system according to any one of claims 1 to 3, wherein the database stores the first information and the second information in a hierarchical structure. The information extracting means extracts an image showing an appearance of the second article together with the second information;
5. The display unit according to claim 1, wherein the display unit displays an image showing the appearance of the second article extracted by the information extraction unit, together with the second information, superimposed on the image captured by the imaging unit . The identification code reading system according to any one of the preceding claims. A photographing step of photographing with a camera a first article with a first identification code readable as an image;
An identification code recognition step of recognizing the first identification code in the captured image by a computer;
The second information related to the second article attached with the second identification code readable by the computer as the first information related to the first article attached with the first identification code readable as an image. A database for storing information in association with information , wherein the database storing an image showing an appearance of the second article with the second identification code is accessed, and the recognized first identification in the image and information extraction step you extracted encoding said second information associated with the first information on the first article attached from said database,
A display step of superimposing the second information extracted in the information extraction step on an image photographed in the photographing step and displaying it on a display;
An identification code reading method comprising: Associating the first information relating to the first article attached with the first identification code readable as an image with the second information relating to the second article attached with the second identification code readable as an image A database that stores the first information and the second information in a hierarchical structure;
Photographing means for photographing the first article to which the first identification code is attached;
An identification code recognition means for recognizing the first identification code in the photographed image;
Information extracting means for accessing the database and extracting the second information associated with the first information relating to the first article with the recognized first identification code in the image from the database; ,
Display means for displaying the second information extracted by the information extraction means superimposed on an image taken by the photographing means;
An identification code reading system comprising: A photographing step of photographing with a camera a first article with a first identification code readable as an image;
An identification code recognition step of recognizing the first identification code in the captured image by a computer;
The second information related to the second article attached with the second identification code readable by the computer as the first information related to the first article attached with the first identification code readable as an image. A database that is stored in association with information, the database storing the first information and the second information in a hierarchical structure, and the first identification code recognized in the image is attached An information extraction step of extracting from the database the second information associated with the first information relating to the first article;
A display step of superimposing the second information extracted in the information extraction step on an image photographed in the photographing step and displaying it on a display;
An identification code reading method comprising: