JP4252519B2 - Image analysis apparatus, object recognition method, card, image data, image data transmission method, and image analysis system - Google Patents

Description

  The present invention relates to a technique for expressing code data by two-dimensionally arranging a plurality of cells, and in particular, an object having code data, image data for displaying code data on a display device, and a method for transmitting the image data About.

In recent years, a technique for recognizing code data by capturing a two-dimensional code with a camera and executing a predetermined process associated with the code data has become widespread. Compared to one-dimensional barcodes, two-dimensional codes have more information that can be encoded, and various types of two-dimensional codes are now available. The code data of the two-dimensional code needs to be read efficiently and accurately from the image data. In view of such a situation, there has been proposed a technique related to image recognition of a two-dimensional code (see, for example, Patent Document 1).
JP 2000-82108 A

If the resolution of the image data is high, a large amount of information of the two-dimensional code can be acquired, and a high recognition rate of the code data can be realized. However, some cameras have low resolution, and even in such a case, it is preferable to increase the recognition rate of code data.
An object of the present invention is to provide a two-dimensional code that can be recognized with high accuracy.

  In order to solve the above-described problem, an aspect of the present invention provides a code data portion constituting code data by two-dimensionally arranging a plurality of square cells, and a plurality of corner cells arranged so as to surround the code data portion. The corner cell provides an object having a larger area than the square cell. This object may be a two-dimensional object such as a card or a three-dimensional object. Further, the square cell and the corner cell may be printed on the object or may be stamped. These cells should just exist in the state in which an imaging device can image.

  Another aspect of the present invention includes a code data portion that configures code data by two-dimensionally arranging a plurality of rectangular cells, and a plurality of corner cells arranged so as to surround the code data portion. An object having a round shape is provided. This object may be a two-dimensional object such as a card or a three-dimensional object. Further, the square cell and the corner cell may be printed on the object or may be stamped. These cells should just exist in the state in which an imaging device can image.

  Still another aspect of the present invention is image data generated in accordance with a data format for displaying a two-dimensional code on a display device, in order to display a plurality of rectangular cells at predetermined coordinate positions on the display device. Data and data for displaying a plurality of corner cells at coordinate positions surrounding a plurality of square cells displayed at a predetermined coordinate position, so that the corner cells are displayed larger than the square cells. The present invention provides image data characterized in that cell and square cell data are set.

  Still another aspect of the present invention is image data generated in accordance with a data format for displaying a two-dimensional code on a display device, in order to display a plurality of rectangular cells at predetermined coordinate positions on the display device. And data for displaying a plurality of corner cells at coordinate positions surrounding a plurality of rectangular cells displayed at a predetermined coordinate position, so that the corner cells are displayed in a round shape. Image data characterized in that data is set is provided.

  Still another aspect of the present invention is a method of transmitting image data generated according to a data format for displaying a two-dimensional code on a display device, wherein the transmitted image data displays a plurality of rectangular cells. Data for displaying at a predetermined coordinate position above and data for displaying a plurality of corner cells at coordinate positions surrounding a plurality of rectangular cells displayed at the predetermined coordinate position, the corner cell being a square cell The present invention provides an image data transmission method characterized in that data of corner cells and rectangular cells are set so as to be displayed larger.

  Still another aspect of the present invention is a method of transmitting image data generated according to a data format for displaying a two-dimensional code on a display device, wherein the transmitted image data displays a plurality of rectangular cells. The corner cell has a round shape with data for displaying at a predetermined coordinate position on the top and data for displaying a plurality of corner cells at a coordinate position surrounding a plurality of rectangular cells displayed at the predetermined coordinate position. An image data transmission method is provided in which corner cell data is set so as to be displayed on the screen.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, a two-dimensional code that can be recognized with high accuracy can be provided.

  FIG. 1 shows a configuration of a game system 1 according to an embodiment of the present invention. The game system 1 includes an imaging device 2, an image processing device 10, and an output device 6. The image processing device 10 includes an image analysis device 20 and a game device 30. The image analysis device 20 and the game device 30 may be configured as separate devices, but may be configured as a single unit. The imaging device 2 is a video camera configured by a CCD imaging device or a MOS imaging device, and has a resolution of 320 pixels × 240 pixels, for example. The imaging device 2 images the real space at a predetermined cycle and generates a frame image for each cycle. The imaging region 5 is a range where the imaging device 2 captures an image, and the position and size of the imaging region 5 are adjusted by adjusting the height and direction of the imaging device 2. The game player moves the game card 4 that is a real object with a finger in the imaging area 5. The game card 4 has a two-dimensional code for uniquely identifying itself.

  The output device 6 includes a display 7 that is a display device. The output device 6 may further include a speaker (not shown). Basically, the image processing apparatus 10 displays the frame image captured by the imaging apparatus 2 on the display 7, and controls display so that a character that is a virtual object is superimposed on the game card 4. The player can easily check whether or not the game card 4 is in the imaging area 5 by looking at the display 7. If not, the player can shift the position of the game card 4 or adjust the orientation of the imaging device 2. Then, the imaging device 2 is caused to image the game card 4.

  In the game application of the embodiment, the player operates the character by performing a predetermined operation on the game card 4. Due to the nature of the game application, it is preferable that the player recognize the unity between the game card 4 and the character, and therefore, the character image is displayed superimposed on the game card 4. When the game card 4 is slowly moved within the imaging area 5, the character moves together following the movement of the game card 4 while maintaining the state of riding on the game card 4.

  The movement of the above characters is controlled by the image processing apparatus 10. First, the image analysis device 20 extracts image information of the game card 4 from the frame image acquired by the imaging device 2. Further, the image analysis device 20 extracts a two-dimensional code printed on the game card 4 from the image information of the game card 4. At this time, the image analysis device 20 determines position information, direction information, distance information, and the like in the space of the game card 4 from the two-dimensional code of the game card 4.

  As will be described later, the image analysis device 20 uses the reference cells and the four corner cells in the image of the game card 4 in the frame image to determine the distance between the imaging device 2 and the game card 4, the orientation of the game card 4, and the like. Calculate to find. As a result, the character is controlled to face forward on the game card 4. Further, the code data of the game card 4 is acquired from the array of a plurality of rectangular cells in the two-dimensional code.

  FIG. 1 shows a state in which the game card 4 is simply placed on the table 3, but the game card 4 is tilted with respect to the table 3, for example, or lifted above the table 3. May be. The image analysis device 20 has a function of recognizing a state where the game card 4 is tilted or a state where the height from the table 3 is changed by image analysis. The result of the image analysis performed by the image analysis device 20 is sent to the game device 30. Note that the frame image captured by the imaging device 2 may be sent to the game device 30 and image analysis performed on the game device 30 side. In this case, the image processing apparatus 10 is configured by only the game apparatus 30.

  The game apparatus 30 performs display control so that the character is positioned on the game card 4 on the display screen of the display 7 based on the image analysis result in the image analysis apparatus 20. The character may be associated with the code data extracted from the game card 4 and appropriately assigned for each game scene. In this case, when the game scene is switched, the displayed character is also switched.

  FIG. 2 shows a two-dimensional code printed on the surface of the game card. A plurality of polygonal cells having a predetermined shape are two-dimensionally arranged on the surface of the game card 4 according to a predetermined arrangement rule. Specifically, the two-dimensional code of the game card 4 is a two-dimensional arrangement of a reference cell 100 and a plurality of rectangular cells 106 serving as a reference for performing recognition processing of the game card 4 from image data captured by the imaging device 2. The code data portion 102 constituting the code data, and a plurality of corner cells 104a, 104b, 104c, 104d arranged so as to surround the code data portion 102 (hereinafter collectively referred to as “corner cell 104”) It has. The code data portion 102 and the corner cell 104 exist in the code portion 110 in the two-dimensional code.

  The reference cell 100, the square cell 106, and the corner cell 104 are each configured to have a predetermined shape. In the two-dimensional code shown in FIG. 2, each cell is painted black. In FIG. 2, a region 108 is configured by connecting a plurality of rectangular cells 106, and constitutes code data together with other rectangular cells 106. When there are a plurality of game cards 4 in the embodiment, the arrangement and size of the reference cell 100 and the corner cell 104 are common, and the game card 4 is uniquely specified by the arrangement of the rectangular cells 106 in the code data portion 102. be able to.

  Details of the two-dimensional code in this embodiment will be described below. Assuming that one side of the square-shaped square cell 106 is one block, the reference cell 100 is a rectangular cell (with a long side in the horizontal direction of 7 blocks and a short side in the vertical direction of 1.5 blocks ( Element). In the embodiment, a square area formed by one vertical block and one horizontal block is called a block area.

  The code data portion 102 is formed in an area surrounded by 7 blocks × 7 blocks. One side of this region is parallel to the long side of the reference cell 100 and is arranged at a position one block away from the long side. As shown in the figure, the code part 110 is an area surrounded by 8 blocks × 8 blocks, and the code data part 102 is included in the area where the code part 110 is formed. The area where the square cells 106 are actually arranged is a portion excluding the area of 2 blocks × 2 blocks at the four corners of the area surrounded by 7 blocks × 7 blocks. That is, the rectangular cell 106 is not arranged in the area of 2 blocks × 2 blocks at the four corners. Therefore, the rectangular cell 106 is arranged in the block area of (7 × 7−2 × 2 × 4) = 33, Configure code data.

  The corner cell 104 a is disposed in the upper left corner area of the code portion 110. This upper left corner area may be within the range of the area of 2 blocks × 2 blocks at the upper left corner of the area surrounded by 7 blocks × 7 blocks as the code data portion 102, or may extend beyond the range. In the example shown in the figure, the corner cell 104a is arranged to protrude 0.5 blocks vertically and 0.5 blocks horizontally from the 2 block × 2 block region in the upper left corner of the code data portion 102. That is, the corner cell 104a exists in the upper left corner region surrounded by 2.5 blocks × 2.5 blocks in the code portion 110. Similarly, the corner cell 104b is disposed in the upper right corner region of the code portion 110, the corner cell 104c is disposed in the lower right corner region of the code portion 110, and the corner cell 104d is disposed in the lower left corner region of the code portion 110. The In the example shown in the drawing, each corner cell 104 is formed in a square shape, specifically, a square cell with one side having a length of 1.5 blocks.

  In the code data portion 102, if one block area is 1 bit, information of 33 bits in total can be encoded. Of these 33 bits, 9 bits constitute check data for confirming that the code data is correct code data. Therefore, information for 24 bits is encoded in the code data portion 102.

  The rectangular cell 106 is an important element that expresses code data, and needs to be accurately recognized by the image analysis apparatus 20. For that purpose, it is ideal to make all the square cells 106 large, but naturally the size of the game card 4 is limited, and the enlargement of the square cells 106 is equivalent to the code data portion. As a result, the number of rectangular cells 106 constituting the number 102 is reduced. This corresponds to reducing the number of bits of information and is not preferable.

  From the above, in this embodiment, the corner cell 104 is configured to have a larger area than the square cell 106. As will be described later, in the image analysis apparatus 20 of this embodiment, the corner cells 104 at the four corners are used to detect the code data portion 102. In other words, if the corner cells 104 at the four corners cannot be detected, the square cell 106 cannot be detected. Therefore, in this two-dimensional code, it is necessary to first reliably recognize the corner cells 104 rather than the square cells 106. is there. By forming the four corner cells 104 larger than the square cells 106, the recognition rate of the four corner cells 104 can be increased.

  The image analysis apparatus 20 extracts candidates for the reference cell 100 in the frame image and assumes a reference cell 100 that satisfies a predetermined reference, and there are four corner cells 104 in the vicinity of the assumed reference cell 100. It is determined whether or not. When the four corner cells 104 are recognized, the image analysis apparatus 20 reads the area surrounded by the four corner cells 104, that is, the arrangement of the rectangular cells 106 in the code data portion 102, and acquires code data. Thereby, it is determined that the assumed reference cell 100 is the true reference cell 100, and the game card 4 can be recognized. The image analysis device 20 calculates and obtains the position and direction of the game card 4 in the virtual three-dimensional coordinate system with the imaging device 2 as a reference (origin).

  FIG. 3 shows the configuration of the image analysis apparatus. The image analysis device 20 includes a frame image acquisition unit 40, a real object extraction unit 42, and a state determination unit 44. The real object extraction unit 42 includes a binarization processing unit 50, a reference cell detection unit 52, a code unit detection unit 54, and a verification processing unit 56. The processing function of the image analysis apparatus 20 in the present embodiment is realized by a CPU, a memory, a program loaded in the memory, and the like, and here, a configuration realized by cooperation thereof is illustrated. The program may be built in the image analysis apparatus 20 or may be supplied from the outside in a form stored in a recording medium. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof. In the illustrated example, the CPU of the image analysis device 20 has functions as a frame image acquisition unit 40, a real object extraction unit 42, and a state determination unit 44.

  The frame image acquisition unit 40 acquires a real space frame image captured by the imaging device 2. The imaging device 2 periodically acquires frame images, and preferably generates frame images at 1/60 second intervals.

  The real object extraction unit 42 extracts a real object image, that is, an image of the game card 4 from the frame image. Specifically, the binarization processing unit 50 converts the frame image into binary bit representation, and expresses the frame image information by turning the bit on and off. The reference cell detection unit 52 detects the reference cell 100 from the binarized data of the frame image. When the reference cell 100 is detected, the code part detection unit 54 detects the code part 110 based on the position of the reference cell 100, and the verification processing unit 56 performs verification processing of the code data included in the code part 110.

  The state determination unit 44 determines the state of the real object in the set coordinate system, and specifically determines the position and direction of the game card 4 and the distance from the imaging device 2. The position information, direction information, and distance information determined by the state determination unit 44 are respectively associated with code data acquired from the two-dimensional code and sent to the game apparatus 30. When a plurality of game cards 4 are present in the imaging area 5, position information, direction information, distance information, and code data are associated with each game card 4 and sent to the game apparatus 30. In the game system 1, the frame image itself is also sent to the game device 30 in order to display the frame image captured by the imaging device 2 on the display 7. The game apparatus 30 reads the character associated with the code data and displays the game card 4 and the character superimposed on each other in a three-dimensional virtual space.

  FIG. 4 is a flowchart showing the processing procedure of the recognition process of the two-dimensional code. The two-dimensional code is imaged by the imaging device 2, and image data for one frame obtained as a result is acquired by the frame image acquisition unit 40. The binarization processing unit 50 performs binarization processing on the image data (S10). In the binarization process, the pixel value of a pixel holding a luminance greater than a predetermined threshold is encoded to “0”, and the pixel is white on display. Hereinafter, a pixel whose pixel value is encoded as “0” in this way is referred to as a white pixel. On the other hand, the pixel value of the pixel holding the luminance value below the threshold is encoded as “1”, and the pixel is black on display. Hereinafter, a pixel whose pixel value is encoded as “1” in this way is referred to as a black pixel. The binarization processing unit 50 collects areas in which black pixels are continuously present as one connected area, and sets (labels) the extracted black pixel connected areas in ascending order from 1 in order. (S12). For example, the labeling numbers may be set in the order in which they are extracted as connected regions, that is, in the specified order. The number is stored in a RAM (not shown) together with the coordinate data. When the two-dimensional code shown in FIG. 2 is used, since the corner cell 104 is displayed larger than the square cell 106, the probability that the corner cell 104 can be detected as a black pixel connection region and the accuracy of the coordinate position can be increased. .

  In the two-dimensional code recognition process executed before the current target image data, the reference cell detection unit 52 displays the recognized position on the display of the two-dimensional code, for example, the center point of the reference cell 100 in the RAM. It is determined whether it is stored (S14). When the center point of the reference cell 100 in the previous frame is stored in the RAM (Y in S14), the stored position is set as the start point of the reference cell detection process (S16). When the reference cell 100 is detected in the previous image frame, the position of the game card 4 is often not changed greatly in the current image frame, so the center point of the reference cell 100 in the previous frame is used. Thus, the reference cell 100 can be efficiently detected in the current frame. On the other hand, when the position of the two-dimensional code is not stored in the RAM (N in S14), the reference cell detection unit 52 sets the center point on the display 7 as the start point of the reference cell detection process (S18). . After setting the starting point, the reference cell detection unit 52 executes the detection process of the reference cell 100 (S20).

  FIG. 5 is a flowchart showing reference cell detection processing. Note that the number of pixels shown in this flow depends on the resolution of the imaging device 2, and the number of pixels is shown as an example here for easy understanding. First, the reference cell detection unit 52 reads the total number M of black pixel connected regions labeled in S12 of FIG. 4 and initializes the counter value j to 1 (S100). Next, from the start point set in S16 or S18 of FIG. 4, the black pixel connection area is searched along the trace of the counterclockwise spiral on the screen of the display 7, and the black pixel connection area detected first is used as a reference. A cell candidate area is selected (S102). The reference cell detection unit 52 extracts the edge portion of the selected reference cell candidate region, and determines the short side and the long side (S104).

  The reference cell detection unit 52 determines whether or not the short side includes a predetermined number of pixels, for example, 20 pixels or more (S106). The short side is configured with a length corresponding to 1.5 blocks, but when the short side is the reference cell 100 configured with less than 20 pixels, the length of one block is a smaller number of pixels. It will be composed of. Therefore, one side of the rectangular cell 106, which is the smallest component in the two-dimensional code, is about 13 pixels or less, and the image pickup apparatus 2 cannot capture images appropriately. From this, when it is determined that the short side is configured by fewer than 20 pixels (N in S106), it is determined that the black pixel connection region selected in S102 is not the reference cell 100. The process proceeds to S116.

  When it is determined that the short side is composed of 20 or more pixels (Y in S106), the reference cell detection unit 52 uses a pixel whose long side of the reference cell candidate region is a predetermined number of pixels, for example, 300 pixels or less. It is determined whether or not it is configured (S108). For example, if the reference cell 100 is composed of more than 300 pixels on the long side, the length of one block obtained at a ratio of 1 to 7 on the long side becomes large. The lower right corner cell 104c and the lower left corner cell 104d located 8.5 blocks apart from 100 are no longer imaged by the imaging device 2. From this, when it is determined that the long side is composed of more than 300 pixels (N in S108), it is determined that the black pixel connection region selected in S102 is not the reference cell 100. The process proceeds to S116.

  When it is determined that the long side is composed of pixels of 300 pixels or less (Y in S108), the reference cell detection unit 52 determines that the total number of black pixels in the reference cell candidate region is 20 pixels or more and less than 1500 pixels. It is determined whether or not there is (S110). When the total number of black pixels is less than 20 pixels (N in S110), the same problem occurs when the short side is composed of less than 20 pixels, and when the total number is 1500 pixels or more (N in S110). The same problem occurs when the long side is composed of more than 300 pixels. Therefore, in these cases, it is determined that the possibility that the reference cell candidate region is the reference cell 100 is small, and the process proceeds to S116.

  When the total number of black pixels in the reference cell candidate region is 20 pixels or more and less than 1500 pixels (Y in S110), the reference cell detection unit 52 determines the squareness of the reference cell candidate region (S112). The squareness is obtained based on the moment of the reference cell candidate region as described in Patent Document 1 described above. When it is determined that it is not a square (N in S112), it is determined that the reference cell candidate region is not the reference cell 100, and the process proceeds to S116. When it is determined that the pixel is a square (Y in S112), the reference cell detection unit 52 sets the reference cell candidate region as the reference cell 100 (S114), and the labeling number of the black pixel connection region that is set as the reference cell candidate region Is stored in the RAM. In this way, the reference cell 100 can be detected. When only one game card 4 is included in the image data, the reference cell detection process ends when the reference cell 100 is detected. On the other hand, if there is a possibility that a plurality of game cards 4 are included, the process proceeds to S116.

  The reference cell detection unit 52 determines whether or not the count value j is equal to the total number M of reference cell candidate areas (S116). If the count value j has not reached the total number M (N in S116), the count value j is incremented by 1 (S118), and the process returns to the step of S102. On the other hand, when the count value j reaches the total number M (Y in S116), the reference cell detection process is terminated.

  Returning to FIG. 4, when the reference cell 100 is not detected in the reference cell detection process of S20 (N of S22), since the presence of the game card 4 cannot be confirmed, the two-dimensional code recognition process is terminated. On the other hand, when the reference cell 100 is detected (Y in S22), a code part detection process is executed (S24).

  FIG. 6 is a flowchart showing the code portion detection process. First, the code part detection unit 54 reads the total number N of the reference cells 100 set in S114 of FIG. 5, and initializes the counter value k to 1 (S200). Note that the reference cells 100 set in S114 are numbered in the order set. Subsequently, the total number M of the black pixel connected regions labeled in S12 of FIG. 4 is read, and the counter value j is initialized to 1 (S202). The code part detection part 54 detects the black pixel connection area of the number corresponding to the count value j, and selects it as a corner cell candidate area in the lower left corner (S204).

  Next, the code portion detection unit 54 detects that the selected lower left corner cell candidate region is within the search range set in advance for the reference cell 100 of the number corresponding to the count value k detected in S20 of FIG. It is determined whether or not it exists (S206). Since the positional relationship between the reference cell 100 and the corner cell 104d at the lower left corner is predetermined as shown in FIG. 2, the search range for the lower left corner cell can be narrowed down from the coordinate data of the reference cell 100. is there. Since the reference cell 100 has a horizontally long rectangular shape, it is possible to set a search range at two positions facing each other across the long side. When it does not exist within the search range (N in S206), it is determined that the black pixel connected region is not the lower left corner cell 104d, and the process proceeds to S232. If it exists within the search range (Y in S206), the black pixel connection region is set as the lower left corner cell 104d (S208).

  After setting the lower left corner cell 104, the code portion detection unit 54 initializes the value l of another counter for counting the number of the black pixel connection region to 1 (S210). The code part detection part 54 detects the black pixel connection area of the number corresponding to the count value l and selects it as a corner cell candidate area in the lower right corner (S212).

  Next, the code part detection unit 54 calculates the ratio between the number of pixels (area) of the lower left corner cell 104d set in S208 and the number of pixels (area) of the lower right corner cell candidate region selected in S212, and the ratio It is determined whether (area ratio) is, for example, 1/6 or more or 6 times or less (S214). In the corner cell 104, the number of pixels acquired changes according to the distance from the imaging device 2. Specifically, it has a large area if it is close to the imaging device 2, and the area is small if it is far away. Therefore, it is assumed that the areas in the image data are different even in the corner cells. However, for example, a case where the ratio of the lower left corner cell to the lower right corner cell is smaller than 1/6 or larger than 6 times (N in S214) cannot normally be assumed. Cannot be regarded as a corner cell in step S226, and the process proceeds to step S226.

  When the area ratio is, for example, 1/6 or more and 6 times or less (Y in S214), the code part detection unit 54 determines the center point of the lower left corner cell 104d and the center point of the lower right corner cell candidate region selected in S212. It is determined whether or not the distance of (2) satisfies a predetermined condition (S216). Here, the predetermined condition may be that the distance between the center points approximates the length of the long side of the reference cell 100, for example. If the predetermined condition is not satisfied (N in S216), the process proceeds to S226, and if the predetermined condition is satisfied (Y in S216), the code portion detection unit 54 sets the black pixel connection region as the lower right corner cell 104c. (S218).

  When the lower left corner cell 104d and the lower right corner cell 104c are set as described above, the code part detection unit 54 changes the set reference cell 100, lower left corner cell 104d, and lower right corner cell 104c to X on the screen of the display 7. Affine transformation is performed in the axial direction and the Y-axis direction (S220). The length of one block is calculated based on the length of the long side or short side of the reference cell 100 set in S20. The code part detection part 54 maps the black pixel connection area included in the area of the code part 110 as a cell from the image obtained by the affine transformation, and generates a code map (S222).

  The code part detection unit 54 detects the corner cells 104 at the four corners from the generated code map cells, and determines whether or not the surrounding three block areas are white pixels (S224). If the three block areas around the corner cell 104 are composed of white pixels (Y in S224), the code map is set as the code part 110 of the two-dimensional code (S230), and the process proceeds to S236. When the three block areas around the corner cell 104 are not white pixels, the process proceeds to S226.

  In S226, it is determined whether or not the count value l is equal to the total number M of black pixel connected regions. If the count value l is equal to M (Y in S226), the process proceeds to S232, and if not equal (N in S226), The count value l is incremented by 1 (S228), and the process returns to S212.

  In S232, it is determined whether or not the count value j is equal to the total number M of the black pixel connected regions. If the count value j is equal to M (Y in S232), the process proceeds to S236, and if not equal (N in S232), The count value j is incremented by 1 (S234), and the process returns to S204.

  In S236, it is determined whether or not the count value k is equal to the set total number N of reference cells 100. If the count value k is not equal to N (N in S236), the count value k is incremented by 1 (S238). , Return to S202. This makes it possible to search for a plurality of two-dimensional codes in the image data. When the count value k is equal to N (Y in S236), the detection process of the code unit 110 is finished as described above.

  Returning to FIG. 4, when the code part 110 is not detected in the code part detection process of S24 (N of S26), since the presence of the two-dimensional code cannot be confirmed, the two-dimensional code recognition process is terminated. On the other hand, when the code part 110 is detected (Y in S26), a code data verification process is executed (S28).

  FIG. 7 is a flowchart showing a code data verification process. As described above, 9-bit check data generated by a predetermined algorithm and 24-bit information data exist in 33-bit code data. In this process, the check data is used to verify the code data. To do. In the following, a process for verifying code data is performed by generating check data from code data based on the check data generated by a predetermined algorithm and collating the check data.

  First, the verification processing unit 56 initially sets a value p of a counter that counts the number of times the reference value calculated in subsequent steps S304 and S308 is shifted to the right by 1 bit to 1 (S300). Next, the verification processing unit 56 calculates code data and check data values from the code map of the code unit 110 (S302).

  The verification processing unit 56 performs an exclusive OR operation between the calculated code data value (bit stream) and 0xFFFFFF (S304), and uses the obtained value (bit stream) as a reference value (reference bit stream). And It is determined whether or not “1” is set in the LSB (Least Significant Bit) of the reference bitstream (S306). If it is determined that “1” is not set (N in S306), the verification processing unit 56 An exclusive OR operation of the reference value (reference bit stream) and 0x8408 is performed (S308), and the value (bit stream) obtained as a result is set as a new reference value (reference bit stream), and the process proceeds to S310. If “1” is in the LSB (Y in S306), the process proceeds to S310 in the same manner.

  The verification processing unit 56 shifts the reference value (reference bit stream) calculated in S304 or S308 to the right by 1 bit (S310), and determines whether or not the count value p is equal to 24 which is a predetermined number of shifts. Determination is made (S312). If it is determined that p is not 24 (N in S312), the count value p is incremented by 1 (S314), and the process returns to S306.

  When it is determined that p = 24 (Y in S312), a logical product operation of the calculated bit stream and 0x1FF is performed (S316). The verification processing unit 56 determines whether or not the value obtained by the logical product operation is equal to the calculated check data value (S318). If it is determined that the value is equal (Y in S318), it is detected in S24 of FIG. The code part 110 thus determined is an appropriate pattern as a two-dimensional code, and the code part 110 of the two-dimensional code is determined (S320). If it is not equal to the check code (N in S318), it is determined that there is an error in reading the code part 110, and the code data verification process is terminated. In FIG. 7, only the verification process for one code unit 110 is shown, but when a plurality of code units 110 are detected, this verification process is executed for each.

  Returning to FIG. 4, when the code part 110 is not fixed in the code data detection process of S28 (N of S30), since the presence of the two-dimensional code cannot be confirmed, the two-dimensional code recognition process is terminated. On the other hand, when the code part 110 is confirmed (Y in S30), the value of the code data, that is, the value of the two-dimensional code is stored in, for example, the RAM and held (S32), and the two-dimensional code recognition process is finished. To do.

  Hereinafter, a method for obtaining the position and direction of the game card 4 in the three-dimensional coordinate system will be described. In order to determine the position and direction of the virtual three-dimensional coordinate system of the game card 4, in this embodiment, the angle of view of the imaging device 2, the screen resolution, the screen coordinate positions of the four corner cells 104, and the actual corner cells 104 Use distance. The screen is a screen of the display 7 that displays an image captured by the imaging device 2.

(Step 1)
First, the distance from the viewpoint to the screen projection plane is obtained. Here, the distance from the viewpoint to the screen projection plane is obtained from the angle of view of the camera and the screen resolution.
Camera horizontal angle of view: θ
Screen horizontal resolution: W
Distance from viewpoint to screen projection plane: P
Then, the following calculation formula holds.
P = (W × 0.5) / tan (θ × 0.5)

(Step 2)
Next, a three-dimensional vector extending from the viewpoint to each corner cell 104 is obtained.
Corner cell screen coordinate position: SX, SY
Distance from viewpoint to screen projection plane: P
3D vector extending to the corner cell: V
Then, the following calculation formula holds.
V = (SX, SY, P)
Note that the screen coordinate position has the origin at the center of the screen.

(Step 3)
A normal vector of a plane formed by the viewpoint and three points of two adjacent corner cells is obtained. A total of four normal vectors are generated.
A three-dimensional vector extending from the viewpoint to the corner cell 104a: V1
A three-dimensional vector extending from the viewpoint to the corner cell 104b: V2
A three-dimensional vector extending from the viewpoint to the corner cell 104c: V3
A three-dimensional vector extending from the viewpoint to the corner cell 104d: V4
Normal vector of the plane formed by the viewpoint and the corner cells 104a and 104b: S12
Normal vector of the plane formed by the viewpoint and the corner cells 104b and 104c: S23
Normal vector of the plane formed by the viewpoint and the corner cells 104c and 104d: S34
Normal vector of the plane formed by the viewpoint and the corner cells 104d and 104a: S41
Then, the following calculation formula holds.
S12 = V1 × V2
S23 = V2 × V3
S34 = V3 × V4
S41 = V4 × V1
“X” indicates an outer product.

(Step 4)
Subsequently, the direction (coordinate axis) of the game card 4 in three dimensions is obtained. Based on the four normal vectors obtained in step 3, the coordinate axes of the card are obtained.
Card local coordinate X-axis (horizontal direction with respect to the card surface): VX
Card local coordinate Y-axis (direction penetrating the card surface): VY
Card local coordinate Z-axis (vertical direction with respect to the card surface): VZ
Then, the following calculation formula holds.
VX = S12 × S34
VZ = S23 × S41
VY = VZ × VX

(Step 5)
Finally, the three-dimensional position of the game card 4 is obtained. In step 4, the local coordinate transformation matrix of the game card 4 is obtained. If the actual distance between corner cells is known, the position of the card can be easily obtained. If the three-dimensional coordinate position of the corner cell 104a is expressed as a variable, the three-dimensional coordinate positions of the corner cells 104b, 104c, and 104d can be expressed using the variable. Since the screen coordinate positions of the four corner cells are known, the three-dimensional coordinate position of the game card 4 can be specified by solving the simultaneous equations of these calculation formulas.

  As shown in the above embodiment, by using the two-dimensional code shown in FIG. 2, the recognition rate of the corner cell 104 in the code unit 110 can be increased, and the coordinates of the corner cell 104 in the two-dimensional coordinate system are used. The position can be obtained accurately. Specifically, since the center position of the corner cell 104 can be accurately obtained by forming the corner cell 104 large, the position of the game card 4 can be accurately identified. In addition, by accurately determining the center positions of the plurality of corner cells 104, the positional relationship between the reference cell 100 and the corner cells 104 can be accurately grasped, and the accuracy of affine transformation can be improved. And the direction in which the game card 4 is facing can be accurately specified.

  FIG. 8 shows another example of the two-dimensional code printed on the surface of the game card. A reference cell 100 and a code portion 110 having a predetermined shape are arranged on the surface of the game card 4. Compared with the two-dimensional code of FIG. 2, in the two-dimensional code shown in FIG. 8, a plurality of corner cells 104a, 104b, 104c, 104d surrounding the code data portion 102 are formed in a circle. As described above, the two-dimensional coordinate position of the corner cell 104 is specified by the center point of the corner cell 104. Since the corner cell 104 has a round shape, the shape of the corner cell 104 imaged by the imaging device 2 does not change regardless of the orientation of the game card 4 with respect to the imaging device 2. For this reason, the center point of the corner cell 104 does not change depending on the orientation with respect to the imaging device 2, and thus the center point can be acquired stably. Thereby, as described above, the position information, direction information, and distance information of the game card 4 can be accurately obtained. Note that the square cell 106 may be formed in a round shape. In addition to making the corner cell 104 circular, as described with reference to FIG. 2, it is possible to obtain the coordinate position of the corner cell 104 with higher accuracy by forming it larger than the rectangular cell 106. .

  FIG. 9 shows still another example of the two-dimensional code printed on the surface of the game card. A reference cell 100 and a code portion 110 are arranged on the surface of the game card 4. Compared with the two-dimensional code of FIG. 2, in the two-dimensional code shown in FIG. 9, a reference cell 100 having a predetermined shape, a plurality of rectangular cells 106 constituting code data by being arranged two-dimensionally, and a code data portion 102 At least one cell of the plurality of corner cells 104 arranged so as to surround the region is colored differently from the other cells. For example, red may be assigned to the reference cell 100, green may be assigned to the square cell 106, blue may be assigned to the corner cell 104, and different colors may be assigned to the respective cells. The color assigned here may be any color that can be captured by the imaging apparatus 2 and binarized with a predetermined luminance threshold as a reference, and is not necessarily required to be visible to the human eye. When the imaging characteristics of the imaging apparatus 2 to be used are known in advance, the cells may be colored according to the characteristics.

  In the recognition process of the two-dimensional code of the present embodiment, as shown in S20 of FIG. Among the entire two-dimensional code recognition processes, the load of the reference cell detection process is high. Therefore, it is preferable that the reference cell 100 can be efficiently extracted from the image data. Therefore, it is preferable that at least the reference cell 100 among the plurality of types of cells is colored differently from the rectangular cell 106 and the corner cell 104.

  An RGB color filter is arranged in front of the light receiving surface of the imaging device 2, and each pixel value of RGB is recorded as data expressed in 256 gradations for each frame image. In this example, the binarization processing unit 50 has a function of setting a range of RGB pixel values for binarization processing. When the reference cell 100 is painted red, the binarization processing unit 50 sets a range of RGB pixel values for the reference cell in the binarization process of S10 of FIG. For example, the binarization processing unit 50 has a red (R) pixel value range of 200 to 255, a green (G) pixel value range of 0 to 100, and a blue (B) pixel value range of 0 to 0. Set to 100. Based on this setting, the binarization processing unit 50 extracts RGB composite pixels within this range, and encodes the pixel values to “1”. For example, the pixel value of the composite pixel in which the R pixel value is 225, the G pixel value is 30, and the B pixel value is 50 is encoded as “1”, and the color as the binary image is set to black. On the other hand, the pixel value of the composite pixel whose RGB pixel value is not within these ranges is encoded to “0”, and the color as the binary image is set to white. As described above, by extracting the red pixels from the image data, it is possible to label connected regions that are black pixels in the binary image in S12 of FIG. The number of noises to be extracted can be reduced by limiting the pixels to be extracted as black pixels to a specific color in the image data, in this case, red, and the reference cell detection unit 52 can extract the reference data from the binarized data of the frame image. The time for detecting the cell 100 can be shortened.

  Since the pixel value is affected by the color filter characteristics, the red pixel of the reference cell 100 is placed with the game card 4 placed on the table 3 in order to set a range of pixel values that also consider the color filter characteristics. The value may be investigated. As a result, the RGB pixel value of the reference cell 100 acquired in the imaging device 2 can be specified, and the black value to be detected by performing binarization processing in a range in which a slight margin is given to the specified pixel value. It is possible to dramatically reduce the number of pixel connection regions. In particular, in the game system 1, since it is preferable to perform processing in real time, there is a great merit that the reference cell 100 can be efficiently extracted.

  The same applies to the case where other types of cells, for example, the corner cell 104 and the rectangular cell 106 are colored. When the corner cell 104 is blue, in S10 of FIG. 4, the binarization processing unit 50 sets a range of RGB pixel values for the corner cell. For example, the binarization processing unit 50 has a red (R) pixel value range of 0 to 100, a green (G) pixel value range of 0 to 100, and a blue (B) pixel value range of 200 to 200. Set to 255. Based on this setting, the binarization processing unit 50 extracts RGB composite pixels within this range, and encodes the pixel values to “1”. For example, the pixel value of the composite pixel in which the R pixel value is 20, the G pixel value is 30, and the B pixel value is 240 is encoded as “1”, and the color as the binary image is set to black. On the other hand, the pixel value of the composite pixel whose RGB pixel value is not within these ranges is encoded to “0”, and the color as the binary image is set to white. In this way, by extracting blue pixels from the image data, it is possible to label connected regions that are black pixels in the binary image in S12 of FIG.

  In this way, after performing the binarization processing for converting a strong blue image into a black image, the code portion detection unit 54 performs the corner shown in the flow of FIG. Used to select as a cell candidate area. The number of noises to be extracted can be reduced by limiting the pixels to be extracted as black pixels to a specific color in the image data, in this case, blue, and the code portion detection unit 54 can calculate the corner from the binarized data of the frame image. The time for detecting the cell 104 can be shortened. When the square cell 106 is green, the binarization process for converting a strong green image into a black image in S10 in FIG. 4 is performed, and then the black pixel connected region in S12 is labeled as shown in FIG. This is used for the code map generation process of S222. When three types of cells are colored differently, the number of times of binarization processing increases. However, since the processing amount for searching for a connected region after binarization processing is greatly reduced, Efficiency can be realized.

  The two-dimensional code shown in FIG. 9 has colored cells, and the colored cells may be formed large, or may be formed round as shown in FIG.

  FIG. 10 shows an image data transmission system in the embodiment. The image data transmission system 200 includes an image data generation device 201 that generates image data, a transmission station 204 that transmits image data, and a terminal device 202 that receives the transmitted image data. The terminal device 202 has a display device 203 for displaying the received image data. The image data transmission system 200 transmits data by wireless communication, but may transmit data by wired communication.

  The image data generation device 201 generates the above-described two-dimensional code, that is, the image data of the two-dimensional code described with reference to FIGS. 2, 8, and 9 according to a data format for displaying on the display device 203 of the terminal device 202. To do. The image data generation device 201 has, as the image data of the two-dimensional code shown in FIG. 2, data for displaying a plurality of rectangular cells at predetermined coordinate positions on the display device 203, and a plurality of data displayed at the predetermined coordinate positions. Data for displaying a plurality of corner cells at coordinate positions surrounding the rectangular cell may be generated. The image data generation device 201 sets the corner cell and square cell data so that the corner cell is displayed larger than the square cell on the display device 203.

  The image data generation device 201 has, as the image data of the two-dimensional code shown in FIG. 8, data for displaying a plurality of rectangular cells at predetermined coordinate positions on the display device 203, and a plurality of data displayed at the predetermined coordinate positions. Data for displaying a plurality of corner cells at coordinate positions surrounding the rectangular cell may be generated. The image data generation device 201 sets the corner cell data so that the corner cell is displayed in a round shape.

  The image data generation device 201 has, as the image data of the two-dimensional code shown in FIG. 9, data for displaying a reference cell having a predetermined shape at a predetermined position on the display device, and an area defined for the reference cell Data for displaying a plurality of polygonal cells within the range of and a data for displaying a plurality of corner cells at coordinate positions surrounding the region may be generated. The image data generation device 201 sets the color of at least one of the reference cell, the polygonal cell, and the corner cell to be different from the colors of the other cells.

  The transmission station 204 transmits the image data generated by the image data generation device 201 to the terminal device 202. The image data is preferably compressed in a format that can be decompressed by the terminal device 202, and needs to be created in a data format displayed by a browser function or the like in the terminal device 202. The transmission station 204 can reduce the amount of data transmission by transmitting image data for each cell in association with the display coordinate position, for example. The terminal device 202 displays a two-dimensional code on the display device 203. In the game system 1 of FIG. 1, the user can use the two-dimensional code displayed on the display device 203 instead of the game card 4 to be read by the imaging device 2. Thereby, even if the user does not have the game card 4, the user can participate in the game system 1 by downloading the image data to the terminal device 202.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

It is a figure which shows the structure of the game system concerning an Example. It is a figure which shows the two-dimensional code printed on the surface of the game card. It is a figure which shows the structure of an image analyzer. It is a flowchart which shows the process sequence of the recognition process of a two-dimensional code. It is a flowchart which shows the detection process of a reference | standard cell. It is a flowchart which shows the detection process of a code part. It is a flowchart which shows the verification process of code data. It is a figure which shows another example of the two-dimensional code printed on the surface of the game card. It is a figure which shows another example of the two-dimensional code printed on the surface of the game card. It is a figure which shows the transmission system of the image data in an Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Game system, 2 ... Imaging device, 4 ... Game card, 6 ... Output device, 7 ... Display, 10 ... Image processing device, 20 ... Image analysis device, DESCRIPTION OF SYMBOLS 30 ... Game device, 40 ... Frame image acquisition part, 42 ... Real object extraction part, 44 ... State determination part, 50 ... Binarization processing part, 52 ... Reference cell detection Part 54... Code part detection part 56 .. verification processing part 100... Reference cell 102. Code data part 104. Corner cell 106. .. Area 110 ... Code part 200 ... Image data transmission system 201 ... Image data generation device 202 ... Terminal device 203 ... Display device 204 ... Transmission station

Claims (6)

  A rectangular reference cell, a plurality of square cells constituting code data, and four corner cells of the same color having an area larger than that of the rectangular cell and colored differently from the reference cell; The card is imaged by the imaging device, and the code data is obtained from the captured frame image by the image analysis device,
  The reference cell is detected from binarized data in which a binarization processing unit in the image analysis apparatus sets a range of RGB pixel values and binarizes the frame image.
  The corner cell performs binarization processing of the frame image by setting a range of RGB pixel values different from the range of RGB pixel values set for the detection of the reference cell by the binarization processing unit. Detected from the binarized data,
  The plurality of rectangular cells are obtained by reading the arrangement of the plurality of rectangular cells by an image analysis device using the detected corner cells, thereby obtaining code data.
  A card characterized by that.   The card according to claim 1, wherein the corner cell is colored differently from the square cell.   Image data generated in accordance with a data format for displaying a two-dimensional code on a display device,
  Data for displaying a rectangular reference cell at a predetermined first coordinate position on the display device;
  Data for displaying a plurality of square-shaped square cells constituting the code data at a predetermined second coordinate position on the display device;
  Data for displaying four corner cells of the same color having a larger area than the reference cell and a color different from the reference cell at a predetermined third coordinate position on a display device,
  Reference cells, square cells, and corner cells displayed at predetermined positions on the display device are captured by the imaging device, and code data is acquired by the image analysis device from the captured frame image.
  The reference cell is detected from binarized data in which a binarization processing unit in the image analysis apparatus sets a range of RGB pixel values and binarizes the frame image.
  The corner cell performs binarization processing of the frame image by setting a range of RGB pixel values different from the range of RGB pixel values set for the detection of the reference cell by the binarization processing unit. Detected from the binarized data,
  The plurality of rectangular cells are obtained by reading the arrangement of the plurality of rectangular cells by an image analysis device using the detected corner cells, thereby obtaining code data.
  Image data characterized by that.   A method of transmitting image data generated according to a data format for displaying a two-dimensional code on a display device, wherein the image data to be transmitted is
  Data for displaying a rectangular reference cell at a predetermined first coordinate position on the display device;
  Data for displaying a plurality of square-shaped square cells constituting the code data at a predetermined second coordinate position on the display device;
  Data for displaying four corner cells of the same color having a larger area than the reference cell and a color different from the reference cell at a predetermined third coordinate position on a display device,
  Reference cells, square cells, and corner cells displayed at predetermined positions on the display device are captured by the imaging device, and code data is acquired by the image analysis device from the captured frame image.
  The reference cell is detected from binarized data in which a binarization processing unit in the image analysis apparatus sets a range of RGB pixel values and binarizes the frame image.
  The corner cell performs binarization processing of the frame image by setting a range of RGB pixel values different from the range of RGB pixel values set for the detection of the reference cell by the binarization processing unit. Detected from the binarized data,
  The plurality of rectangular cells are obtained by reading the arrangement of the plurality of rectangular cells by an image analysis device using the detected corner cells, thereby obtaining code data.
  An image data transmission method characterized by the above.   An image analysis system comprising the card according to claim 1 and an image analysis device that analyzes a frame image obtained by imaging the card,
  The image analyzer is
  A binarization processing unit that sets a range of RGB pixel values and converts a frame image into a binary bit representation;
  A first detector for detecting a reference cell from the binarized data of the frame image;
  A second detector for detecting a corner cell from the binarized data of the frame image,
  The binarization processing unit sets a range of RGB pixel values for the reference cell, extracts pixels within the set pixel value range, and performs a binarization process of the frame image. The first detector detects the reference cell based on the value data,
  The binarization processing unit sets a range of RGB pixel values for the corner cell, extracts pixels within the set pixel value range, performs binarization processing of the frame image, On the premise that the corner cell is larger than the rectangular cell, the second detection unit detects the corner cell based on the binarized data.
  An image analysis system characterized by this.   An image analysis system comprising the card according to claim 1 and an image analysis device that analyzes a frame image obtained by imaging the card,
  The image analyzer is
  A binarization processing unit that sets a range of RGB pixel values and converts a frame image into a binary bit representation;
  A first detector for detecting a reference cell from the binarized data of the frame image;
  A second detector for detecting a corner cell from the binarized data of the frame image,
  The binarization processing unit sets a range of RGB pixel values for the reference cell, extracts pixels within the set pixel value range, and performs a binarization process of the frame image. The first detector detects the reference cell based on the value data,
  The binarization processing unit sets a range of RGB pixel values for the corner cell, extracts pixels within the set pixel value range, performs binarization processing of the frame image, On the assumption that the color of the corner cell is different from the color of the square cell, the second detection unit detects the corner cell based on the binarized data.
  An image analysis system characterized by this.

Images