JP4364186B2 - card - Google Patents

Description

  The present invention relates to a technique for expressing code data by two-dimensionally arranging a plurality of cells.

In recent years, a technique for recognizing code data by capturing a two-dimensional code with a camera and executing a predetermined process associated with the code data has become widespread. Compared to one-dimensional barcodes, two-dimensional codes have more information that can be encoded, and various types of two-dimensional codes are now available. The code data of the two-dimensional code needs to be read efficiently and accurately from the image data. In view of such a situation, there has been proposed a technique related to image recognition of a two-dimensional code (see, for example, Patent Document 1).
JP 2000-82108 A

If the resolution of the image data is high, a large amount of information of the two-dimensional code can be acquired, and a high recognition rate of the code data can be realized. However, some cameras have low resolution, and even in such a case, it is preferable to increase the recognition rate of code data. Further, since the image data acquired by imaging includes the influence of external environmental light or the like, the binarized pixels are taken into account when the acquired image data is binarized. It is preferable to set the range of values suitably.
An object of the present invention is to provide an object such as a card that can be recognized with high accuracy. It is another object of the present invention to provide a technique for suitably setting a pixel value range when binarizing image data.

  In order to solve the above-described problem, an object according to an aspect of the present invention includes a reference cell having a predetermined shape, a plurality of polygonal cells that form code data by two-dimensionally arranging each, and the same shape. A plurality of corner cells. The corner cell is colored differently than the reference cell and the polygonal cell. This object may be a two-dimensional object such as a card or a three-dimensional object.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, a two-dimensional code that can be recognized with high accuracy can be provided.

  FIG. 1 shows a configuration of a game system 1 according to an embodiment of the present invention. The game system 1 includes an imaging device 2, an image processing device 10, and an output device 6. The image processing device 10 includes an image analysis device 20 and a game device 30. The image analysis device 20 and the game device 30 may be configured as separate devices, but may be configured as a single unit. The imaging device 2 is a video camera configured by a CCD imaging device or a MOS imaging device, and has a resolution of 320 pixels × 240 pixels, for example. The imaging device 2 images the real space at a predetermined cycle and generates a frame image for each cycle. The imaging region 5 is a range where the imaging device 2 captures an image, and the position and size of the imaging region 5 are adjusted by adjusting the height and direction of the imaging device 2. The game player moves the game card 4 that is a real object with a finger in the imaging area 5. The game card 4 has a two-dimensional code for uniquely identifying itself.

  The output device 6 includes a display 7 that is a display device. The output device 6 may further include a speaker (not shown). Basically, the image processing apparatus 10 displays the frame image captured by the imaging apparatus 2 on the display 7, and controls display so that a character that is a virtual object is superimposed on the game card 4. The player can easily check whether or not the game card 4 is in the imaging area 5 by looking at the display 7. If not, the player can shift the position of the game card 4 or adjust the orientation of the imaging device 2. Then, the imaging device 2 is caused to image the game card 4.

  In the game application of the embodiment, the player operates the character by performing a predetermined operation on the game card 4. Due to the nature of the game application, it is preferable that the player recognize the unity between the game card 4 and the character, and therefore, the character image is displayed superimposed on the game card 4. When the game card 4 is slowly moved within the imaging area 5, the character moves together following the movement of the game card 4 while maintaining the state of riding on the game card 4.

  The movement of the above characters is controlled by the image processing apparatus 10. First, the image analysis device 20 extracts image information of the game card 4 from the frame image acquired by the imaging device 2. Further, the image analysis device 20 extracts a two-dimensional code printed on the game card 4 from the image information of the game card 4. At this time, the image analysis device 20 determines position information, direction information, distance information, and the like in the space of the game card 4 from the two-dimensional code of the game card 4.

  As will be described later, the image analysis device 20 uses the reference cells and the four corner cells in the image of the game card 4 in the frame image to determine the distance between the imaging device 2 and the game card 4, the orientation of the game card 4, and the like. Calculate to find. As a result, the character is controlled to face forward on the game card 4. Further, the code data of the game card 4 is acquired from the array of a plurality of rectangular cells in the two-dimensional code.

  FIG. 1 shows a state in which the game card 4 is simply placed on the table 3, but the game card 4 is tilted with respect to the table 3, for example, or lifted above the table 3. May be. The image analysis device 20 has a function of recognizing a state where the game card 4 is tilted or a state where the height from the table 3 is changed by image analysis. The result of the image analysis performed by the image analysis device 20 is sent to the game device 30. Note that the frame image captured by the imaging device 2 may be sent to the game device 30 and image analysis performed on the game device 30 side. In this case, the image processing apparatus 10 is configured by only the game apparatus 30.

  The game apparatus 30 performs display control so that the character is positioned on the game card 4 on the display screen of the display 7 based on the image analysis result in the image analysis apparatus 20. The character may be associated with the code data extracted from the game card 4 and appropriately assigned for each game scene. In this case, when the game scene is switched, the displayed character is also switched.

  FIG. 2 shows a two-dimensional code printed on the surface of the game card. A plurality of polygonal cells having a predetermined shape are two-dimensionally arranged on the surface of the game card 4 according to a predetermined arrangement rule. Specifically, the two-dimensional code of the game card 4 is a two-dimensional arrangement of a reference cell 100 and a plurality of rectangular cells 106 serving as a reference for performing recognition processing of the game card 4 from image data captured by the imaging device 2. The code data portion 102 constituting the code data, and a plurality of corner cells 104a, 104b, 104c, 104d arranged so as to surround the code data portion 102 (hereinafter collectively referred to as “corner cell 104”) It has. The code data portion 102 and the corner cell 104 exist in the code portion 110 in the two-dimensional code.

  The reference cell 100, the square cell 106, and the corner cell 104 are each configured to have a predetermined shape. In the two-dimensional code shown in FIG. 2, each cell is painted black. In FIG. 2, a region 108 is configured by connecting a plurality of rectangular cells 106, and constitutes code data together with other rectangular cells 106. The rectangular cells 106 have the same shape, here a square shape. When there are a plurality of game cards 4 in the embodiment, the arrangement and size of the reference cell 100 and the corner cell 104 are common, and the game card 4 is uniquely specified by the arrangement of the rectangular cells 106 in the code data portion 102. be able to.

  Details of the two-dimensional code in this embodiment will be described below. Assuming that one side of the square-shaped square cell 106 is one block, the reference cell 100 is a rectangular cell (with a long side in the horizontal direction of 7 blocks and a short side in the vertical direction of 1.5 blocks ( Element). In the embodiment, a square area formed by one vertical block and one horizontal block is called a block area.

  The code data portion 102 is formed in an area surrounded by 7 blocks × 7 blocks. One side of this region is parallel to the long side of the reference cell 100 and is arranged at a position one block away from the long side. As shown in the figure, the code part 110 is an area surrounded by 8 blocks × 8 blocks, and the code data part 102 is included in the area where the code part 110 is formed. The area where the square cells 106 are actually arranged is a portion excluding the area of 2 blocks × 2 blocks at the four corners of the area surrounded by 7 blocks × 7 blocks. That is, the rectangular cell 106 is not arranged in the area of 2 blocks × 2 blocks at the four corners. Therefore, the rectangular cell 106 is arranged in the block area of (7 × 7−2 × 2 × 4) = 33, Configure code data.

  The corner cell 104 a is disposed in the upper left corner area of the code portion 110. This upper left corner area may be within the range of the area of 2 blocks × 2 blocks at the upper left corner of the area surrounded by 7 blocks × 7 blocks as the code data portion 102, or may extend beyond the range. In the example shown in the figure, the corner cell 104a is arranged to protrude 0.5 blocks vertically and 0.5 blocks horizontally from the 2 block × 2 block region in the upper left corner of the code data portion 102. That is, the corner cell 104a exists in the upper left corner region surrounded by 2.5 blocks × 2.5 blocks in the code portion 110. Similarly, the corner cell 104b is disposed in the upper right corner region of the code portion 110, the corner cell 104c is disposed in the lower right corner region of the code portion 110, and the corner cell 104d is disposed in the lower left corner region of the code portion 110. The In the example shown in the drawing, each corner cell 104 is formed in a square shape, specifically, a square cell with one side having a length of 1.5 blocks.

  In the code data portion 102, if one block area is 1 bit, information of 33 bits in total can be encoded. Of these 33 bits, 9 bits constitute check data for confirming that the code data is correct code data. Therefore, information for 24 bits is encoded in the code data portion 102.

  The rectangular cell 106 is an important element that expresses code data, and needs to be accurately recognized by the image analysis apparatus 20. For that purpose, it is ideal to make all the square cells 106 large, but naturally the size of the game card 4 is limited, and the enlargement of the square cells 106 is equivalent to the code data portion. As a result, the number of rectangular cells 106 constituting the number 102 is reduced. This corresponds to reducing the number of bits of information and is not preferable.

  From the above, in this embodiment, the corner cell 104 is configured to have a larger area than the square cell 106. As will be described later, in the image analysis apparatus 20 of this embodiment, the corner cells 104 at the four corners are used to detect the code data portion 102. In other words, if the corner cells 104 at the four corners cannot be detected, the square cell 106 cannot be detected. Therefore, in this two-dimensional code, it is necessary to first reliably recognize the corner cells 104 rather than the square cells 106. is there. By forming the four corner cells 104 larger than the square cells 106, the recognition rate of the four corner cells 104 can be increased.

  The image analysis apparatus 20 extracts candidates for the reference cell 100 in the frame image and assumes a reference cell 100 that satisfies a predetermined reference, and there are four corner cells 104 in the vicinity of the assumed reference cell 100. It is determined whether or not. When the four corner cells 104 are recognized, the image analysis apparatus 20 reads the area surrounded by the four corner cells 104, that is, the arrangement of the rectangular cells 106 in the code data portion 102, and acquires code data. Thereby, it is determined that the assumed reference cell 100 is the true reference cell 100, and the game card 4 can be recognized. The image analysis device 20 calculates and obtains the position and direction of the game card 4 in the virtual three-dimensional coordinate system with the imaging device 2 as a reference (origin).

  FIG. 3 shows the configuration of the image analysis apparatus. The image analysis device 20 includes a frame image acquisition unit 40, a real object extraction unit 42, and a state determination unit 44. The real object extraction unit 42 includes a binarization processing unit 50, a reference cell detection unit 52, a code unit detection unit 54, and a verification processing unit 56. The processing function of the image analysis apparatus 20 in the present embodiment is realized by a CPU, a memory, a program loaded in the memory, and the like, and here, a configuration realized by cooperation thereof is illustrated. The program may be built in the image analysis apparatus 20 or may be supplied from the outside in a form stored in a recording medium. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof. In the illustrated example, the CPU of the image analysis device 20 has functions as a frame image acquisition unit 40, a real object extraction unit 42, and a state determination unit 44.

  The frame image acquisition unit 40 acquires a real space frame image captured by the imaging device 2. The imaging device 2 periodically acquires frame images, and preferably generates frame images at 1/60 second intervals.

  The real object extraction unit 42 extracts a real object image, that is, an image of the game card 4 from the frame image. Specifically, the binarization processing unit 50 converts the frame image into binary bit representation, and expresses the frame image information by turning the bit on and off. The reference cell detection unit 52 detects the reference cell 100 from the binarized data of the frame image. When the reference cell 100 is detected, the code part detection unit 54 detects the code part 110 based on the position of the reference cell 100, and the verification processing unit 56 performs verification processing of the code data included in the code part 110.

  The state determination unit 44 determines the state of the real object in the set coordinate system, and specifically determines the position and direction of the game card 4 and the distance from the imaging device 2. The position information, direction information, and distance information determined by the state determination unit 44 are respectively associated with code data acquired from the two-dimensional code and sent to the game apparatus 30. When a plurality of game cards 4 are present in the imaging area 5, position information, direction information, distance information, and code data are associated with each game card 4 and sent to the game apparatus 30. In the game system 1, the frame image itself is also sent to the game device 30 in order to display the frame image captured by the imaging device 2 on the display 7. The game apparatus 30 reads the character associated with the code data and displays the game card 4 and the character superimposed on each other in a three-dimensional virtual space.

  FIG. 4 is a flowchart showing the processing procedure of the recognition process of the two-dimensional code. The two-dimensional code is imaged by the imaging device 2, and image data for one frame obtained as a result is acquired by the frame image acquisition unit 40. The binarization processing unit 50 performs binarization processing on the image data (S10). In the binarization process, the pixel value of a pixel holding a luminance greater than a predetermined threshold is encoded to “0”, and the pixel is white on display. Hereinafter, a pixel whose pixel value is encoded as “0” in this way is referred to as a white pixel. On the other hand, the pixel value of the pixel holding the luminance value below the threshold is encoded as “1”, and the pixel is black on display. Hereinafter, a pixel whose pixel value is encoded as “1” in this way is referred to as a black pixel. The binarization processing unit 50 collects areas in which black pixels are continuously present as one connected area, and sets (labels) the extracted black pixel connected areas in ascending order from 1 in order. (S12). For example, the labeling numbers may be set in the order in which they are extracted as connected regions, that is, in the specified order. The number is stored in a RAM (not shown) together with the coordinate data. When the two-dimensional code shown in FIG. 2 is used, since the corner cell 104 is displayed larger than the square cell 106, the probability that the corner cell 104 can be detected as a black pixel connection region and the accuracy of the coordinate position can be increased. .

  In the two-dimensional code recognition process executed before the current target image data, the reference cell detection unit 52 displays the recognized position on the display of the two-dimensional code, for example, the center point of the reference cell 100 in the RAM. It is determined whether it is stored (S14). When the center point of the reference cell 100 in the previous frame is stored in the RAM (Y in S14), the stored position is set as the start point of the reference cell detection process (S16). When the reference cell 100 is detected in the previous image frame, the position of the game card 4 is often not changed greatly in the current image frame, so the center point of the reference cell 100 in the previous frame is used. Thus, the reference cell 100 can be efficiently detected in the current frame. On the other hand, when the position of the two-dimensional code is not stored in the RAM (N in S14), the reference cell detection unit 52 sets the center point on the display 7 as the start point of the reference cell detection process (S18). . After setting the starting point, the reference cell detection unit 52 executes the detection process of the reference cell 100 (S20).

  FIG. 5 is a flowchart showing reference cell detection processing. Note that the number of pixels shown in this flow depends on the resolution of the imaging device 2, and the number of pixels is shown as an example here for easy understanding. First, the reference cell detection unit 52 reads the total number M of black pixel connected regions labeled in S12 of FIG. 4 and initializes the counter value j to 1 (S100). Next, from the start point set in S16 or S18 of FIG. 4, the black pixel connection area is searched along the trace of the counterclockwise spiral on the screen of the display 7, and the black pixel connection area detected first is used as a reference. A cell candidate area is selected (S102). The reference cell detection unit 52 extracts the edge portion of the selected reference cell candidate region, and determines the short side and the long side (S104).

  The reference cell detection unit 52 determines whether or not the short side includes a predetermined number of pixels, for example, 20 pixels or more (S106). The short side is configured with a length corresponding to 1.5 blocks, but when the short side is the reference cell 100 configured with less than 20 pixels, the length of one block is a smaller number of pixels. It will be composed of. Therefore, one side of the rectangular cell 106, which is the smallest component in the two-dimensional code, is about 13 pixels or less, and the image pickup apparatus 2 cannot capture images appropriately. From this, when it is determined that the short side is configured by fewer than 20 pixels (N in S106), it is determined that the black pixel connection region selected in S102 is not the reference cell 100. The process proceeds to S116.

  When it is determined that the short side is composed of 20 or more pixels (Y in S106), the reference cell detection unit 52 uses a pixel whose long side of the reference cell candidate region is a predetermined number of pixels, for example, 300 pixels or less. It is determined whether or not it is configured (S108). For example, if the reference cell 100 is composed of more than 300 pixels on the long side, the length of one block obtained at a ratio of 1 to 7 on the long side becomes large. The lower right corner cell 104c and the lower left corner cell 104d located 8.5 blocks apart from 100 are no longer imaged by the imaging device 2. From this, when it is determined that the long side is composed of more than 300 pixels (N in S108), it is determined that the black pixel connection region selected in S102 is not the reference cell 100. The process proceeds to S116.

  When it is determined that the long side is composed of pixels of 300 pixels or less (Y in S108), the reference cell detection unit 52 determines that the total number of black pixels in the reference cell candidate region is 20 pixels or more and less than 1500 pixels. It is determined whether or not there is (S110). When the total number of black pixels is less than 20 pixels (N in S110), the same problem occurs when the short side is composed of less than 20 pixels, and when the total number is 1500 pixels or more (N in S110). The same problem occurs when the long side is composed of more than 300 pixels. Therefore, in these cases, it is determined that the possibility that the reference cell candidate region is the reference cell 100 is small, and the process proceeds to S116.

  When the total number of black pixels in the reference cell candidate region is 20 pixels or more and less than 1500 pixels (Y in S110), the reference cell detection unit 52 determines the squareness of the reference cell candidate region (S112). The squareness is obtained based on the moment of the reference cell candidate region as described in Patent Document 1 described above. When it is determined that it is not a square (N in S112), it is determined that the reference cell candidate region is not the reference cell 100, and the process proceeds to S116. When it is determined that the pixel is a square (Y in S112), the reference cell detection unit 52 sets the reference cell candidate region as the reference cell 100 (S114), and the labeling number of the black pixel connection region that is set as the reference cell candidate region Is stored in the RAM. In this way, the reference cell 100 can be detected. When only one game card 4 is included in the image data, the reference cell detection process ends when the reference cell 100 is detected. On the other hand, if there is a possibility that a plurality of game cards 4 are included, the process proceeds to S116.

  The reference cell detection unit 52 determines whether or not the count value j is equal to the total number M of reference cell candidate areas (S116). If the count value j has not reached the total number M (N in S116), the count value j is incremented by 1 (S118), and the process returns to the step of S102. On the other hand, when the count value j reaches the total number M (Y in S116), the reference cell detection process is terminated.

  Returning to FIG. 4, when the reference cell 100 is not detected in the reference cell detection process of S20 (N of S22), since the presence of the game card 4 cannot be confirmed, the two-dimensional code recognition process is terminated. On the other hand, when the reference cell 100 is detected (Y in S22), a code part detection process is executed (S24).

  FIG. 6 is a flowchart showing the code portion detection process. First, the code part detection unit 54 reads the total number N of the reference cells 100 set in S114 of FIG. 5, and initializes the counter value k to 1 (S200). Note that the reference cells 100 set in S114 are numbered in the order set. Subsequently, the total number M of the black pixel connected regions labeled in S12 of FIG. 4 is read, and the counter value j is initialized to 1 (S202). The code part detection part 54 detects the black pixel connection area of the number corresponding to the count value j, and selects it as a corner cell candidate area in the lower left corner (S204).

  Next, the code portion detection unit 54 detects that the selected lower left corner cell candidate region is within the search range set in advance for the reference cell 100 of the number corresponding to the count value k detected in S20 of FIG. It is determined whether or not it exists (S206). Since the positional relationship between the reference cell 100 and the corner cell 104d at the lower left corner is predetermined as shown in FIG. 2, the search range for the lower left corner cell can be narrowed down from the coordinate data of the reference cell 100. is there. Since the reference cell 100 has a horizontally long rectangular shape, it is possible to set a search range at two positions facing each other across the long side. When it does not exist within the search range (N in S206), it is determined that the black pixel connected region is not the lower left corner cell 104d, and the process proceeds to S232. If it exists within the search range (Y in S206), the black pixel connection region is set as the lower left corner cell 104d (S208).

  After setting the lower left corner cell 104, the code portion detection unit 54 initializes the value l of another counter for counting the number of the black pixel connection region to 1 (S210). The code part detection part 54 detects the black pixel connection area of the number corresponding to the count value l and selects it as a corner cell candidate area in the lower right corner (S212).

  Next, the code part detection unit 54 calculates the ratio between the number of pixels (area) of the lower left corner cell 104d set in S208 and the number of pixels (area) of the lower right corner cell candidate region selected in S212, and the ratio It is determined whether (area ratio) is, for example, 1/6 or more or 6 times or less (S214). In the corner cell 104, the number of pixels acquired changes according to the distance from the imaging device 2. Specifically, it has a large area if it is close to the imaging device 2, and the area is small if it is far away. Therefore, it is assumed that the areas in the image data are different even in the corner cells. However, for example, a case where the ratio of the lower left corner cell to the lower right corner cell is smaller than 1/6 or larger than 6 times (N in S214) cannot normally be assumed. Cannot be regarded as a corner cell in step S226, and the process proceeds to step S226.

  When the area ratio is, for example, 1/6 or more and 6 times or less (Y in S214), the code part detection unit 54 determines the center point of the lower left corner cell 104d and the center point of the lower right corner cell candidate region selected in S212. It is determined whether or not the distance of (2) satisfies a predetermined condition (S216). Here, the predetermined condition may be that the distance between the center points approximates the length of the long side of the reference cell 100, for example. If the predetermined condition is not satisfied (N in S216), the process proceeds to S226, and if the predetermined condition is satisfied (Y in S216), the code portion detection unit 54 sets the black pixel connection region as the lower right corner cell 104c. (S218).

  When the lower left corner cell 104d and the lower right corner cell 104c are set as described above, the code part detection unit 54 changes the set reference cell 100, lower left corner cell 104d, and lower right corner cell 104c to X on the screen of the display 7. Affine transformation is performed in the axial direction and the Y-axis direction (S220). The length of one block is calculated based on the length of the long side or short side of the reference cell 100 set in S20. The code part detection part 54 maps the black pixel connection area included in the area of the code part 110 as a cell from the image obtained by the affine transformation, and generates a code map (S222).

  The code part detection unit 54 detects the corner cells 104 at the four corners from the generated code map cells, and determines whether or not the surrounding three block areas are white pixels (S224). If the three block areas around the corner cell 104 are composed of white pixels (Y in S224), the code map is set as the code part 110 of the two-dimensional code (S230), and the process proceeds to S236. When the three block areas around the corner cell 104 are not white pixels, the process proceeds to S226.

  In S226, it is determined whether or not the count value l is equal to the total number M of black pixel connected regions. If the count value l is equal to M (Y in S226), the process proceeds to S232, and if not equal (N in S226), The count value l is incremented by 1 (S228), and the process returns to S212.

  In S232, it is determined whether or not the count value j is equal to the total number M of the black pixel connected regions. If the count value j is equal to M (Y in S232), the process proceeds to S236, and if not equal (N in S232), The count value j is incremented by 1 (S234), and the process returns to S204.

  In S236, it is determined whether or not the count value k is equal to the set total number N of reference cells 100. If the count value k is not equal to N (N in S236), the count value k is incremented by 1 (S238). , Return to S202. This makes it possible to search for a plurality of two-dimensional codes in the image data. When the count value k is equal to N (Y in S236), the detection process of the code unit 110 is finished as described above.

  Returning to FIG. 4, when the code part 110 is not detected in the code part detection process of S24 (N of S26), since the presence of the two-dimensional code cannot be confirmed, the two-dimensional code recognition process is terminated. On the other hand, when the code part 110 is detected (Y in S26), a code data verification process is executed (S28).

  FIG. 7 is a flowchart showing a code data verification process. As described above, 9-bit check data generated by a predetermined algorithm and 24-bit information data exist in 33-bit code data. In this process, the check data is used to verify the code data. To do. In the following, a process for verifying code data is performed by generating check data from code data based on the check data generated by a predetermined algorithm and collating the check data.

  First, the verification processing unit 56 initially sets a value p of a counter that counts the number of times the reference value calculated in subsequent steps S304 and S308 is shifted to the right by 1 bit to 1 (S300). Next, the verification processing unit 56 calculates code data and check data values from the code map of the code unit 110 (S302).

  The verification processing unit 56 performs an exclusive OR operation between the calculated code data value (bit stream) and 0xFFFFFF (S304), and uses the obtained value (bit stream) as a reference value (reference bit stream). And It is determined whether or not “1” is set in the LSB (Least Significant Bit) of the reference bitstream (S306). If it is determined that “1” is not set (N in S306), the verification processing unit 56 An exclusive OR operation of the reference value (reference bit stream) and 0x8408 is performed (S308), and the value (bit stream) obtained as a result is set as a new reference value (reference bit stream), and the process proceeds to S310. If “1” is in the LSB (Y in S306), the process proceeds to S310 in the same manner.

  The verification processing unit 56 shifts the reference value (reference bit stream) calculated in S304 or S308 to the right by 1 bit (S310), and determines whether or not the count value p is equal to 24 which is a predetermined number of shifts. Determination is made (S312). If it is determined that p is not 24 (N in S312), the count value p is incremented by 1 (S314), and the process returns to S306.

  When it is determined that p = 24 (Y in S312), a logical product operation of the calculated bit stream and 0x1FF is performed (S316). The verification processing unit 56 determines whether or not the value obtained by the logical product operation is equal to the calculated check data value (S318). If it is determined that the value is equal (Y in S318), it is detected in S24 of FIG. The code part 110 thus determined is an appropriate pattern as a two-dimensional code, and the code part 110 of the two-dimensional code is determined (S320). If it is not equal to the check code (N in S318), it is determined that there is an error in reading the code part 110, and the code data verification process is terminated. In FIG. 7, only the verification process for one code unit 110 is shown, but when a plurality of code units 110 are detected, this verification process is executed for each.

  Returning to FIG. 4, when the code part 110 is not fixed in the code data detection process of S28 (N of S30), since the presence of the two-dimensional code cannot be confirmed, the two-dimensional code recognition process is terminated. On the other hand, when the code part 110 is confirmed (Y in S30), the value of the code data, that is, the value of the two-dimensional code is stored in, for example, the RAM and held (S32), and the two-dimensional code recognition process is finished. To do.

  Hereinafter, a method for obtaining the position and direction of the game card 4 in the three-dimensional coordinate system will be described. In order to determine the position and direction of the virtual three-dimensional coordinate system of the game card 4, in this embodiment, the angle of view of the imaging device 2, the screen resolution, the screen coordinate positions of the four corner cells 104, and the actual corner cells 104 Use distance. The screen is a screen of the display 7 that displays an image captured by the imaging device 2.

(Step 1)
First, the distance from the viewpoint to the screen projection plane is obtained. Here, the distance from the viewpoint to the screen projection plane is obtained from the angle of view of the camera and the screen resolution.
Camera horizontal angle of view: θ
Screen horizontal resolution: W
Distance from viewpoint to screen projection plane: P
Then, the following calculation formula holds.
P = (W × 0.5) / tan (θ × 0.5)

(Step 2)
Next, a three-dimensional vector extending from the viewpoint to each corner cell 104 is obtained.
Corner cell screen coordinate position: SX, SY
Distance from viewpoint to screen projection plane: P
3D vector extending to the corner cell: V
Then, the following calculation formula holds.
V = (SX, SY, P)
Note that the screen coordinate position has the origin at the center of the screen.

(Step 3)
A normal vector of a plane formed by the viewpoint and three points of two adjacent corner cells is obtained. A total of four normal vectors are generated.
A three-dimensional vector extending from the viewpoint to the corner cell 104a: V1
A three-dimensional vector extending from the viewpoint to the corner cell 104b: V2
A three-dimensional vector extending from the viewpoint to the corner cell 104c: V3
A three-dimensional vector extending from the viewpoint to the corner cell 104d: V4
Normal vector of the plane formed by the viewpoint and the corner cells 104a and 104b: S12
Normal vector of the plane formed by the viewpoint and the corner cells 104b and 104c: S23
Normal vector of the plane formed by the viewpoint and the corner cells 104c and 104d: S34
Normal vector of the plane formed by the viewpoint and the corner cells 104d and 104a: S41
Then, the following calculation formula holds.
S12 = V1 × V2
S23 = V2 × V3
S34 = V3 × V4
S41 = V4 × V1
“X” indicates an outer product.

(Step 4)
Subsequently, the direction (coordinate axis) of the game card 4 in three dimensions is obtained. Based on the four normal vectors obtained in step 3, the coordinate axes of the card are obtained.
Card local coordinate X-axis (horizontal direction with respect to the card surface): VX
Card local coordinate Y-axis (direction penetrating the card surface): VY
Card local coordinate Z-axis (vertical direction with respect to the card surface): VZ
Then, the following calculation formula holds.
VX = S12 × S34
VZ = S23 × S41
VY = VZ × VX

(Step 5)
Finally, the three-dimensional position of the game card 4 is obtained. In step 4, the local coordinate transformation matrix of the game card 4 is obtained. If the actual distance between corner cells is known, the position of the card can be easily obtained. If the three-dimensional coordinate position of the corner cell 104a is expressed as a variable, the three-dimensional coordinate positions of the corner cells 104b, 104c, and 104d can be expressed using the variable. Since the screen coordinate positions of the four corner cells are known, the three-dimensional coordinate position of the game card 4 can be specified by solving the simultaneous equations of these calculation formulas.

  As shown in the above embodiment, by using the two-dimensional code shown in FIG. 2, the recognition rate of the corner cell 104 in the code unit 110 can be increased, and the coordinates of the corner cell 104 in the two-dimensional coordinate system are used. The position can be obtained accurately. Specifically, since the center position of the corner cell 104 can be accurately obtained by forming the corner cell 104 large, the position of the game card 4 can be accurately identified. In addition, by accurately determining the center positions of the plurality of corner cells 104, the positional relationship between the reference cell 100 and the corner cells 104 can be accurately grasped, and the accuracy of affine transformation can be improved. And the direction in which the game card 4 is facing can be accurately specified.

  FIG. 8 shows another example of the two-dimensional code printed on the surface of the game card. A reference cell 100 and a code portion 110 having a predetermined shape are arranged on the surface of the game card 4. Compared with the two-dimensional code of FIG. 2, in the two-dimensional code shown in FIG. 8, a plurality of corner cells 104a, 104b, 104c, 104d surrounding the code data portion 102 are formed in a circle. As described above, the two-dimensional coordinate position of the corner cell 104 is specified by the center point of the corner cell 104. Since the corner cell 104 has a round shape, the shape of the corner cell 104 imaged by the imaging device 2 does not change regardless of the orientation of the game card 4 with respect to the imaging device 2. For this reason, the center point of the corner cell 104 does not change depending on the orientation with respect to the imaging device 2, and thus the center point can be acquired stably. Thereby, as described above, the position information, direction information, and distance information of the game card 4 can be accurately obtained. Note that the square cell 106 may be formed in a round shape. In addition to making the corner cell 104 circular, as described with reference to FIG. 2, it is possible to obtain the coordinate position of the corner cell 104 with higher accuracy by forming it larger than the rectangular cell 106. .

  FIG. 9 shows still another example of the two-dimensional code printed on the surface of the game card. A reference cell 100 and a code portion 110 are arranged on the surface of the game card 4. Compared with the two-dimensional code of FIG. 2, in the two-dimensional code shown in FIG. 9, a reference cell 100 having a predetermined shape, a plurality of rectangular cells 106 constituting code data by being arranged two-dimensionally, and a code data portion 102 At least one cell of the plurality of corner cells 104 arranged so as to surround the region is colored differently from the other cells. For example, red may be assigned to the reference cell 100, green may be assigned to the square cell 106, blue may be assigned to the corner cell 104, and different colors may be assigned to the respective cells. The color assigned here may be any color that can be captured by the imaging apparatus 2 and binarized with a predetermined luminance threshold as a reference, and is not necessarily required to be visible to the human eye. When the imaging characteristics of the imaging apparatus 2 to be used are known in advance, the cells may be colored according to the characteristics.

  In the recognition process of the two-dimensional code of the present embodiment, as shown in S20 of FIG. Among the entire two-dimensional code recognition processes, the load of the reference cell detection process is high. Therefore, it is preferable that the reference cell 100 can be efficiently extracted from the image data. Therefore, it is preferable that at least the reference cell 100 among the plurality of types of cells is colored differently from the rectangular cell 106 and the corner cell 104.

  An RGB color filter is arranged in front of the light receiving surface of the imaging device 2, and each pixel value of RGB is recorded as data expressed in 256 gradations for each frame image. In this example, the binarization processing unit 50 has a function of setting a range of RGB pixel values for binarization processing. When the reference cell 100 is painted red, the binarization processing unit 50 sets a range of RGB pixel values for the reference cell in the binarization process of S10 of FIG. For example, the binarization processing unit 50 has a red (R) pixel value range of 200 to 255, a green (G) pixel value range of 0 to 100, and a blue (B) pixel value range of 0 to 0. Set to 100. Based on this setting, the binarization processing unit 50 extracts RGB composite pixels within this range, and encodes the pixel values to “1”. For example, the pixel value of the composite pixel in which the R pixel value is 225, the G pixel value is 30, and the B pixel value is 50 is encoded as “1”, and the color as the binary image is set to black. On the other hand, the pixel value of the composite pixel whose RGB pixel value is not within these ranges is encoded to “0”, and the color as the binary image is set to white. As described above, by extracting the red pixels from the image data, it is possible to label connected regions that are black pixels in the binary image in S12 of FIG. The number of noises to be extracted can be reduced by limiting the pixels to be extracted as black pixels to a specific color in the image data, in this case, red, and the reference cell detection unit 52 can extract the reference data from the binarized data of the frame image. The time for detecting the cell 100 can be shortened.

  Since the pixel value is affected by the color filter characteristics, the red pixel of the reference cell 100 is placed with the game card 4 placed on the table 3 in order to set a range of pixel values that also consider the color filter characteristics. The value may be investigated. As a result, the RGB pixel value of the reference cell 100 acquired in the imaging device 2 can be specified, and the black value to be detected by performing binarization processing in a range in which a slight margin is given to the specified pixel value. It is possible to dramatically reduce the number of pixel connection regions. In particular, in the game system 1, since it is preferable to perform processing in real time, there is a great merit that the reference cell 100 can be efficiently extracted.

  The same applies to the case where other types of cells, for example, the corner cell 104 and the rectangular cell 106 are colored. When the corner cell 104 is blue, in S10 of FIG. 4, the binarization processing unit 50 sets a range of RGB pixel values for the corner cell. For example, the binarization processing unit 50 has a red (R) pixel value range of 0 to 100, a green (G) pixel value range of 0 to 100, and a blue (B) pixel value range of 200 to 200. Set to 255. Based on this setting, the binarization processing unit 50 extracts RGB composite pixels within this range, and encodes the pixel values to “1”. For example, the pixel value of the composite pixel in which the R pixel value is 20, the G pixel value is 30, and the B pixel value is 240 is encoded as “1”, and the color as the binary image is set to black. On the other hand, the pixel value of the composite pixel whose RGB pixel value is not within these ranges is encoded to “0”, and the color as the binary image is set to white. In this way, by extracting blue pixels from the image data, it is possible to label connected regions that are black pixels in the binary image in S12 of FIG.

  In this way, after performing the binarization processing for converting a strong blue image into a black image, the code portion detection unit 54 performs the corner shown in the flow of FIG. Used to select as a cell candidate area. The number of noises to be extracted can be reduced by limiting the pixels to be extracted as black pixels to a specific color in the image data, in this case, blue, and the code portion detection unit 54 can calculate the corner from the binarized data of the frame image. The time for detecting the cell 104 can be shortened. When the square cell 106 is green, the binarization process for converting a strong green image into a black image in S10 in FIG. 4 is performed, and then the black pixel connected region in S12 is labeled as shown in FIG. This is used for the code map generation process of S222. When three types of cells are colored differently, the number of times of binarization processing increases. However, since the processing amount for searching for a connected region after binarization processing is greatly reduced, Efficiency can be realized.

  The two-dimensional code shown in FIG. 9 has colored cells, and the colored cells may be formed large, or may be formed round as shown in FIG.

  FIG. 10 shows still another example of the two-dimensional code printed on the surface of the game card. A reference cell 100 and a code portion 110 are arranged on the surface of the game card 4. The two-dimensional code shown in FIG. 10 is one embodiment of the two-dimensional code shown in FIG. The two-dimensional code shown in FIG. 10 includes a reference cell 100 having a predetermined shape, a plurality of rectangular cells 106 constituting code data by being arranged two-dimensionally, and a plurality of codes arranged so as to surround the area of the code data portion 102. The corner cell 104 is provided. The four corner cells 104a, 104b, 104c, and 104d have the same color scheme, but are colored differently from the reference cell 100 and the rectangular cell 106.

  In order to improve the aesthetics of the game card 4, the corner cell 104 is provided up to the side edge of the game card 4. Since the corner cells 104 are arranged near the four corners of the game card 4 and are formed larger than the square cells 106, the four corner cells 104 are conspicuous in the game card 4. Originally, the corner cell 104 is provided in order to specify the location of the square cell 106, but aesthetics are also important as long as it is used as the game card 4. Therefore, the corner cell 104 is designed to have a design by folding the frame edge 107 that borders the game card 4 inward so as to surround the corner cell 104 at the position of the corner cell 104. .

  Thus, when providing the corner cell 104 in the side edge of the game card 4, it is preferable that the corner cell 104 is given colors other than black. If black is assigned, when the game card 4 is imaged by the imaging device 2 and binarized, the shadow generated at the edge of the game card 4 may be converted as a black pixel. It is assumed that the two corner cells 104 are connected by the shadow of the edge of the game card 4. When the two corner cells 104 are connected by the binarization process, it is difficult to separate the two corner cells 104 from the connected black pixels. Therefore, by assigning a color other than black to the corner cell 104 and performing binarization processing by appropriately setting the RGB threshold when binarizing, that is, the RGB range, the shadow of the edge of the game card 4 It is possible to avoid a situation where the corner cells 104 are connected. Thus, when the corner cell 104 is provided up to the side edge of the game card 4, it is possible to appropriately extract the corner cell 104 by assigning a color other than the shadow color, that is, a color other than black. Become.

  In the game card 4 shown in FIG. 10, the corner cell 104 is formed in a triangular shape. This shape may be an equilateral triangle. By making the corner cell 104 an equilateral triangle, the recognition accuracy of the corner cell 104 can be increased.

  FIG. 11A to FIG. 11C are explanatory diagrams for explaining a method of determining whether or not the acquired black pixel is the corner cell 104. As described above, the corner cell 104 is discriminated based on the connected area of black pixels after binarizing the frame image. Since the binarized frame image is expressed on the two-dimensional XY coordinates, it is possible to determine whether or not the connected area is the corner cell 104 by using the coordinate value of the connected area.

Fig.11 (a) shows an example of the connection area | region extracted from the frame image. The connecting region 109a has a width in the X-axis direction as H ₁ and a width in the Y-axis direction as V ₁ . This width is counted by the number of pixels on the two-dimensional coordinate. If the number of pixels the coupling region 109a and the C _1, the connection region 109a is whether a candidate for corner cell 104, is determined by the following equation.
α × H ₁ × V ₁ ≦ C ₁ ≦ β × H ₁ × V ₁ (α <β <1)
For example, α is 0.3 and β is 0.7.
Pixel number _{C 1} of the connecting region 109a is, if satisfying the above equation, the coupling region 109a may be determined to be a candidate of the corner cells 104. Naturally, the values of the constants α and β may be set to other values.

FIG. 11B shows another example of the connected area extracted from the frame image. Whether or not this connected area 109b is a candidate for the corner cell 104 is similarly determined by the following equation. The number of pixels the connecting region 109b _C 2, the width of the X-axis direction to the width of the _H 2, Y-axis and _{V 2.}
α × H ₂ × V ₂ ≦ C ₂ ≦ β × H ₂ × V ₂ (α <β <1)
The connected regions shown in FIGS. 11A and 11B each satisfy the discriminant, and are examples in which it is determined that the corner cell 104 is a candidate. As shown in FIG. 11A and FIG. 11B, in the frame image, the game card 4 may be arranged in the imaging region 5 at various angles, but the above discriminant is used. By doing so, it becomes possible to appropriately recognize the triangular corner cell 104.

FIG. 11C shows still another example of the connected area extracted from the frame image. In this case, the number of pixels C ₃ in the connection region 109c is smaller than α × H ₃ (width in the X-axis direction) × V ₃ (width in the Y-axis direction). That is,
C ₃ <α × H ₃ × V ₃
The relationship is established. Since this does not satisfy the discriminant (α × H ₃ × V ₃ ≦ C ₃ ≦ β × H ₃ × V ₃ ), the connection region 109 c shown in FIG. 11C is not the corner cell 104. Is determined.

As described above, by setting the shape of the corner cell 104 to a triangle, a candidate for the corner cell 104 can be easily extracted using the coordinate values on the two-dimensional coordinates. In addition, when the shape of the corner cell 104 is an equilateral triangle, the direction dependency at the time of determination by the discriminant can be reduced as compared with other triangular shapes, so that the recognition accuracy of the corner cell 104 can be further increased.
In the game card 4 shown in FIG. 10, the code data portion 102 composed of the square cells 106 is arranged between the corner cells 104c and 104d below the card. As a result, a space can be created in the central portion of the game card 4, and a character picture or the like can be printed there.

  FIG. 12 shows still another example of the two-dimensional code printed on the surface of the game card. In the game card 4 shown in FIG. 12, the corner cell 104 is provided from the side edge of the game card 4 similarly to the game card 4 shown in FIG. The corner cell 104 is formed in a triangular shape, and is colored differently from the reference cell 100 and the code data portions 102a and 102b.

  In the game card 4 shown in FIG. 12, the area in which the square cell 106 is provided is separated into two. Here, the two code data portions 102a and 102b are arranged in a portion other than the region surrounded by the four corner cells 104a, 104b, 104c, and 104d. The code data portion 102a is formed between the reference cell 100 and the upper edge of the game card 4, and the code data portion 102b is formed between the line connecting the corner cells 104c and 104d and the lower edge of the game card 4. Is done. The line connecting the corner cells 104c and 104d is a line connecting triangle vertices facing inward of the card. By arranging the code data portion 102a and the code data portion 102b outside the area surrounded by the four corner cells 104, a space in the central area of the game card 4 can be widened, and a character picture or the like can be provided there. It becomes possible to print. Each of the code data portions 102a and 102b can record 12 bits of information, and thus can hold a total of 24 bits of information. It is desirable that the code data portion 102b arranged below the game card 4 is not continuously colored for a predetermined bit length, for example, 3 bits or more, in order to prevent erroneous recognition as the reference cell 100. .

  When recognizing the rectangular cell 106 in FIG. 12, since the reference cell 100 and the lower left and lower right corner cells 104d and 104c have already been detected, the code data portion 102a and 102b can be easily detected. That is, the code data portion 102a exists on the opposite side of the corner cells 104c and 104d with respect to the reference cell 100, and the code data portion 102b exists on the opposite side of the reference cell 100 with respect to the corner cells 104c and 104d. By setting the reference cell 100 and the code data portion 102a in the vicinity and setting the corner cells 104c and 104d and the code data portion 102b in the vicinity, the search efficiency of the code data portion 102 can be improved.

  In the game card 4 of FIG. 12, the code data portions 102a and 102b are shown in two separate arrangements, but the code data portions 102a and 102b may be arranged adjacent to each other. That is, the code data portions 102 a and 102 b may be disposed between the reference cell 100 and the upper edge of the game card 4, and may be disposed between the corner cells 104 d and 104 c and the lower edge of the game card 4. .

Although the game card 4 is moved on the table 3 as described above, a game mat may be placed on the table 3 and the game card 4 may be moved thereon.
FIG. 13 shows a game mat for placing game cards. With reference to FIG. 1, the game mat 300 is placed on the table 3 so as to enter the imaging region 5 of the imaging device 2. The game mat 300 is preferably made of a soft material such as felt fabric and can be stored in a compact manner while not in use.

  The game mat 300 has nine squares 302a, 302b, 302c, 302d, 302e, 302f, 302g, 302h, and 302i (hereinafter collectively referred to as “mass 302”) by a plurality of partition lines 304. Is formed. The game mat 300 is configured according to the game application, and each cell 302 forms a section in which the game card 4 is placed. When one or more users place the game card 4 on one of the squares 302, the image analysis device 20 identifies the placed square 302, recognizes the two-dimensional code of the game card 4, and the game apparatus 30 You may develop the game story according to the arranged cell. In the game mat 300 shown in FIG. 13, 3 × 3 squares 302 are formed, but the number of squares 302 is not limited to this. When an area where the user can move the game card 4 is called a “play area”, in the game mat 300 shown in FIG. 13, a square area in which a square 302 is formed forms a play area 306. The game mat 300 does not have to be arranged so that the entire game mat 300 enters the imaging region 5, but at least the play area 306 needs to enter the imaging region 5. Note that in the game application in which the game card 4 may be moved freely, the square 302 may not be formed.

  In the game mat 300, gradation areas 310 a, 310 b, 310 c, and 310 d (hereinafter collectively referred to as “gradation areas 310”) whose brightness changes in stages are provided at the four corners of the play area 306. A color has three attributes of hue, brightness, and saturation, but the gradation area 310 is configured by changing the brightness of one hue, and the saturation need not be changed. The outer edge of the gradation region 310 is circular, and the lightness changes in a concentric manner from high lightness to low lightness in a direction from the center of the gradation region 310 toward the outer edge.

  FIG. 14 shows the appearance of the gradation area. The gradation area 310 is composed of one hue, and the brightness is gradually changed in the hue. Hue represents a difference in hue such as red, green, and blue, and lightness represents a difference in brightness. Here, it is assumed that the gradation region 310 has a hue, that is, is expressed by a chromatic color. It should be noted that although there is no hue in achromatic colors such as white, black, and gray, the gradation region 310 can be configured by changing the brightness from white to black in stages. The difference in brightness in the gradation area 310 is acquired as a difference in brightness when digitized via the RGB filter.

  Thus, the brightness of the outer edge 312 of the gradation area 310 is set low, and the brightness of the center 314 is set high. That is, the gradation area 310 has a dark chromatic area on the outside and white on the inside. The four gradation areas 310 may all be configured with the same color, but can be used as an index for determining the orientation of the game mat 300 by changing at least one color. For example, three gradation areas 310a, 310b, and 310c may be configured with the same hue, and one gradation area 310d may be configured with a different hue. At this time, if the four gradation areas 310 can be detected, the position of the gradation area 310d with respect to the gradation areas 310a, 310b, and 310c is known, so the orientation of the game mat 300 can be specified.

  The gradation area 310 is further used to determine a threshold value of RGB pixel values when the image of the reference cell 100, the corner cell 104 and / or the rectangular cell 106 of the game card 4 is binarized and extracted. For example, it is assumed that the hue of the corner cell 104 is green and has a predetermined brightness. At this time, the hue of the gradation area 310 is set to green, and an area having the same brightness as the corner cell 104 is formed in a part of the gradation. Therefore, a cell brightness area 316 having the same brightness as the corner cell 104 is set at a predetermined position between the center 314 of the gradation area 310 and the outer edge 312. Preferably, the cell brightness region 316 is formed concentrically at a midpoint between the center 314 and the outer edge 312. Therefore, in the radial direction of the gradation area 310, an area having a higher brightness than the cell brightness area 316 is provided between the cell brightness area 316 and the center 314, and a cell between the cell brightness area 316 and the outer edge 312 is provided. A region having a lightness lower than that of the lightness region 316 is provided.

  When the corner cell 104 and the cell brightness region 316 of the gradation region 310 have the same hue (G) and the same luminance (gradation) and are not affected by ambient light, the corner cell 104 is acquired via a G filter. It is assumed that the G pixel value is expressed by 127, for example. The binarization processing unit 50 sets a range of RGB pixel values for binarization processing, but when the pixel value of the corner cell 104 is known in advance, the binarization threshold is ideally set. Is set in the vicinity of the pixel value of the corner cell 104, and it is preferable to set the range so that a pixel value lower than that of the corner cell 104 (ie, having a high brightness) is not extracted as much as possible. In this case, theoretically, the binarization processing unit 50 can extract the corner cell 104 by setting the G pixel value range from 120 to 255, for example, by setting the threshold value to 120, and the G pixel value becomes 120. An image that does not satisfy the condition can be converted to “0” by binarization processing. By narrowing the range of G pixel values in the binarization process, other noise can be reduced.

  However, in an actual environment, the corner cell 104 acquired by the imaging device 2 is often different from the theoretical value (127) due to lighting or sunlight. The “theoretical value” is a term used for convenience in describing the present embodiment, and means a pixel value extracted from the image data under a reference environment. The narrower the range of pixel values set for binarization, the more the post-processing load can be reduced. On the other hand, the pixel values that are actually acquired differ from the theoretical values due to the influence of ambient light. Therefore, if the range of pixel values to be binarized is narrow, there is a possibility that the range is out of the range. Therefore, in the game system 1 according to the present embodiment, the pixel value under the actual use environment in which the gradation region 310 is used to calibrate the pixel value range when binarizing and the influence of ambient light or the like is taken into consideration. Set the range.

  FIG. 15 shows another configuration of the image analysis apparatus. The image analysis device 20 includes a frame image acquisition unit 40 and a real object extraction unit 42. The real object extraction unit 42 includes a binarization processing unit 50, a region extraction unit 60, and an RGB adjustment unit 62. Unlike the block diagram shown in FIG. 3, FIG. 15 shows a configuration for calibrating the range of pixel values when binarizing. The processing function of the image analysis device 20 is realized by a CPU, a memory, a program loaded in the memory, and the like, and here, a configuration realized by cooperation thereof is illustrated. The program may be built in the image analysis apparatus 20 or may be supplied from the outside in a form stored in a recording medium. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof. In the illustrated example, the CPU of the image analysis device 20 has functions as a frame image acquisition unit 40 and a real object extraction unit 42.

  The frame image acquisition unit 40 acquires a real space frame image captured by the imaging device 2. The imaging device 2 periodically acquires frame images, and preferably generates frame images at 1/60 second intervals. In the imaging area 5, the game mat 300 and the game card 4 placed on the game mat 300 are arranged. Therefore, the frame image includes the game mat 300 and the game card 4.

  The real object extraction unit 42 extracts a real object image from the frame image. Here, in order to adjust the range of RGB pixel values when the corner cell 104 is binarized, the actual object image to be extracted is the gradation region 310 in the game mat 300. Hereinafter, a case will be described in which the corner cell 104 is configured with green having a predetermined brightness, and the gradation area 310 is formed as a green gradation including the predetermined brightness.

  The binarization processing unit 50 sets a range of RGB to be binarized and converts the frame image into a binary bit representation. Specifically, the binarization processing unit 50 extracts the G pixel value range from 120 to 255 and the R pixel value range from 0 to 0 in order to extract the gradation regions 310a, 310b, and 310c to which the green gradation is applied. 20. The range of the B pixel value is set to 0 to 20, and RGB composite pixels within the RGB pixel value range are extracted. The reason why the pixel value range for RB is not 0 is to consider the characteristics of the color filter. The binarization processing unit 50 extracts a composite pixel within the set RGB pixel value range and sets the pixel value to “1”.

  The area extraction unit 60 extracts the gradation area 310 of the game mat 300 from the binarized data of the frame image. The gradation area 310 forms a gradation in which the G pixel value in the central portion is low and the G pixel value in the outer peripheral portion is high. As a result of the binarization processing, the gradation area 310 is converted to a bit representation in which the composite pixel value of the central portion is 0 and the composite pixel value of the outer peripheral portion is 1, and has a donut shape in which the central portion is blank. Therefore, the region extraction unit 60 detects a donut shape from the binarized data. As a result of the binarization process, it is considered that there are few regions that are converted into a donut shape other than the gradation region 310. Therefore, forming the gradation region 310 in a circle is highly advantageous in terms of narrowing the candidates for the gradation region 310.

  Here, the four gradation regions 310 are arranged at the vertices of a square. When a plurality of donut areas are detected as a result of the binarization process for green and the binarization process for black, those that have a relationship in which the donut areas are at the vertices of a square are extracted and the gradations are extracted. The area 310 is specified. The shapes of the donut areas identified as the green gradation areas 310a, 310b, and 310c are sent to the RGB adjustment unit 62 for binarization threshold calibration.

  Fig.16 (a)-FIG.16 (c) show the donut area | region specified as a gradation area | region. In the following, a method of calibrating the range of G pixel values when binarizing and detecting the corner cells 104 of the same color based on the green gradation areas 310a to 310c will be described. The binarization threshold value of the G pixel value is 120, and the extraction range is 120 to 255. The RGB adjustment unit 62 adjusts the range of RGB pixel values when binarization is performed in the binarization processing unit 50 based on the binarized data in the gradation area 310.

  FIG. 16A shows a donut region in which the ratio of the inner diameter to the outer diameter is about 1: 2, more precisely 121: 256. In this case, the gradation area 310 is ideally binarized without any influence of ambient light in the frame image captured by the imaging device 2. In this donut region, a cell brightness region 316 having the same gradation as the square cell 106 exists at a point where the center and the outer edge are divided into 1: 1, and is included in the black portion of the donut region. Therefore, by setting the threshold value of the G pixel value to 120, the corner cell 104 can be preferably extracted.

  FIG. 16B shows a state in which the inner diameter is very small with respect to the outer diameter of the donut region. In this case, the cell brightness region 316 existing at the point where the center and the outer edge are divided into 1: 1 is included in the black portion of the donut region, but even the inner low-tone region is detected. Yes. For example, when green light is included in the ambient light, the green light and the green color in the gradation area 310 may overlap, and the G component of the detection light may become strong. At this time, the pixel value of the donut region is detected as a value stronger than the theoretical value of the pixel value of the gradation region 310, and therefore, the region that is binarized and encoded to “1” increases more than necessary. Become.

  Therefore, when the binarization processing unit 50 binarizes the image of the game card 4 by setting the G pixel value range to 120 to 255, the corner cell 104 can be extracted, but the corner cell 104 can be extracted. The light green area is extracted in the same manner. As the number of corner cell 104 candidates increases, it takes time to specify the corner cell 104. Therefore, it is preferable to set a pixel value range for binarization so that the number of corner cell 104 candidates decreases.

  The RGB adjustment unit 62 holds in advance the distribution of gradation expressed by gradation in the gradation area 310 and the position of the area where the same gradation as the corner cell 104 exists. Therefore, when receiving the shape of the donut area shown in FIG. 16B, the RGB adjustment unit 62 determines that the set threshold value (120) is low, and determines that the threshold value needs to be increased.

  For example, the RGB adjustment unit 62 calculates the G pixel value of the gradation region 310 at that point from the distance from the center to the inner diameter in the donut region (that is, the radius of the white region) based on the gradation distribution of the gradation region 310. Find the value. The theoretical value of the G pixel value at that point is originally a G pixel value lower than 120, but the G pixel value at that point is detected as 120 in the donut region shown in FIG. Therefore, the RGB adjustment unit 62 may determine the amount of increase in the threshold value by using the difference between the theoretical value of the G pixel value on the gradation region 310 and the set threshold value 120 as an increase due to the ambient light. For example, when the theoretical value of the G pixel value on the gradation area 310 at the boundary between the white area and the black area in the donut area is 60, the difference from the setting threshold 120 is 60. The threshold value of the G pixel value in the binarization process is set to 180 and supplied to the binarization processing unit 50.

  FIG. 16C shows a state where the inner diameter of the donut region is larger than half of the outer diameter. In this case, the cell brightness region 316 existing at the point where the center and the outer edge are divided by 1: 1 is included in the white part of the donut region, and therefore the cell brightness region 316 having the same gradation as the corner cell 104 is It is converted to “0” by the binarization process. For example, when the environment is very dark, the pixel value may be detected lower than the theoretical value of the gradation area 310.

  Therefore, if the binarization processing unit 50 binarizes the game card 4 by setting the G pixel value range to 120 to 255, the corner cell 104 cannot be extracted. Therefore, it is necessary to reset the G pixel value range of the binarization process so that the cell brightness region 316 having the same pixel value as that of the corner cell 104 can be extracted by expanding the G pixel value extraction range.

  When receiving the shape of the donut area shown in FIG. 16C, the RGB adjustment unit 62 determines that the set threshold value (120) is high, and determines that the threshold value needs to be lowered. For example, the RGB adjustment unit 62 calculates the G pixel value of the gradation region 310 at that point from the distance from the center to the inner diameter in the donut region (that is, the radius of the white region) based on the gradation distribution of the gradation region 310. Find the value. The theoretical value of the G pixel value at that point is originally a G pixel value higher than 120, but the G pixel value at that point is detected as 120 in the donut region shown in FIG. Therefore, the RGB adjustment unit 62 may determine the difference between the theoretical value of the G pixel value on the gradation area 310 and the set threshold 120 as the threshold reduction amount. For example, when the theoretical value of the G pixel value on the gradation region 310 at the boundary between the white region and the black region in the donut region is 180, the difference from the setting threshold 120 is 60. The threshold value of the G pixel value in the binarization process is set to 60 and supplied to the binarization processing unit 50.

  When the binarization processing unit 50 receives the range of pixel values adjusted from the RGB adjustment unit 62, the binarization processing unit 50 executes a binarization process on the frame image based on the range. The region extraction unit 60 detects a donut region corresponding to the gradation region 310 from the binarized data and supplies the detected donut region to the RGB adjustment unit 62. The RGB adjustment unit 62 adjusts the threshold value of the binarization process based on the donut area, and feeds it back to the binarization processing unit 50. The above calibration processing may be executed until the donut region is in a state as shown in FIG.

  In the above description, the case of calibrating the range (threshold value) of the G pixel value when binarizing has been described, but the same applies to other R and B pixel values. Although the threshold value of the RB pixel value at the time of binarization processing of the green gradation areas 310a, 310b, and 310c is set to 20 by default, the RGB adjustment unit 62 preferably adjusts this default value as necessary. .

  Note that the gradation areas 310a, 310b, and 310c exist at three spatially different points, and it is considered that the optimum pixel value range during binarization processing differs depending on the position due to the influence of ambient light or the like. It is done. When the vicinity of the gradation areas 310a and 310b is bright and the vicinity of the gradation area 310c is dark depending on the direction of the external illumination, the threshold value for binarization may be set low in accordance with the gradation area 310c. This makes it possible to detect the corner cell 104 regardless of the arrangement position of the game card 4.

  However, in the vicinity of the gradation areas 310a and 310b, the possibility of extracting various areas other than the corner cell 104 increases by setting the binarization threshold value low. Therefore, an optimal binarization threshold value at the position of each gradation area 310 may be obtained, and the binarization threshold value at an arbitrary position in the play area 306 may be derived from these three binarization threshold values. For example, when the binarization threshold in the gradation area 310b is T1 and the binarization threshold in the gradation area 310c is T2, the binarization threshold changes linearly from T1 to T2 on the line connecting the gradation areas 310b and 310c. You may set so that it may. The same applies to other arbitrary positions, and the binarization threshold value at each position may be calculated according to the distance from the binarization threshold value of three points. This makes it possible to properly recognize the code even when the illumination environment differs depending on the position of the card, such as when illumination is close, and the code color to be photographed differs greatly.

  The calibration process of the present embodiment is preferably performed periodically. If there is no change in the environment, the need for frequent calibration processing is not high, so a separate sensor is provided to detect the change in the environment, and if an environmental change is detected by that sensor, calibration is performed. May be performed. The environmental change includes a case where the game mat 300 is moved and a case where the brightness of the external illumination is changed.

  As a method of calibration processing, the binarization threshold value may be corrected based on the difference between the actual pixel value at a predetermined position in the gradation area 310 and the measurement value obtained as a result of photographing. The binarization threshold value may be corrected so that the shape of the donut region when it is converted into a specific shape. In the latter case, for example, the binarization threshold value may be incremented or decremented according to the size of the donut region, and adjusted so that the shape of the donut region approaches the theoretical value. Further, not only binarization based on one binarization threshold value, but also two threshold values of an upper limit and a lower limit may be provided, and binarization may be performed depending on whether or not it falls within the range. For example, the pixel value may be binarized as “0” if the pixel value is 85 to 170, and “1” otherwise. In this case, the gradation region 310 is binarized based on the magnitude of a certain binarization threshold, the outer diameter of the gradation region 310 is grasped, and the donut region of the intermediate region specified by the upper and lower thresholds is actually measured. You may acquire a result and correct | amend an upper limit and a minimum so that the shape may approach a theoretical value. Further, the density may be reversed from the above-described example. In that case, binarization may be performed after performing the reversal process.

  As shown in FIG. 1, since the imaging device 2 images the game mat 300 from obliquely above, even if the shape of the gradation region 310 is a perfect circle, there is a possibility that it is recognized as an ellipse. Even in this case, as described above, the position and direction of the game mat 300 can be grasped in advance, and the shape of the donut area as a recognition result of the gradation area 310 based on this information. The above-described calibration process may be performed after correcting.

  Further, since it is preferable to perform the calibration process after the game mat 300 is fixedly installed, for example, after confirming that the position does not change by performing the recognition process of the gradation region 310 a plurality of times, or The calibration process may be started after confirming that the position does not change by performing the recognition process again after a predetermined time.

  FIG. 17 shows an image data transmission system in the embodiment. The image data transmission system 200 includes an image data generation device 201 that generates image data, a transmission station 204 that transmits image data, and a terminal device 202 that receives the transmitted image data. The terminal device 202 has a display device 203 for displaying the received image data. The image data transmission system 200 transmits data by wireless communication, but may transmit data by wired communication.

  The image data generation device 201 displays the above-described two-dimensional code, that is, the image data of the two-dimensional code described with reference to FIGS. 2, 8, 9, 10 and 12 on the display device 203 of the terminal device 202. It is generated according to the data format. The image data generation device 201 has, as the image data of the two-dimensional code shown in FIG. 2, data for displaying a plurality of rectangular cells at predetermined coordinate positions on the display device 203, and a plurality of data displayed at the predetermined coordinate positions. Data for displaying a plurality of corner cells at coordinate positions surrounding the rectangular cell may be generated. The image data generation device 201 sets the corner cell and square cell data so that the corner cell is displayed larger than the square cell on the display device 203.

  The image data generation device 201 has, as the image data of the two-dimensional code shown in FIG. 8, data for displaying a plurality of rectangular cells at predetermined coordinate positions on the display device 203, and a plurality of data displayed at the predetermined coordinate positions. Data for displaying a plurality of corner cells at coordinate positions surrounding the rectangular cell may be generated. The image data generation device 201 sets the corner cell data so that the corner cell is displayed in a round shape.

  The image data generation device 201 has, as the image data of the two-dimensional code shown in FIG. 9, data for displaying a reference cell having a predetermined shape at a predetermined position on the display device, and an area defined for the reference cell Data for displaying a plurality of polygonal cells within the range of and a data for displaying a plurality of corner cells at coordinate positions surrounding the region may be generated. The image data generation device 201 sets the color of at least one of the reference cell, the polygonal cell, and the corner cell to be different from the colors of the other cells. Further, the image data generation device 201 has the same shape as the data for displaying a reference cell having a predetermined shape at a predetermined position on the display device as the image data of the two-dimensional code shown in FIGS. Data for displaying a plurality of corner cells and data for displaying a plurality of corner cells in a predetermined area may be generated. The image data generation device 201 sets the color of the corner cell to a color different from that of the reference cell and the polygonal cell.

  The transmission station 204 transmits the image data generated by the image data generation device 201 to the terminal device 202. The image data is preferably compressed in a format that can be decompressed by the terminal device 202, and needs to be created in a data format displayed by a browser function or the like in the terminal device 202. The transmission station 204 can reduce the amount of data transmission by transmitting image data for each cell in association with the display coordinate position, for example. The terminal device 202 displays a two-dimensional code on the display device 203. In the game system 1 of FIG. 1, the user can use the two-dimensional code displayed on the display device 203 instead of the game card 4 to be read by the imaging device 2. Thereby, even if the user does not have the game card 4, the user can participate in the game system 1 by downloading the image data to the terminal device 202.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

It is a figure which shows the structure of the game system concerning an Example. It is a figure which shows the two-dimensional code printed on the surface of the game card. It is a figure which shows the structure of an image analyzer. It is a flowchart which shows the process sequence of the recognition process of a two-dimensional code. It is a flowchart which shows the detection process of a reference | standard cell. It is a flowchart which shows the detection process of a code part. It is a flowchart which shows the verification process of code data. It is a figure which shows another example of the two-dimensional code printed on the surface of the game card. It is a figure which shows another example of the two-dimensional code printed on the surface of the game card. It is a figure which shows another example of the two-dimensional code printed on the surface of the game card. (A)-(c) is explanatory drawing for demonstrating the method to discriminate | determine whether it is a corner cell. It is a figure which shows another example of the two-dimensional code printed on the surface of the game card. It is a figure which shows the mat for games for arrange | positioning the card for games. It is a figure which shows the external appearance of a gradation area | region. It is a figure which shows another structure of an image analyzer. (A)-(c) is a figure which shows the donut area | region specified as a gradation area | region. It is a figure which shows the transmission system of the image data in an Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Game system, 2 ... Imaging device, 4 ... Game card, 6 ... Output device, 7 ... Display, 10 ... Image processing device, 20 ... Image analysis device, DESCRIPTION OF SYMBOLS 30 ... Game device, 40 ... Frame image acquisition part, 42 ... Real object extraction part, 44 ... State determination part, 50 ... Binarization processing part, 52 ... Reference cell detection , 54... Code part detection part, 56... Verification processing part, 60... Region extraction part, 62... RGB adjustment part, 100. 104 ... Corner cell, 106 ... Square cell, 108 ... Area, 110 ... Code part, 200 ... Image data transmission system, 201 ... Image data generator, 202 ... Terminal Device 203... Display device 204. Shin stations.

Claims (9)

A reference cell having a predetermined shape;
A plurality of polygonal cells constituting the code data;
A plurality of corner cells having the same shape;
With
The card, wherein the polygonal cell is arranged in a portion other than a region surrounded by the plurality of corner cells.   The card according to claim 1, wherein the corner cell is colored differently from the reference cell and the polygonal cell.   The card according to claim 1, wherein the corner cell is a triangle.   The card according to any one of claims 1 to 3, wherein the corner cell is provided from an edge of the card and is colored other than black. A reference cell having a predetermined shape;
A plurality of polygonal cells constituting the code data;
A plurality of corner cells having the same shape;
A rectangular card with
The plurality of corner cells are provided at first and second opposing edges of the card,
At least some of the plurality of polygonal cells are provided along the third edge in the vicinity of a third edge different from the first edge and the second edge. card.   Some of the plurality of polygonal cells are provided along the fourth edge in the vicinity of the fourth edge facing the third edge, and the third edge and the fourth edge are provided. 6. The card according to claim 5, wherein a plurality of polygonal cells provided on the edge constitute one two-dimensional code.   The card according to claim 5, wherein the polygonal cell and the reference cell are not formed in a region surrounded by the plurality of corner cells.   A reference cell having a predetermined shape;
  A plurality of polygonal cells constituting the code data;
  A plurality of corner cells having the same shape;
  With
  The polygonal cell is arranged in a portion other than a region surrounded by the plurality of corner cells.   A reference cell having a predetermined shape;
  A plurality of polygonal cells constituting the code data;
  A plurality of corner cells having the same shape;
  An image pattern comprising:
  The plurality of corner cells are provided at the first edge and the second edge facing each other in the image pattern,
  At least some of the plurality of polygonal cells are provided along the third edge in the vicinity of a third edge different from the first edge and the second edge. Image pattern.