JP3844482B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing technique, and more particularly to a technique for displaying a real object and a virtual object in association with each other.

2. Description of the Related Art In recent years, a technique has been widely used in which a two-dimensional code is captured and recognized by a camera and a predetermined process based on a two-dimensional code pattern is executed. As one of such techniques, an image processing technique has been proposed in which a two-dimensional barcode is imaged by a video camera and a three-dimensional image corresponding to the two-dimensional barcode is displayed on a display device (for example, Patent Documents). 1). In Patent Document 1, coordinates on a space for displaying a three-dimensional object are determined based on the coordinates of the captured two-dimensional barcode and the focal length of the CCD video camera, and the three-dimensional object is superimposed on the two-dimensional barcode. To display.
JP 2000-322602 A

  The technique disclosed in Patent Literature 1 can realize an excellent visual effect by displaying a combination of a real-world object and a virtual object. However, Patent Document 1 only discloses a technique for displaying a virtual object in association with one two-dimensional barcode, and is conscious of display processing of a virtual object when a plurality of two-dimensional barcodes exist. is not. The inventor pays attention to display processing when a plurality of real objects such as two-dimensional barcodes exist in the real space, and devise the display processing of the virtual object at this time, for example, in the game field or the like I found out that it has the potential to achieve better visual effects when applied to the field.

  An object of the present invention is to provide a technique for displaying a real object and a virtual object in association with each other, and in particular, in a field such as a game, the display mode of a virtual object is controlled based on the positional relationship between a plurality of real objects. The purpose is to provide technology.

  In order to solve the above problems, an aspect of the present invention relates to an image processing apparatus. The image processing apparatus according to this aspect includes a storage unit that stores identification information for identifying a real object and three-dimensional image data of a virtual object in association with each other, and a plurality of frame images captured by the imaging apparatus. A positional relationship detection unit that detects a positional relationship between real object images, a condition determination unit that determines whether or not the positional relationship detected for at least two real object images satisfies a predetermined condition, and a plurality of frames included in the frame image When it is determined that a predetermined condition is satisfied by the reading unit that reads out the three-dimensional image data of the plurality of virtual objects associated with the identification information of the real object image from the storage unit, and the condition determining unit, the condition is Based on the identification information of at least two real object images that satisfy, the display pattern of the virtual object is determined and read. Using a plurality of 3-dimensional image data, the determined display pattern and a display control unit that performs display processing of the virtual object. Here, the real object is a real object that exists as a tangible object in the real space, and the virtual object does not exist in the real space, but is expressed by data in the virtual space.

  According to this image processing apparatus, in order to control the display mode of a virtual object based on the positional relationship between a plurality of captured real object images, a plurality of real objects are placed in a predetermined positional relationship. Visual expression can be realized.

  Another aspect of the present invention relates to a game device. The game device according to this aspect includes a storage unit that stores identification information for identifying a real object and three-dimensional image data of the game character in association with each other, and a plurality of real images included in the frame image captured by the imaging device. A positional relationship detection unit that detects a positional relationship between the object images, a condition determination unit that determines whether the positional relationship detected for at least two real object images satisfies a predetermined condition, and a plurality of actual images included in the frame image The condition is satisfied when the reading unit that reads the three-dimensional image data of the plurality of game characters associated with the identification information of the object image from the storage unit and the condition determination unit determine that the predetermined condition is satisfied Based on the identification information of at least two real object images, the display pattern of the game character is determined and read Using a plurality of 3-dimensional image data, and a display control unit that performs display processing of the game character by the determined display pattern.

  According to this game device, in order to control the display mode of the game character based on the positional relationship between the captured plurality of real object images, a new visual perception of the game character is obtained by placing the plurality of real objects in a predetermined positional relationship. Expression can be realized.

  According to still another aspect of the present invention, a step of detecting a positional relationship between a plurality of real object images included in a frame image captured by an imaging device, and a positional relationship detected for at least two real object images satisfy a predetermined condition. A step of determining whether or not to satisfy, identification information for identifying a real object, and three-dimensional image data of the virtual object are stored in association with each other, and a plurality of real object images included in the frame image are stored. A step of reading the three-dimensional image data of a plurality of virtual objects associated with the identification information, and when it is determined that a predetermined condition is satisfied, based on the identification information of at least two real object images that satisfy the condition To determine the display pattern of the virtual object and Using 3-dimensional image data, the determined display pattern to provide an image processing method comprising the steps of: performing display processing of the virtual object.

  According to still another aspect of the present invention, a function for causing a computer to detect a positional relationship between a plurality of real object images included in a frame image captured by an imaging device and a positional relationship detected for at least two real object images are predetermined. From a storage unit that stores a function for determining whether or not the above condition is satisfied, identification information for identifying a real object, and three-dimensional image data of the virtual object in association with each other. A function of reading the three-dimensional image data of a plurality of virtual objects associated with the identification information of the object image, and at least two real object images satisfying a predetermined condition when it is determined that the predetermined condition is satisfied Based on the identification information of the virtual object and the function to determine the display pattern of the virtual object Using a plurality of 3-dimensional image data issued by the determined display pattern provides a program for implementing a function to perform the display processing of the virtual object.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the technique which controls the display mode of a virtual object based on the positional relationship of a some real object can be provided.

  The present invention provides a technique for detecting a positional relationship between a plurality of real objects imaged by an imaging device and controlling a display mode of a virtual object based on the detected positional relationship. The real object may be an existing one-dimensional object, two-dimensional object, or three-dimensional object, and is preferably configured to have a characteristic portion for identifying the real object itself. For example, the real object may be a two-dimensional object such as a card having code information expressed in two dimensions as a characteristic part, or may be a three-dimensional object having a unique three-dimensional shape as a characteristic part. Note that the two-dimensional shape of the two-dimensional object may constitute a unique feature portion, and code information serving as the feature portion may be attached to the three-dimensional object. The virtual object may be a so-called character such as a person, an animal, or an object that is three-dimensionally expressed in the virtual space. The following embodiment relates to an image processing technique in a game application, and employs a game character appearing in the game application as a virtual object. In the embodiment, an event corresponding to the contact is generated when the player contacts the real objects, and a game application that progresses by sequentially executing the event is shown.

  FIG. 1 shows a configuration of a game system 1 according to an embodiment of the present invention. The game system 1 includes an imaging device 2, an image processing device 10, and an output device 6. The image processing device 10 includes an image analysis device 20 and a game device 30. The image analysis device 20 and the game device 30 may be configured as separate devices, but may be configured as a single unit. The imaging device 2 is a video camera that includes a CCD imaging device or a MOS imaging device. The imaging device 2 images the real space at a predetermined cycle and generates a frame image for each cycle. The imaging region 5 is a range where the imaging device 2 captures an image, and the position and size of the imaging region 5 are adjusted by adjusting the height and direction of the imaging device 2. The game player moves the game card 4 that is a real object with a finger in the imaging area 5. The game card 4 has a characteristic part for uniquely identifying itself.

  The output device 6 includes a display 7 that is a display device. The output device 6 may further include a speaker (not shown). Basically, the image processing apparatus 10 displays the frame image captured by the imaging apparatus 2 on the display 7, and controls display so that a character that is a virtual object is superimposed on the game card 4. In the example of FIG. 1, two game cards 4 exist in the imaging area 5, and characters are superimposed on the respective game cards 4 on the display 7. The player can easily check whether or not the game card 4 is in the imaging area 5 by looking at the display 7. If not, the player can shift the position of the game card 4 or adjust the orientation of the imaging device 2. Then, the imaging device 2 is caused to image the game card 4.

  In the game application of the embodiment, the player operates the character by performing a predetermined operation on the game card 4. Due to the nature of the game application, it is preferable that the player recognize the unity between the game card 4 and the character, and therefore, the character image is displayed superimposed on the game card 4.

  The movement of the above characters is controlled by the image processing apparatus 10. First, the image analysis device 20 extracts image information of the game card 4 from the frame image acquired by the imaging device 2. Further, the image analysis device 20 extracts a unique feature portion for identifying the game card 4 from the image information of the game card 4. At this time, the image analysis device 20 determines position information, direction information, distance information, and the like in the space of the game card 4 from the image information of the game card 4.

  FIG. 2 shows an example of the surface of the game card 4. On the surface of the game card 4, a direction instruction unit 11 and an identification unit 12 are printed. The direction instruction unit 11 is provided to indicate the front part of the game card 4, and the identification unit 12 is provided to express a characteristic portion for uniquely identifying the game card 4. The identification unit 12 is code information created by printing a plurality of blocks in a predetermined section. Among the plurality of blocks, a square block is commonly given to the plurality of game cards 4, and a characteristic portion is substantially formed by blocks other than the four corners. The quadrangular block is used to measure the distance from the imaging device 2.

  Returning to FIG. 1, when the image analysis device 20 recognizes the game card 4 in the frame image, the image analysis device 20, based on the length between the four blocks in the image of the game card 4, Calculate the distance. Further, the image analysis device 20 detects the direction instruction unit 11 of the game card 4 and obtains the direction in which the game card 4 is facing in the frame image. In this case, the direction in which the direction instructing unit 11 is present is the front, and the character is display-controlled so as to face the front on the game card 4. Further, the image analysis device 20 acquires the identification information of the game card 4 from the blocks other than the four corners of the identification unit 12.

  The result of the image analysis performed by the image analysis device 20 is sent to the game device 30. Note that the frame image captured by the imaging device 2 may be sent to the game device 30 and image analysis performed on the game device 30 side. In this case, the image processing apparatus 10 is configured by only the game apparatus 30.

  The game apparatus 30 performs display control so that the character is positioned on the game card 4 on the display screen of the display 7 based on the image analysis result in the image analysis apparatus 20. Characters may be appropriately assigned to the game card 4 for each game scene. In this case, when the game scene is switched, the displayed character is also switched. In the present embodiment, when the game apparatus 30 detects the positional relationship between a plurality of real object images and determines that the positional relationship between the real object images satisfies a predetermined condition, the game apparatus 30 controls the display mode of the virtual object.

  FIG. 3 shows the configuration of the image analysis apparatus 20. The image analysis apparatus 20 includes a frame image acquisition unit 40, a real object extraction unit 42, a state determination unit 44, an identification information acquisition unit 46, an identification information storage unit 48, and a transmission unit 50. The identification information storage unit 48 stores the information of the characteristic part for identifying the real object and the identification information for identifying the real object in association with each other. Specifically, the pattern information of the identification unit 12 of the game card 4 in FIG. 2 and the identification information are stored in a one-to-one correspondence. The identification information is used for character assignment in the game apparatus 30. For example, in the case of the game system 1 that permits simultaneous use of five game cards 4, identification information 1 to 5 may be assigned to each game card 4. Each game card 4 can be recognized by associating each of the plurality of game cards 4 with identification information. The state determination unit 44 determines the state of the real object in the set coordinate system. Specifically, from the position determination unit 52 that determines the position of the game card 4, the direction determination unit 54 that determines the direction, and the imaging device 2. Has a distance determination unit 56 for determining the focal length of the.

  The frame image acquisition unit 40 acquires a real space frame image captured by the imaging device 2. Here, it is assumed that a plurality of game cards 4 are arranged in the imaging region 5 as shown in FIG. The imaging device 2 periodically acquires frame images, and preferably generates frame images at 1/60 second intervals.

  The real object extraction unit 42 extracts a plurality of real object images, that is, images of the game card 4 from the frame image. This process is performed by converting the image information into a binary bit representation and extracting the image of the game card 4 from the bit representation. This is so-called dot processing, and image extraction processing may be performed by detecting on and off of bits. Further, this processing may be performed by a known image matching technique. In that case, the real object extraction unit 42 registers image information of a real object to be used in a memory (not shown) in advance for the matching process. By matching the pre-registered image information with the captured image information, the images of the plurality of game cards 4 can be cut out from the frame image.

  The state determination unit 44 detects the geometry information of the table 3 for placing and moving the game card 4 in advance, and uses the surface of the table 3 as a reference plane, and the posture of the game card 4 placed on the reference plane. Geometry information is recorded as an initial state of the posture of the game card 4. This geometry information may be generated based on the imaging device 2. The state determination unit 44 grasps the position, posture, and the like of the table 3 as coordinate data in the imaging space from the geometry information acquired for the table 3. The position determination unit 52 determines the position of the real object image. Specifically, the coordinates of the center point of the real object image in the frame image are determined. In addition to the position of the game card 4, the position determination unit 52 has a relative state from the attitude of the game card 4 in which the inclination and height of the game card 4 with respect to the reference plane of the game card 4 are recorded as an initial state. You may obtain | require as a difference of quantity, ie, the difference of the coordinate value in imaging space. The direction determination unit 54 determines the direction of the real object image. The direction determination unit 54 may determine the direction of the real object by detecting the direction instruction unit 11 shown in FIG. 2 from the real object image. The distance determination unit 56 determines the distance between the imaging device 2 and the game card 4 based on the length between the four corners of the identification unit 12 in the image of the game card 4. The identification information acquisition unit 46 extracts a feature portion from the real object image and acquires corresponding identification information from the identification information storage unit 48.

  The position information, the direction information, the distance information, and the identification information acquired by the identification information acquisition unit 46 determined by the state determination unit 44 are transmitted from the transmission unit 50 to the game device 30 in association with each other. This association is performed for each game card 4. In the present embodiment, the frame image itself is also sent from the sending unit 50 to the game device 30 in order to display the frame image taken by the imaging device 2 on the display 7.

  FIG. 4 shows the configuration of the game apparatus 30. The game apparatus 30 includes an analysis information acquisition unit 100, a game progress processing unit 102, a character determination unit 104, a character storage unit 106, a positional relationship detection unit 110, a condition determination unit 112, a display control unit 120, and a display pattern storage unit 122. .

  The processing function of the game apparatus 30 in the present embodiment is realized by a CPU, a memory, a program loaded in the memory, and the like, and here, a configuration realized by cooperation thereof is illustrated. The program may be built in the game apparatus 30 or may be supplied from the outside in a form stored in a recording medium. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof. In the illustrated example, the CPU of the game apparatus 30 functions as an analysis information acquisition unit 100, a game progress processing unit 102, a character determination unit 104, a positional relationship detection unit 110, a condition determination unit 112, and a display control unit 120.

  The analysis information acquisition unit 100 receives an analysis result from the image analysis device 20. This analysis result includes position information, direction information, distance information, and identification information of the game card 4 that is a real object. The analysis information acquisition unit 100 passes the received analysis result to the game progress processing unit 102. The analysis information acquisition unit 100 may directly acquire frame image data from the imaging device 2. In this case, the game apparatus 30 has a function in the image analysis apparatus 20 and may perform the same process as the process described for the image analysis apparatus 20.

  The game progress processing unit 102 controls processing of the entire game application. In the game application in this embodiment, the game progress is composed of a plurality of stages, and different game scenes are set in each stage. The player clears the end condition of each stage step by step, and the game ends when the final stage is cleared. The game progress processing unit 102 manages the progress of the game, and immediately after the start of the game or at the time of switching the game stage, the character determination unit 104 sends the name of the game stage to be started next and the analysis information acquisition unit 100. Communicate identification information sent to you. The character storage unit 106 stores the identification information of the game card 4 and the three-dimensional image data of the character in association with each game stage.

  FIG. 5 shows the contents stored in the character storage unit 106. The game system 1 according to the present embodiment can use five game cards 4. The character storage unit 106 stores three-dimensional image data of characters assigned to each of the five game cards 4 in association with the game stage. In stage 1, the character “male” is on the game card with identification information 1, the character “female” is on the game card with identification information 2, the character “drum” is on the game card with identification information 3, and the character “restaurant” is on the game card with identification information 4. The character “post office building” is assigned to the game card of the identification information 5. At stage 2, the character “male” on the game card with identification information 1, the character “female” on the game card with identification information 2, the character “restaurant door” on the game card with identification information 3, and the character on the game card with identification information 4. The character “table and chair” is assigned to the game card of “waiter” and identification information 5. In stage 1 and stage 2, the characters assigned to the identification information 3, 4, and 5 are different.

  The character determination unit 104 reads out three-dimensional image data of a plurality of characters associated with the identification information from the character storage unit 106 based on the game stage name and the plurality of identification information, and supplies the data to the game progress processing unit 102. . Note that the read three-dimensional image data may be directly supplied to the display control unit 120. The game progress processing unit 102 supplies the display control unit 120 with the three-dimensional image data and the position information, direction information, and distance information of the game card 4. The display control unit 120 causes the display 7 to display the character in association with the display position of the game card 4 using the three-dimensional image data.

  Specifically, the display control unit 120 receives a frame image sent from the image analysis device 20 and causes the display 7 to display the frame image. In addition, the display control unit 120 recognizes the position, orientation, and distance of the game card 4 based on the position information, direction information, and distance information of the game card 4, and displays the character displayed on the display 7 using the three-dimensional image data. Determine the position, orientation, and size of the. The display control unit 120 may place the character at any position as long as it overlaps the game card 4. However, in the normal display mode, the display control unit 120 displays the character so as to be positioned above the center point of the game card 4. Determine the location.

  When the player moves the game card 4, the analysis result of the frame image is sequentially sent from the image analysis device 20 to the analysis information acquisition unit 100. The display control unit 120 receives the three-dimensional image data of the character and the position information, direction information, and distance information of the game card 4 from the game progress processing unit 102, and the character image overlaps the display position of the game card 4 on the display 7. Thus, the character is caused to follow the movement of the game card 4. Thereby, the display 7 becomes a display mode in which the character is always on the game card 4, and the player can recognize the sense of unity between the character and the game card 4.

  FIG. 6 shows a display example of the display in stage 1. Here, the game card 4a is the game card of the identification information 1, the game card 4b is the game card of the identification information 2, the game card 4c is the game card of the identification information 3, the game card 4d is the game card of the identification information 4, and the game card 4e. Indicates a game card of identification information 5.

  The character “male” is superimposed on the game card 4a, the character “drum” is superimposed on the game card 4c, the character “restaurant building” is superimposed on the game card 4d, and the character “post office building” is superimposed on the game card 4e. Yes. The game card 4c also displays a “bachi” for hitting the drum along with the drum. At this time, the character “female” is not yet displayed on the game card 4b.

  In the game application of the present embodiment, when the arrangement positions of the plurality of game cards 4 have a predetermined positional relationship, an event is generated and the event is executed, whereby the game story progresses. By moving a plurality of game cards 4 by the player, it is possible to change the action pattern of the character, and to give the player a different pleasure from the operation by the normal game controller or the like. In the example of FIG. 6, the “female” character appears on the game card 4 b by executing a predetermined event determined in the stage 1.

  The positional relationship detection unit 110 detects the positional relationship between the images of the plurality of game cards 4 included in the frame image captured by the imaging device 2. Specifically, first, the game progress processing unit 102 delivers the position information, direction information, and distance information of the plurality of game cards 4 to the positional relationship detection unit 110. The positional relationship detection unit 110 detects the positional relationship between the game cards based on the positional information and distance information of the plurality of game cards 4. Here, it is preferable to detect the positional relationship between game cards for all combinations of two or more game cards 4. For example, the positional relationship detection unit 110 may calculate the distance between the center coordinates based on the center coordinates of each game card 4.

  The condition determination unit 112 determines whether or not the positional relationship between the game cards 4 satisfies a predetermined condition based on the positional relationship detected by the positional relationship detection unit 110. Here, it is determined whether or not the positional relationship detected for the images of at least two game cards 4 satisfies a predetermined condition. As an example of condition determination, the condition determination unit 112 determines whether the images of the game card 4 are in contact with each other. Simply, when the distance between the center coordinates of the two game card images is within a predetermined range, the condition determination unit 112 may determine contact between the game card images.

  The condition determination unit 112 considers the direction of the game card image when determining contact between the game card images. The arrangement of the game card image in the space can be determined by the center coordinates and orientation of the game card image. Thereby, the condition determination part 112 can know the arrangement | positioning of the game card image in space, and can determine whether it is contacting. At this time, it is possible to know which side of the game card image is in contact by considering the direction. Since the direction of the game card image is determined by the direction determination unit 54, it is determined whether the rectangular game card is in contact with the front side or the left side. 112 can be determined based on the determined direction information. When the condition determination unit 112 recognizes the direction of the game card image at the time of contact determination, it is possible to change the event to be generated depending on which direction the game cards are in contact with each other. When determining that the game card images are in contact with each other, the condition determining unit 112 notifies the game progress processing unit 102 of the determination result, the identification information of the contacted game card image, and the orientation of the game card image. To do. The game progress processing unit 102 transmits these pieces of information to the display control unit 120. The processes of the positional relationship detection unit 110 and the condition determination unit 112 described above may be executed simultaneously.

  Furthermore, the condition determination unit 112 may set a virtual viewing angle for one game card image, for example, and may determine whether another game card image exists within the viewing angle. This condition determination is executed based on the positional relationship detected by the positional relationship detection unit 110, and is used to check whether another character exists in the viewing angle of a certain character during the game. When the condition determining unit 112 determines that another game card image exists within the viewing angle, the determination result, the identification information of the game card image in which the viewing angle is set, and the game card existing within the viewing angle The game identification processing unit 102 is notified of image identification information. Such information is transmitted to the display control unit 120.

  When the condition determination unit 112 determines that the predetermined condition is satisfied, the display control unit 120 determines a character display pattern based on the identification information of at least two game card images that satisfy the condition. The display control unit 120 receives the determination result sent from the game progress processing unit 102, the identification information of the game card image that satisfies the conditions, and the orientation of the game card image, and refers to the display pattern storage unit 122, Determine the display pattern.

  The display pattern storage unit 122 stores the display pattern of the virtual object. The display pattern may be, for example, an action pattern between characters corresponding to identification information of at least two game card images that satisfy a condition. This operation pattern is stored in association with identification information of at least two game card images and the orientations of those game card images. Specifically, for example, when the game card 4a of the identification information 1 and the game card 4c of the identification information 3 are brought into contact with each other on the front side, the display control unit 120 displays a predetermined operation pattern as a display pattern storage unit. 122 can be read out. In other words, the requirement for reading the display pattern from the display pattern storage unit 122 is that the image of the game card 4a and the image of the game card 4c are in contact with each other and the front sides are in contact with each other. Is set. Therefore, even if the game card 4a and the game card 4c are in contact with each other on the front side and the left side, the display control unit 120 cannot read the display pattern.

  The display control unit 120 performs character display processing using the determined display pattern on the frame image sent from the image analysis device 20 using the three-dimensional image data of a plurality of characters.

  FIG. 7 shows a state in which two game cards are brought into contact with the state shown in FIG. Here, the game card 4a and the game card 4c are brought into contact with each other. The game card 4a and the game card 4c are in contact with each other at the front sides. Since the “male” faces the front side of the game card 4a and the “drum” is also arranged facing the front side of the game card 4c, the “male” stands in front of the drum, You can strike a drum. In other words, the “man” cannot strike the drum even when standing on the side of the drum. The identification information of the game card 4 to be contacted and the orientation at the time of contact are requirements for reading the display pattern from the display pattern storage unit 122 as described above. Therefore, by bringing the front sides of the respective game cards into contact with each other, the display control unit 120 reads out a predetermined operation pattern, performs a display process in which a man holds a drum placed on the game card 4c and strikes a drum. Execute. This series of movements is defined in the read display pattern.

  In the game application, the operation of tapping the drum is a condition for causing the character of “female” to appear. When the game progress processing unit 102 recognizes that the drum is hit by a man, Appear on 4b. Subsequently, the player moves the male and female characters in front of the restaurant.

  FIG. 8 shows a state where two game cards are brought into contact with one game card from the state shown in FIG. Here, the front side of the game card 4a and the front side of the game card 4b are brought into contact with the left side of the game card 4d. The identification information and direction of the game card to be contacted are set as requirements for reading the display pattern as described above. In stage 1, as shown in FIG. 8, moving the male and female characters in front of the restaurant is set as a transition condition for ending stage 1 and proceeding to stage 2. When the positional relationship detection unit 110 is notified of the positional relationship shown in FIG. 8 with respect to the game cards 4a, 4b, and 4d, the condition determination unit 112 indicates that the front sides of the game cards 4a and 4b correspond to the left side of the game card 4d. The contact is determined and reported to the game progress processing unit 102. Upon receiving this report, the game progress processing unit 102 recognizes that the stage 1 end condition is satisfied, and performs a game stage switching process. Note that the condition determination unit 112 may determine that the end condition is satisfied. The game progress processing unit 102 notifies the character determination unit 104 of the next stage name (stage 2).

  When notified of the stage name, the character determination unit 104 reads the three-dimensional image data of the identification information assigned to the stage 2 with reference to the correspondence relationship shown in FIG. As described above, the character determination unit 104 changes the correspondence relationship referred to according to the stage, so that a new character can appear in each stage, and the game story can be enhanced. At stage 2, as shown in FIG. 5, the character “restaurant door” is displayed on the game card with identification information 3, the character “waiter” is displayed on the game card with identification information 4, and the character “table and chair” is displayed on the game card with identification information 5. "Is assigned, and the character display shown in FIG. 8 is switched.

  In the game system 1 of the present embodiment, various image processing techniques other than those described above can be realized using the positional relationship of the game card images.

FIG. 9 is a diagram for explaining a process of assigning one character to a plurality of game card images.
FIG. 9A shows a state in which buildings are allocated to three game cards. Although the same building is allocated to the game cards 4a, 4b, and 4c, the character target to be allocated does not matter here. FIG. 9B shows a state where three game cards are brought into contact with each other. At this time, one large building is allocated on three game cards.

  The display pattern storage unit 122 stores a display pattern in which one virtual object is assigned to at least two game card images that satisfy the conditions. In this example, the display pattern storage unit 122 stores identification information (identification information 1, 2, 3) of three game card images and display patterns associated with the directions of contact portions of the game card images. Yes. Here, the direction of the contact portion is the left-right direction, that is, that each game card image is in contact with another game card image on the left side or the right side is a condition for reading the display pattern.

  When the condition determination unit 112 determines that the game cards 4a, 4b, and 4c are in contact in the left-right direction, the determination result, the identification information of the game card image, and the image orientation are transmitted via the game progress processing unit 102. To the display control unit 120. The display control unit 120 reads the display pattern from the display pattern storage unit 122 based on these pieces of information. The display control unit 120 performs display processing shown in FIG. 9B based on the display pattern. Thereby, since a character can be displayed largely, a new visual effect can be realized in the game story. The player can attach game cards to each other so that an unexpected character appears, and the game can be made more interesting.

FIG. 10 is a diagram for explaining the process of changing the direction of the character.
FIG. 10A shows the positional relationship between the character “male” of the game card 4a and the character “female” of the game card 4b. Two broken lines 101 extending from the character “male” indicate a virtual viewing angle of the man. The condition determination unit 112 determines whether the image of the game card 4b is within the virtual viewing angle of the game card 4a from the position information and direction information of the game card 4a and the position information of the game card 4b. Note that the positional information and the direction information are delivered from the positional relationship detection unit 110. If it is determined that the character “woman” falls within the virtual viewing angle, the determination result and the identification information of the game cards 4 a and 4 b are transmitted to the game progress processing unit 102. It is assumed that the game card 4a is advanced along the path of the arrow by subsequent player operations.

  FIG.10 (b) shows the state after moving along the arrow shown to Fig.10 (a). The character of the game card 4a is originally set to face forward, but in this example, the character of the game card 4b faces. The display control unit 120 receives from the game progress processing unit 102 identification information of the game cards 4a and 4b and information indicating that the game card 4b is within the viewing angle of the game card 4a. The display pattern storage unit 122 stores an action pattern indicating that the character of the game card 4a changes its direction so as to continue watching the character of the game card 4b. This display pattern is set to be readable on condition that the relationship between these game cards and the viewing angle is established. The display control unit 120 reads this operation pattern and executes the display process shown in FIG. That is, the display control unit 120 changes the direction of the character of the game card 4a according to the position of the character of the game card 4b. Thereby, the player recognizes that the character of the game card 4b may have an important influence on the progress of the game for the character of the game card 4a. Such a display mode gives the player an opportunity to generate a new event by bringing the game card 4a into contact with the game card 4b.

FIG. 11 is a diagram for explaining processing for expanding and contracting a virtual object.
FIG. 11A shows a state where the game card 4a and the game card 4b are in contact with each other. When two game cards are brought into contact with each other, a virtual object 103 is generated between the characters. As shown in FIG. 11B, the virtual object 103 is expressed so as to extend when the game cards are separated from each other. On the other hand, when the game cards are brought closer together from the state shown in FIG. 11B, the virtual object 103 is expressed so as to shrink. Thereby, a new visual effect is realizable. In addition, the display pattern storage part 122 memorize | stores the display pattern which produces | generates the virtual object which can be expanded-contracted according to those positional relationship between characters, when the front sides of the game card 4a and the game card 4b contact. is doing.

  When the positional relationship detection unit 110 detects that the positional relationship between the game card images shifts from the first state to the second state, the display control unit 120 determines that the characters corresponding to the respective identification information A display pattern for expanding and contracting the virtual object connecting the two is read out from the display pattern storage unit 122 and determined. The display control unit 120 performs the display shown in FIGS. 11A and 11B using the display pattern.

  FIG. 12 shows a state in which a problem is given to the player in order to improve the game performance. For example, the problem is that the game card 4 is moved on the table 3 according to the arrow 105 displayed on the display 7 while the two game cards 4 are attached. The positional relationship detection unit 110 calculates each movement vector of the game cards 4a and 4b, and sets the average as the movement direction of the two game cards. The movement vector is calculated from the change in the distance of the game card 4 and the difference between the center coordinates in the previous and subsequent frames by storing the center coordinates and distance information of the game card image in a memory (not shown) for each frame. The condition determination unit 112 determines whether the task has been achieved from the movement vector, specifically, determines whether the movement vector is along the arrow 105, and the direction of the vector and the arrow 105 is substantially the same. If so, it is assumed that the task has been cleared. The player may receive an optional merit on the game by clearing the task.

  FIG. 13 shows an image processing flow of the embodiment. In the game device 30, the positional relationship detection unit 110 detects the positional relationship between the images of the plurality of game cards 4 included in the frame image captured by the imaging device 2 (S10). The condition determination unit 112 determines whether or not the positional relationship between the game cards 4 satisfies a predetermined condition based on the positional relationship detected by the positional relationship detection unit 110 (S12). When the predetermined condition is satisfied (Y in S12), the display control unit 120 determines a display pattern associated with the satisfied condition and reads it from the display pattern storage unit 122 (S14). The display control unit 120 performs a character display process using the determined display pattern (S16).

  While the predetermined condition is not satisfied (N in S12) and the stage continues (N in S18), the positional relationship detection unit 110 continues to detect the positional relationship between the game cards 4. When the stage is switched (Y in S18), this flow ends. When the next stage starts, the three-dimensional image data of the character corresponding to the stage is read, and the above-described flow is executed.

  In the above, this invention was demonstrated based on the Example. This embodiment is an exemplification, and it will be understood by those skilled in the art that various modifications can be made to the combination of each component and each processing process, and such modifications are also within the scope of the present invention. .

  In the embodiment described above, the display mode of the character is controlled based on the positional relationship between the game cards 4. In order to enhance the interest of the game application, not only the display mode of the character but also an effect such as voice may be performed. In this case, when the condition determination unit 112 determines that the positional relationship between the game cards 4 satisfies the predetermined condition, the game progress processing unit 102 notifies the voice control unit (not shown) to that effect and performs voice control. The part may perform an auditory performance of the character through a speaker. At this time, the game apparatus 30 not only functions as an image processing apparatus but also functions as an audio processing apparatus. Therefore, the game apparatus 30 can also be referred to as a processing apparatus capable of controlling both images and sounds. Note that the game apparatus 30 may perform only sound control based on the positional relationship between the game cards 4.

It is a figure which shows the structure of the game system concerning an Example. It is a figure which shows an example of the surface of a game card. It is a figure which shows the structure of an image analyzer. It is a figure which shows the structure of a game device. It is a figure which shows the memory content of a character storage part. 6 is a diagram showing a display example of a display in stage 1. FIG. It is a figure which shows the state which made the two game cards contact from the state shown in FIG. It is a figure which shows the state which made the two game cards contact the one game card from the state shown in FIG. It is a figure for demonstrating the process which allocates one character to several game card images. It is a figure for demonstrating the process which changes the direction of a character. It is a figure for demonstrating the process which expands and contracts a virtual object. It is a figure which shows the state which gave the subject to the player. It is a flowchart of an image process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Game system, 2 ... Imaging device, 4 ... Game card, 5 ... Imaging area, 7 ... Display, 10 ... Image processing device, 20 ... Image analysis device, DESCRIPTION OF SYMBOLS 30 ... Game device, 40 ... Frame image acquisition part, 42 ... Real object extraction part, 44 ... State determination part, 46 ... Identification information acquisition part, 48 ... Identification information storage part , 50 ... sending unit, 52 ... position determining unit, 54 ... direction determining unit, 56 ... distance determining unit, 100 ... analysis information acquiring unit, 102 ... game progress processing unit, 104: Character determination unit, 106: Character storage unit, 110: Position relation detection unit, 112 ... Condition determination unit, 120 ... Display control unit, 122 ... Display pattern storage unit

Claims (1)

A storage unit that stores identification information for identifying a real object and three-dimensional image data of the virtual object in association with each other;
A positional relationship detection unit that detects a positional relationship between a plurality of real object images included in a frame image captured by the imaging device;
A condition determination unit that determines whether the positional relationship detected for at least two real object images satisfies a predetermined condition;
A reading unit that reads out three-dimensional image data of a plurality of virtual objects associated with identification information of a plurality of real object images included in the frame image from a storage unit;
When the condition determining unit determines that the predetermined condition is satisfied, the display pattern of the virtual object is determined based on the identification information of at least two real object images that satisfy the condition, and the plurality of read three A display control unit that performs display processing of the virtual object using the determined display pattern using the dimensional image data;
With
When the positional relationship between the real object images detected by the positional relationship detection unit shifts from the first state to the second state, the display control unit detects whether the virtual object corresponding to the identification information of each real object image And determining a display pattern to be displayed by extending or contracting another virtual object generated so as to connect the virtual objects .

Images