JP2013061870A - Program, information storage medium, and electronic device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve resistance to instability of player viewpoint such as camera shake in a multi-play game using an augmented reality space.SOLUTION: From an image of an AR marker 10 taken by a CCD camera module 1516, a relative position and a relative attitude of a player viewpoint are calculated and stored in a queue a predetermined number of times. The weighted means of relative positions and relative attitudes from past to present stored in the queue are determined to calculate a current position and a current attitude. In accordance with the current position and the current attitude, a virtual camera is disposed in a virtual three-dimensional space. CG of the virtual three-dimensional space as captured by the virtual camera is superimposed on the image taken by the CCD camera module 1516 to generate a game screen.

Description

  The present invention relates to a program for causing a computer to execute a game using augmented reality technology.

  As one of augmented reality (AR) technologies, an image taken with a video camera is analyzed, the coordinates of the real space and the CG space are aligned, and CG is synthesized on the video shot image (so-called live view). A technique for displaying the AR space by displaying is known.

  In recent years, video games using AR technology have also appeared. In a game using such AR technology, the key is how to match the CG space (computer graphics space) where the game character object is placed with the real space captured by the camera. As a known technique related to this, an image of an augmented reality marker (AR marker) captured by a camera at a fixed position is analyzed, and a virtual map is generated so as to match the actual plane on which the augmented reality marker is arranged. The technique to do is known (for example, refer patent document 1).

JP 2010-49690 A

  As shown in Patent Document 1, if the camera that captures the AR marker (that is, the player viewpoint position) is fixed, the image captured by the camera is also stable. However, since a hand-held game apparatus equipped with a CCD camera is played with being held in hand, there is a possibility that camera shake or the like may occur. When camera shake occurs, the coordinate alignment between the real space and the CG space is not stable, and an event in which the CG sometimes shakes in small increments, so-called jitter may occur. In the case of multiplayer, there are many forms in which one of the game machines controls the NPC (non-player character) and distributes the control result. When the game screen is raised, the NPC appears to tremble even when the game screen of another aircraft is not shaken.

  The present invention has been conceived in view of such circumstances, and an object thereof is to improve the quality of a multiplayer game using an augmented reality space. More specifically, the aim is to increase resistance to player viewpoint instability such as camera shake.

In a first mode for solving the above-described problem, an object is synthesized and displayed on a live view photographed by the photographing unit so that a marker having a reference direction is reflected on a computer having a photographing unit and a display unit. A program for running the game,
Position calculation means for periodically calculating the position of the computer in the virtual three-dimensional space with reference to the position of the marker in the live view (for example, the control board 1550 in FIG. 1, the processing unit 200 in FIG. 11, game calculation) Unit 202, own device information calculation unit 206 in FIG. 12, relative position calculation unit 207, step S68 in FIG.
Current position determining means for determining the current position of the computer by using a plurality of calculated positions calculated by the position calculating means for a given time or a given number of times retroactively from the present (for example, FIG. 1 control board 1550, processing unit 200 in FIG. 11, game calculation unit 202, current position determination unit 209 in FIG. 12, step S80 in FIG.
Object placement means for placing an object at a given position in the virtual three-dimensional space (for example, the control board 1550 in FIG. 1, the processing unit 200 in FIG. 11, the game calculation unit 202, the character display control unit 230 in FIG. 12, Step S222 in FIG.
Combining means (for example, the control board 1550 in FIG. 1, the processing unit 200 in FIG. 11, the image generating unit 260, and the like) that synthesizes an image of the object viewed from the current position determined by the current position determining unit with the live view. This is a program for causing the computer to function as the CG synthesis unit 264, steps S312 to S314) of FIG.

  In another embodiment, the electronic device includes a photographing unit and a display unit, and displays an object in a live view photographed by the photographing unit so that a marker indicating a reference direction is captured. Position calculation means for periodically calculating the position of the computer in the virtual three-dimensional space with reference to the position of the marker in the above, and the position calculation for a given time or a given number of times retroactively from the present Current position determining means for determining a current position of the computer using a plurality of calculated positions calculated by the means; object placement means for placing an object at a given position in the virtual three-dimensional space; and Synthesizing means for synthesizing an image of the object viewed from the current position determined by the position determining means with the live view. It is possible to configure the electronic devices.

  According to the first aspect, while periodically calculating the position of the computer with respect to the marker, the current position of the computer is calculated based on the past position for a given time or a given number of times from the latest position. It can be calculated. Then, an image (CG: computer graphics) of the object arranged in the virtual three-dimensional space can be generated from the current position. That is, even if there is a camera shake or a sudden change in the shooting direction, the movement of the object displayed in the AR space is suppressed more than the actual change. Therefore, the game screen is stable and easy to see.

  The second form is the program according to the first form, wherein the current position determining means is means for determining the current position of the computer by weighted averaging the plurality of calculated positions.

  According to the second embodiment, the same effect as the first embodiment can be obtained, and how much the change in the computer position with respect to the marker affects the actual movement of the object by weighted averaging of a plurality of calculated positions. This can be optimized by adjusting the weight.

  A third form is a first weight changing means (for example, in FIG. 12) in which the current position determining means changes the weight of each of the calculation positions used for the weighted average according to the calculation time by the position calculating means. It is a program of the 2nd form which has the present position determination part 209 and step S78 of FIG.

  According to the 3rd form, while the same effect as a 2nd form is acquired, weighting can be performed according to calculation time.

  According to a fourth mode, second weight changing means (for example, FIG. 5) changes the weight of each of the calculated positions used for the weighted average according to the distance between the calculated position and the marker. This is a program in the second or third form having twelve current position determination units 209 and step S78 in FIG.

  According to the 4th form, while being able to acquire the effect similar to a 3rd form, weighting can be adjusted according to the distance of a calculation position and a marker.

The fifth form is a change amount measuring means for measuring a change amount per unit time of the position and / or posture of the computer (for example, control board 1550, 3-axis gyro 1556, 3-axis acceleration sensor in FIG. 1). 1557, game calculation unit 202 in FIG. 25, change amount calculation unit 213F, step S26F in FIG. 26),
Calculation interval changing means for changing the calculation interval of the position calculating means using the measurement result of the change amount measuring means (for example, the control board 1550 in FIG. 1, the game calculation unit 202 in FIG. 25, the calculation interval setting unit 214F, As a step S27F) of FIG. 26, it is a program according to any one of the first to fourth modes for further functioning the computer.

According to the fifth aspect, the same effect as in any of the first to fourth aspects can be obtained, and the calculation interval can be further adjusted to further enhance the effect.
In this case, as a sixth aspect, the calculation interval changing unit further changes the amount of change of the calculation interval according to the calculated position calculated by the position calculating unit and the distance between the markers. A program of a form may be configured.

In the seventh embodiment, the unrecognizable state detecting means for detecting that the marker cannot be recognized from the live view (for example, the control board 1550 in FIG. 1, the processing unit 200 in FIG. 11, the game calculation unit 202). , Recognition state determination unit 222 in FIG. 12, steps S60 to S64 in FIG.
While the unrecognizable state is detected, the unrecognized direction control means (for example, the control board 1550 in FIG. 1, the processing unit 200 in FIG. 11) that controls the display direction of the object in accordance with a change in the posture of the computer. Game calculation unit 202, position change suppression control unit 208 in FIG. 12, step S72 in FIG.
As a program in any one of the first to sixth modes for further functioning the computer.

  According to the seventh embodiment, the same effects as in the first to sixth embodiments can be obtained, and since the marker cannot be recognized, the coordinate alignment between the real space and the CG space cannot be performed accurately. By temporarily controlling the display direction of the object according to the posture change, it becomes difficult to understand that the marker cannot be recognized apparently. If the marker becomes unrecognizable, it is possible to avoid a game interruption immediately.

The eighth form is an unrecognizable state detecting means for detecting that the marker cannot be recognized from the live view (for example, the control board 1550 in FIG. 1, the processing unit 200 in FIG. 11, the game calculation unit 202). , Recognition state determination unit 222 in FIG. 12, steps S60 to S64 in FIG.
Recognition return state detection means (for example, control board 1550 in FIG. 1, processing unit 200 in FIG. 11, game calculation unit 202, FIG. 12) that detects a recognition return state in which the marker can be recognized again from the unrecognizable state. Recognition state determination unit 222, steps S60 to S64 in FIG.
While the unrecognizable state is detected, object position change suppression control means (for example, control board 1550 in FIG. 1, position change suppression control unit in FIG. 12) that suppresses a change in the display position of the object in the live view. 208, step S72 of FIG. 19),
As a program according to any one of the first to sixth embodiments for further functioning the computer.

  According to the 8th form, while the same effect as any of the 1st-6th form is acquired, while the marker is unrecognizable, the display position change of an object can be suppressed.

  In the eighth embodiment, the unrecognized direction control means (for example, the control board 1550 in FIG. 1, FIG. 1) that controls the display direction of the object according to the change in the attitude of the computer while the unrecognizable state is detected. 11 is a program of a seventh form for causing the computer to further function as the processing unit 200, the game calculation unit 202, the position change suppression control unit 208 in FIG. 12, and the step S72 in FIG.

  According to the eighth embodiment, the same effect as that of the seventh embodiment can be obtained, and since the marker cannot be recognized, the coordinate alignment between the real space and the CG space cannot be accurately performed. By tentatively controlling the display direction of the object according to, it becomes difficult to understand that the marker is unrecognizable in appearance. If the marker becomes unrecognizable, it is possible to avoid a game interruption immediately.

  The ninth form is a computer-readable information storage medium storing the program of any one of the first to eighth forms.

  The “information storage medium” mentioned here includes, for example, a magnetic disk, an optical disk, an IC memory, and the like. According to the ninth aspect, by causing a computer to read and execute the program of any of the first to eighth aspects, causing the computer to exert the same effect as any of the first to eighth aspects. Can do.

The figure which shows the structural example of the game device corresponded to the computer used in order to play a game in 1st Embodiment. The conceptual diagram for demonstrating the play style in 1st Embodiment. The figure which shows an example of a game screen and a live view. The conceptual diagram of the area | region set in order to carry out arrangement | positioning control of an enemy character. A conceptual diagram of a basic area. The conceptual diagram which shows the setting method of an element area | region. The conceptual diagram which shows the way of thinking of the depth of an element area | region. The conceptual diagram for demonstrating the utilization form of the relative position and relative attitude | position of a player viewpoint with respect to AR marker. The principle figure explaining the utilization form of the relative position and relative attitude when the AR marker is lost. The conceptual diagram for demonstrating the mechanism of switching of a host function expression machine. The functional block diagram which shows the function structural example in 1st Embodiment. The functional block diagram which shows the functional example of the game calculating part in 1st Embodiment. The figure which shows the data structural example of AR marker registration data. The figure which shows the data structural example of game participating machine registration data. The figure which shows the data structural example of the own machine information queue. The figure which shows the data structural example of an enemy character arrangement | positioning information cue | queue. The flowchart for demonstrating the flow of the main processes in 1st Embodiment. The flowchart for demonstrating the pre-game preparation process flow. The flowchart for demonstrating the own apparatus information calculation process flow. The flowchart for demonstrating a host function expression machine setting process flow. The flowchart for demonstrating the host function execution processing flow. The flowchart for demonstrating the flow of a basic area | region setting process. The flowchart for demonstrating an element area | region setting process flow. The flowchart for demonstrating the host function execution processing flow. The functional block diagram which shows an example of a function structure of the game calculating part in 2nd Embodiment. The flowchart for demonstrating the flow of the main processes in 2nd Embodiment. The conceptual diagram which shows the modification which seems to have the relative positional relationship of a virtual camera, a player viewpoint, and a player character.

[First Embodiment]
As an embodiment to which the present invention is applied, an example in which a multiplayer shooting game is executed in an augmented reality space realized using a marker method AR will be described.

  FIG. 1 is a diagram showing a configuration example of a game apparatus 1500 which is a computer used for game play in the present embodiment and is a kind of electronic equipment, and includes (1) a front external view and (2) a back surface. It is an external view. The game apparatus 1500 includes a direction input key 1502, a home key 1504, a touch panel 1506, a speaker 1510, a microphone 1512, a GPS (Global Positioning System) antenna 1514, a CCD camera module 1516, a control board 1550, and a computer. A memory card reader 1542 that can read and write data from a memory card 1540 that is a readable information storage medium. In addition, a built-in battery, a power button, a volume control button, etc., not shown, are provided.

  The CCD camera module 1516 is a module on which an autofocus mechanism, a CCD image sensor, and an image signal generation chip are mounted, and is arranged so that the back direction of the game apparatus 1500 can be photographed. Note that the image sensor element is not limited to the CCD, but may be an element of another type such as a CMOS.

  The control board 1550 includes various microprocessors such as a CPU (Central Processing Unit) 1551, GPU (Graphics Processing Unit), and DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), and various ICs such as VRAM, RAM, and ROM. A memory 1552 is mounted as appropriate.

The control board 1550 includes a wireless communication module 1553 for wireless connection with a wireless base station such as a mobile phone or a wireless LAN, a GPS module 1554, an electronic compass 1555, a 3-axis gyro 1556, and a 3-axis acceleration sensor. 1557 is mounted.
In addition, a driver circuit of the touch panel 1506, a circuit that receives signals from the direction input key 1502 and the home key 1504, an output amplifier circuit that outputs an audio signal to the speaker 1510, and an input signal that generates an audio signal input from the microphone 1512 A so-called I / F circuit (interface circuit) such as a generation circuit and a signal input / output circuit for the memory card reader 1542 is mounted. Each element mounted on the control board 1550 is electrically connected via a bus circuit or the like, and is connected so as to be able to read / write data and transmit / receive signals.

  The GPS module 1554, together with the GPS antenna 1514, constitutes a means for acquiring position information using GPS. The GPS module 1554, based on the signal from the GPS satellite 3 received by the GPS antenna 1514, at predetermined time intervals (for example, every second) and positional information (for example, latitude / longitude) and other information (absolute time) Is output as data that can be processed by the control board 1550. The system used for positioning is not limited to GPS. Other satellite positioning systems may be used, and positioning systems that do not use satellites can also be used. Examples of the latter include, for example, acquiring position information preset in the base station from a radio base station to which the radio communication module 1553 can wirelessly connect, or using a signal arrival time difference from three radio base stations The position information may be calculated by a surveying method.

The control board 1550 reads game programs and data necessary for game execution from the memory card 1540 and temporarily stores them in the IC memory 1552. Then, the game program is executed to execute arithmetic processing, and the direction change and acceleration change detected by the direction input key 1502, the home key 1504, the operation input from the touch panel 1506, the 3-axis gyro 1556, and the 3-axis acceleration sensor 1557 are detected. Accordingly, each part of the game apparatus 1500 is controlled to execute the video game. In the present embodiment, the game apparatus 1500 is configured to read out necessary programs and various setting data from the memory card 1540, but may be configured to download from the server apparatus or the like via the communication line 1.
Since game device 1500 is a computer, “own device” in this specification can be read as “own computer”. Similarly, “another machine” can be read as “another computer”.

[Game Overview]
Next, an outline of the game in the present embodiment will be described.
FIG. 2 is a conceptual diagram for explaining the play style in the present embodiment, (1) a perspective view showing a situation in a real space, (2) a CCD camera module 1516 of the game apparatus 1500, an AR marker 10, and FIG. 3 is a perspective view showing a state in a virtual three-dimensional space in which objects appearing in a game are arranged.

  The video game in the present embodiment belongs to a multi-play shooting game belonging to a game set in an AR space realized by using the AR marker 10, that is, a so-called AR game. As shown in FIG. 2 (1), the players PL 1 to PL 3 each hold the game apparatus 1500 and play while shooting the AR marker 10 placed on the table 9 or the like with the CCD camera module 1516. The game apparatus 1500 of the player PL1 in the figure is the original host machine, and performs the reception process of the client machine participating in the game, and is responsible for the NPC control of the enemy character 7 and the like. The game apparatuses 1500 of the players PL2 and PL3 are guest machines, which acquire and use NPC control data from the host machine. Of course, the number of players is not limited to three as shown in the figure.

The AR marker 10 in the present embodiment is a card having a prescribed shape in which a reference direction is represented on at least one surface and a design for identifying the type of the marker is provided, and there are a plurality of types.
In the game apparatus 1500 of this embodiment, the CCD camera module 1516 is fixed to the back side of the apparatus main body so as to photograph the back direction of the apparatus main body. Therefore, in this embodiment, the viewpoint position of the player is the position of the game apparatus 1500 and can be handled in the same way as the viewpoint position of the CCD camera module 1516. Similarly, the viewpoint posture of the player is the posture of the game apparatus 1500 and is handled in the same manner as the posture of the CCD camera module 1516.

  As shown in FIG. 2 (2), the game apparatus 1500 detects the AR marker 10 from the image photographed by the CCD camera module 1516 and recognizes its type. Further, the three-dimensional relative position (Xc, Yc, Zc) and relative posture (ωp, ωy, ωr) up to the recognized AR marker 10 are measured. For example, since the actual size of the AR marker 10 (card) is defined, the relative position based on the scale in the real space can be obtained from the pixel size occupied by the AR marker 10 in the captured image of the CCD camera module 1516. .

  Then, based on the measured relative position and relative orientation, a surface (for example, a table surface) on which the AR marker 10 is arranged is a reference surface in the virtual three-dimensional space (for example, also called a ground in the game world, a virtual map, or the like). A game space is formed so as to match. The game space is represented by a coordinate system (Xw, Yw, Zw) in a virtual three-dimensional space, and is matched with a coordinate system (Xm, Ym, Zm) in the real space with the AR marker 10 as a reference.

  As shown in FIG. 2 (3), the virtual three-dimensional space is a so-called CG space. In the virtual three-dimensional space, target enemy characters 7 and objects such as background objects are arranged to form a game space. In addition, in the virtual three-dimensional space, in addition to the player characters PC1 to PC3 of the players PL1 to PL3, the enemy character 7 and the flying object 6 launched from each character (not shown in FIG. 2; shown in FIG. 3) Objects such as are also placed.

  The virtual three-dimensional space in each game device 1500 is defined as a common coordinate axis in principle because the AR marker 10 is used as a reference. Virtual cameras CM1 to CM3 of each game device 1500 are set in the virtual three-dimensional space. The player characters PC1 to PC3 corresponding to each game device 1500 are arranged at predetermined relative positions (Xp, Yp, Zp) with respect to the virtual cameras CM1 to CM3 of the corresponding game device. Specifically, in this embodiment, the player characters PC1 to PC3 are arranged obliquely below and forward of the virtual cameras CM1 to CM3 so as to correspond to a third person viewpoint. It should be noted that the relative positional relationship between the player character and the virtual camera does not mean an absolute fixed value, although it is determined in advance, and can include an appropriate shift or fluctuation.

  FIG. 3 is a diagram showing an example of the game screen and live view in the present embodiment. (1) An example of a game screen in the game device 1500 possessed by the player PL1, (2) Photographed by the CCD camera module 1516. It is an example of a live view.

Each game device 1500 generates a CG (computer graphics) image viewed from the corresponding virtual camera CM <b> 1 to CM <b> 3, and displays it on a live view photographed by the CCD camera module 1516.
Incidentally, in the example of FIG. 3A, the player character PC1 of the player PL1, the enemy character 7 around the AR marker 10, the other player PL2 beside the table 9, and the corresponding player character PC2 are displayed on the screen. It is reflected. In the center of the screen, a shooting sight 8 used by the player PL1 during the game is displayed. In this way, the virtual world recognized by the player by synthesizing CG such as a game character with the video of the real space is the “augmented reality space” that becomes the stage of the game.

In this embodiment, the angle of view of the CCD camera module 1516 and the angle of the virtual cameras CM1 to CM3 are different.
Since the CCD camera module 1516 has a meaning that the photographing range is wide, the photographing field angle is set to a wide angle. For example, the focal length is 15 mm to 24 mm in terms of 35 mm film lens, and the nominal diagonal field angle is about 90 °. On the other hand, the shooting angle of view of the virtual cameras CM1 to CM3 is approximately 50% of the angle of view of the CCD camera module 1516 in consideration of playing the AR game while seeing through the real space through the touch panel 1506 of the game device 1500. It is set so narrow that it becomes about. Considering the play style, it is preferable that the angle of view is such that when the player holds the game device 1500 forward with the elbows slightly bent, the illusion of viewing the real world from the display portion of the touch panel 1506 is as if it is seen through. It is. For example, it is set to 40 mm to 55 mm in terms of a lens for 35 mm film and a nominal diagonal field angle of around 45 °.
Therefore, the background of the game screen shown in FIG. 3A is not the same as the live view taken by the CCD camera module 1516 shown in FIG. 3B, but matches the shooting angle of view of the virtual cameras CM1 to CM3. It is obtained by cropping (using the edges of the image).

[Regarding the placement of enemy characters]
As described above, since the game according to the present embodiment is a marker-type AR game, in order to appropriately match the coordinate system of the real space and the virtual three-dimensional space, the AR marker 10 is always set to the CCD camera module 1516. It is desirable to play so as to be included in the shooting range.

  Due to the characteristics of the shooting game, the player plays while changing the position and posture of the game apparatus 1500 so that the aim 8 is aligned with the enemy character 7. Therefore, when the enemy character 7 is far away from the AR marker 10, the act of aiming at the enemy character 7 induces the lost AR AR (a state in which it does not appear in the shooting range). However, the player tries to keep the position and posture of the game apparatus 1500 so that the aim 8 does not come off the enemy character 7. Therefore, if the position of the enemy character 7 is controlled well, the AR marker 10 can be included in the shooting range (in other words, the AR marker 10 does not fall outside the shooting range).

  Therefore, in the present embodiment, as shown in FIG. 4, an arrangement area 12 in which the enemy character 7 can be arranged so as to make it difficult to lose the AR marker 10 according to the position and posture of the game device participating in the game (shaded in the figure). The displayed range) is set, and the placement control is performed so that the enemy character 7 stays in the placement area 12. In the present embodiment, the arrangement area 12 is set as an overlap area between the basic area 13 and the element area 14 obtained for each game participant (that is, for each player viewpoint). In the example shown in the figure, only two game apparatuses 1500 are displayed for easy understanding.

FIG. 5 is a conceptual diagram showing a method for setting the basic area 13.
The basic area 13 is a range of a predetermined shape centered on the AR marker 10 and defined by the basic length Lb. The basic length Lb is described by a function f (Lc) using a relative distance Lc (see FIG. 2B) from the AR marker 10 of the game participant as a variable.

  The shape of the basic region 13 is preset according to the type of the AR marker 10. In the example of FIG. 5, a spherical shape having a basic length Lb as an example is illustrated. However, for example, the basic length Lb may be a distance from the center of the cube to eight vertices as a cubic range. The dimension to which the shape and the reference length Lb are assigned can be appropriately changed.

  As the relative distance Lc serving as a variable of the function f (Lc), the smallest relative distance Lc is adopted among the individual relative distances Lc with respect to the AR marker 10 of each game participant. However, a minimum value is set for the size of the function f (Lc). This minimum value is set in advance so that the action range of the enemy character 7 does not become too small. Note that the relative distance Lc adopted as a variable of the function f (Lc) is not limited to the minimum value among the relative distances Lc for each game participant, and may be a maximum value, an average value, a median value, or the like.

  FIG. 6 is a conceptual diagram showing a method for setting the element region 14. The element region 14 is defined as a shooting range in which the posture change that allows the AR marker 10 not to deviate from the shooting range of the CCD camera module 1516 with the game device 1500 (that is, the player viewpoint) as a reference.

  Specifically, it is assumed that the AR marker 10 is currently shown in the live view by the CCD camera module 1516 as shown in FIG. The outer rectangular frame is a shooting frame and shows the outline of the live view. On the other hand, an inner rectangular frame indicates an outline that is cropped and used as the background of the game screen.

  In order to obtain the “shooting range in which the AR marker 10 can tolerate a posture change that does not deviate from the shooting range of the CCD camera module 1516”, in the present embodiment, the position of the game apparatus 1500 is first fixed, and to what extent the CCD Consider whether the change in the shooting direction of the camera module 1516 can be allowed.

  Specifically, as shown in FIG. 6 (2), the right end of the AR marker 10 and the right end of the imaging frame are close to each other while satisfying a predetermined condition, with the size of the AR marker 10 currently captured as it is, and the AR The “first posture” that satisfies the condition in which the lower end of the marker 10 and the lower end of the photographing frame are close to each other is obtained, and the optical axis AL1 of the CCD camera module 1516 in the posture is obtained. Then, assuming that the shooting direction is shifted diagonally from the first posture, the left end of the AR marker 10 is close to the left end of the shooting frame, and the upper end of the AR marker 10 and the upper end of the shooting frame are close to each other. 2 ”and the optical axis AL2 of the CCD camera module 1516 in that posture.

  Next, as shown in FIG. 6 (3), the left end of the AR marker 10 and the left end of the shooting frame meet in a predetermined condition and are close to each other, and the lower end of the AR marker 10 and the lower end of the shooting frame meet each other. The “third attitude” is obtained, and the optical axis AL3 of the CCD camera module 1516 in the attitude is obtained. Then, assuming that the shooting direction is shifted diagonally from the third posture, the right end of the AR marker 10 is close to the right end of the shooting frame, and the upper end of the AR marker 10 is close to the upper end of the shooting frame. 4 postures "and the optical axis AL4 of the CCD camera module 1516 in the posture are obtained.

  Then, as shown in FIG. 6 (4), the element region 14 is a quadrangular pyramid region surrounded by the four optical axes AL 1 to AL 4 from the viewpoint position of the CCD camera module 1516.

  The depth Db of the element region 14 corresponding to the height of the quadrangular pyramid region is set based on the depression angle θc with respect to the planar direction of the AR marker 10 viewed from the player viewpoint (see FIG. 7).

FIG. 7 is a conceptual diagram showing the concept of the depth of the element region 14.
As shown in FIG. 7A, the depth Db of the element region 14 is considered as the maximum distance from the viewpoint at which the AR marker 10 can be properly recognized. As shown in FIG. 7B, the recognition performance of the AR marker 10 is highest when the CCD camera module 1516 captures the AR marker 10 from the front (the depression angle θc = π (90 °)), and the depth Db is maximum. (Db1). Then, as the depression angle θc becomes shallower (smaller) with respect to the plane (design surface) of the AR marker 10, the recognition performance decreases (Db2), and the recognition performance eventually reaches an impractical level (θmin). it can.

  The relationship between the depression angle θc and the depth Db is determined by the resolving power based on the specifications of the imaging device and optical system of the CCD camera module 1516, the size of the AR marker 10, and the like. In the present embodiment, AR markers 10 having different sizes can be used, and are categorized in advance according to the size of the marker. Which category it belongs to can be identified by design recognition of the AR marker 10. Therefore, as shown in FIG. 7 (3), a function g (θc) for calculating the depth Db using the depression angle θc as a variable is set for each category of the AR marker 10, and the category of the recognized AR marker 10 is set. The function g (θc) corresponding to is selected, and the depth Db is obtained using the depression angle θc at that time.

  Considering that the shooting angle of view of the virtual cameras CM1 to MC3 is narrower than the shooting angle of view of the CCD camera module 1516 and that the aim 8 is displayed at the center of the game screen, it should be within the shooting range indicated by the element area 14. For example, even if the player slightly changes the position and orientation of the game apparatus 1500, the AR marker 10 is captured within the shooting range of the CCD camera module 1516 if the aiming target 8 is aimed at the enemy character 7. Can do.

  Now, as shown in FIG. 4, the arrangement area 12 is set as an overlapped portion of four areas of the basic area 13 and each element area 14 (14a, 14b) obtained for each game device (that is, for each player viewpoint). Is done.

  As described above, in this embodiment, when the relationship between the shooting angle of view of the virtual camera CM1 to CM3 with respect to the shooting angle of view of the CCD camera module 1516 and the fact that the aim 8 is displayed at the center of the game screen are combined. As long as the enemy character 7 is placed in the AR, the AR marker 10 is always recognized correctly, and unnatural display such as the display position of the game character deviating from the real space does not occur.

Even if the AR marker 10 is not shown on the game screen displayed on the touch panel 1506, the CCD camera module 1516 can continue to capture the AR marker 10 properly. That is, even when the AR marker 10 is not always shown in the game screen, the AR game continues normally. For example, the enemy character 7 seems to walk along the table surface on which the AR marker 10 is placed.
Even if the AR marker 10 is not shown in the game screen during play, the enemy character 7 that is not inconsistent with the real space is displayed, so that it is impossible for the player to play at a place away from the AR marker 10. The sense of blockage is reduced, and an open feeling as if the game is being played using all the wide space where I am now is obtained, and the interest as an AR game is enhanced.

  Further, the arrangement area 12 set in this way has a different shape from the basic area 13 due to the influence of the plurality of element areas 14 that change depending on the situation, so that the action range of the enemy character 7 becomes irregular. The effect which raises the interest of a game is acquired.

  In addition, by including the basic area 13 in the setting of the arrangement area 12, even if the AR marker 10 cannot be recognized by all the game apparatuses 1500 and the element area 14 cannot be obtained, the arrangement position of the enemy character 7 Can be kept properly.

[Auxiliary function assuming loss of AR marker: Part 1]
Even if the arrangement region 12 is set as described above, it is not always possible to continuously capture the AR marker 10. For example, it is assumed that the action of turning the game device 1500 toward the enemy character 7 with excitement of play becomes excessive, and the AR marker 10 is out of the shooting range of the CCD camera module 1516 unintentionally. Therefore, in the present embodiment, two auxiliary functions are provided so that inconvenience does not occur in the game progress even when the AR marker 10 is lost.

First, the first auxiliary function will be described.
FIG. 8 is a conceptual diagram for explaining how to use the relative position and relative posture of the player viewpoint with respect to the AR marker 10 which is the premise of the first auxiliary function.
The game apparatus 1500 according to the present embodiment has a relative position (Xcn, Ycn) of itself (CCD camera module 1516) with respect to the AR marker 10 at times t1, t2,..., Tm (m is a natural number: an identification number of calculation timing) in a predetermined cycle. , Zcn) and relative posture (ωpn, ωyn, ωrn). Then, a predetermined number of the calculated relative positions and relative postures are stored and managed in a queue in a first-in first-out manner. The relative position and the relative attitude stored in the queue are weighted and averaged, and the weighted average value is regarded as the current position and the current attitude, and is used as the position and attitude of the virtual camera CM in the CG space. In the example of FIG. 8, an example is shown in which four sets of relative positions and relative postures are stored in the queue, but the storage number can be set as appropriate.

  The weighting wn in the weighted average (n is a natural number: the identification number of the calculation timing) is determined by the calculation order n and the weighting function h (n, Lc) having the relative distance L from the AR marker 10 to the player viewpoint as a variable. .

  Although the weighting function h (n, Lc) can be set as appropriate, in the present embodiment, with respect to the calculation order n, the weighting wn given to the newest relative position and relative posture is the largest, and the others are relatively It is set to be smaller. Further, the relative distance Lc is set so that the larger the relative distance Lc, the smaller the difference in weighting wn due to the difference in calculation order n, and the past relative position and relative posture are also emphasized.

  By using values obtained by averaging the relative position and relative posture (referred to as current position and current posture) for control of the virtual camera CM related to AR display control, so-called jitter due to hand shake caused by hand shaking over the game device 1500 (Small and unpleasant vibration of an AR display body such as a game character) and a case where the game apparatus 1500 is suddenly moved or changed in posture, and the unnatural display that the player character or the like moves instantaneously is suppressed. be able to.

  In addition, the influence of camera shake or the like increases as the game apparatus 1500 captures the AR marker 10 from a distance, but the characteristic of the weighting function h (n, Lc) with respect to the relative distance Lc is that the relative distance Lc is Since the weight wn for the past value is set to be relatively larger as the value is larger, the influence can be reduced. In addition, according to the characteristic of the relative distance Lc of the weighting function h (n, Lc), when the AR marker 10 is taken from near, the weighting wn to the past value becomes relatively small. The effect of camera shake will be relatively large, but blurring in this case is beneficial because it creates a visual effect that creates the tightness and presence of close combat.

  In the present embodiment, assuming that control using such a weighted average is performed, when the AR marker 10 is lost, the relative position with respect to the AR marker 10 is assumed to be fixed while only the posture change occurs. Control it.

FIG. 9 is a principle diagram for explaining how to use the relative position and the relative posture when the AR marker 10 is lost.
In the present embodiment, when the AR marker 10 is lost, the relative position is fixed to the value immediately before the loss until the recognition is restarted after the AR marker 10 is lost. In the example of FIG. 9, until the time t4, the AR marker 10 is in the imaging range of the CCD camera module 1516 and is correctly recognized and the relative position of the marker reference is obtained correctly. "Lost" is not captured in Therefore, the value of the relative position between the timings t5 and t7 stored in the queue is fixed to the value at the timing t4 immediately before the lost. On the other hand, a weighted average is performed for the posture. In other words, it is considered that the game apparatus 1500 does not move its position during the timing t5 to t7.

  Although the actual game device 1500 may be moved even during the lost, the state of the game screen display can be changed by continuing the processing with the relative position temporarily fixed. Even if there is a possibility that the relative position cannot be calculated correctly, the game can be continued without interruption.

  Even if the control is switched to “premise that there is no movement of the player viewpoint”, the relative position (virtual camera position) is fixed and the AR display control is continued. Instantaneous movement and jerky movements can be effectively suppressed.

[Auxiliary function assuming loss of AR marker: Part 2]
Moreover, in this embodiment, a guest machine can substitute for a host function as a 2nd auxiliary | assistance function supposing the case where the AR marker 10 is lost. In other words, the host function can be inherited in the game participant in accordance with the marker recognition state.

  FIG. 10 is a conceptual diagram for explaining the switching mechanism of the host function developing machine, and shows the recognition state of the AR marker 10 of each game device when it is assumed that there are three game devices 1500 participating in the game. An example of a combination and which game device 1500 expresses the host function in the combination is shown. Note that “In Sight” in the column of the marker recognition state means that the AR marker 10 can be recognized, and “LOST” means that the AR marker 10 is out of the imaging range and cannot be recognized (recognition impossible state). Means.

In the present embodiment, the game device 1500a, which is the host machine from the beginning, performs a process for accepting participation in the game. The accepted game devices 1500b and 1500c are guest machines.
The game device 1500a which is a host machine sets itself as the first expression priority in accordance with the reception process, and assigns the second or higher expression priority to the guest machine in the order of reception. In the figure, the white numbers given to the illustrations of the game devices represent the respective expression priorities.

  In addition, the game devices 1500a, 1500b, and 1500c according to the present embodiment can transmit a predetermined unrecognizable signal to another device when the AR marker 10 is lost. In other words, each game device can know the marker recognition state of the other device depending on whether or not the recognition failure signal from the other device has been received.

As shown in the combination of FIG. 10 (1), while all the game devices participating in the game can recognize the AR marker 10, the game device 1500a that is the original host device is used as the host device. Express function.
That is, in addition to the player character PC1 of the player, the NPC (non-player character) such as the enemy character 7 and the object such as the background object are controlled, and the effect objects such as the bullets flying are controlled. Deliver to guest machines. Of course, the control related to the enemy character 7 includes setting processing of the arrangement area 12.

  On the other hand, in the game devices 1500b and 1500c, which are guest machines, the operation of the player characters PC2 and PC3 of each player and the position in the virtual three-dimensional space are controlled by operation input and movement of the device. Controls the arrangement in the virtual three-dimensional space based on the information obtained from the host machine. That is, with respect to object control, the host machine and the guest machine have a client / server relationship.

  Here, if a state occurs in which the game apparatus 1500a which is the original host machine cannot recognize the AR marker 10, a deviation occurs in the alignment between the real space and the virtual three-dimensional space, and the enemy character 7 is tilted and arranged. There is a high possibility that a problem such as a failure will occur. In the guest machine that acquires the arrangement position information of the enemy character 7 from the host machine, the enemy character 7 is unnaturally displayed even though the AR marker 10 is not lost.

Therefore, the game apparatus 1500 according to the present embodiment detects a lost state of the AR marker 10 and transmits a predetermined “recognition impossible signal” to another device. If the host machine has lost the AR marker 10, the host function is stopped along with the transmission of the recognition failure signal.
On the other hand, each game device that has been a guest machine until then, when all the other machines higher than the expression priority given to itself based on the received unrecognizable signal are issuing unrecognizable signals, It is determined that the host function should be developed, and the host function is developed.

  Specifically, as shown in the combination of FIG. 10 (2), when the original host device (game device 1500a) becomes in an unrecognizable state, the game device 1500a Stop functioning and start functioning as a guest machine. That is, the control related to the enemy character 7 and the distribution of the control result are interrupted.

  On the other hand, the guest machine (game device 1500b) having the second highest expression priority has received the unrecognizable signal from all the machines having higher expression priority than itself, so that the host function is exhibited and the enemy character 7 and the like are controlled and controlled. Start delivering results. That is, the host function is inherited. Note that the guest machine (game device 1500c) with the third highest expression priority functions as a guest machine as it is.

  As shown in the combination of FIG. 10 (3), when the game apparatus 1500b expressing the host function also loses the AR marker 10, the guest machine (game apparatus 1500b) having the second highest expression priority ranks the host function. Stop and start functioning as a guest machine. Instead, since the guest machine (game device 1500c) with the third highest expression priority has received the unrecognizable signal from all the machines higher than itself, it is determined that the host function should be expressed. To express.

  When the recognition of the AR marker 10 is resumed in the game apparatus 1500a which is the original host machine from the combination state of FIG. 10 (3), the game apparatus 1500a resumes the host function and stops sending the notification of unrecognition. To do. Since the game device 1500c that has performed the host function does not receive the unrecognizable signal from the game device 1500a, the host device is stopped and only the function as the guest machine is exhibited again.

  Thus, according to the present embodiment, any aircraft that recognizes the AR marker 10 can always be configured to perform an object positioning process with the AR marker 10 as a reference. Therefore, if the game device that is the host machine becomes unable to recognize the AR marker, a misalignment occurs in the alignment between the real space and the virtual three-dimensional space (CG space), and the enemy character 7 is inclined and arranged. In this embodiment, such a situation can be prevented by the host proxy function of the guest machine.

[Description of functional block]
Next, a functional configuration for realizing the present embodiment will be described.
FIG. 11 is a functional block diagram illustrating an example of a functional configuration of the game apparatus 1500 according to the present embodiment. The game device 1500 includes an operation input unit 100, an imaging unit 104, an attitude detection unit 108, an acceleration detection unit 110, a processing unit 200, a sound output unit 350, an image display unit 360, a communication unit 370, A positioning signal receiving unit 380 and a storage unit 500 are provided.

  The operation input unit 100 outputs an operation input signal to the processing unit 200 in accordance with various operation inputs made by the player. For example, it can be realized not only by a button switch, a joystick, a touchpad, a trackball, etc., but also by an element that detects movement or posture, such as an acceleration sensor, angular velocity sensor, tilt sensor, geomagnetic sensor, etc. . The direction input key 1502 and the home key 1504 in FIG. 1 correspond to this. When the detection values of the 3-axis gyro 1556 and the 3-axis acceleration sensor 1557 are used for operation input, they are also applicable. In addition, the operation input unit 100 of this embodiment includes a touch operation input unit 102, which enables operation input by touching the display screen with a finger or the like. The touch panel 1506 in FIG. 1 corresponds to this.

  The imaging unit 104 receives light from the imaging target, converts it into an electrical signal, generates digital image data, and outputs the digital image data to the processing unit 200. For example, it is realized with a lens, a mechanical shutter, a shutter driver, a photoelectric conversion element such as a CCD image sensor module or a CMOS image sensor module, a digital signal processor (DSP) that reads out the amount of charge from the photoelectric conversion element and generates image data, and an IC memory. The In FIG. 1, the CCD camera module 1516 corresponds to this.

  Posture detection unit 108 detects a change in posture of game device 1500 and outputs a detection signal to processing unit 200. This can be realized with an electronic compass or gyroscope. The electronic compass 1555 and the three-axis gyro 1556 shown in FIG. In the posture detection, it is preferable to calculate three components of roll, pitch, and yaw by setting three orthogonal axes and aligning any one of them with the optical axis direction taken by the imaging unit 104. In this embodiment, since the CCD camera module 1516 is fixed to the game apparatus 1500, it can be said that the attitude detection unit 108 detects the attitude change of the player viewpoint.

  The acceleration detection unit 110 can detect the acceleration acting on the game device 1500. This corresponds to the triaxial acceleration sensor 1557 of FIG. In this embodiment, since the CCD camera module 1516 is fixed to the game apparatus 1500, it can be said that the acceleration detection unit 110 detects the acceleration accompanying the movement of the player viewpoint.

The processing unit 200 is realized by an appropriate combination of, for example, a microprocessor such as a CPU and a GPU, and an electronic component such as an ASIC and an IC memory. The processing unit 200 performs data input / output control with each functional unit, executes various arithmetic processes based on predetermined programs and data, operation input signals from the operation input unit 100, and the like, The operation of 1500 is controlled. In FIG. 1, the control board 1550 corresponds to this.
The processing unit 200 includes a game calculation unit 202, a sound generation unit 250, an image generation unit 260, a communication control unit 270, and a positioning control unit 280.

  The game calculation unit 202 executes various controls necessary for executing the game of the present embodiment. In addition, it is possible to appropriately execute timing processing such as a time limit using the system clock, flag management processing, and the like.

  FIG. 12 is a partial functional block diagram illustrating a configuration example of the game calculation unit 202 in the present embodiment. The game calculation unit 202 of the present embodiment includes a game participation management control unit 204, an own device information calculation unit 206, an own device information transmission control unit 220, an other device information reception control unit 221, and a recognition state determination unit 222. , An unrecognizable signal transmission control unit 223, a host function expression determination unit 224, a character display control unit 230, and an NPC information transmission control unit 240.

  The game participation management control unit 204 executes game participation related processing for multiplayer. For example, a process for causing the player to select whether the own machine is the host machine / guest machine is executed. When the host machine is selected in this selection process, a process for accepting / registering another machine as a game participant and a process for distributing registration information of the game participant are executed. In addition, regardless of the host machine / guest machine, player character setting processing such as allowing the player to select the type of player character is performed. The player character setting information is then distributed to other devices.

  In addition, the game participation management control unit 204 of this embodiment includes an expression priority setting unit 205. The expression priority order setting unit 205 sets the expression priority order in response to participation acceptance and controls distribution to other apparatuses when the own apparatus expresses the host function.

  The own device information calculation unit 206 calculates information related to the positional relationship of the player viewpoint of the own device with respect to the AR marker 10. In the present embodiment, a relative position calculation unit 207, a current position determination unit 209, a relative posture calculation unit 210, a current posture determination unit 211, and a weight setting unit 212 are included.

The relative position calculation unit 207 periodically calculates information indicating the relative positional relationship of the own apparatus with respect to the marker using the captured image of the AR marker 10. That is, the latest relative position of the player viewpoint at that time is measured and calculated.
Although the content of the calculated relative position information can be set as appropriate, for example, a predetermined position of the AR marker 10 (for example, one of the center or corner of the card) is used as a reference point and is based on the reference direction indicated by the AR marker 10. The triaxial orthogonal coordinates (Xm, Ym, Zm) are obtained. Then, the relative position (Xc, Yc, Zc) and the relative distance Lc in the coordinate system are calculated. Specifically, the AR marker 10 is detected by analyzing the image captured by the imaging unit 104, the type and orientation of the AR marker 10 are determined, and the relative position of the marker reference is calculated. These processes can be realized by appropriately using a known AR technique. Further, the relative distance Lc is calculated in units of actual size based on the card size (actual size; for example, 5 cm × 7 cm) set in advance corresponding to the type of the AR marker 10 and the number of dots of the AR marker 10 in the photographed image. calculate.

In the present embodiment, the relative position calculation unit 207 functions as the position change suppression control unit 208.
The position change suppression control unit 208 performs a process of suppressing the position movement of the object in the game space when the AR marker 10 cannot be recognized. Specifically, the relative position is determined on the assumption that there is no movement of the position of the game device (no movement of the position of the player's viewpoint) from when recognition is impossible to when the state returns to the recognition state. More specifically, the relative position newly calculated after the recognition is impossible is fixed to the relative position calculated immediately before the recognition becomes impossible, and the fixation is released when the recognition is restored (see FIG. 9). In other words, while the unrecognizable state is detected, it functions as an unrecognizable direction control unit that controls the display direction of the object according to the change in the posture of the computer.

  The current position determination unit 209 uses a weighted average using the weighting Wn set by the weighting setting unit 212 from the relative position information of the past a predetermined amount calculated by the relative position calculation unit 207 to the latest relative position. The position information applied to CM control is calculated (see FIG. 8).

  The relative attitude calculation unit 210 periodically calculates the attitude information of the own device based on the detection result by the attitude detection unit 108. For example, on the basis of the optical axis direction by the imaging unit 104, the pitch, yaw, and roll components, the predetermined position of the AR marker 10 (for example, one of the center or corner of the card), the reference point and the reference direction indicated by the marker The relative orientation (ωp, ωy, ωr) with respect to the three-axis orthogonal coordinates (Xm, Ym, Zm) based on the above is calculated.

  The current posture determination unit 211 calculates posture information applied to the control of the virtual camera CM by a weighted average using weights set by the weight setting unit 212 from a predetermined amount of past posture information to the latest posture information. (See FIG. 8).

  The weighting setting unit 212 refers to a predetermined function or predetermined table data according to the calculation time associated with the latest relative distance Lc with respect to the weighting Wn (see FIG. 8) for each information having different calculation time that can be processed by the weighted average. And decide. In other words, the distance between the first weight change function weighted according to the calculation time of the own machine position information and the own machine attitude subjected to the weighted average, and the AR marker 10 at the time of calculating the own machine position information and the own machine attitude A second weighting change function for weighting according to L is realized.

The own device information transmission control unit 220 performs processing for transmitting various information related to the own device to other devices participating in the game. For example, this includes operation input information from the operation input unit 100, current position, current posture, relative distance Lc, element region 14 definition information, and the like.
On the other hand, the other device information reception control unit 221 includes information related to the other device (operation input information by the operation input unit 100 in the other device, current position, current posture, relative distance Lc, definition information of the element region 14, and the like). Receive control.

The recognition state determination unit 222 can detect the recognition state of the AR marker 10. Specifically, when the AR marker 10 cannot be detected in the image captured by the imaging unit 104, or when the design of the marker cannot be recognized, the recognition unrecognized state is detected and the recognition return state is established. Is detected.
In the unrecognizable state, for example, a case where the AR marker 10 is far away and sufficient information for design identification cannot be obtained due to the relationship with the resolution capability of the imaging unit 104, or the card surface of the AR marker 10 Similarly, a case where sufficient information for design identification cannot be obtained because the optical axis direction of the CCD camera module 1516 is shallow, and a case where a part of the AR marker 10 remains outside the imaging range and sufficient information for design identification cannot be obtained. Can be considered.

  The unrecognizable signal transmission control unit 223 executes a process for transmitting a predetermined “unrecognizable signal” to other game participants when the unrecognized state is detected by the recognized state determining unit 222. It can also be said that it functions as an inherited signal transmission control unit that transmits an inherited signal for requesting the expression of the host function to another machine.

  The host function expression determination unit 224 determines whether or not the host device should function as a host device based on the presence or absence of recognition of each participating device of the game. In this embodiment, all other devices that are higher than the expression priority of the own device are recognized based on the expression priority set for the own device and the monitoring result of the unrecognizable state of the own device and the other device. When it is disabled, it is determined that it should function as a host machine.

  The character display control unit 230 controls the arrangement of objects such as characters in the virtual three-dimensional space (CG space) based on the positional relationship of the game participants with respect to the AR marker 10. In other words, the display positions of the characters such as the player characters PC1 to PC3 and the enemy character 7 in the augmented reality space are controlled. The character display control unit 230 according to the present embodiment includes an area setting unit 231 and an NPC control unit 235.

  The area setting unit 231 sets an area in the augmented reality space where a predetermined object (enemy character 7 in this embodiment) is displayed. Specifically, it includes a basic area setting unit 232, an element area setting unit 233, and an arrangement area setting unit 234.

  The basic area setting unit 232 sets the basic area 13 based on the relative distance Lc from the AR marker 10 to each player viewpoint. That is, the basic size of the arrangement area 12 is set. Specifically, the basic length Lb of the basic area 13 is obtained by a predetermined function f (Lc) using the minimum value as a variable among the relative positions Lc for each game participant. Then, the basic shape preset according to the type of the AR marker 10 is enlarged / reduced so that the predetermined dimension becomes the basic length Lb, and the size is changed to set the basic region 13.

  The element area setting unit 233 uses the own device shooting range and the other device shooting range when the AR marker 10 is shot at the viewpoint position of each player so that the AR marker 10 is not easily lost from the viewpoint of each player. The element area 14 in which the enemy character 7 can be placed is set. In addition, it functions as a depth changing unit that changes the depth of the photographing range of the own device and the photographing range of the other device using the depression angle θc with respect to the planar direction of the AR marker viewed from the player's viewpoint.

  The arrangement area setting unit 234 uses the basic area 13 and the element area 14 to set the arrangement area 12 in which the enemy character 7 can be arranged while making it difficult for the AR marker 10 to be lost for any game participant. In the present embodiment, overlapping portions of the basic region 13 and each element region 14 are obtained and set as the arrangement region 12.

  The NPC control unit 235 controls the placement of NPCs (non-player characters) such as the enemy character 7, effect objects such as the flying object 6, and background objects that form the background of the game space. So-called “AI control” is performed. The control information (position coordinates, orientation, motion control data, etc.) of an object such as NPC is history information for a predetermined period (for example, several frames of a game image) associated with each object, and play data stored in the storage unit 500. It is stored in the enemy character arrangement information queue 544 of 541 (see FIG. 11). The NPC control unit 235 according to this embodiment includes a control start special control unit 236 and a movement direction control unit 237.

The control start special control unit 236 performs control so as to reduce the movement speed and / or movement speed of the object to be controlled for a predetermined period (for example, several frames of the game image) from the start of control. Specifically, past control information stored in the enemy character arrangement information queue 544 and newly calculated control information are averaged to obtain actual control information.
For example, at the timing when the game device responsible for the host function is switched, a jump of the control information value may occur before and after switching, but the average value of the past value and the latest value should be used for actual control. Therefore, it is possible to suppress unnatural display such as instantaneous movement or shift / blur of the object at the switching timing. In particular, this is effective against a change (shift / blur) of the display position of the object.

  The movement direction control unit 237 determines the movement direction of the object to be controlled based on the image position of the AR marker 10 in the captured image (live view) by the imaging unit 104, and the relative direction of the object in the arrangement area 12. And a process of determining a moving direction based on the position.

  In the former process, the object position of the enemy character 7 moves closer to the image position of the AR marker 10 with a higher probability as the image position of the AR marker 10 in the live view of the host function developing machine is farther from the image center. Control. In other words, the enemy character 7 is closer to the AR marker 10 as the player viewpoint direction of the host function developing machine (the optical axis direction of the CCD camera module 1516 is the same as the visual line direction of the virtual camera) is farther from the AR marker 10. Control the operation. Specifically, the moving direction of the enemy character 7 is determined so that the enemy character 7 approaches the AR marker 10 as the angle between the direction from the player (game device) to the AR marker 10 and the player viewpoint direction increases. . When determining the movement direction and the movement target point by the lottery process, it can be realized by temporarily changing the lottery probability or executing the lottery again when the lottery result does not approach the AR marker 10.

  For example, if the enemy character 7 is in the latest placement area 12, a process for selecting a moving direction so as not to go out of the area, or a distance from the outline of the placement area 12 is calculated and a predetermined approach condition is set. When a satisfying state is detected, a process for temporarily reducing the next movement amount is performed. Further, if the enemy character 7 is outside the area, a process for selecting the movement direction so as to return to the area or a process for temporarily increasing the movement amount so as to return as quickly as possible can be applied. .

  The NPC information transmission control unit 240 controls transmission of the NPC control information obtained by the NPC control unit 235 to another device.

  The area setting unit 231, the NPC control unit 235, and the NPC information transmission control unit 240 are functional units that appear only when the own device should function as a host device, and these are the host function execution control unit 249. Configure.

  Returning to FIG. 11, the sound generation unit 250 is realized by, for example, a digital signal processor (DSP), a processor such as a voice synthesis IC, an audio codec that can play back an audio file, and the like. Sound effects, BGM, and sound signals of various operation sounds are generated and output to the sound output unit 350.

  The sound output unit 350 is realized by a device that outputs sound effects such as sound effects, BGM, telephone communication sound, and the like related to the game based on the sound signal input from the sound generation unit 250. In FIG. 1, the speaker 1510 corresponds to this.

  The image generation unit 260 is, for example, a processor such as a GPU or a digital signal processor (DSP), a program such as a video signal IC or a video codec, an IC memory for drawing frames such as a frame buffer, or an IC used for developing texture data. This is realized by a memory or the like. The image generation unit 260 generates one game screen in one frame time (for example, 1/60 second) based on the processing result by the game calculation unit 202, and outputs the generated image signal of the game screen to the image display unit 360. To do.

  The image display unit 360 displays various game images based on the image signal input from the image generation unit 260. For example, it can be realized by an image display device such as a flat panel display, a cathode ray tube (CRT), a projector, or a head mounted display. In the present embodiment, the touch panel 1506 in FIG. 1 corresponds to this.

The communication control unit 270 executes data processing related to data communication, and realizes data exchange with an external device via the communication unit 370.
A communication unit 370 related to this is connected to the communication line 1 to realize communication. For example, it is realized by a wireless communication device, a modem, a TA (terminal adapter), a cable communication cable jack, a control circuit, and the like, and the wireless communication module 1553 of FIG. 1 corresponds to this.

  The positioning control unit 280 performs control for calculating position information from a signal obtained from the positioning system via the positioning signal receiving unit 380, that is, performs positioning calculation and management of the result. For example, the GPS module 1554 shown in FIG. When the known position information of the mobile phone base station is used for the positioning calculation, the wireless communication module 1553 for wireless communication with the base station is also applicable.

  The storage unit 500 stores a program and various data for realizing various functions for causing the processing unit 200 to control the game apparatus 1500 in an integrated manner. Further, it is used as a work area of the processing unit 200, and temporarily stores calculation results executed by the processing unit 200 according to various programs, input data input from the operation input unit 100, and the like. Such a function is realized by, for example, an IC memory such as a RAM or a ROM, a magnetic disk such as a hard disk, or an optical disk such as a CD-ROM or DVD. In FIG. 1, the IC memory 1552 and the memory card 1540 mounted on the control board 1550 correspond to this.

The storage unit 500 of the present embodiment stores a system program 502 and a game program 504 for the game apparatus 1500.
A system program 502 is a program for realizing basic functions of the game apparatus 1500 as a computer.

  The game chromatogram 504 is a program for causing the processing unit 200 to realize the function as the game calculation unit 202. Although it is assumed that it is stored in advance in the memory card 1540, the present embodiment may be configured to download from a predetermined server before starting the game.

The storage unit 500 stores game space initial setting data 510, character initial setting data 511, AR marker registration data 512, and depth function setting data 514 in advance. These are also stored in the memory card 1540 in advance, but may be downloaded from a predetermined server before the game is started.
In addition, in the storage unit 500, game participation machine registration data 520, own machine information queue 530, current position 537, current attitude 538, and play data 541 are generated as the game progresses and updated as needed. And virtual three-dimensional space data 560 and host function inheritance timing 562 are stored. In addition, data necessary for executing various processes (for example, compressed / decompressed texture data, elapsed time, various timer values, counters, flags) and the like are also stored as appropriate.

  The game space initial setting data 510 is initial setting data that defines a background object for creating a game stage. This includes model data, texture data, arrangement position data, etc. of the background object.

  The character initial setting data 511 stores initial setting data of game characters such as the player characters PC1 to PC3 and the enemy character 7. For each character, initial values of model data, texture data, motion data, and ability parameters (attack power, defense power, HP, etc.) are stored for each character.

  The AR marker registration data 512 is data for registering an AR marker that can be used in a game, and is stored for each type that can be used. Then, as shown in FIG. 13, a marker type, a corresponding marker category, design pattern data, card dimensions, a basic area model that serves as a template for the basic area 13, and a basic that designates a dimension with a basic length Lb. Stores the length applicable dimension.

  The depth function setting data 514 is data for setting a function for calculating the depth Db of the element region 14, and is prepared for each marker category of the AR marker 10 (see FIG. 7).

The game participant registration data 520 is a data group related to registration of game devices participating in a game, and is generated by participation processing. For example, as shown in FIG. 14, in association with the game machine ID 521, address information 522 for specifying the other party by communication connection, expression priority 523, player character type 524, marker recognition state 525, host A host function expression flag 526 indicating whether the machine expresses a function is stored.
The game machine registration data 520 is distributed to each guest machine after an original is generated by the host machine in connection with the game participation process.

  The own device information queue 530 stores information such as the position and posture of the own device based on the AR marker 10 for a predetermined number of times of calculation. For example, as shown in FIG. 15, the relative position 533, the relative posture 534, and the relative distance 535 are stored in association with the calculation time 532.

  The current position 537 and the current posture 538 store the weighted average value of the relative position 533 and the weighted average value of the relative posture 534 stored in the own device information queue 530, respectively.

  The play data 541 is a data group generated and managed by executing the host function, and stores various data describing the game progress status. In this embodiment, player character state data 542, enemy character state data 543, enemy character arrangement information queue 544, flying object arrangement data 545, basic area setting data 550, element area setting data 552, arrangement area Setting data 554.

  The player character state data 542 includes data describing the states of the player characters PC1 to PC3, for example, current values of ability values such as HP (hit points), identification information of applied motion data, motion control data, and the like. Store.

The enemy character state data 543 stores data describing the state of the enemy character 7. For example, the current value of the ability value such as HP of the enemy character, identification information of motion data to be applied, motion control data, and the like are stored.
The enemy character arrangement information queue 544 stores the position and posture of the enemy character 7 in the CG space associated with the calculation time for a predetermined calculation count.
The flying object arrangement data 545 stores information describing the flying state such as the position, posture, speed, and acceleration of each launched flying object 6.

The basic area setting data 550 stores the latest definition data of the basic area 13.
The element area setting data 552 stores the latest definition data of the element area 14 obtained for each game participant, that is, for each player viewpoint position.
The arrangement area setting data 554 stores the latest information of the arrangement area 12.

  The virtual three-dimensional space data 560 stores arrangement information of various objects such as player characters PC1 to PC3, enemy character 7, flying object 6, and background objects arranged in the virtual three-dimensional space.

  The host function inheritance timing 562 stores timing information (for example, system time) when the machine that expresses the host function has changed. In other words, the timing at which the execution function of the host function is inherited by another machine is stored.

[Description of process flow]
Next, the flow of various processes in this embodiment will be described with reference to FIGS. A series of processing described here is realized by the game device 1500 reading and executing the system program 502, the game program 504, and various setting data. It is assumed that the AR marker 10 is arranged in advance on the table 9 and the like, and the player is in a posture to take the AR marker 10 with the CCD camera module 1516 while holding each game device 1500 in his / her hand.

  FIG. 17 is a flowchart for explaining the main processing flow in the present embodiment. The processing unit 200 of the game device 1500 first establishes a communication connection with another game device in an ad hoc mode (step S2). Thereafter, a peer-to-peer (P2P) local network is formed between the game devices.

Next, the processing unit 200 executes pre-game preparation processing (step S4).
FIG. 18 is a flowchart for explaining the flow of the pre-game preparation process in the present embodiment. In this process, the processing unit 200 first executes a host / guest selection process for displaying a selection screen for selecting the own machine as a host machine or a guest machine and causing the player to select one (step S6).

  If the host machine is selected (“host” in step S8), game participation machine registration data 520 is generated and the own machine is registered (step S10). That is, the first game machine ID 521 is issued and address information 522 in the local area network established previously is stored. The expression priority order 523 is set to the first order, and the marker recognition state 525 is initialized to information indicating the recognition state (“1” in the example of FIG. 14).

Then, the processing unit 200 of the host machine executes the player character selection process (step S12) and the game space initialization process (step S14).
The selection result of the player character is stored in the player character type 524 of the game participant registration data 520.
In the game space initialization process, with reference to the game space initial setting data 510 and the character initial setting data 511, the background space is arranged in the virtual three-dimensional space to arrange the game space, and the player character PC1 and the enemy character are arranged. 7 is arranged. As the characters are arranged in the game space, player character state data 542 and enemy character state data 543 are generated and initialized. When the game space initialization process is executed, the processing unit 200 of the host machine executes the participation process as the host machine for a limited time (step S16).

  On the other hand, when the guest is selected in the host / guest selection process (“Guest” in step S6), the processing unit 200 of the game device 1500 that is the guest machine executes the player character selection process. (Step S18), a game participation request signal is transmitted to the host machine, and a participation process as a guest machine is performed (Step S22).

  In the participation process, the processing unit 200 of the host machine sends and receives information to and from the guest machine that requested the game participation and registers it as a participating machine. That is, the newly assigned game machine ID 521 is set in the game participation machine registration data 520, the address information 522 of the guest machine is registered in association with this, and the expression priority 523 is set to the registration order of the guest machine +1 (therefore, Set the first guest machine's expression priority to 2nd). The player character type 524 is acquired from the guest machine, and the marker recognition state 525 is a value indicating the recognition state. When the predetermined reception time ends from the start of the participation process, game participation machine registration data 520 is distributed from the host machine to the guest machine. In the guest machine, the received game participation machine registration data 520 is stored in the storage unit 500.

When the participation process is completed, the process returns to the flowchart of FIG.
If the game is started (step S24), the processing unit 200 repeatedly executes steps S32 to S320 in a predetermined cycle until the game end condition is satisfied.

In one control cycle, the processing unit 200 first executes an automatic information calculation process (step S32).
FIG. 19 is a flowchart for explaining the flow of the device information calculation process. In this process, the processing unit 200 first captures a video image taken by the imaging unit 104 (step S60), and detects the AR marker 10 in the captured image (step S62).

  If the AR marker 10 can be detected (YES in step S64), the processing unit 200 recognizes the AR marker 10 and determines the type (step S66). That is, in the same manner as in the known marker system AR technique, a portion where the AR marker is reflected is detected, and the portion is compared with the design pattern registered in the AR marker registration data 512 (see FIG. 13). The AR marker 10 is recognized, and the marker direction, marker type, and marker category are determined.

  Next, the processing unit 200 measures and calculates the relative position (Xc, Yc, Zc), the relative posture (ωp, ωy, ωr), and the relative distance L with respect to the AR marker 10 (step S68). These calculation results are stored in the own device information queue 530.

  If the AR marker 10 is not detected (NO in step S64), the processing unit 200 does not calculate the relative position and the relative attitude, and does not calculate the relative position 533 and the relative position in the latest past information queue 530. Refer to distance 535.

If those values in the past are stored in the own device information queue 530 (YES in step S70), they are copied and directly stored in the own device information queue 530 as the current relative position and relative distance. Store (step S72). If those values in the latest past are not stored (NO in step S70), predetermined initial values are stored in the own device information queue 530 as the current relative position and relative distance (step S74).
Regarding the relative posture, regardless of whether or not those values in the latest past are stored in the own device information queue 530, the result measured based on the detection result of the posture detecting unit 108 is newly added to the own device information queue 530. (Step S76).

If the relative position and the relative posture for this time are determined, the processing unit 200 next calculates a weighting wn (n: storage order of data stored in the own device information queue 530) (step S78). The current position 537 is obtained by weighted average of the relative positions 533 stored in the own device information queue 530 using the weighted wn thus obtained (step S80).
Further, the current posture 538 is obtained by weighted average of the relative postures 534 stored in the own device information queue 530 (step S82). The weighting wn related to the weighted average is calculated by using the latest relative distance Lc of the own device for a predetermined function h (n, Lc). When the weighting wn is calculated, the own device information calculation process is terminated.

  Returning to the flowchart of FIG. 17, the processing unit 200 next executes host function expressing machine setting processing (step S <b> 110).

FIG. 20 is a flowchart for explaining the flow of host function developing machine setting processing in the present embodiment.
In the process, the processing unit 200 first performs a monitoring process of the recognition state of the AR marker 10 in each game device (steps S112 to S122).
Specifically, when the AR marker 10 cannot be recognized in the own device information calculation process (YES in step S112), the processing unit 200 recognizes the marker recognition state 525 of the own device in the game participant registration data 520 (FIG. 14) is changed to “recognition is not possible (LOST)” (step S114), and a predetermined recognition impossibility signal is transmitted to another device (step S116). If the AR marker 10 can be recognized (NO in step S112), the marker recognition state 525 of the own device is set to “In Sight” (step S118).

  For the unrecognizable signal from the other device, the processing unit 200 changes the marker recognition state 525 of the other device that is the transmission source of the unrecognizable signal to “unrecognizable (LOST)” (step S120). The other devices that have not received the unrecognizable signal are set to “In Sight” (step S122). Thus, the marker identification state in each game participant is updated.

Next, the processing unit 200 selects a game device that expresses a host function based on the monitoring result of the marker recognition state, that is, a “host function expressing machine” (step S140).
Specifically, referring to the marker recognition state 525 of the game machine registration data 520, the marker recognition state of all other machines whose expression priority is higher than the game apparatus is “recognizable (LOST)”. The game machine is selected as the machine that should express the host function.

  Then, the processing unit 200 refers to the host function expression flag 526 of the selected machine. If the flag is “0” (“0” in step S 142), the same flag of other game machines is set to “0” (step S 144), and the current system time is stored in the host function inheritance timing 562. Then, it is updated (step S146), and the host function developing device setting process is terminated. If the host function expression flag 526 of the selected machine is already “1” (“1” in step S142), the host function expression machine setting process is not performed without performing the flag operation or updating the host function inheritance timing 562. finish.

  Returning to the flowchart of FIG. 17, when the own device is set as the host function developing device (YES in step S150), the processing unit 200 determines that the own device should develop the host function and executes the host function. Processing is performed (step S152).

  FIG. 21 is a flowchart for explaining the flow of host function execution processing in the present embodiment. In this process, the processing unit 200 first obtains the current position 537, the current posture 538, and operation input data (in this embodiment, whether or not there is a shooting operation input) as the other machine information from all other machines (step S160). ). Then, basic area setting processing is executed (step S162).

  FIG. 22 is a flowchart for explaining the flow of the basic area setting process in the present embodiment. In this process, the processing unit 200 first obtains the relative distance Lc to the AR marker 10 for each game participating machine, that is, for each player's viewpoint, by obtaining or transmitting from another machine (step S170), and calculating or obtaining the relative distance. A basic length Lb is calculated based on Lc (step S172). In the present embodiment, the basic length Lb is obtained using a predetermined function f (Lc) using the minimum value among the relative distances Lc calculated for each game participant.

  Next, the processing unit 200 compares the calculated basic length Lb with a predetermined minimum value, and if it is below the minimum value (YES in step S174), replaces the basic length Lb with this minimum value (step S176). ). Then, the basic area model is read from the AR marker registration card 512 (see FIG. 13) corresponding to the recognized type / category of the AR marker (see step S66 in FIG. 19), and the basic length Lb set previously is used as the basic length. The basic area 13 assigned to the applicable dimension is set (step S178). Information (for example, functions describing vertex coordinates and contour lines) defining the obtained basic area 13 is stored in the play data 541 as basic area setting data 550. Then, the basic area setting process ends.

  Returning to the flowchart of FIG. 21, the processing unit 200 next executes an element area setting process for each game participant, and sets an element area 14 for each player viewpoint (step S180).

  FIG. 23 is a flowchart for explaining the flow of the element region setting process. In this process, the processing unit 200 first obtains the vertex positions of the four corners of the AR marker 10 shown in the captured image (step S190), and the position and roll of the game apparatus 1500, that is, the position and roll of the CCD camera module 1516 ( Assuming that the rotation around the optical axis is fixed, the right end of the four corners of the AR marker 10 is close to the right end of the imaging range of the CCD camera module 1516, and the lower end of the four corners of the AR marker 10 is the lower end of the imaging range. Is obtained, and the optical axis AL1 of the CCD camera module 1516 in the posture is obtained (step S192; see FIG. 6).

  Similarly, the processing unit 200 obtains the optical axis AL2 by obtaining the second posture in which the left ends and the upper ends are close to each other between the AR marker 10 and the imaging range (step S194), and the left ends are obtained between the AR marker 10 and the imaging range. The optical axis AL3 is obtained by obtaining a third posture in which the lower ends are close to each other (step S196), and the optical posture AL4 is obtained by obtaining a fourth posture in which the right ends and the upper ends are close to each other between the AR marker 10 and the imaging range ( Step S198).

  Next, the processing unit 200 calculates the optical axis angle of the CCD camera module 1516 relative to the design surface (marker plane) of the AR marker 10, that is, the depression angle θc, and selects / references the depth function setting data 514 corresponding to the category of the AR marker 10. Then, the depth Db based on the calculated depression angle θc is calculated (step S200).

  Then, the intersections of the clip plane separated from the camera viewpoint position of the CCD camera module 1516 by the depth Db in the current optical axis direction and the optical axes AL1 to AL4 previously obtained are obtained, and the obtained four intersection points and the camera viewpoint are obtained. A volume surrounded by the position is set as the element area 14 in the target game device 1500 (target player viewpoint) (step S202). Information (for example, functions describing vertex coordinates, contour lines, etc.) defining the obtained element area 14 is stored in the play data 541 as element area setting data 552. Then, the element area setting process ends.

  Returning to the flowchart of FIG. 21, the processing unit 200 next obtains the overlapping region of the basic region 13 and the element region 14 and sets it as the arrangement region 12 (step S220). Information that defines the obtained overlapping area (for example, functions that describe vertex coordinates and contour lines) is stored in the play data 541 as arrangement area setting data 554.

  Next, the processing unit 200 performs motion control so that the enemy character 7 exists within the latest arrangement area 12 (step S222).

The operation control here includes two operations of movement and attack.
With regard to the movement control, for example, if the enemy character 7 is in the latest arrangement area 12, a process for selecting a movement direction so as not to go out of the area, or a distance from the outline of the area is calculated to obtain a predetermined value. When a state satisfying the approach condition is detected, a process for temporarily reducing the next movement amount is executed to determine a new arrangement position. Also, if the enemy character 7 is outside the area, a process for selecting the movement direction so as to return to the area, a process for temporarily increasing the movement amount (movement speed) so as to return as quickly as possible, etc. Execute to determine a new placement position.

  Furthermore, the movement control is performed so that the enemy character 7 approaches the image position of the AR marker 10 with a higher probability as the image position of the AR marker 10 in the captured image (live view) by the imaging unit 104 is farther from the image center. To do. In other words, as the player viewpoint direction (the optical axis direction of the CCD camera module 1516 and the visual line direction of the virtual camera are the same) is further away from the AR marker 10, the operation control is performed so that the enemy character 7 is closer to the AR marker 10.

Specifically, for example, when the moving direction of the enemy character 7 is determined by lottery processing, the lottery probability is temporarily changed so that the moving direction toward the current AR marker 10 is selected with higher probability, This can be realized by, for example, re-drawing when the result does not follow the line-of-sight direction.
The newly determined position and posture of the enemy character 7 are stored in the enemy character arrangement information queue 544. Along with the movement control, the control data of the object related to the enemy character 7 in the virtual three-dimensional space data 560 is appropriately rewritten.
The attack is controlled by a predetermined algorithm (so-called AI control) so as to shoot at the appropriate player characters PC1 to PC3 at an appropriate timing.

  Here, the processing unit 200 compares the current system time with the system time stored in the host function inheritance timing 562 (that is, the control start timing of the host function), and determines a predetermined time from the control start timing of the host function. Determine if it has passed.

If it has not elapsed (NO in step S224), the processing unit 200 performs special control at the start of control for adjusting the movement amount and / or movement speed of the enemy character 7 to be reduced (step S226). ).
Specifically, for example, weighted average values of the positions and postures stored in the enemy character arrangement information queue 544 (weighting is lower as the past value) is obtained, and the latest values of the enemy character arrangement information queue 544 are weighted averaged. Update to position and orientation. Thereby, even if the position and posture of the enemy character 7 due to the replacement of the aircraft responsible for the host function are changed, the influence can be reduced. The apparent movement amount and movement speed of the enemy character 7 are reduced, and an unnatural movement such as an instantaneous increase in the movement amount and movement speed can be prevented.
If the predetermined time has not elapsed from the control start timing of the host function (YES in step S224), such special control at the start of control is skipped.

  Next, the processing unit 200 updates the arrangement positions of the player characters PC1 to PC3 (step S228). Accordingly, the control data of the objects related to the player characters PC1 to PC3 in the virtual three-dimensional space data 560 is appropriately rewritten.

  Next, the processing unit 200 manages the flying object 6 (step S240). For example, when an attacking action is performed by the enemy character 7 or when an attack operation input is detected in each game device, a new flying object 6 launched from the corresponding player character is placed in the virtual three-dimensional space. To do. Further, the arranged flying object 6 is moved. Accordingly, the flying object arrangement data 545 of the play data 541 is appropriately rewritten, and the control data of the object related to the flying object 6 in the virtual three-dimensional space data 560 is appropriately rewritten.

  Next, the processing unit 200 determines the hit of the attack based on the position after the flying object 6 has moved, the updated positions of the player characters PC1 to PC3, and the position of the enemy character 7, and the damage associated with the attack hit. Determination and damage reflection processing are performed (step S242). If it is determined that the flying object 6 has been hit, it goes without saying that the flying object 6 is erased from the virtual three-dimensional space. By the damage reflection process, the player character state data 542 and the enemy character state data 543 of the play data 541 are updated.

  Since the game in one control cycle has progressed through the processing by the host function execution processing so far, the processing unit 200 distributes the latest play data 541 to the other machine (step S244), and executes the host function execution processing. finish.

  Returning to the flowchart of FIG. 17, when it is not determined that the own device should exhibit the host function (NO in step S150), the processing unit 200 performs control to behave as a guest device. That is, the processing unit 200 of the guest machine acquires the current position 537, the current posture 538, and the operation input information from all other machines (Step S300), and acquires play data from the host machine (Step S302). Then, the arrangement position of the object in the virtual three-dimensional space is updated based on the acquired current position 537 and current posture 538 of the other device, the current position 537 and current posture 538 of the own device, and the acquired play data ( Step S304).

By the flow so far, the game progress control and the rearrangement of objects in the virtual three-dimensional space (CG space) according to the result are completed.
Therefore, the processing unit 200 next executes processing related to generation and display of the game screen. That is, the captured image captured by the imaging unit 104 is cropped to generate a background image (step S310), and the virtual camera CM is placed in the virtual three-dimensional space based on the current position 537 and the current posture 538 of the own device. Arrange and render the state of the virtual three-dimensional space photographed by the virtual camera (step S312). Then, the rendered CG is synthesized on the background image, and an information display body such as the sight 8 is synthesized on the center of the screen to complete the game screen (step S314), and this is displayed on the image display unit 360. It is displayed (step S316; see FIG. 3).

  Next, the processing unit 200 determines whether or not a game end condition is satisfied (step S320). If the game end condition is not satisfied (NO in step S320), the process proceeds to the next control cycle. If it is satisfied (YES in step S320), a game end process such as displaying an ending screen is executed (step S322), and the series of processes ends.

As described above, according to the present embodiment, the relative position for each player viewpoint with respect to the AR marker 10 (the game participant represented by the relative position of the viewpoint of the CCD camera module 1516 with respect to the reference point of the AR marker 10 in this embodiment). Relative position), the display position of the character in the augmented reality space can be appropriately controlled conveniently for multiplayer.
Specifically, even if the viewpoint position and the line-of-sight direction of each game device 1500 are changed so as to track the attention character (enemy character 7), the AR marker 10 does not fall outside the imaging range of the CCD camera module 1516. As described above, the arrangement of the attention character can be adjusted. That is, it is possible to improve the quality of a multiplayer game using an augmented reality space and avoid and reduce various inconveniences associated with character placement control.

  In the present embodiment, since the current position for actually controlling the virtual camera can be determined using the relative position for a given time (or given number of times) retroactively from the present, Camera shake can be suppressed. Furthermore, while the AR marker 10 cannot be recognized, the position coordinates are temporarily fixed to the values immediately before the marker lost, while the current position is continuously calculated for the posture. Thereby, the accuracy of coordinate alignment between the real space and the CG space is lower than when the AR marker 10 can be recognized, but it is possible to suppress the display misalignment of characters and the like that the game must be interrupted. That is, it is possible to increase resistance to the interruption of the game due to the lost AR marker.

  In addition, even if a game participant having a host function such as NPC control of the enemy character 7 or the like falls into a state where the AR marker 10 cannot be recognized, another game participant takes over the host function, It is possible to suppress a display position shift such as NPC caused by the lost of the game and continue the game.

In the present embodiment, the components can be changed, added, or omitted as appropriate.
For example, in relation to the switching of the host function developing machine, when the AR marker 10 cannot be recognized, a “recognition impossible signal” is transmitted, and the marker recognition state is managed based on the presence or absence of the recognition unrecognized signal. Alternatively, the “recognition signal” may be transmitted when the AR marker 10 is recognized. While the “recognition signal” is received, the marker recognition state 525 of the transmission source is set to “In Sight (recognizing)”, and the marker recognition state 525 of the aircraft to which the “recognition signal” is not transmitted is set to “LOST ( It is better to set to "Unrecognizable)".

  Further, the expression priority order can be changed as appropriate during the game play from the initial value set in the pre-game preparation process. For example, as shown in FIG. 24, in the host function execution process, the processing unit 200 acquires the relative distance Lc with respect to the latest AR marker 10 from all the other machines (step S250), and the own unit is included in the acquired relative distance Lc. Sorting in order of increasing distance including the latest relative distance Lc, the expression priority order 523 is reset in the sorted order and distributed (step S254). For example, when the processing unit 200 of the game apparatus 1500 that does not express the host function receives a new expression priority 523 following step S304 in FIG. It is assumed that the expression priority order 523 of the registered game participant registration data 520 is updated (see FIG. 17).

  Since the game device 1500 that is far from the AR marker 10 can capture a wider range, it is expected that the AR marker 10 is less likely to be lost. Therefore, by resetting the expression priority 523 in order of increasing relative distance Lc, it is possible to reduce the switching frequency of the aircraft that exhibits the host function.

[Second Embodiment]
Next, a second embodiment to which the present invention is applied will be described. The present embodiment is basically realized in the same manner as the first embodiment, except that the calculation timing of the current position 537 can be adjusted. In addition, the same code | symbol is provided about the component similar to 1st Embodiment, description is abbreviate | omitted, and it shall mainly describe the difference with 1st Embodiment.

  FIG. 25 is a partial functional block diagram illustrating a functional configuration example of the game calculation unit 202 in the present embodiment. The own device information calculation unit 206 of the present embodiment includes a change amount calculation unit 213F and a calculation interval setting unit 214F.

  The change amount calculation unit 213F uses the position information calculated by the posture detection unit 108, the acceleration detection unit 110, and the positioning control unit 280, and the like, that is, the position and posture of the player viewpoint (that is, the game device 1500 and the CCD camera module 1516). The amount of change per unit time of the position and orientation of the viewpoint position with respect to the AR marker 10 is calculated. The amount of change in both position and orientation may be obtained, or either one may be obtained. Note that the amount of change may be a feature point extracted from an image captured by the imaging unit 104, and the amount of movement of the extracted feature point per unit time may be used.

  The calculation interval setting unit 214F can newly set / change the latest calculation information of the own device position information and the own device attitude using the calculation result of the change amount calculation unit 213F. The setting value of the calculation interval is appropriately stored in the storage unit 500.

  FIG. 26 is a flowchart for explaining the main processing flow in the present embodiment. Although it is basically the same as in the first embodiment, the processing unit 200 calculates a change amount per predetermined time of any posture change / acceleration / positioning result at the beginning of the control cycle (step S26F).

  Next, a calculation interval is set according to the calculated change amount and the relative distance Lc calculated at the previous calculation timing (step S27F). Specifically, when it is determined that the calculated change amount has rapidly changed beyond a predetermined change amount reference value, the calculation interval is made shorter than the standard value. The calculation interval is also made shorter than the standard value when it is determined that the latest past relative distance Lc exceeds a predetermined distance reference value.

  If the calculation interval set in step S27F has elapsed since the previous calculation timing (YES in step S28F), the device information calculation process is executed (step S32).

[Modification]
As described above, the embodiments to which the present invention is applied have been described. However, the embodiments of the present invention are not limited to these, and additions, omissions, and changes of components can be appropriately made.

  For example, if the computer used as the game apparatus 1500 in the first embodiment is equipped with an image sensor serving as the player's viewpoint, a gyroscope and an acceleration sensor for detecting a change in posture, and can execute a game program, the portable game apparatus In addition, for example, an electronic device such as a smartphone, a personal computer, or a mobile phone may be used as the “game device”.

  Of course, the genre of the game is not limited to the shooting game, and can be set as appropriate.

  In the above embodiment, the relative positional relationship between the virtual camera CM and the player character PC in the virtual three-dimensional space is described as being substantially fixed, but the present invention is not limited to this. For example, as shown in FIG. 27, the virtual cameras CM1 to CM3 are arranged based on the relative positions of the corresponding game apparatuses 1500a to 1500c with respect to the AR marker 10 (that is, relative positions of the player viewpoint), but the player character PC1. The PC 3 is separately controlled by an operation input using the direction input key 1502 of the game apparatus 1500, and as a result, the relative positional relationship between the virtual camera and the player character may be variable. In this case, the player changes the position of the virtual camera by moving the game apparatus 1500, while controlling the player character with a separate operation input. Accordingly, it is possible to play the game by freely operating the player character while imagining the game space like a miniature garden and watching the state in the game space from a desired angle through the game device.

  In addition, the image display unit 360 of the game apparatus 1500 is configured by a display apparatus that can perform autostereoscopic viewing (for example, a parallax barrier system or a lenticular system), and according to the relative position / relative attitude of the game apparatus 1500 with respect to the AR marker 10, Display control of a three-dimensional image may be performed. In this case, the live view image captured by the CCD camera module 1516 is set as the farthest view, and the characters and the like (objects) arranged on the coordinate system of the game space (virtual three-dimensional space) are three-dimensionally controlled. It is good. In this way, for example, when the game device is moved so as to wrap around the AR marker 10, an object (an enemy character in the case of the above embodiment) that appears on the basis of the AR marker 10 wraps around the object. It is possible to obtain a sense of depth by viewing.

DESCRIPTION OF SYMBOLS 10 ... AR marker 12 ... Arrangement area 13 ... Basic area 14 ... Element area 104 ... Imaging part 108 ... Attitude detection part 110 ... Acceleration detection part 200 ... Processing part 202 ... Game operation part 204 ... Game participation management control part 205 ... Expression priority Order setting unit 206 ... Self-machine information calculation unit 206B ... Position relation calculation unit 207 ... Relative position calculation unit 208 ... Position change suppression control unit 209 ... Current position determination unit 210 ... Relative posture calculation unit 211 ... Current posture determination unit 212 ... Weighting Setting unit 220 ... Own device information transmission control unit 221 ... Other device information reception control unit 222 ... Recognition state determination unit 223 ... Unrecognizable signal transmission control unit 224 ... Host function expression determination unit 230 ... Character display control unit 231 ... Area setting unit 232 ... Basic area setting section 233 ... Element area setting section 234 ... Arrangement area setting section 235 ... NPC control section 23 ... control start special control unit 237 ... movement direction control unit 249 ... host function execution control unit 500 ... storage unit 504 ... game chromatogram 512 ... AR marker registration data 514 ... depth function setting data 520 ... game participant registration data 523 ... Expression priority 524 ... Player character type 525 ... Marker recognition state 526 ... Host function expression flag 530 ... Self-machine information queue 533 ... Relative position 534 ... Relative attitude 535 ... Relative distance 537 ... Current position 538 ... Current attitude 541 ... Play data 544 ... enemy character arrangement information queue 550 ... basic area setting data 552 ... element area setting data 554 ... arrangement area setting data 560 ... virtual three-dimensional space data 562 ... host function inheritance timing 1500 ... game device 1506 ... touch panel 1516 ... CCD Camera module 1550 ... control board 1556 ... triaxial gyro 1557 ... 3-axis acceleration sensor CM, CM1 to CM3 ... virtual camera Db ... depth PL, PL1 to PL3 ... player PL
PC, PC1 to PC2 ... Player characters AL1 to AL4 ... Optical axis

Claims (10)

A program for executing a game by combining and displaying an object on a live view photographed by the photographing means so that a marker indicating a reference direction is reflected on a computer including a photographing means and a display means,
Position calculating means for periodically calculating the position of the computer in a virtual three-dimensional space with reference to the position of the marker in the live view;
Current position determination means for determining the current position of the computer by using a plurality of calculated positions calculated by the position calculation means for a given time or a given number of times retroactively from the present,
Object placement means for placing an object at a given position in the virtual three-dimensional space;
Combining means for combining the live view with an image of the object viewed from the current position determined by the current position determining means;
A program for causing the computer to function as The current position determining means is means for determining the current position of the computer by weighted averaging the plurality of calculated positions.
The program according to claim 1. The current position determining means includes first weight changing means for changing a weight of each of the calculation positions used for the weighted average according to a calculation time by the position calculation means.
The program according to claim 2. The current position determining means includes second weight changing means for changing the weight of each of the calculation positions used for the weighted average according to the distance between the calculation position and the marker.
The program according to claim 2 or 3. A change amount measuring means for measuring a change amount per unit time of the position and / or posture of the computer;
A calculation interval changing unit that changes a calculation interval of the position calculating unit using a measurement result of the change amount measuring unit;
The program as described in any one of Claims 1-4 for making the said computer further function as. The calculation interval changing unit further changes a change amount of the calculation interval according to a distance between the calculation position calculated by the position calculating unit and the marker.
The program according to claim 5. Unrecognizable state detection means for detecting that the marker cannot be recognized from the live view.
An unrecognizable direction control means for controlling the display direction of the object in accordance with a change in the posture of the computer while the unrecognizable state is detected,
The program as described in any one of Claims 1-6 for making the said computer further function as. Unrecognizable state detection means for detecting that the marker cannot be recognized from the live view.
A recognition return state detection means for detecting a recognition return state in which the marker can be recognized again from the unrecognizable state;
Object position change suppression control means for suppressing a change in the display position of the object in the live view while the unrecognizable state is detected;
The program as described in any one of Claims 1-6 for making the said computer further function as.   The computer-readable information storage medium which memorize | stored the program as described in any one of Claims 1-8. An electronic device that includes a photographing unit and a display unit, and displays an object on a live view photographed by the photographing unit so that a marker representing a reference direction is captured,
Position calculating means for periodically calculating the position of the computer in a virtual three-dimensional space with reference to the position of the marker in the live view;
Current position determination means for determining the current position of the computer using a plurality of calculated positions calculated by the position calculation means for a given time or a given number of times retroactively from the present time;
Object placement means for placing an object at a given position in the virtual three-dimensional space;
Combining means for combining the live view with an image of the object viewed from the current position determined by the current position determining means;
With electronic equipment.

Images