JP2002233665A - Game system, game method, and variable recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a game system, a game method, and a variable recording medium which can enable an operator to directly input to an object image on a display screen from an arbitrary inputting position and simultaneously give position direction information of the inputting operator to the object image to provide a diversity of game development. SOLUTION: This game system is equipped with an imaging means 101 to image the target including at least four feature points on a plane and a posture calculating means 521 to calculate a posture parameter of an imaging plane facing the plane based on taken image data obtained by the imaging means. The target is varied based on posture parameter obtained by the posture calculating means.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a game device for changing a display image in accordance with an operation position of an input operator, a game method, and a readable storage medium storing a program for executing a game.

[0002]

2. Description of the Related Art Conventionally, home games and arcade games using a display device such as a television or a CRT of a personal computer have been known. In particular, a shooting game is a game in which a gun-type controller is used to aim directly at the target object image on the display screen, shoot, and compete for a hit or not, and operate the controller directly at the display screen. This is a typical game.

A general method of detecting the position of a shooting game device is to detect light of an electronic scanning line for scanning on a CRT display screen by a light receiving element provided in a gun-type controller. For example, Japanese Patent Application Laid-Open No. Sho 60-17907
No. 9, JP-A-4-51987, JP-A-5-3
No. 22487 is known. Further, as a conventional example of a shooting game apparatus in which a gun-type controller is aimed at a projected image projected on a screen, Japanese Patent Application Laid-Open Nos. 62-32987 and 5-322487 are known.

[0004] Recently, with the speeding up of image processing of game machines, it has become possible to enjoy a three-dimensional shooting game made of CG images full of reality. For example, JP-A-6-213595, JP-A-11-86
No. 038, international application WO97 / 21194, and the like.

[0005] This game is a virtual game on the display.
The target object appearing in the dimensional space is shot by the gun type controller to compete for points. Objects placed in the virtual three-dimensional space
It is a three-dimensional object composed of a plurality of polygon data, and is displayed after being coordinate-transformed into a visual field coordinate system determined by a virtual camera viewpoint (also referred to as an image viewpoint).
JP-A-6-213595 is a representative example of a game device that shoots at an image projected in a three-dimensional space.
Will be described. This is a game in which a player takes a spacecraft on which a gun device is attached in accordance with a predetermined game story and flies while aiming at and shooting at a target in a space space that is regarded as a three-dimensional game space. The viewpoint of the spacecraft on which the player is riding is stored in advance as information for each frame, and the scenery of outer space is projected according to the viewpoint information.

The image viewpoint of a three-dimensional target object is
Which direction is occupied and what spatial coordinates are occupied are arbitrarily set, and the image viewed from this virtual viewpoint is displayed by performing a perspective projection operation on a display screen which is a two-dimensional coordinate plane.

[0007]

However, a conventional shooting game apparatus, for example, Japanese Patent Laid-Open No. 4-519 is disclosed.
The coordinate detection method disclosed in Japanese Patent Laid-Open No. 87 is a light detection method based on electron beam scanning of a CRT screen, and thus has a problem that it cannot be used for a liquid crystal display screen or a wall projection display surface.

Further, the position calculating method disclosed in Japanese Patent Application Laid-Open No. 8-71252 can be applied only when the input operator performs an input operation at a substantially normal position on the screen surface, and the calculated inclination and position are qualitative. It's just a mere amount, and you can't enjoy a game full of reality.

JP-A-6-213595, JP-A-7-213
Although operating means such as that disclosed in Japanese Patent Application Laid-Open No. 116343 can change the viewpoint of a three-dimensional object space outside the display screen, the operating viewpoint coordinates are stored and determined in advance. In the example of the shooting game, the fulcrum to which the gun-type controller is attached is used as the imaging viewpoint for the gaze direction and inclination from the operation means to the object, and a sensor is provided for detecting the rotation angle around the fulcrum. As described above, the input operation space of the player is limited, the degree of freedom of operation is low, and the movement of the display screen and the viewpoint of the target object becomes uniform. As a result, the game development was monotonous and lacked fun and power. Further, there is a problem that such a conventional device becomes large and cannot be easily used as a home game.

An object of the present invention is to enable an operator to directly perform an input operation on an object image on a display screen from an arbitrary input operation position, and at the same time, to give the input operator position and orientation information to the object image. It is an object of the present invention to provide a readable storage medium storing a game device, a game method, and a program for executing a game, which can perform various game developments.

[0011]

According to a first aspect of the present invention, there is provided a game apparatus for imaging a target including at least four feature points on a plane. And posture calculating means for calculating a posture parameter of the imaging plane with respect to the plane based on the captured image data obtained by the imaging means, wherein the target is changed based on the posture parameter obtained by the posture calculating means. It is characterized by making it.

According to a seventh aspect of the present invention, there is provided a game apparatus for displaying and playing a target object image together with at least four feature points on a display screen, and capturing the object image including the feature points. Imaging means, attitude calculation means for calculating an attitude parameter of the imaging surface with respect to the display screen based on image data on the imaging surface of the imaging means, and a display screen based on the attitude parameters obtained by the attitude calculation means. The object image is changed.

Further, according to a twelfth aspect of the present invention, there is provided a game device for displaying and playing a target object image including a plurality of feature points on a display screen, wherein at least four target object images displayed on the display screen are displayed. Image capturing means for capturing an image including a plurality of feature points; position calculating means for calculating a detected position based on the feature point coordinates on the captured image data from the image capturing means; and image capturing means calculated by the position calculating means. An object part judging means for comparing and judging a calculation result of the detected position with a predetermined part object coordinate, and a target object image on a display means is changed based on a result of the part judging means.

According to a nineteenth aspect of the present invention, there is provided a game method for executing a game for changing a target based on the posture information of an operator, wherein a target including at least four feature points on a plane is imaged. An imaging step, and an attitude calculation step of calculating an attitude parameter of the imaging surface with respect to the plane based on the image data on the imaging surface obtained from the imaging step, based on the attitude parameter obtained by the attitude calculation step To change the target.

Further, in the storage medium according to the present invention, a feature point extracting step of extracting a feature point based on image data of a target including at least four feature points located on a plane. And calculating the attitude of the imaging surface with respect to the predetermined plane based on the coordinates of the feature points extracted in the feature point extraction step, and changing the target based on the attitude parameters calculated in the attitude calculation step. Is stored in the program.

[0016]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating an overall configuration of a game device according to an embodiment of the present invention.

The game apparatus includes an input operation means 100, a game apparatus main body 120 for executing image processing and coordinate calculation according to an output signal from the input operation means, and a display means 110 for displaying the processing results. [First Embodiment] FIG. 2 is a conceptual diagram illustrating a game device according to the present embodiment.

Using the gun-type controller 100 as an input operation means, the operator plays a shooting game toward a specific portion (also referred to as a detected position) Ps of the target object image displayed on the screen.
An image 111 is projected on the screen by the projector 130. Reference numeral 100 denotes an XYZ coordinate system (referred to as an image coordinate system), and an origin Om of the XYZ coordinate system.
There is an imaging plane on the XY plane. A screen plane 110 having an X ^* -Y ^* coordinate system (referred to as a screen coordinate system) is placed in a three-dimensional space of the XYZ coordinate system.

First, each configuration of the game device according to the present embodiment will be described. Input operation means 100
Is a CCD camera 101 as an imaging means, a sighting means 10
2. It has a distance measuring means 103 and various control buttons 14 for operating the object. FIG. 3 is a schematic configuration diagram of a gun-type controller as an input operation means.

The image pickup means 101 uses a camera having a CCD image pickup surface for detecting a position to be detected on the display screen.
The distance measuring means 103 measures the distance from the display screen to the imaging surface provided on the imaging means. The distance measurement is performed by a known technique. For example, if the CCD camera lens is provided with an auto-focusing mechanism, the focus amount can be used. Further, a laser distance measuring device may be provided.

The aiming means 102 is used by an operator to determine a position to be detected on the display screen, and is provided with, for example, a finder having a crosshair, a laser with visibility, and the like. 4 and 5 are arrangement diagrams of the optical system of the input operation means of FIG. 3, and FIG. 4 shows a viewfinder 1 as an aiming device.
FIG. 5 shows the visibility laser 120.
This is an example using B. In each of the figures, the aiming position is substantially coincident with the optical axis (center of the imaging surface) of the CCD camera lens. In this embodiment, the aiming position and the optical axis of the lens are matched, but it is not always necessary to match. A detailed description of FIGS. 4 and 5 is omitted.

When the operator sets the aiming position to a desired position to be detected on the display screen and turns on the switch of the imaging means 101 provided in the input operation means, the captured image data is captured. Is used to detect the coordinates of the detected position. That is, the operator can directly control the object image projected onto the display screen.

Further, the input operation means 100 is provided with various object operation control buttons 14 and 15 for moving an object image required for the progress of the game up and down, left and right, and jumping. Returning to FIG. 1, the input / output interface 3 has a frame memory capable of A / D converting an output signal from the input operation means and temporarily storing the signal. Further, a control signal for operating various objects is output to the game apparatus main body 120 from the input operation means. In addition, a voice signal accompanying a result of the image processing, a control signal for giving a vibration to a hand holding the operation means, and the like are exchanged via the input / output interface.

The CPU 4 has a ROM and stores an operation system (OS) and various control programs. The external storage medium 20 stores various programs for games, such as a CD-ROM and a DV.
D and the like. The storage medium 20 stores a method for executing the game according to the present embodiment, and is readable by the game apparatus main body 120, a computer, and the like.

The feature point extracting means 51 includes a difference processing means 511 for obtaining a difference image based on two picked-up images having different luminances or colors in order to extract a mark image characterizing a rectangular shape. A series of processes for extracting a mark such as the means 512 are performed.

The feature point specifying means 513 extracts markers from the obtained two pieces of captured image data, and specifies the barycentric coordinates of at least four marks on the image capturing surface (image coordinate system). If there are four or more captured mark images, the specification of the four marks used for the next posture calculation is also performed.

The position calculating means 52 includes a plane attitude calculating means 5
21 and coordinate calculating means 522 for calculating the detected position, and calculates the coordinates of the detected position on the visual field screen based on the image data obtained by the input operation means.
The attitude calculation means 521 is based on the identified four marker coordinate positions, and the position and attitude of the screen plane in the XYZ coordinate system with the origin being the imaging viewpoint of the imaging surface of the input operation means, and the rotation of the imaging plane. Calculate parameters β, γ, and の, such as angle, elevation, and horizontal angle.

The detected position coordinate calculation means 522 uses one of the posture parameters γ or ψ obtained as a result of the plane posture calculation means to specify a specific portion (detected position) of the target object in the target display screen. Is calculated.
The coordinate calculation processing of the posture parameter of the screen plane and the detected position will be described later in detail.

The input operation position specifying means 7 calculates the rotation α, the elevation angle γ, and the horizontal angle と し て as the posture parameters of the display plane with respect to the imaging viewpoint calculated by the screen plane posture calculating means 521 and the detected position coordinate calculating means 62. Based on the input coordinate values Ps of the detected position on the screen and the distance L from the detected position on the screen to the input operation position calculated by the distance measuring means 102 provided on the input means. , The position of the input operation means with respect to the screen is calculated and specified.

The target object region judging means 8 specifies the target object from among a plurality of various objects based on the result of the position calculation process, and judges which position of the target object is located. After the coordinate positions of various objects in the game space are determined in the image combining means 9, the target object image and the background object image are combined and displayed on the display means.

The image synthesizing means 9 stores the object operation mode image selected from the object image storage means 91 and the object operation mode image corresponding to the specific part of the hit target object in accordance with the input operation parameter. And a drawing unit 93 that pastes background data on the converted screen coordinate system data and outputs the data to a frame buffer. .

Here, the image viewpoint is also referred to as a virtual camera viewpoint, and is a virtual viewpoint that determines the viewing direction of an image displayed on a display screen when drawing computer graphics. The setting of this viewpoint is determined by the position optical axis direction (viewing direction), the angle of view, the rotation around the optical axis, and the like.

The object image storage means 92 stores object operation mode data. The object operation mode data is roughly stored in advance in some visual axis directions with respect to the screen. For example, the operation modes are classified into nine according to the elevation angle γ with respect to the screen size and the ψ setting area of the horizontal angle.

F1 (γ2, ψ2), F2 (γ2, ψ1), F3
(Γ2, ψ3) F4 (γ1, ψ2), F5 (γ1, ψ1), F6
(Γ1, ψ3), F7 (γ3, ψ1), F8 (γ3, ψ2), F
9 (γ3, ψ3) Here, the values of γ1, γ2, γ3, ψ2, ψ1, ψ2, and ψ3 are assumed to be in the following range.

Elevation angle: γ2 <−5 degrees, −5 degrees ≦ γ1 ≦ 5 degrees, 5 degrees <γ3 Horizontal angle: ψ2 <−20 degrees, −20 degrees ≦ ψ1 ≦ 20 degrees, 20
Degree <$ 3 The drawing means 9 performs a process of setting objects such as a target object and a background object in the object space. For example, a drawing process is performed in which the target object appears or moves within the visual field of the input operator as the game progresses.

The screen such as the target object and the background object temporarily stored in the frame memory 94 is combined with the scroll screen, generated as final frame image data, and displayed on the screen 110 by the projector 130. Next, the operation of the game device according to the present embodiment will be described based on an example applied to a specific shooting game.

FIG. 6 is a flowchart of the basic operation of the shooting game according to the embodiment of the present invention. Step S101 is an operation of aiming at a specific portion of the target object image on the screen by the gun-type controller as an input operation means and triggering. This is a shutter-on operation of the imaging unit by turning on the trigger switch. S102 is an operation process for capturing an image captured by the imaging unit when the switch is turned on. After being turned on, the process proceeds to step S103, and if not turned on, the shooting operation returns to the initial state.

Step S103 is a process of extracting at least four mark images, which are characteristic points on the display screen, based on the captured image data, and specifying the coordinates of the characteristic points on the captured image. The four marker images provided at predetermined positions displayed on the screen are captured. In order to perform the difference image processing, marker images of two frames with different display colors displayed on the screen are taken in time series.

As another method of this embodiment, a difference image of two images having different overall luminances may be obtained. In this case, four corners or four sides of all the display images in the display image are set as characteristic points characterizing the rectangular shape, and the game device main body is provided with a luminance changing unit so that two types of display images having different luminances are supplied on the screen. I do. For example,
In synchronization with the shutter of the imaging means, when the shutter is turned on once (when the trigger signal is turned on), images are taken twice at a predetermined time interval. One of the two shots captures the original image projected from the computer, and in the second captured image, synchronizes the display image whose luminance is changed by about ± 40% with respect to the luminance of the original image with the imaging shutter. Image. The difference image between the image captured at the first time and the image captured at the second time is obtained.

In step S104, the posture of the screen plane with respect to the imaging plane is calculated based on the coordinates of the plurality of feature points specified in the image coordinate system in step S103, that is, the values of the marker coordinates. The three attitude parameters calculated are ψ around the X axis, γ around the Y axis, and α or β around the Z axis. In the present embodiment, X on the imaging surface
-Expressed using a YZ coordinate system.

In step S105, coordinates Ps of the detected position on the screen are calculated based on the posture parameters α, ψ, and γ obtained in step S104. In addition,
A detailed description of the processing operation in steps S104 to S105 will be described later.

Step S106 is a process for judging in which part of the target object the detected position has been hit in step S105. If it is determined that there is a hit, the process proceeds to the next step, and if it is not hit, the process proceeds according to the game story. Step S107 is a process of selecting object operation mode data of a specific portion of the hit target object or a target object associated with the target object from object operation storage means stored in advance. When the object stored in advance is a three-dimensional object composed of a plurality of polygon data, the viewpoint of the virtual camera of the object may be switched to an arbitrary position according to the position and posture of the operator. it can.

The next step S108 is a step of obtaining distance data L from the screen from the distance measuring means to the imaging surface of the input operation means. In step S109, the direction and position of the operator with respect to the screen surface are calculated based on the posture parameters and the distance data.

More specifically, the posture information is based on the image coordinate plane when the camera viewpoint (imaging viewpoint) at the operation position is set as the origin, and the visual axis direction (lens optical axis) of the camera is set as the Z axis. , Three angles of rotation, α around the Y axis, and γ around the X axis. The distance information is the distance from the detected position on the screen to the imaging surface held by the operator. Although the position of the operator can be accurately determined from the posture information and the distance information, the visual axis direction can be specified by using only the posture information, and thus the processing in step S108 may be omitted.

The position of the operator is determined only when the operator performs the capturing operation of the captured image by aligning the target object with the collimating direction.
The optical axis direction of the imaging surface at the operation position of the input operation means, that is, the visual axis direction can be specified from the calculated posture parameters. This visual axis direction is a target direction 101 (shown by a broken line) in FIG.

Step S110 is a step of displaying the target object on the screen based on the result of the processing. For example, the object operation mode selected in step S107 is selected according to the input operation position, and the image viewpoint G0 of the object in the object operation mode is made to correspond to the imaging viewpoint (operation viewpoint) P1 and displayed by perspective projection calculation processing. be able to. In addition, if distance information can be obtained, the operator can use the information to display the object image on the display screen while increasing the perspective in accordance with the play. Can be varied and displayed to increase the game effect. , FIG. 7 and FIG.
FIG. 4 is a diagram illustrating a relationship between an image viewpoint G0 and an operation position (imaging viewpoint) P1 when a shot is taken from the operation position P1 to the detected position Ps with respect to the dimensional object image. 7 and 8 are plan views of the target object in a virtual three-dimensional space. The figure below shows the image displayed on the screen viewed from the normal direction. FIG. 7 is a diagram in which an object image is displayed on the screen plane irrespective of the operator. FIG. 8 shows a case where the image viewpoint G0 of the three-dimensional object image is matched with the position of the operation position P1. This figure shows a perspective projection display of an object image in a virtual three-dimensional space with the imaging viewpoint as a perspective viewpoint.

In this case, the target object placed in the virtual three-dimensional space is a static image for the sake of simplicity, but the target object is generally a dynamic image that operates after a shot operation. Through the above processing, a game effect that is full of reality that has not been achieved in the past can be obtained.

Next, the operation of the display plane attitude calculation processing in step S104 and the coordinate calculation processing of the target object specific portion to be detected in step S105 will be described. After the four marker images in the display image are extracted by the feature point extracting means 51 and the coordinate position of each marker is specified, the posture parameter of the imaging surface provided in the input operation means for the screen is determined based on the coordinate positions. , And
The coordinates of the detected position are calculated.

FIG. 9 is a flowchart for calculating the detected position from the coordinate values of the extracted four feature points. (A1) Posture Calculation Processing FIG. 10 shows a captured image q in which the input operator captures a display image from an arbitrary position with the imaging unit facing the screen.
At this time, the captured display image is a captured image q obtained by capturing the detected position Ps, which is a coordinate position on a plane, in accordance with a reference position (origin Om of the imaging surface) set on the imaging surface.

In step S201, the specified q1,
Based on the coordinates of q2, q3, q4, straight lines I1. I2, I
3. Calculate I4. In step S203, vanishing points T0 and S0 of the captured image data are obtained using these linear equations.

When a rectangular plane is picked up, a picked-up image always has a vanishing point. The vanishing point is a point where the parallel group converges. For example, a line on the imaging surface corresponding to Q1Q2 (g1)
q1q2 And q3q4 and q1q corresponding to Q3Q4 (g2)
4, again the right side Q1Q4 If and q2q3 are completely parallel, the vanishing point will be at infinity. When present at infinity, no perspective effect appears even if perspective projection is performed in that direction. That is, X in the XY image coordinate system (XY coordinate system)
When the vanishing point exists at infinity in the axial direction, the vanishing axis is the X axis itself.

Processing image data is often performed because the shape of an object placed in a three-dimensional space is known. In the present embodiment, four corners (corresponding to geometric feature points) on a screen having two sets of parallels in the object coordinate system.
Therefore, one vanishing point exists on each of the X-axis side and the Y-axis side on the captured image plane.

FIG. 10 shows the positions of vanishing points on the image data when an image is taken at an arbitrary position. The vanishing point on the X-axis side is S0, and the vanishing point on the Y-axis side is T0. The intersection of the extended straight lines of q1q2 and q3q4 is the position of the vanishing point. When it is determined that one vanishing point on the X-axis side or the Y-axis side is at infinity, when it is determined that one vanishing point exists on the X-axis or the Y-axis, the vanishing line is the X-axis or the This is the Y axis itself.

Further, at step S202, vanishing points S0,
After obtaining T0, the origin O of the image coordinate system which is the detected position is obtained.
linear vanishing axis S1S2, T1T2 connecting m and these vanishing points
Is performed. In step S203, vanishing point S
Straight lines S, T connecting 0, T0 and the imaging data center Om are
Point T that intersects the straight lines q1q2, q3q4, and q2q3, q1q4
1 ( _Xt1 , _Yt1 ), T2 ( _Xt2 , _Xt2 ), S1 ( _Xs1 , _Ys1 ),
S1 ( _Xs2 , _Ys2 ) is obtained (the intersection of these is called a vanishing feature point). T1T2 and S1S2 on the captured image will be referred to as disappearance axes. These disappearance axes are straight lines orthogonal to each other on the screen with respect to the detected position Ps, and are reference axes for calculating the detected position. The disappearing straight lines correspond to the straight lines S1S2 and T1T2 on the screen in FIG. Next, the process proceeds to step S204.

In step S204, the image coordinate system XY
The coordinate system is rotated by an angle α around Om around the vanishing straight line S on the X-axis side to make it coincident with the X-axis, and a process of setting the coordinate system to the X′-Y ′ coordinate system is performed. At this time, a process may be performed in which the vanishing straight line T on the Y-axis side is rotated by an angle β around the point Om to be coincident with the Y-axis, and the X-axis coordinate system is used. Either one is sufficient in the analysis used in the present embodiment.

FIG. 11 shows that the image coordinate system XY coordinate system is rotated by α degrees or β degrees to obtain an X′-Y ′ coordinate system, X ″
It is a figure explaining coordinate conversion to each -Y '' coordinate system. These rotation input operations correspond to rotation about the Z axis in the three-dimensional space, and are one parameter representing the screen-shaped posture position in the three-dimensional space.

For example, a straight line Q 1 Q 2 (g
1), Q3Q4 (g2) has a positional relationship parallel to the X-axis by matching the vanishing line S on the X-axis. FIG.
Shows the positional relationship between the attitude of an XYZ coordinate system (called an image coordinate system) on an imaging surface in a three-dimensional space and an X * -Y * coordinate system (called a plane coordinate system) on a predetermined plane. It is. An optical axis (optical axis of the imaging lens) extending vertically from the center of the image coordinate system is defined as a Z axis. The viewpoint O on the Z axis is located at a focal distance f from the origin Om of the image coordinate system. An angle ψ around the X axis, an angle γ around the Y axis, and an angle α or β around the Z axis in the XYZ coordinate system. Regarding the rotation around these angles, clockwise is defined as positive.

In the next step S205, the attitude of the imaging surface with respect to the screen plane is calculated. In this step, based on the position coordinates in the X′-Y ′ coordinate system of the obtained captured image on the imaging surface after the XY coordinate conversion, feature points q 1 and q 2 in the image coordinate system X′-Y ′ system , Q3, q4 are associated with each point of the screens Q1, Q2, Q3, Q4 in the object coordinate system. This association is performed by placing the screen and the imaging plane in a three-dimensional space having an image coordinate system (XYZ coordinate system) and performing a perspective projection conversion process with the focal length f of the imaging plane as the origin. Done.

FIG. 13 is a perspective view for explaining the position and orientation of the screen placed in the three-dimensional space. In the figure, a quarter of the rectangle is shown, and the coordinate points corresponding to the position coordinates Q1 (X ^* ₁ , Y ^* ₁ ) and Q2 (X ^* ₂ , Y ^* ₂ ) on the screen on the imaging surface are q1 ( X ₁ , Y ₁ ) and q ₂ (X ₂ , Y ₂ ) are shown. Each feature point to be analyzed is T1, T2 and S2.
The position coordinates of the point are shown.

[0060] _{^{Q3 (X * 3, Y *}} 3), Q4 (X * 4, Y * 4) coordinate point q3 corresponding to (X _3, Y _3), omitted for q4 (X _4, Y ₄₎ It is. Further, the instructed detected position corresponds to the origin Om (0,0, f) in the figure. The perspective point O (0,0,0) is the origin of the three-dimensional space XYZ coordinate system, and f is the focal length. Here, the two-dimensional screen has an ideal shape, the length of the upper side of the screen parallel to the horizontal axis is g1, the length of the lower side is g2, and the length of the right side of the screen parallel to the vertical axis is h1,
The length of the left side is h2. The positional relationship between the screen and the imaging surface is an angle + ψ around the X axis and an angle + γ around the Y axis around the origin Om of the imaging surface around the X axis. In each case, clockwise is defined as positive. This figure shows the result of a rotation input operation about the Z axis (the XY coordinate system is rotated by β degrees).

In the present embodiment, these 4
Although the points are selected, any four feature points that determine the posture position of the screen may be used. In the present embodiment, the detected position on the rectangular shape provided on the imaging surface is set as the origin of the XY coordinate system, and perspective projection conversion processing is performed on three points T1, S2, and Q1 in addition to the origin.

FIG. 14 is an orthographic view of the screen placed in the three-dimensional space shown in FIG. 13 on the XZ plane where Y ′ = 0. Here, only the side OmS2 exists in the XZ plane, and the remaining sides are projected. Focal length f
, The perspective projection transformation is performed by arranging the imaging plane at the position of, and the position coordinates of each point in the XY coordinate system are calculated. The results are shown in Equations 1 and 2.

[0063]

(Equation 1)

FIG. 15 shows the screen of FIG.
FIG. 5 is a diagram of orthographic projection of O on the YZ plane. In the figure, only the description relating to T1 and Q1 is shown, and the description relating to Q2, Q3 and Q4 is omitted. Similar to the processing performed on the XZ plane, the perspective projection conversion processing on the YZ plane is performed, and the coordinate positions of T1 and Q1 are calculated.

[0065]

(Equation 2)

Attention is paid to the feature points T1 and Q1 of the screen.
You. These project onto these two planes, the XZ plane and the YZ plane.
As a result of the perspective projection transformation, the coordinate values of T1 and Q1 are shown in FIG.
From T14 (X^* _t1, Z^* _t1| X) and Q1 (X^* ₁, Z^* ₁
| X), and from FIG.^* _t1, Z^* _t1| Y) and Q1
(Y^* ₁, Z^* ₁| Y) are obtained.

In the orthographic planes on the XZ plane and the YZ plane in FIGS. 14 and 15, the coordinate values for the Z axis have the same value and have the following relationship. Z ^* ₁ | x = Z ^* ₁ | y Z ^* _t1 | x = Z ^* _t1 | y The following two relational expressions (Equation 3) can be obtained from the above conditional expression.

[0068]

(Equation 3)

Equation 3 is a relational expression between the posture parameters of the screen placed in the three-dimensional space. These equations are simple relational expressions between one angle representing the rectangular position and orientation and a plurality of coordinate values on the imaging surface characterizing the image. Further, by substituting this equation into a number, another posture parameter θ can be obtained.

(B2) Coordinate calculation process Next, the process proceeds to step S207 in FIG. 9, in which the detected position on the plane is calculated by adjusting the detected position on the plane to the reference position set on the imaging surface. Do. When the XY coordinate system is rotated by β degrees and converted to the X′-Y ′ coordinate system, the formula for calculating the coordinate position on the screen in the three-dimensional space is represented by Expression 4.

[0071]

(Equation 4)

The coordinate system on the screen is (X ^* i, Y ^* i). When the detected position on the screen is represented by a horizontal axis ratio m and a vertical axis ratio n, the coordinate conversion operation is represented by the following equations (Equations 5 and 6)
It is performed by

[0073]

(Equation 5)

[0074]

(Equation 6)

Umax and Vmax used in Equation 6 are predetermined lengths between feature points Q2Q3 and Q3Q4. FIG.
(A) and (b) show the relationship between the coordinate system of the four feature points Q1, Q2, Q3, and Q4 forming the previously formed rectangular shape and the corresponding coordinate system projected on the screen. FIG.

As described above, in the present embodiment, since the reference position of the imaging surface is provided at the center position, the angle representing the position of the screen with respect to the position of the imaging surface in the three-dimensional space is a simple relational expression. . In addition, two vanishing points occur in the image data of the screen. However, the detection of the posture position on the predetermined plane and the detected position on the predetermined plane in the three-dimensional space includes:
It is possible if at least one vanishing point can be obtained in the obtained image data of the predetermined plane. That is, any shape having at least a pair of parallel lines may be used. [Second Embodiment] FIGS. 17A and 17B are diagrams for explaining a second embodiment of the present invention.

Unlike the first embodiment in which the target object image displayed on the screen is shot using the gun type controller, the second embodiment uses the same gun type controller as the target object. This is a game that tries to shoot a 3D object in real space.

The upper part of FIG.
FIG. 14 is a diagram in which the operator holding the target is aiming at the target three-dimensional object 801 on the cubic base 804 in the real space. A motor is provided in the cubic base 804 so that the motor can rotate around a rotation axis 805. On one surface 804a of the base 804, four light emitting elements (LEDs) 806a and 8
06b, 806c and 806d are provided. FIG.
The lower diagram of FIG. 7A shows a target object image on the imaging surface of the controller at that time. LED images 806a, 806b, 806c, and 80 corresponding to the four LEDs are provided on the imaging surface.
6d is shown as a feature point. The detected position Ps is the center 803a of the imaging surface.

In FIG. 17A, the object 801 in the real space is not suitable for the operator. When the operator makes a shot toward the target object hand 801a and hits, the motor is driven, and the object in the real space moves according to the posture and position of the operator.

Now, the processing operation when a shot and a hit are made will be described. The shot image 80
3 is taken in, and four feature points are extracted. The posture of the cubic-based LED surface 804a with respect to the imaging surface is calculated based on the coordinates of these feature points, and posture parameters ψ, γ, and β are calculated. The detected position is calculated using one of the posture parameters. The coordinates of the detected position are compared with the coordinates of an object provided at a predetermined position, and when it is determined that a hit has occurred, the motor is driven. The cubic base surface 804a is rotationally driven in the direction of the operator.

FIG. 17B is a diagram showing a state after the cubic base surface is rotated by an angle ψ about the rotation shaft 805 so as to face the operator. Although the description has been made with reference to one-axis rotation in the figure, the driving method is not limited. As described above, the game device according to the present embodiment can directly indicate a specific portion of the object image intended by the operator from any position on the display screen, and each time, the posture of the imaging surface with respect to the screen, Since various parameters such as the position can be calculated and the position of the operator can be specified, a new game effect that is more powerful and full of reality is possible. For example, regardless of aiming at the right hand of the target object, when hitting the left foot, the target object performs predetermined operation mode processing according to the shooting situation, and the input operation position direction of the player by the gun input operation, that is, It is also possible to develop an interactive game story in which the player shoots back in the visual axis direction.

Further, since the game device according to the present embodiment can obtain operation position information, it can detect the direction aimed at the screen even if the operation position is a fixed point position. On the other hand, the game can be performed as a motion in which the object motion is full of realism. For example, in a three-dimensional shooting game apparatus, when a shot is directed toward a target object on a display image, the speed at which the bullet hits the target object can be made different depending on the perspective of the object. That is, even if the specific portion of the object image on the screen is at the same coordinate position, the expression of the object motion can be made different depending on the aimed direction, and a game full of variety can be created.

The input operation parameters of the game device according to the present embodiment may be at least one posture parameter even if distance data from the input operation means to the screen is not obtained.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a game device according to an embodiment.

FIG. 2 is an overall configuration perspective view of the game device according to the first embodiment.

FIG. 3 is a configuration perspective view of a gun-type controller according to the present embodiment.

FIG. 4 is a first optical system of an aiming unit according to the embodiment.

FIG. 5 is a second optical system of the aiming unit according to the embodiment.

FIG. 6 is a flowchart illustrating a basic operation according to the embodiment.

FIG. 7 is a view for explaining a relationship between a fixed image viewpoint and an operation viewpoint.

FIG. 8 is a view for explaining the relationship between an image viewpoint and an operation (imaging) viewpoint according to the first embodiment;

FIG. 9 is a detailed flowchart of a position calculation processing unit according to the embodiment.

FIG. 10 is an image of a screen taken.

FIG. 11 shows each coordinate system on a captured image plane.

FIG. 12 is a perspective view illustrating a coordinate system of three-dimensional perspective projection transformation.

FIG. 13 is a perspective view illustrating three-dimensional perspective projection conversion.

FIG. 14 is an orthographic view of the screen in FIG. 13 on an X′-Z ′ coordinate plane.

FIG. 15 is an orthographic view of the screen in FIG. 13 on a Y′-Z ′ coordinate plane.

FIG. 16 is a view for explaining the relationship between a predetermined coordinate system and a screen coordinate system.

FIG. 17 is a diagram illustrating a shooting game in a three-dimensional real space according to the second embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 100 input operation means 101 imaging means 102 aiming means 103 distance measurement means 110 screen 111 image projected on screen 120 game device main body 130 projector 20 external storage medium 4 CPU 51 feature point extraction means 52 position calculation means 521 plane attitude calculation Means 522 Detected position (target) coordinate calculating means 7 Input operation position specifying means 8 Object part determining means 9 Image synthesizing means

Claims (26)

[Claims] 1. A game device for changing a target based on posture information of an operator, comprising: an image pickup means for picking up an image of the target including at least four feature points on a plane; And a posture calculating means for calculating a posture parameter of the imaging plane with respect to the plane based on the obtained captured image data, and changing the target based on the posture parameter obtained by the posture calculation means. apparatus. 2. The game device according to claim 1, wherein the target is an object in a real space, and the object is driven by the posture parameter. 3. The game device according to claim 1, wherein the target is an object in a virtual three-dimensional space, and the object is moved according to the posture parameter. 4. The game apparatus according to claim 1, further comprising a distance measuring unit that measures a distance from the target plane to the imaging plane, and moving the object in consideration of the distance information. 5. The apparatus according to claim 1, further comprising a detected position coordinate calculating unit that calculates a target position coordinate on the object using at least one posture parameter calculated by the posture calculating unit. Game equipment. 6. The game device according to claim 1, wherein the attitude calculation means performs the calculation using a vanishing point on the captured image. 7. A game apparatus for playing by displaying a target object image together with at least four feature points on a display screen, wherein said imaging means captures said object image including said feature points, and said imaging means Attitude calculating means for calculating an attitude parameter of the imaging surface with respect to the display screen based on image data on the imaging surface, and changing an object image of the display screen based on the attitude parameter obtained by the attitude calculating means. Characteristic game device. 8. The game device according to claim 7, wherein the display screen is a projection image projected from a projector. 9. The apparatus according to claim 7, further comprising a distance measuring unit for measuring a distance between the display screen and the imaging surface, wherein the object image is changed in consideration of an output signal from the distance measuring unit. Game equipment. 10. A detected position coordinate calculation means for calculating coordinates of a detected position on the display screen using at least one posture parameter calculated by the posture calculation means. Item 8. The game device according to Item 7. 11. The apparatus according to claim 7, wherein said attitude calculation means performs calculation using a vanishing point on a captured image.
The game device according to the above. 12. A game device for displaying and playing a target object image including a plurality of feature points on a display screen, wherein the target object image displayed on the display screen includes at least four feature points. Imaging means for imaging, position calculation means for calculating a detected position based on the feature point coordinates on the captured image data from the imaging means,
An object part determining means for comparing and determining the calculation result of the detected position calculated by the position calculating means with the coordinates of a predetermined target object part; anda target object image of a display means based on the result of the part determining means. A game device characterized by changing. 13. The position calculating means calculates a coordinate of the detected position using a posture calculating means for calculating a posture parameter of an imaging surface with respect to a display screen and at least one posture parameter calculated from the posture calculating means. 13. The game device according to claim 12, further comprising a detected position coordinate calculating means. 14. The game device according to claim 13, wherein said attitude calculation means performs calculation using a vanishing point on a captured image. 15. The game apparatus according to claim 12, wherein said position calculating means performs the calculation by matching a center position of the imaging surface with a detected position. 16. The game device according to claim 12, wherein the target object image is changed when a value obtained from the position calculation means is close to a predetermined coordinate value. 17. The game apparatus according to claim 12, further comprising: aiming means for setting a target as an index on an imaging surface provided on said imaging means. 18. The game apparatus according to claim 12, further comprising aiming means for determining a target object irrespective of a position on the imaging surface of said imaging means. 19. A game method for executing a game in which a target is changed based on posture information of an operator, wherein an imaging step of imaging a target including at least four feature points on a plane, and the imaging step And a posture calculating step of calculating a posture parameter of the imaging surface with respect to the plane based on the image data on the imaging surface obtained from the above. Characteristic game method. 20. A feature point extraction step of extracting a feature point based on image data of a target including at least four feature points on a plane, and a feature point extracted by the feature point extraction step. A program that executes a step of calculating the attitude of the imaging surface with respect to the predetermined plane based on the coordinates of the target plane and a step of changing the target based on the attitude parameter calculated in the attitude calculation step. Readable storage medium. 21. A program for executing a coordinate calculation step of calculating a coordinate position of the target using at least one of the posture parameters calculated in the posture calculation step. The storage medium according to the above. 22. A game method for playing by displaying an object image together with at least four feature points on a display screen, wherein: an imaging step of capturing the object image including the feature points; And a posture calculation step of calculating a posture parameter of the imaging surface with respect to the display screen based on the captured image data of the above, wherein the object image of the display screen is changed based on the posture parameter obtained by the posture calculation means. And the game method. 23. A feature point extracting step of extracting a feature point based on image data of an object image on a display screen including at least four feature points, and a feature extracted by the feature point extracting step. A program for executing a step of calculating an attitude of the imaging surface with respect to a display screen of the object image based on the coordinates of a point and a step of changing the object image based on the attitude parameter calculated in the attitude calculation step is stored. A readable storage medium characterized by the following. 24. A program for executing a coordinate calculation step of calculating a coordinate position of the target using at least one of the posture parameters calculated in the posture calculation step. The storage medium according to the above. 25. The storage medium according to claim 24, wherein a program for executing a step of calculating using a vanishing point on a captured image is recorded in the coordinate calculation step. 26. A game method for displaying and playing a target object image together with at least four feature points on a display screen, wherein the target object image displayed on the display screen includes at least four feature points. An imaging step of imaging, a coordinate calculation step of calculating a detected position on a display screen based on the coordinates of the feature point on the imaging surface obtained from the imaging step, and the calculation performed by the coordinate calculation step An object part determining step of comparing and determining the calculation result of the detected position and the coordinates of a predetermined target object image part; and changing the target object image on the display screen based on the result of the part determining step. Characteristic game method.