JP2000132306A - Device and method for inputting information and game device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the operability of an input tool such as a virtual image solid synthesizer and to constitute a virtual character bleeding game device or the like with high reproducibility while using that input tool. SOLUTION: This device is provided with nine light emitting diodes(LED) of different flicker patterns attached on the specified face of an image pickup enable simplified commander 4, consecutive photographing CCD 23' for picking up images so as to let these LED pan in a prescribed image pickup direction, arithmetic means 6 for finding position information Dp of these respective LED by performing image processing to the luminance information of these LED, and data generating means 7 for generating input information Din corresponding to the position information Dp of LED from the arithmetic means 6. In this case, when any arbitrary flicker pattern of LED is shielded, the data generating means 7 generates the input information Din corresponding to the position information Dp of any LED excepting for the LED which shields the flicker pattern at least.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information input device, an information input method, and a game device suitable for application to a so-called virtual character breeding game device in which a TV program character is played with a palm in a virtual space.

More specifically, there is provided means for generating input information in accordance with positional information of light sources having different blinking patterns attached to a specific surface of an arbitrary object that can be imaged, and blinking an arbitrary light source among a plurality of light sources. When the pattern is shielded, at least, it is possible to generate input information according to the position information of the light source other than the light source whose blinking pattern is shielded,
A simple commander optimal for an input tool such as the above-described virtual character breeding game device can be configured.

[0003]

2. Description of the Related Art In recent years, with the improvement of display technology based on virtual reality, a virtual image three-dimensional synthesizing apparatus for providing a viewer with virtual reality over a plurality of image display surfaces has appeared. are doing.

[0004] This type of stereoscopic display device is disclosed in Japanese Unexamined Patent Publication No. 9-23.
7353 can be found in the technical literature. According to this technical document, a projection space of about several meters in length and width is provided, and a display device is arranged on each surface, and a dinosaur,
Images of virtual bodies such as monsters and weapons are displayed in three dimensions. Then, when the observer wears glasses with a liquid crystal shutter and stands in the projection space, it is as if he were at the same place as the virtual object displayed on each display device.

[0005] Further, a weapon that an observer holds in a virtual space is imaged by a camera, and image processing is performed so that a virtual body reacts according to the movement of the weapon. This allows the observer to slip back in time to the primitive era thousands of years ago, and perform dinosaur extermination, etc., as if playing a game.

[0006]

According to the conventional three-dimensional display device, an input tool such as a jog dial is used to specify a virtual object to be projected into a projection space. In the image processing system, a virtual body such as a dinosaur appears in the virtual space and is synthesized and displayed based on the displacement information of the jog dial operated by the observer.

[0007] Therefore, the characters of the TV program are virtually projected on the reference plane of the real space to which the observer belongs,
When trying to construct a virtual image stereoscopic synthesis device that makes the character play with the palm of the observer in the virtual space, if the conventional jog dial is applied as it is,
There is a problem that the lead wire is routed from the input tool to the image processing system or the input operation becomes complicated, resulting in poor operability as a commander of the game device.

Accordingly, the present invention has been made in view of the above-mentioned problems, and has been made to improve the operability of an input tool such as a virtual image three-dimensional synthesizing device, and to use the input tool to reproduce a TV program. An information input device that can be configured with good reproducibility, such as a virtual character breeding game device for playing a character with a palm in a virtual space,
It is an object to provide an information input method and a game device.

[0009]

SUMMARY OF THE INVENTION An object of the present invention is to provide a plurality of light sources having different blinking patterns attached to a specific surface of an arbitrary object which can be photographed, and a plurality of light sources having different blinking patterns to flow in a predetermined photographing direction. Image processing means for obtaining image information of the light source by the image processing means; calculating means for obtaining position information of each of the light sources; and data generation for generating input information corresponding to the position information of the light source by the operation means. And an information input device characterized by comprising:

According to the information input device of the present invention, the input information corresponding to the position information of the light sources having different blinking patterns attached to a specific surface of an arbitrary object that can be imaged is generated by the data generating means. For example, when a blinking pattern of an arbitrary light source among a plurality of light sources is shielded, input information corresponding to the position information of the light source of which the blinking pattern is shielded and the position information of other light sources is generated.

Therefore, it is possible to configure a simple commander which is most suitable for an input tool such as a virtual character breeding game device in which a TV program character can be played with a palm in a virtual space.

In the information input method according to the present invention, a plurality of light sources having different blinking patterns are attached to a specific surface of an arbitrary object which can be photographed, and the light sources having different blinking patterns are imaged so as to flow in a predetermined imaging direction. Here, image processing is performed on the luminance information of the captured light source to obtain position information of each of the light sources, and when an arbitrary blinking pattern of the light source is shielded, at least:
The present invention is characterized in that input information is generated in accordance with position information of a light source other than the light source that shields the blinking pattern.

According to the information input method of the present invention, when a blinking pattern of an arbitrary light source is shielded from a plurality of light sources having different blinking patterns, position information of a light source other than the light source whose blinking pattern is shielded is provided. Therefore, a simple commander comparable to a wireless remote controller most suitable for an input tool such as the above-described virtual character breeding game device can be configured with good reproducibility.

A game device according to the present invention is a device for stereoscopically combining an arbitrary character image with an external image to which an observer belongs, and a plane recognition device for recognizing an arbitrary reference plane in a real space to which the observer belongs. Means, a synthesizing means for synthesizing a character image on a reference plane in a virtual space recognized by the plane recognizing means, and at least input means for instructing input information for changing the character image, the input means comprising: A plurality of light sources having different blinking patterns attached to a specific surface of an arbitrary object that can be photographed, an imager for imaging the light sources having different blinking patterns in a predetermined imaging direction, and a light source by the imager. Calculating means for obtaining position information of each of the light sources by performing image processing on the luminance information of the light source; and data generating means for generating input information corresponding to the position information of the light source by the calculating means. It is characterized in that.

According to the game device of the present invention, since the above-mentioned information input device is applied to the input means, the input operation is simple and the virtual character raising game device having a simple commander with good operability is provided. Can be configured.

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information input device, an information input method and a game device according to an embodiment of the present invention will be described below with reference to the drawings.

(1) Information Input Device as Embodiment FIG. 1 shows an information input device 1 as each embodiment according to the present invention.
It is a block diagram which shows the example of a structure of 00.

In this embodiment, there is provided means for generating input information in accordance with positional information of light sources having different blinking patterns attached to a specific surface of an arbitrary object that can be imaged, and an arbitrary light source among a plurality of light sources is provided. When the blinking pattern is shaded, input information can be generated at least in accordance with positional information of a light source other than the light source whose blinking pattern is shaded, and a TV program character can be played with a palm in a virtual space. A simple commander most suitable for an input tool such as a virtual character training game device can be configured.

The information input device 100 of the present invention has a simple commander 4 shown in FIG. 1 as an input tool. The simple commander 4 is an arbitrary object that can be imaged. For example, nine light emitting diodes LED are mounted on a specific surface as a plurality of light sources having different blinking patterns. The blinking pattern of the light emitting diode LED is controlled by the blinking control circuit 5 described with reference to FIG. In this example, when the finger slides on the light emitting diode LED of the simple commander 4, the light emitting position of the light emitting diode LED hidden by the finger changes, and it is possible to have almost the same function as the jog dial. Therefore, this simple commander 4 is
Can be used instead of jog dial.

Further, the information input device 100 is provided with a panning CCD device 23 'as an image pickup means so that the nine light emitting diodes LED which are blinked so that the above-mentioned blinking patterns are different are caused to flow in a predetermined photographing direction. It is imaged. Here, panning refers to a shooting mode in which signal charges are read out of a photoelectric conversion element (such as a photodiode) a plurality of times during the same field period in the panning CCD device 23 '.

In this example, when a two-dimensional imaging device of an interline transfer system having a vertical transfer unit is used as the panning CCD device 23 ', a plurality of times from the photoelectric conversion element to the vertical transfer unit during the same field period. The signal charge is read. When a frame transfer type two-dimensional imaging device having a charge storage unit is used as the panning CCD device 23 ', signal charges are read from the photoelectric conversion element to the charge storage unit a plurality of times during the same field period. .

The output means of the panning CCD device 23 'is connected to a calculating means 6, and the luminance information of the nine light-emitting diodes LED is input from the panning CCD device 23' and image-processed. Position information Dp of each light emitting diode LED is obtained. Data output means 7 is connected to the output stage of the operation means 6, and input information Din corresponding to the position information Dp of the nine light emitting diodes LED attached to the simple commander 4 is generated. The data generating means 7 generates input information Din corresponding to at least the position information Dp of the light-emitting diodes LED other than the light-emitting diode LED whose light-emitting diode LED is light-shielded when an arbitrary blinking pattern of the light-emitting diode LED is light-shielded.

Therefore, when the information input device 100 is applied to a virtual character breeding game device or the like in which a TV program character can be played with a palm in a virtual space, for example, the right hand of the nine light emitting diodes LED can be used. Since the input information Din corresponding to the position of the light emitting diode LED covered with the forefinger can be generated, a complicated operation such as a jog dial is not required.

In this example, the data generating means 7 is provided with a storage means 16 for storing input information Din associated with position information Dp of nine light emitting diodes LED. The storage means 16 may use a ROM (read only memory) and store an input software program for a jog dial. With this configuration, 9
The input information Din can be read from the storage means 16 using the position information Dp of the light emitting diodes LED as an address. The contents of the storage means 16 are described in FIGS.
Will be described.

In this example, the simple commander 4 has a diamond-shaped plate portion 14 shown in FIG. 2 and a ring-shaped ring portion 15 joined to the plate portion 14. The ring portion 15 is moved by the plate portion 14 of the panning CCD device 2.
It is good to attach to the left index finger or the like so as to be within the shooting range of 3 '. The plate portion 14 forms a light emitting surface, and is formed, for example, in a mountain shape (three-dimensional shape) with a raised central portion in order to facilitate panning. The size of the plate portion 14 is about the size of a wristwatch, and the length of one side is about 3 cm, and at most about 5 cm.

On a diagonal line of the plate portion 14, nine light emitting diodes LED10 to LED12, LED21, 23, LED
31, 32 and LEDs 41, 42 are arranged along the diagonal. In this example, the light emitting diodes LED10 to L
ED12, LED21, LED23, LED31, LED32, LE
D <b> 41 and D <b> 42 are embedded in a cross shape in the plate portion 14 in which a resin member is formed in a rhombus shape. Of course, the simple commander 4 is not limited to the one shown in FIG.
Has a flat shape, and 12 light emitting diodes LE are provided around the surface of the plate portion 14 '.
D may be embedded.

In this example, the light emitting diodes LED10 to LED12, LED2 arranged in a cross shape shown in FIG.
1, 23, LEDs 31, 32, and LEDs 41, 42 are controlled to blink in four groups. The first group is 3
Consisting of two light emitting diodes LED10 to LED12,
As the position coordinates of the three light emitting points p10 to p12, for example,
(X5, y5), (x6, y6), (x8, y8) are given, and the LEDs are numbered as the first to third light emitting diodes of the G1 group.

The second group includes two light emitting diodes LE.
D21, LED22 and two light emitting points p21, p21
For example, (x7, y7), (x9,
y9), and are numbered as the first and second light emitting diodes of the G2 group.

The third group includes two light emitting diodes LED31 and LED32, and two light emitting points p3.
1, (x4, y4) as the position coordinates of p32,
(X2, y2) and numbering such as the first and second light emitting diodes of the G3 group. The fourth group includes two light emitting diodes LED41 and LED42, and two light emitting points p4.
1, (x3, y3) as the position coordinates of p42,
(X1, y1) are given, and numbering is performed such as the first and second light emitting diodes of the G4 group.

The nine light emitting diodes LED10-L
ED12, LED21, LED23, LED31, LED32, LE
D41 and D42 are blinked at least in different blinking patterns so as to clarify the attachment position, and are divided into four groups G1 to G4 so as to exhibit the function as the mark portion. Flashing is controlled.

In the simple commander 4 of this example, the light emitting diodes LED10 to LED12, LED21, LED23,
If the LEDs 31 and 32 and the LEDs 41 and 42 are not mounted separately, the flash position cannot be recognized with good reproducibility by the panning CCD device 23 '. Output.

The nine light emitting diodes LED10-L
ED12, LED21, LED23, LED31, LED32, LE
In order to recognize the position information Dp of D41 and D42, the panning CCD device 23 ignores the blinking pattern every quarter cycle (time division), and the panning CCD device 23 '
May be provided and operated at every 1/4 period. In the present embodiment, the former case will be described. The recognition of the position information Dp will be described with reference to FIGS.

Next, a method of using the simple commander 4 will be described. FIG. 5 is a top view showing a usage example of the simple commander 4. The simple commander 4 shown in FIG. 5 shows a case where the first and second light emitting diodes LED41 and LED42 of the G4 group are shielded by the right forefinger 30R, for example.

In this case, the first group other than the G4 group,
Light-emitting diodes LE of the second and third groups G1 to G3
D10 to LED12, LED21, LED23, LED31,
The flashing pattern is imaged by the panning CCD device 23 '. Therefore, the calculating means 6 can determine which group of which light emitting diode has disappeared.

In this example, the light emitting diodes LED10 to LED10 by the simple commander 4
12, the luminance information of the LEDs 21 and 23, the LEDs 31 and 32 is input from the panning CCD device 23 ', and the image is processed and the light emitting diodes LED10 to LED12, LED
Position information Dp of each of the LEDs 21 and 23 and the LEDs 31 and 32 is obtained. As a result of this image processing, for example, the light source shaded by the right forefinger 30R is the first light source of the G4 group.
And second light emitting diodes LED41, LED42
Is recognized.

That is, when nothing is covered on the plate portion 14, the nine light emitting diodes LED10 to LED12, LED2 of the simple commander 4 having different blinking patterns are provided.
FIG. 6 shows an image of the LEDs 1, 23, the LEDs 31, 32, and the LEDs 41, 42 taken by the ordinary CCD image pickup device 25. However, with respect to the blinking pattern of the panning signal (luminance signal) SOUT output from the panning CCD device 23 ', nine panning bright points p10 to p12 in the image area defined by the window W shown in FIG. , P2
It is converted into a spatial arrangement pattern forming an XY plane including 1, p22, p31, p32, p41p, and p42.

After that, the arrangement pattern is scanned,
At least nine bright points p10 to p12, p21, p2
2, p31, p32, p41p, p42 position coordinates (X
1, Y1), (X2, Y2), (X3, Y3), (X
4, Y4), (X5, Y5) (X6, Y6), (X7,
Y7), (X8, Y8), and (X9, Y9) are obtained. For example, on the arrangement pattern shown in FIG. 7, when the flowing imaging direction is the Y axis and the direction orthogonal to the Y axis is the X axis, the arithmetic unit 6 sets the luminance signal in the same direction as the flowing imaging direction or in the opposite direction. The values are added. This added value is X
When plotted on the axis, nine positions at which the luminance signal value plotted on the X axis is maximum are detected, and X coordinate values X1, X2, X3, X4, X corresponding to the nine positions are detected.
5, X6, X7, X8 and X9 are determined.

Further, the acquired image is represented by Y on the arrangement pattern.
When scanning in the Y direction, among the plurality of luminescent points arranged in the Y direction, the luminescent spot position that first emits light has Y coordinate values Y1, Y2, Y3, Y4, Y5, Y6, Y
7, Y8 and Y9. These nine bright spots p1
0 to p12, p21, p22, p31, p32, p41
p and p42 are, for example, nine light emitting diodes LED10 to L of the simple commander 4 attached to the left forefinger of the observer.
ED12, LED21, LED23, LED31, LED32, LE
The position information Dp of D41 and D41 is shown.

Accordingly, the right forefinger 30 shown in FIG.
In the simple commander 4 in which the first and second light emitting diodes LED41 and LED42 of the G4 group are shielded by R, the first, second and third groups G other than the G4 group are used.
1 to G3 light emitting diodes LED10 to LED12, L
The seven position coordinates by the EDs 21 and 23 and the LEDs 31 and 32 are (X2, Y2), (X4, Y4), (X
5, Y5) (X6, Y6), (X7, Y7), (X8,
Y8) and (X9, Y9) are obtained, and the position coordinates (X1,
Y1) and (X3, Y3) are not required. Therefore, G4
The input information Din based on the position coordinates of the first to third groups G1 to G3 other than the group can be read from the storage unit 16. FIG. 8 shows nine light emitting diodes LED10 to LED12, LEDs 21, 23, and LE.
It is a table | surface figure which shows the number of D31,32, LED41,42, and the example of its mounting position information.

Next, an example of input information stored in the storage means 16 will be described. FIG. 9 shows four groups G1 to G
6 is a reference table showing an example of input information for position information Dp of No. 4; In this example, a case where the position information Dp is input for each group in order to simplify the description of the simple commander 4 will be described. Therefore, the four groups G1 to G1
By combining "cover" or "not cover" G4, 16 types of input information Din can be obtained.

In this example, a simple commander 4 for a virtual character breeding game device is assumed in which a 3D polygon (snowman) 10 as a virtual body exists on the reference plane of the display means 24 shown in FIG. In this case, the icon 24A is displayed on the screen of the display means 24, and “input standby”, “pop out”, “move to the left”,
16 types of instruction information such as "move right" ... "disappear" are displayed. The icon 24A can be designated by a cursor 24B displayed on the screen. The cursor 24B moves based on the simple commander 4.

Here, an example will be described in which the 3D polygon 10 shown in FIG. 11 is moved in the x direction and the y direction. The simple commander 4 can perform an input operation like a jog dial. That is, the simple commander 4 shown in FIG.
Light emitting diodes LED10 to LED12, LED2
With respect to 1, 23, LEDs 31, 32, and LEDs 41 and 42, position information Dp = “1” when no light shielding is performed.
111 ", the input information D
in = “waiting for input” is stored in the storage unit 16 in advance. The non-light-shielding pattern when no light-shielding is performed is PT
= 0 and PT = 7.

In this example, the light emitting diodes LED10 to LED12, LED21,
3, the LEDs 31, 32, and the LEDs 41, 42 are sequentially covered in a horizontal direction with, for example, the index finger, the middle finger, or the ring finger of the right hand. FIG. 13 shows a case where the light-emitting diodes LED42 of the group G4 are shielded from light. This light shielding pattern is set to PT = 1. Further, in FIG.
4 shows a case where the two light emitting diodes LED 41 and 42 are shielded from light. This light-shielding pattern is set to PT = 2.

In FIG. 15, the light emitting diodes LED41 of the group G4, the light emitting diodes LED10 to 12 of the groups G1 and G3, and the light emitting diode LED31
To 32 are shaded. This light-shielding pattern is set to PT = 3. Further, in FIG. 16, the groups G1, G3
Light emitting diodes LED10-12, light emitting diodes LED31-32, and light emitting diode L of group G2
This is the case where the ED 21 is shielded from light. This light-shielding pattern is set to PT = 4.

FIG. 17 shows a case where the two light emitting diodes LED21 and LED22 in the group G4 are shielded from light. It is assumed that this light-shielding pattern PT = 5. Further, FIG. 18 shows a case where the light emitting diodes LED22 of the group G2 are shielded from light. This is set to PT = 6 for the light shielding pattern.

In this example, the jog dial mode can be set by one of the commands in FIG. 9, and when the light shielding pattern PT = 0 → 7 is input, it becomes equivalent to the right rotation operation of the jog dial. PT = 7 →
Entering 0 is equivalent to turning the jog dial left. For example, when PT = 0,1,2,3 → 2,1,0 is continuously input, the same result as when the jog dial is switched from right rotation to left rotation is obtained.
The cursor moves upward or downward on the left side of the type icon 24A. In this example, when the cursor 24B stops, an instruction to "determine the operation" is issued.

In the determination operation of the simple commander 4 like a jog dial, "YE"
S ”and“ NO ”when the light-shielding pattern PT = 5 → 0. For example, if the light-shielding pattern PT = 0 → 5 is continued once or twice, and if the light-shielding pattern PT = 0 → 5 is input again as an answer such as “Do you want to do something?”, “Y
ES ”, and inputting the light shielding pattern PT = 5 → 0 can indicate“ NO ”. Alternatively, the light emitting diodes LE of the group G4 may be pressed with the forefinger of the right hand shown in FIG.
A jog dial-like determination operation can also be instructed by shielding D42 from light and shielding the light emitting diodes LED22 of group G2 with the middle finger.

FIGS. 11 to 18 show the simple commander 4.
The case where the finger of the right hand is slid in the horizontal direction has been described above, but the present invention is not limited to this, and FIGS.
As shown in FIG. 4, the finger of the right hand may be slid vertically on the simple commander 4 '. FIG. 20 shows a case where the light emitting diodes LED12 of the group G1 are shielded from light. It is assumed that this light-shielding pattern PT = 1. Further, FIG. 21 shows a case where the light-emitting diodes LEDs 12 and 11 of the group G1 are shielded from light. This is set to PT = 2 for the light shielding pattern.

In FIG. 22, three light emitting diodes LED10-12 of group G1, a light emitting diode LED41 of group G4, and a light emitting diode LE of group G2.
D21 and D21 are shielded from light. This light shielding pattern P
Let T = 3. Further, in FIG. 23, the light emitting diodes LED10 to 12 of the group G1, the light emitting diode LED41 of the group G4, and the light emitting diode L of the group G2
ED21 and light emitting diode LED31 of group G3
Are light-shielded. This light shielding pattern PT = 4
And Further, in FIG. 24, the light emitting diodes LED10 to 12 of the group G1, the light emitting diode LED41 of the group G4, and the light emitting diode LED2 of the group G2.
1 and light-emitting diodes LED31, 32 of group G3
Are light-shielded. This is a light shielding pattern of PT
= 5.

Also in this example, the jog dial mode can be set by one of the commands in FIG. 9, and when the light shielding pattern PT is changed in order from 0 to 5, an input state equivalent to the right rotation operation of the jog dial is obtained. On the contrary, if the light-shielding pattern PT is sequentially changed from 5 to 0, an input state equivalent to the left rotation operation of the jog dial can be obtained. For example, when PT = 0,1,2,3 → 2,1,0 is continuously changed, the same result as when the jog dial is switched from right rotation to left rotation is obtained, and the 16 types shown in FIG. 10 are obtained. Is moved upward or downward on the left side of the icon 24A. Also in this example, when the cursor 24B stops, an instruction of “operation decision” can be given in the same manner as the simple commander 4.

The input information may be instructed and input as follows. In this example, four groups of light emitting diodes LED10 to LED12, LED2
With respect to 1, 23, LEDs 31, 32, and LEDs 41 and 42, position information Dp = “1” when no light shielding is performed.
111 ", the input information D
in = “waiting for input” is stored in the storage unit 16 in advance. Similarly, three groups G2 and G other than the G1 group
3. Position information Dp = “1000” when G4 is shielded from light
, The input information Din = “pop out” having this as an address is stored in the storage means 16.

The position information Dp = "0" when three groups G1, G3 and G4 other than the G2 group are shielded from light.
100 ”, input information D using this as an address
in = “move to the left” is stored in the storage unit 16 in advance. Similarly, three groups G1, G other than the G3 group
2. Position information Dp = “0010” when G4 is shielded from light
, The input information Din = “move right” with this as an address is stored in the storage means 16 in advance.

Further, position information Dp = "0" when three groups G1, G2 and G3 other than the G4 group are shielded from light.
001 ”is input information D
in = “jump” is stored in the storage unit 16 in advance. Hereinafter, similarly, the position information Dp = “11” when the group G4 other than the G1, G2, and G3 groups is shielded from light.
10 ", the input information Din having this address
= “Speak” is stored in the storage means 16 in advance. Then, for the position information Dp = “0000” when all the groups G1 to G4 are shielded from light, the input information Din = “disappear” having this as an address is stored in the storage means 16 in advance.

Therefore, four groups of light emitting diodes LED10 to LED12, LED21, 23, LED3
In the case where no light is shielded for the LEDs 1 and 32 and the LEDs 41 and 42, the position information Dp = “1111” is instructed to the storage unit 16 as an address. Information Din can be read.

When three groups G2, G3, and G4 other than the G1 group are shielded from light, the position information Dp =
Since “1000” is instructed to the storage means 16 as the address, the input information Din “jumping out” can be read from the storage means 16. Based on the input information Din of "popping out", it is possible to perform image processing such that characters of a TV program pop out in the virtual space.

Then, with the right hand, all the groups G1 to G
When the light 4 is shielded, the position information Dp = “0000” is instructed to the storage means 16 as an address, so that the input information Din “disappear” can be read from the storage means 16. Image processing such as erasing a TV program character from the virtual space can be performed based on the input information Din of “erasure”.

As described above, according to the information input device 100 of the present embodiment, nine light emitting diodes LED10 to LED12, LED21, LED23,
31, 32, LED41, 42, light emitting diodes LED10-12 of any group G1-G4, LED
When the blinking pattern of the LEDs 21 and 23, the LEDs 31 and 32, and the LEDs 41 and 42 is shielded, input information Din corresponding to the position information Dp of the light emitting diodes in the group other than the group in which the blinking pattern is shielded can be generated. .

With this configuration, it is possible to configure a simple commander 4 with good reproducibility that is optimal and operable for an input tool such as a virtual character breeding game device in which a TV program character can be played with a palm in a virtual space. In addition, since no lead wire is routed from the simple commander 4 to the image processing system, input information can be instructed as if by an infrared remote controller or a wireless remote controller.

Thus, the input processing such as executing the input software program by the jog dial can be performed by the simple commander 4.
Therefore, the present invention can be sufficiently applied as an input commander for a virtual character breeding game device or the like. Moreover, in this example, a panning CCD for recognizing a reference plane on which characters of a TV program appear is used.
The device 23 and the panning CCD device 23 'for the simple commander 4 can be used as both.

(2) First Embodiment Next, a description will be given of a game device 200 for virtual character cultivation and the like to which the information input device 100 according to the present invention is applied. FIG. 25 shows a game device 20 according to the first embodiment.
FIG. 2 is a perspective view showing a configuration example of a No. 0.

In this example, in a so-called virtual character breeding game device in which a TV program character is played in the palm of an observer in a virtual space, the information input device 100 according to the present invention is applied to convert the character into a virtual character. Make it appear in space, move it left and right,
It is made to jump or erase from the virtual space.

The virtual character cultivating game device 200 shown in FIG. 25 is a device for stereoscopically displaying a virtual body image such as a TV program character on the external image to which the observer belongs. This game device 200 has a simple commander 4, a reference surface setting bracelet 1, a special glasstron 2, and an image processing device 3. The simple commander 4 is attached to, for example, the forefinger 30L of the left hand of the observer, and the bracelet 1 is used by being attached to the left hand.
The main body 21 of the special glasstron 2 is provided with a belt 22. The main body 21 is attached to the face of the observer as if wearing glasses, and the belt 22 extends along the outer periphery of the observer's head. Fixed.

The special glasstron 2 is provided with at least a panning CCD device 23 and display means 24. Normal CC depending on the model of Special Glasstron 2
A D imaging device 25 is provided. The bracelet 1 for setting the reference plane, the panning CCD device 23, and the image processing apparatus 3 constitute a plane recognition unit, and can recognize an arbitrary reference plane in the real space to which the observer belongs.

Further, an image processing device 3 is connected to the special glasstron 2, and image processing for recognizing a reference plane or the like is performed based on image data output from the panning CCD device 23. The display unit 24 is connected to the image processing apparatus 3 and displays the reference plane recognized by the plane recognition unit. The special glasstron 2 may be provided with optical means such as a polarizing beam splitter.
The image of the virtual body is synthesized on the reference plane of the virtual space displayed by 4. In this example, a 3D polygon (snowman) 10 as a virtual body exists at the position to which the reference plane in the real space belongs.

In this example, the reference surface setting bracelet 1
Has a plate portion 11 and a bracelet portion 12 shown in FIG. The plate portion 11 forms a reference surface, and is formed in a flat shape without irregularities. The size of the plate portion 11 is substantially the same as the size of the plate portion 14 of the simple commander 4, or has a rectangular shape slightly larger than this. Light-emitting diodes (LED1 to LED4) are attached to the four corners of the surface of the plate portion 11, respectively.
(X1, y1), (x2, y2), (x3,
y3) and (x4, y4) (corresponding to a reference plane on which an image is to be synthesized in the virtual space).

The four light emitting diodes LED1 to LED4
Is flashed so as to at least have a different flashing pattern in the same manner as the light-emitting diode LED of the simple commander 4 so as to exhibit its function as a mark portion, that is, so that its mounting position becomes clear. The light-emitting diodes LED1 to LED4 are controlled by a blinking pattern by a blinking control circuit 13 described with reference to FIG.

The light emitted from the light emitting diodes LED1 to LED4 is imaged by the panning CCD device 23 in the special glasstron 2 so as to flow in a predetermined panning imaging direction. This panning is for specifying the reference plane from the mounting positions of the four light emitting diodes LED1 to LED4. The specification of the reference plane will be described with reference to FIGS.

The bracelet portion 12 preferably has a structure that can be extended and contracted so as to fit the wrist of the observer. For example,
For example, Velcro (registered trademark) is provided on the band member so that the mounted state can be adjusted. The plate portion 11 is preferably mounted on the back side of the left hand so as to be within the shooting range of the panning CCD device 23.

The special glasstron 2 shown in FIG. 27 constitutes a non-transmissive head-mounted display, and includes a normal CCD image pickup device 25, the above-mentioned panning CCD device 23, and a first image display element. It has a liquid crystal display device (hereinafter referred to as LCD) 26 for right-eye display and an LCD 27 for left-eye display as a second image display element.

That is, a normal CCD image pickup device 25 and a panning CCD device 23 are arranged side by side at a position corresponding to the eyebrows of the observer, and the former captures an outside world image to which the observer belongs, and the latter captures a simplified image. Nine light emitting diodes LED10-12, LED21,2 of commander 4
2. The four light emitting diodes LED1 to LED4 of the LEDs 31, 32, the LEDs 41 and 42 and the bracelet 1 are shot in a panning manner. Therefore, when the observer looks at the commander 4 or the reference surface setting bracelet 1, the panning CCD device 23 is directed toward the light emitting surface or the reference surface.

An LCD 26 is attached to the special glasstron 2 at a position facing the right eye of the observer. For example, an observer's bracelet 1 photographed by a normal CCD image pickup device 25 and a computer One of the stereo images synthesized with the image of the 3D polygon 10 by graphics (CG) is displayed.

An LCD 27 is mounted at a position facing the left eye of the observer, and the simple commander 4 described above is
The other of the stereo image synthesized with the image of the bracelet 1 and the 3D polygon 10 is displayed. This special glasstron 2 is attached to the face or head of the observer, and the above-mentioned LCD
The stereo image of 26 and the stereo image of LCD 27 are directed to the eyeball of the observer. This allows
The background image to which the observer belongs and the 3D polygon 10 are combined in the head.

The special glasstron 20 shown in FIG. 28 constitutes a transmission type head mounted display, and does not have a normal CCD image pickup device 25 mounted thereon. Therefore, a transmissive head-mounted display is a panning CCD
A device 23, a liquid crystal shutter 28 for capturing an external image,
It has an LCD 29 as an image display element.

For example, a panning CCD device 23 is disposed at a position corresponding to the observer's eyebrow, and when the observer looks at the simple commander 4 or the bracelet 1 for setting the reference plane, the simple commander 4 9 light emitting diodes L
ED10-12, LED21,22, LED31,3
2. The four light emitting diodes LED1 to LED4 of the bracelet 1 and the LEDs 41 and 42 are panned. The liquid crystal shutter 2 is located at a position corresponding to the left and right eyes of the observer.
For example, when the liquid crystal shutter 28 is opened, a real image of the bracelet 1 or the simple commander 4 of the observer passing through the liquid crystal shutter 28 is directly guided to the eyeball.

An LCD 29 is attached to a portion of the special glasstron 2 located beside the left or right eye of the observer, and a character image is displayed in the same manner as the special glasstron 2 described above. Although not shown, the liquid crystal shutter 2
An optical means such as a polarizing beam splitter is provided between the LCD 8 and the LCD 29 so that the real image of the bracelet 1 and the simple commander 4 of the observer and the image of the 3D polygon 10 are guided to the observer's eyeball. I have. Thus, the background image to which the observer belongs and the 3D polygon 10 are combined in the head.

Next, the internal configuration of the follow-up CCD device 23 of the interline transfer system will be described. FIG.
The panning CCD device 23 shown in FIG. On the substrate 31, photodiodes PHij (i = 1 to n, j =
1 to m) are arranged in a matrix of n columns × m rows.

In the column direction of the substrate 31, m vertical transfer sections 32 are provided as charge transfer sections, and the signal charges read from the photodiodes PHij are used as vertical read signals S.
1 is transferred in the vertical direction (the panning direction).
The vertical transfer section 32 is connected to a horizontal transfer section 33, and the signal charges are transferred in the horizontal direction based on the horizontal read signal S2.
Is output. In this example, the signal charge is read out from the photodiode PHij to the vertical transfer unit 32 at least a plurality of times during the same field period in order to perform a panning shot.

The panning CCD device 23 has a fisheye lens 35 shown in FIG. The fisheye lens 35 is provided on the optical axis of the CCD image sensor 36, for example. The fish-eye lens 35 allows a wide range of images of the simple commander 4, the reference surface setting bracelet 1, and the like. Of course, a normal lens may be used, but since the field of view is narrowed, the observer must tilt the head more toward the simple commander 4 or the bracelet 1.

Next, the circuit configuration of the game device 200 will be described. The game device 200 shown in FIG. 31 is roughly composed of four circuit blocks. The first circuit block is a bracelet 1 for setting a reference plane. The bracelet 1 is provided with a blinking control circuit 13, which controls blinking by applying a predetermined voltage to the four light emitting diodes LED1 to LED4. In the blinking control circuit 13, the light emitting diodes LED1, LED2, LED3
And the blinking interval of the LED 4 is controlled. A small dry battery or button battery (not shown) is used as a power source for the light emitting diodes LED1 to LED4.

The second circuit block is a simple commander 4. The simple commander 4 is provided with a blinking control circuit 5, and includes nine light emitting diodes LED10 to LED12,
A predetermined voltage is applied to the LEDs 1, 22, the LEDs 31, 32, and the LEDs 41, 42 to control the blinking. In the blinking control circuit 5, the blinking pattern is different for each of the groups G1 to G4, and the light emitting diodes LED10 to 12, LED21, 22, LED31, and LED3 are arranged such that the blinking pattern is different within the group.
2. The blinking interval of the LEDs 41 and 42 is controlled. Light emitting diodes LED10-12, LED21,22, LED
As in the case of the bracelet 1, a small dry cell or button cell is used as a power source for the LEDs 31 and 32 and the LEDs 41 and 42.

The third circuit block is a special glasstron 2
In the case of the non-transmissive type, the panning CCD device 23 described above, the normal CCD image capturing device 25, and the LC for displaying the right eye are used.
D26 and LCD 27 for left eye display.

The fourth circuit block is the image processing device 3 and has an internal bus 41. The internal bus 41 includes an interface (I / O) 42, an image capture unit 43, an image processing unit 44, a CPU 45, a ROM 46, a RAM 47, and an E ² PROM (read only memory capable of writing and erasing electrical information) 48. It is connected. The panning CCD device 23, the normal CCD imaging device 25, the LCD 26 for right-eye display, and the LCD 27 for left-eye display are connected to the internal bus 41 via the interface 42.

The internal bus 41 has E ² as a recording medium.
The PROM 48 is connected, and the 3D
An algorithm for three-dimensionally synthesizing the image of the polygon 10 is stored. For example, the E ² PROM 48 captures an arbitrarily set reference plane in the real space to which the observer belongs, displays the reference plane in the virtual space, and displays an image of the 3D polygon 10 in the virtual space. An algorithm for superimposing and synthesizing is stored.

Therefore, by executing this algorithm, the reference plane in the real space can be easily recognized with a small amount of calculation. As a result, the 3D polygon 10
Can be image-processed so that there is a game device 20 such as a virtual character breeding game device.
0 can be configured with good reproducibility.

Further, a ROM 46 as the storage means 16 is connected to the internal bus 41, and stores input information Din for the position information Dp of the four groups G1 to G4 described with reference to FIG. Therefore, 16 kinds of input information Din can be obtained by a combination of “covering” or “not covering” the four groups G1 to G4. In addition, the ROM 46 stores a system program for controlling the game device 200, and control information such as a memory reading procedure.

The internal bus 41 has a working R
The AM 47 is connected, and the input information Din and the display information for displaying the system program and the character image are temporarily recorded. Further, a CPU 45 is connected to the internal bus 41, and an interface 42, an image capture unit 43, an image processing unit 44, a ROM 46, a RAM 47, and an E ² PRO
M48 input / output control, panning CCD device 23, C
Input / output control of the CD imaging device 25, the LCD 26, and the LCD 27 is performed.

The interface 42 has an image processing unit 4
For example, images of the four light-emitting diodes LED1 to LED4 for setting a reference plane shown in FIG. 32 captured by the normal CCD imaging device 25 are processed together with a control instruction of the CPU 45 via the interface 42. The image is taken into the unit 44, where predetermined image processing is performed, and the LCD 2 in the special glasstron 2 is again transmitted via the interface 42.
6 and the LCD 27.

An image capture unit 43 is connected to the interface 42. Upon receiving a control command from the CPU 45, a predetermined capture process for acquiring image data of a blinking pattern input from the panning CCD device 23 is performed. The image data of this blinking pattern is expressed as a change in luminance corresponding to the passage of time. The image capture unit 43 is connected to an image processing unit 44 as the arithmetic unit 6, and for the image data on which the predetermined image processing has been performed, the synchronization deviation of the blinking pattern is corrected, or the reference plane to which the observer belongs is obtained. .

For example, in the image processing section 44, the panning CC
Panning signal (luminance signal) S output from D device 23
The blinking pattern of OUT is converted into a spatial arrangement pattern that forms an XY plane including four follow shot bright points P1 to P4 in the image area defined by the window W shown in FIG. Then, by scanning the arrangement pattern, at least the position coordinates (X
1, Y1), (X2, Y2), (X3, Y3), (X
4, Y4) are required. These four luminescent spots P1 to P4 are used for setting the reference plane of the bracelet 1 attached to the observer.
Three light emitting diodes LED1 to LED4. 4 in real space
The position coordinates of the two light emitting diodes LED1 to LED4 are known, and the position coordinates are (x1, y1), (x2, y2),
(X3, y3) and (x4, y4).

Therefore, the above-described reference plane in the real space can be obtained by calculating a conversion matrix projected on the mounting positions of the four light emitting diodes LED1 to LED4. Here, the point (xi, yi, 0) on the plane of the real space is moved by a certain translation / rotational motion, and the point projected on the image coordinate system by the perspective transformation is represented by (Xi, Yi). There is a relationship represented by equation (1).

[0091]

(Equation 1)

Where a1... A8 are unknown coefficients and C
External parameters (position and direction) such as CD imaging device 25
And internal parameters such as focal length. These parameters are the position coordinates (x1, y) of a known point in the real space.
1), (x2, y2), (x3, y3), (x4, y
4) and four sets of position coordinates (X1, Y1), (Y2, Y2), (X3, Y3),
If (X4, Y4) exists, it can be obtained by solving the equation (2).

[0093]

(Equation 2)

The position coordinates of the four points obtained here (X1, Y
1), (X2, Y2), (X3, Y3), (X4, Y
By connecting 4), the reference plane in the real space shown in FIG. 32 is recognized.

More specifically, when the panning imaging direction is the Y axis on the arrangement pattern shown in FIG. 33 and the direction orthogonal to the Y axis is the X axis, the same direction as the panning imaging direction is set by the image processing unit 44. Alternatively, the luminance signal values are added in the opposite direction. When the added value is plotted on the X axis, four positions where the luminance signal value plotted on the X axis is maximum are detected, and the X coordinate values X1 and X corresponding to the four positions are detected.
2, X3 and X4 are determined. Further, when the acquired image is scanned in the Y direction on the arrangement pattern, among the plurality of luminescent points arranged in the Y direction, the luminescent spot position that first emits light has a Y coordinate value Y1 corresponding to the X coordinate value. , Y2, Y3, Y4
Is required.

Here, the position coordinates of the four light emitting diodes LED1 to LED4 in the real space are defined as wi (i = 1 to 4), and the position coordinates wi of the four light emitting diodes LED1 to LED4 on the camera coordinate system. Assuming that the expression vector is Ci, the position coordinates of the four light-emitting diodes LED1 to LED4 on the LCD screen are Pi, the rotation matrix of the panning CCD device 23 is R, and the movement vector is T, the following equation (3) is obtained.
That is, Ci = R · wi + T (3) where Ci = Pi · ki (ki is a scalar). Therefore, the normal CCD imaging device 25
Of the rotation matrix R and its movement vector T,
Using this as a parameter, coordinate conversion between the real space and the virtual space can be easily performed, so that a character image can be synthesized on a reference plane in the virtual space.

Next, the operation of the game device 200 will be described with respect to the information input method of the present invention. In this example, after the image of the 3D polygon 10 is three-dimensionally combined with the external image to which the observer belongs, the 3D polygon 1
It is assumed that 0 is changed. Obviously, the observer wears the simple commander 4 on the left forefinger 30L so that the light emitting surface faces upward, and moves the special glasstron 2 shown in FIG.
To the head.

First, in step A1 of the flowchart shown in FIG. 34, in order to set an arbitrary reference plane in the real space to which the observer belongs, the observer places his or her left arm on the left arm, for example, so that the plate portion 11 faces upward. The reference surface setting bracelet 1 is mounted. Thereafter, the blinking control unit 13 is turned on to blink the four light emitting diodes LED1 to LED4 in a predetermined blinking pattern. Before or after this, the blinking control circuit 5 is turned on, and the nine light emitting diodes LED10 to LED12, LED21,
2. The LEDs 31 and 32 and the LEDs 41 and 42 blink in a predetermined blink pattern.

Next, in step A 2, on the other hand, a reference plane in real space is photographed using the ordinary CCD image pickup device 25, and stereo images are displayed on the LCD 26 and the LCD 27. On the other hand, the simple commander 4 and the reference plane in the real space are shot using the panning CCD device 23. For example, four light emitting diodes LED1 to LED4 attached at positions where an image of the 3D polygon 10 is to be synthesized,
Since the blinking pattern is blinked differently, an image is taken so that the blinking pattern flows in a predetermined imaging direction.

Thereafter, in step A3, image processing is performed to recognize an arbitrarily set reference plane in the real space to which the observer belongs. In the image processing unit 44, for example, a subroutine shown in FIG. 35 is called to execute a video capture process in step B1. Then, in step B2, the light emitting diodes LED1 to LED4 at the four corners are recognized. Specifically, the blinking pattern of the luminance signal by the four light emitting diodes LED1 to LED4 captured by the panning CCD device 23 is converted into a spatial arrangement pattern forming an XY plane including the four bright points P1 to P4. You.

Then, the arrangement pattern is scanned, and
At least the position coordinates (X1,
Y1), (X2, Y2), (X3, Y3), (X4, Y
4) is calculated, and the above-described expressions (1) and (2) are calculated, and the mounting positions of the four light emitting diodes LED1 to LED4 in the real space and the position coordinates (X1, Y) of the four points of the image processing system.
1), (X2, Y2), (X3, Y3), (X4, Y
4) is obtained, and a reference plane is obtained by connecting these four points. Then, in step B3, the image processing unit 44 performs an arithmetic process based on the above equation (3).
The positional relationship between the panning CCD device 23 and the reference plane is detected.

On the other hand, the nine light-emitting diodes LED10 to LED12 of the simple commander 4 described with reference to FIGS.
21, 22, LED31, 32, LED41, 42-9
Bright points p10 to p12, p21, p22, p31, p
32, p41p, p42 position coordinates (X1, Y1),
(X2, Y2), (X3, Y3), (X4, Y4),
(X5, Y5) (X6, Y6), (X7, Y7), (X
8, Y8) and (X9, Y9). At this time, if the light emitting surface of the simple commander 4 is not covered at all, the light emitting diodes LED of the four groups G1 to G4
10 to LED 12, LED 21, 23, LED 31, 3
2. Regarding the LEDs 41 and 42, the position information Dp = “11
11 "is instructed to the ROM 46 as an address.
From the ROM 46, the input information Din of "input standby" can be read.

Thereafter, the flow returns to step A4, and image processing is performed based on the instruction from the simple commander 4 by the observer (see FIG. 5). For example, G of the simple commander 4
When an instruction is given to block the three groups G2, G3, and G4 other than one group with the right forefinger 30R, the images of the 3D polygons 10 are superimposed on the reference plane of the virtual space. . At this time, the position information Dp = “1000” by the panning CCD device 23
Is instructed to the ROM 46 as an address.
The input information Din “pop out” is read from the OM 46, and image processing such that the 3D polygon 10 appears on the reference plane can be performed based on the input information Din “pop out”.

Therefore, in the special glasstron 2 worn by the observer, one of the stereo images obtained by synthesizing the external world image in the real space by the LCD 26 and the image of the 3D polygon 10 is guided to the right eyeball of the observer. . The other stereo image obtained by synthesizing the external world image in the real space by the LCD 27 and the image of the 3D polygon 10 is guided to the left eyeball of the observer.

Thus, although the 3D polygon 10 does not appear on the reference plane in the real space shown in FIG. 36A, the 3D polygon 10 can appear on the reference plane in the virtual space shown in FIG. 36B. . Therefore, since the background image in the real space to which the observer belongs and the 3D polygon 10 appearing in the virtual space are synthesized in the head, it is as if the 3D polygon 10 is located at the position to which the reference plane in the real space belongs. Can exist.

Then, using the simple commander 4, the three groups G1, G3, G other than the G2 group are further added.
4, the position information Dp = "0100" is instructed to the ROM 46 as an address.
The input information Din “move to the left” can be read from M46. Image processing such as moving the 3D polygon 10 on the reference plane to the left can be performed based on the input information Din of “move to the left”.

When three groups G1, G2, and G4 other than the G3 group are shielded from light, the position information Dp =
Since "0010" is instructed to the ROM 46 as an address, the input information D "move to the right" is read from the ROM 46.
in can be read. Image processing such as moving the 3D polygon 10 on the reference plane to the right can be performed based on the input information Din of “move right”.

Further, when three groups G1, G2, and G3 other than the G4 group are shielded from light, the position information Dp =
Since “0001” is instructed to the ROM 46 as an address, the input information Din “jump” can be read from the ROM 46. This "jump white"
3D polygon 10 on the reference plane based on the input information Din
Image processing such as jumping can be performed.

Similarly, when the group G4 other than the groups G1, G2, and G3 is shielded from light, the position information Dp = "1110" is instructed to the ROM 46 as an address. Can be read out. 3D polygon 10 on the reference plane based on the input information Din of "speak"
Voice processing can be performed to make the user speak.

Then, with the right hand, all the groups G1 to G
4 is shielded, the position information Dp = “0000” is instructed to the ROM 46 as an address.
The input information Din of "disappearing" can be read from M46. 3 based on the input information Din of this “disappear”
Image processing can be performed to erase the D polygon 10 from its reference plane.

As described above, according to the game device 200 to which the information input device 100 according to the first embodiment is applied,
Nine light emitting diodes LED10 to LED12, LED
Since the simple commander 4 can perform an input process for executing a jog dial software program on the basis of the position information Dp based on the blinking patterns of the LEDs 21, 32, the LEDs 31, 32, and the LEDs 41, 42, the virtual character training game device Operability as an input tool can be improved. In addition, a panning CCD for recognizing a reference plane on which the 3D polygon 10 appears.
Device 23 and panning CCD device 2 for this simple commander
3 ′ can also be used.

Therefore, the virtual character breeding game device 200 and other game devices that allow a TV program character or the like to jump out to the position where the reference plane in the real space to which the observer belongs and play with the palm of the observer are used. It can be easily configured. Further, it is possible to reduce the cost of these virtual character breeding game devices and the like.

FIG. 37 is a perspective view showing a configuration example of another game apparatus 300. As shown in FIG. In this example, the 3D polygon 10 appears in the virtual space using the simple commander 4 as a reference plane. Light emitting diode LED of simple commander 4
Can be used for recognition of the reference plane, so that the 3D polygon 10 can appear on the reference plane created by the simple commander 4 in the virtual space. Therefore, even without the reference surface setting bracelet 1, the 3D polygon 10 can appear.

In this example, the simple commander 4 is not limited to the one described above, but may be one using the data glove 90. FIG. 38A shows an example of an image of the data globe 90 in the real space, and FIG. 38B is an image diagram in which the 3D polygon 10 appears on the data globe 90 in the virtual space. Also in this example, a plurality of light emitting diodes LED having different blinking patterns are attached to the data glove 90. When the reference plane is recognized by the panning CCD device 23, the 3D polygon 10 can appear on the data glove 90.

FIG. 39 is an image diagram showing a configuration example of the glove type simple commander 40. As shown in FIG. This simple commander 40
A plurality of light-emitting diodes LED are linearly attached to a glove part covering a forefinger at a position where a right thumb can reach.

When the light emitting diode LED of the forefinger of the glove portion is shielded from right to left or from left to right in order with the right thumb, the light shielding pattern like the simple commander 4 can be changed. The jog dial operation can be performed by the glove type simple commander 40. Therefore, since the image processing system can recognize the rotation and the motion determination by the simple commander 40, the operability as an input tool such as a virtual character breeding game device can be improved.

FIG. 40 shows another glove type simple commander 4.
It is an image figure showing the example of composition of 0 '. In this simple commander 40 ', a plurality of light emitting diodes LED are attached to a glove portion covering a middle finger at a position where the right thumb can reach. If the light emitting diode LED of the middle finger of the glove portion is shielded from right to left or from left to right in order with the thumb of the right hand, the light shielding pattern like the simple commander 4 described above can be changed. The jog dial operation can be performed by another simple commander 40 '.

Therefore, in the image processing system, the simple commander 40 '
Since the rotation and the action determination by the user can be recognized, the operability as an input tool of a virtual character breeding game device or the like can be improved.

(3) Second Embodiment FIG. 41 is a diagram showing a configuration example of a two-dimensional barcode 50 for setting a reference plane used in a game device according to a second embodiment. In this embodiment, four light emitting diodes LED1
A reference plane is recognized using a two-dimensional barcode 50 shown in FIG. Of course, the movement of the 3D polygon 10 appearing on the two-dimensional barcode 50 can be controlled by the information input device 100 described above.

The surface recognizing means of this example has a two-dimensional matrix code 50 and is used for recognizing a reference plane in a real space. The two-dimensional barcode 50 is at least
A black and white matrix of n rows × n columns printed in black on a white background,
It is composed of a black frame part 51 having the same thickness as the black and white matrix.
In this example, 5 × 5 pixels surrounded by a black frame portion 51 are a code region portion 52, and among the 25 pixels, 1 × 5 pixels shown in FIG.
Two pixels are painted black.

The barcode 50 is provided at a position where an image of the 3D polygon 10 is to be synthesized, for example, on the plate 11 of the bracelet 1 shown in FIG. The two-dimensional matrix code 50 is imaged by the ordinary CCD imaging device 25 instead of the panning CCD device 23. The image processing device 3 as described with reference to FIG. 31 is connected to the output stage of the imaging device 25. The image processing device 3 is provided with a calculating means 6, which performs image processing on an image pickup signal (luminance signal) output from the CCD image pickup device 25 to obtain a reference plane from the two-dimensional matrix code 50.

For example, the image processing section 44 performs preprocessing. In this process, first, the acquired image is set at 2 with an appropriate threshold.
Valued. Since the barcode portion is printed in black on a white background, the background image and the code area can be separated quite stably by the fixed threshold value. Next, labeling is performed for each connected region of black pixels. The black frame portion 51 of the two-dimensional barcode 50 is included in any of the labeled connection regions. Therefore, considering the size and aspect ratio of the circumscribed rectangle of the connection area, the code area 52
The background image (region) that is unlikely to include the symbol is removed.

Thereafter, a barcode frame is applied to each element of the connected area obtained as a result of the preprocessing. For example, a black area is searched from each side of the circumscribed rectangle toward the inside, and a point sequence of the black frame part 51 is obtained. A line segment is applied to this point sequence by the least squares method. After that, the code area section 52 given to the two-dimensional barcode 50 is recognized.

The above-mentioned reference plane is obtained by calculating a transformation matrix for projecting the four vertices of the black frame 51 to the vertices of a square. Here, the point (xi, yi, 0) on the plane of the real space is moved by a certain translation / rotational motion, and the point projected on the image coordinate system by the perspective transformation is represented by (Xi, Yi). There is a similar relationship between the expressions (1) described in the first embodiment.

Accordingly, these parameters are the position coordinates (x1, y1), (x2, y2),
(X3, y3), (x4, y4) and their corresponding four sets of position coordinates (X1, Y1), (Y2,
If (Y2), (X3, Y3) and (X4, Y4) exist, they can be obtained by solving the equation of the above-described equation (2).

The position coordinates (x1, y1) obtained here,
Regarding (x2, y2), (x3, y3), and (x4, y4), assuming that four vertices of a square whose side length is “1”, a plane connecting these four vertices is a reference plane in the real space. Become. Although the black frame portion 51 on the screen is distorted by the attitude of the CCD imaging device 25 and perspective projection, a rectangular vertex on the screen can be projected to each square vertex by an external parameter and an internal parameter. Therefore, since the cubic 53 can be created from the position coordinates of the four corners of the two-dimensional barcode 50 in the virtual space shown in FIG. 42, the 3D polygon 10 and the like can be combined with the cubic 53.

Then, in the same manner as in the first embodiment, based on an instruction from the simple commander 4 by the observer, the 3D polygon 10 is projected onto the two-dimensional bar code 50 in the virtual space, or the bar code 50 is displayed. Move the 3D polygon 10 to the left or right, jump it,
Make 3D polygon 1 from the barcode 50
0 can be deleted.

As described above, according to the game apparatus of the present embodiment, a virtual character breeding game in which a TV program character or the like jumps out onto the two-dimensional barcode 50 in the virtual space and plays with the palm of the observer. When the device or another game device is configured, the information input device 100 described above can be used as the simple commander 4.

In this embodiment, the case where the non-transmission type special glasstron 2 or the transmission type special glasstron 20 is used has been described. However, the present invention is not limited to this, and the transmission type and the non-transmission type can be switched. Of course, it is possible to use a special type of glasstron which can be used for both purposes.

The game device 200 of this embodiment is disclosed in Japanese Patent Application Laid-Open Nos. 10-123453 and 9-304727,
JP-A-9-304730, JP-A-9-21374
JP-A-8-160348, JP-A-8-94960
JP-A-7-325265, JP-A-7-27071
4 and JP-A-7-67055.

In this embodiment, the case where the two-dimensional imaging device of the interline transfer system is used for the panning CCD 23 has been described. However, the present invention is not limited to this, and the case where the two-dimensional imaging device of the frame transfer system is used is described. Even if there is, the same effect can be obtained.

In this embodiment, nine light emitting diodes LED10 to LED12, LED21, 22,
The case where the LEDs 31 and 32 and the LEDs 42 and 42 are mounted has been described. However, the present invention is not limited to this. Further, in this embodiment, the case where the input information Din is generated in units of groups has been described. However, the present invention is not limited to this, and the first and second light emitting diodes are individually shielded as lower layers of the group. Then, the amount of the input information Din may be increased to 16 or more. With this configuration, it is possible to control the display of the expression of the 3D polygon 10 more finely.

[0133]

As described above, according to the information input device of the present invention, input information corresponding to the position information of the light source having a different blinking pattern attached to a specific surface of an arbitrary imageable object is generated. The data generating means is provided.

With this configuration, when a blinking pattern of an arbitrary light source among a plurality of light sources is shielded, at least
It is possible to generate input information corresponding to position information of a light source other than the light source whose blinking pattern is shielded.

Further, according to the information input method of the present invention, when a blinking pattern of an arbitrary light source is shielded from light among a plurality of light sources having different blinking patterns attached to a specific surface of an arbitrary imageable object. In addition, at least input information corresponding to position information of a light source other than the light source that shields the blinking pattern is generated.

With this configuration, a simple commander comparable to a wireless remote controller most suitable for an input tool such as a virtual character breeding game device for playing a TV program character with a palm in a virtual space can be configured with high reproducibility.

According to the game device of the present invention, since the above-mentioned information input device is applied to the input means, the input operation is simple and the virtual character raising game device provided with a simple commander having good operability. Can be configured.

The present invention is extremely suitable for application to a virtual character breeding game device and the like.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration example of an information input device 100 as each embodiment according to the present invention.

FIG. 2 is a perspective view showing a configuration example of the simple commander 4;

FIG. 3 is a perspective view showing a configuration example of another simple commander 4;

FIG. 4 is a conceptual diagram showing an example of a group of nine light emitting diodes of a simple commander 4;

FIG. 5 is a top view showing an example of use of the simple commander 4;

FIG. 6 is an image diagram showing an example of a normal image of the simple commander 4;

FIG. 7 is a schematic diagram showing an example of calculating position coordinates of the nine light emitting diodes.

FIG. 8 is a table showing examples of light emitting diode LED numbers and their mounting position information.

FIG. 9 is a table showing an example of input information for position information Dp by the four groups.

FIG. 10 is an image diagram showing a display example of the input information.

FIG. 11 is an image diagram showing an example of movement of the 3D polygon in the x and y directions.

FIG. 12 is an image diagram showing an operation example (PT = 0, 7) of the simple commander 4 like a jog dial.

FIG. 13 shows an example of an operation like a jog dial (PT = 1).
FIG.

FIG. 14 shows an example of an operation similar to a jog dial (PT = 2).
FIG.

FIG. 15 shows an example of an operation similar to a jog dial (PT = 3).
FIG.

FIG. 16 shows an example of a jog dial operation (PT = 4).
FIG.

FIG. 17 shows an example of an operation like a jog dial (PT = 5).
FIG.

FIG. 18 shows an example of a jog dial operation (PT = 6).
FIG.

FIG. 19 is an image diagram showing an input example of an “operation decision” instruction of the simple commanders 4 and 4 ′.

FIG. 20 is an image diagram showing another jog dial-like operation example (PT = 1) of the simple commander 4 ′.

FIG. 21 shows an example of an operation like a jog dial (PT = 2).
FIG.

FIG. 22 shows an example of an operation similar to a jog dial (PT = 3).
FIG.

FIG. 23 shows an example of an operation similar to a jog dial (PT = 4).
FIG.

FIG. 24 shows an example of an operation similar to a jog dial (PT = 5).
FIG.

FIG. 25 is a perspective view showing a configuration example of a game device 200 to which the information input device according to the present invention is applied.

FIG. 26 is a perspective view showing a configuration example of a reference surface setting bracelet 1 used in the first embodiment.

FIG. 27 is a conceptual diagram illustrating a configuration example of a special glasstron 2 used in the game apparatus 200 as viewed from the front.

FIG. 28 is a conceptual diagram illustrating a configuration example of another special glasstron 20 used in the game apparatus 200, as viewed from the front.

FIG. 29 is a plan view showing an example of the internal configuration of the panning CCD device 23 of the special glasstron 2.

FIG. 30 is a cross-sectional view showing a configuration example of an optical system of the panning CCD device 23.

FIG. 31 is a diagram showing an example of a circuit block of the game device 200.

FIG. 32 is an image diagram showing an example of a normal image of the plate portion 11 forming the reference plane.

FIG. 33 is a schematic diagram showing an example of calculating position coordinates of the reference plane.

FIG. 34 is a flowchart of a main routine showing an operation example (No. 1) of the game device 200.

FIG. 35 is a flowchart of a subroutine showing an operation example (2) of the game apparatus 200.

FIG. 36A is an example of a real image of the plate unit 11 in the real space, and FIG. 36B is an image diagram showing an example of combining 3D polygons on a reference plane in the virtual space.

FIG. 37 shows another game device 30 to which the information input device is applied.
FIG. 2 is a perspective view showing a configuration example of a No. 0.

38A is an example of a real image when the data globe 90 is applied, and FIG. 38B is an image diagram showing an example of the combination of 3D polygons on the data globe 90 in a virtual space.

FIG. 39 is an image diagram showing a configuration example (forefinger) of a glove type simple commander 40.

FIG. 40 is an image diagram showing a configuration example (middle finger) of another glove-type simple commander 40 ′.

FIG. 41 shows a reference plane setting 2 used in the second embodiment.
FIG. 4 is a plan view illustrating an example of a dimensional barcode 50.

FIG. 42 is a perspective view showing a synthesis example of the 3D polygon 10 on the two-dimensional barcode 50.

[Explanation of symbols]

1. Bracelet for setting reference surface (surface recognition means),
2, 20 special glasstron, 3 image processing device, 4, 4'40, 40 'simple commander, 6 ...
.Calculating means, 7 ... data generating means, 10 ... 3D
Polygon, 16 storage means, 23, 23 'panning CCD device (imaging means), 24 display means,
25: CCD imaging device, 26: L for right eye display
CD (first image display element), 27 LCD for left eye display (second image display element), 32 vertical transfer unit (charge transfer unit), 33 horizontal transfer unit 46 ... R
OM, 50: two-dimensional barcode, 90: data glove, 100: information input device, 200, 300
... Game devices

Claims (13)

[Claims] 1. A plurality of light sources having different blinking patterns attached to a specific surface of an arbitrary object that can be photographed, an imaging unit configured to image the light sources having different blinking patterns in a predetermined imaging direction, and Computing means for performing image processing on the luminance information of the light source by the imaging means to obtain position information of each of the light sources, and data generating means for generating input information according to the positional information of the light source by the computing means An information input device characterized by the above-mentioned. 2. The object that can be imaged has a plate part of a predetermined shape on which a plurality of light sources having different blinking patterns are attached to a specific surface, and a ring part joined to the plate part. The information input device according to claim 1, wherein 3. The light emitting device according to claim 2, wherein the plate portion has a rhombic shape, and the light sources having different blinking patterns are arranged in a cross shape along a diagonal line of the rhombic plate portion. Information input device. 4. The information input device according to claim 3, wherein the light sources having different blink patterns arranged in a cross shape are controlled to blink in four groups. 5. When the light source and the data generating means are provided, the data generating means, when an arbitrary blinking pattern of the light source is shielded, at least a light source other than the light source which shields the blinking pattern. 2. The information input device according to claim 1, wherein input information is generated according to the position information of the light source. 6. A case where the light source and the data generating unit are provided, wherein the data generating unit is provided with a storage unit, and the storage unit stores input information associated with the position information of the light source. The information input device according to claim 1, wherein the information is stored. 7. A two-dimensional imaging device having a plurality of photoelectric conversion elements constituting each pixel is used as said imaging means, and when transferring signal charges obtained from said photoelectric conversion elements in a predetermined direction. 2. The information input device according to claim 1, wherein the signal charge is read from the photoelectric conversion element at least a plurality of times during the same field period. 8. The method according to claim 1, wherein the imaging unit and the calculation unit are provided, wherein the calculation unit determines a blinking pattern of a luminance signal by the imaging unit.
2. The image processing apparatus according to claim 1, wherein the image data is converted into a spatial arrangement pattern forming an XY plane including a plurality of bright points, and the arrangement pattern is scanned to obtain at least a plurality of position coordinates. Information input device. 9. The panning imaging method according to claim 1, wherein the panning imaging direction is a Y-axis on an arrangement pattern forming an XY plane including a plurality of bright points, and an axis orthogonal to the Y-axis is an X-axis. The luminance signal values are added in the directions and plotted on the X axis. The position where the luminance signal value plotted on the X axis is the maximum is detected to determine a plurality of X coordinate values. When scanning in the Y-axis direction in (1), among the plurality of bright spots arranged in the panning imaging direction, the position of the bright spot that first emits light is obtained as a Y coordinate value corresponding to the X coordinate value. Item 2. The information input device according to Item 1. 10. A plurality of light sources having different blinking patterns are attached to a specific surface of an arbitrary object that can be imaged, and images are taken so that the light sources having different blinking patterns flow in a predetermined imaging direction. Image processing is performed on the luminance information to obtain position information of each of the light sources. When an arbitrary blinking pattern of the light source is shielded, at least input information corresponding to position information of a light source other than the light source that shields the blinking pattern An information input method characterized by generating 11. An apparatus for stereoscopically combining an arbitrary character image with an external world image to which an observer belongs, a plane recognizing means for recognizing an arbitrary reference plane in a real space to which the observer belongs, and the plane A synthesizing unit for synthesizing the character image on the reference plane of the virtual space recognized by the recognizing unit; and at least an input unit for instructing input information for changing the character image, the input unit comprising: A plurality of light sources having different blinking patterns attached to a specific surface of any possible object; an imager for imaging the light sources having different blinking patterns in a predetermined imaging direction; and An arithmetic unit for performing image processing on the luminance information to obtain position information of each of the light sources; Game apparatus characterized by having means. 12. An image capturing means for capturing an external image to which an observer belongs, and a first image for displaying one of a stereo image synthesized with the external image by the image capturing means and an image of a virtual body prepared in advance. And a second image display element for displaying the other of the stereo images. The head mounted display is mounted on a face or a head of an observer, and the first 12. The game device according to claim 11, wherein the stereo image by the image display element and the stereo image by the second image display element are superimposed and guided to an eyeball of an observer. 13. A liquid crystal shutter for opening and closing incident light for capturing an external image to which an observer belongs, an image display element for displaying an image of a virtual body to be synthesized with the external image, An image of the virtual body by the image display element, and a head mounted display having an optical unit that guides an external image to which the observer who has passed through the liquid crystal shutter belongs to the observer's eyeball, wherein the head mount display includes: When mounted on the face or head of the observer and the liquid crystal shutter is opened, the image of the virtual body by the image display element is superimposed on the external image in real space to which the observer who has passed through the liquid crystal shutter belongs. The game device according to claim 11, wherein the game device is guided to an eyeball of an observer.