JP2008225985A - Image recognition system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image recognition system, capable of performing processing reflecting a user's motion in detail by three-dimensionally analyzing the user's motion without needing to install an emitter to the user's hand, foot or the like. <P>SOLUTION: The shape or position of a detection object is three-dimensionally analyzed to perform various processings by acquiring three-dimensional shape information or three-dimensional position information of a detection object is acquired in a three-dimensional coordinate system of an imaging space PS of a detection unit 16. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image recognition system.

Conventionally, a user's image is taken after attaching a light emitter to the system user's hand or foot, etc., and a user's movement is detected based on a change in the position of the light emitter in the image. There is known an image recognition system that displays an image according to the movement of the camera.
JP 2000-33184 A

  However, in the conventional image recognition system, it is necessary to attach the light emitter to the user's hand or foot, and the change in the position of the light emitter in the image is analyzed two-dimensionally. An image corresponding to the user's movement or the like is generated simply and in detail and cannot be used for communication or a game.

  The present invention has been made in view of the problems as described above. The object of the present invention is to detect a detection object such as a user in a simple and three-dimensional manner to generate an image, for example, a keyboard. It is an object of the present invention to provide an image recognition system that can be used for net game communication on a game-dedicated machine that does not have a problem, or that can simplify the operation or expand the range of communication expression even with a keyboard. Another object of the present invention is to provide an image recognition system capable of performing operation input by a controller that does not require an actual circuit, power supply, cable, or the like. Another object of the present invention is to provide an image recognition system capable of simplifying operation input. Another object of the present invention is to provide an image recognition system capable of generating an image reflecting what the user actually has.

(1) The present invention
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
An object generation unit that generates object shape information that defines the shape of a three-dimensional object set in a given three-dimensional object space based on the detected three-dimensional shape information;
A communication control unit that transmits and receives the generated object shape information to and from an external device;
An object space setting unit for setting the received object shape information in the three-dimensional object space;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
It is related with the image recognition system characterized by including.

  The present invention also relates to a program for causing a computer to function as each unit and an information storage medium storing the program.

  In the present invention, the “detected three-dimensional shape information” can be, for example, coordinate information of a coordinate system using the center point of the detection unit as a reference point, and not only defines the shape of the detection target but also the detection unit. The information for defining the position and orientation of the detection object in the detection direction.

  In this invention, a detection part detects the reflected wave of the output wave which the output part output, and the detection three-dimensional shape information of a detection target object is acquired based on the detected reflected wave. Then, object shape information is generated based on the acquired detected three-dimensional shape information, and the object shape information is transmitted to and received from an external device. Thus, according to the present invention, a three-dimensional object based on the detected three-dimensional shape information acquired in the terminal device of the user (person) is displayed on the terminal device of another person, or the detected three-dimensional object acquired in the terminal device of the other person. Communication can be performed while a three-dimensional object based on the shape information is displayed on its own terminal device. For example, in the terminal device of another person, by acquiring the detected three-dimensional shape information for which the terminal device itself is a detection object, generating the object shape information corresponding to itself, and transmitting this to the terminal device of the other person A three-dimensional object (character) corresponding to itself can be displayed.

(2) The present invention also provides:
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
An object shape information storage unit for storing a plurality of types of object shape information defining the shapes of a plurality of types of three-dimensional objects set in a given three-dimensional object space;
An object shape information selection unit that selects the object shape information corresponding to the detected three-dimensional shape information based on the detected three-dimensional shape information;
A communication control unit for transmitting and receiving the selected object shape information to and from an external device;
An object space setting unit for setting the received object shape information in the three-dimensional object space;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
It is related with the image recognition system characterized by including.

  The present invention also relates to a program for causing a computer to function as each unit and an information storage medium storing the program.

  Thus, according to the present invention, the 3D object selected according to the detected 3D shape information acquired in the terminal device of the user (person) is displayed on the terminal device of the other person, or acquired by the terminal device of the other person. It is possible to perform communication while displaying the three-dimensional object selected according to the detected three-dimensional shape information on its own terminal device. For example, in the terminal device of another person, by acquiring the detected three-dimensional shape information for which the terminal device itself is a detection object, selecting the object shape information corresponding to itself, and transmitting the information to the terminal device of the other person A three-dimensional object corresponding to itself can be displayed.

(3) In the image recognition system according to the present invention,
An object shape information correction unit that corrects the object shape information selected by the object shape information selection unit based on the detected three-dimensional shape information;
The communication control unit
The corrected object shape information corrected by the object shape information correction unit may be transmitted / received to / from an external device.

  According to the present invention, for example, when object shape information corresponding to its own detected three-dimensional shape information is selected, the selected object shape information can be corrected so as to be closer to its own detected three-dimensional shape information. Thus, according to the present invention, it is possible to display a three-dimensional object closer to the detection target while suppressing the data volume of the object shape information stored in advance.

(4) In the image recognition system according to the present invention,
A texture mapping unit may be further included that maps a given texture to a part of the generated object shape information and sets the object shape information in the three-dimensional object space.

  According to the present invention, for example, when object shape information corresponding to oneself is generated, when a three-dimensional object (character) corresponding to oneself is displayed on another person's terminal device, for example, a part corresponding to a part of the face By mapping a given texture, it is possible to maintain anonymity while giving uniqueness to a three-dimensional object displayed on another person's terminal device.

(5) The present invention also provides:
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
A reference information storage unit that stores reference three-dimensional shape information that defines a predetermined three-dimensional shape;
A comparison unit that compares the reference three-dimensional shape information with the detected three-dimensional shape information;
A command signal output unit that outputs a corresponding command signal based on the comparison result of the comparison unit;
An output information storage unit that stores at least one of image information and sound information corresponding to the command signal;
And an output control unit that outputs at least one of the corresponding image information and sound information based on the output command signal.

  The present invention also relates to a program for causing a computer to function as each unit and an information storage medium storing the program.

  In the present invention, the acquired detected three-dimensional shape information is compared with the reference three-dimensional shape information, and a corresponding command signal is output based on the comparison result. As a result, according to the present invention, the shape formed by the hand at the position overlapping the user's body with respect to the detection unit, such as sign language, is analyzed three-dimensionally, and the shape according to the shape of the hand is determined. A command signal can be output. Thus, according to the present invention, for example, the shape of the hand in the sign language can be recognized by the image recognition system, and for example, the character information and voice information corresponding to the command signal corresponding to the shape of the hand is output to translate the sign language. It can also be configured as follows.

(6) In the image recognition system according to the present invention,
The three-dimensional information acquisition unit
Obtaining detection three-dimensional motion information defining a change in a predetermined period of the three-dimensional shape of the detection object reflected by the output wave based on the reflected wave;
The reference information storage unit is
Storing reference three-dimensional motion information defining a change in a predetermined period of a predetermined three-dimensional shape;
The comparison unit is
The reference three-dimensional motion information and the detected three-dimensional motion information may be compared.

  In the present invention, the acquired detected three-dimensional motion information is compared with the reference three-dimensional shape motion, and a corresponding command signal is output based on the comparison result. Thus, according to the present invention, for example, not only the shape of the hand but also the movement of the hand can be analyzed three-dimensionally, and a command signal corresponding to the three-dimensional movement of the hand can be output. Thus, according to the present invention, it is possible to realize an image recognition system that outputs various command signals according to a three-dimensional movement instead of a two-dimensional movement of a detection target.

(7) In the image recognition system according to the present invention,
A communication control unit that transmits and receives the output command signal to and from an external device;
The output control unit is
Based on the received command signal, at least one of the corresponding image information and sound information may be output.

  In the present invention, for example, based on a command signal corresponding to the detected three-dimensional shape information acquired in the terminal device of the user, corresponding character information or voice information is output in the terminal device of another person, or is acquired in the terminal device of another person. Based on the command signal corresponding to the detected three-dimensional shape information, it is possible to perform communication while outputting the corresponding character information and voice information on the terminal device of the user.

(8) The present invention also provides:
A movement / motion processing unit that controls at least one of movement and motion of a given object based on an operation signal;
A drawing processing unit for drawing an image in which the given object performs at least one of movement and movement;
An image recognition system including:
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
Based on the reflected wave, at least one of detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave and detection three-dimensional position information that defines the three-dimensional position of the detection object. A three-dimensional information acquisition unit to acquire;
A reference information storage unit that stores reference three-dimensional shape information that defines a predetermined three-dimensional shape;
A comparison unit that compares the reference three-dimensional shape information with the detected three-dimensional shape information;
An operation detection target setting unit that sets the detection target corresponding to the detection three-dimensional shape information as an operation detection target based on the comparison result of the comparison unit;
An operation signal output unit that outputs the operation signal based on a change in at least one of the detected three-dimensional shape information and the detected three-dimensional position information acquired for the set operation detection target;
In addition, the present invention relates to an image recognition system.

  The present invention also relates to a program for causing a computer to function as each unit and an information storage medium storing the program.

  In the present invention, if the detection target has a predetermined three-dimensional shape, the detection target is set as the operation detection target. When the shape or position of the detection target changes, this is detected as an operation input, and a corresponding operation signal is output. As described above, according to the present invention, it is possible to construct an image recognition system that uses a detection unit as an input interface for operating movement / movement of a given object.

(9) In the image recognition system according to the present invention,
An operation algorithm storage unit that stores an operation algorithm corresponding to each reference 3D shape information of the plurality of reference 3D shape information stored in the reference information storage unit;
An operation algorithm setting unit that sets an operation algorithm according to the reference three-dimensional shape information corresponding to the detected three-dimensional shape information based on the comparison result of the comparison unit;
Further including
The operation signal output unit is
An operation signal corresponding to the set operation algorithm is output based on a change in at least one of the detected three-dimensional shape information and the detected three-dimensional position information acquired for the set operation detection object. May be.

  In the present invention, a plurality of operation algorithms are prepared according to the three-dimensional shape of the detection object. For example, when an annular detection object is detected, this is set as an operation detection object called a car handle. The operation algorithm corresponding to the steering wheel of the car is set. Then, when the user rotates the operation detection target so as to rotate the handle, an image for changing the direction of the car object, for example, is drawn according to the rotation amount of the operation detection target. For example, when a rod-shaped detection target is detected, this is set as an operation detection target called an airplane control stick, and an operation algorithm corresponding to the airplane control stick is set. When the user performs an operation of tilting the operation detection target so as to operate the control stick, for example, an image for changing the direction of the airplane object is drawn according to the tilt amount of the operation detection target. As described above, according to the present invention, it is possible to realize an input interface capable of performing various operation inputs according to the three-dimensional shape of the operation detection target.

(10) The present invention also provides:
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
A reference information storage unit that stores reference three-dimensional shape information that defines a predetermined three-dimensional shape;
A comparison unit that compares the reference three-dimensional shape information with the detected three-dimensional shape information;
An object space setting unit that sets given object shape information in the three-dimensional object space based on a comparison result of the comparison unit;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
It is related with the image recognition system characterized by including.

  The present invention also relates to a program for causing a computer to function as each unit and an information storage medium storing the program.

  In the present invention, the “reference three-dimensional shape information” may be three-dimensional shape information for comparing and determining shape information, or may be object shape information itself set in the object space.

  In the present invention, the acquired detected three-dimensional shape information is compared with the reference three-dimensional shape information, and the object shape information is set in the three-dimensional object space based on the comparison result. For example, if the user has a detection target in the shape of a tennis racket, an image in which the character object has a tennis racket is drawn. Here, according to the present invention, for example, even if the user holds the tennis racket at a position overlapping the user's body with respect to the detection unit, the shape of the tennis racket is analyzed three-dimensionally and the user holds it. You can recognize what you are. Therefore, according to the present invention, it is possible to acquire the detected three-dimensional shape information of the detection target regardless of the arrangement state of the detection target and set the corresponding object shape information in the three-dimensional object space.

(11) In the image recognition system according to the present invention,
The object shape information storage unit
Stores multiple types of object shape information,
The object space setting unit
Based on the comparison result of the comparison unit, the object shape information corresponding to the detected three-dimensional shape information among the plurality of types of object shape information may be set in the three-dimensional object space.

  According to the present invention, for example, if the user has a detection object in the shape of a sword, the character object can draw an image having a sword item and have the detection object in the shape of a shield. For example, an image in which a character object has a shield item can be drawn. As described above, according to the present invention, images of various three-dimensional objects can be drawn according to the three-dimensional shape of the detection target.

(12) In the image recognition system according to the present invention,
The object space setting unit
The object shape information set in the three-dimensional object space may be changed based on the occurrence of a given event.

  According to the present invention, for example, when a user has a detection object in the shape of a sword and a character object is drawing an image having a sword item, based on the occurrence of a given event You can draw an image that breaks the sword item.

(13) The present invention also provides:
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional position information that defines a three-dimensional shape of the detection object based on the reflected wave;
An object space setting unit that sets given object shape information in the three-dimensional object space based on the acquired detected three-dimensional shape information;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
It is related with the image recognition system characterized by including.

  The present invention also relates to a program for causing a computer to function as each unit and an information storage medium storing the program.

  In this invention, a detection part detects the reflected wave of the output wave which the output part output, and the detection three-dimensional shape information of a detection target object is acquired based on the detected reflected wave. Based on the acquired detected three-dimensional shape information, an image in which the shape, position, and orientation of a given object change according to changes in the shape, position, and orientation of the detection target in the real world can be drawn. . For example, when a given object is thrown toward the detection unit, an image in which the corresponding object moves (throws) with the movement of the thrown object can be drawn.

(14) In the image recognition system according to the present invention,
You may make it further contain the determination part which determines the overlap in the said three-dimensional object space with the said object shape information set to the said three-dimensional object space, and other object shape information.

  According to the present invention, for example, tennis that an operator has can be obtained by performing hit determination between an object corresponding to a detection target possessed by the user, an object corresponding to the detection target thrown by the user, and another object. A tennis game in which a ball is hit by a tennis racket object corresponding to the racket can be configured, or a snowball game in which a snowball battle is performed by a snowball object corresponding to a paper scrap thrown by the operator can be configured.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. System Overview FIG. 1 is a schematic external view of a game system 10 to which the image recognition system of this embodiment is applied. The game system 10 according to the present embodiment includes a display unit 14 that displays a game image such as a target object TO on the display screen 12, and a three-dimensional shape of a detection target that is installed on the upper surface of the display unit 14 and in the front direction of the display unit 14. A detection unit 16 for detecting information and three-dimensional position information, and a game machine 18 that performs various processes using the three-dimensional shape information of the detection target are included. The detection unit 16 is installed at the left end of the detection unit 16, and outputs an output wave of a predetermined wavelength such as infrared light toward the front direction of the display unit 14 (the direction of the player P). A detection unit 22 that is installed at the right end of the detection unit 16 and detects a reflected wave of the output wave is provided. In particular, in the present embodiment, the output unit 20 is configured by an infrared LED that outputs infrared light, and the detection unit 22 is configured by an infrared light image sensor. The detection unit 22 receives infrared light incident from the front direction of the display unit 14 (reflected light of infrared light output from the output unit 20) on the light receiving surface, and acquires (captures) this as an image. .

  Further, the detection unit 16 is provided with a three-dimensional information acquisition unit 24 composed of an ASIC, MCU, or the like. The three-dimensional information acquisition unit 24 obtains an image of an imaging target (detection target) such as the player P based on a phase shift of the reflected wave at each light receiving position on the light receiving surface of the detection unit 22 that receives the reflected wave. And 3D coordinate system information (detected 3D shape information, detected 3D position information) of the imaging space including the imaging object. Then, the game machine 18 recognizes the posture and movement of the player P based on the three-dimensional coordinate information of the player P that is the imaging object, and displays a corresponding image on the display screen 12 or outputs a sound. . Also, the 3D position information and information based on the 3D position information are transmitted to the game machine 18 of the other person via a network such as the Internet, and an image corresponding to the user is displayed on the display screen 12 of the display unit 14 of the other person. .

  In the present embodiment, a filter that transmits only light in a wavelength band corresponding to infrared light (light having the same wavelength as the light from the output unit 20) is provided on the front side of the detection unit 22, and the detection unit 22 receives light. The light to be emitted is light of a predetermined wavelength band. Note that an image sensor having a light receiving sensitivity characteristic in a predetermined wavelength band including infrared light (light having a predetermined wavelength) may be used as the detection unit 22 without providing a filter. Further, a controller for performing various operation inputs may be connected to the game machine 18.

2. Next, the principle by which the detection unit 16 of the present embodiment acquires an image of an imaging target such as the player P as three-dimensional coordinate system information (detected three-dimensional shape information, detected three-dimensional position information). explain. In the present embodiment, the three-dimensional information acquisition unit 24 acquires distance information as information that defines the relative positional relationship between the detection unit 22 and the player P that is the imaging target.

  FIG. 2 is a plan view of the display unit 14, the detection unit 16 disposed on the upper surface of the display unit 14, and the player P, which is an imaging target, as seen from the side. As shown in FIG. 2, in the present embodiment, when the output unit 20 of the detection unit 16 outputs infrared light DL in the detection direction of the detection unit 16 (the front direction of the display unit 14), the detection unit 22 of the detection unit 16. However, the reflected light RL from the detection direction of the detection unit 16 is detected. Here, the reflected light RL is the reflected light of the direct light output from the output unit 20, but is relatively different from the reflected light RL1 reflected by the right hand RH of the player P whose distance from the detection unit 22 is relatively short. A phase difference occurs between the reflected light RL <b> 2 reflected by the head HD of the far player P and reaching the detection unit 22. Therefore, the detection unit 22 of the present embodiment detects (images) the phase difference of the reflected light RL received on the light receiving surface of the detection unit 22 for each light receiving element (pixel) arranged on the light receiving surface. Then, based on the phase difference for each pixel, the three-dimensional information acquisition unit 24 calculates the distance information between the imaging object and the detection unit 22 existing in the detection direction (imaging space, in the angle of view θ) of the detection unit 22 as a pixel. Get every.

  FIG. 3 is a diagram for explaining distance information for each pixel acquired by the three-dimensional information acquisition unit 24. In the detection unit 22 of the present embodiment, since the light receiving elements are arranged in a matrix on the rectangular light receiving surface, the three-dimensional information acquisition unit 24 is arranged in a matrix in the rectangular imaging area PA shown in FIG. The distance information between the imaging object PO (player P) and the detection unit 22 is acquired for each pixel. Therefore, as shown in FIG. 3, the pixel corresponding to the right hand RH of the player P, the pixel corresponding to the head HD, the pixel corresponding to the body BD, the pixel corresponding to the left hand LH, and the background wall WL For the corresponding pixel and the pixel corresponding to the floor GS, distance information from the detection unit 22 is stored for each pixel.

  FIG. 4 is a diagram for explaining the three-dimensional coordinate system information in the imaging space PS (within the angle of view θ) of the detection unit 22. In the three-dimensional information acquisition unit 24 of the present embodiment, as shown in FIG. 4, in the three-dimensional coordinate system in which the center of gravity WP of the light receiving surface of the detection unit 22 is the origin O and the normal N of the light receiving surface is the Z axis. The three-dimensional coordinate information of the surface seen from the light receiving surface of the imaging object PO in the imaging space PS is acquired. That is, a three-dimensional coordinate value group for each pixel corresponding to the surface of the imaging object PO that can be seen from the detection unit 22 (the surface on which the detected reflected wave is reflected) is stored in the memory area. Thus, in the detection unit 16 of the present embodiment, in the three-dimensional coordinate system of the imaging space PS, three-dimensional shape information that defines the three-dimensional shape of the imaging object PO and three-dimensional that defines the three-dimensional position of the imaging object PO. Position information can be acquired.

  Thus, according to the present embodiment, the shape formed by the hand at the position overlapping the body of the player P with respect to the detection unit 22 as in, for example, sign language is also analyzed three-dimensionally. A command signal corresponding to the shape can be output. Thus, according to the present embodiment, for example, the hand shape in sign language can be recognized by the image recognition system 10. For example, character information or voice information corresponding to a command signal corresponding to the hand shape is output to It can also be configured to translate.

  In the present embodiment, the detection unit 22 receives (refreshes) the reflected light RL every 1/30 seconds (with 1/30 seconds as one frame), and the three-dimensional information acquisition unit 24 performs the three-dimensional imaging region SA. Update information.

3. Functional Block Next, the configuration of the indicated position calculation system (game system) in the present embodiment will be described with reference to FIG. FIG. 7 is an example of a functional block diagram of the indicated position calculation system in the present embodiment. Note that the pointing position calculation system of this embodiment may have a configuration in which some of the components (each unit) in FIG. 7 are omitted.

  The detection unit 16 includes an output unit 20, a detection unit 22, and a three-dimensional information acquisition unit 24.

  The output unit 20 outputs a predetermined output wave in the detection direction of the detection unit 16. Specifically, it can be realized by a device that outputs an output wave having a predetermined wavelength, such as a light emitting unit such as an infrared LED or an ultrasonic wave generating unit.

  The detection unit 22 has a predetermined positional relationship with the output unit 20 and detects a reflected wave of the output wave output by the output unit 20. Specifically, it can be realized by a device that detects a reflected wave having a predetermined wavelength, such as an infrared image sensor.

  The three-dimensional information acquisition unit 24 can be realized by hardware such as various processors (CPU, MPU, DSP, etc.), ASIC (gate array, etc.), and a program, and defines the three-dimensional shape of the detection object reflected by the output wave. The detected three-dimensional shape information is acquired based on the reflected wave. The three-dimensional information acquisition unit 24 acquires detection three-dimensional position information that defines the three-dimensional position of the detection target based on the reflected wave. The three-dimensional information acquisition unit 24 acquires detection three-dimensional motion information that defines a change in the three-dimensional shape of the detection target object for a predetermined period based on the reflected wave. The three-dimensional information acquisition unit 24 may be realized as a function of the processing unit 100 without being provided in the detection unit 16.

  The operation unit 24 is used by the player to input operation data, and the function can be realized by a lever, a button, a steering, a microphone, a touch panel display, or a housing.

  The storage unit 170 serves as a work area for the processing unit 100, the communication unit 196, and the like, and its function can be realized by a RAM (VRAM) or the like. The storage unit 170 according to the present embodiment includes a main storage unit 171 used as a work area, a drawing buffer 172 that stores final display images and the like, and model data of a plurality of types of objects (three-dimensional object data). Object data storage unit 173 storing object shape information defining the shape), texture storage unit 174 storing texture for object data, and Z buffer 175 storing Z values during object image generation processing And including. Note that some of these may be omitted.

  The information storage medium 180 (computer-readable medium) stores programs, data, and the like, and functions as an optical disk (CD, DVD), magneto-optical disk (MO), magnetic disk, hard disk, and magnetic tape. Alternatively, it can be realized by a memory (ROM).

  The information storage medium 180 stores a program (data) for the processing unit 100 to perform various processes of the present embodiment. That is, the information recording medium 180 stores a program for causing a computer to function as each unit of the present embodiment (a program for causing a computer to execute processing of each unit).

  The display unit 14 outputs an image generated according to the present embodiment, and its function can be realized by a CRT, LCD, touch panel display, or the like.

  The sound output unit 192 outputs the sound generated by the present embodiment, and its function can be realized by a speaker, headphones, or the like.

  The portable information storage device 194 stores player personal data, game save data, and the like. Examples of the portable information storage device 194 include a memory card and a portable game device.

  The communication unit 196 performs various controls for communicating with the outside (for example, a host device or other image generation system), and functions thereof are hardware such as various processors or communication ASICs, It can be realized by a program.

  Note that a program (data) for causing a computer to function as each unit of the present embodiment is distributed from the information storage medium of the host device (server) to the information storage medium 180 (storage unit 170) via the network and communication unit 196. May be. Use of the information storage medium of such a host device (server) can also be included in the scope of the present invention.

  The processing unit 100 (processor) performs processing such as game processing, image generation processing, or sound generation processing based on operation data from the indicator 16, a program, and the like. Here, the game process includes a process for starting a game when a game start condition is satisfied, a process for advancing the game, a process for placing an object such as a character or a map, a process for displaying an object, and a game result calculation. Or a process for ending the game when a game end condition is satisfied. The processing unit 100 performs various processes using the storage unit 170 as a work area. The functions of the processing unit 100 can be realized by various processors, hardware such as an ASIC, and programs.

  In particular, the processing unit 100 of this embodiment includes a comparison unit 101, a command signal output unit 102, an output control unit 103, an operation detection target setting unit 104, an operation algorithm setting unit 105, an operation signal output unit 106, and an object generation unit 107. , Object space setting unit 108, movement / motion processing unit 109, determination unit 110, texture mapping unit 111, drawing processing unit 120, sound generation unit 130, and communication control unit 140. Note that some of these may be omitted.

  The comparison unit 101 includes reference three-dimensional shape information (comparison shape information, template information, and object shape information) that prescribes a three-dimensional shape stored in advance in the reference information storage unit 176, and the detection target detected by the detection unit 22. The detected three-dimensional shape information is compared. Specifically, the player's posture and shape, such as the yoga pose and sign language hand shape that the player takes, items such as swords and shields that can be used in the game, operational objects such as tennis rackets and baseball bats, vehicles A bounding volume corresponding to an operation detection target such as a steering wheel or a control stick of an airplane is stored in the reference information storage unit 176 to determine which bounding volume the detected three-dimensional shape information matches. .

  The comparison unit 101 also includes reference three-dimensional motion information that preliminarily stores changes in a predetermined three-dimensional shape stored in the reference information storage unit 176 and a three-dimensional shape of the detection target detected by the detection unit 22. The detected three-dimensional motion information that defines a change in a predetermined period is compared. Specifically, for example, an animation corresponding to a player's action such as a bowing action or an object throwing action is stored in the reference information storage unit 176, and whether or not the detected three-dimensional action information matches any animation. Determine whether.

  The command signal output unit 102 outputs a corresponding command signal based on the comparison result of the comparison unit 101. Specifically, the command signal output unit 102 outputs, for example, a command signal corresponding to the shape of the hand in sign language, or a command signal for performing evaluation according to the posture of the player.

  Based on the output command signal, the output control unit 103 refers to the output information storage unit 177 that stores image information and sound information corresponding to the command signal, and outputs at least one of the corresponding image information and sound information. . Specifically, character information and voice information corresponding to the shape of the hand in the sign language are output, the sign language is translated, and character information and voice information for evaluating the posture of the player are output.

  The output control unit 103 performs display control of a display object (object) displayed and output on the display unit 14 as display control processing. Specifically, a display object (game character, background, target, car, ball, item, building, tree, pillar, wall, map) is generated, the display position of the display object is indicated, or the display object disappears. Display control such as That is, display control is performed such as registering the generated display object in the display object list, transferring the display object list to the drawing processing unit 120, and deleting the disappeared display object from the display object list.

  The operation detection target setting unit 104 sets a detection target corresponding to the detected three-dimensional shape information as an operation detection target based on the comparison result of the comparison unit 101. Specifically, the operation detection target setting unit 104 sets the detection target as the operation detection target if the detection target has a predetermined three-dimensional shape such as a car handle or an airplane control stick. Set.

  The operation algorithm setting unit 105 sets an operation algorithm according to the reference three-dimensional shape information corresponding to the detected three-dimensional shape information based on the comparison result of the comparison unit 101. Specifically, the operation algorithm setting unit 105 stores an operation algorithm storage that stores an operation algorithm corresponding to each reference 3D shape information of a plurality of reference 3D shape information (object shape information) stored in the reference information storage unit 176. With reference to the unit 178, for example, when an annular detection object is detected, this is handled as a car handle, and an operation algorithm corresponding to the car handle is set. For example, when a rod-shaped detection target is detected, this is handled as an airplane control stick, and an operation algorithm corresponding to the airplane control stick is set.

  The operation signal output unit 106 outputs an operation signal corresponding to the set operation algorithm based on at least one change in the detected three-dimensional shape information and the detected three-dimensional position information acquired for the set operation detection target. Output. Specifically, when the operation algorithm corresponding to the steering wheel of the car is set, the operation signal output unit 106 rotates the operation detection object so that the player rotates the steering wheel. For example, an operation signal for changing the direction of the car object according to the amount of rotation is output. Further, for example, when an operation algorithm corresponding to the control stick of an airplane is set, if the player performs an operation of tilting the operation detection target so as to operate the control stick, the tilt amount of the operation detection target is set. In response, for example, an operation signal for changing the direction of the airplane object is output.

  The object generation unit 107 generates object shape information that defines the shape of a three-dimensional object set in a given three-dimensional object space based on the detected three-dimensional shape information. Specifically, the object generation unit 107 generates a character object corresponding to itself as a detection target, or generates various objects using various objects around us. A room as a background can also be generated as an object as it is.

  Further, the object shape information selection unit 1071 is adopted, and object shape information corresponding to the detected three-dimensional shape information is obtained from object data corresponding to the detected three-dimensional shape information from a plurality of types of object data stored in the object data storage unit 173. You may make it select. In this case, the object shape information correction unit 1072 may be further employed to correct the object shape information selected by the object shape information selection unit based on the detected three-dimensional shape information.

  The object space setting unit 108 includes various objects (such as character objects generated in the present embodiment, character objects stored in advance, buildings, trees, pillars, walls, maps (terrain) and other display objects ( A process of placing and setting objects (objects composed of primitive surfaces such as polygons, free-form surfaces, or subdivision surfaces) in the object space is performed. That is, the object space setting unit 108 determines the position and rotation angle (synonymous with direction and direction) of the object (model object) in the world coordinate system, and sets the rotation angle (X, Y, Z) to the position (X, Y, Z). , Rotation angle around Y, Z axis).

  In particular, in the present embodiment, the object space setting unit 108 may change the object shape information set in the three-dimensional object space based on the occurrence of a given event. For example, when a user has a sword-shaped detection target and a character object is drawing an image with a sword item, the sword item will break based on the occurrence of a given event. An image can be drawn. At this time, the object data of the sword item that is not broken and the object data that is broken may be prepared, and the object data set in the object space may be changed based on the occurrence of an event. Then, a destruction simulation may be performed according to the damaged part of the object data of the sword item.

  The movement / motion processing unit 109 controls at least one of movement and motion of a given object based on the operation signal. Specifically, the movement / motion processing unit 109 performs a movement / motion calculation (movement / motion simulation) of a moving object (a character or a tool used by the character). That is, the movement / motion processing unit 109 is configured such that an operation signal corresponding to the detected three-dimensional information detected by the detection unit 22, an operation signal from the operation unit, a set parameter or attribute, a program (movement / motion algorithm), various types Based on data (motion data) or the like, the mobile object is moved in the object space, or a process for controlling the motion (motion, animation) of the mobile object is performed.

  Specifically, the movement / motion processing unit 109 according to the present embodiment stores object movement information (position, rotation angle, speed, or acceleration) and movement information (position or rotation angle of each part object) for one frame. A simulation process is sequentially obtained every (for example, 1/60 seconds). Here, the frame is a unit of time for performing object movement / motion processing (simulation processing) and image generation processing. In this embodiment, the frame rate may be fixed every frame or may be variable according to the processing load.

  The determination unit 110 determines an overlap in the three-dimensional object space between the object shape information set in the three-dimensional object space and other object shape information.

  The drawing unit 120 performs drawing processing based on the results of various processing (game processing) performed by the processing unit 70, thereby generating an image and outputting the image to the display unit 190. In the case of generating a so-called three-dimensional game image, the drawing unit 120 of the present embodiment firstly stores vertex data (vertex position coordinates, texture coordinates, color data, normal vector, or α value) of each vertex of the object (model). Etc.) is input, and vertex processing is performed based on the vertex data included in the input object data. When performing the vertex processing, vertex generation processing (tessellation, curved surface division, polygon division) for re-dividing the polygon may be performed as necessary.

  In the vertex processing, geometric processing such as vertex movement processing, coordinate transformation (world coordinate transformation, camera coordinate transformation), clipping processing, perspective transformation, or light source processing is performed. The given vertex data is changed (updated or adjusted) for the vertex group to be configured. Then, rasterization (scan conversion) is performed based on the vertex data after the vertex processing, and the surface of the polygon (primitive) is associated with the pixel. Subsequent to rasterization, pixel processing (fragment processing) for drawing pixels constituting an image (fragments constituting a display screen) is performed.

  In pixel processing, various processes such as texture reading (texture mapping), color data setting / changing, translucent composition, anti-aliasing, etc. are performed to determine the final drawing color of the pixels that make up the image, and perspective transformation is performed. The drawing color of the object is output (drawn) to the frame buffer 174 (buffer that can store image information in units of pixels; VRAM, rendering target). That is, in pixel processing, per-pixel processing for setting or changing image information (color, normal, luminance, α value, etc.) in units of pixels is performed.

  Thereby, an image that can be seen from the virtual camera (given viewpoint) set in the object space is generated. Note that when there are a plurality of virtual cameras (viewpoints), an image can be generated so that an image seen from each virtual camera can be displayed as a divided image on one screen.

  Note that the vertex processing and pixel processing performed by the drawing unit 120 are performed by hardware that enables the polygon (primitive) drawing processing to be programmed by a shader program written in a shading language, so-called programmable shaders (vertex shaders and pixel shaders). It may be realized. Programmable shaders can be programmed with vertex-level processing and pixel-level processing, so that the degree of freedom of rendering processing is high, and the expressive power can be greatly improved compared to fixed rendering processing by hardware. .

  The drawing unit 120 performs geometry processing, texture mapping, hidden surface removal processing, α blending, and the like when drawing an object.

  In the geometry processing, processing such as coordinate conversion, clipping processing, perspective projection conversion, or light source calculation is performed on the object. Then, object data (positional coordinates of object vertices, texture coordinates, color data (luminance data), normal vector, α value, etc.) after geometry processing (after perspective projection conversion) is stored in the storage unit 170.

  In texture mapping, a process of mapping a texture (texel value) stored in the texture storage unit 174 of the storage unit 170 to an object is performed. Specifically, the texture (surface properties such as color (RGB) and α value) is read from the texture storage unit 174 of the storage unit 170 using the texture coordinates set (applied) to the vertex of the object. Map an image texture to an object. In this case, processing for associating pixels with texels, bilinear interpolation or the like is performed as texel interpolation. In particular, in the present embodiment, a process of mapping a texture in which the color of the surface of the polygon mesh is distributed in the texture space for each surface set is performed.

  In the present embodiment, when an object is drawn, a process for mapping a given texture may be performed. In this case, the color distribution (texel pattern) of the texture to be mapped can be dynamically changed.

  In this case, textures having different color distributions (pixel patterns) may be dynamically generated, or a plurality of textures having different color distributions are prepared in advance, and the texture to be used is dynamically switched. May be. The texture color distribution may be changed in units of objects.

  In particular, the texture mapping unit 121 of the present embodiment maps a given texture to a part of the object shape information generated by the object generation unit 107, and sets the object shape information in the three-dimensional object space. For example, when object shape information corresponding to itself is generated, when a 3D object (character) corresponding to itself is displayed on another person's terminal device, for example, a given texture is applied to a part corresponding to a part of the face. Map.

  In the hidden surface removal processing, hidden surface removal processing is performed by a Z buffer method (depth comparison method, Z test) using a Z buffer (depth buffer) in which the Z value (depth information) of the drawing pixel is stored. That is, when drawing the drawing pixel corresponding to the primitive of the object, the Z value stored in the Z buffer is referred to, and the Z value of the referenced Z buffer and the Z value at the drawing pixel of the primitive are obtained. In comparison, if the Z value at the drawing pixel is a Z value (for example, a small Z value) that is on the near side when viewed from the virtual camera, the drawing pixel is drawn and the Z value in the Z buffer is updated. Update to the correct Z value.

  In α blending (α synthesis), the rendering unit 120 performs translucent synthesis processing (normal α blending, addition α blending, subtraction α blending, or the like) based on an α value (A value). The α value is information that can be stored in association with each pixel (texel, dot), for example, plus alpha information other than color information. The α value can be used as mask information, translucency (equivalent to transparency and opacity), bump information, and the like.

  The sound generation unit 130 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.

  The communication control unit 140 performs communication control processing such as what information is transmitted to and received from which external device (server, terminal) based on operation information (request), a program, and the like. Specifically, the communication control unit 140 transmits / receives the object shape information generated by the object generation unit 107 to / from an external device. For example, the communication control unit 140 acquires the detected three-dimensional shape information of the terminal device itself as a detection target, generates object shape information corresponding to itself, and transmits this to other person's terminal device, The object shape information generated based on the detected three-dimensional shape information with the other person as the detection target in the other person's terminal device is received.

  Further, the communication control unit 140 may transmit / receive the output command signal to / from an external device. Thereby, for example, the detected three-dimensional shape information in which the shape of the sign language hand is detected in the terminal device of the user is acquired, the corresponding command signal is transmitted to the terminal device of the other person, and the hand of the sign language is transmitted to the terminal device of the other person. Character information and voice information corresponding to the shape may be output.

  Note that the image generation system of the present embodiment may be a system dedicated to the single player mode in which only one player can play, or may be a system having a multiplayer mode in which a plurality of players can play.

  In addition, when a plurality of players play, game images and game sounds provided to the plurality of players may be generated using one terminal, or connected via a network (transmission line, communication line), etc. It may be generated by distributed processing using a plurality of terminals (game machine, mobile phone).

4). Processing of this Embodiment Next, the flow of various processing performed in this embodiment will be described with reference to the drawings and flowcharts.

4-1. Avatar Character Control Processing First, so-called avatar (self-portrait) communication is performed using the image recognition system 10 of the present embodiment, in which communication such as mail or chat is performed while displaying a character corresponding to the user on another person's terminal. An example will be described. In the present embodiment, as shown in FIG. 6, with the image recognition system 10-1 corresponding to the player P1 and the image recognition system 10-2 corresponding to the player P2 connected via the Internet INT, The avatar character control process shown in the flowchart of FIG. 7 is performed. In FIG. 6, the game machines 18-1 and 18-2 are constituted by PCs (personal computers) 18-1 and 18-2 having a communication function.

  As shown in FIG. 7, in the present embodiment, the detection unit 16-1 of the image recognition system 10-1 acquires detection three-dimensional shape information for the player P1 as a detection target, and also detects the image recognition system 10-2. The unit 16-2 acquires detected three-dimensional shape information for the player P2 as a detection target (step S20). Then, the PC 18-1 of the image recognition system 10-1 generates object shape information of the avatar character corresponding to the player P1 based on the detected three-dimensional shape information for the player P1, and the image recognition system 10- The PC 18-2 of No. 2 generates object shape information of the avatar character corresponding to the player P2 based on the detected three-dimensional shape information targeting the player P2 (step S22).

  Then, the object shape information of the avatar characters AC1 and AC2 generated by the PC 18-1 and PC 18-2 is transmitted and received to each other (step S24 and step S26), and the object shape information generated in another image recognition system is used as the object. A space is set (step S28). Then, the player P1 and the player P2 map textures set in advance for the avatar characters AC1 and AC2 corresponding to the players P1 and P2, respectively, to a part of the faces of the avatar characters AC1 and AC2 (step S30). For example, textures such as the eyes and mouth of the animation character are mapped to positions corresponding to the eyes and mouth of the avatar characters AC1 and AC2. Then, the object space in which the object shape information of the avatar characters AC1 and AC2 is set is drawn (step S32).

  Thus, according to the present embodiment, as shown in FIG. 6, the display unit 14-1 of the image recognition system 10-1 is generated based on the detected three-dimensional shape information acquired by using the player P2 as a detection target. The avatar character AC2 is displayed, and the avatar character AC1 generated based on the detected three-dimensional shape information acquired by using the player P1 as the detection target is displayed on the display unit 14-2 of the image recognition system 10-2. Can conduct avatar communication.

4-2. Command Output Processing Next, using the image recognition system 10 of the present embodiment, the acquired detected three-dimensional shape information is compared with the reference three-dimensional shape information, and processing for outputting a corresponding command signal based on the comparison result is performed. An example will be described. In the present embodiment, as shown in FIG. 8, sign language is performed in front of the detection unit 16, and character information corresponding to the hand shape of the sign language is displayed on the display unit 14.

  As shown in the flowchart of FIG. 9, in the present embodiment, first, the detection unit 16 acquires detected three-dimensional shape information for the player as a detection target (step S40). Then, the reference three-dimensional shape information defining the shape of the sign language hand stored in the storage unit is compared with the acquired detected three-dimensional shape information (step S42). The detected 3D shape information acquired here includes 3D shape information corresponding to the shape of the right hand RH of the player, 3D shape information corresponding to other parts of the player, 3D shape information of the background, and the like. Is also included. In addition, sign language is often performed with the hand positioned in front of the body, and the hand is often in a position overlapping the user's body with respect to the detection unit. However, according to this embodiment, in the three-dimensional coordinate system of the imaging space of the detection unit 16, the three-dimensional shape information of the hands constituting the sign language can be acquired. Can be compared with the reference three-dimensional shape information. That is, according to the present embodiment, desired information can be extracted quickly and accurately from the detection target by analyzing the shape of the detection target at the position overlapping with the detection unit 16 three-dimensionally. .

  As a result of the comparison, if there is matching reference three-dimensional shape information (Y in step S44), a corresponding command signal is output (step S46), and an image is output based on the command signal (step S48). For example, as shown in FIG. 8, when the player P raises the index finger with the right hand RH, the shape of the right hand RH is extracted and compared with the reference three-dimensional shape information, and the character “most” is displayed on the display unit 14. Command signal to be output.

4-3. Operation Input Processing Next, using the image recognition system 10 of the present embodiment, if the detection target has a predetermined three-dimensional shape, the detection target is set as the operation detection target, and the player operates A description will be given of an example in which when a detection object is manipulated and the shape or position of the detection object changes, this is detected as an operation input and a corresponding operation signal is output. In FIG. 10A, an annular operation body RM is set as a steering wheel of a car, and a corresponding operation signal is output by rotating the annular operation body RM about the direction toward the detection unit 16 as a rotation axis. FIG. 10B is an example, and the rod-like operation body BM is set as an airplane control stick, and the corresponding operation signal is obtained by tilting the rod-like operation body BM forward, backward, left and right with respect to the detection unit 16. This is an example of output.

  As shown in the flowchart of FIG. 11, in the present embodiment, before executing the game, first, a setting mode for setting an operation detection target is executed (step S60). Then, the detection unit 16 obtains detected three-dimensional shape information whose detection target is the operating body operated by the player (step S62). Then, the reference three-dimensional shape information that defines the shape of the operating tool stored in the storage unit is compared with the acquired detected three-dimensional shape information (step S64). The detected 3D shape information acquired here includes 3D shape information corresponding to the shape of the operating body, 3D shape information corresponding to the player, 3D shape information of the background, and the like. As described above, according to the present embodiment, in the three-dimensional coordinate system of the imaging space of the detection unit 16, the three-dimensional shape information of the operating body is acquired, and only the operating body is extracted and compared with the reference three-dimensional shape information. To do.

  As a result of the comparison, if there is matching reference three-dimensional shape information (Y in step S66), a detection target corresponding to the detected three-dimensional shape information is set as an operation detection target (step S68). For example, in the case of FIG. 10A, since the annular operating body RM matches the reference three-dimensional shape information of the car handle, the annular operating body RM is set as the car handle, and FIG. ), The rod-like operation body BM matches the reference three-dimensional shape information of the airplane control stick, so the rod-like operation body BM is set as the airplane control stick. Then, an operation algorithm corresponding to the set type of operation detection object is set (step S69). For example, in the case of FIG. 10A, when the player performs an operation of rotating the annular operation body RM with the direction toward the detection unit 16 as the rotation axis, a car object (moving object) set in the object space. An operation algorithm for changing the moving direction of the MO is set, and in the case of FIG. 10B, when the player performs an operation of tilting the rod-shaped operation body BM forward, backward, left and right with respect to the detection unit 16, the object space is entered. An operation algorithm for changing the moving direction of the set airplane object (moving object) AO is set.

  When the operation detection object and the operation algorithm are set in this way, a game corresponding to the set operation detection object and the operation algorithm is executed (step S70). Then, the detection unit 16 acquires detected three-dimensional shape information for the operation object operated by the player as a detection target (step S72). Then, an operation signal is output based on the change in the detected three-dimensional shape information (step S74), and the moving / moving process of the moving object set in the object space is performed based on the operation signal (step S76). For example, in the case of FIG. 10A, the moving direction of the vehicle object (moving body object) is changed according to the amount of rotation of the annular operating body RM with the direction toward the detection unit 16 as the rotation axis. In the case of 10 (B), the moving direction of the airplane object (moving body object) is changed according to the tilting direction and the tilting amount of the rod-shaped operating body BM with respect to the detection unit 16. Then, an image in which the moving object moves and moves in the object space is drawn (step S78).

4-4. Item Recognition Processing Next, if the object to be detected has a predetermined three-dimensional shape using the image recognition system 10 of the present embodiment, the object shape information corresponding to the predetermined three-dimensional shape is set to 3. An example of performing processing for setting in a dimensional object space will be described. FIG. 12A is an example of drawing an image in which the character object CO has the sword item object KO if the player has a sword-shaped sword item body KM. This is an example of drawing an image in which the character object CO has the shield item object TO if it has the item body TM.

  As shown in the flowchart of FIG. 13, in the present embodiment, first, the detection unit 16 obtains detected three-dimensional shape information for detecting an item body held by the player (step S90). Then, the reference 3D shape information defining the shape of the item body stored in the storage unit is compared with the acquired detected 3D shape information (step S92). As a result of the comparison, if there is matching reference three-dimensional shape information (Y in step S94), object shape information corresponding to the detected three-dimensional shape information is set in the object space (step S96), and the object space is set. Drawing is performed (step S98). For example, in the case of FIG. 12A, since the sword item body KM matches the sword item object KO, an image in which the character object CO has the sword item object KO is drawn, and in the case of FIG. Since the shield item body TM matches the shield item object TO, the image in which the character object CO has the shield item object TO is drawn. When the set object shape information overlaps with the object shape information corresponding to another object in the object space, that is, when it is determined that the object has been hit (Y in step S100), the other object is broken. And hit processing is performed (step S102).

4-5. Movement Recognition Processing Next, an example of performing processing for drawing an image in which the position of a given object changes in accordance with the change in the position of the detection target will be described using the image recognition system 10 of the present embodiment. FIG. 14 shows an image in which when the player throws a thrown object HM made of foamed polystyrene or paper toward the detection unit 16, the corresponding ball object BO moves (throws) along with the movement of the thrown thrown object HM. Is an example of drawing.

  As shown in the flowchart of FIG. 15, in the present embodiment, first, the detection unit 16 acquires detected three-dimensional position information for the object HM that the player has as a detection target (step S110). Then, based on the detected three-dimensional position information, the ball object BO is set in the object space (step S112), and the object space is drawn (step S114). When the set ball object BO overlaps with another object in the object space, that is, when it is determined that the ball object BO has been hit (Y in step S116), the other object is broken or the ball object BO is moved. Hit processing such as changing the direction is performed (step S118). Then, the processes from step S110 to step S118 are repeated every 1/30 seconds to draw an image in which the ball object BO moves in the object space.

  In this way, in the image recognition system 10 of the present embodiment, the shape and position of the detection target are acquired by acquiring the three-dimensional shape information and the three-dimensional position information of the detection target in the three-dimensional coordinate system of the imaging space of the detection unit 16. The above-described various processes can be performed by three-dimensional analysis.

  Note that for the operation body in the above-described operation input process and the item body in the item recognition process, the reference 3D information serving as a determination criterion should be set strictly, and the detected 3D shape information should be substantially the same as the reference 3D information. If the detected three-dimensional shape information approximates the reference three-dimensional information, the reference three-dimensional information serving as the determination reference is not strictly set. May be. Here, when the matching determination is strictly performed, an operation body or item body having substantially the same shape as the reference three-dimensional information may be provided to the player. On the other hand, when the matching determination is not strictly performed, an object around the shape approximate to the reference three-dimensional information may be used as the operation body or the item body.

  In addition, information related to the human skeleton and human movement is stored in advance, and by specifying the three-dimensional information corresponding to the player from the three-dimensional information of the imaging space, the three-dimensional information corresponding to the sign language hand, You may make it specify 3D information, such as an operation body, an item body, and a to-be-projected body.

5. Hardware Configuration FIG. 16 shows an example of a hardware configuration that can realize this embodiment. The main processor 900 operates based on a program stored in a CD 982 (information storage medium), a program downloaded via the communication interface 990, a program stored in the ROM 950, or the like, and includes game processing, image processing, sound processing, and the like. Execute. The coprocessor 902 assists the processing of the main processor 900, and executes matrix operation (vector operation) at high speed. For example, when a matrix calculation process is required for a physical simulation for moving or moving an object, a program operating on the main processor 900 instructs (requests) the process to the coprocessor 902.

  The geometry processor 904 performs geometry processing such as coordinate conversion, perspective conversion, light source calculation, and curved surface generation based on an instruction from a program operating on the main processor 900, and executes matrix calculation at high speed. The data decompression processor 906 performs decoding processing of compressed image data and sound data, and accelerates the decoding processing of the main processor 900. Thereby, a moving image compressed by the MPEG method or the like can be displayed on the opening screen or the game screen.

  The drawing processor 910 executes drawing (rendering) processing of an object composed of primitive surfaces such as polygons and curved surfaces. When drawing an object, the main processor 900 uses the DMA controller 970 to pass the drawing data to the drawing processor 910 and, if necessary, transfers the texture to the texture storage unit 924. Then, the drawing processor 910 draws the object in the frame buffer 922 while performing hidden surface removal using a Z buffer or the like based on the drawing data and texture. The drawing processor 910 also performs α blending (translucent processing), depth cueing, mip mapping, fog processing, bilinear filtering, trilinear filtering, anti-aliasing, shading processing, and the like. When an image for one frame is written in the frame buffer 922, the image is displayed on the display 912.

  The sound processor 930 includes a multi-channel ADPCM sound source and the like, generates game sounds such as BGM, sound effects, and sounds, and outputs them through the speaker 932. Data from the game controller 942 and the memory card 944 is input via the serial interface 940.

  The ROM 950 stores system programs and the like. In the case of an arcade game system, the ROM 950 functions as an information storage medium, and various programs are stored in the ROM 950. A hard disk may be used instead of the ROM 950. The RAM 960 is a work area for various processors. The DMA controller 970 controls DMA transfer between the processor and the memory. The DVD drive 980 accesses a DVD 982 in which programs, image data, sound data, and the like are stored. The communication interface 990 performs data transfer with the outside via a network (communication line, high-speed serial bus).

  The processing of each unit (each unit) in this embodiment may be realized entirely by hardware, or may be realized by a program stored in an information storage medium or a program distributed via a communication interface. Also good. Alternatively, it may be realized by both hardware and a program.

  When the processing of each part of this embodiment is realized by both hardware and a program, a program for causing the hardware (computer) to function as each part of this embodiment is stored in the information storage medium. More specifically, the program instructs the processors 902, 904, 906, 910, and 930, which are hardware, and passes data if necessary. Each processor 902, 904, 906, 910, 930 realizes the processing of each unit of the present invention based on the instruction and the passed data.

6). Modifications The present invention is not limited to that described in the above embodiment, and various modifications can be made. For example, terms cited as broad or synonymous terms in the description in the specification or drawings can be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

  Further, the present invention is applied to various image generation systems such as a business game system, a home game system, a large attraction system in which a large number of players participate, a simulator, a multimedia terminal, a system board for generating a game image, and a mobile phone. it can.

  Further, the present invention can be applied to various pointing position calculation systems. In the above-described embodiment, the case where the present invention is applied to a game system has been described as an example. However, the present invention is applied to various pointing position calculation systems such as a presentation system and a pointer used in the pointing position calculation system. can do.

The figure which shows an example of the schematic external appearance of the system of this embodiment. The figure for demonstrating the principle of the system of this embodiment. The figure for demonstrating an example of the image imaged by this embodiment. The figure for demonstrating the principle of the system of this embodiment. The figure which shows an example of the functional block of the system of this embodiment. The figure for demonstrating an example of the process of the system of this embodiment. The flowchart figure which shows an example of the process of this embodiment. The figure for demonstrating an example of the process of the system of this embodiment. The flowchart figure which shows an example of the process of this embodiment. FIGS. 10A and 10B are diagrams for explaining an example of processing of the system of this embodiment. The flowchart figure which shows an example of the process of this embodiment. FIGS. 12A and 12B are diagrams for explaining an example of processing of the system of this embodiment. The flowchart figure which shows an example of the process of this embodiment. The figure for demonstrating an example of the process of the system of this embodiment. The flowchart figure which shows an example of the process of this embodiment. The figure which shows an example of the hardware constitutions of the system of this embodiment.

Explanation of symbols

P player, TO target object, DL output wave, RL reflected wave, WL wall,
PO imaging object, PA imaging area, PS imaging space, WP center of gravity position, N normal,
10 game systems, 12 display screens, 14 display units, 16 detection units,
18 game machine, 20 output unit, 22 detection unit, 24 3D information acquisition unit,
100 processing unit, 101 comparison unit, 102 command signal output unit, 103 output control unit,
104 operation detection object setting unit, 105 operation algorithm setting unit,
106 operation signal output unit, 107 object generation unit,
108 object space setting unit, 109 movement / motion processing unit, 110 determination unit,
120 drawing processing units, 121 texture mapping units, 130 sound generation units,
140 communication control unit, 170 storage unit, 180 information storage medium

Claims (14)

An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
An object generation unit that generates object shape information that defines the shape of a three-dimensional object set in a given three-dimensional object space based on the detected three-dimensional shape information;
A communication control unit that transmits and receives the generated object shape information to and from an external device;
An object space setting unit for setting the received object shape information in the three-dimensional object space;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
An image recognition system comprising: An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
An object shape information storage unit for storing a plurality of types of object shape information defining the shapes of a plurality of types of three-dimensional objects set in a given three-dimensional object space;
An object shape information selection unit that selects the object shape information corresponding to the detected three-dimensional shape information based on the detected three-dimensional shape information;
A communication control unit for transmitting and receiving the selected object shape information to and from an external device;
An object space setting unit for setting the received object shape information in the three-dimensional object space;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
An image recognition system comprising: In claim 2,
An object shape information correction unit that corrects the object shape information selected by the object shape information selection unit based on the detected three-dimensional shape information;
The communication control unit
An image recognition system, wherein the object shape information corrected by the object shape information correction unit is transmitted / received to / from an external device. In any one of Claims 1-3,
An image recognition system, further comprising a texture mapping unit that maps a given texture to a part of the generated object shape information and sets the object shape information in the three-dimensional object space. An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
A reference information storage unit that stores reference three-dimensional shape information that defines a predetermined three-dimensional shape;
A comparison unit that compares the reference three-dimensional shape information with the detected three-dimensional shape information;
A command signal output unit that outputs a corresponding command signal based on the comparison result of the comparison unit;
An output information storage unit that stores at least one of image information and sound information corresponding to the command signal;
An image recognition system comprising: an output control unit that outputs at least one of corresponding image information and sound information based on the output command signal. In claim 5,
The three-dimensional information acquisition unit
Obtaining detection three-dimensional motion information defining a change in a predetermined period of the three-dimensional shape of the detection object reflected by the output wave based on the reflected wave;
The reference information storage unit is
Storing reference three-dimensional motion information defining a change in a predetermined period of a predetermined three-dimensional shape;
The comparison unit is
An image recognition system that compares the reference three-dimensional motion information with the detected three-dimensional motion information. In any one of Claims 5 and 6,
A communication control unit that transmits and receives the output command signal to and from an external device;
The output control unit is
An image recognition system for outputting at least one of corresponding image information and sound information based on the received command signal. A movement / motion processing unit that controls at least one of movement and motion of a given object based on an operation signal;
A drawing processing unit for drawing an image in which the given object performs at least one of movement and movement;
An image recognition system including:
An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
Based on the reflected wave, at least one of detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave and detection three-dimensional position information that defines the three-dimensional position of the detection object. A three-dimensional information acquisition unit to acquire;
A reference information storage unit that stores reference three-dimensional shape information that defines a predetermined three-dimensional shape;
A comparison unit that compares the reference three-dimensional shape information with the detected three-dimensional shape information;
An operation detection target setting unit that sets the detection target corresponding to the detection three-dimensional shape information as an operation detection target based on the comparison result of the comparison unit;
An operation signal output unit that outputs the operation signal based on a change in at least one of the detected three-dimensional shape information and the detected three-dimensional position information acquired for the set operation detection target;
An image recognition system, further comprising: In claim 8,
An operation algorithm storage unit that stores an operation algorithm corresponding to each reference 3D shape information of the plurality of reference 3D shape information stored in the reference information storage unit;
An operation algorithm setting unit that sets an operation algorithm according to the reference three-dimensional shape information corresponding to the detected three-dimensional shape information based on the comparison result of the comparison unit;
Further including
The operation signal output unit is
Outputting an operation signal corresponding to the set operation algorithm based on a change in at least one of the detected three-dimensional shape information and the detected three-dimensional position information acquired for the set operation detection object. A featured image recognition system. An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional shape information that defines the three-dimensional shape of the detection object reflected by the output wave, based on the reflected wave;
A reference information storage unit that stores reference three-dimensional shape information that defines a predetermined three-dimensional shape;
A comparison unit that compares the reference three-dimensional shape information with the detected three-dimensional shape information;
An object space setting unit that sets given object shape information in the three-dimensional object space based on a comparison result of the comparison unit;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
An image recognition system comprising: In claim 10,
The object shape information storage unit
Stores multiple types of object shape information,
The object space setting unit
An image recognition system, wherein the object shape information corresponding to the detected three-dimensional shape information among the plurality of types of object shape information is set in the three-dimensional object space based on a comparison result of the comparison unit. In claim 11,
The object space setting unit
An image recognition system, wherein object shape information set in the three-dimensional object space is changed based on occurrence of a given event. An output unit for outputting a predetermined output wave;
A detection unit that has a predetermined positional relationship with the output unit and detects a reflected wave of the output wave;
A three-dimensional information acquisition unit that acquires detection three-dimensional position information that defines a three-dimensional shape of the detection object based on the reflected wave;
An object space setting unit that sets given object shape information in the three-dimensional object space based on the acquired detected three-dimensional shape information;
A drawing processing unit for drawing an image of the three-dimensional object space viewed from a virtual camera;
An image recognition system comprising: In any one of Claims 10-13,
An image recognition system, further comprising: a determination unit that determines an overlap in the three-dimensional object space between the object shape information set in the three-dimensional object space and other object shape information.

Images