JP6448196B2 - Image generation system and program - Google Patents

Description

  The present invention relates to an image generation system, a program, and the like.

  2. Description of the Related Art Conventionally, a system that generates a projection image projected on a curved screen (dome-shaped screen) by a projection device (projector) is known. As a conventional technique for projecting a projection image with little distortion on a curved screen, there is a technique disclosed in Patent Document 1, for example.

  There is also known a game system that detects the position of the spotlight of the light from the light emitting unit and performs hit determination in the shooting game. As a conventional technique of such a game system, there is a technique disclosed in Patent Document 2, for example.

JP 2003-85586 A JP 2001-4327 A

  However, in an image generation system that generates a projection image on a curved screen, the position of the light spot light from the light emitting unit is detected, and an appropriate hit determination process between the light emitting direction of the light emitting unit and the object is performed. No method has been proposed for realization.

  According to some aspects of the present invention, it is possible to provide an image generation system, a program, and the like that can realize an appropriate hit determination process while generating a projection image for projection.

  One aspect of the present invention includes an object space setting unit that performs an object space setting process, an image generation unit that generates a projection image for projection based on information on a plurality of objects arranged in the object space, A receiving unit that receives a captured image from an imaging unit that captures a projection area of the projected image, and a position of the spot light on the projection screen based on the position of the spot light on the captured image by the light from the light emitting unit. A screen spot light position that is a position, and a direction from the set position set as the representative position of the light emitting unit or player to the screen spot light position is obtained as the light emitting direction of the light emitting unit, Based on the obtained emission direction, a hit determination process with an object in the object space is performed. And the hit judging unit that performs, related to an image generation system comprising. The present invention also relates to a program that causes a computer to function as each of the above-described units, or a computer-readable information storage medium that stores the program.

  According to one aspect of the present invention, a screen spot light position, which is the position of the spot light on the projection screen, is obtained based on the position of the spot light on the captured image. Then, a direction from the set position set as the light emitting unit or the representative position of the player to the screen spot light position is obtained as the light emitting direction, and hit determination processing with an object in the object space based on the emitting direction Is executed. In this way, an appropriate hit determination process that reflects the shape of the projection screen can be realized in an image generation system that generates a projection image for projection. In other words, if the direction from the set position to the screen spot light position is set as the emission direction, an appropriate emission reflecting the shape of the projection screen can be obtained when the projection screen is not a screen composed of only one plane. The direction can be obtained and the hit determination process based on the emission direction can be executed.

  In one aspect of the present invention, the projection screen is a screen configured by one curved surface or a plurality of surfaces, and the image generation unit performs a distortion correction process based on shape information of the projection screen. The projection image may be generated, and the hit determination unit may obtain the screen spot light position based on the shape information of the projection screen.

  As described above, if the distortion correction processing based on the shape information of the projection screen is performed, an appropriate projection image reflecting the shape of the projection screen constituted by one curved surface or a plurality of surfaces can be generated. Further, if the screen spot light position is obtained based on the shape information of the projection screen, the screen spot light position reflecting the shape of the projection screen constituted by one curved surface or a plurality of surfaces can be obtained. It is possible to realize an appropriate hit determination process reflecting the shape of the screen for use.

  In the aspect of the invention, the hit determination unit obtains a direction vector of the spot light viewed from the imaging unit based on a position of the spot light on the captured image, and follows the direction vector. An intersection position between the straight line and the projection screen may be obtained as the screen spot light position.

  In this way, the direction vector of the spot light corresponding to the position of the spot light on the captured image is obtained, and the screen spot light is obtained by obtaining the intersection point between the straight line along the direction vector and the projection screen. The position can be determined. As a result, for example, it is possible to obtain a screen spot light position reflecting information on the optical system of the imaging unit.

  In one aspect of the present invention, the image generation unit includes a straight line connecting the intersection position between the light beam emitted from the pixel on the drawing buffer through the optical system of the projection apparatus and the projection screen and the representative viewpoint position. The color of the pixel on the drawing buffer may be determined as the line of sight of the virtual camera.

  In this way, it becomes possible to generate a projection image for projection by performing distortion correction processing in units of pixels.

  In the aspect of the invention, the image generation unit may perform drawing corresponding to the object based on an intersection position between the projection screen and a straight line connecting the vertex position of the object and the representative viewpoint position in the object space. A vertex position on the drawing buffer for the object may be obtained, and the drawing object may be drawn in the drawing buffer based on the vertex position.

  In this way, it becomes possible to generate a projection image for projection by performing distortion correction processing in units of vertices.

  In one aspect of the present invention, the object space setting unit obtains an arrangement position of the aiming object representing the aim of the light emitting unit in the object space based on the emission direction, and determines the obtained arrangement position. The aiming object may be arranged.

  In this way, the aiming object of the light emitting unit can be arranged at a position obtained based on the emitting direction, and an image of the aiming object can be generated on the projection image.

  In the aspect of the invention, the object space setting unit may arrange the aiming object on a straight line along the emission direction.

  In this way, it is possible to display an image of the aiming object that moves on the projection image in conjunction with a change in the emission direction of the light emitting unit.

  In the aspect of the invention, the image generation unit may generate the projection image in which the spot light detection adjustment object is displayed inside the imaging range of the imaging unit.

  In this way, for example, even when there are variations in the mounting position and mounting direction of the imaging unit, it is possible to appropriately realize the adjustment of the spot light position detection using the detection adjustment object.

  In the aspect of the invention, the object space setting unit may detect the detection adjustment object so that the detection adjustment object is displayed inside the imaging range of the imaging unit in a projection area of the projection image. May be arranged in the object space.

  In this way, it becomes possible to appropriately display the detection adjustment object for realizing the adjustment of the position detection of the spot light on the projection image for projection.

  Moreover, in one aspect of the present invention, an imaging range specifying unit that specifies the imaging range of the imaging unit based on the second captured image obtained by capturing the projection image or the history information of the detection position of the spot light may be included. .

  In this way, it becomes possible to specify the imaging range of the imaging unit, and it is possible to prevent problems and the like from occurring in spot light detection processing based on the captured image of the imaging unit.

  In one embodiment of the present invention, the imaging unit may include the imaging unit and a fisheye lens.

  If the fisheye lens is provided in the imaging unit in this manner, it is possible to detect the position of the spot light over a wide range of the projection area of the projection image.

  In one embodiment of the present invention, the projector may be included, and the imaging unit may be attached to the projector.

  In this way, for example, it becomes easy to align the projection direction of the projection apparatus and the imaging direction of the imaging unit, and the accuracy of hit determination processing can be improved.

  According to another aspect of the present invention, an object space setting unit that performs an object space setting process, an image generation unit that generates a projection image for projection based on information on a plurality of objects arranged in the object space, A light projection unit that projects the projection image; an imaging unit that captures a projection area of the projection image; a light emission unit that emits light; and a light output from the light emission unit based on a captured image from the imaging unit. And a hit determination unit that performs a hit determination process with an object in the object space based on the determined emission direction, and the imaging unit includes an image sensor and a fisheye lens. Related to the generation system.

  According to another aspect of the present invention, the projection image is projected by the projection device, and the projection area of the projection image is captured by the imaging unit. Then, based on the captured image, the light emitting direction of the light emitting unit is obtained, and hit determination processing with the object in the object space is executed. In another aspect of the present invention, the imaging unit is provided with a fisheye lens, and the light emitting direction of the light emitting unit is obtained based on the captured image captured through the fisheye lens. If the fisheye lens is provided in the imaging unit in this manner, the detection process can be executed for the hit determination process over a wide range of the projection area of the projection image. Therefore, hit determination processing suitable for an image generation system that generates a projection image for projection can be realized.

An example of the game system to which the image generation system of this embodiment is applied. The vertical sectional view of the game system to which the image generation system of this embodiment is applied. 1 is a configuration example of an image generation system according to the present embodiment. Explanatory drawing of the distortion correction method per pixel of a drawing buffer. Explanatory drawing of the method of using a surrogate plane for distortion correction per pixel. The figure which shows the relationship between the drawing buffer by the method of using a substitute plane for distortion correction per pixel, a UV map, and a render texture. Explanatory drawing of the distortion correction method in the vertex unit of an object. An example of a projection image generated by the image generation system of this embodiment. FIG. 9A and FIG. 9B are explanatory diagrams of a shooting game realized by the image generation system of this embodiment. FIGS. 10A and 10B are also explanatory diagrams of a shooting game realized by the image generation system of the present embodiment. Explanatory drawing of the hit determination method of this embodiment. The flowchart explaining the hit determination process of this embodiment. Explanatory drawing of the arrangement | positioning method of the aiming object of this embodiment. Explanatory drawing of the arrangement | positioning method of the aiming object of this embodiment. FIGS. 15A and 15B are explanatory diagrams of a method for calculating the direction vector of the spot light. FIGS. 16A and 16B are also explanatory diagrams of a method for calculating the direction vector of the spot light. FIG. 17A and FIG. 17B are explanatory diagrams of objects for detecting and adjusting spot light. FIG. 18A and FIG. 18B are also explanatory diagrams of the spot light detection adjustment object. Explanatory drawing of the object for detection adjustment of a spot light. 20A and 20B are explanatory diagrams of the imaging range of the imaging unit. Explanatory drawing of the arrangement method of the spot light detection adjustment object. FIG. 22A and FIG. 22B are explanatory diagrams of the display control method for the aiming object.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. Configuration FIG. 1 shows a configuration example of a game system to which the image generation system of this embodiment is applied. FIG. 2 is a vertical sectional view of the game system.

  The game system shown in FIGS. 1 and 2 is a business game system installed in an amusement facility for a player to play a shooting game. This game system includes a player seat 10, a control board 16, a curved screen 20 on which a projection image that is a game image is projected, a projection device 30 that projects a projection image (video) on the screen 20, and spot light. And a light emitting unit 50 (gun-type controller) simulating a gun, and a speaker (not shown) for outputting game sounds.

  The player seat 10 is provided with its orientation and height adjusted so that the assumed normal viewing direction of the seated player faces the center of the screen 20. In the present embodiment, the front direction of the player who is seated is the assumed normal viewing direction. The screen 20 is formed in a convex shape with respect to the front direction (assumed normal viewing direction) of the player seated on the player seat 10.

  The control board 16 is mounted with various processors such as a CPU, GPU or DSP, and various memories such as an ASIC, VRAM, RAM or ROM. Then, the control board 16 performs various processes for realizing a shooting game based on a program and data stored in the memory, an operation signal of a player who operates the light emitting unit 50, and the like.

  The projection device 30 (projector) is supported by the support column 12 and the housing frame 14 installed behind the player seat 10 and is located above the player seat 10 so as not to interfere with the player seated on the player seat 10. The projection center direction is set so as to face the vicinity of the center of the screen 20. In other words, the projection center direction is set so as to face the intersection point between the assumed normal viewing direction of the player and the screen 20. In addition, the projection device 30 is provided with a wide-angle lens (for example, a fish-eye lens, that is, a super-wide-angle lens with an angle of view exceeding 180 degrees) as a projection lens. Projected to the whole.

  The projection device 30 is attached with an imaging unit 40 that captures a projection area of the projection image. For example, the shooting direction of the imaging unit 40 is set to the projection direction of the projection image. Using the picked-up image picked up by the image pickup unit 40, the position of the spot light formed on the screen 20 by the light emitted from the light emitting unit 50 is detected. Then, the light emission direction of the light emitting unit 50 is specified based on the detected position of the spot light, and hit determination with an enemy object shown in the game image is performed.

  The light emitting unit 50 is a gun-type controller that is modeled on a gun. The light emitting portion 50 is provided with an infrared light emitting portion, and spot light is formed on the screen 20 when the infrared light hits. Since the imaging unit 40 is provided with an infrared filter (IR filter) that blocks visible light and transmits infrared light, the position of the spot light of the infrared light is determined based on the captured image of the imaging unit 40. It can be detected.

  The player sits on the player seat 10, watches the game image displayed on the screen 20, listens to the game sound from the speaker, and shoots the enemy appearing in the game image with the light emitting unit 50 simulating a gun. And enjoy the game.

  In the shooting game of the present embodiment, a game space is configured by arranging objects such as background objects in an object space (virtual three-dimensional space). In this object space, enemy objects and the like that are targets for shooting are also arranged, and a virtual camera is arranged at the viewpoint position of the player. Then, the image of the object space viewed from the virtual camera is projected (displayed) on the screen 20 by the projection device 30 as a game image.

  FIG. 3 shows an example of a block diagram of the image generation system of the present embodiment. Note that the configuration of the image generation system of the present embodiment is not limited to FIG. 3, and various modifications such as omitting some of the components (each unit) or adding other components are possible. .

  The imaging unit 40 (camera) includes an optical system such as a lens 42 and an imaging element 44. The lens 42 is, for example, a lens that projects an image on the image sensor 44 and is a wide-angle lens (for example, a fisheye lens) for capturing the entire screen 20. The image sensor 44 is realized by, for example, a CCD or CMOS sensor. In addition to this, the imaging unit 40 is provided with the above-described infrared filter and the like.

  The operation unit 160 is for a player to input operation data. When the image generation system is applied to the game system of FIGS. 1 and 2, the operation unit 160 can be realized by the light emitting unit 50 that is a gun-type controller, an operation button (not shown), or the like.

  The storage unit 170 serves as a work area for the processing unit 100, the communication unit 196, and the like, and its function can be realized by a RAM (DRAM, VRAM) or the like. Then, the game program and game data necessary for executing the game program are held in the storage unit 170.

  An information storage medium 180 (a computer-readable medium) stores programs, data, and the like, and functions thereof by an optical disk (CD, DVD), HDD (hard disk drive), memory (ROM, etc.), and the like. realizable. The processing unit 100 performs various processes of the present embodiment based on a program (data) stored in the information storage medium 180. That is, in the information storage medium 180, a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (a program for causing the computer to execute processing of each unit). Is memorized.

  The display unit 190 displays an image generated according to the present embodiment. When the image generation system is applied to the game system of FIGS. 1 and 2, the display unit 190 can be realized by an LCD in a liquid crystal projector, a DMD in a DLP projector, or the like. The sound output unit 192 outputs the sound generated by the present embodiment, and its function can be realized by a speaker, headphones, or the like.

  The auxiliary storage device 194 (auxiliary memory, secondary memory) is a storage device used to supplement the capacity of the storage unit 170, and can be realized by a memory card such as an SD memory card or a multimedia card.

  The communication unit 196 communicates with the outside (for example, another image generation system, a server, or a host device) via a wired or wireless network, and functions as a communication ASIC, a communication processor, or the like. It can be realized by hardware and communication firmware.

  The processing unit 100 (processor) performs game processing, image generation processing, sound generation processing, and the like based on operation data from the operation unit 160, a program, and the like. The processing unit 100 performs various processes using the storage unit 170 as a work area. The functions of the processing unit 100 can be realized by hardware such as various processors (CPU, GPU, etc.), ASIC (gate array, etc.), and programs.

  The processing unit 100 includes a game calculation unit 102, an object space setting unit 104, a moving body calculation unit 106, a virtual camera control unit 108, a reception unit 110, a hit determination unit 114, an imaging range specifying unit 116, an image generation unit 120, and sound generation. Part 130. Various modifications may be made such as omitting some of these components (each unit) or adding other components.

  The game calculation unit 102 performs game calculation processing. Here, as a game calculation, a process for starting a game when a game start condition is satisfied, a process for advancing the game, a process for calculating a game result, or a process for ending a game when a game end condition is satisfied and so on.

  The object space setting unit 104 performs an object space setting process in which a plurality of objects are arranged. For example, various objects (polygons, free-form surfaces) representing display objects such as moving objects (people, animals, robots, cars, airplanes, marine aircraft, etc.), maps (terrain), buildings, courses (roads), trees, walls, water surfaces, etc. Or an object configured with a primitive surface such as a subdivision surface) is set in the object space. In other words, the position and rotation angle of the object in the world coordinate system (synonymous with direction and direction) are determined, and the rotation angle (rotation angle around the X, Y, and Z axes) is determined at that position (X, Y, Z). Place the object. Specifically, the object data storage unit 172 of the storage unit 170 stores object data such as the position, rotation angle, moving speed, moving direction, etc. of the object (part object) in association with the object number. . The object space setting unit 104 performs a process of updating the object data for each frame, for example.

  The moving object calculation unit 106 performs control processing for moving a moving object (moving object) such as a person, an animal, a car, or an airplane. Also, control processing for operating the moving body is performed. That is, based on operation data input by the player through the operation unit 160, a program (movement / motion algorithm), various data (motion data), etc., a moving body (object, model object) is moved in the object space, Performs control processing to move the moving body (motion, animation). Specifically, a simulation process for sequentially obtaining movement information (position, rotation angle, speed, or acceleration) and motion information (part object position or rotation angle) of a moving body for each frame (1/60 second). I do. A frame is a unit of time for performing a moving / movement process (simulation process) and an image generation process of a moving object.

  The virtual camera control unit 108 performs control processing of a virtual camera (viewpoint, reference virtual camera) for generating an image that can be seen from a given (arbitrary) viewpoint in the object space. Specifically, processing for controlling the position (X, Y, Z) or rotation angle (rotation angle about the X, Y, Z axis) of the virtual camera (processing for controlling the viewpoint position, the line-of-sight direction or the angle of view) I do.

  For example, when a moving body is photographed from behind using a virtual camera, the position (viewpoint position) and direction (gaze direction) of the virtual camera are controlled so that the virtual camera follows changes in the position or direction of the moving body. In this case, the virtual camera can be controlled based on information such as the position, direction, or speed of the moving object obtained by the moving object computing unit 106. Alternatively, control may be performed such that the virtual camera is rotated at a predetermined rotation angle or moved along a predetermined movement path. In this case, the virtual camera is controlled based on virtual camera data for specifying the position (movement path) or direction of the virtual camera.

  The sound generation unit 130 performs sound processing based on the results of various processes performed by the processing unit 100, generates game sounds such as BGM, sound effects, or sounds, and outputs the game sounds to the sound output unit 192.

  The image generation system according to the present embodiment includes an object space setting unit 104, a reception unit 110, a hit determination unit 114, and an image generation unit 120, as shown in FIG. Moreover, the image generation system (game system) of this embodiment can include the imaging unit 40, the projection device 30 and the light emitting unit 50 of FIGS.

  The object space setting unit 104 performs the object space setting process as described above. The object space is a three-dimensional game space.

  The image generation unit 120 generates a projection image. Specifically, a projection image for projection (a projection image generated by projection mapping processing) is generated based on information on a plurality of objects (enemy objects, background objects, etc.) arranged in the object space. The projected image is an image that the projection device 30 projects onto the curved screen 20 (dome-shaped screen). This projection image is displayed on a display unit 190 realized by an LCD in a liquid crystal projector or a DMD in a DLP projector, and is projected on the screen 20 by an optical system such as a wide-angle lens of the projection device 30.

  The accepting unit 110 accepts a captured image from the image capturing unit 40 that captures a projection area (screen area) of the projection image. For example, the imaging direction (imaging direction) of the imaging unit 40 is set to the projection direction of the projection image (projection direction of the projection device). Specifically, the imaging unit 40 is installed such that the imaging direction is parallel (substantially parallel) to the projection direction, for example. When the imaging unit 40 captures an image of the projection area of the screen 20, the captured image data is received by the receiving unit 110 that functions as an image interface.

  The hit determination unit 114 performs a hit determination process between the light from the light emitting unit 50 and the object. That is, assuming that the bullet of the gun has flew in the light emitting direction of the light emitting unit 50, the hit determination process between the bullet represented by the light from the light emitting unit 50 and the object is performed.

  Specifically, the hit determination unit 114 obtains the position of the spot light on the captured image (the position in the coordinate system of the image sensor) by the light (for example, infrared light) from the light emitting unit 50. For example, image processing or the like is performed on the captured image from the imaging unit 40 to detect the position of spot light (infrared spot light) reflected in the captured image. The hit determination unit 114 obtains a screen spot light position, which is the position of the spot light on the projection screen, based on the position of the spot light on the captured image. For example, the position of the spot light on the captured image and the position of the spot light on the projection screen can be associated one to one. The screen spot light position is the position of the spot light on the projection screen that is associated one-to-one with the position of the spot light on the captured image.

  Then, the hit determination unit 114 obtains the direction from the set position set as the light emitting unit 50 or the representative position of the player toward the screen spot light position as the light emitting direction of the light emitting unit 50. That is, the representative position assumed as the position of the light emitting unit 50 (or the position of the player) is set as the set position, and the direction of the straight line connecting the set position and the screen spot light position is obtained as the emission direction. This emission direction does not necessarily match the actual emission direction of the light emitting unit 50, but in order to realize hit determination processing, the direction of a straight line connecting the set position and the above-described intersection position is assumed to be a virtual emission direction. Set to.

  Then, the hit determination unit 114 performs hit determination processing with an object in the object space based on the emission direction. That is, hit determination with a target object such as an enemy object is performed. For example, by setting a straight line (light ray) extending in the obtained emission direction and performing an intersection determination between the straight line and the object, it is determined whether or not the light (bullet, shot) from the light emitting unit 50 has hit the object. A hit determination process for checking is performed.

  When the projection screen is a screen composed of one curved surface or a plurality of surfaces, the image generation unit 120 performs a distortion correction process based on the shape information of the projection screen to generate a projection image. The shape information of the projection screen is, for example, information representing the shape of the projection screen by a mathematical expression (for example, an ellipsoidal equation). The image generation unit 120 generates a projection image by performing distortion correction processing for reflecting the shape of the projection screen formed by one curved surface or a plurality of surfaces.

  The hit determination unit 114 obtains the screen spot light position based on the shape information of the projection screen. For example, the screen spot light position is obtained based on information representing the shape of the projection screen by a mathematical expression or the like. For example, as will be described later, when the position of the intersection of the straight line along the direction vector and the projection screen is obtained as the screen spot light position, the mathematical expression of the straight line along the direction vector and the mathematical expression representing the shape of the projection screen Based on the above, the screen spot light position can be obtained.

  Further, the hit determination unit 114 obtains the direction vector of the spot light viewed from the imaging unit 40 based on the position of the spot light on the captured image. For example, in the camera coordinate system of the imaging unit 40, a direction vector representing the direction of the spot light reflected on the screen 20 is obtained. Next, the hit determination unit 114 determines the intersection position of the straight line along the direction vector (the straight line along the direction of the direction vector from the representative position of the imaging unit 40) and the projection screen as the screen spot light position. Asking. Then, the hit determination unit 114 obtains the direction of a straight line connecting the set value assumed as the position of the light emitting unit 50 (or the position of the player) and the screen spot light position that is the intersection position as the emission direction, Perform hit determination processing.

  The object space setting unit 104 performs an aiming object placement process. For example, the object space setting unit 104 obtains the arrangement position of the aiming object (gun site) representing the aim of the light emitting unit 50 in the object space based on the emission direction described above. Then, the aiming object is arranged at the obtained arrangement position. Specifically, the object space setting unit 104 arranges the aiming object on a straight line along the emission direction. For example, an aiming object that is a three-dimensional object is arranged so as to intersect the straight line.

  The image generation unit 120 generates a projection image on which the spot light detection adjustment object is displayed. For example, a spot light detection adjustment object (initial adjustment object) generates a projection image displayed inside the imaging range (imaging area) of the imaging unit 40. Specifically, the object space setting unit 104 detects the detection adjustment object (three-dimensional) so that the detection adjustment object is displayed inside the imaging range of the imaging unit 40 in the projection area (screen area) of the projection image. Object) in the object space. By using such an object for detection adjustment, adjustment processing (correction processing) or the like of spot light position detection is realized.

  The imaging range specifying unit 116 performs processing for specifying the imaging range of the imaging unit 40. For example, in the present embodiment, the imaging range is narrower than the projection area (projected image display area). For example, the imaging range area is, for example, an area included in the projection area. Then, due to variations in the mounting position and mounting direction of the imaging unit 40, variations occur in the boundaries of the imaging range. The imaging range specifying unit 116 specifies the boundary of the imaging range.

  In this case, the imaging range specifying unit 116 specifies the imaging range of the imaging unit 40 based on the second captured image obtained by capturing the projection image or the history information of the spot light detection position. The second captured image is an image different from the captured image captured by the imaging unit 40 for spot light detection. For example, when the captured image is a captured image for infrared light, for example, the second captured image is a captured image for visible light. The second captured image may be captured by providing a second image capturing unit separate from the image capturing unit 40, or by switching a filter (for example, an infrared filter) of the image capturing unit 40 or the like. Good. The spot light detection position history information is a spot light detection position history obtained by an operator such as a player operating the light emitting unit 50. The history information of the detection position of the spot light may be history information of a detection position by a past game play, or may be history information of a detection position by a game play of another player.

  The image generation unit 120 generates a projection image for projection. The generated projection image is projected onto the screen 20 by the projection device 30 shown in FIGS. As a result, the player can view the image of the object space viewed from the virtual camera as a game image. The projection image for projection is, for example, a projection image generated by projection mapping processing. Projection mapping is a method of projecting with the projection device 30 taking into account the state (shape, etc.) of the object (screen) to be projected and the state (position, direction, etc.) of the projection device 30.

  Specifically, the image generation unit 120 determines that the pixel on the drawing buffer 176 (pixel of the projection image) is the intersection point between the light beam emitted through the optical system (such as a wide-angle lens) of the projection device 30 and the projection screen. The color of the pixel on the drawing buffer 176 is determined with the straight line connecting the representative viewpoint positions as the line of sight of the virtual camera. For example, using this straight line as the line of sight of the virtual camera, the pixel color is determined from the information in the object space. Specifically, the color of the pixel is determined according to the intersection position of this straight line and the object in the object space (the position of the point on the object where this straight line first arrives in the object space). This processing can be realized by the ray tracing method, but the drawing load is large and it is difficult to implement in real time. Therefore, a more practical method can be realized by saving the rendering result to a render texture on a flat screen that is as close as possible to a curved screen (this will be referred to as a “proxy plane”). Good. Then, the projection image is generated by determining the color of the pixel in this way. The representative viewpoint position is a position assumed to be the player's viewpoint position (the position of the virtual camera).

  Alternatively, the image generation unit 120 obtains a straight line connecting the vertex position of the object in the object space and the representative viewpoint position, and obtains an intersection position between the straight line and the projection screen. Then, based on the obtained intersection position, the vertex position on the drawing buffer 176 for the drawing object corresponding to the object is obtained. Then, a projection image is generated by drawing a drawing object in the drawing buffer 176 based on the obtained vertex position.

  Note that the projection screen is, for example, a virtual screen for generating a projection image that is arranged and set in an object space that is a virtual three-dimensional space, corresponding to the screen 20 in FIGS. In the image generation unit 120, distortion correction processing (also referred to as projection mapping processing) is performed in accordance with the shape of the projection screen. This projection screen can be composed of one curved surface or a plurality of surfaces (curved surface, flat surface).

  The drawing object is a two-dimensional object corresponding to the three-dimensional object to be drawn. For example, a three-dimensional object means an object arranged in a three-dimensional space (object space), and is an object having, for example, a three-dimensional coordinate value (X, Y, Z coordinate value) as a coordinate value of its vertex. On the other hand, a drawing object is an object having, for example, a two-dimensional coordinate value (X, Y coordinate) as a coordinate value of its vertex. The drawing buffer 176 is a buffer capable of storing image information in units of pixels, such as a frame buffer and a work buffer.

2. Next, the method of this embodiment will be described in detail. In the following, the case where the projection screen is a curved screen (dome-shaped screen) will be mainly described as an example, but the present embodiment is not limited to this. A projection screen refers to any screen that is not a single plane. For example, the projection screen may be a screen composed of one curved surface or a plurality of surfaces (plane, curved surface). That is, as the screen, a screen configured by one curved surface, a screen configured by a plurality of planes, a screen including a curved surface portion and a plane portion, and the like can be assumed.

2.1 Distortion Correction Method in Pixel Unit First, a method for generating a projection image for projection will be described. For example, when the projection screen is a screen composed of one curved surface or a plurality of surfaces, in this embodiment, a distortion correction process based on the shape information of the projection screen is performed to generate a projection image. Hereinafter, a specific example of a distortion correction method performed when generating a projection image will be described.

  When projecting an image on a dome-shaped (curved surface) screen, if the position of the projection device and the position of the player (observer) are separated, image distortion will be conspicuous. Therefore, by taking this distortion into consideration in advance and generating a projection image from the projection apparatus (an image drawn in the drawing buffer of the projection apparatus), it is possible to present an image without distortion as viewed from the player.

  In this case, if the screen is a single plane, the distortion becomes linear (distortion due to the perspective), and thus correction can be easily performed by using only one projective transformation matrix.

  However, in the case of a screen that is not a single plane (a screen composed of one curved surface or a plurality of surfaces), distortion that is not linear is added, so a simple correction method that uses only one projection transformation matrix is not possible. The distortion cannot be corrected, and it is necessary to perform fine correction according to the portion of the image.

  As a technique for realizing such distortion correction, a technique for performing distortion correction in units of pixels of the drawing buffer and a technique for performing distortion correction in units of vertices of a three-dimensional object are conceivable. First, a distortion correction method for each pixel will be described.

In the distortion correction method for each pixel of the drawing buffer, the processes (1), (2), (3), and (4) shown in FIG.
(1) The pixel PX (XP, YP) on the drawing buffer (frame buffer) obtains a light ray RY emitted through the lens of the projection device.
(2) The position of the intersection PS (XS, YS, ZS) where the ray RY intersects the screen SC is obtained. For example, when the screen SC is represented by an equation such as an elliptical equation, the intersection PS is obtained using an equation representing the straight line RY and an equation representing the screen SC. A mathematical expression representing the screen SC is shape information of the screen SC.
(3) The color of the intersection PS (XS, YS, ZS) needs to be a color when the player (observer) observes the object space (virtual space). Therefore, a straight line LV connecting the position VP of the virtual camera VC corresponding to the representative viewpoint of the player and the position of the intersection PS (XS, YS, ZS) is obtained.
(4) Using the straight line LV as the line of sight of the virtual camera VC, the color of the pixel of the projected image on the drawing buffer is determined from the information in the object space. For example, the position of the point PB (XB, YB, ZB) on the three-dimensional object OB that reaches first in the object space is obtained, and the color of the pixel of the projection image on the drawing buffer is determined accordingly.

  In this case, in the improvement method of the distortion correction method in pixel units, instead of obtaining the color of the intersection PB (XB, YB, ZB) between the straight line LV and the three-dimensional object OB in the last (4), The color of the pixel of the projection image is determined by using the color of the intersection (render texture coordinates (U, V)) between the plane (render texture) drawn in advance and the straight line LV. A render texture can be created by selecting a plane as close as possible to the projection plane (hereinafter referred to as a proxy plane) in advance and performing a normal drawing method, that is, drawing with the plane as the projection plane.

  FIG. 5 shows an example of such proxy planes PL1 and PL2. Point PP is the intersection of straight line LV and proxy plane PL1 (PL2).

  The render texture reference position only needs to be calculated once, as long as the viewpoint and the position of the projection device do not change. As a typical method for storing the data, for each pixel of the drawing buffer, a pixel value (texel value) at which position (U, V) of the render texture is referred to one sheet. There is a method to keep it as a texture. This texture is called a “UV map”. FIG. 6 shows the relationship among the drawing buffer, UV map, and render texture of the projection apparatus.

  Obtaining and storing the corresponding points (UV coordinates) of the render texture of FIG. 6 for all the pixels on the drawing buffer requires a large resource. If there is no room to prepare such a large resource, the corresponding point (XP, YP) on the drawing buffer and the corresponding point (U, V) on the render texture for the representative vertex appropriately arranged on the screen SC. And make a mesh with triangular polygons connecting them. When the triangle polygon is drawn, the (U, V) coordinates recorded as information at the vertices of the triangle polygon are referred to, and the (U, V) coordinates interpolated therefrom are used for the points inside the triangle polygon. Like that. In this way, resources can be saved significantly.

2.2 Distortion Correction Method for Each Vertex Next, a method for correcting distortion for each vertex of a drawing object will be described. In this method, the vertex of the three-dimensional object in the object space is converted into a point on the drawing buffer of the projection apparatus. Specifically, the processes (1), (2), (3), and (4) shown in FIG. This corresponds to performing the process of the method of FIG. 4 in the reverse order.
(1) A straight line LV connecting the position of the vertex V (XV, YV, ZV) of the three-dimensional object OB in the object space and the position VP of the virtual camera VC corresponding to the representative viewpoint of the player is obtained.
(2) The position of the intersection PS (XS, YS, ZS) between the obtained straight line LV and the screen SC is obtained. For example, when the screen SC is represented by an equation such as an equation of an ellipsoid, the intersection PS is obtained using an equation representing the straight line LV and an equation representing the screen SC. A mathematical expression representing the screen SC is shape information of the screen SC.
(3) A straight line LR connecting the position of the intersection PS (XS, YS, ZS) and the position of the projection device PJ is obtained.
(4) The position of the point PD (XD, YD) on the drawing buffer corresponding to the straight line LR is obtained. This point PD corresponds to the vertex of the drawing object OBD on the drawing buffer corresponding to the three-dimensional object OB. When obtaining the point PD from the straight line LR, information on the optical system such as the lens characteristics and arrangement of the projection device PJ is used.

  Thereafter, a projection image can be generated on the drawing buffer by performing a rasterizing process for connecting the vertices of the drawing object OBD and painting colors.

  In this case, it is desirable to use the following method. That is, vertex division processing is performed on the three-dimensional object OB in the object space with the vertex division number set by the vertex division number setting unit. Then, an intersection position between the straight line connecting the vertex position of the three-dimensional object OB after the vertex division process and the position of the virtual camera VC and the projection screen SC is obtained. Then, based on the obtained intersection position, the vertex position in the drawing buffer for the drawing object OBD corresponding to the three-dimensional object OB is obtained. Then, based on the obtained vertex position of the drawing object OBD, the drawing object OBD is drawn in the drawing buffer to generate a projection image for projection. By adopting such a method, for example, a straight line in the object space is drawn as a straight line on the drawing buffer, and as a result, a situation in which a distorted image is observed can be suppressed.

2.3 Hit Determination Processing FIG. 8 shows an example of a projection image generated by the image generation system of this embodiment. In the present embodiment, a projection image as shown in FIG. 8 is generated on the drawing buffer, and the generated projection image is projected onto the screen 20 by the projection device 30. This projection image is an image that has been subjected to the above-described distortion correction in pixel units or distortion correction in vertex units.

  In the present embodiment, the projected image shown in FIG. 8 is projected onto the screen 20 in FIGS. 1 and 2, and the player plays a shooting game on a target target object such as an enemy object while viewing the projected image reflected on the screen 20. To do. That is, the light emitting unit 50 imitating a gun is held in the hand, the target target object is aimed, and a trigger (firing switch) provided in the light emitting unit 50 is pulled to perform shooting. In this case, the emitting direction of light (infrared light) from the light emitting unit 50 is set on the game as the direction in which the bullet of the gun flies, and hit determination processing between the target target object and the bullet (emitted light) is performed. Is called. When a bullet hits, a known shot effect process such as destruction of the target target object is performed.

  As shown in FIG. 8, an image of an object constituting a game image such as a tree or a rock is displayed on the projection image. In addition, aiming objects SG1 and SG2 are also displayed in the projection image. The aiming objects SG1 and SG2 are objects representing the aiming of the light emitting unit 50 simulating a gun. When the player aims at the aiming object (SG1 or SG2) and pulls the trigger of the light emitting unit 50, a bullet (light ray) will fly in the direction of the aiming object. FIG. 8 shows an example of a game image when two players are multi-playing. SG1 is an aiming object for the first player, and SG2 is an aiming object for the second player. These aiming objects SG1 and SG2 are objects (three-dimensional objects) arranged in the object space, and the arrangement positions of the aiming objects SG1 and SG2 in the object space are obtained based on the emitting direction of the light emitting unit 50, for example. It is done. Therefore, the images of the aiming objects SG1 and SG2 are also images that have been subjected to the above-described distortion correction in units of pixels or vertices.

  9A and 10A are explanatory diagrams of a shooting game realized by a conventional image generation system, and FIGS. 9B and 10B are realized by the image generation system of the present embodiment. It is explanatory drawing of a shooting game to be performed.

  In the conventional shooting game of FIG. 9A, for example, a game image is displayed on a flat screen 520 of an LCD or CRT, and the player operates the light emitting unit 550 (gun-type controller) while viewing the game image. Play a shooting game. Since the game image is displayed on the flat screen 520, the shooting range by the light emitting unit 550 is narrow as indicated by A1.

  On the other hand, in the shooting game of this embodiment shown in FIG. 9B, a projection image is projected by the projection device 30 onto the dome-shaped (curved surface) screen 20. The player plays the shooting game while holding the light emitting unit 50 while viewing the projection image projected on the screen 20. Since the projected image, which is a game image, is projected onto the dome-shaped screen 20, the shooting range by the light emitting unit 50 is wide as shown by A2. That is, it is possible to realize a shooting game in which bullets are hit against an enemy attacking from the upper side of the screen 20 and an enemy attacking from the lower direction.

  FIG. 10A and FIG. 10B are examples of a multiplayer shooting game played by two players.

  In the conventional multiplayer shooting game of FIG. 10A, an image is displayed on a flat screen 520. Accordingly, the shooting range by the light emitting unit 550 of the first player is narrow as shown by A3, and the shooting range by the light emitting unit 552 of the second player is also narrow as shown by A4.

  On the other hand, in the multiplayer shooting game of this embodiment shown in FIG. 10B, the projection image from the projection device 30 is projected on the entire dome-shaped screen 20 provided so as to cover the first and second players. Is done. Therefore, the shooting range by the light emitting unit 50 of the first player and the shooting range by the light emitting unit 52 of the second player are very wide as indicated by A5. For example, as shown at A6, the first player can hit a bullet against an enemy that appears in the front direction of the second player. Similarly, as shown at A7, the second player can hit a bullet against an enemy that appears in the front direction of the first player.

  Thus, according to the present embodiment, the huge dome-shaped screen 20 that completely wraps the field of view of two (or one) players is employed. Therefore, it is possible to enjoy a shooting game (gun game) of a new experience that aims at the enemy attacking from the top, bottom, left and right while feeling an overwhelming immersive feeling together. Accordingly, it is possible to provide a shooting game that can be enjoyed by a wide range of customers such as couples, families, and groups.

  Now, when such a dome-shaped (curved surface) screen 20 is adopted, it is an important issue how to accurately realize the bullet hit determination process (hit determination process) by the light emitting unit 50. . In order to solve this problem, the present embodiment employs the following hit determination method.

FIG. 11 is a diagram for explaining a hit determination method according to this embodiment. In the present embodiment, the processes (1), (2), (3), (4), and (5) shown in FIG.
(1) First, the position PSL (XL, YL) on the captured image of the spot light by the light from the light emitting part GN is detected. The coordinates of the position PSL (XL, YL) are represented by, for example, a coordinate system (for example, a CCD coordinate system) of the image sensor.
That is, infrared light is emitted from the light emitting portion GN (gun-type device) of the player, and the spot light of the infrared light (spot light formed on the screen 20 in FIGS. 1 and 2) is converted into a wide-angle lens ( For example, an image is taken (photographed) by an imaging unit CM provided with a fisheye lens. Then, image analysis processing of the obtained captured image is performed to obtain coordinates (CCD coordinate system) of the position PSL (XL, YL) of the spot light (red point) of the infrared light reflected in the captured image.
(2) Next, a direction vector VSL corresponding to the spot light is obtained based on the position PSL (XL, YL) of the spot light on the captured image. That is, the direction vector VSL of the spot light viewed from the imaging unit CM is obtained. This direction vector VSL is a direction vector in the camera coordinate system of the imaging unit CM, for example.
For example, light from the spot light formed on the screen 20 in FIGS. 1 and 2 by the light emitting unit GN passes through the optical system of the imaging unit CM and is reflected at the position PSL (XL, YL) of the imaging device. Therefore, based on the information of the optical system (lens) of the imaging unit CM, the direction vector VSL (the direction opposite to the incident direction) corresponding to the incident direction of the light from the spot light can be obtained. For example, the imaging unit CM has a wide viewing angle due to the refraction effect of a wide-angle lens (fisheye lens). For example, the viewing angle is widened such that the left and right viewing angle is about 120 degrees and the vertical viewing angle is about 90 degrees. By using such a fish-eye lens that widens the viewing angle, a wide projection area of the dome-shaped screen 20 can be accommodated in the imaging range of the imaging unit CM. And when calculating | requiring the direction vector VSL, a calculation will be performed in consideration of such a viewing angle.
(3) Next, the position of the intersection SL (XC, YC, ZC) between the straight line LL along the direction vector VSL and the projection screen SC is obtained. That is, in this embodiment, the screen spot light position, which is the spot light position on the screen 20, is obtained based on the spot light position PSL (XL, YL) on the captured image. XC, YC, ZC) corresponds to the screen spot light position.
Specifically, the direction vector VSL of the camera coordinate system is converted into a real world world coordinate system (dome coordinate system). That is, the direction of the direction vector VSL in the camera coordinate system is converted into the direction in the world coordinate system, and the installation direction (imaging direction) of the imaging unit CM is reflected in the direction vector VSL. For example, as shown in FIGS. 1 and 2, the imaging unit CM is installed inclined at a predetermined angle in the downward direction, and the inclination angle and the like are reflected in the direction vector VSL.
Then, the position of the intersection SL (XC, YC, ZC) between the straight line LL and the screen SC along the direction of the direction vector VSL is obtained. For example, when the screen SC is represented by an equation such as an equation of an ellipsoid, the position of the intersection SL is obtained using an equation representing the straight line LL along the direction vector VSL and an equation representing the screen SC. A mathematical expression representing the screen SC is shape information of the screen SC. Further, the position of the obtained intersection SL (XC, YC, ZC) corresponds to the position of the spot light on the screen SC.
(4) Next, in the direction from the set position PG set as the representative position of the light emitting part GN to the position (screen spot light position) of the intersection SL (XC, YC, ZC), the light of the light emitting part GN The emission direction DG (gun direction) is determined.
This set position PG is an ideal position of the light emitting part GN assumed in the real world. For example, the set position PG can be represented by fixed coordinate values (X, Y, Z coordinates) with the seat surface position of the player seat 10 in FIGS. 1 and 2 as a reference position. That is, the coordinates of the setting position PG are determined on the assumption that the light emitting portion GN is located at the setting position PG. The set position PG may be a position set as a representative position of the player (a position of a predetermined part of the player). For example, when the position of the light emitting unit GN (or the position of the player) can be detected, the detected position may be set as the set position PG. In this case, the setting position PG is a variable position.
(5) Next, based on the obtained emission direction DG, a hit determination process with the object TOB in the object space is performed. That is, assuming that the emission direction DG (emission angle) is the direction of the light emission part GN (gun) that the player holds, the object TOB hit determination process is performed based on the straight line LG along the emission direction DG. Do. For example, the hit determination process is realized by the intersection determination between the straight line LG and the object TOB. That is, a ray along the emission direction DG is skipped, and this ray is extended to perform hit determination processing with the object TOB.

  FIG. 12 is a flowchart showing the hit determination processing of the present embodiment described above.

  First, the position PSL of the spot light on the captured image (CCD coordinate system) is obtained (step S1). Then, a direction vector VSL (camera coordinate system) corresponding to the position PSL of the spot light is obtained (step S2).

  Next, the obtained direction vector VSL is converted from the camera coordinate system to the world coordinate system (step S3). Then, the position of the intersection SL between the straight line LL and the screen SC along the direction vector VSL is obtained (step S4).

  Next, the direction from the gun setting position PG to the position of the intersection SL is obtained as the emission direction DG (step S5). Then, hit determination processing is performed using the straight line LG along the obtained emission direction DG (step S6).

  According to the method of the present embodiment described above, an appropriate hit determination process reflecting the dome shape and the like can be realized in the image generation system that projects the projection image onto the dome-shaped screen 20.

  That is, in the present embodiment, the screen spot light position that is the position of the spot light on the screen SC is obtained from the position PSL of the spot light on the captured image. Specifically, the direction vector VSL corresponding to the spot light position PSL on the captured image is obtained, and the position of the intersection SL with the screen SC is obtained as the screen spot light position. Therefore, the position of the intersection SL reflecting the shape of the dome-shaped screen SC can be obtained. In addition, for example, when the shape of the screen SC is changed, this can be easily handled. Further, when the direction vector VSL is obtained from the position PSL, the information of the optical system of the imaging unit CM can be used. Therefore, the position of the intersection SL that also reflects the information of the optical system of the imaging unit CM can be obtained.

  The position of the intersection SL corresponds to the position of the spot light formed on the screen 20 by the light emitting part GN in the real world. Accordingly, by setting the light emission direction DG of the light emitting portion GN using the position of the intersection SL and performing hit determination processing with the object TOB, appropriate hit determination processing reflecting the dome shape and the like can be realized. It becomes like this.

  For example, as a method of a comparative example of the present embodiment, a projected image is projected onto a single flat screen composed of one plane, and the position of the spot light formed on the single flat screen by the light emitting unit A method is conceivable in which hit detection is performed by detecting based on the captured image, obtaining the light emission direction only based on the position of the spot light.

  However, in the method of this comparative example, when the screen is a single flat screen, an accurate hit determination process is possible. However, by using one curved surface or a plurality of surfaces like the projection screen of the present embodiment. In the case of using a configured screen, there is a problem that an accurate hit determination process cannot be realized. That is, in a single flat screen, the light emission direction can be uniquely determined based only on the position of the spot light on the screen, but in the projection screen in projection mapping, the light emission direction is uniquely determined. I can't.

  In this regard, in the present embodiment, a set position set as the representative position of the light emitting unit 50 or the player is prepared. Then, the direction from the set position to the screen spot light position is set as the light emission direction, and hit determination processing is executed. In this way, even when a game such as that shown in FIGS. 9B and 10B is realized using a projection screen composed of one curved surface or a plurality of surfaces, for projection. An accurate hit determination process reflecting the shape of the screen can be realized.

  Further, in the present embodiment, the position of the spot light may be detected by imaging the projection area by the imaging unit CM having a fisheye lens as a wide-angle lens. In this way, it is possible to cope with the case where the screen 20 has a dome shape and the projection area of the projection image is very wide.

  That is, in the present embodiment, as shown by A2 in FIG. 9B and A5 in FIG. 10B, the shooting range by the light emitting unit 50 is very wide. Therefore, for example, in FIG. 9B, it is possible to hit a bullet against an enemy attacking from above or below the screen 20. In FIG. 10B, as shown at A6, the first player can hit a bullet against an enemy appearing in the front direction of the second player. As shown at A7, The second player can hit a bullet against an enemy that appears in the front direction of the first player. For example, in FIG. 10B, the projection image from the projection device 30 is projected onto the entire dome-shaped screen 20 provided so as to cover the first and second players. Therefore, it is difficult for an imaging unit using a normal lens to detect the position of the spot light formed on such a screen 20.

  In this regard, if a fish-eye lens is provided as a wide-angle lens in the imaging unit CM, the imaging range can be set by the fish-eye lens so that the entire giant dome-shaped screen 20 as shown in FIG. The position of the spot light formed on the top can be detected appropriately. Accordingly, it is possible to realize a spot light detection system optimal for a shooting game as shown in FIGS. 9B and 10B.

2.4 Aiming object In a shooting game, an aiming object is required to aim at the enemy. In a shooting game, the aiming object is generally realized by a two-dimensional object arranged on a screen.

  However, in the image generation system that performs projection mapping onto the dome-shaped screen 20, the projected image is a deformed image reflecting the dome shape as shown in FIG. Therefore, it is difficult to display an appropriate aim on such a projection image by the method of arranging a two-dimensional aiming object on the screen as described above.

  Therefore, in the present embodiment, a method is employed in which the arrangement position of the aiming object in the object space is obtained based on the emission direction DG obtained in FIG.

  For example, in FIG. 13, the emission direction DG of the light emission part GN is obtained by the method of FIG. In this case, in the present embodiment, the arrangement position of the aiming object SG in the object space is obtained based on the emission direction DG, and the aiming object SG is arranged at the obtained arrangement position. That is, the aiming object SG is arranged not as a two-dimensional object on the screen but as a three-dimensional object (an object having three-dimensional coordinates) in an object space that is a virtual three-dimensional space. Specifically, the aiming object SG is arranged on the straight line LG along the emission direction DG. For example, in FIG. 13, the aiming object SG is arranged at the position of the intersection between the straight line LG and the screen SC.

  In this way, the aiming object SG is displayed in the direction aimed by the player holding the light emitting part GN in his / her hand. For example, when the player moves the light emitting unit GN up, down, left, and right, the aiming object SG also moves up, down, left, and right in conjunction therewith. From the viewpoint of the player, when the aiming object SG overlaps the object TOB and the trigger of the light emitting part GN is pulled, the bullet from the light emitting part GN can be hit by the object TOB.

  That is, in an image generation system that performs projection mapping, it is extremely difficult to determine what shape / display mode the aiming object is to be arranged by the technique of arranging the two-dimensional aiming object on the screen.

  On the other hand, in the present embodiment, the aiming object SG is arranged in the object space by effectively utilizing the position of the intersection SL and the emission direction DG obtained for the hit determination process. By doing so, as shown in FIG. 8, images of appropriate aiming objects SG1 and SG2 reflecting the shape of the screen 20 are displayed on the projection image. Then, by projecting this projection image onto the screen 20, it is possible to display images of the aiming objects SG1 and SG2 that appear to be in an appropriate shape when viewed from the player. Therefore, it is possible to display an aiming object suitable for an image generation system that performs projection mapping.

  In FIG. 13, the aiming object SG is arranged on the straight line LG along the emission direction DG, but the present embodiment is not limited to this. For example, as shown in FIG. 14, the aiming object SG may be arranged on a straight line LSV connecting the position VP of the virtual camera VC and the position of the intersection SL. In this way, the aiming object SG can be arranged so that it looks appropriate from the viewpoint of the player.

  Further, the arrangement position of the aiming object SG is not limited to the intersection position with the screen SC. For example, as shown in FIGS. 13 and 14, the aiming object SG ′ may be arranged behind the intersection position with the screen SC. In the case of FIG. 14, the aiming objects SG and SG ′ appear to be at the same position when viewed from the viewpoint of the player (virtual camera VC), but in the case of FIG.

2.5 Direction vector VSL
Next, details of a method for obtaining the direction vector VSL will be described.

  FIGS. 15A and 15B are diagrams illustrating a method for obtaining the direction vector VSL in an ideal lens LEN such as a pinhole camera model.

  In FIG. 15A, the position PSL (x, y) of the spot light in the coordinate system of the CCD that is the image sensor is detected. Then, a direction vector VSL in the camera coordinate system is obtained from the position PSL (x, y) in the CCD coordinate system.

For example, in FIG. 15A, the value range in the CCD coordinate system is (0 to 1), but in FIG. 15B, the range is converted from (0 to 1) to (−1 to +1). Further, in FIG. 15B, the direction of the Y axis is reversed with respect to FIG. The direction of the Y axis in FIG. 15B is forward with respect to the paper surface. Accordingly, when the position PSL in FIG. 15A is (x, y) and the position PSL in FIG. 15B is (X, Y), the following equation (1) is obtained.
(X, Y) = (2x-1,-(2y-1)) (1)

If the horizontal angle of view is Ax, the vertical angle of view is Ay, and the Z coordinate of the virtual screen SCV in FIG. 15B is Z = 1, the coordinates of the point PR in FIG. It is expressed as
PR = (tan (Ax / 2), 0, 1) (2)

Therefore, the coordinates of the point PV are expressed as the following formula (3).
PV = ((2x-1) × (tan (Ax / 2)),-(2y-1) × (tan (Ay / 2)), 1) (3)

The coordinates of this point PV become a direction vector VSL in the camera coordinate system as shown in the following equation (4).
VSL = ((2x-1) x (tan (Ax / 2)),-(2y-1) x (tan (Ay / 2)), 1) (4)

  FIG. 16 is a diagram for explaining a method for obtaining a direction vector VSL in an fθ lens often used for fisheye lenses.

  In B1 and B2 of FIG. 16, coordinate conversion from (x, y) to (X, Y) is performed in the same manner as in FIGS. 15A and 15B. That is, the lens center is set to the center (origin) of the CCD in accordance with the lens center. The direction of the Y axis is reversed.

In B3 of FIG. 16, since the image is reversed by the lens, the coordinates of PSL (X, Y) are reversed vertically and horizontally. The distance from the origin to PSL (X, Y) is L = (X ² + Y ² ) ^1/2 .

The fθ lens is a lens in which a light beam incident at an angle of θ forms an image at a distance fθ from the lens center (optical axis) at a position separated by a focal length f. Therefore, the following equation (5) holds for θ corresponding to L.
L = fθ (5)

In this fθ lens, the distance from the lens center (the origin in the CCD) corresponding to θ = 0 to π / 2 is 0 to M. That is, let M be the distance when a light beam incident at an angle θ = π / 2 in the fθ lens forms an image at a position separated by the focal length f. Then, the following formula (6) is established.
M = f × (π / 2) (6)

From the above equations (5) and (6), θ can be obtained as in the following equation (7).
θ = (L / M) × (π / 2) (7)

Therefore, as is apparent from B4 of FIG. 16, the direction vector VSL is obtained as in the following equation (8).
VSL = ((X / L) × sin θ, (Y / L) × sin θ, cos θ) (8)

  Various modifications can be made as a method for obtaining the direction vector VSL. For example, a known image correction process for correcting distortion caused by the fθ lens is performed on the captured image of the imaging unit 40 having an fθ lens (fisheye lens). Then, the direction vector VSL may be obtained by applying the method of FIGS. 15A and 15B to the captured image after the image correction processing. An actual lens is neither an ideal lens nor an fθ lens, and often has a distortion that cannot be expressed by a simple expression. Create a map that corrects the distortion to an image with an ideal camera model by shooting an image with a grid pattern printed in advance, and converts it to an image with an ideal camera model Generally, the technique to be used is often used, but after the conversion by the technique, the technique of FIG. 15A and FIG. 15B may be applied to obtain the direction vector VSL.

2.6 Adjustment of Spot Light Position Detection When spot light position detection is performed as in the present embodiment, position detection adjustment work (gun initialization) is required.

  For example, in FIGS. 17A to 19, spot light detection adjustment objects IT1, IT2, IT3, IT4, and IT5 are displayed on the projection image at the time of initial adjustment.

  Specifically, in FIG. 17A, the detection adjustment object IT1 is displayed on the upper left of the screen. Then, the operator (for example, an amusement facility clerk) aims at the detection adjustment object IT1 and pulls the trigger of the light emitting unit 50. Similarly, in FIGS. 17B, 18A, 18B, and 19, detection adjustment objects IT2, IT3, IT4, and IT5 are respectively displayed at the upper right, lower left, lower right, and middle of the screen. indicate. The operator then targets these detection adjustment objects IT2, IT3, IT4, and IT5 and pulls the trigger of the light emitting unit 50. FIGS. 17A to 19 are projection images on which distortion correction processing has been performed in the same manner as in FIG. 8, and the detection adjustment objects IT1, IT2, IT3, IT4, and IT5 are arranged in the object space. Since it is a dimensional object, IT1, IT2, IT3, IT4, and IT5 are deformed images.

  For example, it is assumed that the operator pulls the trigger of the light emitting unit 50 toward the detection adjustment object IT1 in a state where the detection adjustment object IT1 is displayed in FIG. Then, the difference angle between the emission direction DG (see FIG. 11) detected at this time and the target emission direction DGT is obtained. In this way, when the game is played by an actual player, the accurate emission direction DG can be obtained by correcting the detected emission direction DG with the above-described differential angle. In other words, accurate position detection of the spot light can be realized.

  For example, in FIGS. 1 and 2, there are variations in the mounting position and mounting direction of the imaging unit 40. That is, the imaging unit 40 is attached to the projection device 30 such that the imaging direction matches (substantially matches) the projection direction of the projection device 30. In addition, the imaging unit 40 is attached at a predetermined position with respect to the projection device 30. However, there are variations in the mounting position and mounting direction of the imaging unit 40 due to assembly errors during the manufacture of the game system. Then, variation also occurs in the imaging range of the imaging unit 40 in the projection area.

  For example, in FIGS. 20A and 20B, RSC is a projection area (screen area) of a projection image, and RCM is an imaging range of the imaging unit 40 in this projection area RSC. The wide-angle lens (for example, fisheye lens) of the projection device 30 is a wider-angle lens than the wide-angle lens (for example, fisheye lens) of the imaging unit 40. That is, the projection range of the projection device 30 is wider than the imaging range of the imaging unit 40. For this reason, as shown in FIG. 20A, the imaging range RCM of the imaging unit 40 is narrower than the projection area RSC of the projection image. If there are variations in the mounting position and mounting direction of the imaging unit 40, as shown in FIG. 20B, the imaging range RCM shifts, for example, in the DRA direction with respect to the projection area RSC. The direction in which the imaging range RCM shifts may be various depending on the mounting position of the imaging unit 40 and the variation in the mounting direction.

  Therefore, in the present embodiment, as shown in FIG. 21, a projected image is generated so that the spot light detection adjustment objects IT1, IT2, IT3, IT4, and IT5 are displayed inside the imaging range RCM of the imaging unit 40. . That is, the detection adjustment objects IT1, IT2, IT3, IT4, and IT5 are displayed on the inner side by a given distance from the boundary of the imaging range RCM as shown in DR1, DR2, DR3, DR4, and the like. Specifically, the detection adjustment objects IT1 to IT5 are arranged in the object space so that the detection adjustment objects IT1 to IT5 are displayed inside the imaging range RCM in the projection area RSC of the projection image.

  In this way, as shown in FIGS. 20A and 20B, when the imaging range RCM is shifted with respect to the projection region RSC due to variations in the mounting position and mounting direction of the imaging unit 40, and the like. In addition, the detection adjustment objects IT1 to IT5 can be accommodated in the imaging range RCM.

  For example, if the detection adjustment objects IT1 to IT5 are not displayed inside the imaging range RCM, adjustment work (gun initialization) as shown in FIGS. 17A to 19 cannot be executed in the first place.

  For example, it is assumed that the detection adjustment object IT3 at the lower left of the screen does not enter the imaging range RCM due to the imaging range RCM shifting in the upper right direction as shown in FIG. Then, even if the operator aims at the detection adjustment object IT3 and pulls the trigger of the light emitting unit 50 at the time of initial adjustment, the spot light aimed at the detection adjustment object IT3 is caused by the imaging unit 40. The image is not captured. Accordingly, since the emission direction DG corresponding to the detection adjustment object IT3 cannot be detected, a situation in which the above-described correction processing cannot be realized occurs.

  In this regard, in the present embodiment, as shown in FIG. 21, since the detection adjustment objects IT1 to IT5 are displayed in the inner region away from the boundary of the imaging range RCM, the occurrence of the above situation can be prevented.

  It is to be noted that a detection adjustment object or the like is displayed on the projection image, the detection adjustment object is imaged using an imaging unit capable of imaging visible light, and image analysis processing of the obtained captured image is performed. Light position detection adjustment may be performed. For example, barcode image objects serving as detection adjustment objects are displayed at the four corners of the projected image. Then, using an imaging unit capable of capturing visible light, these bar code image objects are imaged, and by performing image analysis processing of the captured image, a shift in the display position of the bar code image object is detected, and a spot is detected. Perform light position detection adjustment. In this case, an imaging unit capable of imaging visible light is realized by, for example, switching a filter of the imaging unit 40 that emits infrared light. For example, when detecting spot light of infrared light, an image is picked up through an infrared filter, and when picking up the above-described barcode image object, the image is not passed through the infrared filter. Image. Further, the detection adjustment object in this case is not limited to the barcode image object. For example, four objects having different colors may be displayed at the four corners of the projected image. Alternatively, the position detection adjustment of the spot light may be performed by irradiating the four corners of the projected image with laser or the like and performing image analysis processing of the irradiation position of the laser.

  In the present embodiment, the process of specifying the imaging range RCM of the imaging unit 40 may be performed. For example, the imaging range RCM of the imaging unit 40 is specified based on the second captured image obtained by capturing the projection image or the history information of the spot light detection position.

  For example, a projection image of a specific image pattern is displayed, and the above-described imaging unit capable of capturing visible light captures the projection image of the specific image pattern, and image analysis processing of a second captured image that is the obtained captured image Thus, the boundary of the imaging range RCM is specified. Alternatively, the operator may perform an operation of swinging the light emitting unit 50 up, down, left, and right, and the boundary of the imaging range RCM may be detected based on the history of the spot light detection position at that time. Alternatively, the boundary of the imaging range RCM may be detected on the basis of the spot light detection position history based on the operation of the player who played before the current player.

  For the aiming object SG, it is desirable to perform display control as shown in FIGS. 22 (A) and 22 (B).

  For example, as shown in FIG. 22A, when the aiming object SG is located within the imaging range RCM of the imaging unit 40, the light emission direction DG is detected by the method described in FIG. The aiming object SG can be displayed on the projection image by arranging the aiming object SG on the projection image.

  However, when the spot light is located outside the imaging range RCM, the emission direction DG cannot be detected as shown in FIG. 13, and the aiming object SG cannot be displayed at an appropriate position. That is, the imaging range RCM is narrower than the projection area RSC of the projection image. Therefore, an area RF that is inside the projection area RSC and outside the imaging range RCM is an area where a projection image exists but spot light cannot be detected. Accordingly, even if the player points the light emitting unit 50 toward the region RF so that spot light is formed in the region RF, the method of FIG. 13 cannot display the aiming object SG in the region RF. .

  Therefore, in FIGS. 22A and 22B, a method of shifting the display position of the aiming object SG in the outward direction DRS from the center of the imaging range RCM toward the outside is adopted. That is, the display position of the aiming object SG is shifted in the outward direction DRS from the display position specified by the method of FIG. For example, when the display position of the aiming object SG specified by the method of FIG. 13 is a position near the boundary of the imaging range RCM, the display position of the aiming object SG becomes a position near the boundary of the projection area RSC. Next, the display position of the aiming object SG is moved in the outer direction DRS. By doing so, the display position of the aiming object SG is not accurate, but the aiming object SG is displayed as shown in FIG. 22B even when the player points the emitting unit 50 toward the region RF. As a result, the player's unnatural feeling can be reduced.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term described at least once together with a different term having a broader meaning or the same meaning in the specification or the drawings can be replaced with the different term in any part of the specification or the drawings. In addition, a projection image generation method for projection, a spot light position detection method, a hit determination method, and the like are not limited to those described in this embodiment, and techniques equivalent to these are also included in the scope of the present invention. The present invention can be applied to various games. Further, the present invention can be applied to various image generation systems such as an arcade game system and a large attraction system in which a large number of players participate.

  In addition, the present invention can be applied to a head tracking system and glasses-type stereoscopic vision. When the head tracking system is introduced, the representative viewpoint position of the player is determined based on the position of the head, and image correction processing and aim correction processing corresponding to this position are performed, so that a more natural image and aim movement can be performed. realizable.

  In order to support stereoscopic viewing, drawing on the proxy plane and image distortion correction processing are performed by dividing the viewpoint position of the player into the left eye and the right eye, and then combining the two images as a stereoscopic video. That's fine. As described above, the representative viewpoint position may be used as a starting point of a straight line that determines the projecting direction of the bullet. However, for this processing, the representative viewpoint is not divided into the left eye and the right eye. This is performed using one point such as the center position of both eyes. As a result, stereoscopic viewing becomes possible, and it is possible to provide a more realistic video.

SC screen, PJ projector (projector), VC virtual camera,
VP virtual camera position, OB object, OBD drawing object,
The position of the spot light on the PSL captured image, CM image capturing unit, VSL direction vector,
LL, LG straight line, SL intersection, GN light emission part, PG setting position, DG emission direction,
TOB object, SG, SG1, SG2 Aiming object,
IT1-IT5 detection adjustment object, RCM imaging range, RSC projection area,
10 player seats, 12 struts, 14 frame, 16 control board 20 screen, 30 projector, 40 imaging unit, 42 lens,
44 image sensor, 50 light emitting part,
100 processing unit, 102 game calculation unit, 104 object space setting unit,
106 moving body computing unit, 108 virtual camera control unit, 110 receiving unit,
114 hit determination unit, 116 imaging range specifying unit, 120 image generation unit,
130 sound generation unit, 160 operation unit, 170 storage unit,
172 Object data storage unit, 176 drawing buffer, 180 information storage medium,
190 Display unit, 192 Sound output unit, 194 Auxiliary storage device, 196 Communication unit

Claims (12)

An object space setting unit for performing object space setting processing;
An image generation unit that generates a projection image for projection based on information of a plurality of objects arranged in the object space;
A receiving unit that receives a captured image from an imaging unit that captures a projection area of the projected image;
Based on the position of the spot light by the light from the light emitting section on the captured image, a screen spot light position that is the position of the spot light on the projection screen is obtained, and the representative position of the light emitting section or player is obtained. A direction from the set position to the screen spot light position is obtained as the light emission direction of the light emitting unit, and hit determination with an object in the object space is determined based on the obtained emission direction. A hit determination unit for processing;
Only including,
The projection screen is a screen composed of one curved surface or a plurality of surfaces,
The image generation unit
Performing distortion correction processing based on the shape information of the projection screen to generate the projection image;
The hit determination unit
An image generation system characterized in that the screen spot light position is obtained based on shape information of the projection screen . In claim 1 ,
The hit determination unit
Based on the position of the spot light on the captured image, a direction vector of the spot light viewed from the imaging unit is obtained, and an intersection position between the straight line along the direction vector and the projection screen is determined as the screen. An image generation system characterized by obtaining a spot light position. In claim 1 or 2 ,
The projected image is an image projected onto the projection screen by a projection device,
The image generation unit
The pixel on the drawing buffer, with a straight line connecting the intersection position of the light beam emitted through the optical system of the projection device and the projection screen and the representative viewpoint position as a virtual camera line of sight. An image generation system characterized by determining the color of the image. In claim 1 or 2 ,
The image generation unit
Based on the intersection position between the projection screen and the straight line connecting the vertex position of the object and the representative viewpoint position in the object space, the vertex position on the drawing buffer for the drawing object corresponding to the object is obtained, An image generation system for drawing the drawing object in the drawing buffer based on a vertex position. In any one of Claims 1 thru | or 4 ,
The object space setting unit
An image generation system characterized in that an arrangement position of the aiming object representing the aim of the light emitting unit in the object space is obtained based on the emission direction, and the aiming object is arranged at the obtained arrangement position. In claim 5 ,
The object space setting unit
An image generation system, wherein the aiming object is arranged on a straight line along the emission direction. In any one of Claims 1 thru | or 6 .
The image generation unit
The image generation system, wherein the spot light detection adjustment object generates the projection image displayed inside an imaging range of the imaging unit. In claim 7 ,
The object space setting unit
An image generation system, wherein the detection adjustment object is arranged in the object space so that the detection adjustment object is displayed inside the imaging range of the imaging unit in a projection area of the projection image. . In any one of Claims 1 thru | or 8 .
An image generation system comprising: an imaging range specifying unit that specifies an imaging range of the imaging unit based on history information of a second captured image obtained by capturing the projected image or the detection position of the spot light. In any one of Claims 1 thru | or 9 ,
Including the imaging unit;
The imaging unit
An image generation system comprising an imaging element and a fisheye lens. In claim 10,
A projection device for projecting the projection image onto the projection screen ;
The image generation system, wherein the imaging unit is attached to the projection device. An object space setting unit for performing object space setting processing;
An image generation unit that generates a projection image for projection based on information of a plurality of objects arranged in the object space;
A receiving unit that receives a captured image from an imaging unit that captures a projection area of the projected image;
Based on the position of the spot light by the light from the light emitting section on the captured image, a screen spot light position that is the position of the spot light on the projection screen is obtained, and the representative position of the light emitting section or player is obtained. A direction from the set position to the screen spot light position is obtained as the light emission direction of the light emitting unit, and hit determination with an object in the object space is determined based on the obtained emission direction. As a hit determination unit that performs processing,
Make the computer work ,
The projection screen is a screen composed of one curved surface or a plurality of surfaces,
The image generation unit
Performing distortion correction processing based on the shape information of the projection screen to generate the projection image;
The hit determination unit
A program characterized in that the screen spot light position is obtained based on shape information of the projection screen .