JP2012101024A - Game program, image processing device, image processing system, and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To assist a user in acquisition, collection, etc. of face images and to enable the user to enjoy the virtual world reflecting the real world with the face images.SOLUTION: The face image is obtained during a predetermined game or before the start of the predetermined game to create a first character object including the face image. In the predetermined game, a game related to the first character object is advanced in accordance with the player's operation, and when a success in the game related to the first character object is determined at least, the face image is stored in an accumulating manner in a storage area.

Description

  The present invention relates to a game program, an image processing device, an image processing system, and an image processing method.

  Conventionally, an apparatus that displays an image obtained by combining a real world and a virtual world has been proposed (see, for example, Patent Document 1). In addition, an information processing technique such as a game using a user image, which is information obtained in the real world, has been proposed (see, for example, Patent Document 2). In this technique, image data is captured at a timing when a user image (such as an image of a human face area) included in the image data satisfies a predetermined condition while the game is in progress.

JP 2006-72669 A JP 2010-142592 A

  An image of a face area (hereinafter referred to as a face image) is an image of the most characteristic part of a person or a living thing, and is extremely useful as information for reflecting the real world in a virtual world. However, in the conventional technology, the feature of the face image that the situation in the real world can be reflected in the virtual world has not been fully utilized.

  An object of the present invention is to support acquisition and collection of face images by a user, and to represent a virtual world that reflects the real world by the face image.

  In order to achieve the above object, the present invention may employ the following configuration, for example. When interpreting the description of the claims, it is understood that the scope should be interpreted only by the description of the claims, and the description of the claims and the description in this column are contradictory. In that case, priority is given to the claims.

  One configuration example of the game program of the present invention is executed by a computer of a game device that displays an image on a display device. The game program causes a computer to execute an image acquisition step, a step of creating a first character object, a first game processing step, a determination step, and a step of accumulatively storing in a second storage area. In the image acquisition step, a face image is acquired during a predetermined game or before the game is started and temporarily stored in the first storage area. The step of creating the first character object creates a first character object that is a character object including a face image stored in the first storage area. In the first game processing step, a game related to the first character object in a predetermined game is advanced according to the operation of the player. The determination step determines success in the game related to the first character object. The step of accumulatively storing in the second storage area stores the face image stored in the first storage area accumulatively in the second storage area when at least the success in the game is determined in the determination step.

  According to the above, until the success in the game is determined, the player can save the face image acquired during or before the game and temporarily stored in the first storage area in the second storage area. You will experience a sense of tension. On the other hand, when the success in the game is determined, the face image is stored in the second storage area, so that the acquired face image can be used even after the game in the first game processing step is finished, for example. If possible, the player will work on the game in the first game processing step with great concentration.

  In the image acquisition step, a face image may be acquired and temporarily stored in the first storage area before the predetermined game is started.

  According to the above, the game in the first game processing step can be performed with the face image acquired before the start of the predetermined game being stored in the second storage area.

  The game program may further cause the computer to execute a step of creating the second character object. The step of creating the second character object creates a second character object including a face image selected automatically or from the face image stored in the second storage area. In this case, in the first game processing step, a game related to the second character object in the predetermined game may be further advanced according to the operation of the player.

  According to the above, the face image stored in the second storage area can be used for the game. Therefore, a game related to the character object including the face image acquired in the predetermined game this time and the character object including the face image saved up to the previous time can be progressed, so that various expressions can be made in the game.

  The game program may further cause the computer to execute a step of creating a second character object and a second game processing step. The step of creating the second character object creates a second character object including a face image selected automatically or from the face image stored in the second storage area. In the second game processing step, a game related to the second character object is advanced according to the operation of the player.

  According to the above, the player can create the second character object including the selected face image from the face images stored in the second storage area, and execute the game in the second game processing step. In other words, the player can enjoy the game in the second game processing step by utilizing the face image saved by the success in the game in the first game processing step. In this case, since the character object appearing in the game includes a face image selected by the player's operation or automatically selected, the player can easily bring the real world image into the virtual world. it can.

  The game device may be able to acquire an image from the imaging device. In this case, in the image acquisition step, a face image may be acquired from the imaging device before the start of the predetermined game.

  Based on the above, it is possible to cause a character object including a face image captured by the imaging device to appear in the game, and to accumulately store the face image captured by the imaging device.

  In addition, the game device obtains an image from the first imaging device that images the front direction of the outer surface of the display surface of the display device and the second imaging device that images the back direction of the display surface of the display device as the imaging device. It may be possible. In this case, the image acquisition step includes a step of preferentially acquiring the face image captured by the first imaging device over the acquisition of the face image captured by the second imaging device, and the face image from the first imaging device. May be acquired after the image data is stored in the second storage area, and allowing a face image to be acquired by imaging by the second imaging device.

  According to the above, since the face image acquisition in the first imaging device that images the front direction of the outer surface of the display surface of the display device is preferentially executed, the player of the game device or the like who looks at the display surface of the display device The possibility that the face image is acquired with priority is increased. Therefore, there is a high possibility that the act of acquiring an image from the second imaging device that captures the back direction of the display surface of the display device can be restricted in a state where the player of the game device is not specified.

  The game program may further cause the computer to execute a step of specifying an attribute of the face image and a step of prompting the player to acquire the face image. The step of specifying the attribute of the face image specifies the attribute of the face image stored in the second storage area. The step of prompting the player to acquire the face image prompts the player to acquire a face image corresponding to an attribute different from the attribute specified from the face image stored in the second storage area.

  According to the above, it is possible to support a player who wants to store face images having various attributes in an accumulative manner by reducing the attribute bias of the face images stored in an accumulative manner.

  The first game processing step may include a step of advancing a game related to the first character object by attacking the character object in accordance with an operation of the player. In this case, in the first game processing step, an attack on the first character object is an effective attack for making the game related to the first character object successful, and an attack on the second character object is a success of the game. On the other hand, it may be an invalid attack.

  According to the above, since it is necessary for the player to suppress the attack on the second character object in the game by the first game processing step, the player selects and attacks the first character object. Therefore, the player is required to accurately recognize the first character object, and is required to concentrate.

  The game program may further cause the computer to execute a step of creating the third character object. The step of creating the third character object creates a third character object including a face image different from the face image included in the second character object. In this case, the second game processing step may include a step of advancing a game related to the second character object by attacking the character object in accordance with the operation of the player. In the second game processing step, an attack on the second character object is an effective attack for making the game related to the second character object successful, and an attack on the third character object is invalid for the success of the game May be considered as a serious attack.

  According to the above, since it is necessary for the player to suppress the attack on the third character object in the game by the second game processing step, the player selects and attacks the second character object. Therefore, the player is required to accurately recognize the second character object and is required to concentrate.

  The game program may further cause the computer to execute a step of creating a third character object and a step of creating a fourth character object. The step of creating the third character object creates a third character object that includes the face image stored in the first storage area and has a smaller dimension than the first character object. The step of creating the fourth character object creates a fourth character object that includes a face image different from the face image stored in the first storage area and has a size smaller than that of the first character object. In this case, the first game processing step includes a step of advancing a game related to the first character object by attacking the character object according to the operation of the player, and a case where the fourth character object is attacked, The step of proceeding with the deformation of the face image included in one character object and when the third character object is attacked, the face image included in the first character object is changed to the original face image stored in the first storage area. A step of retracting the deformation so as to approach.

  Based on the above, the player is required to accurately recognize the first character object, the third character object, and the fourth character object, and is required to concentrate. In particular, when the acquired face image is a face image of a person who has a close relationship with the player, the player can more and more eagerly tackle the game in the first game processing step.

  The game program may further cause the computer to execute a step of creating a third character object and a step of creating a fourth character object. The step of creating the third character object creates a third character object that includes the same face image as the face image contained in the second character object and has a smaller dimension than the second character object. The step of creating the fourth character object creates a fourth character object that includes a face image different from the face image included in the second character object and has a smaller size than the second character object. In this case, the second game processing step includes a step of advancing a game related to the second character object by attacking the character object according to the operation of the player, and a case where the fourth character object is attacked, The step of proceeding with the deformation of the face image included in the two character object and when the third character object is attacked, the face image included in the second character object is changed to the original face image stored in the second storage area. A step of retracting the deformation so as to approach.

  According to the above, the player is required to accurately recognize the second character object, the third character object, and the fourth character object, and is required to concentrate.

  In the step of creating the first character object, a character object including a face image obtained by deforming the face image stored in the first storage area may be created as the first character object. In this case, the first game processing step includes a step of returning the deformed face image to the original face image stored in the first storage area when the game related to the first character object is successful. May be included.

  According to the above, for example, when the acquired face image is a face image of a person who is intimately related to the player, the player is more eager to return the deformed face image to the original face image. The game can be tackled at the game processing step.

  In the step of creating the second character object, a character object included in a face image obtained by deforming the face image stored in the second storage area may be created as the second character object. In this case, the second game processing step includes a step of returning the deformed face image to the original face image stored in the second storage area when the game related to the second character object is successful. May be included.

  According to the above, for example, when the acquired face image is a face image of a person who has a close relationship with the player, the player is more eager to return the deformed face image to the original face image. The game can be tackled at the game processing step.

  In the image acquisition step, a face image may be acquired and temporarily stored in the first storage area during a predetermined game. In this case, in the first game processing step, the first character object is made to appear in the predetermined game in response to the face image being acquired and the first character object being created during the predetermined game. A game related to the character object may be advanced.

  According to the above, it is possible to play the game in the first game processing step as a target for saving the face image acquired during the predetermined game in the second storage area.

  The game program may further cause the computer to execute a captured image acquisition step, a display image generation step, and a display control step. In the captured image acquisition step, a captured image captured by the real camera is acquired. The display image generation step generates a display image in which virtual character objects appearing in a predetermined game are arranged with the captured image acquired in the captured image acquisition step as a background. The display control step displays the image generated in the display image generation step on the display device. In this case, in the image acquisition step, at least one face image may be extracted from the captured image displayed on the display device during a predetermined game and temporarily stored in the first storage area.

  According to the above, the face image included in the captured image of the real world displayed as the background appears as a character object, and the face image is accumulated in accordance with the success in the game related to the character object. Can do.

  In the display image generation step, the display image may be generated by arranging the first character object so that the face image extracted in the image acquisition step is displayed on the display device so as to overlap the position in the captured image. Good.

  According to the above, the first character object can be expressed as if it appeared from within the captured image, and an image as if the first character object existed in the real space captured by the real camera is displayed. Can do.

  Further, in the captured image acquisition step, a real-world captured image captured in real time by a real camera may be repeatedly acquired. In the display image generation step, the captured images repeatedly acquired in the captured image acquisition step may be sequentially set as the background. In the image acquisition step, face images corresponding to face images that have already been extracted may be repeatedly acquired from captured images that are sequentially set as a background. In the step of creating the first character object, the first character object may be repeatedly created so as to include the face image repeatedly obtained in the image obtaining step. In the display image generation step, the first character object that is repeatedly created is arranged so as to be displayed on the display device so as to overlap the position in the captured image of the face image repeatedly obtained in the image acquisition step. A display image may be generated.

  According to the above, even if the imaging position and imaging direction of the real camera change, the first character object can be arranged according to the change, and the position and expression of the person who is the subject from which the face image is acquired change. However, since the change can be reflected in the first character object, it can be displayed as if the first character object exists in the real space represented by the captured image captured in real time.

  The game device may be able to use image data stored in a storage unit that stores data non-temporarily. In this case, in the image acquisition step, at least one face image may be extracted from the image data stored in the storage unit before the predetermined game is started and temporarily stored in the first storage area.

  According to the above, since a face image is acquired from image data stored in advance in the game device, a face image acquired in advance by another application or the like (for example, an image captured by a camera imaging application or a communication application) Thus, a face image included in an image received from another device can be an acquisition target.

  Further, the present invention may be implemented in the form of an image processing apparatus and an image processing system provided with means for executing the above steps, and an image processing method including operations performed in the respective steps.

  According to the present invention, it is possible to support acquisition and collection of a face image by a user and to express a virtual world reflecting the real world by the face image.

Front view showing an example of the game apparatus 10 in an opened state Side view showing an example of the game apparatus 10 in the opened state Left side view showing an example of the game apparatus 10 in the closed state Front view showing an example of the game apparatus 10 in a closed state A right side view showing an example of the game apparatus 10 in a closed state Rear view showing an example of the game apparatus 10 in a closed state The figure which shows an example of a mode that a user holds the game apparatus 10 with both hands. The figure which shows an example of a mode that a user holds the game apparatus 10 with one hand. Block diagram showing an example of the internal configuration of the game apparatus 10 List of face images displayed as a list Other examples of face images listed Screen example of processing to insert face image into enemy object The figure which illustrates a face image selection screen The figure which shows an example of the various data memorize | stored in main memory according to performing the image processing program in this embodiment with the game device of FIG. An example of the data structure of face image management information Example of summary table that classifies acquired face images by attribute The flowchart which shows an example of operation | movement of the game device 10 which concerns on 1st Embodiment. The flowchart which shows an example of the detailed process of a face image acquisition process The flowchart which shows an example of operation | movement of the game device 10 which concerns on 2nd Embodiment. Flow chart showing an example of detailed processing of list display processing The flowchart which shows an example of the detailed process of a casting determination process The flowchart which shows the detailed process example of the face image management assistance process 1 The flowchart which shows the detailed process example of the face image management assistance process 2 The flowchart which shows the detailed process example of the face image management assistance process 3 The figure which shows the outline of the virtual space which is an example of the image processing program Diagram showing relationship between screen model and α texture A diagram showing an example of a virtual space The figure which shows the virtual three-dimensional space (game world) defined in the game program which is an example of an image processing program The figure which shows an example of each process step of the example of a display form displayed on upper LCD of the game device which is an example of the apparatus which performs an image processing program The figure which shows an example of each process step of the example of a display form displayed on upper LCD of the game device which is an example of the apparatus which performs an image processing program The figure which shows an example of each process step of the example of a display form displayed on upper LCD of the game device which is an example of the apparatus which performs an image processing program The figure which shows an example of each process step of the example of a display form displayed on upper LCD of the game device which is an example of the apparatus which performs an image processing program The figure which shows an example which looked at the silhouette model of a shadow object from the top Figure showing an example of a silhouette model of a shadow object Diagram showing an example of opacity of each object A flowchart showing an example of an operation of image processing by a game device by executing an image processing program Subroutine flowchart showing an example of detailed operation of enemy object related processing of the image processing program Subroutine flowchart showing an example of detailed operation of bullet object related processing of the image processing program Subroutine flowchart showing an example of detailed operation of display image update processing (first drawing method) of the image processing program Flowchart of a subroutine showing an example of detailed operation of display image update processing (second drawing method) of the image processing program in the present embodiment. Explanatory drawing for demonstrating an example of the rendering process in a 1st drawing method Explanatory drawing for demonstrating the positional relationship of each object of FIG. Explanatory drawing for demonstrating an example of the process which renders a camera image Explanatory drawing for demonstrating an example of the coordinate at the time of rendering a camera image Explanatory drawing for demonstrating an example of the process which renders virtual space FIG. 37 is an explanatory diagram for explaining the positional relationship between the objects. Explanatory drawing for demonstrating an example of the coordinate of the boundary surface at the time of rendering virtual space The figure which shows an example of the display image produced | generated by the image processing program Example of screen of game device according to second embodiment The flowchart which shows an example of operation | movement of the game device which concerns on 2nd Embodiment. Example of screen of game device according to third embodiment The flowchart which shows an example of operation | movement of the game device which concerns on 3rd Embodiment. Example of a screen according to the fourth embodiment The flowchart which shows an example of operation | movement of the game device which concerns on 4th Embodiment. The figure which shows an example of the screen displayed on upper LCD of the game device which concerns on 5th Embodiment The figure which shows an example of the screen displayed on upper LCD of the game device which concerns on 5th Embodiment The subroutine which shows an example of detailed operation | movement of the in-game face image acquisition process performed by executing the image processing program which concerns on 5th Embodiment Subroutine showing an example of detailed operation of the non-appearance process performed in step 202 of FIG. Subroutine showing an example of detailed operation of the appearance processing performed in step 208 of FIG.

(First embodiment)
Hereinafter, an image processing apparatus that executes an image processing program according to an embodiment of the present invention will be described with a specific example. However, the following embodiments are merely examples, and the present invention is not limited to the configurations of the following embodiments.

  In the embodiment described below, data processed by a computer is exemplified using a graph or natural language. More specifically, a pseudo language, a command, a parameter, a machine language, an array that can be recognized by a computer. Etc. The present invention does not limit the data representation method.

<Example of hardware configuration>
First, with reference to the drawings, a portable game apparatus 10 will be described as an example of an image processing apparatus that executes an image processing program according to the present embodiment. However, the image processing device according to the present invention is not limited to a game device. The image processing apparatus according to the present invention may be any computer system such as a general-purpose computer.

  The image processing program according to the present embodiment shown below is a game program, but the image processing program according to the present invention is not limited to the game program. The image processing program of the present invention can be applied by being executed by an arbitrary computer system. In addition, each processing of the present embodiment may be distributed by a plurality of networked devices, or may be performed by a network system that distributes processing results to a terminal after performing main processing by a server or a so-called cloud network. You may implement.

  1, 2, 3 </ b> A, 3 </ b> B, 3 </ b> C, and 3 </ b> D are plan views illustrating an example of the appearance of the game apparatus 10. The game apparatus 10 shown in FIGS. 1 to 3D has a built-in imaging unit (camera), captures an image by the imaging unit, displays the captured image on a screen, and saves data of the captured image. It is possible to do. The game apparatus 10 can execute a game program stored in a replaceable memory card, or a game program received from a server or another game apparatus via a network. In addition, the game apparatus 10 can generate an image captured by a virtual camera set in a virtual space by computer graphics processing and display it on a screen. In this specification, capturing image data with a camera is referred to as “imaging”, and storing captured image data is referred to as “imaging”.

  The game apparatus 10 shown in FIGS. 1 to 3D includes a lower housing 11 and an upper housing 21. The lower housing 11 and the upper housing 21 are connected to each other by a hinge structure so as to be openable and closable (foldable). That is, the upper housing 21 is attached to the lower housing 11 so as to be rotatable (swingable). Thereby, the game apparatus 10 is in a closed state (FIGS. 3A and 3C) in which the upper housing 21 is in close contact with the lower housing 11, and the upper housing 21 rotates with respect to the lower housing 11. It has two forms, a state where the outside of the contact field is released (open state). Rotation of the upper housing 21 is allowed to a position where the upper housing 21 and the lower housing 11 are substantially parallel in the open state as shown in FIG. 2 (see FIG. 2).

  FIG. 1 is a front view showing an example of the game apparatus 10 in an open state (open state). Each of the lower housing 11 and the upper housing 21 of the game apparatus 10 has a planar shape in the longitudinal direction (lateral direction (left-right direction): x direction in the figure) and short direction ((vertical direction): y direction in the figure). It is formed in the plate shape which is a horizontally long rectangle which has. A lower outer edge portion of the upper housing 21 in the longitudinal direction and an upper outer edge portion of the lower housing 11 in the longitudinal direction are coupled to each other by a hinge structure so as to be rotatable. When the user uses the game apparatus 10, the game apparatus 10 is normally opened. Then, when the user stores the game apparatus 10, the game apparatus 10 is closed. Further, the upper housing 21 can maintain a stopped state at an angle desired by the user with the lower housing 11 due to a frictional force generated at a connection portion with the lower housing 11. That is, the game apparatus 10 can make the upper housing 21 stationary at a desired angle with respect to the lower housing 11. In general, from the viewpoint of the visibility of the screen provided in the upper housing 21, the upper housing 21 is opened to a position that forms a right angle or an obtuse angle with the lower housing 11. Hereinafter, when the game apparatus 10 is in the closed state, the opposing surfaces of the upper housing 21 and the lower housing 11 are referred to as an inner surface or a main surface. Moreover, the surface on the opposite side to each inner surface (main surface) of the upper housing 21 and the lower housing 11 is called an outer surface.

  On the upper long side portion of the lower housing 11 of the game apparatus 10, a protrusion (bearing portion) 11 </ b> A protruding in a direction (z direction in the drawing) perpendicular to the inner surface (main surface) 11 </ b> B of the lower housing 11. Is provided. Further, the lower long side portion of the upper housing 21 is provided with a protrusion (bearing portion) 21A that protrudes from the lower surface of the upper housing 21 in a direction perpendicular to the lower surface. In the protrusions 11A, 21A, 11A, for example, a rotating shaft (not shown) that extends in the x direction from one protrusion 11A through the protrusion 21A to the other protrusion 11A is accommodated. The upper housing 21 is rotatable relative to the lower housing 11 around the rotation axis. In this way, the lower housing 11 and the upper housing 21 are connected so as to be foldable.

  An inner side surface 11B of the lower housing 11 shown in FIG. 1 has a lower LCD (Liquid Crystal Display) 12, a touch panel 13, operation buttons 14A to 14L, an analog stick 15, a first LED 16A, and a microphone. A service hole 18 is provided.

  The lower LCD 12 is housed in the lower housing 11. The planar shape of the lower LCD 12 is a horizontally long rectangle, and is arranged so that the long side direction thereof coincides with the longitudinal direction of the lower housing 11 (x direction in FIG. 1). The lower LCD 12 is provided at the center of the inner surface (main surface) of the lower housing 11. The screen of the lower LCD 12 is exposed from an opening provided on the inner surface of the lower housing 11. When the game apparatus 10 is not used, it is possible to prevent the screen of the lower LCD 12 from being soiled or damaged by keeping the closed state. The number of pixels of the lower LCD 12 is, for example, 320 dots × 240 dots (horizontal × vertical). Unlike the upper LCD 22 described later, the lower LCD 12 is a display device that displays an image in a planar manner (not stereoscopically viewable). In the first embodiment, an LCD is used as the display device. However, other display devices such as a display device using EL (Electro Luminescence) may be used. A display device having a desired resolution can be used as the lower LCD 12.

  The touch panel 13 is one of input devices of the game apparatus 10. The touch panel 13 is mounted so as to cover the screen of the lower LCD 12. In the first embodiment, the touch panel 13 is a resistive film type touch panel. However, the touch panel 13 is not limited to the resistive film type, and any pressing type touch panel such as a capacitance type can be used. In the first embodiment, the touch panel 13 has the same resolution (detection accuracy) as the resolution of the lower LCD 12. However, the resolution of the touch panel 13 and the resolution of the lower LCD 12 are not necessarily matched.

  Each operation button 14A-14L is an input device for performing a predetermined input. On the inner surface (main surface) of the lower housing 11, among the operation buttons 14A to 14L, the cross button 14A (direction input button 14A), button 14B, button 14C, button 14D, button 14E, power button 14F, select A button 14J, a HOME button 14K, and a start button 14L are provided.

  The cross button 14A has a cross shape, and has buttons for instructing at least the vertical and horizontal directions. The cross button 14 </ b> A is provided in a lower area in the area on the left side of the lower LCD 12. The cross button 14 </ b> A is designed at a position where it can be operated with the thumb of the left hand holding the lower housing 11.

  The four buttons, the button 14B, the button 14C, the button 14D, and the button 14E, are arranged in a cross shape in the upper part of the area on the right side of the lower LCD 12. The button 14B, the button 14C, the button 14D, and the button 14E are installed at a position where the thumb of the right hand of the user holding the lower housing 11 is naturally positioned. The power button 14 </ b> F is disposed in the lower part of the right area of the lower LCD 12.

  The select button 14J, the HOME button 14K, and the start button 14L are installed in the lower area of the lower LCD 12, respectively.

  Functions according to a program executed by the game apparatus 10 are appropriately assigned to the buttons 14A to 14E, the select button 14J, the HOME button 14K, and the start button 14L. For example, the cross button 14A is used for a selection operation, a character movement operation during a game, and the like. For example, the operation buttons 14B to 14E are used for a determination operation, a cancel operation, and the like. The power button 14F is used to turn on / off the power of the game apparatus 10.

  The analog stick 15 is a device that indicates a direction. The analog stick 15 is provided in the upper part of the region on the left side of the lower LCD 12 on the inner surface (main surface) of the lower housing 11. That is, the analog stick 15 is provided above the cross button 14A. The analog stick 15 is designed at a position where it can be operated with the thumb of the left hand holding the lower housing 11. Since the analog stick 15 is provided in the upper region, the analog stick 15 is disposed where the thumb of the left hand of the user holding the lower housing 11 is naturally positioned. The cross button 14A is disposed at a position where the thumb of the left hand of the user holding the lower housing 11 is slightly shifted downward. Therefore, the user can operate the analog stick 15 and the cross button 14 </ b> A simply by moving the thumb of the left hand holding the lower housing 11 up and down. The analog stick 15 is configured such that its key top slides parallel to the inner surface of the lower housing 11. The analog stick 15 functions according to a program executed by the game apparatus 10. For example, when a game in which a predetermined object appears in the three-dimensional virtual space is executed by the game apparatus 10, the analog stick 15 functions as an input device for moving the predetermined object in the three-dimensional virtual space. In this case, the predetermined object is moved in the direction in which the key top of the analog stick 15 slides. As the analog stick 15, an analog stick that allows analog input by tilting by a predetermined amount in any direction of up / down / left / right and oblique directions may be used.

  The four buttons 14B, 14C, 14D, and 14E and the analog stick 15 are arranged at symmetrical positions with the lower LCD 12 in between. Thus, depending on the game program, for example, a left-handed person can input a direction instruction using the four buttons of the button 14B, the button 14C, the button 14D, and the button 14E.

  1st LED16A (FIG. 1) notifies a user of the ON / OFF state of the power supply of the game device 10. FIG. The first LED 16 </ b> A is provided on the right side of the end shared by the inner surface (main surface) of the lower housing 11 and the lower surface of the lower housing 11. Thereby, the user can visually recognize whether or not the first LED 16A is lit regardless of the open / closed state of the game apparatus 10.

  The microphone hole 18 is a microphone hole built in the game apparatus 10 as a voice input device. The built-in microphone detects an external sound from the microphone hole 18. The microphone and the microphone hole 18 are provided below the power button 14 </ b> F on the inner surface (main surface) of the lower housing 11.

  An insertion port 17 (indicated by a dotted line in FIGS. 1 and 3D) of the touch pen 28 is provided on the upper side surface of the lower housing 11. A touch pen 28 used for performing an operation on the touch panel 13 can be accommodated from the insertion port 17. The input using the touch panel 13 is normally performed using the touch pen 28. However, the user's finger can be used instead of the touch pen 28.

  An insertion port 11D (indicated by a dotted line in FIGS. 1 and 3D) for inserting the game apparatus 10 and an external memory 45 in which a game program is recorded is provided on the upper side surface of the lower housing 11. Inside the insertion slot 11D, a connector (not shown) for electrically connecting the external memory 45 and the internal circuit in a detachable manner is provided. When the external memory 45 is connected to the game apparatus 10, a predetermined game program is executed by a processor included in the internal circuit. The connector and the insertion port 11D may be provided on the other side surface (for example, the right side surface) of the lower housing 11.

  A speaker hole 21E, an upper LCD 22, an inner imaging unit 24, a 3D adjustment switch 25, and a 3D indicator 26 are provided on the inner side surface 21B of the upper housing 21 shown in FIG. The inner imaging unit 24 is an example of a first imaging device.

  The upper LCD 22 is a display device capable of displaying a stereoscopically visible image. The upper LCD 22 can display a left-eye image and a right-eye image using substantially the same display area. Specifically, the upper LCD 22 is a display device in which a left-eye image and a right-eye image are alternately displayed in a horizontal direction in a predetermined unit (for example, one column at a time). Note that the upper LCD 22 may be a display device in which a left-eye image and a right-eye image are displayed alternately. The upper LCD 22 is a display device capable of autostereoscopic viewing. In this case, the upper LCD 22 is of a lenticular method or a parallax barrier method (parallax barrier method) so that the left-eye image and the right-eye image alternately displayed in the horizontal direction are seen as being decomposed into the left eye and the right eye, respectively. Is used. In the first embodiment, the upper LCD 22 is a parallax barrier display device. The upper LCD 22 uses the right-eye image and the left-eye image to display an image (stereoscopic image) that can be stereoscopically viewed with the naked eye. That is, the upper LCD 22 displays a stereoscopic image (stereoscopically viewable) having a stereoscopic effect for the user by using the parallax barrier to visually recognize the left-eye image for the user's left eye and the right-eye image for the user's right eye. can do. Further, the upper LCD 22 can invalidate the parallax barrier. When the parallax barrier is invalidated, the upper LCD 22 can display an image in a planar manner (in the sense opposite to the above-described stereoscopic view, the planar LCD is planar). (This is a display mode in which the same displayed image can be seen by both the right eye and the left eye.) As described above, the upper LCD 22 is a display device capable of switching between a stereoscopic display mode for displaying a stereoscopically viewable image and a planar display mode for displaying an image in a planar manner (displaying a planar view image). . This display mode switching is performed by a 3D adjustment switch 25 described later.

  The upper LCD 22 is housed in the upper housing 21. The upper LCD 22 is a horizontally long rectangle, and is arranged in the center of the upper housing 21 in a state where the long side direction coincides with the long side direction of the upper housing 21. For example, the screen area of the upper LCD 22 is set larger than the screen area of the lower LCD 12. Specifically, the screen of the upper LCD 22 is set to be horizontally longer than the screen of the lower LCD 12. In other words, the screen of the upper LCD 22 in which the ratio of the width in the aspect ratio of the screen of the upper LCD 22 is set larger than the ratio of the width in the aspect ratio of the screen of the lower LCD 12 is the inner surface (main surface) 21B of the upper housing 21. The screen of the upper LCD 22 is exposed from the opening provided in the inner side surface 21B of the upper housing 21. The inner surface of the upper housing 21 is covered with a transparent screen cover 27. The screen cover 27 protects the screen of the upper LCD 22 and is integrated with the upper LCD 22 and the inner side surface of the upper housing 21, thereby providing a sense of unity. The number of pixels of the upper LCD 22 is, for example, 800 dots × 240 dots (horizontal × vertical). In the first embodiment, the upper LCD 22 is described as a liquid crystal display device. However, not limited to this, for example, a display device using EL may be used. In addition, a display device having an arbitrary resolution can be used as the upper LCD 22.

  The speaker hole 21 </ b> E is a hole for outputting sound from a speaker 44 as a sound output device of the game apparatus 10. The speakers 21E are installed symmetrically across the upper LCD. Sound from a speaker 44 described later is output from the speaker hole 21E.

  The inner imaging unit 24 is an imaging unit whose imaging direction is the inward normal direction of the inner side surface 21 </ b> B of the upper housing 21. The inner imaging unit 24 includes an imaging element having a predetermined resolution and a lens. The imaging element is, for example, a CCD image sensor or a CMOS image sensor. The lens may have a zoom mechanism.

  The inner imaging unit 24 is disposed above the upper end of the screen of the upper LCD 22 on the inner side surface 21B of the upper housing 21, and the center position (upper housing 21 (screen of the upper LCD 22) in the left-right direction) On the line that bisects the line. By arranging the inner imaging unit 24 in this way, when the user views the upper LCD 22 from the front, the inner imaging unit 24 can capture the user's face from the front. The outer left imaging unit 23a and the outer right imaging unit 23b will be described later.

  The 3D adjustment switch 25 is a slide switch, and is a switch used to switch the display mode of the upper LCD 22 as described above. The 3D adjustment switch 25 is used to adjust the stereoscopic effect of a stereoscopically viewable image (stereoscopic image) displayed on the upper LCD 22. The 3D adjustment switch 25 is provided at an end shared by the inner side surface 21 </ b> B and the right side surface of the upper housing 21 so that the user can visually recognize regardless of the open / closed state of the game apparatus 10. The 3D adjustment switch 25 has a slider that can slide to an arbitrary position in a predetermined direction (for example, the vertical direction), and the display mode of the upper LCD 22 is set according to the position of the slider.

  For example, when the slider of the 3D adjustment switch 25 is disposed at the lowest point position, the upper LCD 22 is set to the flat display mode, and a flat image is displayed on the screen of the upper LCD 22. Note that the upper LCD 22 may remain in the stereoscopic display mode, and the left-eye image and the right-eye image may be planarly displayed by using the same image. On the other hand, when the slider is arranged above the lowest point position, the upper LCD 22 is set to the stereoscopic display mode. In this case, a stereoscopically viewable image is displayed on the screen of the upper LCD 22. Here, when the slider is arranged above the lowest point position, the appearance of the stereoscopic image is adjusted according to the position of the slider. Specifically, the shift amount of the horizontal position in the right-eye image and the left-eye image is adjusted according to the position of the slider.

  The 3D indicator 26 indicates whether or not the upper LCD 22 is in the stereoscopic display mode. For example, the 3D indicator 26 is an LED, and lights up when the stereoscopic display mode of the upper LCD 22 is valid. The 3D indicator 26 is provided on the inner surface 21 </ b> B of the upper housing 21 and is provided in the vicinity of the screen of the upper LCD 22. For this reason, when the user views the screen of the upper LCD 22 from the front, the user can easily view the 3D indicator 26. Therefore, the user can easily recognize the display mode of the upper LCD 22 even while viewing the screen of the upper LCD 22.

  FIG. 2 is a right side view showing an example of the game apparatus 10 in the open state. On the right side surface of the lower housing 11, a second LED 16B, a wireless switch 19, and an R button 14H are provided. The second LED 16B notifies the user of the wireless communication establishment status of the game apparatus 10 by lighting. The game apparatus 10 can perform wireless communication with other devices, and the second LED 16B lights up when wireless communication with the other devices is established. The game apparatus 10 is, for example, IEEE 802.11. It has a function of connecting to a wireless LAN by a method compliant with the b / g standard. The wireless switch 19 enables / disables this wireless communication function. The R button 14H will be described later.

  FIG. 3A is a left side view illustrating an example of the game apparatus 10 in a closed state (closed state). An openable / closable cover portion 11C, an L button 14H, and a volume button 14I are provided on the left side surface of the lower housing 11 shown in FIG. 3A. The volume button 14I is a button for adjusting the volume of the speaker 44 provided in the game apparatus 10.

  Inside the cover portion 11C, a connector (not shown) for electrically connecting the game apparatus 10 and the data storage external memory 46 (see FIG. 1) is provided. The external data storage memory 46 is detachably attached to the connector. The data storage external memory 46 is used, for example, for storing (saving) data of an image captured by the game apparatus 10. The connector and cover portion 11 </ b> C may be provided on the right side surface of the lower housing 11. The L button 14H will be described later.

  FIG. 3B is a front view showing an example of the game apparatus 10 in the closed state. An outer left imaging section 23a, an outer right imaging section 23b, and a third LED 29 are provided on the outer surface of the upper housing 21 shown in FIG. 3B.

  The outer left imaging unit 23a and the outer right imaging unit 23b each include an imaging element (for example, a CCD image sensor or a CMOS image sensor) having a predetermined common resolution and a lens. The lens may have a zoom mechanism. The imaging directions of the outer left imaging unit 23a and the outer right imaging unit 23b are both outward normal directions of the outer surface 21D. That is, the imaging direction of the outer left imaging unit 23a and the imaging direction (camera viewing axis) of the outer right imaging unit 23b are parallel. Hereinafter, the outer left imaging unit 23a and the outer right imaging unit 23b are collectively referred to as the outer imaging unit 23. The outer imaging unit 23 is an example of a second imaging device.

  The outer left imaging unit 23 a and the outer right imaging unit 23 b constituting the outer imaging unit 23 are arranged side by side in the horizontal direction of the screen of the upper LCD 22. That is, the outer left imaging unit 23 a and the outer right imaging unit 23 b are arranged so that a straight line connecting the two outer left imaging units 23 a and the outer right imaging unit 23 b is arranged in the horizontal direction of the screen of the upper LCD 22. In addition, when the user swings the upper housing 21 with respect to the lower housing 11 by a predetermined angle (for example, 90 °) and visually recognizes the screen of the upper LCD 22 from the front, the outer left imaging unit 23a displays the screen of the user who visually recognizes the screen. Located on the left side, the outer right imaging unit 23b is positioned on the right side of the user (see FIG. 1). The interval between the outer left imaging unit 23a and the outer right imaging unit 23b is set to about the interval between both eyes of a human, and may be set in a range of 30 mm to 70 mm, for example. In addition, the space | interval of the outer left imaging part 23a and the outer right imaging part 23b is not restricted to this range. In the first embodiment, the outer left imaging unit 23a and the outer right imaging unit 23b are fixed to the upper housing 21, and the imaging direction cannot be changed.

  The outer left imaging unit 23a and the outer right imaging unit 23b are arranged at positions symmetrical with respect to a line that bisects the upper LCD 22 of the upper LCD 22 (upper housing 21) to the left and right. Further, the outer left imaging unit 23a and the outer right imaging unit 23b are arranged on the upper side of the upper housing 21 and on the back side of the upper side of the screen of the upper LCD 22 with the upper housing 21 opened. (See FIG. 1). That is, the outer left imaging unit 23a and the outer right imaging unit 23b are arranged on the outer surface of the upper housing 21 and above the upper end of the projected screen of the upper LCD 22 when the upper LCD 22 is projected onto the outer surface. .

  As described above, the outer left imaging unit 23a and the outer right imaging unit 23b are arranged symmetrically with respect to the center line in the lateral direction of the upper LCD 22, so that when the user views the upper LCD 22 from the front, the outer imaging unit 23 is viewed. The respective imaging directions can be matched with the line-of-sight directions of the left and right eyes of the user. Further, since the outer imaging unit 23 is disposed at a position on the back side above the upper end of the screen of the upper LCD 22, the outer imaging unit 23 and the upper LCD 22 do not interfere inside the upper housing 21. Further, the outer left imaging unit 23a and the outer right imaging unit 23b project the inner imaging unit 24 provided on the inner surface of the upper housing 21 indicated by a dotted line in FIG. The inner imaging unit 24 is provided symmetrically. Accordingly, the upper housing 21 can be made thinner than when the outer imaging unit 23 is arranged on the back side of the screen of the upper LCD 22 or when the outer imaging unit 23 is arranged on the back side of the inner imaging unit 24. .

  The outer left imaging unit 23a and the outer right imaging unit 23b can be used as a stereo camera by a program executed by the game apparatus 10. In addition, depending on the program, one of the two outer imaging units (the outer left imaging unit 23a and the outer right imaging unit 23b) can be used alone, and the outer imaging unit 23 can be used as a non-stereo camera. When a program for causing the outer imaging units 23a and 23b to function as a stereo camera is executed, the outer left imaging unit 23a captures an image for the left eye that is visually recognized by the user's left eye, and the outer right imaging unit 23b A right-eye image that is visually recognized by the right eye is captured. Further, depending on the program, it is also possible to perform imaging with a wider imaging range by combining or complementarily using images captured by the two outer imaging units (the outer left imaging unit 23a and the outer right imaging unit 23b). Is possible. In addition, a pseudo-stereo as if the left-eye image and the right-eye image having parallax were generated from a single image captured using one of the outer imaging units 23a and 23b and captured by two cameras. An image can also be generated. In the generation of the pseudo stereo image, the distance between the virtual cameras can be set as appropriate.

  The third LED 29 is lit when the outer imaging unit 23 is operating, and notifies that the outer imaging unit 23 is operating. The third LED 29 is provided in the vicinity of the outer imaging unit 23 on the outer surface of the upper housing 21.

  FIG. 3C is a right side view showing an example of the game apparatus 10 in the closed state. FIG. 3D is a rear view showing an example of the game apparatus 10 in the closed state.

  An L button 14G and an R button 14H are provided on the upper side surface of the lower housing 11 shown in FIG. 3D. The L button 14 </ b> G is provided at the left end portion of the upper surface of the lower housing 11, and the R button 14 </ b> H is provided at the right end portion of the upper surface of the lower housing 11. Functions corresponding to the program executed by the game apparatus 10 are appropriately assigned to the L button 14G and the R button 14H. For example, the L button 14G and the R button 14H function as shutter buttons (shooting instruction buttons) of the above-described imaging units.

  Although not shown, the lower housing 11 stores a rechargeable battery that serves as a power source for the game apparatus 10, and the terminal is provided via a terminal provided on a side surface (for example, the upper side surface) of the lower housing 11. The battery can be charged.

  4 and 5 each show an example of the usage state of the game apparatus 10. FIG. 4 is a diagram illustrating an example of how the user holds the game apparatus 10 with both hands.

  In the example shown in FIG. 4, the user moves the side and outer surfaces (inner surface of the inner surface) of the lower housing 11 with the palm and middle finger, ring finger and little finger of both hands with the lower LCD 12 and the upper LCD 22 facing the user. Grasp the opposite surface. By gripping in this way, the user operates the operation buttons 14A to 14E and the analog stick 15 with the left and right thumbs while holding the lower housing 11, and performs the operation on the L buttons 14G and R14H with the left and right index fingers. Can be done.

  FIG. 5 is a diagram illustrating an example of how the user holds the game apparatus 10 with one hand. In the example shown in FIG. 5, when performing an input on the touch panel 13, the user releases one hand holding the lower housing 11 and holds the lower housing 11 only with the other hand. As a result, the input to the touch panel 13 can be performed with the one hand.

  FIG. 6 is a block diagram illustrating an example of the internal configuration of the game apparatus 10. In addition to the components described above, the game apparatus 10 includes an information processing unit 31, a main memory 32, an external memory interface (external memory I / F) 33, an external memory I / F 34 for data storage, and an internal memory 35 for data storage. , A wireless communication module 36, a local communication module 37, a real time clock (RTC) 38, an acceleration sensor 39, an angular velocity sensor 40, a power supply circuit 41, and an interface circuit (I / F circuit) 42. These electronic components are mounted on an electronic circuit board and accommodated in the lower housing 11 (or the upper housing 21).

  The information processing unit 31 is information processing means including a CPU (Central Processing Unit) 311 for executing a predetermined program, a GPU (Graphics Processing Unit) 312 for performing image processing, and the like. In the first embodiment, a predetermined program is stored in a memory (for example, the external memory 45 connected to the external memory I / F 33 or the internal data storage memory 35) in the game apparatus 10. The CPU 311 of the information processing unit 31 executes image processing and game processing described later by executing the predetermined program. Note that the program executed by the CPU 311 of the information processing unit 31 may be acquired from another device through communication with the other device. The information processing unit 31 includes a VRAM (Video RAM) 313. The GPU 312 of the information processing unit 31 generates an image in response to a command from the CPU 311 of the information processing unit 31 and draws it on the VRAM 313. Then, the GPU 312 of the information processing unit 31 outputs the image drawn in the VRAM 313 to the upper LCD 22 and / or the lower LCD 12, and the image is displayed on the upper LCD 22 and / or the lower LCD 12.

  A main memory 32, an external memory I / F 33, a data storage external memory I / F 34, and a data storage internal memory 35 are connected to the information processing unit 31. The external memory I / F 33 is an interface for detachably connecting the external memory 45. The data storage external memory I / F 34 is an interface for detachably connecting the data storage external memory 46.

  The main memory 32 is a volatile storage unit used as a work area or a buffer area of the information processing unit 31 (CPU 311). That is, the main memory 32 temporarily stores various data used in image processing and game processing, and temporarily stores programs acquired from the outside (such as the external memory 45 and other devices). In the first embodiment, for example, a PSRAM (Pseudo-SRAM) is used as the main memory 32.

  The external memory 45 is a nonvolatile storage unit for storing a program executed by the information processing unit 31. The external memory 45 is constituted by a read-only semiconductor memory, for example. When the external memory 45 is connected to the external memory I / F 33, the information processing section 31 can read a program stored in the external memory 45. A predetermined process is performed by executing the program read by the information processing unit 31. The data storage external memory 46 is composed of a non-volatile readable / writable memory (for example, a NAND flash memory), and is used for storing predetermined data. For example, the data storage external memory 46 stores an image captured by the outer imaging unit 23 and an image captured by another device. When the data storage external memory 46 is connected to the data storage external memory I / F 34, the information processing section 31 reads an image stored in the data storage external memory 46 and applies the image to the upper LCD 22 and / or the lower LCD 12. An image can be displayed.

  The data storage internal memory 35 is configured by a readable / writable nonvolatile memory (for example, a NAND flash memory), and is used for storing predetermined data. For example, the data storage internal memory 35 stores data and programs downloaded by wireless communication via the wireless communication module 36.

  The wireless communication module 36 is, for example, IEEE 802.11. It has a function of connecting to a wireless LAN by a method compliant with the b / g standard. The local communication module 37 has a function of performing wireless communication with the same type of game device by a predetermined communication method (for example, infrared communication). The wireless communication module 36 and the local communication module 37 are connected to the information processing unit 31. The information processing unit 31 transmits / receives data to / from other devices via the Internet using the wireless communication module 36, and transmits / receives data to / from other game devices of the same type using the local communication module 37. You can do it.

  An acceleration sensor 39 is connected to the information processing unit 31. The acceleration sensor 39 detects the magnitude of linear acceleration (linear acceleration) along the direction of three axes (in this embodiment, the xyz axis). The acceleration sensor 39 is provided, for example, inside the lower housing 11. As shown in FIG. 1, the acceleration sensor 39 is configured such that the long side direction of the lower housing 11 is the x axis, the short side direction of the lower housing 11 is the y axis, and the inner surface (main surface) of the lower housing 11 is. Then, the magnitude of the linear acceleration generated in each axial direction of the game apparatus 10 is detected using the vertical direction as the z-axis. The acceleration sensor 39 is, for example, a capacitance type acceleration sensor, but other types of acceleration sensors may be used. The acceleration sensor 39 may be an acceleration sensor that detects a uniaxial or biaxial direction. The information processing unit 31 receives data (acceleration data) indicating the acceleration detected by the acceleration sensor 39 and calculates the attitude and movement of the game apparatus 10.

  An angular velocity sensor 40 is connected to the information processing unit 31. The angular velocity sensor 40 detects angular velocities generated around the three axes (xyz axes in the present embodiment) of the game apparatus 10 and outputs data (angular velocity data) indicating the detected angular velocities to the information processing section 31. The angular velocity sensor 40 is provided, for example, inside the lower housing 11. The information processing unit 31 receives the angular velocity data output from the angular velocity sensor 40 and calculates the attitude and movement of the game apparatus 10.

  An RTC 38 and a power supply circuit 41 are connected to the information processing unit 31. The RTC 38 counts the time and outputs it to the information processing unit 31. The information processing unit 31 calculates the current time (date) based on the time counted by the RTC 38. The power supply circuit 41 controls power from the power supply (the rechargeable battery housed in the lower housing 11) of the game apparatus 10 and supplies power to each component of the game apparatus 10.

  An I / F circuit 42 is connected to the information processing unit 31. A microphone 43, a speaker 44, and the touch panel 13 are connected to the I / F circuit 42. Specifically, a speaker 44 is connected to the I / F circuit 42 via an amplifier (not shown). The microphone 43 detects the user's voice and outputs a voice signal to the I / F circuit 42. The amplifier amplifies the audio signal from the I / F circuit 42 and outputs the audio from the speaker 44. The I / F circuit 42 includes a voice control circuit that controls the microphone 43 and the speaker 44 (amplifier), and a touch panel control circuit that controls the touch panel 13. The voice control circuit performs A / D conversion and D / A conversion on the voice signal, or converts the voice signal into voice data of a predetermined format. The touch panel control circuit generates touch position data in a predetermined format based on a signal from the touch panel 13 and outputs it to the information processing unit 31. The touch position data indicates the coordinates of the position (touch position) where the input is performed on the input surface of the touch panel 13. The touch panel control circuit reads signals from the touch panel 13 and generates touch position data at a rate of once per predetermined time. The information processing section 31 can know the touch position where an input has been made on the touch panel 13 by acquiring the touch position data.

  The operation button 14 includes the operation buttons 14 </ b> A to 14 </ b> L and is connected to the information processing unit 31. From the operation button 14 to the information processing section 31, operation data indicating the input status (whether or not the button is pressed) for each of the operation buttons 14A to 14I is output. The information processing unit 31 acquires the operation data from the operation button 14, thereby executing processing corresponding to the input to the operation button 14.

  The lower LCD 12 and the upper LCD 22 are connected to the information processing unit 31. Lower LCD 12 and upper LCD 22 display images in accordance with instructions from information processing unit 31 (GPU 312). In the first embodiment, the information processing unit 31 displays an image for handwritten image input operation on the lower LCD 12 and displays an image acquired from either the outer imaging unit 23 or the inner imaging unit 24 on the upper LCD 22. . That is, the information processing unit 31 displays a stereoscopic image (stereoscopically visible image) using the right-eye image and the left-eye image captured by the outer imaging unit 23 on the upper LCD 22 or is captured by the inner imaging unit 24. A planar image is displayed on the upper LCD 22, or a planar image using one of the right-eye image and the left-eye image captured by the outer imaging unit 23 is displayed on the upper LCD 22.

  Specifically, the information processing section 31 is connected to an LCD controller (not shown) of the upper LCD 22 and controls ON / OFF of the parallax barrier for the LCD controller. When the parallax barrier of the upper LCD 22 is ON, the right-eye image and the left-eye image stored in the VRAM 313 of the information processing unit 31 (imaged by the outer imaging unit 23) are output to the upper LCD 22. More specifically, the LCD controller alternately repeats the process of reading pixel data for one line in the vertical direction for the image for the right eye and the process of reading pixel data for one line in the vertical direction for the image for the left eye. Thus, the right-eye image and the left-eye image are read from the VRAM 313. As a result, the image for the right eye and the image for the left eye are divided into strip-like images in which pixels are arranged vertically for each line. The strip-like images for the right-eye image and the strip-like images for the left-eye image are alternately displayed. The image arranged on the upper LCD 22 is displayed on the screen. Then, when the user visually recognizes the image through the parallax barrier of the upper LCD 22, the right eye image is visually recognized by the user's right eye and the left eye image is visually recognized by the user's left eye. As a result, a stereoscopically viewable image is displayed on the screen of the upper LCD 22.

  The outer imaging unit 23 and the inner imaging unit 24 are connected to the information processing unit 31. The outer imaging unit 23 and the inner imaging unit 24 capture an image in accordance with an instruction from the information processing unit 31, and output the captured image data to the information processing unit 31. In the first embodiment, the information processing unit 31 issues an imaging instruction to one of the outer imaging unit 23 and the inner imaging unit 24, and the imaging unit that receives the imaging instruction captures an image and obtains image data. The information is sent to the information processing unit 31. Specifically, an imaging unit to be used is selected by an operation using the touch panel 13 or the operation button 14 by the user. Then, the information processing unit 31 (CPU 311) detects that the imaging unit has been selected, and the information processing unit 31 issues an imaging instruction to the outer imaging unit 23 or the inner imaging unit 24.

  When activated by an instruction from the information processing unit 31 (CPU 311), the outer imaging unit 23 and the inner imaging unit 24, for example, capture images at a speed of 60 sheets per second. The captured images captured by the outer imaging unit 23 or the inner imaging unit 24 are sequentially sent to the information processing unit 31 and displayed on the upper LCD 22 or the lower LCD 12 by the information processing unit 31 (GPU 312). When the captured image is output to the information processing unit 31, it is stored in the VRAM 313, output to the upper LCD 22 or the lower LCD 12, and then deleted at a predetermined timing. Thus, for example, by capturing images at a speed of 60 images per second and displaying the captured images, the game apparatus 10 can view the scenery in the imaging range of the outer imaging unit 23 and the inner imaging unit 24 in real time. It can be displayed on the upper LCD 22 or the lower LCD 12.

  The 3D adjustment switch 25 is connected to the information processing unit 31. The 3D adjustment switch 25 transmits an electrical signal corresponding to the position of the slider to the information processing unit 31.

  The 3D indicator 26 is connected to the information processing unit 31. The information processing unit 31 controls lighting of the 3D indicator 26. For example, the information processing section 31 turns on the 3D indicator 26 when the upper LCD 22 is in the stereoscopic display mode.

<Description of functions>
Next, an outline of an example of game processing executed by the game apparatus 10 will be described. The game apparatus 10 obtains and stores a face image of a person, for example, through the inner imaging unit 24, the outer imaging unit 23, or the like in accordance with an operation of a user (hereinafter also referred to as a player). Provides the ability to collect. In collecting the face images, the user executes a game using the acquired face image (first game), and when the result of the game is successful, the user can save the acquired image. . It should be noted that the user can save the face image to be saved by capturing an image captured by the inner imaging unit 24 or the outer imaging unit 23 before executing the first game, or before executing the first game. The image can be acquired from an image acquired by another application, an image captured by the inner imaging unit 24, the outer imaging unit 23, or the like during the execution of the first game. As will be described later, the game apparatus 10 stores the face image acquired before or during the first game in the saved data storage area Do (see FIG. 11) accessible during the execution of the game process. save. The user can then repeat the same operation on the game apparatus 10 to collect a plurality of face images, add them to the save data storage area Do, and store them. Since the save data storage area Do is an area that can be accessed by the game apparatus 10 that is executing the game, the saved face image is stored in the save data storage area Do so that it can be used in later processing. It becomes. Furthermore, the game apparatus 10 displays a list of face images collected as a result of the first game by reading the data accumulated in the save data storage area Do. Then, the game device 10 executes a game (second game) using the face image selected by the user from the displayed list of face images or the face image automatically selected by the game device 10.

  The game (first game, second game) executed by the game apparatus 10 is, for example, a game that attacks the enemy object EO while aiming and destroys the enemy object EO. In the first embodiment, for example, the face image obtained by the user and stored in the save data storage area Do or the face image stored in the save data storage area Do is stored in the texture of the character object such as the enemy object EO. Thus, it is mapped to the enemy object EO or the like.

  First, in the first game execution stage in the first embodiment, the user can acquire a desired face image via an imaging unit such as a camera and execute the first game. Then, when the user succeeds in the first game, the acquired face images are accumulated in the save data storage area Do, displayed as a list, and can be used for the second game. Here, “accumulating” means that a new face image is added when the user acquires a new face image and succeeds in the first game.

  In the execution stage of the second game, the user can select a desired face image from the collected face images and create an enemy object EO. Then, the user can execute a game using the enemy object EO created using a desired face image, for example, a game for destroying the created enemy object EO. However, the game apparatus 10 may automatically select a face image from the saved data storage area Do, for example, and create an enemy object EO or the like regardless of the user's operation. Even in the execution stage of the first game, a character object is created using the face image already collected in the save data storage area Do, together with the character object using the face image before saving in the save data storage area Do. You may make it appear in the game. In the following, when differentiating a plurality of enemy objects EO, they are called as enemy objects EO1 and EO2. On the other hand, when the enemy objects EO1, EO2, etc. are called generically, and in a context where it is not necessary to distinguish between a plurality of enemy objects, they are called like the enemy object EO.

  Next, display examples of the game apparatus 10 according to the first embodiment will be described with reference to FIGS. FIG. 7 is an example of face images displayed as a list on the upper LCD 22 of the game apparatus 10. As described above, when the user succeeds in the first game, the acquired face images can be saved and displayed as a list as shown in FIG. The face images displayed as a list are obtained by texture mapping image data acquired by the inner imaging unit 24, the outer left imaging unit 23a, the outer right imaging unit 23b, and the like onto a three-dimensional model of a human face. For example, image data is pasted as a texture on the surface of a three-dimensional model formed by combining a plurality of polygons. However, in the game executed by the game apparatus 10, the face image is not limited to a texture image mapped to a three-dimensional model. For example, the game apparatus 10 may display image data held in a simple two-dimensional pixel array as a face image. Also, among the face images displayed as a list as shown in FIG. 7, any one or more face images are held in a simple two-dimensional pixel array, and the rest are face images texture-mapped to a three-dimensional model. There may be.

  In FIG. 7, the face image G1 is surrounded by a thick line L1. A thick line L1 indicates that the face image G1 is in a selected state. The selected state means that the user is in a state selected as a processing target by operating the operation button 14 or the like, for example. The selected state is also called a focused state. Each time the user presses the operation button 14, for example, the selected face image moves from left to right and from top to bottom. For example, in the state shown in FIG. 7, when the user presses the cross button 14A to the right, the face images in the selected state are right like the face image G1 to the face image G2 and the face image G2 to the face image G3. Transition in the direction. The transition of the selected face image means that the face image surrounded by the thick line L1 is changed on the screen of the upper LCD 22.

  In FIG. 7, the horizontal arrangement of the face images is referred to as one stage. When the rightmost face image G4 is in the selected state in this one stage and the user further presses the cross button 14A in the right direction, the face image in the selected state becomes the next lower left face image G5. . Conversely, for example, when the lower left face image G5 in the lower row is in the selected state, the user presses the cross button 14A to the left, so that the selected face image is changed from the face image G5 to the face image G4, As in the face image G3 from the face image G4, a transition is made in the upward direction and the right direction.

  However, the change of the selection state is not limited to pressing the cross button 14A to the left and right, and the selection state may be changed by pressing the button up and down. The change of the selection state is not limited to the pressing of the cross button 14A, but the selection target image is changed by pressing another operation button 14, for example, the operation button 14B (A button). It may be. Alternatively, the selected face image may be switched by operating the touch panel 13 on the lower LCD 12. For example, the game apparatus 10 displays a face image list similar to the face image list displayed on the upper LCD 22 on the lower LCD 12. Then, the game apparatus 10 may detect which face image has been selected by detecting an operation on the touch panel 13. Then, the game apparatus 10 may display the face image in the selected state, for example, the face image G1 surrounded by the thick line L1.

  As described above, the user can view a list of currently acquired face images on the screen illustrated in FIG. 7 displayed on the upper LCD 22. Further, the user can select a desired face image from the currently acquired screen list. For example, the selection state can be determined by selecting a predetermined determination button, for example, a determination button displayed on the lower LCD 12 with the touch panel 13 or the operation button 14. Further, for example, when the user presses the button 14C (also referred to as the B button), the face image list screen of the upper LCD 22 may be closed and a menu selection waiting screen (not shown) may be displayed.

  FIG. 8 shows another example of face images displayed as a list. In the example of FIG. 8, the face image G2 is in a selected state and is surrounded by a thick line L2. In the example of FIG. 8, when the face image G2 is selected, the face image related to the face image G2 reacts. For example, a heart mark is displayed in the vicinity of the face image G0, and the face image G0 closes one eye and sends a line of sight toward the face image G2 in the selected state. In addition, for example, the face image G3, the face image G7, and the like also send their lines of sight toward the face image G2 in the selected state.

  However, the reaction of the face image related to the selected face image is directed toward the face, a heart mark is displayed in the vicinity, the eye is closed with one eye, the eye is directed toward the face, etc. It is not limited only to operation. For example, a related face image may show a reaction such as laughing or making a voice. Conversely, a facial image that is not related to the selected facial image may be changed from a smiling expression to a non-laughing expression. In addition, a face image not related to the selected face image may be directed in a direction opposite to the direction of the selected face image.

  Here, the related face image may be defined, for example, at the face image acquisition stage. For example, a group for classifying face images may be set in advance, and when a face image is acquired, the user may input which group the acquired face image belongs to. Further, for example, a group of face images may be defined according to the progress of the game, and the face images may be classified. For example, when a face image G1 is newly acquired during the game using the face image G0, the face image G0 and the face image G1 are related face images belonging to the same group. You may decide that there is.

  As shown in FIG. 8, when one face image, for example, the face image G2 is selected by the operation button 14 or the like, if other face images G0, G3, G7, etc. react, the interval between the acquired face images You will be able to express a sense of familiarity. In other words, instead of simply classifying the collected face images into groups, the closeness between multiple face images can be expressed by facial expressions and reactions, and the relationship between people in the real world Intimacy can be expressed in the virtual game device 10 world.

  FIG. 9 and FIG. 10 are screen examples of processing for fitting a face image into the enemy object EO, which is one of the game characters. FIG. 9 is an example of the head selection screen of the enemy object EO. For example, when the user selects a menu list “select boss” by operating the touch panel 13 from the menu displayed on the lower LCD 12 of the game apparatus 10, the game apparatus 10 is prepared as shown in FIG. 9. A list of head shapes of the enemy object EO is displayed. However, such an operation is not limited to the operation using the touch panel 13. For example, a list of head shapes as shown in FIG. 9 may be displayed by the operation using the operation buttons 14 or the like.

  In FIG. 9, three types of head shapes H1, H2, and H3 are displayed so as to be selectable. For example, the head shape H1 includes a face part H12 formed by a three-dimensional model and a peripheral part H13 surrounding the face part H12. As shown in FIGS. 7 and 8, the enemy object EO appearing in the game is formed by texture mapping the face image on the face portion H12 which is a three-dimensional model.

  The peripheral portion H13 may have a shape that implies the characteristics of the enemy object EO appearing in the game. For example, if the peripheral portion H13 has a shape imitating a beetle, a war-like image of the enemy object EO can be expressed. In FIG. 9, three types of head shapes H1, H2, and H3 are illustrated in a list of head shapes in 2 rows and 4 columns, and an undefined mark H0 is displayed in other cases. Accordingly, the head shapes H1-H3 are merely examples, and the types of head shapes are not limited to three. For example, a new head shape may be added from a game program upgrade medium or a website that provides game parts on the Internet.

  In FIG. 9, the label LB “boss” indicates that the head shape H1 is selected by the user. The label LB can be moved by, for example, the cross button 14A. When the user moves the label LB by operating the cross button 14A or the like, for example, the head shape H2 or H3 can be selected. Then, after selecting any one of the head shapes, the selection state may be determined by the user pressing the operation button 14B (A button) or the like. By determining the selection state, the label LB is arranged, and the selection of the head shape in the selection state, for example, the head shape H1 in the example of FIG. 9, is confirmed. When the screen of FIG. 9 is displayed on the upper LCD 22, when the user presses the operation button 14C (B button), the head selection screen of the enemy object EO is closed and the previous operation screen is returned. What should I do?

  FIG. 10 is a diagram illustrating a face image selection screen. For example, when the head selection screen of the enemy object EO of FIG. 9 is displayed on the upper LCD 22 and the user determines the selection state of the head shape by pressing the operation button 14B (A button) or the like, The screen of FIG. 10 is displayed. The screen of FIG. 10 is the same as that of FIGS. 7 and 8, except that the peripheral portion H13 of the head shape selected in FIG. 9 is added to the face image in the selected state. Is different.

  In the screen of FIG. 10, the user can change the face image to be selected by operating the cross button 14A or the like. That is, the user can move the selected face image from the face image G0 to G1, and further from the face image G1 to G2. The selected face image is texture-mapped and displayed on the head-shaped face portion. For example, in the example of FIG. 10, the face image G2 is texture-mapped on the face portion H12 of the head shape H1 selected on the screen of FIG. 9, and is displayed in combination with the peripheral portion H13.

  By displaying such a combination of the peripheral portion H13 and the face image G2, the enemy object EO is temporarily created. The user imagines an image of an enemy confronting in the game by the enemy object EO temporarily displayed. The user can replace the face image of the enemy object EO by changing the face image to be selected by operating the cross button 14A or the like. That is, the user can create an enemy object EO that matches the image of fighting in the game by successively changing the faces of the enemy objects EO.

  That is, in the game apparatus 10, for example, the face images collected by succeeding in the first game as described above and accumulated in the save data storage area Do are used for the next second game. That is, in the game apparatus 10, game processing using the enemy object EO created using the collected face images is executed. For example, the game apparatus 10 executes a process called a casting determination process before the game is executed according to the user's operation, and generates an enemy object EO that matches the user's image. Alternatively, the game apparatus 10 automatically generates an enemy object EO before the game is executed. “Automatically” means that, for example, the game apparatus 10 randomly selects the required number of face images from the collected face images, that is, the number of enemy objects EO appearing in the game, and generates enemy objects EO. Say what you can do. Further, for example, the game apparatus 10 creates an enemy object EO by selecting a face image estimated to be next desired by the user based on the user's characteristics, orientation, etc., from the history of game processing executed by the user in the past. May be. The game apparatus 10 has attributes of the subject of the face image, for example, age, sex, friendship relationship (family, friend, exchange relationship at work / school / region, etc.), and subject is a creature such as a pet. The face image to be used next may be selected from the game execution history by the user up to the present time based on the ownership relationship of the subject. Further, for example, the game apparatus 10 may select a face image to be used next from a user's score in a game executed in the past.

  The game apparatus 10 executes a game process (second game) using the created enemy object EO by the designation by the user's operation or the process of the game apparatus 10. Here, as an example, the description will be made with the name of the enemy object EO. However, the character object appearing in the game is not limited to an object having a hostile relationship, and may be a friendly character object. Further, the game is not limited to a game in which there is a relationship of an enemy ally, and may be a game in which a player object simulating the user himself / herself appears. Further, for example, a game in which an object called an agent that supports a user who executes the game appears.

  The game apparatus 10 executes a game in which various character objects appear, such as the enemy object EO as described above. A face image collected when the user succeeds in the first game is pasted to the character object appearing in this game by a technique such as texture mapping. Therefore, in a game executed on the game apparatus 10, a character object including a face image collected by the user himself appears. For this reason, by using an image of a part symbolizing the characteristics of a person or a living thing such as a face image for various character objects, the user can interact with the person of the face image or the living thing of the face image. You can run games that reflect real-world relationships. For example, it is possible to execute a game with feelings such as love, favor, good feeling, and hatred.

  In the face image selection screen of FIG. 10, as in FIG. 8, when one face image is selected, other face images related to the face image that is the selection target show a reaction. do it. For example, in the example of FIG. 10, the face image G2 that is a selection target and combined with the peripheral portion 13 of the head shape H1 is related to the face image, for example, the face image G4 with a smile and eyes. Is sending. In addition, the face image G5 raises the face upward and sends an enviable line of sight to the face image G2. Further, the face images G8 and G9 are also directed toward the face image G2. On the other hand, in FIG. 10, there is no response to the face image G2 being selected except for the face images G4, G5, G8, and G9. By such a difference in reaction, it is possible to express a close relationship between a plurality of face images. As described above, the game apparatus 10 indicates the relationship between the person in the real world and the person when the face image has a specific relationship with the user, for example, when the face image belongs to a plurality of friend groups. It is possible to bring it into the virtual world to express and describe it.

<Examples of various data>
FIG. 11 is a diagram illustrating an example of various data stored in the main memory 32 in response to executing the image processing program.

  Note that a program for executing processing of the game apparatus 10 is included in a memory (for example, the data storage internal memory 35), the external memory 45, or the data storage external memory 46 built in the game apparatus 10. When the power of the game apparatus 10 is turned on, the main memory is stored from the internal memory or from the external memory 45 or the data storage external memory 46 via the external memory I / F 33 or the data storage external memory I / F 34. 32 is read and executed by the CPU 311.

  In FIG. 11, the main memory 32 stores a program read from the internal memory, the external memory 45, or the data storage external memory 46 and temporary data generated in the image processing. In FIG. 11, the data storage area of the main memory 32 includes operation data Da, real camera image data Db, real world image data Dc, boundary surface data Dd, back wall image data De, enemy object data Df, bullet object data Dg. The score data Dh, motion data Di, virtual camera data Dj, rendering image data Dk, display image data Dl, aiming cursor image data Dm, management data Dn, save data storage area Do, and the like are stored. The program storage area of the main memory 32 stores various program groups Pa constituting the image processing program.

<< Operation data Da >>
The operation data Da is data indicating operation information on the operation of the game apparatus 10 by the user. The operation data Da includes operation element data Da1 and angular velocity data Da2. The operation element data Da1 is data indicating that the user has operated an operation element such as the operation button 14 or the analog stick 15 of the game apparatus 10. The angular velocity data Da2 is data indicating the angular velocity detected by the angular velocity sensor 40. For example, the angular velocity data Da2 includes the x-axis angular velocity data indicating the angular velocity around the x-axis detected by the angular velocity sensor 40, the y-axis angular velocity data indicating the angular velocity around the y-axis, and the z-axis indicating the angular velocity around the z-axis. Angular angular velocity data is included. For example, the operation data from the operation button 14 and the analog stick 15 and the angular velocity data from the angular velocity sensor 40 are acquired every time unit (for example, 1/60 second) processed by the game apparatus 10, and the operation is performed according to the acquisition. It is stored and updated in the child data Da1 and the angular velocity data Da2.

  In the game processing described later (for example, the processing in FIG. 20A and the following), the operation data Da1 and the angular velocity data Da2 will be described using an example in which each frame is updated. It does not matter if it is updated periodically. For example, in a mode in which the operator data Da1 is updated for each cycle in which it is detected that the user has operated an operator such as the operation button 14 or the analog stick 15, and the updated operator data Da1 is used for each processing cycle. It doesn't matter. Further, the angular velocity data Da2 may be updated every angular velocity detection cycle of the angular velocity sensor 40, and the updated angular velocity data Da2 may be used every processing cycle. In this case, each cycle for updating the operator data Da1 and the angular velocity data Da2 is different from the processing cycle.

<< Real camera image data Db >>
The actual camera image data Db is data indicating an actual camera image captured by either the outer imaging unit 23 or the inner imaging unit 24. In the description of processing to be described later, a mode is used in which the actual camera image data Db is updated using the actual camera image captured by either the outer imaging unit 23 or the inner imaging unit 24 in the step of acquiring the actual camera image. Note that the cycle in which the outer imaging unit 23 or the inner imaging unit 24 captures and updates the actual camera image data Db using the captured actual camera image is a unit time of processing in the game apparatus 10 (for example, 1/60 seconds). ) Or shorter than the unit time. If the cycle of updating the actual camera image data Db is shorter than the cycle of processing in the game apparatus 10, the actual camera image data Db may be updated as appropriate independently of the processing described later. In this case, in the step of acquiring an actual camera image to be described later, the latest actual camera image indicated by the actual camera image data Db may be always used for processing. Hereinafter, in the present embodiment, it is assumed that the actual camera image data Db is data indicating an actual camera image captured by the outer imaging unit 23 (for example, the outer left imaging unit 23a).

<< Real World Image Data Dc >>
In the game process described later, for example, the game execution process in step 18 of FIG. 14, more specifically, in the processes shown in FIG. A boundary surface 3 for texture mapping of the captured real camera image is introduced. The real world image data Dc is a real world image that appears to be present on the boundary surface 3 using a real camera image captured by a real camera (the outer imaging unit 23 or the inner imaging unit 24) of the game apparatus 10. Data to generate. In a first rendering method described later, for example, the real world image data Dc includes texture data of a real camera image for pasting the real world image on a boundary surface (screen object in the display range of the virtual camera). In the second rendering method described later, for example, the real world image data Dc includes plane polygon data for generating a real world image, texture data of a real camera image for mapping to the plane polygon, and It includes data indicating the position of the planar polygon in the virtual space (position from a real-world drawing camera described later).

<< Boundary surface data Dd >>
The boundary surface data Dd is data for generating a real world image that appears to be present on the boundary surface 3 together with the real world image data Dc. In the first drawing method, for example, the boundary surface data Dd is data related to the screen object, and data for opening determination (alpha texture described later) indicating the state of each point (for example, presence or absence of an opening) constituting the boundary surface 3. Data indicating the arrangement position of the boundary surface 3 in the virtual space (coordinates of the boundary surface 3 in the virtual space) and the like. In the second drawing method, for example, the boundary surface data Dd is data for expressing an opening in the planar polygon of the real world image, and the state of each point constituting the boundary surface 3 (for example, the opening surface) Data for opening determination (corresponding to α texture data to be described later) indicating the presence / absence), data indicating an arrangement position of the boundary surface 3 in the virtual space (coordinates of the boundary surface 3 in the virtual space), and the like. The data indicating the arrangement position of the boundary surface 3 in the virtual space is, for example, a conditional expression of the sphere surface (a relational expression defining the sphere surface in the virtual space), and the existence range of the boundary surface 3 in the virtual space is Show.

  The opening determination data indicating the opening state is, for example, two-dimensional texture data (for example, a rectangular shape such as 2048 pixels × 384 pixels) in which an alpha value (opacity) of each point can be set. The alpha value is a value from “0” to “1” where “0” is the smallest and “1” is the largest. The alpha value is “0” indicating transparency and “1” indicating opaqueness. The opening determination data can indicate that a position where “0” is stored in the opening determination data is an open state, and a position where “1” is stored is a non-open state. The alpha value can be set, for example, in an image of the game world generated in the game apparatus 10 or in a pixel block unit composed of a pixel or a plurality of pixels in the upper LCD 22. In this embodiment, the non-opening area stores “a predetermined value exceeding 0 and less than 1 (0.2 is set in this embodiment)”. This data is not used when applied to real world images. When applied to a real world image, the alpha value “0.2” stored in the aperture determination data is treated as “1”. The alpha value “0.2” is used when the shadow ES of the enemy object EO is drawn. However, the setting of the alpha value and the range of values that the alpha value can take do not limit the image processing program of the present invention.

  In the first drawing method, the image processing program according to the present embodiment uses a region corresponding to the range of the virtual camera viewing space in the aperture determination data and the texture color of the real world image pasted on the boundary surface 3. By multiplying the information (pixel value), a real world image having an opening can be generated. In the second drawing method, an area corresponding to the range of the viewing space of the virtual world drawing camera and the real world image (specifically, the real world image data Dc) are used. A real world image having an opening can be generated by multiplying color information (pixel value) of rendering image data of a real camera image rendered by parallel projection described later. Because the alpha value “0” stored at the opening position is multiplied by the color information of the real world image at the position, the color information value of the real world image is “0” (completely transparent state). Because it becomes.

  In the first drawing method, as described later, an image displayed on the upper LCD 22 by rendering a virtual space image of each virtual object including an object of a real world image to which the aperture determination data is applied. Is generated.

  In the second drawing method, specifically, as described later, the virtual space image is rendered in consideration of the opening determination data. That is, based on the opening determination data, the priority of each virtual object with respect to the boundary surface (the priority with respect to the real world image) is determined, and each virtual object is rendered to generate a virtual space image. Then, the image displayed on the upper LCD 22 is generated by synthesizing the real world image and the virtual space image generated in this way.

  In the image processing program according to the present embodiment, the shape of the boundary surface 3 is a spherical surface (see FIGS. 20A and 20B). The shape of the opening determination data in this embodiment may be a rectangular shape. By mapping this rectangular opening determination data to the center of the sphere surface as shown in FIGS. 20A and 20B, each point of the opening determination data can be associated with each point on the boundary surface. it can.

  In the present embodiment, since the opening determination data is only data corresponding to the central portion of the sphere surface shown in FIG. 20A, depending on the orientation of the virtual camera (virtual world drawing camera in the second drawing method), There may be no data for opening determination. When there is no aperture determination data in this way, the real world image is drawn as it is. That is, rendering is performed assuming that “1” is set as the α value.

  Image processing of the opening created on the boundary surface 3 will be described later.

<< Back wall image data De >>
The back wall image data De is data related to the back wall BW existing in the second space 2. For example, the back wall image data De includes image data for generating an image of the back wall BW, data indicating the position of the polygon model that defines the back wall BW in the virtual space, and the like.

  The polygon model that defines the back wall BW is typically larger than the radius of the sphere shown in FIG. 20A around the vertical axis extending from the position of the virtual camera (the virtual world drawing camera in the second drawing method). It is a model having a radius and the same shape as the central part of the sphere shown in FIG. 20A. That is, the model that defines the back wall BW includes the boundary surface 3. Further, it may be a planar polygon arranged behind the planned opening position formed on the boundary surface 3. Each time an opening is formed in the boundary surface 3, a planar polygon that defines the projection plane of the opening may be arranged in the second space 2.

  Image data (texture) pasted on the polygon model of the back wall BW may be arbitrary. However, since this image data represents another space (second space 2) that exists behind the real world image, it is an image that represents unreality such as an image showing outer space, the sky, and the water. It is preferable that the player can be given a strange feeling as if there is an unreal space beyond the real space. For example, when playing the game of this embodiment in a room, it is possible to give a feeling as if there is an unreal space beyond the room. Further, the texture of the back wall may represent a landscape that is not usually seen, such as a desert or a wilderness. In this way, by selecting the texture of the back wall BW, the player can be informed of a desired image for another world hidden behind the real image expressed as the background of the game world.

  For example, if the image data is an image that can use repetitive expressions such as a space image, the data size of the image data (texture) can be reduced. If the image data is such an image, the image of the back wall BW can be drawn without specifying the position where the back wall BW in the virtual space should be drawn. This is because, if the image can use repetitive expressions, the drawn image does not depend on the position (repetitive patterns can be expressed in the entire polygon model).

  In the present embodiment, since drawing priority, which will be described later, is determined by an alpha value, it is assumed that the image data has an alpha value. In the present embodiment, it is assumed that the alpha value defined for the image data is “1”.

<< Enemy Object Data Df >>
The enemy object data Df is data related to the enemy object EO, and includes entity data Df1, silhouette data Df2, and opening shape data Df3.

  The entity data Df1 is data for drawing the entity of the enemy object EO, and is, for example, a polygon model that defines the three-dimensional shape of the entity of the enemy object EO and texture data for mapping to the polygon model. The texture data may be, for example, a face photograph of a user or the like captured by each imaging unit of the game apparatus 10. In the present embodiment, since drawing priority, which will be described later, is determined by an alpha value, it is assumed that an alpha value is defined for the texture data. In the present embodiment, it is assumed that the alpha value defined for the texture data is “1”.

  The silhouette data Df2 is data for rendering the shadow of the enemy object EO existing in the second space 2 translucently on the real world image, and includes a polygon model and texture data pasted on the polygon model. For example, as described above, this silhouette model includes eight planar polygons and is arranged at the same position as the enemy object EO existing in the second space 2. The shadow model of the enemy object EO existing in the second space 2 can be expressed by drawing the silhouette model with the texture pasted on the real world image as seen from the virtual world drawing camera. Further, the texture data of the silhouette data Df2 may be, for example, an image of the enemy object EO viewed from each direction as shown in FIGS. 27A and 27B (for example, eight planar polygons). Furthermore, the image may be an image obtained by simplifying the silhouette model of the enemy object EO. In the present embodiment, since drawing priority, which will be described later, is determined by an alpha value, it is assumed that an alpha value is defined for texture data to be pasted on a silhouette model. In the present embodiment, it is assumed that the alpha value defined in the texture data is “1” for the shadow image portion and “0” for the portion without the shadow image (surrounding portion).

  The opening shape data Df3 is data relating to the shape of the opening generated on the boundary surface 3 when the enemy object EO travels between the first space 1 and the second space 2. In the present embodiment, the opening shape data Df3 is data for setting the alpha value at the position of the opening determination data corresponding to the position on the boundary surface 3 where the opening is generated to “0”. For example, the opening shape data Df3 is texture data having an alpha value “0” that matches the shape of the opening to be generated. In the present embodiment, with respect to the opening determination data, the alpha value is set to “0” by the shape of the opening shape data Df3 around the location corresponding to the position where the enemy object EO has passed through the boundary surface 3. Set. Image processing when the enemy object EO generates an opening on the boundary surface 3 will be described later.

<< Bullet Object Data Dg >>
The bullet object data Dg is data related to the bullet object BO that is fired in response to the player's attack operation. For example, the bullet object data Dg includes a polygon model and bullet image (texture) data for drawing the bullet object BO, data indicating the arrangement direction and arrangement position of the bullet object BO, and the movement speed and movement direction of the bullet object BO ( For example, it includes data indicating a movement speed vector. In the present embodiment, since drawing priority, which will be described later, is determined by an alpha value, it is assumed that an alpha value is defined for bullet image data. In the present embodiment, it is assumed that the alpha value defined in the bullet image data is “1”.

<Scoring data Dh>
The score data Dh is data indicating the score of the game in which the enemy object EO appears. For example, as described above, the score of the game is added by defeating the enemy object EO by an attack operation, and the enemy object EO reaches the position of the user (that is, the arrangement position of the virtual camera in the virtual space). Points are deducted.

<Motion data Di>
The motion data Di is data indicating the motion of the game apparatus 10 in the real space. As an example, the movement of the game apparatus 10 is calculated based on the angular velocity detected by the angular velocity sensor 40.

<< Virtual Camera Data Dj >>
The virtual camera data Dj is data related to the virtual camera installed in the virtual space. In the first drawing method, for example, the virtual camera data Dj includes data indicating the arrangement direction and the arrangement position of the virtual camera in the virtual space. In the second drawing method, for example, the virtual camera data Dj includes data indicating the arrangement direction and arrangement position of the real world drawing camera in the virtual space, the arrangement direction of the virtual world drawing camera in the virtual space, Each of the data indicating the arrangement position is included. For example, the data indicating the arrangement direction and the arrangement position in the virtual space of the virtual camera in the first drawing method and the virtual world drawing camera in the second drawing method are movements of the game apparatus 10 indicated by the movement data Di ( It changes according to (angular velocity). Further, the virtual camera data Dj includes angle of view (drawing range) data of the virtual camera. As a result, the drawing range (drawing position) on the boundary surface 3 changes according to changes in the position and orientation of the virtual camera in the first drawing method and the virtual world drawing camera in the second drawing method.

<< Rendered image data Dk >>
The rendered image data Dk is data related to an image rendered by processing to be described later.

  In the first drawing method, since the real world image is rendered as an object in the virtual space, the rendering image data Dk includes rendering image data in the virtual space. Rendered image data in the virtual space is rendered by perspective projection of the virtual space where the enemy object EO, the bullet object BO, the boundary surface 3 (screen object) on which the real world image is pasted as a texture, and the back wall BW are arranged. It is the data which shows the virtual world image obtained by doing.

  On the other hand, in the second rendering method, since the real world image and the virtual world image are rendered by separate virtual cameras, the rendering image data Dk includes the rendering image data of the real camera image and the rendering image data of the virtual space. . The rendered image data of the real camera image is data indicating a real world image obtained by rendering the planar polygon onto which the texture of the real camera image is mapped by parallel projection by the real world image drawing camera. The rendered image data of the virtual space indicates a virtual world image obtained by rendering the virtual space in which the enemy object EO, the bullet object BO, the boundary surface 3, and the back wall BW are arranged by perspective projection with a virtual world drawing camera. It is data.

<< Display image data Dl >>
The display image data Dl is data indicating a display image displayed on the upper LCD 22. In the first drawing method, for example, a display image displayed on the upper LCD 22 is generated by a virtual space rendering process. In the second drawing method, for example, the display image displayed on the upper LCD 22 is generated by combining the rendered image data of the camera image and the rendered image data of the virtual space by a method described later.

<< Aiming cursor image data Dm >>
The aiming cursor image data Dm is image data of the aiming cursor AL displayed on the upper LCD 22. The image data may be arbitrary data.

  In the present embodiment, data relating to each object used for drawing (boundary surface data Dd, back wall image data De, entity data Df1, silhouette data Df2, bullet image data) is priority information that determines drawing priority. Is included. In the present embodiment, an alpha value is used for the priority information. The relationship between the alpha value and image processing will be described later.

  In the present embodiment, the data regarding each object used for drawing includes data indicating whether or not depth determination is performed between the core objects. As described above, the data is set so that the depth determination is valid between the enemy object EO, the bullet object BO, the translucent enemy object, the effect object, and the screen object (boundary surface 3). . The data includes “between the shadow plane polygon (silhouette data Df2) and the enemy object EO (substance data Df1)”, “between the shadow plane polygon (silhouette data Df2) and the bullet object BO”, “shadow The plane polygon (silhouette data Df2) and the semi-transparent enemy object "and" between the shadow plane polygon (silhouette data Df2) and the effect object "are set so that the depth determination is effective. Further, the data is set such that the depth determination is invalid between the shadow plane polygon (silhouette data Df2) and the screen object (boundary surface data Dd).

<< Management data Dn >>
The management data Dn is data for managing data processed by the game apparatus 10 such as collected face images or data accumulated by the game apparatus 10. The management data Dn includes, for example, face image management information Dn1, a face image attribute count table Dn2, and the like. In the face image management information Dn1, the storage destination of each face image data (for example, the address of the main memory 32, etc.), the acquisition source of the face image (for example, the inner imaging unit 24, the outer imaging unit 23), the face image management information The attributes (for example, the gender and age of the subject of the face image), information on other face images related to the face image, and the like are stored. The face image attribute count table Dn2 stores the number of points collected for each attribute of the face image currently collected by the user. For example, the collection result value for each classification when the subjects of the collected face images are classified by sex, age, or the like is stored. An example of the data structure of the face image management information Dn1 and the face image attribute count table Dn2 will be described later.

《Save data storage area Do》
The save data storage area Do stores data to be processed by the information processing unit 31 and data obtained as a result of processing by the information processing unit 31 when the information processing unit 31 executes an image processing program such as a game program. It is an area to do. As an example, in the present embodiment, the game device 10 stores face image data acquired by the game device 10 via the inner imaging unit 24, the outer imaging unit 23, the wireless communication module 36, the local communication module 37, and the like. Is done. In the present embodiment, for example, the information processing unit 31 executes the first game in a state where the face image acquired by the game apparatus 10 is temporarily stored in the main memory 32. When it is determined that the user has succeeded in the first game in accordance with the user's operation, the information processing section 31 saves the face image temporarily stored in the main memory 32 in the save data storage area Do. To do. The face image saved in the save data storage area Do can be used in subsequent game processing and the like.

  There is no particular limitation on the structure of the save data storage area Do. For example, the save data storage area Do may be placed in the same physical address space as a normal memory and accessible from the information processing unit 31. Further, for example, the information processing section 31 may be secured (also referred to as allocation or allocation) in a predetermined block unit or page unit at a necessary timing. Further, for example, the save data storage area Do may be connected by management information such as a point for connecting blocks, similarly to a file system of a computer.

  Further, the save data storage area Do may have a separate area reserved for each program executed by the game apparatus 10, for example. Accordingly, when one game program is loaded into the main memory 32, the information processing section 31 accesses the save data storage area Do (input / output data) based on the management information of the game program. Also good.

  Furthermore, the save data storage area Do of one program may be accessible from the information processing section 31 that is executing another program. In this way, data processed by one program may be transferred to another program. For example, as a result of executing a first game, which will be described later, the information processing unit 31 that is executing the second game can read out the face image data stored in the save data storage area Do and create a character object. You may do it. The save data storage area Do is an example of a second storage area.

<Structure of various data>
With reference to FIG. 12 and FIG. 13, an example of a data structure for managing face images in the game apparatus 10 will be described.

  FIG. 12 shows an example of the data structure of face image management information Dn1 for managing face images stored in the game apparatus 10. The game apparatus 10 records the stored face image data in the face image management information Dn1, so that a list of face images can be displayed on the screen of the upper LCD 22 or the like in the format shown in FIGS. become. The face image management information Dn1 is created as information prepared for one record for each face image, for example. The face image management information Dn1 is stored in, for example, the data storage internal memory 35 and the data storage external memory 46. In FIG. 12, a record 1 illustrates the elements of the record. Further, in FIG. 12, details after the record 2 are omitted. Further, for example, although not shown, the information processing section 31 stores the total number of records of the face image management information Dn1, that is, the total number of acquired face images in the data storage internal memory 35, the data storage external memory 46, and the like. Just keep it.

  In the example of FIG. 12, the face image management information Dn1 includes, for example, face image identification information, face image data address, face image acquisition destination, gender estimation, age estimation, related face image information 1-N, and the like. is doing. However, FIG. 12 is an example of the face image management information Dn1, and the face image management information is not limited to the elements illustrated in FIG.

  The face image identification information is information for uniquely identifying the stored face image. The face image identification information may be a serial number, for example.

  The address of the face image data is, for example, the address of the data storage internal memory 35 or the data storage external memory 46 where the face image data is stored. However, for example, when face image data is stored in a storage medium in which the file system is constructed by the OS (operating system), a path name, a file name, etc. in the file system are set as the address of the face image data. You may make it do.

  The acquisition source of the face image is, for example, information for identifying the imaging device that acquired the face image. As a face image acquisition source, for example, information for identifying the inner imaging unit 24, the outer left imaging unit 23a, and the outer right imaging unit 23b is set. However, when both the outer left imaging unit 23a and the outer right imaging unit 23b are used for face image acquisition, information indicating both is set. In addition, for example, when a face image is acquired by another imaging device other than the game device 10 other than the inner imaging unit 24, the outer left imaging unit 23a, and the outer right imaging unit 23b, for example, this is indicated. Information (for example, “other”) is set. The case where the face image is acquired by another imaging device outside the game device 10 is, for example, another game device 10 similar to the game device 10 through the external memory interface 33, the wireless communication module 36, the local communication module 37, and the like. This is the case where an image captured by the method is acquired. In addition, a case where an image obtained by a camera other than the game apparatus 10, an image obtained by a scanner, an image from a video device, or the like is acquired through the external memory interface 33, the wireless communication module 36, or the like.

  The gender estimation is information indicating whether the face image is male or female. The gender estimation may be executed in accordance with, for example, processing shown in another embodiment described later. Age estimation is information indicating the age of a person in a face image. The estimation of age may be executed according to the processing shown in another embodiment described later, for example.

  The related image identification information 1-N is information indicating other face images related to the face image. For example, the related image identification information 1-N may set face image identification information of up to N other related face images. Other related face images may be specified by a user operation through a GUI, for example. For example, when a new face image is acquired, the user selects a face image 1 or more related to the acquired face image with the operation button 14 or the like, and then performs an operation of the GUI that instructs setting of the related image May be detected by the information processing section 31. Or you may make it classify | categorize the acquired face image only by the classification | category which the game device 10 prepared, such as a person, a friend, a colleague, and others. And you may make it link the face image which belongs to the same classification | category using the identification information 1-N of a related image. However, when classifying a face image according to the classification prepared by the game apparatus 10, instead of preparing an entry of identification information 1-N of the related face image, an element “classification of the face image” is simply provided, , Friends, colleagues, others, etc. may be set. In FIG. 12, the related image identification information 1 to N is shown as a fixed number, but the number of N may be a variable number. In that case, the set number N may be held in the face image management information Dn1. Further, for example, the records of the face image management information Dn1 may be connected with a chain of pointers between related face images.

  FIG. 13 shows an example of the data structure of the face image attribute summary table Dn2. The face image attribute count table Dn2 is a table in which acquired face images are classified according to attributes and the number of classified images is totaled. Hereinafter, the face image attribute summary table Dn2 is also simply referred to as a summary table. The information processing unit 31 stores the summary table of FIG. 13 in, for example, the data storage internal memory 35 and the data storage external memory 46. In the example of FIG. 13, the tabulation table is classified by combination with gender (male, female) and age group (under 10 years old, teens, 20s, 30s, 40s, 50s, 60s, 70 and above). The number of face image acquisitions is stored in each line. That is, each row in the table of FIG. 13 includes elements such as gender, age, and number of acquired face images. The acquired face images are classified into the number of acquired face images, and the acquired number is totaled. However, the classification or attribute of the face image is not limited to the gender or age group illustrated in FIG.

<Example of processing flow>
An example of the operation by the image processing program executed by the information processing unit 31 of the game apparatus 10 will be described with reference to FIGS. First, when the power supply (power button 14F) of the game apparatus 10 is turned on, a boot program (not shown) is executed by the CPU 311 and thereby stored in the built-in memory or the external memory 45 or the data storage external memory 46. Are expanded in a format that can be executed by the CPU 311 in the main memory 32. Here, the format executable by the CPU 311 is, for example, in which machine instructions of the CPU 311 are described in a predetermined processing order, arranged at an appropriate address in the main memory 32, and readable from a control unit that processes the machine instructions of the CPU 311 Form. Extracting in an executable form is also simply loading. In FIGS. 14 to 19, descriptions of processes not directly related to the first embodiment are omitted.

  FIG. 14 is a flowchart illustrating an example of an operation performed by the information processing unit 31. The information processing unit 31 performs a user operation, for example, a graphical user interface (hereinafter, GUI) displayed on the lower LCD 12, for example, a graphic object such as a menu or an icon after a series of processes after the power is turned on. When an operation through the touch panel 13 or the operation button 14 is detected, the process of FIG. 14 is executed. Hereinafter, a user operation on the GUI through the touch panel 13 or the operation button 14 is simply referred to as “operation on the GUI”. In the example of FIG. 14, the information processing section 31 waits for a user operation (step 8). Hereinafter, in the drawings, “step” is abbreviated as “S”.

  Next, when a user operation is detected, the information processing section 31 performs the processing from step 9 onwards. For example, when the user's operation on the GUI is an instruction of “acquisition of a face image by the inner imaging unit 24” (Yes in Step 9), the information processing unit 31 executes the face image acquisition process 1 (Step S1). 10). Here, the instruction of “acquisition of face image by the inner imaging unit 24” refers to an instruction of acquisition using the inner imaging unit 24 by an operation to the user through a GUI or the like, for example. Thereafter, the information processing section 31 advances the control to step 19. The face image acquisition process 1 will be described later with reference to FIG. On the other hand, when the user's operation on the GUI is not an instruction of “acquisition of face image by the inner imaging unit 24” (No in Step 9), the information processing unit 31 advances the control to Step 11.

  Next, for example, when the user's operation on the GUI is an instruction of “acquisition of a face image by the outer imaging unit 23” (Yes in Step 11), the information processing unit 31 executes the face image acquisition process 2 (Step 12). Thereafter, the information processing section 31 advances the control to step 19. Here, the instruction “acquisition of a face image by the outer imaging unit 23” refers to an instruction to acquire using the outer imaging unit 23 by an operation to a user through a GUI or the like, for example. The face image acquisition process 2 will be described later with reference to FIG. On the other hand, when the user's operation on the GUI is not an instruction of “acquisition of face image by outer imaging unit 23” (No in step 11), the information processing unit 31 advances the control to step 13.

  Next, when the user's operation on the GUI is an instruction to display a list of collected face images (Yes in Step 13), the information processing section 31 executes a list display process (Step 14). Thereafter, the information processing section 31 advances the control to step 19. The list display process will be described later with reference to FIG. On the other hand, if the user's operation on the GUI is not an instruction to display a list of collected face images (No in step 13), the information processing section 31 advances the control to step 15.

  When the user's operation on the GUI is a casting determination instruction (Yes in Step 15), the information processing section 31 executes a casting determination process (Step 16). Thereafter, the information processing section 31 advances the control to step 19. The casting determination process will be described later with reference to FIG. On the other hand, if the user's operation on the GUI is not an instruction to determine casting (No in step 15), the information processing section 31 advances the control to step 17.

  If the user operation is a game execution instruction (Yes in Step 17), the information processing section 31 executes the game (Step 18). The process of step 18 is an example of a second game process step. The game apparatus 10 executes a game process in which various character objects such as the enemy object EO created in the casting determination process in step 16 appear. There is no limitation on the type of game. For example, the game executed in step 18 may be a game in which the user battles the enemy object EO created by the casting determination process. In this case, for example, the user battles with the enemy object EO of the face image collected in the face image acquisition process 1 in step 10 and the face image acquisition process 2 in step 12 and displayed in the list display process in step 14. It will be. Further, for example, this game may be an adventure game in which a player object that symbolizes a user moves over various barriers, obstacles, and the like. This game may be a game such as a war simulation in which historical characters appear, a management simulation in which a player object appears, or a driving simulation of a vehicle in which the player object appears. The game may be a novel game in which the character object appears, modeled on the original novel. In addition, the game may be a so-called role-playing game (RPG) in which a user operates a hero or a character appearing in a story and plays the role. Further, the game may be a game in which the user receives some support from an agent who appears.

  To the character object appearing in such a game process, a face image collected when the user succeeds in the first game in step 10 is pasted by a technique such as texture mapping. Therefore, in the game executed in step 18, a character object including a face image collected by the user himself appears. For this reason, the user reflects the real-world relationship with the person (or creature) of the face image by using the face image, which is an image of a part that symbolizes a person or creature, for various character objects. You can run the game. For example, it is possible to execute a game process that includes emotions such as love, favor, good feeling, and hatred.

  On the other hand, when the user's operation is not a game execution instruction (No in Step 17), the information processing section 31 advances the control to Step 19.

  Then, the information processing section 31 determines whether or not to end the process. When the information processing unit 31 detects an instruction to end the process through the GUI, the information processing unit 31 ends the process of FIG. On the other hand, when the information processing unit 31 detects an instruction that does not end (for example, a re-execution instruction) through the GUI, the information processing unit 31 executes the face image management support process 1 (step 1A). The face image management support process 1 is a process that provides information to the user regarding the attribute of the face image to be acquired next based on the already acquired face image and supports the user's face image acquisition. The face image management support process 1 will be described later in detail with reference to FIG. 19A. Thereafter, the information processing section 31 returns the control to step 8.

  FIG. 15 is a flowchart showing an example of detailed processing of face image acquisition processing 1 (step 10 in FIG. 14). In this process, the information processing section 31 first executes the face image management support process 2 (step 100). The face image management support process 2 is a process that provides information to the user regarding the attribute of the face image to be acquired next based on the already acquired face image and supports the user's face image acquisition. A detailed processing example of the face image management support processing 2 will be described later with reference to FIG. 19B.

  Next, the information processing section 31 executes face image acquisition processing (step 101). The CPU 311 of the information processing unit 31 executes the process of step 101 as an example of an image acquisition unit.

  For example, the information processing unit 31 captures an image captured through the inner imaging unit 24, the outer left imaging unit 23a, and / or the outer right imaging unit 23b at a predetermined cycle, and displays the captured image on the upper LCD 22. In this case, the display cycle may be the same as the processing unit time (for example, 1/60 seconds) in the game apparatus 10 or may be shorter than the unit time. Immediately after the game apparatus 10 is turned on and the image processing program is loaded, or in the initial state immediately after the start of processing in FIG. 14, the information processing unit 31 displays, for example, an image from the inner imaging unit 24 on the upper LCD 22. To do. Note that, for example, at least one of the inner imaging unit 24, the outer left imaging unit 23a, and the outer right imaging unit 23b (both the outer left imaging unit 23a and the outer right imaging unit 23b may be used as the lower LCD 12). An imaging unit selection GUI for selecting (including) is prepared. In the processing of FIG. 15, the user can freely switch the imaging unit to be used by the imaging unit selection GUI. Hereinafter, the inner imaging unit 24, the outer left imaging unit 23a, and / or the outer right imaging unit 23b used for imaging in the initial state or the operation of the imaging unit selection GUI are simply referred to as an imaging unit.

  For example, when the inner imaging unit 24 is used, when the user turns his / her face to the inner side surface 21 </ b> B of the upper housing 21 with the upper housing 21 opened, the user's face is projected on the upper LCD 22. . When the user presses the R button 14H (or L button 14G), for example, the information processing unit 31 acquires an image from the inner imaging unit 24 projected on the upper LCD 22 as data, and temporarily stores it in the main memory 32. Remember me. At this time, the image data is only present on the main memory 32 and is not saved in the save data storage area Do described later. The data on the main memory 32 is only used in the game in step 106 described later, and is discarded when the game ends without success as described later. The main memory 32 is an example of a first data storage area.

  In the process of the present embodiment, the face image acquired in step 101 is texture-mapped to the face portion of the enemy object EO and the game is executed. For this reason, in the process of step 101, it is desirable to acquire a face image by cutting out a face portion from the image acquired from the imaging unit. In the present embodiment, for example, the following processing is executed. (1) The information processing unit 31 detects the outline of the face from the acquired image. The outline of the face is estimated from the interval between the eyes and the positional relationship between the eyes and the mouth. That is, the information processing section 31 recognizes the boundary line between the face outline and the background using the standard face dimensions from the arrangement of the eyes and mouth. The boundary line can be acquired by combining normal image processing such as differentiation processing (outline emphasis) and average processing (smoothing operation), for example. The face contour detection method may be performed by another known method. (2) The information processing unit 31 enlarges or reduces the obtained face image so as to match the size of the face portion of the head shape of the enemy object EO. According to this process, it is possible to acquire a face image having a variation in dimensions to some extent by the game apparatus 10 and paste it on the enemy object EO.

  However, in the game apparatus 10 of the present embodiment, the process of acquiring a face image is not limited to the above procedure. For example, when acquiring a face image, instead of acquiring an image with an arbitrary size from an arbitrary distance, a face image with a target size may be acquired from the imaging unit. For example, it may be acquired after establishing the distance from the subject so that the distance between the eyes of the face image obtained from the subject is close to a predetermined number of pixels. For example, the information processing unit 31 may guide the distance to the subject. In addition, after establishing the distance to the subject, the information processing unit 31 guides the person who is the subject or the user who is the photographer so as to adjust the angle of the face of the subject with respect to the visual axis direction of the imaging unit, for example. May be. Further, for example, instead of saving the image by pressing the R button 14H (or L button 14G), the user determines that the adjustment of the distance to the subject and the face angle with respect to the visual axis direction of the imaging unit is completed. When it is possible, the information processing section 31 may store the image. For example, the information processing unit 31 may display a mark indicating the eye position and the target position for mouth alignment on the upper LCD 22 so as to be superimposed on the face image of the subject. Then, when the positions of both eyes and mouth of the subject acquired from the imaging unit fall within a predetermined tolerance range from the target position marks corresponding to both eyes and mouth, the information processing unit 31 stores the image in the memory. You may make it do.

  Note that when acquiring the face image in step 101, the information processing section 31 updates the number of face image acquisitions in the corresponding row of the face image attribute summary table Dn2 in FIG. The relevant row means, for example, a row of attributes corresponding to gender and age estimated in step 1002 of FIG. 19B described later.

  Next, the information processing section 31 displays the image acquired by the processing in step 101 on, for example, the upper LCD 22 (step 102).

  Next, the information processing section 31 executes an enemy object EO selection process (step 103). Here, the information processing section 31 prompts the user to select the head shape of the enemy object EO. For example, a list of head shapes as shown in FIG. 9 may be displayed and a user's selection may be received through the GUI. Then, the information processing section 31 sets the acquired face image as the texture of the enemy object EO (step 104), and generates the enemy object EO (step 105). The enemy object generated in step 105 is an example of a first character object. The information processing section 31 performs the process of step 105 as an example of a means for creating a character object.

  Then, the information processing section 31 executes a game using the generated enemy object EO (step 106). The CPU 311 of the information processing unit 31 executes the process of step 106 as an example of the first game processing means. Here, the kind of game is not ask | required. The game is, for example, a game that simulates a battle with the enemy object EO. Further, the game may be, for example, a game that competes with the enemy object EO for a large number of points. After the game is executed, the information processing section 31 determines whether or not the user has succeeded in the game (step 107). The information processing section 31 executes the processing of step 107 as an example of means for determining success or failure. Success is, for example, a case where the user wins the enemy object EO in a game in which the user fights the enemy object EO. Success is, for example, a case where the user has scored more points than the enemy object EO in a game in which the enemy object EO competes for a large number of scores. The success may be, for example, a game in which the user overcomes the obstacle set by the enemy object EO and the user reaches the goal.

  In the game at step 106, in addition to the character object including the face image acquired at step 101, a character object using the face image already collected in the past may appear. For example, a face image that has already been collected in the past appears pasted on an enemy object EO or an ally object, so that the user can play a game reflecting human relationships in the real world.

  When the user succeeds in the game, the information processing section 31 stores the face image data on the main memory 32 acquired in step 101 described above in the saved data storage area Do of the game so far. In addition to the data, it is stored (step 109). The CPU 311 of the information processing unit 31 executes the process of step 109 as an example of a storing unit. The save data storage area Do of the game is constructed in, for example, the main memory 32, the internal data storage memory 35, the external data storage memory 46, and the like that can be written and read by the information processing unit 31 that executes the game. It is a storage area. When the new face image data is stored in the save data storage area Do of the game, the information processing section 31 that executes the game displays the face described in FIGS. 7 and 8 on the screen of the upper LCD 22, for example. In addition to the image list, new face image data can be displayed. Thus, according to the process of FIG. 15, the user executes the game (first game) in order to save the face image acquired in step 101 in the game save data storage area Do. In the game, for example, a user who executes a game on the game apparatus 10 by making a character object using a face image stored so far in the saved data storage area Do appear so far can be displayed in the real world. New face images can be collected and added to the saved data storage area Do while reflecting the human relationship of the user.

  At this time, the information processing section 31 newly generates the face image management information Dn1 described with reference to FIG. 12 to manage the face image stored in the save data storage area Do of the game, and stores the internal data storage memory. 35 or in the external memory 46 for data storage. That is, the information processing section 31 newly generates face image identification information and sets it in the record of the face image management information Dn1. Further, the address of the face image stored in the save data storage area Do of the game is newly set as the address of the face image data. Furthermore, a face image acquisition source, gender estimation, age estimation, and related face image identification information 1-N are set.

  Furthermore, the information processing section 31 may estimate the attribute of the face image added to the save data storage area Do and update the count result of the face image attribute count table Dn2 described with reference to FIG. In other words, the information processing section 31 may newly estimate the gender, age, and the like of the face image added to the save data storage area Do and reflect it in the count result of the face image attribute count table Dn2.

  Furthermore, the information processing section 31 may allow the user to copy or change data stored in the save data storage area Do of the game, transfer data through the wireless communication module 36, or the like. Then, the information processing unit 31 only needs to execute saving, copying, changing, transferring, and the like of the face image stored in the save data storage area Do according to the operation through the user GUI or the operation through the operation button 14.

  On the other hand, if the user has not succeeded in the game, the information processing section 31 inquires of the user whether or not to retry the game (step 108). For example, an inquiry message as to whether or not to retry is displayed on the upper LCD 22, and an operation through the touch panel 13 on a GUI (affirmative, negative icon, menu, etc.) on the lower LCD 12 or an operation button 14 is used. A user's selection is received by operation. If the user instructs a retry, the information processing section 31 returns control to step 106. On the other hand, if the user has not instructed the retry, the information processing section 31 discards the face image acquired in step 101 (step 110) and ends the process. If the game is not successful, the information processing section 31 may discard the face image acquired in step 101 without waiting for the retry instruction in step 108.

  Hereinafter, a detailed processing example of the face image acquisition processing 2 (step 12 in FIG. 14) will be described with reference to FIG. In this process, in the face image acquisition process, the user is guided to acquire the face image by the inner imaging unit 24 before the two outer imaging units 23 (the outer left imaging unit 23a and the outer right imaging unit 23b). A processing example will be described. The game device 10 causes the user to execute the face image acquisition by the inner imaging unit 24 first, for example, by clarifying the owner of the game device 10 by the face image acquisition by the inner imaging unit 24, and by another person of the game device 10 This is to increase the possibility of restricting the use.

  An example of an operation by the image processing program executed by the information processing unit 31 of the game apparatus 10 will be described with reference to FIG. In this process, the information processing section 31 first determines whether or not the face image acquisition by the inner imaging section 24 has been completed (step 121). Whether or not the face image has been acquired by the inner imaging unit 24 is, for example, whether or not there is a record in which the inner imaging unit 24 is set as a face image acquisition destination by the face image management information Dn1 shown in FIG. It may be determined by Further, for example, when the game device 10 is provided with a function of registering the face image of the owner of the game device 10 in the game device 10 by the inner imaging unit 24, whether or not the owner's face image has been registered is determined. You may judge.

  If the face image has not been acquired by the inner imaging unit 24 (No in step 121), the information processing unit 31 prompts the user to execute the face image acquisition process by the inner imaging unit 24 first. (Step 124), the processing by the subroutine is terminated. More specifically, for example, a message stating that “the face image acquisition process by the outer imaging unit 23 cannot be executed unless the face image by the inner imaging unit 24 has been acquired in the game apparatus 10” is displayed on the upper LCD 22. Alternatively, the information processing section 31 may request the user to register the owner's face image first.

  On the other hand, when the face image has been acquired by the inner imaging unit 24 (Yes in Step 121), the information processing unit 31 executes the face image management support process 3 (Step 122). The face image management support process 3 will be described later with reference to FIG. 19C. And the information processing part 31 performs the face image acquisition process by the outer side imaging part 23 (step 123). For example, when the outer imaging unit 23 is used, when the user points the outer surface 21D of the upper housing 21 toward another person's face with the upper housing 21 opened, the upper LCD 22 displays the other The person's face is projected. When the user presses the R button 14H (or L button 14G), for example, the information processing unit 31 acquires an image from the outer imaging unit 23 projected on the upper LCD 22 as data, and temporarily stores it in the main memory 32. Remember me. At this time, the image data exists only on the main memory 32 and is not saved in the save data storage area Do. The data on the main memory 32 is only used in the game in step 129 described later, and is discarded when the game ends without success as described later.

  In the process of the present embodiment, the face image acquired in step 123 can be texture-mapped on the face portion or the like of the enemy object EO, and the game can be executed. Therefore, in the process of step 123, it is desirable to cut out a face part and acquire a face image from among images acquired from the imaging unit by the same process as in step 101 above. Further, the information processing section 31 also updates the number of face image acquisitions in the corresponding row of the face image attribute summary table Dn2 in FIG. The corresponding row refers to, for example, a row of attributes corresponding to gender and age estimated in step 1202 of FIG. 19C described later.

  Next, the information processing section 31 displays the image acquired by the process of step 123 on, for example, the upper LCD 22 (step 125).

  Next, the information processing section 31 executes an enemy object EO selection process (step 126). Here, the information processing section 31 prompts the user to select the head shape of the enemy object EO. For example, a list of head shapes as shown in FIG. 9 may be displayed and a user's selection may be received through the GUI. Then, the information processing section 31 sets the acquired face image as the texture of the enemy object EO (step 127), and generates the enemy object EO (step 128). The enemy object generated in step 128 is also an example of the first character object. The information processing section 31 executes the process of step 128 as an example of a means for creating a character object.

  Then, the information processing section 31 executes a game using the generated enemy object EO (step 129). The CPU 311 of the information processing unit 31 executes the process of step 129 as an example of the first game processing unit. The game performed in step 129 is the same as that in step 106. That is, there are various types of games performed in the above step 129. For example, a game that simulates a battle with the enemy object EO and a game that competes with the enemy object EO for scores are considered. Then, after the game is executed, the information processing section 31 determines whether or not the user has succeeded in the game (step 130). The information processing unit 31 executes the process of step 130 as an example of means for determining success or failure. Success is, for example, a case where the user wins the enemy object EO in a game in which the user fights the enemy object EO. Success is, for example, a case where the user has scored more points than the enemy object EO in a game in which the enemy object EO competes for a large number of scores. The success may be, for example, a game in which the user overcomes the obstacle set by the enemy object EO and the user reaches the goal.

  In the game executed in step 129, in addition to the character object including the face image acquired in step 123, a character object using the face image already collected in the past may appear. For example, a face image that has already been collected in the past appears pasted on an enemy object EO or an ally object, so that the user can play a game reflecting human relationships in the real world.

  If the user succeeds in the game, the information processing section 31 stores the face image data on the main memory 32 acquired in step 123 described above in the save data storage area Do of the game so far. In addition to the data, the data is stored (step 132), and the processing by the subroutine is terminated. The CPU 311 of the information processing unit 31 executes the process of step 132 as an example of a storing unit. When the new face image data is stored in the save data storage area Do of the game, the information processing unit 31 executing the game displays the face described in FIGS. 7 and 8 on the screen of the upper LCD 22, for example. In addition to the image list, new face image data can be displayed. Thus, according to the process of FIG. 16, the user executes the game (first game) in order to save the face image acquired in step 123 in the game save data storage area Do. In the game, for example, a user who executes a game on the game apparatus 10 by making a character object using a face image stored so far in the saved data storage area Do appear so far can be displayed in the real world. New face images can be collected and added to the saved data storage area Do while reflecting the human relationship of the user.

  At this time, in the same way as the face image acquisition process 1 in step 10 above, the information processing unit 31 newly manages the face image stored in the save data storage area Do of the game. Image management information Dn1 is generated and stored in the data storage internal memory 35 or the data storage external memory 46. That is, the information processing section 31 newly generates face image identification information and sets it in the record of the face image management information Dn1. Further, the address of the face image stored in the save data storage area Do of the game is newly set as the address of the face image data. Furthermore, a face image acquisition source, gender estimation, age estimation, and related face image identification information 1-N are set. Furthermore, the information processing section 31 may estimate the attribute of the face image added to the save data storage area Do and update the count result of the face image attribute count table Dn2 described with reference to FIG. In other words, the information processing section 31 may newly estimate the gender, age, and the like of the face image added to the save data storage area Do and reflect it in the count result of the face image attribute count table Dn2. Furthermore, the information processing section 31 may allow the user to copy or change data stored in the save data storage area Do of the game, transfer data through the wireless communication module 36, or the like. Then, the information processing unit 31 only needs to execute saving, copying, changing, transferring, and the like of the face image stored in the save data storage area Do according to the operation through the user GUI or the operation through the operation button 14.

  On the other hand, if the user has not succeeded in the game, the information processing section 31 inquires of the user whether or not to retry the game (step 131). For example, an inquiry message as to whether or not to retry is displayed on the upper LCD 22, and an operation through the touch panel 13 on a GUI (affirmative, negative icon, menu, etc.) on the lower LCD 12 or an operation button 14 is used. A user's selection is received by operation. If the user instructs a retry, the information processing section 31 returns the control to step 129. On the other hand, if the user has not instructed the retry, the information processing section 31 discards the face image acquired in step 123 (step 133) and ends the processing by the subroutine. If the game is not successful, the information processing section 31 may discard the face image acquired in step 123 without waiting for a retry instruction in step 131.

  FIG. 17 is a flowchart illustrating an example of detailed processing of the list display processing (step 14 in FIG. 14). In this process, the information processing section 31 first reads a registered face image from the saved data storage area Do of the data saving internal memory 35 or the data saving external memory 46 and displays it on the upper LCD 22 (step 140). More specifically, the information processing section 31 acquires the address of each face image data from the face image management information Dn1 stored in the save data storage area Do. Then, a face image may be read from the address in the data storage internal memory 35 or the data storage external memory 46 and displayed on the upper LCD 22.

  Next, the information processing section 31 waits for a user operation (step 141). Then, it is determined by the user's operation whether or not the face image is in a selected state (step 142). Whether or not the face image is in the selected state is determined after the user presses the operation button 14 or the like when the list of face images is displayed on the upper LCD 22 as shown in FIGS. Judgment is made based on the state or the operation state through the GUI. Note that the face image in the selected state is displayed surrounded by, for example, the thick line L1, the thick line L2, etc., as shown in FIGS.

  Then, when any face image is in the selected state, the information processing section 31 uses the face image management information Dn1 (see FIG. 12) to search for a face image related to the selected face image (see FIG. 12). Step 143).

  Then, the information processing section 31 executes a process in which the searched face image reacts, such as directing the line of sight of the searched face image to the selected face image (step 144). The process in which the searched face image reacts can be executed by the following procedure, for example. For example, a plurality of eye patterns in which the direction of eyes is directed to other face images in FIG. 8 and a plurality of face patterns in which the direction of faces are directed to other face images in FIG. 8 are prepared in advance. Then, based on the relationship between the position of the searched face image and the position of the face image in the selected state, the corresponding eye orientation and face orientation pattern are selected. Then, instead of the eye orientation and face orientation pattern of the face image already displayed, the eye orientation and face orientation pattern of the corresponding face image may be displayed. That is, the image of the eye part determined from the relationship between the position of the searched face image and the position of the face image in the selected state may be replaced with the eye part of the original face image. Further, when changing the orientation of the face, the entire face image may be replaced and displayed. Further, for example, a pattern for changing the direction in the direction of 360 degrees in a predetermined angle, for example, 15 degrees may be prepared in advance. Then, an angle may be determined from the positional relationship between the face image in the selected state and the searched face image, and an eye pattern having an angle closest to the determined angle may be selected.

  Furthermore, with regard to the orientation of the face, a face image pattern is prepared in which the orientation is 0 degrees when facing in the normal direction of the screen and the orientation is changed at an angle of 30 degrees, 60 degrees, 90 degrees, etc. in the left-right direction. To do. Also, a pattern whose direction is changed by about 30 degrees in the vertical direction is prepared. Furthermore, with respect to a face image whose direction has been changed by 90 degrees in the left-right direction, it is further up and down, that is, diagonally upward (for example, 15 degrees upward, 30 degrees upward, 45 degrees upward, etc.), diagonally downward (for example, It is sufficient to prepare a pattern whose direction has been changed to 15 degrees, 30 degrees, 45 degrees, or the like. Then, the angle may be determined from the positional relationship between the face image in the selected state and the searched face image, and the face angle closest to the corresponding angle may be selected. Furthermore, in order to emphasize intimacy, an expression such as closing one eye may be displayed by animation of a three-dimensional model. Also, a heart mark or the like may be prepared and displayed near the face image related to the face image in the selected state.

  If it is determined in step 142 that the face image is not in the selected state, the information processing section 31 performs other processing (step 145). The other processes include, for example, operations on other GUIs on the lower LCD 12 and processes on the operation buttons 14 other than the operation buttons 14 (buttons 14a, 14b, 14c, etc.) used for selecting a face image. Thereafter, the information processing section 31 determines whether or not to end the process (step 146). For example, when the information processing unit 31 detects that the button 14c (B button) is pressed while the screen of FIG. 8 is displayed, the information processing unit 31 determines that a “return” instruction has been issued and ends the processing of FIG. . If the process is not terminated, the information processing section 31 returns the control to step 140.

  FIG. 18 is a flowchart showing an example of detailed processing of the casting determination processing (step 16 in FIG. 14). In this process, the information processing section 31 first displays a list of head shapes of the enemy object EO (step 160). Here, the enemy object EO is described as an example, but the process is the same as the following when a character object other than the enemy object EO is generated. The information processing section 31 stores the head shape of the enemy object EO in the data storage internal memory 35 in advance, for example, before shipment of the game apparatus 10, at the time of installation of the image processing program, or at the time of version upgrade. The information processing section 31 reads the head shape of the enemy object EO currently stored in the data storage internal memory 35 and displays it in the arrangement as illustrated in FIG.

  Next, the information processing section 31 detects a selection operation by the user GUI or the operation button 14 and receives the selection of the head shape of the enemy object EO (step 161). When the selection of the head shape of the enemy object EO is completed, the information processing section 31 then displays a face image list (step 162). Then, the information processing section 31 detects a selection operation by the user's GUI or the operation button 14 and receives a selection of a face image (step 163). In the processing example of FIG. 18, in step 163, the information processing unit 31 determines a face image by detecting a selection operation. However, instead of such processing, the information processing section 31 may automatically determine a face image. For example, the information processing unit 31 may randomly select a face image from the face images stored in the save data storage area Do. In addition, the information processing unit 31 stores a user's game history on the game apparatus 10 in the main memory 32, the external memory 45, the data storage external memory 46, the data storage internal memory 35, and the like. A face image may be selected according to the user characteristics, orientation, etc. estimated from the history. For example, the face image to be selected next may be determined according to the frequency ratio with which the user has selected the face image in the past.

  Then, the information processing section 31 sets the selected face image as the texture of the enemy object EO (step 164). Then, the enemy face EO is generated by texture mapping the selected face image on the face portion of the enemy object EO (step 165). The enemy object generated in step 165 is an example of a second character object. Then, the information processing section 31 displays the generated enemy object EO on the screen of the upper LCD 22 in a manner like the enemy object EO in FIG.

  Furthermore, the information processing section 31 executes a process in which the related face image reacts (step 166). This processing is the same as the processing in step 143 and step 144 in FIG. Further, the information processing section 31 determines whether or not to confirm the generated enemy object EO according to the user's operation on the GUI (step 167). If the enemy object EO is not confirmed, the information processing section 31 returns control to step 162 and accepts selection of a face image. However, when the enemy object EO is not confirmed, the information processing section 31 may return the control to Step 160 and accept the selection of the head shape of the enemy object EO. On the other hand, when confirming the enemy object EO, the information processing section 31 ends the process. The information processing section 31 executes the game process of step 18 in FIG. 14 using the confirmed enemy object EO. However, although not explicitly shown in FIG. 18, a GUI menu or the like may be prepared so as to end the processing of FIG. 18 without determining the enemy object EO.

  FIG. 19A is a flowchart illustrating an example of detailed processing of face image management support processing 1 (step 1A in FIG. 14). In this process, the information processing section 31 reads the acquired face image attributes from the face image attribute count table Dn2 (step 1A1). Then, the information processing section 31 searches the face image attribute count table Dn2 that has been read for an attribute that has not been acquired or that has a small number of face image acquisitions. The unacquired attribute is, for example, an attribute whose face image acquisition count is 0 in the table illustrated in FIG. In addition, the attribute having a small number of face image acquisitions is an attribute in which the number of face image acquisitions is included in a predetermined order from the bottom when sorting is performed using the number of face image acquisitions in the table illustrated in FIG. 13 as a sorting key. Say.

  Next, the information processing section 31 executes processing for prompting the user to acquire a face image corresponding to an unacquired attribute (step 1A2). For example, the information processing unit 31 displays a message combining the attribute “male”, the attribute “10's”, and the term “image acquisition number 0” on the lower LCD 12 or the upper LCD 22 from the table illustrated in FIG. That's fine. Further, for example, the information processing section 31 may display a message in which the attribute “male”, the attribute “10s” and the term “the number of image acquisition is small” are combined. However, the number of attribute combinations to be displayed (the rows in the table of FIG. 13) is not limited to one, and two or more may be displayed. For example, when detecting that the user presses the operation button 14B (A button), the information processing section 31 may end the process of FIG. 19A. Then, the information processing section 31 returns control to step 8 in FIG.

  Here, the face image management support processing 1 shown in FIG. 19A has been described as a detailed processing example at the time of termination determination (step 1A) in FIG. However, the face image management support processing 1 is not limited to the execution at the timing of the end determination (step 1A) after the game is executed. For example, the information processing unit 31 performs facial image management in order to prompt the user to acquire facial images during processing such as list display processing (step 14), casting determination processing (step 16), and game execution (step 18). Support process 1 may be executed.

  FIG. 19B is a flowchart illustrating an example of detailed processing of face image management support processing 2 (step 100 in FIG. 15). In this processing, the information processing section 31 first determines the presence or absence of an acquired image by referring to the number of acquired images (step 1000). The information processing section 31 can store the number of acquired images as, for example, the number of records of the face image management information Dn1 in the main memory 32, the external memory 45, the external data storage memory 46, the internal data storage memory 35, and the like. That's fine.

  If there is no acquired image (No in step 1000), the information processing section 31 ends the process. On the other hand, if there is an acquired image (Yes in step 1000), the information processing section 31 advances the control to step 1001. Then, the information processing section 31 accepts a face image acquisition request (step 1001). When the information processing unit 31 receives an acquisition instruction from the L button 14G or the R button 14H in a state where a face image is projected on the upper LCD 22 through the inner imaging unit 24 or the outer imaging unit 23, for example, Recognize image acquisition requests.

  Then, the information processing unit 31 estimates the attributes of the face image acquired through the inner imaging unit 24 or the outer imaging unit 23 and the face image is projected on the upper LCD 22, such as gender and age (step 1002). For example, the gender can be estimated from the cheekbone included in the face image, the size of the skeleton including the jawbone, and the face size. In other words, the information processing unit 31 calculates the relative dimensions of the face outline (for example, the width of the face, the distance between the eyes and the jaw) with respect to the distance between the eyes and the distance between the eyes. Then, when the relative dimension is close to the statistically obtained male average value, the face image may be determined to be male. Further, for example, when the relative dimension is close to the male average value obtained statistically, it may be determined that the face image is male.

  In addition, the information processing unit 31 sets the facial parts for each gender, for each age group (for example, younger than 10 years, 10s, 20s, 30s, 40s, 50s, 60s, 70 or more). Characteristic information such as the average position and the number of wrinkles in the face portion may be stored in advance. For example, the information processing unit calculates the feature information of the face image acquired through the outer imaging unit 23 and the face image is projected on the upper LCD 22, and estimates the age group with the closest calculated feature information. Good. However, the above description of specifying the sex and age is an example, and the determination of the sex and age is not limited to the above processing. In the processing of step 1002, various gender determination techniques and age identification techniques that have been proposed in the past can be applied.

  Next, the information processing section 31 prompts the user to acquire a face image having an unacquired attribute (step 1003). For example, the information processing unit 31 may display a message prompting the upper LCD 22 to acquire a face image having an unacquired attribute. This process is the same as the process of FIG. 19A. Then, the information processing section 31 determines whether or not the face image to be acquired has been changed by a user operation (step 1004). For example, the information processing unit 31 changes the characteristics of the face image included in the image acquired through the inner imaging unit 24 or the outer imaging unit 23, for example, the distance between the eyes and the distance between the eyes and the mouth. It is determined that the face image to be acquired has changed. Further, for example, when the acquisition instruction by the L button 14G or the R button 14H is simply canceled and the acquisition instruction by the L button 14G or the R button 14H is made again, the information processing unit 31 displays the face image to be acquired. You may determine with having changed. Then, when the face image to be acquired is changed, the information processing section 31 returns the control to step 1002.

  On the other hand, when the face image to be acquired is not changed, the information processing section 31 ends the process as it is. Here, the case where the face image to be acquired has not been changed is, for example, a case where the user has finished the face image management support processing 2 by using the GUI or the operation button 14C (B button). For example, the information processing unit 31 may determine that the face image to be acquired has not been changed when the face image to be acquired has not been changed for a predetermined time. In this case, the information processing section 31 advances the control to step 101 in FIG. 15 and executes face image acquisition processing. As already described in the processing of step 101 in FIG. 15, the information processing section 31 updates the number of face image acquisitions in the corresponding row in the face image attribute summary table Dn2 in FIG.

  The case where the face image to be acquired is not changed in the determination processing in step 1004 is, for example, a case where the amount of change in the distance between the eyes and the distance between the eyes and the mouth is within an allowable value. Also, for example, when the acquisition instruction continues for a predetermined time without simply canceling the acquisition instruction by the L button 14G or the R button 14H, the information processing section 31 determines that the acquisition instruction has not been canceled. It's okay.

  FIG. 19C is a flowchart illustrating an example of detailed processing of face image management support processing 3 (step 122 in FIG. 16). The processing in FIG. 19C (step 1201 to step 1204) is the same as the processing in step 1001 to step 1004 in FIG.

  19A to 19C, the information processing section 31 guides the user to preferentially acquire face images corresponding to unacquired attributes. By such a process, it is possible to support the face image collection process of a user who desires to acquire a face image with no attribute bias as much as possible.

  In addition, in this embodiment, the process example which classifies an age group like less than 10 years old, 10 generations, 20 generations, 30 generations, 40 generations, 50 generations, 60 generations, 70 or more was shown. However, the implementation of the present invention is not limited to such age group classification. For example, the age group may be further classified. Conversely, the age group may be roughly classified as a child, an adult, an elderly person, or the like.

  In the present embodiment, the information processing unit 31 recognizes a face image acquisition request when an acquisition instruction by the L button 14G or the R button 14H is received. However, instead of such processing, for example, as described in the present embodiment, in order for the information processing unit 31 to acquire a face image having a target size from the imaging unit, the game apparatus 10 and the face In the process of guiding the distance, the angle between the visual axis of the imaging unit and the face, the attributes of the face image, such as sex and age, may be estimated. That is, for such guidance processing, for example, when the information processing unit 31 acquires a face image in real time or every frame period (1/60 seconds or the like), the face image is acquired from the acquired face image. It is sufficient to specify the attribute of.

<Detailed example of game processing>
Here, an example of a game process in which a character object using a face image appears will be described with reference to FIGS. 20A to 40. The following processing is processing that is executed, for example, in step 18 of FIG. 14 using the enemy object EO including the face image accumulated in the save data storage area Do.

  First, in this embodiment, an outline of a game that a player can play by executing a game program by the game apparatus 10 will be described. The game in the present embodiment is a so-called shooting game in which a player shoots down an enemy character that appears in a virtual three-dimensional space prepared as a game world as the main character in the game. For example, a virtual three-dimensional space (virtual space (also referred to as a game space)) forming the game world is displayed on the display screen (for example, the upper LCD 22) of the game apparatus 10 from the player's viewpoint (so-called subjective viewpoint). . Of course, it may be an objective viewpoint. A score is added when the player shoots down an enemy character. On the other hand, when the enemy character and the player collide (specifically, when the enemy character reaches within a certain distance from the position of the virtual camera), the score is reduced.

  In addition, the game of the present embodiment combines a real world image (hereinafter referred to as “real world image”) acquired by an imaging unit included in the game apparatus 10 and a virtual world image representing a virtual space. indicate. Specifically, the virtual space is divided into a region close to the virtual camera (hereinafter referred to as “front side region”) and a region far from the virtual camera (hereinafter referred to as “back side region”). An image representing a virtual object existing in the area is displayed in front of the real world image, and a virtual object existing in the back area is displayed behind the real world image. More specifically, as will be described later, the virtual object existing in the near side area is given priority over the real world image, and the real world image is given priority over the virtual object existing in the back side area.

  There is no limitation on the method of combining the real world image and the virtual world image. For example, a real world image exists as an object in the same virtual space as the virtual object (more specifically, for example, pasted as a texture of the virtual object), and the real world image is shared with the virtual object by a common virtual camera. It may be rendered.

  In another example, a real world image is rendered from a first virtual camera (hereinafter referred to as a real world drawing camera) to form a first rendered image, and a second virtual camera (hereinafter referred to as a virtual world drawing camera) is rendered. A virtual object is rendered as a second rendered image from the camera), the first rendered image and the second rendered image are given priority over the virtual object existing in the near side area over the real world image, and The real world image may be synthesized so as to give priority to the virtual object existing in the back area.

  In the former method, typically, an object to which a real-world image is applied as a texture (hereinafter referred to as a screen object) is placed at a position that is a boundary between the near side area and the far side area, and an enemy object, etc. You may draw by seeing from a common virtual camera with these virtual objects. In this case, typically, the object to which the real-world image is pasted is an object having a certain distance from the virtual camera and a surface whose normal line coincides with the viewing direction of the virtual camera. Hereinafter, the real world image may be pasted as a texture on the boundary surface.

  In the latter method, the virtual object is rendered while performing depth determination (determination by the Z buffer) on the boundary surface between the near side region and the back side region (hereinafter simply referred to as the boundary surface), and the second method described above. The real world image is pasted as a texture on a surface at a constant distance from the virtual camera and the normal line coincides with the viewing direction of the virtual camera, and rendered as the first rendered image. Then, if the second rendering image is synthesized with priority over the first rendering image, an image in which the real world image appears to exist on the boundary surface is synthesized.

  In either method, the relationship between the distance and viewing angle from the virtual camera and the size of the real world image object (size in the line of sight) is set so that the real world image includes the visual field range of the virtual camera. Is done.

  Hereinafter, the former method is referred to as a first drawing method, and the latter method is referred to as a second drawing method.

  Further, when a predetermined event occurrence condition on the game is satisfied, a part of the real world image is opened and displayed so that the virtual space in the back area can be seen through the opening. An enemy character object exists in the near side area, and a special enemy character (so-called “boss character”) appears in the back side area when a predetermined condition is satisfied. The stage is cleared by shooting down the boss character. Several stages are prepared, and the game is cleared by clearing all stages. On the other hand, when a predetermined game over condition is satisfied, the game is over.

  In the typical example of the first drawing method described above, the opening for the real world image may be set with data indicating the position of the opening with respect to the boundary surface of the screen object. More specifically, an opening or non-opening may be indicated by the opacity of a texture (so-called α texture) applied to the boundary surface. In the second drawing method, data indicating the position of the opening with respect to the boundary surface may be set.

  In this embodiment, opening / non-opening is set for the real world image, but other image processing may be performed on the real world image. For example, it is possible to perform arbitrary image processing based on the common general knowledge of those skilled in the art, such as smearing or blurring a real world image. Also in these examples, data indicating the position where image processing is performed may be set for the boundary surface or the boundary surface.

<Game World>
As described above, in the game according to the present embodiment, it is felt that there is a virtual space (back side region) behind the real image, and a game screen representing the virtual space with an improved sense of depth is displayed. The real world image may be a normal image captured by a monocular camera or a stereo image captured by a compound eye camera.

  In the game according to the present embodiment, an image captured by the outer imaging unit 23 is used as a real world image. That is, a real world image (real world moving image acquired in real time) around the player captured by the outer imaging unit 23 during the game play is used. Accordingly, when the user (game player) holding the game apparatus 10 changes the imaging range of the outer imaging unit 23 by changing the orientation of the game apparatus 10 in the left-right direction or the up-down direction while playing the game, the upper LCD 22 is displayed. The displayed real world image is also changed following the change of the imaging range.

  Here, the direction change of the game apparatus 10 during the game play is roughly performed in accordance with (1) the intention of the player or (2) the intention (scenario) of the game. When the player intentionally changes the direction of the game apparatus 10 during play, the real world image captured by the outer imaging unit 23 changes, and thereby the real world image displayed on the upper LCD 22 is intentionally changed. can do.

  In addition, a change in the orientation of the game device 10 is detected by the angular velocity sensor 40 included in the game device 10, and the orientation of the virtual camera is changed accordingly. More specifically, the virtual camera is changed from the current direction in the direction in which the direction of the outer imaging unit 23 changes. Further, the virtual camera is changed from the current direction by the change amount (angle) of the direction of the outer imaging unit 23. That is, when the orientation of the game apparatus 10 is changed, the real world image changes and the displayed range of the virtual space changes. That is, by changing the orientation of the game apparatus 10, the real world image and the virtual world image change in conjunction with each other, thereby displaying a composite image as if the real world and the virtual world are related to each other. Can do. In the present embodiment, the position of the virtual camera is not changed, but the movement of the game apparatus 10 may be detected to change the position of the virtual camera.

  In the second drawing method, the virtual camera orientation changing process is applied to the virtual world drawing camera, and is not applied to the real world drawing camera.

  In addition, if an object is displayed locally such as at the edge of the screen (for example, the right edge or the left edge) during game play, the player will have a psychological feeling of trying to catch the object at the center of the screen, and the player Moves the game apparatus 10 (outside imaging unit 23). As a result, the real world image displayed on the screen changes. Such a change in the orientation of the game apparatus 10 (real world image change) is performed by the player naturally by programming the object displayed according to the game scenario to be intentionally displayed on the edge of the screen. Can be.

<Details of virtual space>
(Drawing real world images)
The real world image captured by the outer imaging unit 23 is synthesized so that it appears to exist at the boundary position between the near side region and the far side region of the virtual space. FIG. 20A shows an example of a virtual space in the present embodiment. FIG. 20B shows the relationship between the screen model and the α texture in the present embodiment. In the first drawing method, for the display of the real world image, as shown in FIG. 20A, a sphere model (the above-described screen model) centered on the position of the virtual camera is set in the virtual space, and the inner surface of the sphere is displayed. A real world image may be pasted to form a screen object. More specifically, it is pasted as a texture on the entire portion of the screen model that is within the viewing volume of the virtual camera. The screen model other than this part is not visible on the screen because it is set to be transparent. In this example, the boundary surface is a sphere surface, that is, as shown in FIG. 20A, the virtual camera side is the front side region (corresponding to the “second region” of the present invention) by the surface of the sphere, and from the surface of the sphere The far side from the virtual camera is the back side region (corresponding to the “first region” of the present invention).

  In the second drawing method, a planar polygon for pasting the texture of the real world image is arranged in the virtual space for displaying the real world image. In the virtual space, the relative position of the planar polygon with respect to the real world drawing camera is always fixed. That is, the plane polygon is arranged at a certain distance from the real world drawing camera, and is arranged so that the normal direction thereof coincides with the viewpoint (visual axis) of the real world drawing camera.

  Further, the planar polygon is set so as to encompass the visual field range of the real world drawing camera. Specifically, the size of the planar polygon and its distance from the virtual camera are set so that the planar polygon can include the visual field range of the virtual camera. Since the real world image is pasted on the entire surface of the plane polygon on the virtual camera side, when the plane polygon on which the real world image is pasted is drawn with the virtual camera, in the image generated by the virtual camera, The real world image is displayed so as to correspond to the entire area.

  As shown in FIG. 21, the boundary surface may be cylindrical. FIG. 21 shows another example of the virtual space in the present embodiment. In this case, in the virtual space, the vertical axis set up at the position of the virtual camera (in this embodiment, the Y axis of the virtual space corresponds to the vertical direction, and the X axis and the Z axis correspond to the horizontal direction). An imaginary cylindrical circumferential surface (boundary surface) having a central axis is arranged. However, as described above, this cylindrical circumferential surface is not an object to be visually recognized, but an object used in the opening process. The outer peripheral surface of the cylinder is divided into a virtual space, a first space where the virtual camera is located (corresponding to a “second region” of the present invention) and a second space existing around the first space (“ Equivalent to “first region”).

(Process for opening real world images)
Furthermore, in the game according to the present embodiment, an opening is provided in the real world image, and the player is made aware of the existence of the back side area behind the real world image. More specifically, the opening portion of the real world image is displayed as transparent or translucent, and the world behind it is combined with this portion. For this reason, in the game, when a predetermined event occurs, a part of the real world image is opened (erased) and another virtual space (back side) existing behind the real world image through the opening. Area) is displayed.

  In this embodiment, the boundary surface is a spherical surface in the processing for providing an opening in the real world image and displaying the back side region. In the first drawing method, as shown in FIGS. 20A and 20B, This is realized by a texture pasted on the inner surface of the spherical screen object. Hereinafter, this texture is referred to as a screen α texture (opening determination data described later). In the present embodiment, the screen α texture is attached to a portion that circulates 360 degrees in at least a certain direction around the virtual camera. More specifically, as shown in FIG. 20B, the α texture for the screen is rotated around 360 degrees in the direction parallel to the XY plane around the center of the sphere, in other words, the position of the virtual camera. Affixed to a portion having a predetermined width in the direction (hereinafter referred to as an α texture application unit). In this way, it is possible to simplify the way of holding the screen α texture data. Specifically, in this embodiment, the screen α texture has a rectangular shape. By pasting this α texture on the portion shown in FIG. 20A, information on each dot of the screen α texture corresponds to each coordinate of the α texture application unit in the screen object.

  As described above, the real world image is pasted, and the screen object with the α texture set is drawn with the virtual camera, so that the real world image having the opening is drawn so that it exists on the boundary surface (sphere inner surface). Will be. The part of the α texture corresponding to the real world image is calculated by drawing with a virtual camera.

  Also in the second drawing method, data indicating the position of the opening is set with respect to the boundary surface of the virtual space (here, the inner surface of the sphere). Typically, data indicating the presence or absence of an opening is set for each point on the boundary surface. More specifically, the same spherical object as described above is arranged in the virtual world where the virtual object exists, and the same α texture is set for the spherical object. Then, at the time of rendering the real world image, the α texture of the part corresponding to the part drawn by the virtual world drawing camera among the spherical objects set with the α texture is applied to the above-described planar polygon and rendered. Alternatively, the alpha texture of this part is used to perform the process of making the opening transparent with respect to the real world image, and then the real world image after pasting this process is pasted on the planar polygon. Render with Note that this spherical object is an object that is used only for calculating the opening, and is not drawn when the virtual world is drawn.

  In the present embodiment, the data indicating the opening is data having information for each point on the boundary surface, but may be information defining a position having the opening on the boundary surface by a calculation formula.

  In the second space, a polygon (object) for pasting a background image (texture) of the second space that fits in the visual field of the virtual camera through the opening is arranged. The background of the second space may be called “back wall”.

  In the first space, objects representing various characters representing enemy characters and bullets for shooting down enemy characters are arranged. A predetermined object (for example, a part of enemy characters) is also arranged in the second space. Each object arranged in the virtual space moves in the virtual space according to a preprogrammed logic (algorithm).

  Further, some of the enemy characters can go back and forth between the first space and the second space through an opening formed in the boundary surface. Alternatively, it is possible to go back and forth between the first space and the second space by forming an opening on the boundary surface. The predetermined event on the game that forms the opening is, for example, an event (collision event) in which the enemy character collides with the boundary surface. Alternatively, it is an event (enemy character appearance event) in which the boundary surface is destroyed at a predetermined timing and the enemy character in the second space enters the first space as the game scenario progresses. Alternatively, the opening may be automatically formed as time elapses. Further, the opening can be repaired in accordance with a predetermined game operation by the player. For example, the player can reduce (repair) the opening by hitting a bullet against the formed opening.

  FIG. 22 shows a virtual three-dimensional space (game world) defined in a game program that is an example of an image processing program in the embodiment. As described above, in this embodiment, the boundary surface is a sphere, but in FIG. 22, the boundary surface is shown in a cylindrical shape for the sake of simplicity. As described above, in the game according to the present embodiment, the upper LCD 22 of the game apparatus 10 displays the synthesized virtual world image representing the virtual three-dimensional space and the real world image.

  Further, as shown in FIG. 22, the virtual space on the game in the present embodiment is divided into the first space 1 and the second space 2 by the boundary surface 3 formed by a spherical surface with the position of the virtual camera as the central axis. It is divided.

  On the boundary surface 3, a camera image CI (FIG. 23), which is a real world image captured by a real camera built in the game apparatus 10, is described later in steps 81 and 82 in the first drawing method. In the second drawing method, the CI is synthesized with the virtual world image as if it were present at the position of the boundary surface 3 by the processing from step 83 to step 85 described later.

  The real world image in the present embodiment is a planar view image. The virtual world image is also a planar view image. That is, a planar view image is displayed on the upper LCD 22. However, the real world image may be a stereoscopically viewable image. The present invention is not limited by the type of real world image. The camera image CI in the present embodiment may be a still image or a real-time real world image (moving image). In the game program in the present embodiment, the camera image CI is assumed to be a real-time real world image. Further, the camera image CI that is a real world image is not limited by the type of camera. For example, the camera image CI may be an image obtained by a camera that can be externally connected to the game apparatus 10. Furthermore, in the present embodiment, the camera image CI may be an image acquired from the outer imaging unit 23 (compound eye camera) and / or the inner imaging unit 24 (monocular camera). In the game program in the present embodiment, the camera image CI is an image acquired by using one of the outer left imaging unit 23a and the outer right imaging unit 23b of the outer imaging unit 23 as a monocular camera.

  As described above, the first space 1 is a space in front of the boundary surface 3 when viewed from the virtual camera, and is a space surrounded by the boundary surface 3. The second space 2 is a space behind the boundary surface 3 when viewed from the virtual camera. Although not shown in FIGS. 20A and 21, there is a back wall BW surrounding the boundary surface 3. That is, the second space 2 is a space that exists between the boundary surface 3 and the back wall BW. An arbitrary image is pasted on the back wall BW. For example, an image representing a prepared space is pasted on the back wall BW, and the presence of the second space 2 that is a space is displayed behind the first space 1. That is, as viewed from the virtual camera, the first space 1, the boundary surface 3, the second space 2, and the back wall BW are arranged in this order from the front.

  However, as described above, the image processing program of the present invention is not limited to the game program, and these settings and rules do not limit the image processing program of the present invention. As shown in FIG. 22, the enemy object EO can move in the virtual three-dimensional space, and can travel between the first space 1 and the second space 2 through the boundary surface 3 described above. In the image displayed on the upper LCD 22 when the enemy object EO moves between the first space 1 and the second space 2 through the area captured by the virtual camera in the boundary surface 3, It is expressed that the enemy object EO moves through the real world image and moves from the back side to the near side, or from the near side to the far side.

  The screen displays the appearance of the enemy object EO that travels between the first space 1 and the second space 2 using openings (holes) created in the real world image due to game scenarios and events. In FIGS. 22 and 24, the enemy object EO makes an opening in the boundary surface 3 or passes through the opening already existing on the boundary surface 3 to move between the first space 1 and the second space 2. Is shown.

  In the image processing program in the present embodiment, there are three objects existing in the first space 1 or the second space 2, the enemy object EO, the bullet object BO, and the back wall BW. The program is not limited to the type of object. In the image processing program according to the present embodiment, the object is a virtual object that exists in the virtual space (the first space 1 and the second space 2). For example, in the image processing program according to the present embodiment, an arbitrary object such as an obstacle object may exist.

<Display example>
23 to 26 show examples of game screens displayed on the upper LCD 22. Hereinafter, examples of display forms in each figure will be described.

  First, the aiming cursor AL displayed in common in FIGS. 23 to 26 will be described. 23 to 26, the upper LCD 22 has a common aiming cursor AL for the bullet object BO that is fired in response to an attack operation using the game apparatus 10 (for example, pressing the button 14B (A button)). It is displayed. In the game program of the present embodiment, the aiming cursor AL is set to face a predetermined direction according to the program executed by the game apparatus 10.

  For example, the aiming cursor AL is set so as to be fixed in the viewing direction of the virtual camera, that is, in the center of the screen of the upper LCD 22. In this case, as described above, in the present embodiment, the virtual camera (the virtual camera in the first drawing method or the virtual world drawing camera in the second drawing method) according to the change in the imaging direction of the outer imaging unit 23 in the present embodiment. Since the line-of-sight direction is changed, the player can change the direction of the aiming cursor AL in the virtual space by changing the direction of the game apparatus 10. Then, for example, the player performs an attack operation by pressing the button 14B (A button) provided in the game apparatus 10 with the thumb of the right hand holding the lower housing 11. As a result, the player fires the bullet object BO by the attack operation, and extinguishes the enemy object EO on the game of the present embodiment or repairs the opening existing on the boundary surface 3.

  Next, FIGS. 23 to 26 will be described individually.

  In FIG. 23, the upper LCD 22 displays an enemy object EO present in the first space 1 and a camera image CI captured by a real camera built in the game apparatus 10. The enemy object EO is arbitrarily set.

  The enemy object EO uses, for example, an image (for example, a human face photograph) stored in the data storage external memory 46 of the game apparatus 10 as a texture, and uses a three-dimensional polygon model (3 of the human head) of a predetermined shape. The polygon model representing a dimensional shape) is an object pasted by a predetermined method.

  In the present embodiment, the camera image CI displayed on the upper LCD 22 is a real-time real world image captured by a real camera built in the game apparatus 10 as described above. In addition, for example, the camera image CI may be an image (for example, a landscape photograph) stored in the external data storage memory 46 of the game apparatus 10 or the like.

  In a state where the camera image CI is displayed on the upper LCD 22, the enemy object EO can arbitrarily move. For example, the enemy object EO existing in the first space 1 can move to the second space 2. FIG. 24 shows an example of how the enemy object EO existing in the first space 1 moves from the first space 1 to the second space 2. In the example shown in FIG. 24, the enemy object EO existing in the first space 1 moves to the second space 2 by creating an opening on the boundary surface 3. The enemy object EO that has moved to the second space 2 is displayed as a shadow (silhouette model) ES at a position viewed from the virtual camera in the non-opening region on the boundary surface 3. The second space 2 is visually recognized through the opening on the boundary surface 3. That is, when an opening exists on the boundary surface 3 in the visual field of the virtual camera, a part of the image of the second space 2 is displayed on the upper LCD 22 through the opening. Specifically, the image of the second space 2 is an object that exists in the second space 2, for example, an enemy object EO or a back wall BW that exists in the second space 2. The shadow ES is a display showing the shadow of the enemy object EO. FIG. 27A shows a top view of the silhouette model of the shadow of the enemy object EO. FIG. 27B shows an example of a silhouette model of the shadow of the enemy object EO. As shown in FIG. 27A and FIG. 27B, in this embodiment, a silhouette model is set for each of a plurality of directions for the enemy object EO. Specifically, the silhouette model is, for example, eight planar polygons shown in FIG. 27A. This silhouette model (eight planar polygons) is arranged at the same position as the enemy object EO that is a substantial model. Each planar polygon has a size included in the entity model (does not protrude from the entity model). Also, a texture drawn by a shadow image of the enemy object EO viewed from the normal direction of the surface is pasted to each planar polygon. When the enemy object EO is behind the non-opening area of the boundary surface 3, the shadow ES is displayed by drawing this silhouette model.

  Note that all eight planar polygons are rendered. When the enemy object EO exists behind the non-opening area of the boundary surface 3, the entity model of the enemy object EO is hidden by the boundary surface (screen object) 3 due to depth determination (depth determination), and thus is not drawn. However, since the silhouette model is set so as not to perform depth determination on the boundary surface (screen object) 3, the enemy object EO (and its silhouette model) exists behind the non-opening region of the boundary surface 3. Even in this case, the silhouette model is drawn and a shadow is displayed as shown in FIGS. However, when the enemy object EO exists in front of the boundary surface 3 or behind the opening area of the boundary surface 3, the silhouette model exists behind the entity model, and the entity model of the enemy object EO is drawn. As a result, the silhouette model is not drawn, so no shadow is displayed. This is because the silhouette model is set to be included in the entity model of the enemy object EO.

  Images are synthesized and displayed on the upper LCD 22 in the following priority order.

  (1) An image of an object existing in the first space 1 (2) For a non-opening region in the real world image, a combined image (for example, a shadow image of the shadow image of the object existing in the second space 2 and the real world image) (3) The opening area of the real world image is preferentially synthesized with the image of the object (substance image) existing in the second space 2 and further on the back. The wall image is synthesized. However, depending on the movement state of the enemy object EO existing in the second space 2, there is a scene where the enemy object EO exists over both the open area and the non-open area. That is, there is a scene where the enemy object EO is present at the edge of the opening at the position viewed from the virtual camera. FIG. 25 shows a state in which the enemy object EO existing in the second space 2 has moved to the edge of the opening set on the boundary surface 3. As shown in FIG. 25, regarding the enemy object EO existing in the second space 2, as viewed from the virtual camera, an image as it is in the range in which the second space 2 can be viewed through the opening, the second space 2 is viewed through the opening. The shadow ES is displayed on the upper LCD 22 for the range where it cannot.

  More specifically, as shown in FIG. 28, opacity (alpha value) data is set for each object. FIG. 28 shows an example of the opacity (alpha value) set for each object in the present embodiment. In the actual model of the enemy object, the texture is set to have an opacity of 1 for the entire model. In the silhouette model (planar polygon) of the enemy object, the opacity of the entire shadow image is set to 1 for the texture. The same applies to the bullet model. For the enemy object, a semi-transparent object model and an effect model, for example, an opacity of 0.6 is set for the entire model. In the screen object (spherical screen model shown in FIG. 20A), an opacity of 0.2 is set as a material, and 1 or 0 is set for each point in the α texture which is the texture. 1 indicates a non-opening and 0 indicates an opening. That is, the screen object is subjected to two opacity values: material setting and texture setting.

  Further, the depth determination is effective between the enemy object, the bullet object, the translucent enemy object, the effect object, and the screen object. "Between shadow plane polygons and enemy objects" "Between shadow plane polygons and bullet objects" "Between shadow plane polygons and translucent enemy objects" "Shadow plane polygons and effects “Between objects” is effective for depth determination. Depth determination is invalid between the shadow plane polygon and the screen object.

  When the depth determination is valid, rendering is performed according to normal perspective projection. The hidden surface is erased according to the depth direction from the virtual camera. When the depth determination is invalid, rendering is performed even if the target object is closer to the virtual camera than the object itself.

  In the present embodiment, a rendering expression can be set for each object during rendering. Specifically, it is set as follows.

  The enemy object entity, bullet object, translucent enemy object, and effect object are drawn using the following formula.

"Own color x own opacity + background color x (1-own opacity)"
The screen object is drawn with the following formula.

"Own color (color of real world image) x Opacity of own texture + Background color x (1-Opacity of own texture)"
The silhouette model of the enemy object is drawn by the following formula.

"Own color x (1-Opacity of background material) + Background color x Opacity of background material"
When an enemy object is drawn, the background is a screen object (boundary surface 3). Therefore, in the above equation, the “background material opacity” is “the opacity of the material of the screen object (boundary surface 3)”. Is.

  When the enemy object exists behind the non-opening portion of the boundary surface due to various settings as described above, the shadow is displayed without displaying the entity, and when the enemy object exists in front of the boundary surface and the enemy object Is present in the opening of the boundary surface, the entity is displayed without displaying the shadow.

  In the game according to the present embodiment, the opening existing on the boundary surface 3 can be repaired by hitting the bullet object BO. FIG. 26 shows a state in which a bullet object BO is applied to an opening existing on the boundary surface 3 to close the opening. As shown in FIG. 26, for example, when the bullet object BO collides with a non-opening area existing on the boundary surface 3, non-opening data is set for the boundary surface within a certain range from the collision point. Thereby, when there is an opening within a certain range from the collision point, the opening is closed. In this embodiment, the bullet object BO that hits the opening disappears (therefore, the bullet object BO disappears in FIG. 26). When the bullet object BO hits the opening on the boundary surface 3, it passes through the opening and moves to the second space.

  As described above, on the upper LCD 22, the real-time real world image captured by the real camera built in the game apparatus 10 is displayed as an image that looks as if it is on the boundary surface 3. By changing the direction of the game apparatus 10 in the real space, the imaging range captured by the game apparatus 10 changes, so that the camera image CI displayed on the upper LCD 22 also changes. In this case, the game apparatus 10 changes the direction of the virtual camera (for the second drawing method, a virtual world drawing camera) in the virtual space according to the movement of the game apparatus 10 in the real space. As a result, the enemy object EO displayed so as to be arranged in the real space and the opening existing on the boundary surface 3 have the same position in the real space even if the direction of the game apparatus 10 changes in the real space. It is displayed as if it is arranged in. For example, it is assumed that the imaging direction of the game device 10 by the real camera is changed to the left. In this case, the display position of the enemy object EO displayed on the upper LCD 22 or the opening on the boundary surface 3 moves in the opposite direction (right direction) to the direction in which the imaging direction of the real camera is changed, that is, the enemy object EO. And the direction of the virtual camera (virtual world drawing camera for the second drawing method) in the virtual space in which the opening existing on the boundary surface 3 is arranged moves to the left as the real camera. Accordingly, the opening present on the enemy object EO or the boundary surface 3 is arranged in the real space as if it is represented by the camera image CI even if the direction of the game apparatus 10 changes and the imaging range of the real camera changes. As shown in FIG.

《Image processing operation example》
Next, with reference to FIG. 29 to FIG. 31, FIG. 32A, and FIG. 32B, a specific example of the image processing operation by the image processing program in the present embodiment executed by the game apparatus 10 will be described. FIG. 29 is a flowchart illustrating an example of an operation example in which the game apparatus 10 performs image processing by executing the image processing program. FIG. 30 is a flowchart of a subroutine showing an example of detailed operation of the enemy object-related process performed in step 53 of FIG. FIG. 31 is a subroutine flowchart showing an example of detailed operation of the bullet object related process performed in step 54 of FIG. FIGS. 32A and 32B are flowcharts of subroutines showing an example of detailed operation of the display image update process (first drawing method and second drawing method) performed in step 57 of FIG. 29.

《Image processing example》
The operation of the information processing unit 31 will be described with reference to FIG. First, when the power supply (power button 14F) of the game apparatus 10 is turned on, a boot program (not shown) is executed by the CPU 311 and is thereby stored in the internal memory or the external memory 45 or the data storage external memory 46. Is loaded into the main memory 32. Then, when the loaded program is executed by the information processing unit 31 (CPU 311), the steps shown in FIG. 29 are executed. In FIGS. 29 to 32A, descriptions of image processing and other peripheral processing not directly related to the image processing of the present invention are omitted.

  In FIG. 29, the information processing section 31 performs initial setting in image processing (step 51), and proceeds to the next step 52. For example, the information processing unit 31 sets the initial position and initial direction of a virtual camera for generating a virtual world image (virtual space image) in the virtual camera data Dj, and the coordinate axis of the virtual space in which the virtual camera is arranged (For example, XYZ axes) is set. Next, the information processing unit 31 acquires various data from each component of the game apparatus 10 (step 52), and proceeds to the next step 53. For example, the information processing unit 31 updates the actual camera image data Db using the camera image captured by the currently selected image capturing unit (the outer image capturing unit 23 in the present embodiment). Further, the information processing section 31 acquires data indicating that the operation button 14 or the analog stick 15 has been operated, and updates the operator data Da1. Furthermore, the information processing section 31 acquires angular velocity data indicating the angular velocity detected by the angular velocity sensor 40, and updates the angular velocity data Da2.

  Next, the information processing section 31 executes enemy object related processing (step 53), and proceeds to the next step 54. Hereinafter, enemy object-related processing will be described with reference to FIG.

  In FIG. 30, the information processing section 31 determines whether or not the condition for the enemy object EO to appear is satisfied (step 61). For example, as a condition for the appearance of the enemy object EO, the enemy object EO may appear every predetermined time, or a new enemy object EO may appear when the enemy object EO disappears from the virtual world. Then, the enemy object EO may appear at random timing. The condition for the appearance of the enemy object EO is set by various program groups Pa held in the main memory 32, for example.

  Then, the information processing unit 31 proceeds to the next step 62 when the condition for the appearance of the enemy object EO is satisfied. On the other hand, the information processing unit 31 proceeds to the next step 63 when the condition for the appearance of the enemy object EO is not satisfied.

  In step 62, the information processing section 31 generates enemy object data Df corresponding to the enemy object EO that satisfies the appearance condition, performs initial setting, and proceeds to the next step 63. For example, the information processing unit 31 uses the various program groups Pa stored in the main memory 32 to acquire the entity data Df1, the silhouette data Df2, the opening shape data Df3, and polygon data corresponding to the enemy object EO. . Then, the information processing section 31 generates enemy object data Df including the data. In addition, for example, the information processing unit 31 includes, in the generated enemy object data Df, data indicating the arrangement direction and position of the polygon corresponding to the enemy object EO in the virtual space, and the enemy object EO in the virtual space. Initial setting of data indicating the moving speed and moving direction is performed. The initial setting is performed by a known method.

  Next, the information processing section 31 moves the enemy object EO arranged in the virtual space (step 63), and proceeds to the next step 64. As an example, the information processing unit 31 uses data indicating the placement position of the enemy object EO included in the enemy object data De to indicate the moving speed and moving direction of the enemy object EO included in the enemy object data De in the virtual space. Update based on data. At this time, the information processing section 31 updates the data indicating the arrangement direction of the enemy object EO included in the enemy object data De based on the data indicating the movement direction. After the update, data indicating the moving speed and moving direction of the enemy object EO in the virtual space included in the enemy object data De may be arbitrarily updated. By updating the data indicating the moving speed and moving direction, the enemy object EO can move in an arbitrary speed and in an arbitrary direction in the virtual space.

  Next, the information processing section 31 determines whether or not the enemy object EO has reached a certain distance from the position of the virtual camera (the virtual camera in the first drawing method or the virtual world drawing camera in the second drawing method). Determination is made (step 64). For example, the information processing unit 31 includes data indicating the arrangement position of the enemy object EO included in the enemy object data De, and a virtual camera (virtual camera in the first drawing method or the second drawing method) included in the virtual camera data Dj. The data indicating the arrangement position of the virtual world rendering camera in FIG. When the two data satisfy a predetermined condition (for example, the distance between the placement position of the enemy object EO and the placement position of the virtual camera is smaller than a predetermined value), the information processing unit 31 When it is determined that the object EO has reached a certain distance from the position of the virtual camera and the two data do not satisfy the predetermined condition, it is determined that the enemy object EO has not reached the position of the virtual camera. In the following, when the first drawing method and the second drawing method are not distinguished and are simply expressed as “virtual camera”, the virtual camera in the first drawing method or the virtual world drawing camera in the second drawing method is used. Point to. If the information processing unit 31 determines that the enemy object EO has reached the position of the virtual camera, the information processing unit 31 proceeds to the next step 65. On the other hand, if the information processing section 31 determines that the enemy object EO has not reached the position of the virtual camera, the information processing section 31 proceeds to step 66.

  In step 65, the information processing section 31 performs a point deduction process and proceeds to the next step 66. For example, the information processing section 31 deducts the score of the game indicated by the score data Dh by a predetermined value, and updates the score data Dh using the score after deduction. In the deduction process, the information processing section 31 performs a process of eliminating the enemy object EO that has reached the position of the virtual camera from the virtual space (for example, enemy object data De regarding the enemy object EO that has reached the position of the virtual camera). May be performed so that the enemy object EO does not exist in the virtual space). In addition, the predetermined value in the deduction process may be an arbitrary value, and may be set by various program groups Pa held in the main memory 32, for example.

  In step 66, the information processing section 31 determines whether the enemy object EO passes through the boundary surface 3 (the enemy object EO moves between the first space 1 and the second space 2). For example, the information processing unit 31 compares data indicating the arrangement position of the enemy object EO included in the enemy object data De with data indicating the arrangement position of the boundary surface 3 included in the boundary surface data Dd. Then, the information processing unit 31 determines that the enemy object EO passes through the boundary surface 3 when the two data satisfy the predetermined condition, and determines that the enemy object EO is the boundary when the two data does not satisfy the predetermined condition. It is determined that the surface 3 is not passed. The predetermined condition is, for example, that the coordinates (arrangement position) of the enemy object EO in the virtual space satisfy the conditional expression of the sphere surface of the boundary surface 3. As described above, the data indicating the arrangement position of the boundary surface 3 in the virtual space indicates the existence range of the boundary surface 3 in the virtual space. It is. When the arrangement position of the enemy object EO satisfies the conditional expression, the enemy object EO exists on the boundary surface 3 in the virtual space. In the present embodiment, for example, in such a case, it is determined that the enemy object EO passes through the boundary surface 3.

  If the information processing unit 31 determines that the enemy object EO passes through the boundary surface 3, the information processing unit 31 proceeds to the next step 67. On the other hand, if the information processing section 31 determines that the enemy object EO does not pass through the boundary surface 3, the information processing section 31 ends the processing by the subroutine.

  In step 67, the information processing section 31 updates the opening determination data included in the boundary surface data Dd, and ends the processing by the subroutine. This process is a process for registering opening information on the boundary surface 3 that is created when the enemy object EO passes through the boundary surface 3 in the boundary surface data Dd. For example, in the first drawing method and the second drawing method, the information processing unit 31 sets the opening determination data included in the boundary surface data Dd to a position on the virtual space where the enemy object EO passes through the boundary surface 3. The alpha value of the area centered on the corresponding position is multiplied by the alpha value of the aperture shape data Df3. The opening shape data Df3 is texture data in which an alpha value “0” is stored with the arrangement position of the enemy object EO as the center. Therefore, the alpha value of the opening determination data in the region where the opening is generated around the position where the enemy object EO is arranged (coordinates on the boundary surface 3 through which the enemy object EO passes) is “0”. That is, the information processing unit 31 can update the state of the boundary surface (specifically, data for opening determination) without determining whether or not an opening already exists on the boundary surface 3. Note that it may be determined whether an opening already exists at the collision position between the enemy object and the boundary surface. When there is no opening, an effect may be displayed in which the real world image corresponding to the collision position is scattered as a fragment.

  In addition, in the opening determination data updating process, the information processing section 31 may perform an effect process that an opening has been generated (for example, an effect that causes a wall to collapse at the position where the opening is generated). . In this case, the information processing unit 31 needs to determine whether or not the position where the enemy object EO passes through the boundary surface 3 (the range in which the opening is generated) is already in the open state. For example, the information processing section 31 multiplies data obtained by inverting the alpha value of the opening shape data Df3 from “0” to “1” by the alpha value of the multiplied opening determination data, thereby generating an opening. It can be determined whether the range is already open. That is, when the entire range in which the opening is generated is already in the opening state, the alpha value of the opening determination data is “0”, and thus the multiplication result is “0”. On the other hand, if even a part of the range for generating the opening is not in the opening state, the multiplication result is other than “0” because there is a portion where the alpha value of the opening determination data is not “0”.

  The opening shape data Df3 of the enemy object EO is texture data in which an alpha value “0” corresponding to the shape of the enemy object EO is stored. However, the information processing unit 31 performs an alpha value of the texture data according to a predetermined event. May be converted to “1”. When the process is performed after the change, the alpha value of the opening determination data is not changed because the alpha value of the opening shape data Df3 is “1”. In this case, the enemy object EO passes through the boundary surface 3 without making an opening. That is, it is possible to produce an effect as if the enemy object EO slips through the boundary surface 3 (see FIG. 22). The predetermined event may be, for example, a random number or a time interval determined by a predetermined interval, or may be that a predetermined condition on the game is satisfied. These may be set by various program groups Pa held in the main memory 32, for example.

  Returning to FIG. 29, after the enemy object related processing in step 53, the information processing section 31 performs bullet object related processing (step 54), and proceeds to the next step 55. Hereinafter, bullet object related processing will be described with reference to FIG.

  In FIG. 31, the information processing section 31 moves the bullet object BO in the virtual space according to the set moving speed vector (step 71), and proceeds to the next step 72. For example, the information processing unit 31 updates the data indicating the arrangement direction and the arrangement position of the bullet object BO based on the data indicating the moving speed vector included in the bullet object data Dg. At this time, the information processing section 31 may update the data indicating the movement speed vector by a known method. For example, the information processing unit 31 may change the update method of data indicating the moving speed vector according to the type of the bullet object BO. For example, when the bullet object BO is a ball, the information processing unit 31 may update the data indicating the movement speed vector in consideration of the influence of gravity in the vertical direction in the virtual space.

  Next, the information processing section 31 determines whether or not a launch operation has been performed by the user of the game apparatus 10 (step 72). For example, the information processing section 31 refers to the operator data Da1 and determines whether or not the user has performed a predetermined firing operation (for example, pressing the button 14B (A button)). Then, when the launch operation is performed, the information processing section 31 proceeds to the next step 73. On the other hand, the information processing unit 31 proceeds to the next step 74 when the launch operation is not performed.

  In step 73, the information processing section 31 arranges the bullet object BO at the position of the virtual camera in the virtual space in accordance with the launch operation, sets the moving speed vector for the bullet object BO, and proceeds to the next step 74. To proceed. For example, the information processing unit 31 generates bullet object data Dg corresponding to the firing operation. Then, for example, the information processing section 31 uses data indicating the placement position and placement direction (gaze direction) of the virtual camera included in the virtual camera data Dj as the placement position of the bullet object BO included in the generated bullet object data Dg, Store in the data indicating the direction of placement. Further, for example, the information processing unit 31 stores an arbitrary value in the data indicating the moving speed vector included in the generated bullet object data Dg. The value stored in the data indicating the moving speed vector may be set by various program groups Pa held in the main memory 32.

  In step 74, the information processing section 31 determines whether or not the enemy object EO and the bullet object BO have contacted each other in the virtual space. For example, the information processing unit 31 compares the data indicating the placement position of the enemy object EO included in the enemy object data Df with the data indicating the placement position of the bullet object BO included in the bullet object data Dg. It is determined whether the EO and the bullet object BO are in contact with each other in the virtual space. For example, the information processing unit 31 determines that the enemy object EO and the bullet object BO are in contact when the data indicating the arrangement position of the enemy object EO and the data indicating the arrangement position of the bullet object BO satisfy a predetermined condition. To do. Otherwise, the information processing section 31 determines that the enemy object EO and the bullet object BO are not in contact. The predetermined condition is, for example, that the distance between the arrangement position of the enemy object EO and the arrangement position of the bullet object BO is smaller than a predetermined value. The predetermined value may be a value based on the size of the enemy object EO, for example.

  If the information processing unit 31 determines that the enemy object EO and the bullet object BO have contacted each other, the information processing unit 31 proceeds to the next step 75. On the other hand, if the information processing unit 31 determines that the enemy object EO and the bullet object BO are not in contact, the information processing unit 31 proceeds to the next step 76.

  In step 75, the information processing section 31 performs a point addition process and proceeds to the next step 76. For example, in the above score process, the information processing section 31 scores a predetermined number of game scores indicated by the score data Dh, and updates the score data Dh using the score after the score. In addition, in the point addition process, the information processing section 31 performs a process (for example, a bullet object BO) that causes both of the touches determined to have contacted in step 84 (that is, the enemy object EO and the bullet object BO) to disappear from the virtual space. The enemy object data De and the bullet object data Dg relating to the enemy object EO that has touched and the bullet object BO that has come into contact with the enemy object EO are initialized, and the enemy object EO and the bullet object BO are not in the virtual space). Do. The predetermined value in the point addition process may be an arbitrary value, and may be set by various program groups Pa held in the main memory 32, for example.

  In step 76, the information processing section 31 determines whether or not the bullet object BO has contacted the non-opening area on the boundary surface 3. For example, the information processing unit 31 uses the data indicating the arrangement position of the bullet object BO included in the bullet object data Dg and the opening determination data, so that the bullet object BO contacts the non-opening area on the boundary surface 3. It is determined whether or not.

  For example, the information processing unit 31 determines whether or not the data indicating the placement position of the bullet object BO included in the bullet object data Dg satisfies the conditional expression of the sphere surface of the boundary surface 3 as in the processing of the enemy object BO. Determine. Then, the information processing unit 31 determines that the bullet object BO is not in contact with the boundary surface 3 when the data indicating the arrangement position of the bullet object BO does not satisfy the conditional expression of the sphere surface. On the other hand, when the data indicating the arrangement position of the bullet object BO satisfies the conditional expression of the sphere surface of the boundary surface 3, the bullet object BO exists on the boundary surface 3 in the virtual space. At this time, for example, the information processing section 31 acquires the alpha value of the opening determination data for a predetermined area centered on a position corresponding to the position on the boundary surface 3 where the bullet object BO exists. This predetermined area is a predetermined area centered on the contact point between the bullet object BO and the boundary surface 3. When the alpha value of the opening determination data corresponding to at least a part of the predetermined area is “1” corresponding to the non-opening area, the bullet object BO has contacted the non-opening area on the boundary surface 3. Is determined.

  If the information processing unit 31 determines that the bullet object BO has contacted the non-opening region on the boundary surface 3, the information processing unit 31 proceeds to the next step 77. On the other hand, when the information processing unit 31 determines that the bullet object BO is not in contact with the non-opening region on the boundary surface 3, the information processing unit 31 proceeds to the next step 78.

  In step 77, the information processing section 31 updates the opening determination data and proceeds to the next step 78. For example, in the update process, the information processing section 31 opens a predetermined area centered on a position on the boundary surface 3 corresponding to the arrangement position of the bullet object BO determined to be in contact with the non-opening area of the boundary surface 3. The alpha value of the determination data is updated to the alpha value “1” corresponding to the non-opening area. When the bullet object BO comes into contact with the non-opening area by this update process, all the alpha values of the opening determination data in the predetermined area centered on the contact point are updated to “1”. Therefore, if there is a portion where the alpha value of the opening determination data is set to “0” within a predetermined area centered on the contact point, the alpha value of the opening determination data in that portion is also set to “1”. Will be updated. That is, when the bullet object BO comes into contact with the edge of the opening provided on the boundary surface 3, the opening included in the predetermined area centered on the contacted position is restored to the non-opening state. Further, in the update process, the information processing section 31 performs a process of eliminating the bullet object BO determined to have been contacted in step 76 from the virtual space (for example, a bullet object that has contacted the non-opening area on the boundary surface 3). The bullet object data Dg related to BO is initialized, and the bullet object BO does not exist in the virtual space). Note that the predetermined area in the update process may be an arbitrary area, and may be set by various program groups Pa held in the main memory 32, for example.

  In step 78, the information processing section 31 determines whether or not the bullet object BO has reached a predetermined position in the virtual space. The predetermined position may be, for example, a position where the back wall BW exists in the virtual space. In this case, for example, the information processing section 31 determines whether data indicating the arrangement position of the bullet object BO included in the bullet object data Dg collides with the back wall BW.

  Then, when the bullet object BO has reached a predetermined position, the information processing section 31 proceeds to the next step 77. On the other hand, when the bullet object BO has not reached the predetermined position, the information processing section 31 ends the processing by the subroutine.

  In step 77, the information processing section 31 performs a process of eliminating the bullet object BO determined to have reached the predetermined position in step 76 from the virtual space, and ends the process of the subroutine. For example, the information processing section 31 initializes the bullet object data Dg related to the bullet object BO, for example, to erase the bullet object BO determined to have reached the predetermined position in step 76 from the virtual space. The object BO does not exist in the virtual space).

  Returning to FIG. 29, after the bullet object related processing in step 54, the information processing section 31 calculates the movement of the game apparatus 10 (step 55), and proceeds to the next step 56. As an example, the information processing unit 31 calculates the movement of the game apparatus 10 (for example, the change in the imaging direction in the real camera provided in the game apparatus 10) using the angular velocity indicated by the angular velocity data Da2, and uses the movement. To update the motion data Di. Specifically, when the user changes the imaging direction of the real camera provided in the game device 10 in real space, the orientation of the game device 10 as a whole also changes, so the game device 10 has an angular velocity corresponding to the change. Arise. And the angular velocity sensor 40 detects the angular velocity which arose in the game device 10, and the data which show the said angular velocity are stored in the angular velocity data Da2. Therefore, the information processing unit 31 uses the angular velocity indicated by the angular velocity data Da2 to calculate the direction in which the imaging direction of the real camera provided in the game device 10 has changed and the amount (angle) of the change as the motion of the game device 10. It becomes possible to do.

  Next, the information processing section 31 changes the direction of the virtual camera in the virtual space according to the movement of the game apparatus 10 (step 56), and proceeds to the next step 57. For example, the information processing unit 31 uses the motion data Di to give the same change in the imaging direction of the real camera of the game device 10 in the real space to the virtual camera in the virtual space, and the position and direction of the virtual camera after the change Is used to update the virtual camera data Dj. As an example, when the imaging direction of the real camera of the game device 10 in the real space changes A ° to the left, the direction of the virtual camera in the virtual space also changes A ° to the left. Thereby, the enemy object EO and the bullet object BO displayed so as to be arranged in the real space are arranged at the same position in the real space even if the direction and the position of the game apparatus 10 in the real space are changed. Is displayed.

  Next, the information processing section 31 performs a display screen update process (step 57), and proceeds to the next step 58. Hereinafter, the display screen update process will be described with reference to FIGS. 32A and 32B. FIG. 32A shows display screen update processing in the first drawing method. FIG. 32B shows display screen update processing in the second drawing method.

  First, display screen update processing in the first drawing method will be described.

  In FIG. 32A, the information processing section 31 performs a process of pasting the real camera image acquired in the above step 52 on the screen object (boundary surface 3) within the visual volume of the virtual camera (step 81). Proceed with the process. For example, the information processing section 31 updates the texture data of the actual camera image included in the actual world image data Dc using the actual camera image data Db updated in step 52. Then, the information processing unit 31 uses the data indicating the arrangement direction and the arrangement position of the virtual camera in the virtual space included in the virtual camera data Dj to obtain a point where the visual line direction of the virtual camera and the boundary surface 3 overlap. The information processing unit 31 pastes the texture data of the real camera image included in the real world image data Dc around the obtained point, and updates the boundary surface data Dd. At this time, the information processing section 31 acquires the opening determination data set in the area where the texture data is pasted for the area corresponding to each pixel of the texture data. Then, the information processing section 31 applies the alpha value (“0” or “0.2”) set in the acquired aperture determination data to the texture data. Specifically, the information processing unit 31 multiplies the color information of each pixel of the texture data of the actual camera image to be pasted by the alpha value at the corresponding position of the aperture determination data. By this processing, as described above, the opening is expressed in the real world image. The alpha value “0.2” (non-aperture area) stored in the aperture determination data is handled as the alpha value “1” as the above-described material in the multiplication. In the present embodiment, the texture data of the actual camera image pasted on the boundary surface 3 is image data having a wider area than the visual field range of the virtual camera C0.

  Next, the information processing section 31 generates a display image by processing for rendering the virtual space (step 82), and ends the processing by the subroutine. For example, the information processing unit 31 generates an image obtained by rendering a virtual space in which the boundary surface 3 (screen object), the enemy object EO, the bullet object EO, and the back wall BW are arranged, and the rendered image is used by using the image. The rendering image data of the virtual space included in the data Dk is updated. Further, the information processing unit 31 updates the display image data Dl using the rendering image data in the virtual space. Hereinafter, an example of the rendering process will be described with reference to FIGS. 33 and 34.

  FIG. 33 shows an arrangement example of the enemy object EO, the bullet object BO, the boundary surface 3 (screen object for which opening determination data is set), and the back wall BW in the virtual space. FIG. 34 shows the positional relationship of each object when the virtual camera C0 in FIG. 33 is oriented in the (X, Y, Z) = (0, 0, 1) direction from the origin. Thus, the data indicating the placement positions of the enemy object EO, the bullet object BO, the boundary surface 3 and the back wall BW included in the enemy object data Df, the bullet object data Dg, the boundary surface data Dd and the back wall image data De, respectively. Are arranged according to. In the virtual space, a virtual camera C0 for rendering the virtual space is arranged according to data indicating the arrangement direction and the arrangement position included in the virtual camera data Dj.

  The information processing section 31 includes the boundary surface 3 as shown in FIG. 33 (or FIG. 34), and renders the enemy object EO, the bullet object BO, and the back wall BW arranged in the virtual space by perspective projection with the virtual camera C0. . At this time, the information processing unit 31 considers drawing priority information. In normal perspective projection, the object existing in the second space 2 is not drawn by the boundary surface 3. In the game of the present embodiment, an opening is provided in the boundary surface 3 (real world image) so that a part of the second space 2 can be seen from the opening. In addition, the shadow of the object in the second space 2 is combined with the real world image and drawn. By doing so, it is possible to give the user a feeling as if there is a virtual world beyond the real world image. Specifically, the information processing section 31 performs the rendering process using drawing priority information. Note that the image processing program in the present embodiment uses an alpha value as an example of the drawing priority.

  In the perspective projection, an object (in this embodiment, the enemy object EO or the back wall BW) existing in the second space 2 exists behind the boundary surface 3. Here, the boundary surface 3 is a screen object in which the texture data of the real camera image is pasted in the visual field direction (visual field range) of the virtual camera C0 in step 81 described above. Further, as described above, aperture determination data corresponding to each position is applied to the texture data of the actual camera image. Therefore, a real world image to which the aperture determination data is applied exists in the visual field range of the virtual camera C0.

  In the present embodiment, for example, the information processing unit 31 has an area (opening area) in which the alpha value “0” is stored in the opening determination data in the area that can be seen from the opening area in the second space 2. The virtual object or the image of the back wall BW is drawn (rendered). In addition, the information processing unit 31 stores an area where the alpha value “0.2” corresponding to the non-opening area is stored in the opening determination data (an area treated as an area where the alpha value “1” is stored as the non-opening area) ), The virtual object and the back wall BW existing in the second space 2 are not drawn. That is, in the displayed image, the real world image pasted in the above-described step 81 is drawn in the portion corresponding to the area.

  Therefore, the region in which “0” is stored in the aperture determination data as viewed from the virtual camera C0 is rendered so that the image data included in the entity data Df1 or the back wall image data De is drawn. Then, on the upper LCD 22, a virtual object or an image of the back wall BW is displayed in a portion corresponding to the area.

  In addition, an area in which the alpha value “0.2” indicating the non-opening area is stored in the opening determination data as viewed from the virtual camera C0 (an area handled as an area in which the alpha value “1” is stored as the non-opening area) As for, the virtual object and the back wall BW existing in the second space 2 are not drawn. That is, in the image displayed on the upper LCD 22, a real world image is drawn in a portion corresponding to the area. However, for the shadow ES (silhouette model) of the enemy object EO existing in the second space 2 described above, depth determination with the boundary surface 3 is set to be invalid, and the alpha value “0. Since the alpha value “1” of the silhouette model is larger than “2”, the alpha value “1” indicating the non-opening area is stored (the alpha value “0.2” is stored in the opening determination data). Drawn). Thereby, an image of the shadow ES is drawn on the real world image.

  In addition, the shadow ES (silhouette model) of the enemy object EO is sized to be included in the entity model and arranged as such, and the entity model of the enemy object EO and the silhouette model of the shadow ES are set to enable depth determination. When the enemy object EO exists in the first space 1, the silhouette model is not drawn because it is hidden by the entity model.

  In addition, since the shape of the boundary surface 3 in this embodiment is the center part of the spherical surface as shown in FIG. In this case, the above processing is performed on the assumption that there is aperture determination data in which an alpha value “0.2” is stored in a pseudo manner. That is, the area where the opening determination data does not exist is handled as an area in which the alpha value “1” indicating the non-opening area is stored.

  Further, the silhouette data Df2 included in the enemy object data Df corresponding to the enemy object EO in the present embodiment is set so that a plurality of planar polygons have respective radial directions when the normal direction is viewed from the enemy object EO. The texture of a silhouette image obtained by viewing the enemy object EO from the direction is applied to each planar polygon. Therefore, in the image processing program in the present embodiment, the shadow of the enemy object EO in the virtual space image is expressed as an image reflecting the direction of the enemy object EO in the second space 2.

  Further, the information processing section 31 performs the rendering process so that the image data included in the aiming cursor image data Dm is drawn with priority over the center of the visual field of the virtual camera C0 (the center of the image to be rendered).

  Through the above processing, the information processing section 31 renders the enemy object EO, the bullet object BO, and the back wall BW arranged in the virtual space by perspective projection, and displays the virtual world image (the aim cursor AL viewed from the virtual camera C0). Including the image) and updating the rendering image data in the virtual space (step 82). Then, the display image data Dl is updated using the updated rendering image data of the virtual space.

  Next, display screen update processing in the second drawing method will be described.

  In FIG. 32B, the information processing section 31 performs processing for rendering the actual camera image acquired in step 52 (step 83), and proceeds to the next step 84. For example, the information processing section 31 updates the texture data of the actual camera image included in the actual world image data Dc using the actual camera image data Db updated in step 52. Then, the information processing unit 31 generates an image obtained by rendering the real camera image using the updated real world image data Dc, and uses the generated image to render the real camera image included in the rendering image data Dk. Update the data. Hereinafter, a processing example for rendering an actual camera image will be described with reference to FIGS. 35 and 36.

  In this embodiment, as shown in FIG. 35, the information processing section 31 sets a real camera image obtained from a real camera of the game apparatus 10 as a texture, and generates a planar polygon to which the texture is mapped. Then, the information processing section 31 generates, as a real world image, an image obtained by rendering the planar polygon by parallel projection from the real world image drawing camera C1. Here, an example of a real world image generation method in the case where the entire real camera image obtained from the real camera of the game apparatus 10 is displayed on the entire display screen of the upper LCD 22 will be described. In the present embodiment, the composite image (composite image of the real world image and the virtual world image) of the present embodiment is displayed on the entire display screen of the upper LCD 22, but a partial area of the display screen of the upper LCD 22. A composite image may be displayed on the screen. In this case, the entire real camera image is displayed on the entire composite image.

  First, consider a planar polygon that maps an i-pixel texture to one unit of coordinates in the virtual space where the planar polygon is placed. In this case, a texture of i pixels × i pixels is mapped to an area of 1 unit × 1 unit of the coordinates. Here, it is assumed that the display screen of the upper LCD 22 is horizontal Wdot × vertical Hdot, and the entire display screen of Wdot × Hdot corresponds to the entire texture of the actual camera image. That is, the size of the texture data of the camera image is assumed to be W horizontal pixels × H vertical pixels.

  In this case, it is only necessary to consider arranging the plane polygon so that 1 dot × 1 dot of the display screen corresponds to the texture 1 pixel × 1 pixel of the actual camera image, and the coordinates are determined as shown in FIG. That's fine. That is, the virtual space in which the plane polygon is arranged so that the width of the plane polygon in which the texture of the camera image is mapped on the entire main surface is the W / i coordinate and the height of the plane polygon is the H / i coordinate. Set the XY coordinates. Then, the principal surface center of the planar polygon to which the texture is mapped is made coincident with the origin of the XY coordinates of the virtual space, the horizontal direction of the planar polygon is the X-axis direction (the right direction is the X-axis positive direction), and the planar polygon The planar polygons are arranged so that the vertical direction is the Y-axis direction (upward is the positive Y-axis direction). In this case, the upper right corner position on the main surface of the planar polygon to which the texture is mapped is arranged at (X, Y) = (W / 2i, H / 2i), and the lower right corner position is (X, Y) = (W / 2i, -H / 2i), the upper left corner position is placed at (X, Y) = (-W / 2i, H / 2i), and the lower left corner position is (X, Y) = (-W / 2i, -H / 2i).

  By arranging in this way, the area of 1 unit × 1 unit of the coordinates corresponds to the area of i pixels × i pixels of the texture, so that the horizontal (W / i) × vertical (H / i) of the planar polygon. The region corresponds to the size of W pixels × H pixels of the texture.

  As described above, by rendering the plane polygon arranged at the coordinates of the virtual space by parallel projection so that 1 dot on the display screen corresponds to 1 pixel of the real camera image (texture), the game device 10 can be actually displayed. A real world image corresponding to the real camera image obtained from the camera is generated.

  As described above, the texture data of the real camera image included in the real world image data Dc is updated by the real camera image data Db, and the size of the horizontal A × vertical B of the image of the real camera image data Db There is a case where the size of horizontal W × longitudinal H of the texture data does not match. In this case, the information processing unit 31 updates the texture data by an arbitrary method. For example, the information processing unit 31 uses an image obtained by enlarging and reducing the size of the horizontal A and vertical B of the image of the actual camera image data Db so as to match the W × H size image (texture data image). The texture data may be updated. For example, it is assumed that the horizontal A and vertical B sizes of the image of the actual camera image data Db are larger than the horizontal W and vertical H sizes of the texture data. In this case, for example, the information processing unit 31 may cut out a W × H size image (texture data image) from a predetermined position of the image of the actual camera image data Db and update the texture data. For example, it is assumed that at least one of the horizontal A and vertical B sizes of the image of the actual camera image data Db is smaller than the horizontal W and vertical H sizes of the texture data. In this case, for example, the information processing unit 31 enlarges the image of the actual camera image data Db so as to be larger than the size of the texture data, and then, from a predetermined position of the enlarged image, the W × H size image ( The texture data image) may be cut out and the texture data may be updated.

  In the present embodiment, the horizontal × vertical size of the display screen of the upper LCD 22 and the horizontal × vertical size of the texture data of the actual camera image match, but they do not have to match. In this case, the display screen of the upper LCD 22 and the size of the real world image do not match, but the information processing unit 31 uses a known method to display the real world image on the display screen of the upper LCD 22. You can change the size of.

  Next, as illustrated in FIG. 32B, the information processing section 31 performs processing for rendering the virtual space (step 84), and proceeds to the next step 85. For example, the information processing unit 31 generates an image obtained by rendering a virtual space in which the enemy object EO, the bullet object BO, and the back wall BW are arranged in consideration of the opening determination data, and uses the image, The rendering image data of the virtual space included in the rendering image data Dk is updated. Hereinafter, an example of the rendering process will be described with reference to FIGS.

  FIG. 37 shows an arrangement example of the enemy object EO, the bullet object BO, the boundary surface 3 (opening determination data), and the back wall BW in the virtual space. FIG. 38 shows the position of each object when the virtual camera (virtual world drawing camera) in FIG. 37 is oriented in the (X, Y, Z) = (0, 0, −1) direction from the origin. Show the relationship. Thus, the data indicating the placement positions of the enemy object EO, the bullet object BO, the boundary surface 3 and the back wall BW included in the enemy object data Df, the bullet object data Dg, the boundary surface data Dd and the back wall image data De, respectively. It is assumed that they are arranged according to In the virtual space, a virtual world drawing camera C2 for rendering the virtual space is arranged according to data indicating the arrangement direction and the arrangement position included in the virtual camera data Dj.

  First, the position of the boundary surface 3 (opening determination data) will be described. As described above, in the image processing program according to the present embodiment, the aperture determination data and the color information of the real world image (rendered image data of the actual camera image) are multiplied to generate a real image with an aperture. Is done. Therefore, for example, the horizontal coordinate x vertical coordinate of the rendering image data of the real camera image (refer to the positional relationship of the planar polygons in FIGS. 35 and 36) and the boundary surface 3 in the virtual space (specifically, aperture determination) Suppose that 1 horizontal coordinate × 1 vertical coordinate of (data) corresponds. That is, when the boundary surface 3 is viewed through the perspective projection from the virtual world drawing camera C2 shown in FIG. 37 or 38, the visual field range of the boundary surface 3 by the virtual world drawing camera C2 is the rendering image data of the real camera image. Assume that the size corresponds to the horizontal x vertical size.

  FIG. 39 shows an example of the positional relationship between the virtual world drawing camera C <b> 2 and the boundary surface 3. Suppose that the boundary surface 3 is perspective-projected by the virtual world drawing camera C2 oriented in the (X, Y, Z) = (0, 0, −1) direction from the origin. In this case, if the boundary surface 3 is arranged at the position of Z = Z0 shown in FIG. 39, the horizontal coordinate of the aperture determination data × the vertical coordinate of 1 and the horizontal coordinate of the rendered image data of the actual camera image × One vertical coordinate corresponds. Here, the position of Z = Z0 is determined from the gazing point of the virtual world drawing camera C2 when the angle of view in the Y axis direction of the virtual world drawing camera C2 that perspectively projects the boundary surface 3 is θ. This is the position where the length of the display range in the direction is H / 2i. As described above, H is the number of vertical dots on the display screen of the upper LCD 22, and i is the number of texture pixels to be mapped to one unit of coordinates in the virtual space. When the distance from the center of the virtual world drawing camera C2 to the position of Z = Z0 is D (D> 0), the following formula 1 is established.

tan θ = (H / 2i) / D = H / 2Di
Therefore, when a virtual world image is generated by perspective projection of an enemy object EO or the like, which will be described later, in consideration of the boundary surface 3, the virtual world drawing camera C2 that generates the virtual world image has a “viewing angle in the Y-axis direction”. θ = tan−1 (H / 2Di), aspect ratio = W: H ”. The boundary surface 3 (specifically, opening determination data indicating the state of the boundary surface 3) is arranged at view coordinates Z = Z0 from the virtual world drawing camera C2. Thereby, the visual field range of the boundary surface 3 by the virtual world drawing camera C2 becomes the size W × H.

  Next, a virtual space rendering process will be described. Assuming that the boundary surface 3 exists at the above position, the information processing section 31 generates an image in which the virtual space is rendered. The information processing unit 31 performs the rendering process in consideration of later synthesis of the real world image. Hereinafter, an example of the rendering process will be specifically described.

  The information processing section 31 assumes that the boundary surface 3 exists as shown in FIG. 37 (or FIG. 38), and draws the enemy object EO, the bullet object BO, and the back wall BW arranged in the virtual space for virtual world drawing. Rendering is performed by perspective projection by the camera C2. At this time, the information processing unit 31 considers drawing priority information. In normal perspective projection, rendering is performed such that an object in front of the virtual camera is drawn preferentially in the virtual space. Therefore, in normal perspective projection, since the boundary surface 3 exists, the object in the second space 2 is not drawn. In the game of the present embodiment, an opening is provided in the boundary surface 3 (real world image) so that a part of the second space 2 can be seen from the opening. In addition, the shadow of the object in the second space 2 is combined with the real world image and drawn. By doing so, it is possible to give the user a feeling as if there is a virtual world beyond the real world image. Specifically, the information processing section 31 performs the rendering process using drawing priority information. Note that the image processing program in the present embodiment uses an alpha value as an example of the drawing priority information.

  In the perspective projection, an object (in this embodiment, the enemy object EO or the back wall BW) existing in the second space 2 exists behind the boundary surface 3. Here, opening determination data is set on the boundary surface 3. As described above, the opening determination data is rectangular texture data storing an alpha value, and each coordinate on the texture data corresponds to each position on the boundary surface of the virtual space. Therefore, the information processing section 31 can specify the area of the aperture determination data corresponding to the visual field range of the object existing in the second space 2 by the virtual world drawing camera C2.

  In the present embodiment, for example, the information processing unit 31 has an area (opening area) in which the alpha value “0” is stored in the opening determination data in the area that can be seen from the opening area in the second space 2. Draw (render) a virtual object or back wall image. In addition, the information processing unit 31 stores an area where the alpha value “0.2” corresponding to the non-opening area is stored in the opening determination data (an area treated as an area where the alpha value “1” is stored as the non-opening area) ), A virtual object and a back wall existing in the second space 2 are not drawn. That is, a real world image is drawn in a portion corresponding to the region in the displayed image by a combining process in step 85 described later.

  Therefore, the region in which “0” is stored in the opening determination data as viewed from the virtual world drawing camera C2 is rendered so that the image data included in the entity data Df1 or the back wall image data De is drawn. The Then, a virtual object / back wall image is displayed on the upper LCD 22 in a portion corresponding to the region by a combining process in step S85 described later.

  In addition, an area where the alpha value “0.2” indicating the non-aperture area is stored in the aperture determination data as viewed from the virtual world drawing camera C2 (the area where the alpha value “1” is stored as the non-aperture area) is treated. The virtual object and the back wall existing in the second space 2 are not drawn. That is, in the image displayed on the upper LCD 22, a real world image is drawn in a portion corresponding to the area by a combining process in step 85 described later. However, with regard to the shadow ES (silhouette model) of the enemy object EO described above, the depth determination with the boundary surface 3 is set to be invalid, and the silhouette model has an alpha value of “0.2”. Since the alpha value “1” is larger, the drawing is performed in the area in which the alpha value “1” indicating the non-opening area is stored. Thereby, the shadow ES of the enemy object EO is drawn on the real world image. In addition, the silhouette model of the enemy object EO is sized and included in the entity model, and the entity model and silhouette model of the enemy object EO are set to enable depth determination. If it exists in the first space 1, the silhouette model is not drawn because it is hidden by the entity model.

  In addition, since the shape of the boundary surface 3 in this embodiment is the center part of the spherical surface as shown in FIG. 20A, there is no opening determination data depending on the viewing direction of the virtual world drawing camera C2. In this case, the above processing is performed on the assumption that there is aperture determination data in which an alpha value “0.2” is stored in a pseudo manner. That is, the area where the opening determination data does not exist is handled as an area in which the alpha value “1” indicating the non-opening area is stored.

  Note that the silhouette data Df2 included in the enemy object data Df corresponding to the enemy object EO in the present embodiment is set so that a plurality of planar polygons have respective radial directions when the normal direction is viewed from the enemy object. The texture of a silhouette image of an enemy object viewed from the direction is applied to each planar polygon. Therefore, in the image processing program in the present embodiment, the shadow ES of the enemy object EO in the virtual space image is expressed as an image reflecting the direction of the enemy object in the second space 2.

  Through the above processing, the information processing section 31 renders the enemy object EO, the bullet object BO, and the back wall BW arranged in the virtual space by perspective projection, and generates a virtual world image viewed from the virtual world drawing camera C2. Then, the rendering image data in the virtual space is updated (step 84 in FIG. 32B). Note that the image generated by this processing is an image obtained by removing the real world image from the display image shown in FIG.

  Next, the information processing section 31 generates a display image obtained by synthesizing the real world image and the virtual space image (step 85), and ends the processing by the subroutine.

  For example, the information processing unit 31 combines the rendering image data of the real camera image and the rendering image of the virtual space with priority given to the rendering image of the virtual space, thereby combining the real world image and the virtual space image. Generated images. Then, the information processing section 31 preferentially combines the image data included in the aiming cursor image data with the center of the combined image (the center of the visual field of the virtual world drawing camera C2) to display the display image (FIG. 40). Is generated. FIG. 40 shows an example of a display image generated by the first drawing method or the second drawing method. Note that when the virtual space image is not stored in the virtual space rendering image data, the information processing unit 31 may store the real world image stored in the camera image rendering image data as it is in the display image data D1.

  Thus, the process (subroutine) related to the update of the display image is completed by the first drawing method or the second drawing method.

  Returning to FIG. 29, after the processing related to the updating of the display image in step 57, the information processing section 31 displays the updated display image on the upper LCD 22 (step 58), and proceeds to the next step 59. For example, the CPU 311 of the information processing unit 31 stores the display image data Dl (display image) updated in step 57 in the VRAM 313. Then, the GPU 312 of the information processing unit 31 outputs the display image drawn in the VRAM 313 to the upper LCD 22 so that the display image is displayed on the upper LCD 22.

  Next, the information processing section 31 determines whether or not to end the game (step 59). As a condition for ending the game, for example, the predetermined condition (game clear or game over) is satisfied, or the user performs an operation to end the game. If the information processing section 31 does not end the game, the information processing section 31 returns to step 52 and repeats the processing. On the other hand, the information processing section 31 ends the processing according to the flowchart when the game is ended.

<Operation and Effect of Image Processing According to First Embodiment>
As described above, in the image processing program according to the present embodiment, as exemplified in the processing of FIGS. 15 and 16, the face image acquired by the face image acquisition processing is a game (first game) by the user. Will not be released to the user until successful. In other words, until the game is successful, the user cannot save the acquired face image in the game save data storage area Do. Also, the user cannot copy, change, transfer, etc. the acquired face image. On the contrary, if the game is retried while the game is unsuccessful, the acquired face image is discarded and the process ends. Further, depending on the processing method of the first game, for example, the execution mode or the specification, when the result of the first game is unsuccessful, the face image is immediately discarded and the processing ends. Therefore, until the acquired face image is delivered, that is, until the face image is stored in the save data storage area Do, the user remains attached to the game and eagerly aims for success. That is, according to the image processing program of this embodiment, the user can tackle the game very seriously.

  In addition, as an operation on the GUI of the user at the start of the game, “acquisition of face image by the inner imaging unit 24” is instructed (Yes in step 9 of FIG. 14), and face image acquisition processing 1 (step of FIG. 14) When 10) is executed, the following effects can also be expected. That is, when a face image is acquired by capturing an image using the inner imaging unit 24 at the start of a game (typically before starting the first game), a different face image is obtained for each image capture. For this reason, for example, compared with the case where a specific image stored in another device is selected and the selected image is used in the game, the willingness to succeed in the game is increased. The same effect can also be expected in the face image acquisition process 2 (step 12 in FIG. 14) by the outer imaging unit 23. When a game is started (typically, before starting the first game), even if a face image is acquired by imaging using the outer imaging unit 23, a different face image is obtained for each imaging. It is.

  In addition, according to the image processing program according to the present embodiment, when the first game is successful, the saved data storage area Do is stored in the user's own face image, the person's face image around the user, and the video device. Various face images such as a face image in the obtained image and a face image of a living creature held by the user can be collected. The game apparatus 10 can display the collected face images on a screen as shown in FIGS. 7 and 8, for example. And the game device 10 can express the state in which the face image in the selected state and the related face image show a reaction, for example, on the screen as shown in FIG. The reaction is, for example, squeezing one eye toward the face image in the selected state, sending a line of sight, or turning the face. Therefore, the game apparatus 10 can express a relationship based on human relationships, intimacy, and the like in the real world in the virtual reality world based on the face images collected in the game apparatus 10. As a result, it is possible to cause the user who has collected the face image to feel familiarity with the virtual reality world by the face image, familiarity with the collected face image, emotions similar to those of the real world person, or living things. .

  Further, according to the image processing program of the present embodiment, the enemy object EO can be generated and the game can be executed by texture mapping the collected face image onto the face portion of the enemy object EO. The user can freely determine the casting by attaching the collected face image to the enemy object EO appearing in the game. Accordingly, when the game is executed, the user can be more likely to be enthusiastic about the game due to the effect obtained from the face of the enemy object EO.

  Furthermore, according to the image processing program of the present embodiment, when generating an enemy object EO, the face image that is in the selected state when the face image is in the selected state to be pasted on the enemy object EO. The face image related to the response shows the reaction. Therefore, it is possible to cause the user who decides the casting of the enemy object EO to have a sense of familiarity with the virtual reality world, familiarity with the face images displayed in a list, emotions similar to those of the real world people, and the like.

  In the first embodiment, the enemy object EO is generated by pasting a face image on the enemy object EO. However, such processing is not limited to generation of enemy objects EO, but can be applied to generation of general character objects appearing in the game. For example, the face image acquired by the user may be pasted on an agent that guides the operation of the game apparatus 10 or the progress of the game. In addition, the face image obtained by the user is pasted on a character object that simulates the user himself / herself, a character object that appears in the game in a friendly relationship with the user, a character object that represents the owner of the game device, or the like. You may make it attach.

  In the above description, a human face is assumed as the face image. However, the embodiment of the present invention is not limited to the human face image, and can be applied to an animal face image. For example, in order to acquire face images of various animals such as mammals such as dogs, cats, horses, birds, fish, reptiles, amphibians, insects, etc., the face image acquisition process described in the first embodiment is executed. , Face images may be collected. For example, according to the game apparatus 10, as exemplified by the relationship between people on the screen of FIG. 8, a bird sings or a dog barks and a cat turns its gaze toward a selected person. In this way, we can express the relationship between people and animals in the real world. On the contrary, in the relationship between the pet and the master, when the pet's face image is selected, the face image corresponding to the master and his family smiles, or turns the line of sight toward the pet. 10 can reflect the feelings, consciousness, and relationships of the real world in the virtual world. Then, the game can be executed while making the user aware of the real-world relationship by pasting the collected faces on the enemy object EO and other character objects by the casting determination process.

  The relationship between the pet and the master, the relationship between the master and the family, etc. may be defined by the face image management information Dn1 through a UI (User Interface) so that it can be referred to as the relationship between the face images. It is sufficient to be able to simply define the feelings of likes and dislikes, good and bad feelings such as favorite pets and disliked pets. In addition, for example, when the game result of the master's face image is successful, the setting that the animal's face image saved in the game save data storage area Do is intimately related to the master is set. The face image management information Dn1 may be recorded. According to the game apparatus 10, the user can generate a character object based on the various facial images collected as described above, and can execute a game reflecting the real world consciousness.

  Further, in the image processing program according to the present embodiment, as shown in FIG. 16, the inner imaging unit 24 performs imaging before the two outer imaging units 23 (the outer left imaging unit 23a and the outer right imaging unit 23b). The user is guided to acquire a face image. The inner imaging unit 24 mainly captures a user who operates the game apparatus 10, and thus is difficult to use for imaging people other than the user. Moreover, since the inner side imaging part 24 mainly images the user who operates the game device 10, it is suitable for an owner's imaging. Therefore, according to this process, there is an effect of prohibiting the use of the outer imaging unit 23 in a state where no user or owner of the game apparatus 10 stores a face image in the game apparatus 10. Therefore, for example, it is possible to increase the possibility of prohibiting a third party from taking an image using the game device 10 whose owner is not specified or the game device 10 whose user is not specified.

  In the image processing program according to the present embodiment, as illustrated in FIGS. 19A to 19C, the information processing unit 31 prompts the user to capture a face image corresponding to an unacquired attribute. Induce. By such a process, it is possible to support the face image collection process of a user who desires to acquire a face image with no attribute bias as much as possible.

  Further, in the image processing program according to the present embodiment, a real world image obtained from a real camera and a virtual space image including an object existing behind the real world image are combined and displayed.

  Therefore, the image processing program according to the present embodiment can generate an image that can attract the user's interest by performing drawing representing unreal on a background using a real-world image.

  In addition, an object (for example, an enemy object EO existing in the second space 2) that exists behind the real world image in the synthesized and displayed image has an opening in the real world image (boundary surface 3). A real image is displayed in the area. In addition, a shadow image of the object is displayed in a region where no opening exists in the real world image (see FIG. 24). Furthermore, the real image and the shadow image are images corresponding to the arrangement direction of the object in the virtual space or the direction according to the movement direction.

  Therefore, the image processing program according to the present embodiment can generate an image that allows the user to recognize trends such as the number of objects and the moving direction behind the real world image.

  In the image processing program according to the present embodiment, an image of an unreal space such as a space image can be used as the image data of the back wall BW. The image of the unreal space can be viewed through the opening of the real world image. The position of the opening in the virtual space is specified. The direction of the real camera is associated with the direction of the virtual camera.

  Therefore, the image processing program according to the present embodiment can provide an opening at a position corresponding to the orientation of the real camera, and can express the opening at the same position in the real world image. In other words, the image processing program according to the present embodiment expresses an opening at the same position in the real space even if the orientation of the real camera is changed, so that the image to be recognized by the user as if the real space and the unreal space were connected. Can be generated.

  The real world image in which the aperture is expressed is generated by multiplying the real world image obtained from the real camera and the alpha value.

  Therefore, the image processing program according to the present embodiment can express and generate the opening by a simple method.

  Further, the opening of the real world image generated by the enemy object EO passing through the boundary surface 3 is generated by multiplying the opening shape data Df3 included in the enemy object data Df by a predetermined position of the opening determination data. The

  Therefore, the image processing program according to the present embodiment can set an opening corresponding to the shape of the collided character by a simple method.

  Further, in the image processing program according to the present embodiment, drawing of a shadow image can be realized by comparing alpha values. Further, the shadow image drawing can be switched on or off by changing the alpha value set in the silhouette data Df2.

  Therefore, the image processing program according to the present embodiment can leave the drawing of shadow images to the GPU, and can switch between displaying and hiding shadows with a simple operation.

(Second Embodiment)
Hereinafter, an image processing apparatus that executes an image processing program according to the second embodiment of the present invention will be described with reference to FIGS. 41 and 42. In the first embodiment, in the face image acquisition process 1 (step 10 in FIG. 14) and the face image acquisition process 2 (step 12 in FIG. 14), the first game is executed, and the user succeeds in the first game. In such a case, the game apparatus 10 allows the images acquired in the face image acquisition process 1 and the face image acquisition process 2 to be stored in the save data storage area Do. And if a user succeeds in a 1st game, he can add the face image acquired by the same process to the save data storage area Do sequentially. Then, the game apparatus 10 creates a character object such as an enemy object EO from the face images collected in the save data storage area Do, for example, according to a user operation or automatically. Then, the game device 10 gives the user a character object created based on the face images collected by the user in the first game (step 106 in FIG. 15, step 129 in FIG. 16) or the second object. It is made to appear in a game (step 18 in FIG. 14) and the like, and a virtual world that reflects human relationships in the real world is provided to the user. Therefore, the user can execute and enjoy a game such as that shown in FIGS. 20A to 26 in a virtual world that reflects such a real world. In the present embodiment, an example of another game process in a virtual world reflecting such a real world will be described. Similarly to the first embodiment, the game of the present embodiment is also provided to the user when the information processing unit 31 of the game apparatus 10 executes the image processing program developed in the main memory 32. Further, the game of the present embodiment is the same as the game of the first embodiment, for example, the first game for the face image acquisition process 1 in step 10 and the face image acquisition process 2 in step 12 shown in FIG. (Step 106 in FIG. 15, step 129 in FIG. 16) may be executed. The game of this embodiment is executed on the assumption that face images are collected and stored in the game save data storage area Do, for example, as in the second game (step 18 shown in FIG. 14). You may be made to do.

  FIG. 41 is an example of a screen displayed on the upper LCD 22 of the game apparatus 10 according to the present embodiment. The procedure for creating this screen is the same as that in the first embodiment. That is, for example, it is assumed that the user holds the lower housing 11 in both hands as shown in FIG. 4 with the lower housing 11 and the upper housing 21 opened in the game apparatus 10. At this time, the user can see the display screen of the upper LCD 22. And in this state, the outer side imaging part 23 can image the space ahead of a user's eyes | visual_axis. During execution of the game of the present embodiment, the game apparatus 10 projects the image captured by the outer imaging unit 23 on the background of the screen. More specifically, the information processing section 31 texture maps the image captured by the outer imaging section 23 on the background portion of the game screen in units of frames. When the user changes the direction while holding the game apparatus 10 in his / her hand, an image acquired through the outer imaging unit 23 from the line-of-sight direction of the outer imaging unit 23 after the change is displayed on the background of the game. That is, an image acquired from the direction in which the user faces the outer imaging unit 23 of the game apparatus 10 is embedded in the background of the screen in FIG.

  Furthermore, on this screen, for example, the enemy object EO1 created according to the procedure described in the screen example of FIG. 10 and the cast determination process of FIG. 18 is displayed against the background. On the face portion of the enemy object EO1, the face image selected in the casting determination process of FIG. 18 is texture-mapped and displayed. The face portion of the enemy object EO1 does not necessarily have to be formed by texture mapping. The enemy object EO1 is simply formed by combining the peripheral portion H13 of the head shape of the enemy object EO and the face image shown in FIG. EO1 may be displayed. Therefore, in the following, it is described that a face image is pasted on the face portion of the enemy object.

  Further, in FIG. 41, enemy objects EO2-EO7 having a smaller shape than the enemy object EO1 are displayed around the enemy object EO1. Thus, in the screen of FIG. 41, a total of seven enemy objects EO of the enemy objects EO1-EO7 are displayed. However, in the process of the game apparatus 10 according to the present embodiment, the number of enemy objects EO is not limited to seven. As already described in the first embodiment, when there is no need to distinguish the enemy objects EO1-EO7, etc., they are called enemy objects EO.

  Further, the same face image as that of the enemy object EO1 is pasted on one or more of the enemy objects EO1 to EO7, for example, the enemy object EO6. On the other hand, a face image different from that of the enemy object EO1 is pasted on the other enemy objects EO2-EO5 and the like.

  Further, an aiming cursor AL for attacking the enemy object EO1 or the like is displayed in the screen of FIG. The positional relationship between the aiming cursor AL, the background and the enemy object EO1, and the relationship of relative movement are the same as those described in the first embodiment (FIGS. 23 to 26).

  That is, the enemy object EO1 freely moves around in the screen of FIG. More specifically, it freely moves around in a virtual space with the image captured by the outer imaging unit 23 as a background. Therefore, it looks to the user who sees the upper LCD 22 that the enemy object EO1 freely moves in the space where the user is located. The enemy objects EO2-EO7 are arranged around the enemy object EO1.

  Therefore, when the user changes the direction of the game apparatus 10 with respect to the enemy objects EO1-EO7 that freely move around in the virtual space, the aiming cursor AL displayed on the screen can be aligned with the enemy objects EO1-EO7. . By pressing the operation button 14B (A button) corresponding to the trigger button in a state where the aiming cursor AL is aligned with the enemy object EO1, it is possible to shoot bullets toward the enemy objects EO1-EO7.

  However, in the game of the present embodiment, an attack on the enemy objects EO1 to EO7 other than the enemy object EO1 that has the same face image as the enemy object EO1 is not an effective attack. For example, when the user attacks an enemy object EO1 and an enemy object having the same face image as the enemy object EO1, the user's score is obtained. Or, it is a penalty for enemy objects. Further, it is assumed that the user score is higher when the enemy objects EO2 to EO7 having a smaller size than the enemy object EO1 are attacked. Alternatively, it is assumed that the loss of the enemy object when the enemy objects EO2 to EO7 are attacked is larger than that when the enemy object EO1 is attacked. However, of the enemy objects EO2 to EO7, an attack on an enemy object having a face image different from that of the enemy object EO1 is an invalid attack. That is, the user is obliged to attack the same enemy object as the enemy object EO1. Hereinafter, an enemy object having a face image different from that of the enemy object EO1 is referred to as a misidentification object. In FIG. 41, enemy objects EO2-EO7 have the same type of head shape. However, of the enemy objects EO2-EO7, any of the enemy objects EO2-EO5, EO7, etc., which are misidentification objects, may have different types of head shapes. As described above, when the individual enemy objects EO1-EO7 are not distinguished, they are simply referred to as enemy objects EO.

  With reference to FIG. 42, an example of an operation by the image processing program executed by the information processing unit 31 of the game apparatus 10 will be described. FIG. 42 is a flowchart illustrating an example of an operation performed by the information processing unit 31. In this process, the information processing section 31 receives, for example, selection of a face image and generates an enemy object (step 30). The process of step 30 is the same as the casting determination process of FIG. 18, for example.

  Next, the information processing section 31 generates a misidentification object (step 31). For example, the misidentification object may be generated by pasting a face image other than the face image of the enemy object EO specified in step 30 on the head-shaped face portion of the enemy object EO. There is no limitation on the designation of the face image of the misidentification object. For example, the face image of the misidentification object may be selected from the face images already acquired by the user as shown in FIGS. Further, for example, a face image of a misidentification object may be stored in advance in the data storage internal memory 35 before the game apparatus 10 is shipped. Further, when the image processing program is installed or upgraded, the face image of the misidentification object may also be stored in the data storage internal memory 35. Further, for example, a face image obtained by deforming the face image of the enemy object EO, for example, a face image in which face parts such as eyes, nose, and mouth are replaced with other face images may be used as the misidentification object.

  Next, the information processing section 31 starts a game with the enemy object EO and the misidentification object (step 32). Then, the information processing section 31 determines whether or not there has been an attack by the user (step 33). The attack by the user is detected by inputting a trigger in a state where the aiming cursor AL shown in FIG. 41 is aligned with the enemy object EO, for example, pressing the operation button 14B. If there is an attack by the user, the information processing section 31 determines whether or not the attack is on an appropriate enemy object EO (step 35). When an attack is made against the appropriate enemy object EO, the information processing section 31 destroys the enemy object EO and adds a user's score (step 36). On the other hand, if it is determined in step 35 that an attack on a misidentification object other than the proper enemy object EO is detected, the information processing section 31 does nothing as an invalid attack. Furthermore, when it is determined in step 33 that there is no attack by the user, the information processing section 31 executes another process (step 34). The other processing is, for example, processing specific to each game. For example, there are processing for multiplying the enemy object EO6 and the misidentification objects EO2-EO5 in FIG. 41, and processing for exchanging the positions of the enemy objects EO6 and misidentification objects EO2-EO7 in FIG.

  Then, the information processing section 31 determines whether or not the game is over (step 37). The end of the game is, for example, when the user's score exceeds a reference value when the user completely destroys all of the multiplying enemy objects EO. Alternatively, the end of the game is, for example, when the enemy object EO has proliferated beyond a predetermined limit, or when the user's goal has exceeded a predetermined limit. If the game is not finished, the information processing section 31 returns the control to step 33.

  As described above, according to the image processing program of the present embodiment, it is possible to create an enemy object EO and execute a game using the collected face images. Therefore, the user can perform a game in the virtual reality world based on a human face image existing in the real world.

  Further, according to the game apparatus 10 of the present embodiment, the game is executed while causing confusion for the user by a combination of the appropriate enemy object EO and the object for misidentification. Therefore, the user is required to accurately grasp the face image of the enemy object EO. As a result, the user is required to have a processing ability and a concentration ability to distinguish the enemy object EO. Therefore, the game apparatus 10 according to the present embodiment generates a sense of tension when the user executes the game, or enables the brain activation associated with the face image recognition.

  In the second embodiment, for example, the enemy object EO is based on the face image stored in the save data storage area Do of the game, as in the cast determination process in step 16 and the game execution process in step 18 of FIG. Created and ran the game. However, as the processing in step 30 in FIG. 42, for example, the processing in steps 100 to 105 in FIG. 15 may be executed. That is, the game according to the present embodiment may be executed in a state obtained by the face image acquisition process and before being stored in the game save data storage area Do. Similarly to Step 107 to Step 110 in FIG. 15, when the game is successful, the acquired face image may be stored in the game save data storage area Do. With such a configuration, as in the first embodiment, the user becomes more enthusiastic about the game in order to save the acquired face image.

(Third embodiment)
Hereinafter, an image processing apparatus that executes an image processing program according to the third embodiment of the present invention will be described with reference to FIGS. 43 and 44. The game of this embodiment is the same as the game of the first embodiment, for example, the first game for the face image acquisition process 1 in step 10 and the face image acquisition process 2 in step 12 shown in FIG. 15 step 106 and step 129) of FIG. The game of this embodiment is executed on the assumption that face images are collected and stored in the game save data storage area Do, for example, as in the second game (step 18 shown in FIG. 14). You may be made to do.

  That is, as in the case described in the second embodiment, the information processing unit 31 of the game apparatus 10 performs the casting determination process (step 16 in FIG. 14) of the first embodiment and the second game. As a processing example of this game (game executed in step 18 of FIG. 14), it is executed. The information processing unit 31 of the game apparatus 10 can also execute the game of the present embodiment as the first game of the first embodiment (the game executed in step 106 in FIG. 15 and step 129 in FIG. 16).

  In the second embodiment, an example of a game process using a face image acquired and an enemy object EO including the acquired face image and a misidentification object has been described. In the second embodiment, the attack on the misidentification object is an invalid attack.

  In this embodiment, instead of the game of the second embodiment, a description will be given of a game in which a face image part of an enemy object EO is replaced with another face image part when an attack on a misidentification object is detected. . For example, the enemy object EO is configured by combining the peripheral portion of the enemy object EO (see H13 in FIG. 9) and the user's face image. Instead of the face image of the user, a face image of a person close to the user may be used. In this case, the face image of the user (or the face image of a close person) may simulate the state of being restricted by the enemy object EO. Then, when the user wins the enemy object EO in the game, it is only necessary to simulate that the user's face image (or a close person's face image) is released from the enemy object EO. On the other hand, if the user continues to fail in the game, the face image of the user (or the face image of a close person) restrained by the enemy object EO will gradually be deformed. Then, the game may be ended when the deformation limit exceeds a certain limit.

  FIG. 43 is an example of a screen displayed on the upper LCD 22 according to the present embodiment. In the example of FIG. 43, an enemy object EO1 (an example of a first character object) is displayed. In addition, an enemy object EO11 (an example of a second character object) having a size smaller than that of the enemy object EO1 and misidentification objects EO12-EO16 (an example of a third character object) are displayed around the enemy object EO1. Yes. The same face image as the enemy object EO1 is pasted on the enemy object EO11. On the other hand, the configuration of the misidentification objects EO12 to EO16 is the same as that in the second embodiment, and a face image different from the enemy object EO1 is pasted. Note that the configuration in FIG. 43 is merely an example, and the number of enemy objects EO11 having dimensions smaller than the enemy object EO1 is not limited to one.

  In the game of the present embodiment, for example, since the aiming cursor AL is easily aligned with the enemy object EO1 having a large size, even if the enemy object EO1 is attacked and a bullet hits, the score obtained by the user or the enemy It is assumed that the damage given to the object EO1 is small. Further, since it is difficult to position the aiming cursor AL on the enemy object EO11 having a small size, when the enemy object EO11 is attacked and a bullet hits, the score that the user can obtain or the damage to the enemy object EO1 is as follows: It is assumed that it is larger than that of the enemy object EO1.

  Furthermore, in this embodiment, when the user attacks the misidentification objects EO12 to EO16, the parts of the face image pasted on the enemy object EO1 are replaced with other face images. For example, in the case of FIG. 43, the eyebrows and the eye portions of the face image pasted on the enemy object EO1 have already been replaced with the eyebrows and eye portions of the face image EO13. In this way, for the user who tries to attack the small enemy object EO11, the misidentification objects EO12-EO16 confuse the user and attempt to lead to deformation of the enemy object EO1. In FIG. 43, the misidentification objects EO12-EO16 have the same type of head shape. However, any of the misidentification objects EO12 to EO16 may have a different type of head shape.

  With reference to FIG. 44, an example of an operation by an image processing program executed by the information processing unit 31 of the game apparatus 10 will be described. FIG. 44 is a flowchart illustrating an example of an operation performed by the information processing unit 31. In this process, the process from step 40 to step 42 is the same as the process from step 30 to step 32 in FIG.

  When the information processing unit 31 detects an attack on the enemy object EO (step 43), the information processing unit 31 reduces the deformation of the face image and brings the face image of the enemy object EO11 closer to the originally pasted face image (step 44). ). In this case, the attack on the enemy object EO11 having a small size shown in FIG. 43 may be given a higher score than the attack on the enemy object EO1 having a large size. Further, the attack on the enemy object EO11 having a small size may be made larger in the degree of reducing the deformation of the face image than the attack on the enemy object EO1 having a large size.

  On the other hand, when the information processing unit 31 detects an attack on the misidentification objects EO12 to EO16 and the like (step 45), the information processing unit 31 further replaces parts of the face image pasted on the enemy object EO1 as compared with the present. That is, the information processing section 31 adds a deformation of the face image (step 44). Further, when the information processing unit 31 detects a state other than the attack on the enemy object EO and the attack on the object for misidentification, the information processing unit 31 performs other processing (step 47). Other processes are the same as those in step 34 of FIG. For example, the information processing unit 31 propagates the enemy object EO.

  Then, the information processing section 31 determines whether or not the game is over (step 48). The determination of the end of the game is, for example, when the deformation of the face image of the enemy object EO exceeds the reference limit. The determination of the end of the game is, for example, when the user destroys the enemy object EO and obtains a predetermined limit score. If the game is not finished, the information processing section 31 returns the control to step 43.

  As described above, according to the game device 10 of the present embodiment, when the user succeeds in attacking the enemy object EO, the deformed face image is restored. Further, when the user attacks the misidentification object, the deformation of the face image further proceeds. Therefore, the user is required to work on the game with concentration, and the tension at the time of game execution increases, and it becomes possible to develop concentration. Furthermore, according to the game device 10 of the present embodiment, the face image of the user and the face image of the close person of the user are deformed, so that the user is enthusiastic about the game in the virtual reality world that is a projection of the real world. The possibility can be increased.

  In the third embodiment, for example, in the same manner as the step 30 in FIG. 42, the casting determination process in step 16 in FIG. 14, and the game execution process in step 18, it is already stored in the game save data storage area Do. The description has been made assuming that the enemy object EO is created based on the face image and the game is executed. However, for example, the processing of Step 100 to Step 105 of FIG. 15 may be executed as the processing of Step 40 of FIG. In other words, the game of the third embodiment may be executed in a state obtained by the face image acquisition process and before being stored in the game save data storage area Do. Similarly to Step 107 to Step 110 in FIG. 15, when the game is successful, the acquired face image may be stored in the game save data storage area Do. With this configuration, as in the first embodiment, the user becomes more enthusiastic about the game.

(Fourth embodiment)
Hereinafter, with reference to FIGS. 45 and 46, an image processing apparatus that executes an image processing program according to the fourth embodiment of the present invention will be described. The game of this embodiment is the same as the game of the first embodiment, for example, the first game for the face image acquisition process 1 in step 10 and the face image acquisition process 2 in step 12 shown in FIG. 15 step 106 and step 129) of FIG. The game of this embodiment is executed on the assumption that face images are collected and stored in the game save data storage area Do, for example, as in the second game (step 18 shown in FIG. 14). You may be made to do.

  That is, as in the case described in the second embodiment, the information processing section 31 of the game apparatus 10 determines that the game of this embodiment is the casting determination process of the first embodiment (step 16 in FIG. 14) or As a processing example of the second game (the game executed in step 18 in FIG. 14), it can be executed. The information processing unit 31 of the game apparatus 10 can also execute the game of the present embodiment as the first game of the first embodiment (the game executed in step 106 in FIG. 15 and step 129 in FIG. 16).

  In the third embodiment, an example of a game process using a face image acquired and an enemy object EO and a misidentification object using the acquired face image has been described. Furthermore, in the third embodiment, when an attack on the misidentification object is detected, the face image part of the enemy object EO is replaced with another face image part.

  In this embodiment, instead of the game of the second embodiment or the third embodiment, the facial parts included in the enemy object EO are replaced with other facial image parts at the start of the game. The process of returning the facial parts included in the enemy object EO to the original facial image when the user wins the game will be described.

  FIG. 45 is an example of a screen displayed on the upper LCD 22 according to the present embodiment. On the left side of the screen, a list of face images (character column) that can be pasted on the enemy object EO is displayed as characters. However, the face image list may not be provided on the screen of the game apparatus 10. Further, for example, when the information processing unit 31 detects a list display display operation from the user through the GUI, the face image list may be displayed. Further, the display location of the face image list is not limited to the left side of the screen as shown in FIG. In the character column, face images such as the face images PS1 to PS5 are displayed.

  In addition, enemy objects EO20, EO21-EO25 are displayed on the screen of FIG. The enemy objects EO20, EO21-EO25, and the like are, for example, enemy objects EO created by the selection operation as shown in the screen of FIG. 7-9 of the first embodiment or the casting determination process of FIG. However, in this embodiment, the enemy object EO20 is drawn large in the center of the screen, and the enemy objects EO21 to EO25 are drawn around the enemy object EO20. For example, the face image PS1 was originally pasted on the enemy object EO20. For example, the face image PS2 is pasted on the enemy object EO22. Further, the face image PS5 is pasted on the enemy image EO25.

  However, the face parts are exchanged between the enemy object EO20 and the EO21-EO25 before the start of the game. For example, the nose is switched between the enemy object EO20 and the enemy object EO22. Further, for example, the left eyebrow and the left eye are interchanged between the enemy object EO20 and the enemy object EO25. For example, the face parts may be replaced in units of polygons constituting a three-dimensional model to be texture-mapped when a face image is texture-mapped.

  Such face part replacement may be performed by, for example, randomly replacing the number of parts to be replaced and the part to be replaced. Also, for example, the number of parts to be replaced may be determined based on the success or score of the game already played or another game. For example, in a game that has already been performed, if the user's grade and achievement level are good, the number of parts to be replaced is reduced, and if the user's grade and achievement level are poor, the number of parts to be exchanged is increased. . Further, the game may be divided into levels, and the number of parts to be replaced may be changed according to the level of the game. For example, the number of parts to be replaced is reduced at the introductory level, and the number of parts to be replaced is increased at the advanced level.

  Alternatively, the user's face image may be acquired by the inner imaging unit 24, face recognition may be performed, and the number of parts to be replaced may be determined according to the recognized facial expression. For example, it may be determined whether the face image is laughing, surprised, sad, or expressionless, and the number of parts to be replaced may be determined according to the determination result. The facial expression may be determined from the position of the cheek outline, etc., using the eye dimensions, mouth area, mouth shape, eye center, mouth center, nose, etc. as reference points. For example, a facial image of the user's expressionless state is registered in advance, and the eye size, mouth area, mouth shape, cheek from the reference point obtained when the user's face image is newly acquired The facial expression of the user may be estimated from the difference value between the value of the contour position and the like and the value obtained from the face image registered in advance. Note that such a facial expression estimation method is not limited to the above procedure, and various procedures can be used.

  In the present embodiment, in the game apparatus 10, the user attacks the enemy object EO with the face parts replaced as shown in FIG. 45 and executes a game in which the user fights the enemy object EO. The manner of attack is the same as the procedure described in the first to third embodiments. Then, when the user wins the battle with the enemy object EO, the replaced parts may be restored to the original face image.

  With reference to FIG. 46, an example of an operation by the image processing program executed by the information processing unit 31 of the game apparatus 10 will be described. Of these processes, the process of Step 90 is the same as Step 30 of FIG. 42 and Step 40 of FIG. In the present embodiment, the information processing section 31 of the game apparatus 10 next performs face part replacement (step 91). Then, the information processing section 31 executes a game (step 92). Then, the information processing section 31 determines whether or not the game is successful (step 93). If the game is successful, the information processing section 31 restores the face whose parts have been replaced in the process of step 91 to the original imaged face (step 94).

  As described above, according to the image processing program of the present embodiment, for example, a face image of a user is acquired by the inner imaging unit 24, and a facial part is acquired between the acquired face image and another face image. The game starts in the state where is replaced. When the user succeeds in the game, such as when the user wins a battle with the enemy object EO, the face image with the parts replaced is restored to the original face image.

  Therefore, for example, when the face image whose face parts are to be replaced is the user's own face image or the face image of a person who is intimate with the user, the user succeeds in the game, so that the motivation is high. Is given.

  In addition, by replacing the face parts at the start of the game in this embodiment according to the results of other games, the user can be given a handicap or advantage depending on the results of the other games. Also, by replacing the face parts according to the game level, the difficulty level of the game level can be expressed by the degree of face deformation.

  In the fourth embodiment, for example, the game save data storage is performed in the same manner as the step 30 in FIG. 42, the step 40 in FIG. 44, or the casting determination process in step 16 in FIG. 14 and the game execution process in step 18. The description has been made on the assumption that the enemy object EO is created based on the face image stored in the area Do and the game is executed. However, as the processing of step 90 of FIG. 45, for example, the processing of step 100 to step 105 of FIG. 15 may be executed. That is, the game according to the present embodiment may be executed in a state obtained by the face image acquisition process and before being stored in the game save data storage area Do. Similarly to Step 107 to Step 110 in FIG. 15, when the game is successful, the acquired face image may be stored in the game save data storage area Do. With such a configuration, as in the first embodiment, the user becomes more enthusiastic about the game in order to save the acquired face image.

(Fifth embodiment)
Hereinafter, an image processing apparatus that executes an image processing program according to the fifth embodiment of the present invention will be described with reference to FIGS. 47 to 51. In the embodiment described above, the camera image captured by the inner imaging unit 24 or the outer imaging unit 23 is used before starting a game (for example, the first game) for storing the face image in the save data storage area Do. The face image to be acquired is acquired by the acquired face image acquisition process. Then, after the face image is acquired, the game is executed, and when the user succeeds in the game, the face image is allowed to be stored in the save data storage area Do. In the fifth embodiment, a face image to be acquired is acquired from a camera image captured by the inner imaging unit 24 or the outer imaging unit 23 during execution of a game fighting an enemy object EO, and the acquisition of the face image is successful. By satisfying the conditions, it is allowed to store the face image acquired during the game in the save data storage area Do. That is, in the fifth embodiment, a face image is acquired from a camera image captured during execution of a predetermined game, and the face image is saved by satisfying a condition for successful acquisition of the face image in the game. Since the storage in the data storage area Do is allowed, the game is allowed to store the face image in the save data storage area Do as a result. The game of this embodiment is also provided to the user when the information processing unit 31 of the game apparatus 10 executes the image processing program developed in the main memory 32, as in the above embodiment. In the following description, as an example of acquiring the face image during the game, a mode in which the face image is acquired during the execution of the second game (step 18 in FIG. 14) described above is used.

  FIG. 47 is an example of a screen displayed on the upper LCD 22 of the game apparatus 10 according to the present embodiment. The procedure for creating this screen is the same as that in the first embodiment. That is, for example, it is assumed that the user holds the lower housing 11 in both hands as shown in FIG. 4 with the lower housing 11 and the upper housing 21 opened in the game apparatus 10. At this time, the user can see the display screen of the upper LCD 22. And in this state, the outer side imaging part 23 can image the space ahead of a user's eyes | visual_axis. During execution of the game of this embodiment, the game apparatus 10 projects the camera image CI captured by the outer imaging unit 23 on the background of the screen. More specifically, the information processing section 31 texture maps the image captured by the outer imaging section 23 on the background portion of the game screen in units of frames. That is, on the upper LCD 22, a real-time real world image (moving image) captured by a real camera built in the game apparatus 10 is displayed in the background portion. When the user changes the direction while holding the game apparatus 10 in his / her hand, an image acquired through the outer imaging unit 23 from the line-of-sight direction of the outer imaging unit 23 after the change is displayed on the background of the game. That is, an image acquired from the direction in which the user faces the outer imaging unit 23 of the game apparatus 10 is embedded in the background of the screen in FIG. In the example of the screen shown in FIG. 47, a person facing the front to the outer imaging unit 23 is included as the subject of the camera image CI displayed as the background of the screen.

  Furthermore, on this screen, for example, the enemy object EO and the aiming cursor AL created according to the procedure described in the above embodiment are displayed against the background. On the face portion of the enemy object EO, the face image selected in the casting determination process or the like is displayed with texture mapping. Then, the user of the game apparatus 10 shoots a bullet at the enemy object EO by pressing the operation button 14B (A button) corresponding to the trigger button while the aiming cursor AL is positioned on the enemy object EO. Can do.

  Even during the game, the game apparatus 10 sequentially performs a predetermined face recognition process on the camera image CI captured by the real camera (for example, the outer imaging unit 23), and the person in the subject in the camera image CI Judging the presence or absence of a face. Then, in the face recognition process, the game apparatus 10 can identify that a human face exists in the camera image CI, and if the condition for causing the acquisition target object AO to appear is satisfied, the game apparatus 10 identifies the face from the camera image CI. The acquisition target object AO appears from the part that has been processed.

  As shown in FIG. 48, in the acquisition target object AO, the face image extracted from the camera image CI is textured on a predetermined portion of a predetermined polygon (for example, a face portion in a three-dimensional model imitating a human head shape). It is mapped and displayed. As an example, the acquisition target object AO is displayed by pasting a partial image identified as a face from the camera image CI as a texture on the surface of a head-shaped three-dimensional model formed by combining a plurality of polygons. Note that the game in which the acquisition target object AO appears is not limited to a face image whose face is recognized and texture-mapped to a three-dimensional model. For example, the acquisition target object AO may be displayed as a plate-like object in which the image identified from the face cut out from the camera image CI is pasted on the main surface, and is simply held in a two-dimensional pixel array. It may be displayed as a processed image.

  For example, the acquisition target object AO is arranged in the above-described virtual space in the same manner as the enemy object EO, and an image (virtual world image) of the virtual space obtained by viewing the acquisition target object AO and / or the enemy object EO from the virtual camera is displayed on the camera. By synthesizing with the real world image obtained from the image CI, the acquisition target object AO and / or the enemy object EO is displayed on the upper LCD 22 so as to be arranged in the real space. Further, in response to an attack operation using the game apparatus 10 (for example, pressing of the button 14B (A button)), the bullet object BO is fired in the direction of the aiming cursor AL, and the acquisition target object AO is also a user's attack target. . When the user wins the battle with the acquisition target object AO, the face image pasted on the acquisition target object AO can be stored in the save data storage area Do.

  In addition to winning the battle with the acquisition target object AO, the game that attacks the enemy object EO, that is, the game already executed when the face image pasted on the acquisition target object AO is recognized is cleared. This may be added to the conditions for storing the face image of the acquisition target object AO in the save data storage area Do. For example, as a condition for clearing a game game that attacks the enemy object EO, it is conceivable to defeat a predetermined number or more of the enemy objects EO. In this case, when executing a game that attacks the enemy object EO, the game specification is such that if the acquisition target object AO appears and is defeated in the middle, a face image of the acquisition target object AO can be additionally acquired.

  The face image used for the acquisition target object AO may be a single face image (still image) whose face is recognized from the camera image CI, or the camera image CI that has been repeatedly picked up is repeatedly recognized as a face. An identified face image (moving image) may be used. For example, in the latter case, when the facial expression or the like of a person repeatedly captured as the camera image CI changes, the change is reflected in the texture of the acquisition target object AO. That is, the facial expression of the face image pasted on the acquisition target object AO can reflect the facial expression of the person captured by the real camera of the game apparatus 10 in real time.

  Further, the acquisition target object AO appearing from the part whose face is recognized from the camera image CI may be arranged so as to always overlap with the part when it is combined with the camera image CI and displayed. For example, since the imaging range captured by the game apparatus 10 is changed by changing the direction and position of the game apparatus 10 in real space (that is, the direction and position of the outer imaging unit 23), the camera displayed on the upper LCD 22 The image CI also changes. In this case, the game apparatus 10 changes the position and direction of the virtual camera in the virtual space according to the movement of the game apparatus 10 in the real space. As a result, the acquisition target object AO displayed so as to be arranged in the real space seems to be arranged at the same position in the real space even if the direction and the position of the game apparatus 10 in the real space change. Is displayed. The upper LCD 22 displays a real-time real world image captured by a real camera built in the game apparatus 10, and the subject may move in real space. In this case, the game apparatus 10 sequentially performs face recognition processing on the repeatedly captured camera image CI so that when the game apparatus 10 is combined with the camera image CI, the game apparatus 10 is displayed so as to overlap the face recognized position. The acquisition target objects AO are sequentially arranged in the virtual space. Therefore, the position and size of the face image that appears as the acquisition target object AO is changed by the camera image CI when the imaging direction or imaging position of the game apparatus 10 changes or the position of the person being imaged changes. The acquisition target object AO can be drawn so as to overlap with the face image by these processes.

  The acquisition target object AO displayed on the upper LCD 22 is displayed by deforming the face image that is actually captured and displayed on the camera image CI, such as enlargement, reduction, deformation, or the like. You may change the display direction of the pasted model. Since the difference between the actually captured face image and the acquisition target object AO is generated by these image processes, the user of the game apparatus 10 can easily determine that the acquisition target object AO has appeared from the camera image CI. It becomes possible to do.

  Next, a specific processing operation by executing the image processing program according to the fifth embodiment will be described with reference to FIGS. 49 to 51. FIG. 49 is a subroutine showing an example of detailed operation of the in-game face image acquisition process performed by executing the image processing program. FIG. 50 is a subroutine showing an example of detailed operation of the non-appearance process performed in step 202 of FIG. FIG. 51 is a subroutine showing an example of detailed operation of the appearance processing performed in step 208 of FIG.

  Note that a program for executing these processes is included in a memory (for example, the data storage internal memory 35), the external memory 45, or the data storage external memory 46 built in the game apparatus 10, and the game apparatus 10 is read from the external memory 45 or the data storage external memory 46 to the main memory 32 from the built-in memory or via the external memory I / F 33 or the data storage external memory I / F 34. And is executed by the CPU 311.

  The processing operation by executing the image processing program according to the fifth embodiment is different from the processing operation by executing the image processing program according to the first embodiment. In the game process described in 29, it is only executed every game process cycle (for example, executed once in the middle of steps 52 to 59). Therefore, in the following description, only processing operations added to the first embodiment will be described, and detailed descriptions of other processing operations will be omitted.

  The various data stored in the main memory 32 in response to the execution of the image processing program according to the fifth embodiment further includes appearance flag data, face recognition data, and acquisition target object data. Except for this, it is the same as the various data stored in response to the execution of the image processing program according to the first embodiment. The appearance flag data is data indicating an appearance flag indicating whether the current appearance state of the enemy object EO is “Not Appearing”, “Now Appearing”, or “Appearing”, and the appearance flag is The initial setting in step 51 (FIG. 29) is set to “not appearing”. The face recognition data is data indicating the latest face image that has been successively recognized from the repeatedly captured camera image CI and the position of the face image in the camera image CI. Further, the acquisition target object data includes three-dimensional model data corresponding to the acquisition target object AO, texture data for mapping to the three-dimensional model, data indicating the arrangement direction and arrangement position of the three-dimensional model, and the like. Contains.

  In FIG. 49, the information processing section 31 determines whether or not the acquisition target object AO has not yet appeared (step 201). For example, the information processing section 31 refers to the appearance flag data and performs the determination in step 201 based on whether or not the appearance flag is set to “non-appearance”. Then, when the acquisition target object AO has not yet appeared, the information processing section 31 advances the processing to the next step 202. On the other hand, if the acquisition target object AO does not appear yet, the information processing section 31 proceeds to the next step 203.

  In step 202, the information processing section 31 performs a non-appearance process and proceeds to the next step 203. Hereinafter, the non-appearance process performed by the information processing unit 31 in step 203 will be described with reference to FIG.

  In FIG. 50, the information processing section 31 performs a predetermined face recognition process on the camera image indicated by the actual camera image data Db, and stores the face recognition result as face recognition data in the main memory 32 (step 211). Proceed to the next step. Here, the face recognition processing may be performed by the information processing section 31 using the camera image sequentially, independently of the processing according to the flowchart shown in FIG. In this case, when the face of a person is identified from the camera image, the information processing unit 31 acquires the face recognition result in step 211 and stores the face recognition result in the main memory 32 as face recognition data.

  Next, the information processing section 31 determines whether or not the acquisition target object AO satisfies a condition for appearing in the virtual space (step 212). For example, the condition for the acquisition target object AO to appear is that the person's face is identified from the camera image in step 211, and the acquisition target object AO is acquired only once from the start of the game to the end of the game. The object AO may appear, the acquisition target object AO may appear every predetermined time, or a new acquisition target object AO appears in response to the disappearance of the acquisition target object AO from the virtual world. Alternatively, the acquisition target object AO may appear at random timing. The information processing unit 31 proceeds to the next step 213 when the condition for the acquisition target object AO to appear is satisfied. On the other hand, if the condition for the acquisition target object AO to appear does not satisfy, the information processing section 31 ends the processing by the subroutine.

  In step 213, the information processing section 31 sets the face image identified by the face recognition process in step 211 as the texture of the acquisition target object AO, and proceeds to the next step. For example, the information processing unit 31 sets an image within the face range indicated by the face recognition result of the face recognition process in step 211 from the camera image indicated by the actual camera image data Db as the texture of the acquisition target object AO. The acquisition target object data is updated using the set texture.

  Next, the information processing section 31 sets the acquisition target object AO using the face image identified in the face recognition process in step 211 (step 214), and proceeds to the next step. As an example, the information processing unit 31 determines a polygon (for example, a plane polygon) corresponding to the state at the start of appearance of the acquisition target object AO according to the range of the face image identified by the face recognition process in step 211. Set the size and shape, paste the texture of the face image set in step 213 on the main surface of the polygon, set the acquisition target object AO corresponding to the state at the start of appearance, and update the acquisition target object data To do.

  Next, the information processing section 31 newly arranges the acquisition target object AO in the virtual space (step 215), and proceeds to the next step. For example, when the camera image is displayed on the upper LCD 22, the information processing section 31 places the acquisition target object AO at a position in the virtual space that is perspectively projected so as to overlap the position of the face image recognized in step 211. Arrange and update the acquisition target object data.

  In the present embodiment, a virtual space in which the acquisition target object AO is newly arranged in addition to the enemy object EO is rendered by perspective projection from a virtual camera to generate an image, and a display image including at least the image is displayed. Here, in order to express the acquisition target object AO so as to appear from the face image of the camera image displayed on the upper LCD 22, the information processing unit 31 performs the processing on the boundary surface 3 on which the texture of the camera image is mapped. The acquisition target object AO is superimposed on the range corresponding to the face image and arranged in the virtual space, and the arranged acquisition target object AO is perspectively projected from the virtual camera. Note that the method of arranging the acquisition target object AO in the virtual space is the same as the arrangement example of the enemy object EO described with reference to FIGS.

  Next, the information processing section 31 sets the appearance flag to “appearing”, updates the appearance flag data (step 216), and ends the processing by the subroutine.

  Returning to FIG. 49, in step 203, the information processing section 31 determines whether or not the acquisition target object AO is appearing. For example, the information processing section 31 refers to the appearance flag data and performs the determination in step 203 depending on whether or not the appearance flag is set to “appearing”. Then, when the acquisition target object AO is appearing, the information processing section 31 proceeds to the next step 204. On the other hand, if the acquisition target object AO is not appearing, the information processing section 31 proceeds to the next step 207.

  In step 204, the information processing section 31 performs an appearance process and proceeds to the next step. For example, in step 204 described above, the information processing section 31 expresses the appearance of the acquisition target object AO by gradually changing the face image included in the camera image into a three-dimensional object. Specifically, the information processing section 31 sets the texture of the acquisition target object AO based on the face recognition result for the camera image as in step 211 described above. As an example, the acquisition target object AO is set by performing a morphing process that changes from a plane polygon to a predetermined three-dimensional polygon (for example, a three-dimensional model imitating a human head formed by combining a plurality of polygons). . Then, the information processing unit 31 positions the acquisition target object AO that has been subjected to the morphing process in a virtual space where the face image recognized in step 204 overlaps with the position of the face image recognized in the same manner as in step 215. And update the acquisition target object data. By performing such a morphing process, when the acquisition target object AO appears from the face image identified from the real world image, the acquisition target object AO is expressed on the face image so as to gradually become a solid from a plane. Is done.

  It should be noted that the polygon that is changed from the planar polygon by the morphing process may be polygons having various shapes. As a first example, the acquisition target object AO is generated by changing from a planar polygon to a three-dimensional polygon having a predetermined character head shape by the morphing process. In this case, the face image identified by the face recognition process with respect to the camera image becomes a texture and is mapped to the face of the head-shaped polygon. As a second example, the acquisition target object AO is generated by changing from a flat polygon to a flat polygon having a predetermined thickness by the morphing process. In this case, the face image identified by the face recognition process with respect to the camera image is textured and mapped to the main surface of the flat polygon. As a third example, the acquisition target object AO is generated by changing from a plane polygon to a three-dimensional polygon having a predetermined weapon shape (for example, a missile-shaped polygon) by the morphing process. In this case, the face image identified by the face recognition process with respect to the camera image is textured and mapped to a part of the weapon-shaped polygon (for example, the tip of the missile with respect to the missile-shaped polygon). Mapped to).

  Next, the information processing section 31 determines whether or not the appearance process for the acquisition target object AO has ended (step 205). For example, when the morphing process for the acquisition target object AO is in the final stage, the information processing unit 31 determines that the in-appearance process has ended. Then, when the in-appearance process for the acquisition target object AO is completed, the information processing section 31 proceeds to the next step 206. On the other hand, if the in-appearance process for the acquisition target object AO has not ended, the information processing section 31 proceeds to the next step 207. For example, the information processing unit 31 changes the polygon corresponding to the acquisition target object AO to a three-dimensional model imitating a human head shape formed by combining a plurality of polygons by repeating the morphing process in step 204 above. In this case, it is determined that the morphing process for the acquisition target object AO is the final stage.

  In step 206, the information processing section 31 sets the appearance flag to “appeared”, updates the appearance flag data, and proceeds to the next step 207.

  In step 207, the information processing section 31 determines whether or not the acquisition target object AO has already appeared. For example, the information processing section 31 refers to the appearance flag data and performs the determination in step 207 depending on whether or not the appearance flag is set to “appeared”. Then, when the acquisition target object AO has already appeared, the information processing section 31 advances the processing to the next step 208. On the other hand, when the acquisition target object AO has not yet appeared, the information processing section 31 ends the processing by the subroutine.

  In step 208, the information processing section 31 performs the appearance processing and ends the processing by the subroutine. Hereinafter, with reference to FIG. 51, the appearance processing that the information processing unit 31 performs in step 208 will be described.

  In FIG. 51, the information processing section 31 performs a predetermined face recognition process on the camera image indicated by the actual camera image data Db and stores the face recognition result as face recognition data in the main memory 32 (step 221). Proceed to the next step. Here, the face recognition process may also be performed by the information processing section 31 using the camera image sequentially independently of the process according to the flowchart shown in FIG. In this case, when the face of a person is identified from the camera image, the information processing section 31 acquires the face recognition result in step 221 and stores the face recognition result in the main memory 32 as face recognition data.

  Next, the information processing section 31 sets the face image (image in the face area of the camera image) identified by the face recognition process in step 221 as the texture of the acquisition target object AO (step 222). Proceed to the next step. For example, the information processing section 31 sets an image within the face range indicated by the face recognition result of the face recognition process in step 221 from the camera image indicated by the real camera image data Db as the texture of the acquisition target object AO. The acquisition target object data is updated using the set texture.

  Next, the information processing section 31 sets an acquisition target object AO corresponding to the face image identified in the face recognition process in step 221 (step 223), and proceeds to the next step. For example, the information processing unit 31 pastes the texture of the face image set in step 222 on the face portion of the three-dimensional model imitating the head shape of a human body formed by combining a plurality of polygons. AO is set and the acquisition target object data is updated. In step 223, the face image identified in the face recognition process in step 221 is subjected to deformation such as enlargement, reduction, deformation, etc., or the texture of the face image is deformed. It doesn't matter.

  Next, the information processing section 31 places the acquisition target object AO set in step 223 in the virtual space (step 224), and advances the processing to the next step. For example, as in step 215, the information processing unit 31 overlaps with the position of the face image recognized in step 221 when the camera image is displayed on the upper LCD 22. The acquisition target object AO is arranged at the position, and the acquisition target object data is updated. In step 223, the acquisition target object AO may be arranged so that the face part to which the texture of the face image is pasted is opposed to the virtual camera, or the acquisition target object AO is set according to the progress of the game. The direction may be changed in any direction.

  Next, the information processing section 31 determines whether or not the acquisition target object AO and the bullet object BO have contacted each other in the virtual space (step 225). For example, the information processing unit 31 uses the position of the acquisition target object AO indicated by the acquisition target object data and the position of the bullet object BO indicated by the bullet object data Dg to obtain a virtual space between the acquisition target object AO and the bullet object BO. It is determined whether or not contact has been made. Then, when the acquisition target object AO and the bullet object BO come into contact with each other, the information processing section 31 proceeds to the next step 226. On the other hand, if the acquisition target object AO and the bullet object BO are not in contact with each other, the information processing section 31 proceeds to the next step 229.

  In step 226, the information processing section 31 performs a point addition process and proceeds to the next step. For example, in the above score process, the information processing section 31 scores a predetermined number of game scores indicated by the score data Dh, and updates the score data Dh using the score after the score. Further, in the above point addition process, the information processing section 31 performs a process of erasing the bullet object BO determined to be touched in the above step 225 from the virtual space (for example, a bullet object related to the bullet object BO that has touched the acquisition target object AO). The data Dg is initialized, and the bullet object BO is not in the virtual space).

  Next, the information processing section 31 determines whether or not acquisition of the face image pasted on the acquisition target object AO that has contacted the bullet object BO has succeeded (step 227). The information processing unit 31 executes the process of step 227 as an example of a unit that determines whether or not the acquisition of the face image is successful. If the information processing unit 31 succeeds in acquiring the face image, the information processing unit 31 proceeds to the next step 228. On the other hand, if acquisition of the face image has not succeeded, the information processing section 31 proceeds to the next step 228.

  Here, success in the acquisition of the face image is, for example, a case where the user wins a battle with the acquisition target object AO. As an example, when a predetermined life value for existing in the virtual space is set in the acquisition target object AO, and the acquisition target object AO comes into contact with the bullet object BO, a predetermined number is subtracted from the life value. Then, when the life value of the acquisition target object AO becomes 0 or less, it is assumed that the acquisition target object AO disappears from the virtual space and the face image pasted on the acquisition target object AO has been successfully acquired.

  If the face image is successfully acquired in step 228, the information processing section 31 stores the data indicating the face image on the main memory 32 recognized in step 221 in the save data storage area Do so far. In addition to the stored face image data, the process proceeds to the next step 229. The CPU 311 of the information processing unit 31 executes the process of step 228 as an example of a storing unit. As described above, the save data storage area Do is a storage area that can be written and read by the information processing section 31, for example, the data storage internal memory 35, the data storage external memory 46, and the like. By storing new face image data in the save data storage area Do, the information processing section 31 adds to the screen of the upper LCD 22, for example, the list of face images described with reference to FIGS. New face image data can be displayed.

  At this time, the information processing unit 31 newly generates and stores the face image management information Dn1 described with reference to FIG. 12 in order to manage the face image stored in the save data storage area Do of the game. That is, the information processing section 31 newly generates face image identification information and sets it in the record of the face image management information Dn1. Further, the address of the face image stored in the save data storage area Do is newly set as the address of the face image data. Furthermore, a face image acquisition source, gender estimation, age estimation, and related face image identification information 1-N are set.

  Furthermore, the information processing section 31 may estimate the attribute of the face image added to the save data storage area Do and update the count result of the face image attribute count table Dn2 described with reference to FIG. In other words, the information processing section 31 may newly estimate the gender, age, and the like of the face image added to the save data storage area Do and reflect it in the count result of the face image attribute count table Dn2.

  Further, the information processing section 31 may allow the user to copy or change data stored in the save data storage area Do, transfer data through the wireless communication module 36, or the like. Then, the information processing unit 31 only needs to execute saving, copying, changing, transferring, and the like of the face image stored in the save data storage area Do according to the operation through the user GUI or the operation through the operation button 14.

  Further, in the processing of step 228, the information processing section 31 may cause the acquisition target object AO, which is a target of facial image acquisition success, to disappear from the virtual space. In this case, the information processing section 31 initializes the acquisition target object data related to the acquisition target object AO that is the target of the successful acquisition of the face image, and makes the acquisition target object AO not exist in the virtual space.

  In step 229, the information processing section 31 determines whether or not the acquisition of the face image pasted on the acquisition target object AO existing in the virtual space has failed. When the information processing unit 31 fails to acquire a face image for any one acquisition target object AO, the information processing unit 31 proceeds to the next step 230. On the other hand, if the acquisition of the face image has not failed for any acquisition target object AO, the information processing section 31 ends the processing by the subroutine.

  Here, the failure to acquire the face image is, for example, a case where the user loses the battle with the acquisition target object AO. As an example, when the acquisition target object AO continues to exist in the virtual space for a predetermined time or longer, it is assumed that acquisition of the face image pasted on the acquisition target object AO has failed.

  If acquisition of the face image fails in step 230, the information processing section 31 discards the data indicating the face image on the main memory 32 recognized in step 221 and ends the processing by the subroutine. In the process of step 230, the information processing section 31 may cause the acquisition target object AO, which is a target of face image acquisition failure, to disappear from the virtual space. In this case, the information processing section 31 initializes the acquisition target object data related to the acquisition target object AO that is the target of face image acquisition failure, and sets the acquisition target object AO in a state that does not exist in the virtual space.

  As described above, according to the processing of FIGS. 49 to 51 according to the fifth embodiment described above, the face image recognized from the camera image captured in the middle of the game attacking the enemy object EO is saved data. It becomes a target to be newly saved in the storage area Do. In order to save the face image acquired during the game in the save data storage area Do, the user is required to obtain a game result that succeeds in acquiring the face image in the game already executed. In the game, a virtual world image indicating an acquisition target object expressed as if a face image in the real world image is extracted from a real world image obtained from a real camera is displayed and combined. A new image in which the target object exists in the real space can be displayed. In the game, for example, in addition to the character object using the face image stored so far in the save data storage area Do, the acquisition target object AO to which the face image acquired during the game is pasted is used. By making it appear, the user who executes the game on the game apparatus 10 collects a new face image and adds it to the save data storage area Do while reflecting the real world in which the game is being executed and the human relationship in the real world. be able to.

  In the above-described fifth embodiment, when the acquisition target object AO that appears in the game is attacked and won, the face image pasted on the acquisition target object AO is stored in the save data storage area Do. However, it may be permitted to store the face image in the save data storage area Do by executing another game that fights the acquisition target object AO. As an example, in a game in which a player competes with an enemy object for a score, an acquisition target object AO to which a face image included in a camera image captured during the game is pasted appears in the game. When the user obtains more points than the acquisition target object AO that appeared during the game, the face image pasted on the acquisition target object AO is allowed to be stored in the save data storage area Do. As another example, in a game in which a user overcomes an obstacle set by an enemy object, an acquisition target object AO in which a face image included in a camera image taken during the game is pasted appears in the game. To do. When the user reaches the goal by overcoming the obstacle set by the acquisition target object AO that appeared during the game, the face image pasted on the acquisition target object AO is stored in the save data storage area Do. It is allowed to be stored.

  In the first to fifth embodiments described above, the face image acquired in the image processing based on the flowchart shown in FIG. 14 (the face image is stored in the save data storage area Do in the first to fourth embodiments). Face image acquired by the face image acquisition process before executing the game for performing, in the fifth embodiment, the face image acquired during the execution of the game for storing the face image in the save data storage area Do), Although an example in which the target is stored in the save data storage area Do has been used, a face image already acquired in an application different from the application of the image processing may be stored in the save data storage area Do. For example, the game apparatus 10 has a built-in image pickup unit (camera), and the image pickup unit picks up an image by a camera image pickup application different from the image processing based on the flowchart shown in FIG. Can be displayed on the screen, and the captured image data can be stored in a recording medium such as the internal data storage memory 35 or the external data storage memory 46. Further, the game apparatus 10 can receive data including a captured image from another apparatus, and the data received by executing the communication application is stored in the internal data storage memory 35, the external data storage memory 46, or the like. It is also possible to store it on a recording medium. As described above, a face image whose face is recognized from an image obtained in advance by executing a camera imaging application or a communication application may be stored in the save data storage area Do.

  When a face image already acquired in an application different from the image processing application is a target to be stored in the save data storage area Do, a captured image saved when the other application is executed At least one face image is extracted by performing face recognition processing, and the extracted face image is a target to be stored. Specifically, when the character is displayed on the upper LCD 22 and / or the lower LCD 12 in order to prompt selection of a character (face image) to appear in the first game or the second game (for example, step) 30, step 40, step 90, step 103, step 126, step 140, step 160, step 162, etc.), and at least one character including a face image acquired in advance by execution of another application is also displayed as a selection target. . In this case, at least one face image is extracted by performing face recognition processing on the pre-stored captured image before displaying the character, and an arbitrary one is selected from the extracted face images. Is displayed in addition to the above character. Then, when the user chooses to make the character including the face image obtained by the extraction appear in the game, and when the user wins the battle with the character, the face image is stored in the save data storage area Do. The

  In this way, the face image already acquired in an application different from the image processing application is also stored in the save data storage area Do, so that the variation of the face image that can be acquired by the user is expanded. Facial images can be easily collected, and face images that cannot be predicted by the user are suddenly added as participation targets of the game, so that it is possible to prevent fatigue in collecting facial images.

  In the above description, an example in which the angular velocity generated in the game apparatus 10 is detected and the movement of the game apparatus 10 in the real space is calculated using the angular velocity is used. However, the movement of the game apparatus 10 using another method is used. May be calculated. As a first example, the motion of the game apparatus 10 may be calculated using the acceleration detected by the acceleration sensor 39 built in the game apparatus 10. As an example, when processing is performed on the computer side on the assumption that the game apparatus 10 equipped with the acceleration sensor 39 is in a static state (that is, processing is performed assuming that the acceleration detected by the acceleration sensor 39 is only gravitational acceleration). ) If the game apparatus 10 is actually static, it is possible to know whether or how much the attitude of the game apparatus 10 is inclined with respect to the direction of gravity based on the detected acceleration. it can. As another example, when it is assumed that the game apparatus 10 equipped with the acceleration sensor 39 is in a dynamic state, the acceleration sensor 39 generates an acceleration corresponding to the movement of the acceleration sensor 39 in addition to the gravitational acceleration component. Therefore, if the gravitational acceleration component is removed by a predetermined process, the movement direction of the game apparatus 10 can be known. Specifically, when the game apparatus 10 including the acceleration sensor 39 is dynamically accelerated and moved by a user's hand, various kinds of the game apparatus 10 are processed by processing an acceleration signal generated by the acceleration sensor 39. Movement and / or position can be calculated. Even when it is assumed that the acceleration sensor 39 is in a dynamic state, if the acceleration according to the movement of the acceleration sensor 39 is removed by a predetermined process, the inclination of the game apparatus 10 with respect to the direction of gravity is reduced. It is possible to know.

  As a second example, the movement of the game apparatus 10 is calculated using the movement amount of the camera image captured in real time by the real camera (the outer imaging section 23 or the inner imaging section 24) built in the game apparatus 10. It doesn't matter. For example, when the imaging direction and imaging position of the real camera change due to the movement of the game apparatus 10, the camera image captured by the real camera also changes. Therefore, if the change in the camera image captured by the real camera incorporated in the game apparatus 10 is used, the angle at which the imaging direction of the actual camera changes, the amount of movement of the imaging position, and the like can be calculated. As an example, by identifying a predetermined object from a camera image captured by a real camera built in the game apparatus 10 and comparing the angle at which the object is captured and the position at which the object is captured in time series The angle at which the imaging direction of the actual camera changes, the amount of movement of the imaging position, and the like can be calculated from these changes. As another example, by comparing the entire camera image captured by the real camera built in the game apparatus 10 in time series, the actual image can be obtained from the imaging direction and the amount of change in the imaging range of the entire image. The angle at which the imaging direction of the camera changes, the amount of movement of the imaging position, and the like can be calculated.

  As a third example, the motion of the game apparatus 10 is calculated by combining at least two of the angular velocity generated in the game apparatus 10 described above, the acceleration generated in the game apparatus 10, and the camera image captured by the game apparatus 10. It doesn't matter. Thus, in a situation where it is difficult to estimate the motion when calculating the motion of the game apparatus 10 from one parameter, the game apparatus compensates for the situation by calculating the motion of the game apparatus 10 by combining other parameters. Ten motions can be calculated. As an example, in calculating the movement of the game apparatus 10 according to the second example, if the captured camera image moves in the horizontal direction in time series, has the shooting angle of the game apparatus 10 been rotated around the vertical direction? It may be difficult to accurately determine whether the game apparatus 10 has moved horizontally. In this case, if the angular velocity generated in the game apparatus 10 is used, it is possible to easily determine whether the game apparatus 10 has been rotated or moved horizontally.

  As a fourth example, the movement of the game apparatus 10 may be calculated by a so-called AR (augmented reality) technique.

  In the above description, the upper LCD 22 has a real-world planar image (opposite of the above-described stereoscopically viewable image) based on the camera image CI acquired from either the outer imaging unit 23 or the inner imaging unit 24. Although an example in which a planar image in terms of meaning) is displayed is mainly used, an image (stereoscopic image) that can be stereoscopically viewed with naked eyes can be displayed on the upper LCD 22. For example, as described above, the game apparatus 10 can display a stereoscopically viewable image (stereoscopic image) using the camera images acquired from the outer left imaging unit 23a and the outer right imaging unit 23b on the upper LCD 22. is there. In this case, the rendering is performed such that the enemy object EO exists in the stereoscopic image displayed on the upper LCD 22 or the acquisition target object AO appears from the stereoscopic image.

  For example, when the enemy object EO and the acquisition target object AO are drawn on the stereoscopic image, the left eye image obtained from the outer left imaging unit 23a and the right eye image obtained from the outer right imaging unit 23b are used as described above. Perform image processing. Specifically, in the image processing described above, one of the left-eye image and the right-eye image is used as the camera image from which the face image is extracted by performing face recognition processing, and is obtained from the one image. The enemy object EO and the acquisition target object AO to which the texture of the face image is mapped are set in the virtual space. Further, the enemy object EO, the acquisition target object AO, the bullet object BO, and the like arranged in the virtual space are perspective-transformed from two virtual cameras (stereo cameras), so that the left-eye virtual world image and the right-eye virtual world respectively. Get an image and. Then, the left-eye real world image and the left-eye virtual world image are combined to generate the left-eye display image, and the right-eye real-world image and the right-eye virtual world image are combined to generate the right-eye display image. Then, the display image for the left eye and the display image for the right eye are output to the upper LCD 22.

  In the above description, the real-time moving image captured by the real camera built in the game apparatus 10 is displayed on the upper LCD 22, and the enemy object EO and the moving image (camera image) captured by the real camera are displayed. Although the acquisition target object AO is displayed so as to appear, various variations are conceivable for the image displayed on the upper LCD 22 in the present invention. As a first example, a moving image recorded in advance, a moving image obtained from a television broadcast or another device, and the like are displayed on the upper LCD 22. In this case, the moving image is displayed on the upper LCD 22, and the enemy object EO and the acquisition target object AO appear in the moving image. As a second example, a still image obtained from a real camera built in the game apparatus 10 or another real camera is displayed on the upper LCD 22. In this case, the still image obtained from the real camera is displayed on the upper LCD 22, and the enemy object EO and the acquisition target object AO appear in the still image. Here, the still image obtained from the real camera may be a real world still image captured in real time by the real camera built in the game apparatus 10, or a real world previously captured by the real camera or another real camera. Or still images obtained from television broadcasting or other devices.

  Further, in the above embodiment, assuming that the upper LCD 22 is a parallax barrier liquid crystal display device, the stereoscopic display and the flat display can be switched by controlling the ON / OFF of the parallax barrier. In another embodiment, for example, a stereoscopic image and a planar image may be displayed using a lenticular liquid crystal display device as the upper LCD 22. Even in the case of the lenticular method, two images picked up by the outer image pickup unit 23 are divided into strips in the vertical direction and alternately arranged, and the images are displayed in three dimensions. Even in the case of the lenticular method, it is possible to display the image on a plane by visually recognizing one image captured by the inner imaging unit 24 to the left and right eyes of the user. That is, even in a lenticular type liquid crystal display device, the same image is divided into strips in the vertical direction, and the divided images are alternately arranged to make the same image visible to the left and right eyes of the user. it can. Thereby, it is possible to display the image imaged by the inner imaging unit 24 as a planar image.

  In the above embodiment, the portable game apparatus 10 has been described. However, the image processing program of the present invention is executed by an information processing apparatus such as a stationary game apparatus or a general personal computer, The present invention may be realized. In another embodiment, the present invention is not limited to a game device, and may be any portable electronic device such as a PDA (Personal Digital Assistant), a mobile phone, a personal computer, a camera, or the like. For example, the mobile phone may include two display units and a real camera on the main surface of one housing.

  In the above description, the example in which the image processing is performed by the game apparatus 10 is used. However, at least a part of the processing steps in the image processing may be performed by another apparatus. For example, when the game apparatus 10 is configured to be communicable with another apparatus (for example, a server or another game apparatus), the game apparatus 10 and the other apparatus cooperate in the processing step in the image processing. It may be executed by. As an example, the game device 10 performs a face image acquisition process and a game process for allowing accumulative storage of the face image, and the face image that has been permitted by the success of the game is another device. You can save it in In this case, the plurality of game devices 10 accumulate face images in other devices in a cumulative manner, thereby further facilitating the collection of face images and browsing the face images stored by other game devices 10. Doing so may create a different way of enjoying it. As another example, another device may perform the processing from step 52 to step 57 in FIG. 29, and the game device 10 may cooperate to execute the processing in step 58 and step 59 in FIG. As described above, by performing at least a part of the processing steps on the image with another apparatus, the same processing as the above-described image processing can be performed. That is, the above-described image processing can be executed by cooperation between one processor or a plurality of processors included in an information processing system including at least one information processing apparatus. In the above embodiment, the processing according to the above-described flowchart is performed by the information processing unit 31 of the game device 10 executing a predetermined program. However, a part of the above processing is performed by a dedicated circuit included in the game device 10. Or all may be done.

  In addition, the shape of the game device 10 described above and the various operation buttons 14, the analog stick 15, the shape, number, and installation position of the touch panel 13 are merely examples, and other shapes, numbers, and It goes without saying that the present invention can be realized even at the installation position. Further, it is needless to say that the present invention can be realized even if the processing order used in the above-described image processing, the set value, the value used for determination, and the like are merely examples and have other orders and values.

  The image processing program (game program) is not only supplied to the game apparatus 10 through an external storage medium such as the external memory 45 or the data storage external memory 46, but also to the game apparatus 10 through a wired or wireless communication line. It may be supplied. The program may be recorded in advance in a non-volatile storage device inside the game apparatus 10. As the information storage medium for storing the program, in addition to the nonvolatile memory, CD-ROM, DVD, or similar optical disk storage medium, flexible disk, hard disk, magneto-optical disk, magnetic tape, etc. But you can. The information storage medium that stores the program may be a volatile memory that temporarily stores the program. Such a storage medium can be referred to as a computer-readable recording medium. For example, various functions described above can be provided by causing a computer or the like to read and execute the program of the recording medium.

Although the present invention has been described in detail above, the above description is merely illustrative of the present invention in all respects and is not intended to limit the scope thereof. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. It is understood that the scope of the present invention should be construed only by the claims. Moreover, it is understood that those skilled in the art can implement an equivalent range from the description of the specific embodiments of the present invention based on the description of the present invention and the common general technical knowledge. In addition, it is to be understood that the terms used in the present specification are used in the meaning normally used in the art unless otherwise specified. Thus, unless defined otherwise, all technical and technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. In case of conflict, the present specification, including definitions, will control.
(Appendix)
The said embodiment can be illustrated with the following aspects (it calls an appendix). The components included in each of the following supplementary notes can be combined with the components included in the other supplementary notes.

(Appendix 1)
A game program executed on a computer that displays an image on a display device,
An image acquisition step of acquiring a face image;
Creating a first character object based on the acquired face image;
A game processing step of displaying a second character object different from the first character object together with the first character object and executing a game;
The game processing step includes
Contributing to success in the game by attacking the first character object by a player's operation;
And a step of invalidating an attack on the second character object by the operation of the player.

(Appendix 2)
A game program executed on a computer that displays an image on a display device,
An image acquisition step of acquiring a face image;
Creating a first character object including the acquired one face image;
Along with the first character object, the second character object including the one face image having a smaller size than the first character object and the one face image having a smaller size than the first character object. A game processing step of displaying a third character object including a face image other than and executing the game,
The game processing step includes
Advancing deformation of the face image included in the first character object by an attack on the third character object by a player's operation;
A step of reversing the deformation so that a face image included in the first character object approaches the acquired original face image by an attack on the second character object by a player's operation. program.

(Appendix 3)
A game program executed on a computer that displays an image on a display device,
An image acquisition step of acquiring a face image;
Creating a character object including a face image obtained by transforming the acquired face image;
A game processing step for accepting a player's operation and advancing a game related to the face image;
Determining success or failure in the game by an operation of the player;
And a step of returning the deformed face image to the acquired original face image when the game result is successful.

(Appendix 4)
An image processing apparatus to which a display device can be connected,
Image acquisition means for acquiring a face image;
Means for creating a first character object based on the acquired face image;
Game processing means for displaying a second character object different from the first character object together with the first character object on the display device and executing a game;
The game processing means includes
Means for contributing to success in the game by attacking the first character object by a player's operation;
An image processing apparatus including: means for invalidating an attack on the second character object by a player's operation.

(Appendix 5)
An image processing apparatus to which a display device can be connected,
Image acquisition means for acquiring a face image;
Means for creating a first character object including the acquired one face image;
Along with the first character object, the second character object including the one face image having a smaller size than the first character object and the one face image having a smaller size than the first character object. Game processing means for displaying a third character object including a face image other than the above on the display device and executing a game,
The game processing means includes
Means for advancing deformation of the face image included in the first character object by an attack on the third character object by a player's operation;
Means for retracting the deformation so that a face image included in the first character object approaches the acquired original face image by an attack on the second character object by a player's operation. Processing equipment.

(Appendix 6)
An image processing apparatus to which a display device can be connected,
Image acquisition means for acquiring a face image;
Means for creating a character object including a face image obtained by transforming the acquired face image;
Game processing means for receiving a player's operation, displaying the character object on the display device, and proceeding with a game related to the face image;
Means for determining success or failure in the game by operation of the player;
An image processing apparatus comprising: means for returning the deformed face image to the acquired original face image when the game result is successful.

(Appendix 7)
A display device;
Image acquisition means for acquiring a face image;
Means for creating a first character object based on the acquired face image;
Game processing means for displaying a second character object different from the first character object together with the first character object on the display device and executing a game;
The game processing means includes
Means for contributing to success in the game by attacking the first character object by a player's operation;
An image processing apparatus including: means for invalidating an attack on the second character object by a player's operation.

(Appendix 8)
A display device;
Image acquisition means for acquiring a face image;
Means for creating a first character object including the acquired one face image;
Along with the first character object, the second character object including the one face image having a smaller size than the first character object and the one face image having a smaller size than the first character object. Game processing means for displaying a third character object including a face image other than the above on the display device and executing a game,
The game processing means includes
Means for advancing deformation of the face image included in the first character object by an attack on the third character object by a player's operation;
Means for retracting the deformation so that a face image included in the first character object approaches the acquired original face image by an attack on the second character object by a player's operation. Processing equipment.

(Appendix 9)
A display device;
Image acquisition means for acquiring a face image;
Means for creating a character object including a face image obtained by transforming the acquired face image;
Game processing means for receiving a player's operation, displaying the character object on the display device, and proceeding with a game related to the face image;
Means for determining success or failure in the game by operation of the player;
An image processing apparatus comprising: means for returning the deformed face image to the acquired original face image when the game result is successful.

(Appendix 10)
An imaging device;
A display device for displaying information including an image acquired by the imaging device;
An image processing system comprising an image processing device that cooperates with the imaging device and the display device,
The image processing apparatus includes:
Image acquisition means for acquiring a face image;
Means for creating a first character object based on the acquired face image;
Game processing means for displaying a second character object different from the first character object together with the first character object on the display device and executing a game;
The game processing means includes
Means for contributing to success in the game by attacking the first character object by a player's operation;
Means for invalidating an attack on the second character object by the operation of the player.

(Appendix 11)
An imaging device;
A display device for displaying information including an image acquired by the imaging device;
An image processing system comprising an image processing device that cooperates with the imaging device and the display device,
The image processing apparatus includes:
Image acquisition means for acquiring a face image;
Means for creating a first character object including the acquired one face image;
Along with the first character object, the second character object including the one face image having a smaller size than the first character object and the one face image having a smaller size than the first character object. Game processing means for displaying a third character object including a face image other than the above on the display device and executing a game,
The game processing means includes
Means for advancing deformation of the face image included in the first character object by an attack on the third character object by a player's operation;
Means for retracting the deformation so that a face image included in the first character object approaches the acquired original face image by an attack on the second character object by a player's operation. Processing system.

(Appendix 12)
An imaging device;
A display device for displaying information including an image acquired by the imaging device;
An image processing system comprising an image processing device that cooperates with the imaging device and the display device,
The image processing apparatus includes:
Image acquisition means for acquiring a face image;
Means for creating a character object including a face image obtained by transforming the acquired face image;
Game processing means for receiving a player's operation, displaying the character object on the display device, and proceeding with a game related to the face image;
Means for determining success or failure in the game by operation of the player;
An image processing system comprising: means for returning the deformed face image to the acquired original face image when the game result is successful.

(Appendix 13)
A computer displaying an image on a display device
An image acquisition step of acquiring a face image;
Creating a first character object based on the acquired face image;
A game processing step of displaying a second character object different from the first character object together with the first character object and executing a game; and
The game processing step includes
Contributing to success in the game by attacking the first character object by a player's operation;
Invalidating an attack on the second character object by the operation of the player.

(Appendix 14)
A computer displaying an image on a display device
An image acquisition step of acquiring a face image;
Creating a first character object including the acquired one face image;
Along with the first character object, the second character object including the one face image having a smaller size than the first character object and the one face image having a smaller size than the first character object. A game processing step of displaying a third character object including a face image other than and executing a game,
The game processing step includes
Advancing deformation of the face image included in the first character object by an attack on the third character object by a player's operation;
Reversing the deformation so that a face image included in the first character object approaches the acquired original face image by an attack on the second character object by a player's operation. Processing method.

(Appendix 15)
A computer displaying an image on a display device
An image acquisition step of acquiring a face image;
Creating a character object including a face image obtained by transforming the acquired face image;
A game processing step for accepting a player's operation and advancing a game related to the face image;
Determining success or failure in the game by an operation of the player;
And a step of returning the deformed face image to the acquired original face image when the game result is successful.

  The game program, the image processing device, the image processing system, and the image processing method according to the present invention can generate a new image by combining a real world image and a virtual world image, and display various images. It is useful as a game program, an image processing apparatus, an image processing system, an image processing method, etc.

10 ... Game device 11 ... Lower housing 12 ... Lower LCD
13 ... Touch panel 14 ... Operation button 15 ... Analog stick 16 ... LED
17 ... Insert 18 ... Microphone hole 19 ... Wireless switch 21 ... Upper housing 22 ... Upper LCD
23 ... Outside imaging unit 23a ... Outside left imaging unit 23b ... Outside right imaging unit 24 ... Outside imaging unit 25 ... 3D adjustment switch 26 ... 3D indicator 27 ... Screen cover 28 ... Touch pen 31 ... Information processing unit 311 ... CPU
312 ... GPU
313 ... VRAM
32 ... Main memory 33 ... External memory I / F
34 ... External memory I / F for data storage
35 ... Internal data storage memory 36 ... Wireless communication module 37 ... Local communication module 38 ... RTC
39 ... Acceleration sensor 40 ... Angular velocity sensor 41 ... Power supply circuit 42 ... I / F circuit 43 ... Microphone 44 ... Speaker 45 ... External memory 46 ... External memory for data storage

Claims (22)

A game program executed by a computer of a game device that displays an image on a display device,
In the computer,
An image acquisition step of acquiring a face image and temporarily storing it in the first storage area during a predetermined game or before the game starts;
Creating a first character object that is a character object including a face image stored in the first storage area;
A first game processing step of causing a game related to the first character object in the predetermined game to progress according to a player's operation;
A determination step of determining success in a game related to the first character object;
A game program for executing the step of accumulatively storing the face image stored in the first storage area in a second storage area when success in the game is determined in the determination step.   The game program according to claim 1, wherein in the image acquisition step, the face image is acquired and temporarily stored in the first storage area before the predetermined game is started. In the computer,
A step of creating a second character object including a face image selected automatically or from a face image stored in the second storage area by the player;
The game program according to claim 1 or 2, wherein in the first game processing step, a game related to the second character object is further advanced in accordance with an operation of a player in the predetermined game. In the computer,
Creating a second character object including a face image selected automatically or by a player from the face images stored in the second storage area;
The game program according to claim 1, further causing a second game processing step of causing a game related to the second character object to proceed in accordance with a player's operation. The game device can acquire an image from an imaging device;
The game program according to any one of claims 2 to 4, wherein, in the image acquisition step, the face image is acquired from the imaging device before the predetermined game is started. The game device, as the imaging device, captures an image from a first imaging device that images the front direction of the outer surface of the display surface of the display device, and a second imaging device that images the back direction of the display surface of the display device. Is available,
The image acquisition step includes
Preferentially acquiring a face image captured by the first imaging device over acquiring a face image captured by the second image capturing device; and
6. The game according to claim 5, further comprising: allowing acquisition of a face image by imaging of the second imaging device after the facial image from the first imaging device is stored in the second storage area. program. Identifying the attributes of the face image stored in the second storage area;
7. The computer according to claim 1, further causing the computer to further execute a step of prompting the player to acquire a face image corresponding to an attribute different from the attribute specified from the face image stored in the second storage area. The game program according to item 1. The first game processing step includes a step of advancing a game related to the first character object by attacking the character object according to an operation of the player,
In the first game processing step, an attack on the first character object is an effective attack for making the game related to the first character object successful, and an attack on the second character object is a success of the game The game program according to claim 3, wherein the game program is an invalid attack. Causing the computer to further execute a step of creating a third character object including a face image different from the face image included in the second character object;
The second game processing step includes a step of advancing a game related to the second character object by attacking the character object according to an operation of the player,
In the second game processing step, an attack on the second character object is an effective attack for making the game related to the second character object successful, and an attack on the third character object is a success of the game The game program according to claim 4, wherein the game program is an invalid attack. Creating a third character object including a face image stored in the first storage area and having a size smaller than that of the first character object;
Causing the computer to further execute a step of creating a fourth character object including a face image different from the face image stored in the first storage area and having a size smaller than that of the first character object;
The first game processing step includes:
Advancing a game related to the first character object by attacking the character object in accordance with an operation of the player;
When the fourth character object is attacked, the step of proceeding the deformation of the face image included in the first character object;
Reversing the deformation so that the face image included in the first character object approaches the original face image stored in the first storage area when the third character object is attacked. The game program according to claim 3. Creating a third character object that includes the same face image as the face image included in the second character object and is smaller in size than the second character object;
A step of creating a fourth character object having a face image different from the face image included in the second character object and having a smaller dimension than the second character object;
The second game processing step includes
Advancing a game related to the second character object by attacking the character object in response to an operation of the player;
When the fourth character object is attacked, the step of proceeding the deformation of the face image included in the second character object;
Reversing the deformation so that the face image included in the second character object approaches the original face image stored in the second storage area when the third character object is attacked. The game program according to claim 4. In the step of creating the first character object, a character object including a face image obtained by deforming the face image stored in the first storage area is created as the first character object,
In the first game processing step, when the game related to the first character object is successful, the deformed face image is restored to the original face image stored in the first storage area. The game program according to claim 1, comprising: In the step of creating the second character object, a character object included in a face image obtained by deforming the face image stored in the second storage area is created as the second character object,
In the second game processing step, when the game related to the second character object is successful, the deformed face image is restored to the original face image stored in the second storage area. The game program according to claim 4, including: In the image acquisition step, during the predetermined game, the face image is acquired and temporarily stored in the first storage area,
In the first game processing step, the first character object is caused to appear in the predetermined game in response to the face image being acquired and the first character object being created during the predetermined game. The game program according to claim 1, wherein a game related to the first character object is advanced. A captured image acquisition step of acquiring a captured image captured by a real camera;
A display image generation step of generating a display image in which a virtual character object appearing in the predetermined game is arranged with the captured image acquired in the captured image acquisition step as a background;
Further causing the computer to execute a display control step of displaying the image generated in the display image generation step on the display device,
15. In the image acquisition step, at least one face image is extracted from the captured image displayed on the display device during the predetermined game and temporarily stored in the first storage area. The described game program.   In the display image generation step, the display unit generates the display image by arranging the first character object so that the face image extracted in the image acquisition step is displayed on the display device so as to overlap the position in the captured image. The game program according to claim 15. In the captured image acquisition step, captured images of the real world captured in real time by the real camera are repeatedly acquired,
In the display image generation step, the captured images repeatedly acquired in the captured image acquisition step are sequentially set as a background,
In the image acquisition step, a face image corresponding to the face image that has already been extracted is repeatedly acquired from the captured images that are sequentially set as a background,
In the step of creating the first character object, the first character object is repeatedly created so as to include the face image repeatedly obtained in the image obtaining step,
In the display image generation step, the first character object that has been repeatedly created is displayed on the display device so as to overlap the position in the captured image of the face image that has been repeatedly acquired in the image acquisition step. The game program according to claim 16, wherein the display image is generated by being arranged. The game device can use image data stored in storage means for storing data non-temporarily,
The at least one face image is extracted from the image data stored in the storage unit before the predetermined game is started and temporarily stored in the first storage area in the image acquisition step. The game program described in. An image processing apparatus to which a display device can be connected,
Image acquisition means for acquiring a face image and temporarily storing it in the first storage area during a predetermined game or before the game starts;
Means for creating a character object including a face image stored in the first storage area;
Game processing means for displaying a game related to the character object in the predetermined game on the display device and advancing according to the operation of the player;
Determination means for determining success in a game related to the character object;
An image processing apparatus comprising: a unit that cumulatively saves the face image stored in the first storage area in the second storage area when the determination unit determines at least success in the game. A display device;
Image acquisition means for acquiring a face image and temporarily storing it in the first storage area during a predetermined game or before the game starts;
Means for creating a character object including a face image stored in the first storage area;
Game processing means for displaying a game related to the character object in the predetermined game on the display device and advancing according to the operation of the player;
Determination means for determining success in a game related to the character object;
An image processing apparatus comprising: a unit that cumulatively saves the face image stored in the first storage area in the second storage area when the determination unit determines at least success in the game. An image processing system for displaying information including an image on a display device,
Image acquisition means for acquiring a face image and temporarily storing it in the first storage area during a predetermined game or before the game starts;
Means for creating a character object including a face image stored in the first storage area;
Game processing means for displaying a game related to the character object in the predetermined game on the display device and advancing according to the operation of the player;
Determination means for determining success in a game related to the character object;
An image processing system comprising: means for accumulatively storing the face image stored in the first storage area in the second storage area when the determination means determines at least success in the game. An image processing method executed by a computer displaying an image on a display device, wherein the computer acquires a face image during a predetermined game or before the game starts and temporarily stores it in a first storage area Steps,
Creating a character object including a face image stored in the first storage area;
A game processing step of causing a game related to the character object in the predetermined game to progress in accordance with a player's operation;
A determination step of determining success in a game related to the character object;
An image processing method that executes a step of accumulatively storing the face image stored in the first storage area in a second storage area when it is determined that at least the success in the game is determined in the determination step.

Images