JP5689103B2 - GAME PROGRAM, GAME SYSTEM, GAME DEVICE, AND GAME CONTROL METHOD - Google Patents

Description

  The present invention relates to a game program and a game apparatus capable of progressing a game based on image data obtained by imaging a user.

  2. Description of the Related Art Conventionally, a game program that captures a user's face and advances the game based on image data obtained by capturing the image is known (see, for example, Non-Patent Document 1). In this game program, a user is imaged at a timing when a predetermined time has elapsed from the start of the game, and the captured image of the user is displayed together with the game result at the end of the game.

"Adult DS Face Training" Instruction Manual, Nintendo Co., Ltd., released on August 2, 2007

  In the above-described prior art, the user is imaged at a timing when a predetermined time has elapsed from the start of the game, and the user's facial expression at that timing is only displayed at the end of the game.

  It is an object of the present invention to provide a novel game program and a game apparatus that display a facial expression of a user who is playing a game while progressing a game based on image data obtained by capturing an image of the user.

  According to the first aspect of the present invention, the image pickup means (23: reference numeral corresponding to the embodiment; the same applies hereinafter) and the display means (12, 22) are executed by the computer of the game apparatus (100) that can be used. Provide a game program to be played. Here, the imaging means is adapted to image the user. The game program includes a game progress means (110, 120, 130, 132) that advances a game based on a user image included in image data acquired by an imaging means, and at least a condition in which the user image is predetermined. When the condition is satisfied, the display unit is made to function as a display control unit (136) for causing the display unit to display an image based on image data acquired by the imaging unit.

  According to the first aspect, it is possible to acquire image data that shows the appearance of the user at a certain timing when the user is playing the game provided by the game program. That is, the user who is playing the game can be imaged at a timing when the user image included in the image data obtained by imaging the user satisfies a predetermined condition. Then, by displaying these image data after the game is over, it is possible to enjoy watching an unexpected or interesting expression of the user who is playing the game. The user image means an image including at least an area where the user is captured as a subject in the image data acquired by the imaging unit.

  According to a preferred second aspect, the display control means is the image data acquired by the imaging means when the game by the game progress means is not in progress and at least when the user image satisfies a predetermined condition. An image based on the is displayed on the display means.

  According to the second aspect, in a period when the user is not playing the game, it is possible to enjoy watching the unexpected or interesting expression of the user who is playing the game.

  According to a preferred third aspect, the display control means displays, on the display means, an image based on the image data acquired by the imaging means when the user image satisfies a predetermined condition at least after the game ends. Display.

  According to the third aspect, after the game is over, the user who is playing the game can enjoy watching the unexpected or interesting facial expression.

  According to a preferred fourth aspect, the game progress means determines whether or not the game is successful based on the user image, and the display control means is imaging means when it is determined that at least the game is successful. An image based on the image data acquired by the above is displayed on the display means.

  According to the fourth aspect, it is possible to take an image of the user's facial expression around the timing when the game is successful. Therefore, the user's facial expression when the game is successful can be displayed.

  According to a preferred fifth aspect, the game advancing means determines whether or not the game is successful based on the position of the user image and the position of the object displayed in accordance with the game effect.

  According to the fifth aspect, it is possible to capture the user's facial expression and the like around the timing when the game is successful. Therefore, after the game is over, the user's facial expression when the game is successful can be displayed.

  Further, according to the fifth aspect, when the success condition associated with the position of the object displayed in accordance with the game effect is satisfied, the user's face or hand is positioned according to the success condition. Therefore, even if any user plays the game, it is possible to acquire image data having substantially the same composition. As a result, when an image object or the like prepared in advance is displayed on the image data, the contents intended by the developer can be expressed more reliably.

  According to a more preferable sixth aspect, the game program causes the computer to further function as detection means (126) for detecting a user area indicating a user area in the image data as a user image. The game progress means determines whether or not the game is successful based on the position of the object displayed in accordance with the game effect and the position of the user area detected by the detection means.

  According to the sixth aspect, since the area in which the user is captured can be accurately specified, erroneous detection by a subject other than the user can be prevented.

  According to a more preferred seventh aspect, the game program causes the computer to further function as a user image, as a detection means for detecting a user area indicating a user area in the image data. Whether or not the game is successful is determined based on the position of the displayed object and the position of the user area detected by the detecting means.

  According to a further preferred eighth aspect, the recognition result reflecting means displays the user area in a manner different from the image picked up by the image pickup means.

  According to the eighth aspect, since the user's facial expression acquired by the imaging means is not displayed while the user is playing the game, the user plays the game without being aware that the image is being captured. be able to. As a result, it is possible to acquire image data showing a natural expression of the user who is playing the game.

  According to a more preferable ninth aspect, the detection means includes a skin color area detection means for detecting a skin color area in the image data, and the game progress means includes the position of the object displayed according to the game effect, and the skin color area. Based on the position of the skin color area detected by the detecting means, it is determined whether or not the game is successful.

  According to the ninth aspect, it is possible to more reliably determine that the imaging unit is imaging the user's skin, so that erroneous detection by subjects other than the user can be prevented.

  According to a more preferable tenth aspect, the skin color area detecting means sequentially detects the skin color area with respect to the image data acquired by the imaging means in a predetermined cycle, and the game program causes the computer to perform the skin color while the game is in progress. It further functions as a recognition result reflecting means for displaying the skin color area detected by the area detecting means.

  According to the tenth aspect, the user can play the game while confirming the skin color area detected for the image data obtained by imaging the user.

  According to a more preferred eleventh aspect, the recognition result reflecting means displays the skin color area in a manner different from the image picked up by the image pickup means.

  According to the eleventh aspect, while the user is playing the game, the user's facial expression or the like acquired by the imaging means is not displayed, so the user plays the game without being conscious of being imaged. be able to. As a result, it is possible to acquire image data showing a natural expression of the user who is playing the game.

  According to a preferred twelfth aspect, the game program further causes the computer to function as a face area detecting means (122) for detecting a human face area in the image data as a user image. The game progress means determines whether or not the game is successful based on the position of the object displayed in accordance with the game effect and the position of the human face area detected by the face area detection means.

  According to the twelfth aspect, since the human face area is specified, it can be more reliably determined that the imaging means is imaging the user's face. Thereby, erroneous detection by a subject other than the user can be prevented.

  According to a preferred thirteenth aspect, the game program further causes the computer to function as presence detection means for detecting the presence of the user, and the game progress means is configured to detect when the presence of the user is detected by the presence detection means. Determine if the game was successful.

  According to the thirteenth aspect, when the user does not exist in front of the game device, it is possible to avoid the game from proceeding due to an erroneous determination.

  According to a preferred fourteenth aspect, the display control means outputs the image data each time a plurality of different conditions are satisfied from the start of the game to the end of the game, including a condition for determining that the game is successful. Each of them is stored, and the display control means switches and displays the plurality of image data picked up by the image pickup means in a predetermined order.

  According to the fourteenth aspect, not only the facial expressions of the users in the vicinity where the game is successful, but also a series of facial expressions of the users who are playing the game can be appropriately imaged. For this reason, after the game is over, a series of facial expressions of the user who is playing the game can be displayed on the moving image-like, and the user can enjoy watching the unexpected or interesting facial expressions of the user.

  According to a preferred fifteenth aspect, the display control means causes a predetermined object to be displayed superimposed on the image data picked up by the image pickup means.

  According to the fifteenth aspect, after the game is over, an object having some meaning is displayed superimposed on the image data, so the user's facial expression is used instead of the facial expression of the user who is playing the game. You can enjoy with various expressions.

  According to a more preferable sixteenth aspect, the display control means causes an object corresponding to the game effect when the image data is stored to be superimposed on the image data.

  According to the sixteenth aspect, after the game is over, the user can enjoy an image that reproduces the game effect when the image data is stored.

  Alternatively, according to a more preferred seventeenth aspect, the display control means causes an object independent of the game effect when the image data is stored to be superimposed on the image data.

  According to the seventeenth aspect, it is possible to perform an expression having a completely different meaning using the expression of the user who is playing the game.

  According to a more preferable eighteenth aspect, the game program further comprises a computer as an object display unit that causes the display unit to display an object while the game is in progress and during the display of an image based on the image data captured by the imaging unit. The object display means replaces the object with another object at the position of the object displayed while the game is in progress, and displays it while displaying an image based on the image data acquired by the imaging means.

  According to the eighteenth aspect, an object different from the object displayed during the game is displayed together with the image data acquired during the game, thereby giving the object meant during the game. An image having a different meaning from the existing meaning can be displayed.

  According to a more preferable nineteenth aspect, the game program further includes a computer as an object display unit that causes the display unit to display an object while the game is in progress and during the display of an image based on the image data captured by the imaging unit. The function display means causes the object display means to display an object not related to the object displayed during the game while the image based on the image data picked up by the image pickup means is being displayed.

  According to the nineteenth aspect, an object that is not related to the object displayed during the game is displayed together with the image data acquired during the game, thereby giving the object that was meant during the game. It is possible to display an image having a different meaning that is not related to the existing meaning.

  According to a more preferable twentieth aspect, the game progress means advances the game by determining success of the game based on success conditions associated with each stage in a plurality of predetermined stages, The program causes the computer to further function as an object display unit that causes the display unit to switch objects associated with success conditions corresponding to the respective stages in accordance with the progress of the game.

  According to the twentieth aspect, a plurality of objects having various meanings can be displayed at any time as the game progresses.

  According to a more preferable twenty-first aspect, the game progress means advances the game by determining success of the game based on success conditions associated with each stage in a plurality of predetermined stages, and displays the game. The control means includes a plurality of image data acquired at each of a plurality of stages in progress of the game and picked up by the image pickup means, an object associated with a success condition associated with the stage at which each image data is acquired, and , And the synthesized image is displayed.

  According to the twenty-first aspect, a plurality of pieces of image data representing a user's appearance acquired during a series of game play are added with an object corresponding to the situation in which each image is acquired. Can do.

  According to a more preferred twenty-second aspect, the game progress means advances the game by determining success of the game based on success conditions associated with each stage at a plurality of predetermined stages, and displays the game. The control means continuously displays the plurality of image data acquired at each of the plurality of stages during the game and stored in the storage means.

  According to the twenty-second aspect, a plurality of image data representing the user's appearance obtained during a series of game play can be displayed on the moving image like expression, and the user's unexpected or interesting expression You can enjoy watching.

  According to a twenty-third aspect of the present invention, a game device (100) including an image pickup unit (23) adapted to pick up an image of a user, a display unit (12, 22), and an arithmetic processing unit (31). I will provide a. The arithmetic processing unit converts the image data acquired by the imaging unit when the game progresses based on the user image included in the image data acquired by the imaging unit and at least the user image satisfies a predetermined condition. It is possible to execute display on the display unit of the image based on it.

  According to the twenty-third aspect, it is possible to acquire image data showing the appearance of the user at a certain timing when the user is playing the game provided by the game device. That is, a user who is playing a game can be imaged at a timing when the result of human body recognition satisfies predetermined conditions for image data obtained by imaging the user. Then, by displaying these image data after the game is over, it is possible to enjoy watching an unexpected or interesting expression of the user who is playing the game.

  In the above description, in order to help understanding of the present invention, reference numerals and supplementary explanations for indicating correspondence with the embodiments described later are given, but these do not limit the present invention.

  According to the present invention, the user can enjoy the game while proceeding based on the image data obtained by imaging himself / herself, and can also enjoy watching his / her facial expression while playing the game.

It is an external view (open state) of the game device according to the embodiment of the present invention. It is an external view (closed state) of the game device according to the embodiment of the present invention. It is a block diagram showing an example of the internal configuration of the game device according to the embodiment of the present invention. It is a figure which shows the provision form of the game program to the game device according to embodiment of this invention. It is a schematic diagram which shows the mode of the user who is playing the game device according to embodiment of this invention. It is a figure for demonstrating temporally an example of the processing content performed with the game device according to embodiment of this invention. It is a figure (the 1) which shows an example of the scene of the 1st game according to embodiment of this invention. (No. 2) showing an example of the scene of the first game according to the embodiment of the present invention. FIG. (No. 3) showing an example of the scene of the first game according to the embodiment of the present invention. FIG. It is a figure which shows the example of a display of the image after the 1st game completion | finish according to embodiment of this invention. (No. 1) shows an example of a scene during and after the game of the second game according to the embodiment of the present invention. FIG. (No. 2) shows an example of a scene during and after the end of the second game according to the embodiment of the present invention. FIG. (No. 3) showing an example of a scene during and after the game of the second game according to the embodiment of the present invention. FIG. It is a figure (the 1) which shows an example of the scene of the 3rd game according to embodiment of this invention. It is a figure (the 2) which shows an example of the scene of the 3rd game according to embodiment of this invention. It is a figure (the 3) which shows an example of the scene of the 3rd game according to embodiment of this invention. (No. 4) showing an example of the scene of the third game according to the embodiment of the present invention. FIG. (FIG. 5) showing an example of the scene of the third game according to the embodiment of the present invention. It is a figure (the 1) which shows the example of a display of the image after the 3rd game completion | finish according to embodiment of this invention. It is a figure (the 1) which shows the example of a display of the image after the 3rd game completion | finish according to embodiment of this invention. It is a block diagram which shows the control structure of the game device according to this Embodiment. It is a figure which shows notionally the process in the rendering engine shown in FIG. It is a figure for demonstrating the conditions of the game progress of the 1st game according to embodiment of this invention. It is a flowchart which shows the process sequence of the 1st game according to embodiment of this invention. It is a flowchart which shows the process sequence of the 2nd game according to embodiment of this invention. It is a figure for demonstrating the conditions of the game progress of the 1st mini game contained in the 2nd game according to embodiment of this invention. It is a flowchart which shows the process sequence of the 1st mini game contained in the 2nd game according to embodiment of this invention. It is a figure for demonstrating the conditions of the game progress of the 2nd mini game contained in the 2nd game according to embodiment of this invention. It is a flowchart which shows the process sequence of the 2nd mini game contained in the 2nd game according to embodiment of this invention. It is a figure for demonstrating the conditions of the game progress of the 3rd mini game contained in the 2nd game according to embodiment of this invention. It is a flowchart which shows the process sequence of the 3rd game according to embodiment of this invention.

  Embodiments of the present invention will be described in detail with reference to the drawings. Note that the same or corresponding parts in the drawings are denoted by the same reference numerals and description thereof will not be repeated.

  As a representative example of the game apparatus (computer) according to the present invention, a portable game apparatus 100 will be described below. The game apparatus 100 can use an imaging unit (or an imaging unit) and a display unit (or a display unit). In addition, as an example of the game program according to the present invention, a game program that is executed by the game device 100 that is a computer will be described. The computer according to the present invention is not limited to a game device, and may be realized as a personal computer capable of executing various applications. Furthermore, the game program according to the present invention may be incorporated as a partial function of various applications executed on the personal computer.

<Terminology>
In this specification, “available” means that a device such as an imaging unit (or imaging unit) and a display unit (or display unit) is connected to a computer (typically, the game apparatus 100) in a wired or wireless manner. It means that data communication is possible. At this time, devices such as an imaging unit (or imaging unit) and a display unit (or display unit) may be integrated with the game apparatus 100 or may be separate.

<Appearance>
1 and 2 are external views of game device 100 according to the embodiment of the present invention.

  Referring to FIGS. 1 and 2, game device 100 according to the present embodiment is a foldable portable game device. FIG. 1 shows the game apparatus 100 in an open state (open state), and FIG. 2 shows the game apparatus 100 in a closed state (closed state). The game apparatus 100 is configured in a size that allows the user to hold it with both hands or one hand even in an open state.

  The game apparatus 100 includes a first housing 11 and a second housing 21. The first housing 11 and the second housing 21 are connected so as to be openable and closable (foldable). In the example shown in FIG. 1, the first housing 11 and the second housing 21 are each formed in a rectangular plate shape, and are connected to each other by a hinge at their long side portions.

  Normally, when the user uses game device 100, game device 100 is in an open state. On the other hand, when the user does not use game device 100, game device 100 is placed in a closed state. In the game apparatus 100, the angle formed by the first housing 11 and the second housing 21 can be maintained between the closed position (approximately 0 degrees) and the open position (approximately 180 degrees) as necessary. That is, the first housing 11 can be stationary at an arbitrary angle with respect to the second housing 21. At this time, it is possible to use a frictional force or the like generated at a connection portion between the first housing 11 and the second housing 21. In addition to the friction force or instead of the friction force, a latch mechanism may be employed at the connecting portion between the first housing 11 and the second housing 21.

  The first housing 11 is provided with a first LCD (Liquid Crystal Display) 12 as a display unit (display means). The first LCD 12 has a rectangular shape and is arranged such that the long side direction thereof coincides with the long side direction of the first housing 11. In this embodiment, an LCD is used as the display unit (display unit). However, for example, another appropriate display device such as a display device using EL (Electro Luminescence) is adopted. Also good. Further, the resolution of the display unit (display unit) can be appropriately designed according to the application to be executed.

  The first housing 11 is provided with buttons 14 </ b> A to 14 </ b> K for performing various operations on the game apparatus 100 as an input unit (input unit). Among the buttons 14A to 14K, the direction input button 14A, the operation button 14B, the operation button 14C, the operation button 14D, the operation button 14E, the power button 14F, the start button 14G, and the select button 14H are the first housing 11 and the second housing. 21 is provided on the inner main surface of the first housing 11 which is the inner side when folded.

  That is, in the arrangement example shown in FIG. 1, the direction input button 14 </ b> A and the power button 14 </ b> F are on one side of the left and right sides (left side in FIG. 1) with respect to the first LCD 12 provided near the center of the inner main surface of the first housing 11. Provided on the main surface. The buttons 14 </ b> B to 14 </ b> E, the start button 14 </ b> G, and the select button 14 </ b> H are provided on the inner main surface of the first housing 11 that is the left and right other side (the right side in FIG. 1) with respect to the first LCD 12.

  The L button 14 </ b> I is provided at the left end of the upper surface of the first housing 11, and the R button 14 </ b> J is provided at the right end of the upper surface of the first housing 11. The volume button 14 </ b> K is provided on the left side surface of the first housing 11.

  Direction input button 14A, L button 14I, and R button 14J are used, for example, for a selection operation. The buttons 14B to 14E are used, for example, for a determination operation or a cancel operation. The power button 14F is used to turn on / off the power of the game apparatus 100. The volume button 14K is used to adjust the volume of a speaker mounted on the game apparatus 100.

  The game apparatus 100 further includes a touch panel 13 as an input unit (input unit) different from the buttons 14A to 14K. The touch panel 13 is mounted so as to cover the screen of the first LCD 12, and detects the coordinates when an input operation is performed by the user. That is, the touch panel 13 is disposed in association with the display surface of the first LCD 12.

  As the touch panel 13, for example, a resistive film type touch panel can be adopted, but not limited to the resistive film type, various press-type touch panels can be adopted. The resolution (detection accuracy) of the touch panel 13 is preferably approximately the same as the resolution (display accuracy) of the first LCD 12. However, the resolution of the touch panel 13 and the resolution of the first LCD 12 do not necessarily match.

  An insertion port (shown by a broken line in FIG. 1) of the touch pen 27 is provided on the right side surface of the first housing 11. A touch pen 27 used for performing an input operation on the touch panel 13 can be accommodated in the insertion opening. Note that the input operation on the touch panel 13 is normally performed using the touch pen 27, but it is also possible to perform the input operation on the touch panel 13 with a user's finger or the like instead of the touch pen 27.

An insertion slot (indicated by a two-dot chain line in FIG. 1) for accommodating the memory card 28 is further provided on the right side surface of the first housing 11. A connector (not shown) for electrically connecting the game apparatus 100 and the memory card 28 is provided inside the insertion slot. The memory card 28 is detachably attached to this connector. The memory card 28 is used for reading game programs and image data acquired from other information processing devices and game devices, and storing (saving) image data taken and / or processed by the game device 100. . The memory card 28 is, for example, SD (Secure
Digital) It consists of non-volatile storage media such as cards.

  An insertion slot (indicated by a one-dot chain line in FIG. 1) for storing the memory card 29 is provided on the upper side surface of the first housing 11. A connector (not shown) for electrically connecting the game apparatus 100 and the memory card 29 is provided inside the insertion slot. The memory card 29 is detachably attached to this connector. The memory card 29 stores a game program and the like.

  Three LEDs 15 </ b> A to 15 </ b> C are attached to the left side of the connection portion between the first housing 11 and the second housing 21. As will be described later, game device 100 can perform wireless communication with other devices, and first LED 15A is lit when power to game device 100 is on. The second LED 15B is lit according to the state of the battery of the game apparatus 100 (charging, low battery level, etc.). The third LED 15C lights up according to the state of wireless communication. Therefore, the three LEDs 15A to 15C can notify the user of the power on / off status of the game apparatus 100, the battery status, and the wireless communication status.

  The second housing 21 is provided with a second LCD 22 as a display unit (display means). The second LCD 22 has a rectangular shape and is arranged so that the long side direction thereof coincides with the long side direction of the second housing 21. Similar to the first LCD 12, other appropriate display devices may be employed instead of the LCD. The game apparatus 100 employs a configuration in which a touch panel as an input unit (input unit) is mounted so as to cover the screen of the first LCD 12. However, another touch panel is mounted on the screen of the second LCD 22. Also good.

  The second housing 21 is further provided with two cameras (an inner camera 23 and an outer camera 25) as imaging means (imaging device). As shown in FIG. 1, the inner camera 23 is attached to the inner main surface near the connecting portion of the second housing 21. On the other hand, as shown in FIG. 2, the outer camera 25 is a surface opposite to the inner main surface to which the inner camera 23 is attached, that is, the outer main surface of the second housing 21 (the game apparatus 100 is in a closed state). It corresponds to the outer surface in the case). In FIG. 1, the outer camera 25 is indicated by a broken line.

  With such an arrangement position, the inner camera 23 can image the direction in which the inner main surface of the second housing 21 faces. That is, the inner camera 23 is adapted to take an image of a user who holds and plays the game apparatus 100. On the other hand, the outer camera 25 can capture an image in the direction opposite to the imaging direction of the inner camera 23, that is, the direction in which the outer main surface of the second housing 21 faces.

  A microphone (a microphone 43 shown in FIG. 3) is housed in the connecting portion of the game apparatus 100 as a voice input device. A microphone hole 16 for the microphone 43 to detect sounds around the game device 100 is provided on the inner main surface near the connecting portion of the game device 100. Note that the position where the microphone 43 is accommodated and the position of the microphone hole 16 do not need to be a connecting portion of the game apparatus 100. For example, the microphone 43 is accommodated in the first housing 11 and the microphone hole 16 is the first hole. You may provide in the position corresponding to the accommodation position of the microphone 43 on the housing 11. FIG.

  A fourth LED 26 (FIG. 2) is provided on the outer main surface of the second housing 21 at a position close to the outer camera 25. The fourth LED 26 is turned on according to the imaging state of the outer camera 25. In other words, the fourth LED 26 notifies the person existing in the object scene and the surrounding people that the game apparatus 100 is imaging. More specifically, the fourth LED 26 is lit while the outer camera 25 is imaging. Further, the fourth LED 26 may blink while the moving image is acquired by the outer camera 25 (the captured image data is continuously stored). In order to prevent the light emitted by the lighting from being reflected on the screen to be imaged, until the shooting by the camera is instructed, until the storage of the image data acquired by the camera in the memory or the like by the instruction is completed During this time, the fourth LED 26 may be turned off.

  With respect to the second LCD 22 provided in the vicinity of the center of the inner main surface of the second housing 21, sound release holes 24 are respectively provided on the left and right main surfaces. A speaker (speaker 45 shown in FIG. 3) is housed in the second housing 21 communicating with the sound release hole 24 as an audio output device. That is, the sound release hole 24 guides the sound emitted from the speaker 45 to the outside of the game apparatus 100.

  As described above, in the first housing 11, the input unit (touch panel 13 and buttons 14 </ b> A to 14 </ b> K) for performing operation input to the game apparatus 100 and the first LCD 12 which is a display unit for displaying various images. Are provided. The second housing 21 is provided with an inner camera 23 and an outer camera 25 for acquiring image data, and a second LCD 22 which is a display means for displaying various images.

<Internal configuration of game device>
FIG. 3 is a block diagram showing an example of the internal configuration of game device 100 according to the embodiment of the present invention.

  Referring to FIG. 3, game device 100 includes CPU 31, main memory 32, memory control circuit 33, storage data memory 34, preset data memory 35, and memory card interface (memory card I / F). 36, 37, a wireless communication module 38, a local communication module 39, a real time clock (RTC) 40, a power supply circuit 41, an interface circuit (I / F circuit) 42, and the like. These electronic components are mounted on an electronic circuit board and housed in the first housing 11 (or the second housing 21).

  The CPU 31 is an arithmetic processing unit that executes various programs. The CPU 31 expands and executes a game program stored in any of the memory (typically, the storage data memory 34), the memory card 28, and the memory card 29 inside the game apparatus 100 in the main memory 32. Execution of the game program of the CPU 31 provides various processes according to the present embodiment to be described later. As will be described later, the game program according to the present embodiment is typically supplied to game device 100 through a wired or wireless communication line from a distribution server device connected to the network or the like. The game program supplied to the game apparatus 100 is stored in the storage data memory 34.

  In addition, the CPU 31 has a VRAM (Video Random Access Memory) (not shown) for controlling display on the first LCD 12 and the second LCD 22 exclusively. This VRAM temporarily stores image data for displaying various images described later. The data stored in the main memory 32 is transferred to the VRAM, or the file (data) stored in the storage data memory 34 is directly read and the contents are written. .

  FIG. 4 is a diagram showing a form in which a game program is provided to game device 100 according to the embodiment of the present invention.

  Referring to FIG. 4, a typical system for distributing a game program according to the present embodiment mediates wireless communication between distribution server device SRV connected to network NW such as the Internet and network NW of game device 100. Access point AP. Distribution server SRV holds a plurality of game programs including a game program according to the present embodiment in a downloadable state, and is requested after a predetermined procedure in response to access from game device 100 or the like. Start sending (downloading) the game program.

  The access point AP is wired to the network NW and establishes a wireless connection with the wireless communication module 38 (FIG. 3). Game device 100 accesses distribution server device SRV via this access point AP. Then, as described above, when a game program distributed from distribution server device SRV is received, CPU 31 of game device 100 stores the received game program in storage data memory 34 or the like. The game program once stored in the storage data memory 34 of the game apparatus 100 is prohibited from moving outside the game apparatus 100 in principle. Therefore, since the game program once taken into the game apparatus 100 is executed only in the game apparatus 100, various settings, parameters, and the like are determined according to the use record and usage form of the owner of the game apparatus 100. Can be customized or arranged independently for the game apparatus 100.

  Instead of the network distribution configuration as shown in FIG. 4, the game program may be provided by selling the memory card 28 or the memory card 29 storing the game program. In this case, the storage medium for storing the game program according to the present invention is not limited to the semiconductor storage device such as the memory card 28 or the memory card 29, but may be an optical storage medium such as a CD-ROM or DVD.

  Referring to FIG. 3 again, main memory 32, memory control circuit 33, and preset data memory 35 are connected to CPU 31. In addition, a storage data memory 34 is connected to the memory control circuit 33.

  The main memory 32 is a storage means used as a work area or buffer area for the CPU 31. That is, the main memory 32 temporarily stores data used for various processes, or develops and temporarily stores game programs read from the storage data memory 34, the memory card 28, the memory card 29, and the like. To do. In the present embodiment, for example, PSRAM (Pseudo-SRAM) is used as the main memory 32.

  The storage data memory 34 is a storage unit that stores a game program executed by the CPU 31, image data captured by the inner camera 23 and the outer camera 25, and the like. The storage data memory 34 is composed of a nonvolatile storage medium such as a NAND flash memory, for example. The memory control circuit 33 is a circuit that controls data reading from the storage data memory 34 and data writing to the storage data memory 34 in accordance with instructions from the CPU 31.

The preset data memory 35 is storage means for storing data (preset data) such as various parameters set in advance in the game apparatus 100. The preset data memory 35 is a CPU 31 via an SPI (Serial Peripheral Interface) bus.
A flash memory connected to can be employed.

  Memory card I / Fs 36 and 37 are each connected to CPU 31. The memory card I / F 36 reads and writes data to and from the memory card 28 attached to the connector in accordance with instructions from the CPU 31. The memory card I / F 37 reads and writes data from and to the memory card 29 attached to the connector according to instructions from the CPU 31.

  In the present embodiment, image data acquired by the inner camera 23 and the outer camera 25 and image data acquired from other devices are written to the memory card 28 via the memory card I / F 36, and the memory card 28. Is read from the memory card 28 via the memory card I / F 36 and written as a file in the storage data memory 34. Various programs stored in the memory card 29 are read from the memory card 29 via the memory card I / F 37 and written to the main memory 32.

  The wireless communication module 38 is, for example, IEEE 802.11. It has a function of connecting to a wireless LAN by a method compliant with the b / g standard. The local communication module 39 has a function of performing wireless communication with the same type of game device by a predetermined communication method. The wireless communication module 38 and the local communication module 39 are connected to the CPU 31. The CPU 31 transmits / receives data to / from other devices via a network line such as the Internet using the wireless communication module 38, and transmits / receives data to / from other game devices of the same type using the local communication module 39. You can send and receive.

  Further, an RTC 40 and a power supply circuit 41 are connected to the CPU 31. The RTC 40 counts the time and outputs it to the CPU 31. For example, the CPU 31 can calculate the current time (date) based on the time counted by the RTC 40. The power supply circuit 41 controls power supplied from a power supply (typically a battery, which is stored in the first housing 11) of the game apparatus 100, and supplies power to each component of the game apparatus 100.

  Game device 100 further includes an I / F circuit 42 connected to CPU 31. A microphone 43, amplifier 44, and touch panel 13 are connected to the I / F circuit 42. I / F circuit 42 includes a voice control circuit that controls microphone 43 and amplifier 44 (and speaker 45), and a touch panel control circuit that controls touch panel 13.

  The microphone 43 detects a user's voice or the like uttered toward the game apparatus 100, and outputs a voice signal indicating the detected voice to the I / F circuit 42. The amplifier 44 amplifies the audio signal from the I / F circuit 42 and outputs it from the speaker 45. That is, the audio control circuit included in the I / F circuit 42 performs A / D conversion on the audio signal detected by the microphone 43, outputs the result to the CPU 31, and outputs D to the signal generated by the CPU 31 or the like. / A conversion is performed and the result is output to the amplifier 44. Furthermore, the audio control circuit converts the audio signal into audio data of a predetermined format suitable for storage.

  Further, the touch panel control circuit included in the I / F circuit 42 generates touch position data according to the detection signal from the touch panel 13 and outputs the touch position data to the CPU 31. For example, the touch position data includes a coordinate value (hereinafter also referred to as “input coordinates”) indicating a position where an input is performed on the input surface of the touch panel 13. The touch panel control circuit cyclically executes reading of signals from the touch panel 13 and generation of touch position data at a predetermined cycle. The CPU 31 can detect input coordinates where an input operation is performed on the touch panel 13 by the user by acquiring touch position data via the I / F circuit 42 (touch panel control circuit).

  The button 14 generically indicates the above-described buttons 14 </ b> A to 14 </ b> K and is connected to the CPU 31. Operation data indicating an input state (whether or not the button 14A to 14K is pressed) is output from the button 14 to the CPU 31. The CPU 31 acquires the operation data from the button 14 to execute processing corresponding to the user operation on the button 14.

  The inner camera 23 and the outer camera 25 are each connected to the CPU 31. The inner camera 23 and the outer camera 25 perform imaging in accordance with an instruction from the CPU 31 and output image data acquired by the imaging to the CPU 31. Each of the inner camera 23 and the outer camera 25 includes an image sensor such as a charge coupled device (CCD) or a CMOS image sensor (CIS) and a peripheral circuit for reading image data acquired by the image sensor. For example, the CPU 31 instructs the inner camera 23 or the outer camera 25 to take an image, and outputs image data acquired by the camera that has received the imaging instruction to the CPU 31.

  Further, both the first LCD 12 and the second LCD 22 are connected to the CPU 31. First LCD 12 and second LCD 22 each display an image in accordance with an instruction from CPU 31. As a form, the CPU 31 displays an image acquired by the inner camera 23 or the outer camera 25 on one of the first LCD 12 and the second LCD 22, and accepts an operation from the user on the other of the first LCD 12 and the second LCD 22. Alternatively, a screen (image) for explaining the operation to the user is displayed.

<Overview of application>
An overview of an application provided by executing a game program according to the present embodiment will be described. In game device 100 according to the present embodiment, an application including game processing and display processing after the game ends is provided. As the game processing, the user's face or hand is mainly imaged using an imaging unit (typically, the inner camera 23), and the game progresses based on the user image included in the image data obtained by this imaging. Is done. Here, the user image means an image including at least an area in which the user appears as a subject in the image data acquired by the imaging unit. More specifically, in order to specify a user image, a human body recognition process (typically a skin color area detection process and / or a face recognition process to be described later) is performed on the acquired imaging data. Then, based on the recognition result (user image), it is sequentially determined whether or not a predetermined condition corresponding to the progress of the game is satisfied, and it is determined how to proceed with the game based on the determination result. The Further, as will be described later, the game may be progressed based on the user's voice data collected by the microphone 43 in addition to the user image included in the image data acquired by the imaging unit. In the present specification, the “game” includes a case of a series of games according to a predetermined scenario and a case of a collection of a plurality of mini games.

  In game device 100 according to the present embodiment, during the progress of such a series of games, the image is played by the user in association with or independently of the condition determination result for progressing the game. Appearance is captured as appropriate. An image based on the image data obtained by such imaging is displayed on the display unit (typically, the second LCD 22) when the game is not in progress (typically after a series of games is completed). Is displayed. The user and those around him can enjoy watching the displayed facial expressions of the user during play.

  FIG. 5 is a schematic diagram showing a state of the user who is playing game device 100 according to the embodiment of the present invention. As shown in FIG. 5, the game apparatus 100 is typically placed on a desk or the like, that is, the user himself / herself has its imaging area of the inner camera 23 fixed, and the user himself / herself has its head, trunk, hand, etc. Move according to the game production. Thereby, the recognition result in the game device 100 (that is, the user image included in the acquired image data) changes, and it is sequentially determined whether or not the game progress condition is satisfied based on the changed recognition result. The

  FIG. 6 is a diagram for temporally explaining an example of the processing content executed by game device 100 according to the embodiment of the present invention. Referring to FIG. 6, in game device 100 according to the present embodiment, when game progress is started, acquisition of image data by imaging unit (inside camera 23) and human body recognition processing based on the acquired image data are performed. Execution is repeatedly executed at a predetermined cycle. And a game progresses based on the recognition result acquired (updated) periodically. That is, the game progresses sequentially when the recognition result satisfies the conditions 1, 2,. If all the conditions (or the game success conditions) are not satisfied within a predetermined time, the game may be determined to be “failed”. In the present specification, a state where all the conditions prepared in advance for the game progress are satisfied may be referred to as “success” of the game.

  In parallel with this game progress, the image of the user playing is imaged by the imaging unit. Note that the image data acquired for performing the human body recognition processing is basically discarded without being saved. On the other hand, image data showing the appearance of the user playing is stored in the main memory 32 or the like. Therefore, from the viewpoint of clarifying the difference between the processes, the process of storing (storing) image data showing the appearance of the user playing is also referred to as “capture”.

  Note that, in both the human body recognition process and the capture, the image data acquired by the imaging unit is used in common, but the timing of using the image data in each process may be different. That is, in the human body recognition process, the image data is processed at a constant cycle, whereas in the capture, the image data is stored at an event timing such as establishment of a predetermined condition.

  As shown in FIG. 6, typically, image data is captured at a timing when the recognition result satisfies the conditions 1, 2,. That is, it is determined whether or not the game is successful based on the user image included in the image data acquired by the imaging unit (inner camera 23), and at least when the game is determined to be successful. The image data acquired by the unit is stored. FIG. 6 illustrates the case where the conditions used for the game progress are the same as the conditions for determining the capture execution timing, but it is not always necessary to match the two, depending on the game scenario, etc. Independent conditions may be set.

  Thereafter, when the series of games is completed, the image data 1, 2,... Captured during the game play are displayed (replayed). After the game is over, image data is displayed in various manners corresponding to the content of the game.

  In game device 100 according to the present embodiment, typically, the following three types of games are provided. Hereinafter, an example of the content of these games and the display mode of the image displayed after the game ends will be described. Note that some or all of these games are selectively executed in accordance with user instructions. Further, the display timing may be such that the image data can be replayed at an arbitrary timing designated by the user, even after the game is not over.

<First game>
First, with reference to FIGS. 7 to 10, display modes of the first game and the image after the first game provided in game device 100 according to the present embodiment will be described. During the play of the first game, the user plays each time a plurality of different conditions are satisfied from the start to the end of the game, including a condition for determining that the first game is “successful”. Each figure is captured. Then, after the first game is finished, these captured plural pieces of image data are displayed by switching them in the order of the captured time or in a random order (continuously). Therefore, after the first game is over, the appearance of the user during the play of the first game is displayed in a moving image-like manner. That is, as the image display based on the stored image data, in the first game, a moving image-like image that switches a plurality of pieces of image data in a predetermined time is displayed. Note that a more enjoyable image display can be provided by changing the display order of the image data to a different one from the original imaging order. Further, by making the display speed (switching speed of image data) shorter than the actual imaging interval, it is possible to display an image with more movement.

  7 to 9 are diagrams showing an example of the scene of the first game according to the embodiment of the present invention. FIG. 10 is a diagram showing a display example of an image after the first game according to the embodiment of the present invention.

  In the first game according to the present embodiment, the user places a face on game device 100 (inner camera 23) so that the recognition result for the image data acquired by inner camera 23 exceeds the size of a predetermined determination area. It is a game that gets closer.

  First, when the user instructs the start of the first game, a screen as shown in FIG. 7 is displayed on the first LCD 12 and the second LCD 22. 7 to 9, the entire images (screen shots) respectively displayed on the first LCD 12 and the second LCD 22 are drawn side by side for convenience of explanation. The same applies to the subsequent drawings.

  More specifically, in the scene shown in FIG. 7, a notification image INF is displayed on the first LCD 12 for notifying the user what kind of movement should be performed. That is, in the example shown in FIG. 7, an image indicating a message “Look at me!” That notifies the user that the face is approaching is displayed on the first LCD 12. On the other hand, the second LCD 22 displays an image object on the game effect (in the example shown in FIG. 7, an image of a human mouth) and is obtained by human body recognition processing on the image data acquired by the inner camera 23. A recognition display REC indicating the range of the user's face is reflected. In principle, the recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later.

  FIG. 7 shows a state in which the user is shown only in a relatively lower position in the image data acquired by the inner camera 23. For this reason, the recognition display REC corresponding to the position of the face of the user who has been imaged is reflected only on the lower side of the second LCD 22.

  Thereafter, when the user brings his / her face close to the game apparatus 100 and the range in which the user is captured in the image data acquired by the inner camera 23 is enlarged, the screen is changed to a screen as shown in FIG. In the state shown in FIG. 8, an image object of the human mouth that is larger than that in FIG. 7 is displayed, and a wider range of recognition display REC is reflected.

  Further, when the user brings his / her face closer to the game apparatus 100 and the range in which the user is captured in the image data acquired by the inner camera 23 is further enlarged, the screen is changed to a screen as shown in FIG. FIG. 9 shows a state where the first game is “successful”. When the first game is “successful”, an image object of a human mouth that is larger than that in FIG. 8 is displayed, and an image object in a state where a bird appears from between the teeth is displayed. Is done.

  In the series of games shown in FIGS. 7 to 9 as described above, the user refers to the game effect displayed on the second LCD 22 and the recognition display REC according to the progress of the game while referring to the relative position with the game device 100. Change. During the play of the first game, the appearance of such a user in the game is captured multiple times. Then, after the first game is finished, the plurality of captured image data are sequentially displayed. An example of how this image data is displayed is shown in FIG.

  In FIG. 10A, the face of the user who played the first game is smaller than that in FIG. 10B. This corresponds to the state shown in FIG. 8 described above, and shows a case where the user's face is at a position relatively distant from the game apparatus 100. On the other hand, in FIG. 10 (b), the user's face is shown in a larger range compared to FIG. 10 (a). This corresponds to the state shown in FIG. 9 described above, and shows a case where the user's face is relatively close to the game apparatus 100.

  By continuously switching and displaying such images, it is possible to display the appearance of the user playing the first game on the moving image-like screen. In other words, the state where the user is playing the first game can be replayed.

  Note that FIG. 10 shows an example in which two pieces of image data are typically switched sequentially, but more image data is acquired during the first game, and these are sequentially switched and displayed. Good.

<Second game>
Next, with reference to FIG. 11 to FIG. 13, the display mode of the second game provided by game device 100 according to the present embodiment and the image after the end of the second game will be described. This second game is typically composed of a plurality of mini-games, and during the play of each mini-game, at least one image of the user playing at a timing when a predetermined condition is satisfied is captured. Then, after the end of a series of mini-games, that is, after the end of the second game, for each of these captured image data, an object related to the corresponding mini-game or an object having a different meaning from the corresponding mini-game Are displayed in layers. Therefore, after the end of the second game, the appearance of the user at a certain timing during the play of the second game (typically around the timing when the mini-game is “successful”) is expressed together with the object corresponding to the game effect. Alternatively, it is expressed together with an object independent of (unrelated to) the game effect. In addition, it is preferable that the image data captured in each of the plurality of mini games is switched and displayed at predetermined time intervals, that is, displayed as a so-called slide show.

  In addition, when the start of a 2nd game is instruct | indicated by the user, you may make it determine at random which mini game should be performed among several prepared mini games.

  11 to 13 are diagrams showing an example of a scene during and after the game of the second game according to the embodiment of the present invention.

(1) First Mini Game In the first mini game included in the second game shown in FIG. 11 (a), the “glasses” object OBJ10 is displayed at a predetermined position on the second LCD 22, and the user The relative position of the player with respect to the game apparatus 100 (inner camera 23) is adjusted so that the position of the recognized eye matches the object OBJ10.

  In the scene shown in FIG. 11A, an image object (object OBJ10) on the game effect is displayed on the second LCD 22, and the user's body obtained by the human body recognition process on the image data acquired by the inner camera 23 is displayed. A recognition display REC indicating the face range is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later. In addition, an image object indicating “eye” at the position is displayed on the pupil of the user whose human body has been recognized.

  When the user adjusts the relative position of the user with respect to the game apparatus 100 and it is determined that the position of the user's recognized pupil in the image data acquired by the inner camera 23 matches the arrangement position of the object OBJ10. The first mini game is “success”. As described above, based on the result of the human body recognition, it is determined whether or not the success condition associated with the position of the object displayed in accordance with the game effect is satisfied.

  Referring to FIG. 11 (b), after the second game ends, image data captured during the play of the first mini game shown in FIG. 11 (a) is displayed on the second LCD 22. This displayed image is typically overlaid with the “glasses” object OBJ10 # displayed during the play of the first mini game. In the example shown in FIG. 11B, the same “glasses” object as the “glasses” displayed in the corresponding first mini-game is displayed, but another form of “glasses” is superimposed. You may make it display. Alternatively, an image object that is completely different from “glasses” may be displayed in an overlapping manner.

  The first LCD 12 displays a button image for controlling display (slide show) of image data captured in each of the plurality of mini games. More specifically, the first LCD 12 displays an order notification image 202, a stop button image 204, a return button image 206, a forward button image 208, and a closing button image 210. A touch panel 13 is provided on the first LCD 12, and when the user touches any position on the first LCD 12, coordinates touched by the touch panel 13 are detected. When any one of the above-described button images is displayed at the detected coordinates, processing assigned to the corresponding button image as described below is executed.

  The order notification image 202 notifies which mini-game the image data currently being displayed is captured. The example shown in FIG. 11B shows a case where five mini games have been played first, and also shows that the currently displayed image data has been captured in the first mini game. The stop button image 204 receives an instruction to stop image data display switching (slide show). When the slide show is in a stopped state, a playback button image for receiving a slide show restart instruction is displayed instead of the stop button image 204. The return button image 206 accepts an instruction for returning one of the series of image data to be displayed from the currently displayed image data. The forward button image 208 and an instruction to advance one of the series of image data to be displayed from the currently displayed image data are received. The end button image 210 receives an instruction to end display of image data.

(2) Second Mini Game Next, in the second mini game included in the second game shown in FIG. 12A, the object OBJ11 indicating “mosquito” is moved and displayed on the second LCD 22, and the user is recognized. Strike the hand so that the object OBJ11 is “crushed” by the movement of its own hand.

  In the scene shown in FIG. 12A, an image (object OBJ11) on the game effect is displayed on the second LCD 22 while changing its position in time, and an object OBJ12 that prompts the user to tap the hand is displayed. The Further, the second LCD 22 reflects a recognition display REC indicating the range of the user's hand and face obtained by the human body recognition process on the image data acquired by the inner camera 23. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later. On the other hand, the first LCD 12 displays a notification image INF indicating a message “Take!” Notifying the user of clapping.

  When the user claps his / her hand after adjusting his / her relative position with respect to the game apparatus 100 and the user recognizes that the user has clashed his / her hand at the position of the object OBJ11 by the human body recognition process on the image data acquired by the inner camera 23. The second mini-game is “successful”, and an effect is produced in which “mosquito” falls to the bottom of the screen.

  Referring to FIG. 12B, after the second game is finished, image data captured during the play of the second mini game shown in FIG. 12A is displayed on the second LCD 22. On this displayed image, an object OBJ13 having a different meaning independent of the “mosquito” object OBJ11 displayed during the play of the second mini game is overlaid. In other words, the movement of hitting “mosquito” by the user is replaced with the user's lovely expression and displayed.

  The first LCD 12 displays a button image for controlling display (slide show) of image data captured in each of the plurality of mini games, as in FIG. 11B described above.

  As shown in FIG. 12, during the display of the image based on the image data captured during the play of the second mini game after the end of the second mini game, the object displayed during the progress of the second mini game is displayed. The image is displayed by replacing the object with a different object at the position. In particular, in the example shown in FIG. 12, the object (in the example shown in FIG. 12A, “mosquito” in the example shown in FIG. 12A) that is not associated with the object displayed during the progress of the second mini game (in the example shown in FIG. 12B). "Heart mark" and the like are displayed together with the image data captured during the play of the second mini game.

(3) Third Mini Game Next, in the third mini game included in the second game shown in FIG. 13A, play by two users is possible. That is, in the third mini game shown in FIG. 13A, two “totem pole” objects OBJ14 and OBJ15 are displayed on the second LCD 22, and obtained by the human body recognition process on the image data acquired by the inner camera 23. The recognition display REC indicating the range of each user is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later. Each of these “totem pole” objects OBJ14 and OBJ15 is provided with a circular cutout portion, and each user has a recognized face position that matches the corresponding cutout portion. In this way, the relative position with respect to the game apparatus 100 (inner camera 23) is adjusted.

  Further, the first LCD 12 displays a notification image INF indicating a message “Match together!” For notifying the user that his / her face matches a predetermined position.

  Both users adjust their relative positions with respect to the game apparatus 100 and determine that the positions of the faces recognized by the human body in the image data acquired by the inner camera 23 coincide with the cutout portions of the corresponding objects OBJ14 and OBJ15. Then, the third mini game becomes “success”.

  Referring to FIG. 13B, after the second game is finished, image data captured during the play of the third mini game shown in FIG. 13A is displayed on the second LCD 22. In this displayed image, an object OBJ16 having a different meaning independent of the objects “OBJ14” and “OBJ15” of the “totem pole” displayed during the play of the third mini game is overlaid. In other words, the display in which both users are positioned in the respective cutout portions provided on the “totem pole” is replaced with an expression in which both users are protruding from the large “nose”.

  The first LCD 12 displays a button image for controlling display (slide show) of image data captured in each of the plurality of mini games, as in FIG. 11B described above.

  As described above, the user captured during the play of the second game (included therein) is independent of the object corresponding to the game effect of the corresponding mini game or the game effect of the corresponding mini game. The objects (with different meanings) are expressed in layers. Thereby, it can enjoy with the more pleasant image using the figure which the user played the mini game.

  As shown in FIG. 13, during the display of the image based on the image data captured during the play of the third mini game after the end of the third mini game, the object displayed during the progress of the third mini game is displayed. Is replaced with another object and the image is displayed. In particular, in the example shown in FIG. 13, an object (example shown in FIG. 13B) that is not related to the object displayed in the progress of the third mini game (“Totem Pole” in the example shown in FIG. 13A). Is displayed together with image data captured during the play of the third mini game.

<Third game>
Next, with reference to FIGS. 14 to 19, a third game provided by game device 100 according to the present embodiment and an image display mode after the end of the third game will be described. The third game is composed of a series of a plurality of mini-games, and during the play of each mini-game, at least one image of the user playing at a timing when a predetermined condition is satisfied is captured. Then, after the end of the series of mini-games, that is, after the end of the third game, an object having a specific meaning is superimposed on each of the captured image data, so that it follows a prepared scenario. Manga-like expression is provided. Note that the comic-like image data is sequentially switched and displayed every predetermined time.

  14 to 18 are diagrams showing an example of the scene of the third game according to the embodiment of the present invention. 19 and 20 are diagrams showing a display example of an image after the end of the third game according to the embodiment of the present invention. In the following description, the case where the third game is composed of five mini-games and “5-frame cartoon” is displayed using a total of five image data captured in each mini-game is illustrated.

(1) Fourth Mini Game In the fourth mini game included in the third game shown in FIG. 14, an object OBJ17 representing a human head and back is displayed at a predetermined position on the second LCD 22, and the user Moves the hand so as to “tap” the portion corresponding to the “shoulder” of the object OBJ17 by its recognized movement.

  In the scene shown in FIG. 14, an image (object OBJ17) on the game effect is displayed on the second LCD 22, and the range of the user's hand obtained by the human body recognition process on the image data acquired by the inner camera 23 is displayed. The recognition display REC shown is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later.

  The user moves the hand up and down after adjusting the relative position of the hand with respect to the game apparatus 100, and the user corresponds to the predetermined position (“shoulder”) of the object OBJ 17 by human body recognition processing on the image data acquired by the inner camera 23. If the position is determined to be “tapped” for a predetermined number of times, the fourth mini game is “successful”. When it is determined that “shoulder” is “tapped” while the game is in progress, an object OBJ 18 indicating that is displayed. Further, the second LCD 22 also displays an object OBJ19 indicating the remaining number of times that the user “taps” the shoulder necessary for the game to be “successful”.

(2) Fifth Mini Game In the fifth mini game included in the third game shown in FIG. 15, an object OBJ20 representing a separate scene is displayed on the second LCD 22, and the user moves his / her hand to the left and right. “Bye-by” operation.

  In the scene shown in FIG. 15, an image (object OBJ20) on the game effect is displayed on the second LCD 22, and the range of the user's hand obtained by the human body recognition process on the image data acquired by the inner camera 23 is displayed. The recognition display REC shown is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later.

  The user moves the hand to the left and right after adjusting the relative position of the hand with respect to the game apparatus 100, and the human body recognition process for the image data acquired by the inner camera 23 recognizes that the user has moved the hand to the left and right a predetermined number of times. Then, the fifth mini game is “success”. Note that an object OBJ21 that prompts the user to touch the hand while the game is in progress is displayed.

(3) Sixth Mini Game In the sixth minigame included in the third game shown in FIG. 16, the object OBJ22 representing “nose” is displayed on the second LCD 22, and the user recognizes his / her index finger and middle finger. Moves the hand so that “is inserted” into a position corresponding to the “hole” of the object OBJ22 representing “nose”.

  In the scene shown in FIG. 16, an image (object OBJ22) on the game effect is displayed on the second LCD 22, and the range of the user's hand obtained by the human body recognition process for the image data acquired by the inner camera 23 is displayed. The recognition display REC shown is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later.

  The user adjusts the relative position of the hand with respect to the game apparatus 100 and then adjusts the position of the hand, and the user's index finger and middle finger are placed in the “hole” of the object OBJ22 by human body recognition processing on the image data acquired by the inner camera 23. If it is recognized that it has been inserted into the corresponding position, the sixth mini-game becomes “success”.

(4) Seventh Mini Game In the seventh mini game included in the third game shown in FIG. 17, an object OBJ 23 representing “a character wearing a cape” is displayed on the second LCD 22, and the user While approaching the face, breath is blown toward the microphone hole 16 (and the microphone 43 accommodated therein) disposed close to the inner camera 23.

  In the scene shown in FIG. 17, an image (object OBJ23) on the game effect is displayed on the second LCD 22, and the range of the user's hand obtained by the human body recognition process for the image data acquired by the inner camera 23 is displayed. The recognition display REC shown is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later.

  The user blows after adjusting the relative position of his / her face with respect to the game apparatus 100, so that the recognition area of the user's face exceeds a predetermined threshold value and the sound pressure when blowing is predetermined When the threshold is exceeded, the seventh mini game is determined to be “success”. If it is determined that the player blows while the game is in progress, the displayed object OBJ23 is “blown away” according to the determination.

(5) Eighth mini game In the eighth minigame included in the third game shown in FIG. 18, an object OBJ24 representing "crying baby" is displayed on the second LCD 22, and the user covers his face with both hands. Such a motion and a motion of releasing both hands from the face are repeated, so-called “not in the hand”.

  In the scene shown in FIG. 18, an image (object OBJ24) on the game effect is displayed on the second LCD 22, and the range of the user's hand obtained by the human body recognition process on the image data acquired by the inner camera 23 is displayed. The recognition display REC shown is reflected. This recognition display REC is sequentially updated every processing cycle of a skin color area detection process described later.

  When it is recognized that the user has repeated the action of covering his / her face with both hands and the action of releasing both hands for a predetermined number of times, the eighth mini game is determined to be “success”.

(6) Display example after the end of the game After the fourth to eighth mini-games (third game) as described above, a cartoon-like expression according to a scenario prepared in advance as shown in FIG. 19 and FIG. Provided.

  After the end of the third game, a title screen prepared in advance as shown in FIG. 19A is displayed on the second LCD 22, and then a screen as shown in FIG. 19B is displayed. In the screen shown in FIG. 19B, an image object according to a scenario prepared in advance is superimposed on the image data captured while the user is playing the fourth mini game (FIG. 14) included in the third game. It is a thing. Then, following the screen shown in FIG. 19B, the screen shown in FIG. 19C is displayed. In the screen shown in FIG. 19C, an image object according to a scenario prepared in advance is overlaid on image data captured while the user is playing the fifth mini game (FIG. 15) included in the third game. It is a thing.

  In the same manner, following the screen shown in FIG. 19C, the screen shown in FIG. 20A, the screen shown in FIG. 20B, and the screen shown in FIG. 20C are sequentially displayed on the second LCD 22. Is displayed. The screen shown in FIG. 20A, the screen shown in FIG. 20B, and the screen shown in FIG. 20C are the sixth mini game (FIG. 16) and the seventh mini game ( FIG. 17) and the eighth mini game (FIG. 18) are obtained by superimposing the respective image objects in accordance with a prepared scenario on the image data captured while the user is playing.

  As shown in FIG. 19 and FIG. 20, objects in accordance with scenarios prepared in advance are superimposed on the appearances of users captured during a series of mini games. Thereby, it can enjoy with the cartoon-like image which made the user who played the 3rd game the hero.

  As shown in FIGS. 14 to 18 described above, in the third game, the success condition associated with each stage (mini game) is determined in a plurality of predetermined stages (fourth to eighth mini games). The game is advanced by judging the success of the game based on each. At this time, in each mini game, objects associated with the corresponding success conditions are switched and displayed.

  Further, as shown in FIG. 19 and FIG. 20 described above, in the third game, the success associated with each stage (mini game) in a plurality of predetermined stages (fourth to eighth mini games). The game is advanced by judging the success of the game based on the conditions. After the end of the third game, a plurality of image data acquired in each of a plurality of stages (mini game) in progress of the game and stored in the image storage unit 116, and a stage in which each image data is acquired (mini game) ) And the object associated with the success condition associated with () are displayed.

<Control structure>
Next, a control structure for providing the above processing will be described.

FIG. 21 is a block diagram showing a control structure of game device 100 according to the present embodiment.
Referring to FIG. 21, game device 100 has, as its control structure, an audio data buffer 102, an audio recognition unit 104, a determination unit 110, an image data buffer 112, a capture controller 114, and an image storage unit 116. A human body recognition unit 120, a game image controller 130, a game image DB (Data Base) 132, a recognition result reflection unit 134,
A replay controller 136, a replay image DB 138, and a rendering engine 140 are included. The control structure shown in FIG. 21 is typically provided such that the CPU 31 executes a game program using hardware such as the main memory 32. Alternatively, all or a part thereof may be embodied by a hardware circuit.

  The audio data buffer 102 accumulates a digital audio signal obtained by D / A conversion of audio acquired by the microphone 43 (FIG. 3) by the I / F circuit 42 (FIG. 3) for a predetermined time. The audio data buffer 102 is provided as a partial area of the main memory 32.

  The voice recognition unit 104 performs known voice recognition on the voice data stored in the voice data buffer 102. Typically, the voice recognition unit 104 determines whether or not the user in the seventh mini game included in the third game shown in FIG. 17 is blowing. Note that it may be determined whether the user is blowing on the basis of whether the sound acquired by the microphone 43 is equal to or higher than a predetermined sound pressure. The recognition result by the voice recognition unit 104 is sent to the determination unit 110.

  The image data buffer 112 accumulates image data acquired from the cameras (the inner camera 23 and the outer camera 25) for a predetermined number of frames. In each game described above, the inner camera 23 is basically used. Therefore, the image data captured by the inner camera 23 is accumulated in the image data buffer 112 as needed. It is provided as a partial area of the main memory 32. The image data buffer 112 is connected to the capture controller 114 and the human body recognition unit 120.

  The capture controller 114 is arranged between the image data buffer 112 and the image storage unit 116, and in response to a capture command from the determination unit 110 described later, the image data stored in the image data buffer 112 is stored in the image storage unit 116. Output to. That is, during the game play (while the inner camera 23 is activated), image data captured by the inner camera 23 is periodically written to the image data buffer 112, and among these image data, Necessary items are selectively stored in the image storage unit 116.

  Image storage unit 116 stores image data displayed after the game according to the present embodiment is over. The image storage unit 116 is provided as a partial area of the main memory 32 or the storage data memory 34. Note that the image data stored in the image storage unit 116 may be stored in the memory card 28 attached to the game apparatus 100 from the outside.

  The human body recognition unit 120 performs human body recognition on the image data acquired by the inner camera 23 and outputs the recognition result to the determination unit 110. More specifically, it includes a face recognition unit 122, a color coordinate conversion unit 124, and a skin color area detection unit 126.

  The face recognition unit 122 performs face recognition processing on the image data stored in the image data buffer 112. That is, the face recognition unit 122 detects a human face area from the captured image data as a user image. At this time, the face recognition unit 122 also extracts feature points of the face included in the face area. The face recognition unit 122 outputs these face recognition results to the determination unit 110. More specifically, a coordinate value indicating a range specified as a face area, a center coordinate such as a pupil, and the like are output as detection results. Note that the face recognition processing in the face recognition unit 122 is a known technique, and therefore will not be described in detail here.

  The color coordinate conversion unit 124 converts image data described in the RGB color system to the HVC color system as preprocessing in the skin color area detection unit 126 described later. That is, in the image data captured by the camera, the color information of each pixel is expressed by gradation values of R (red), G (green), and B (blue), and the color coordinate conversion unit 124 The color information of each pixel is converted into a coordinate system of hue (Hue), luminance (Value), and saturation (Chroma).

  The skin color area detection unit 126 detects a skin color area in the image data acquired by the inner camera 23 based on the color information of the HVC coordinate system output from the color coordinate conversion unit 124 (skin color area detection processing). That is, the skin color area detection unit 126 detects a skin color area in the image data acquired by the inner camera 23 as a user image. As a result, the skin color area detection unit 126 detects a user area indicating a user area in the image data. More specifically, the skin color area detection unit 126 determines whether the hue, luminance, and saturation are included in the corresponding threshold ranges for each pixel, and is included in any threshold range. It is determined that the pixel is “skin color”. In addition, since the color information obtained by imaging the user's skin portion varies depending on the surrounding lighting environment, the user is imaged once or a plurality of times in advance, and based on the image data obtained by these imaging. It is preferable to dynamically determine the threshold range to be used as the skin color. The skin color area detection unit 126 outputs the thus detected skin color area (the coordinate value and / or the number of pixels of each pixel determined to be a skin color) to the determination unit 110.

  Based on the recognition results from the speech recognition unit 104 and the human body recognition unit 120 (the face recognition unit 122 and the skin color area detection unit 126), the determination unit 110 satisfies the conditions set specifically for each game during the game. Sequentially determine whether or not. That is, based on the recognition result, the determination unit 110 determines whether or not a condition associated with the position of the object displayed according to the ongoing game effect is satisfied. More specifically, the determination unit 110 stores a condition group 110a including a plurality of conditions in advance in association with each game, and sets a corresponding condition according to a game selected according to a user command. Select and determine whether the input recognition result satisfies the condition. Then, the determination unit 110 determines that the game is successful when a condition (success condition) for determining that each game is successful is satisfied.

  Note that the determination unit 110 may determine whether or not the game is successful only when the presence of the user is detected by the skin color area detection unit 126 and / or the face recognition unit 122. Here, as a criterion for determining whether or not the presence of the user is detected, when a skin color area of a predetermined area or more is detected by the skin color area detection unit 126, or a face area is detected by the face recognition unit 122. Including cases. By adding such a condition, it is possible to avoid a situation in which the game proceeds due to erroneous determination even though the user does not exist in front of the game apparatus 100.

  Further, the determination unit 110 determines whether or not a condition for outputting a capture command to the capture controller 114 is satisfied in addition to a success condition for determining whether or not the game is successful. To do.

  The determination result by the determination unit 110 is output to the game image controller 130 and the recognition result reflection unit 134. Further, the determination unit 110 outputs the recognition result from the human body recognition unit 120 to the recognition result reflection unit 134.

  The game image controller 130 and the game image DB 132 advance the game based on the recognition result from the determination unit 110. More specifically, the game image DB 132 stores image objects respectively corresponding to a plurality of games prepared in advance, and the game image controller 130 determines the game image based on the recognition result from the determination unit 110. Necessary image objects are output from the DB 132 to the rendering engine 140 at an appropriate timing. As described above, the determination unit 110, the game image controller 130, and the game image DB 132 advance the game based on the result of human body recognition.

  The recognition result reflection unit 134 displays the skin color area (user image) detected by the human body recognition unit 120 during the game as an area recognized as the user's face or hand (recognition display shown in FIG. 7 or the like). REC). The recognition result reflecting unit 134 displays the detected skin color area as “shadow” as shown in FIG. That is, the recognition result reflection unit 134 displays the skin color area (user image) detected during the game in a different manner from the image captured by the inner camera 23.

  The replay controller 136 and the replay image DB 138 perform image display (replay operation) after the above-described game ends. More specifically, the replay image DB 138 stores an image object to be superimposed on the image data after the game ends, and the replay controller 136 reads the image data stored in the image storage unit 116 after the game ends. At the same time, corresponding image objects are read from the replay image DB 138 as necessary, and both are output to the rendering engine 140. As shown in FIG. 10, when displaying a plurality of image data without overlapping any image object, the replay controller 136 sends only the image data stored in the image storage unit 116 to the rendering engine 140. Output.

  The rendering engine 140 controls image display on the first LCD 12 and the second LCD 22. More specifically, the rendering engine 140 displays the image object from the game image controller 130 and the image indicating the recognized area from the recognition result reflecting unit 134 in an overlapping manner while the game is in progress. In addition, after the game ends, the rendering engine 140 displays the image data and the image object from the replay controller 136 in an overlapping manner.

  FIG. 22 is a diagram conceptually showing processing in the rendering engine shown in FIG. FIG. 22 shows an example in which the display on the second LCD 22 shown in FIG.

  Referring to FIG. 22, in rendering engine 140, a video signal for displaying an image on first LCD 12 and second LCD 22 is generated using a plurality of layers. More specifically, the rendering engine 140 includes at least a first layer LYR1 for displaying the image data IMG and a second layer LYR2 for displaying the image object. The image data IMG is expanded on the first layer LYR1, and the object OBJ10 # is expanded on the second layer LYR2. Each of the layers LYR1 and LYR2 is provided by forming a memory area large enough for display on the first LCD 12 or the second LCD 22 in a VRAM (not shown) in the CPU 31. By synthesizing these layers LYR1 and LYR2, a video signal is generated.

  In this way, by expanding the necessary objects in the second layer LYR2, any object can be switched and displayed during the game and during the display of the image based on the image data stored in the image storage unit 116. Can do.

<Processing contents and processing procedure of first game>
Next, processing contents and processing procedures for providing the first game (FIGS. 7 to 10) will be described. In the first game, the game proceeds mainly based on the position of the object displayed according to the game effect and the position and size of the skin color area detected by the skin color area detection unit 126 (FIG. 22).

  FIG. 23 is a diagram for describing the game progress conditions of the first game according to the embodiment of the present invention. Referring to FIG. 23, in the first game, the reference point RP1 set in the image data acquired by the inner camera 23 is centered in association with the position of the object displayed on the second LCD 22 shown in FIGS. A plurality of determination areas AREA1 to AREA5 divided into concentric circles are set. Of the pixels included in these determination areas AREA1 to AREA5, the number of pixels (ratio) detected as “skin color” is used as the game progress condition.

  More specifically, in the first game, the game progress is divided from the first stage to the fifth stage, and the number of pixels detected as “skin color” among the pixels included in the determination areas AREA1 to AREA5, respectively. A ratio with respect to the total number of pixels included in the corresponding determination region is calculated, and the steps are sequentially performed on condition that the calculated ratio exceeds a corresponding threshold value. In the first game, it is assumed that the user gradually approaches the face of the game apparatus 100, and it is assumed that pixels detected as “skin color” gradually spread due to such user movement. It is to be done.

  That is, after the start of the first game, first, the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA1 to the total number of pixels included in the determination area AREA1 is a predetermined threshold Th1. If it exceeds, proceed to the second stage. In the second stage, whether the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA2 to the total number of pixels included in the determination area AREA2 exceeds a predetermined threshold value Th2. Is continuously determined, and if the threshold value Th2 is exceeded, the process proceeds to the third stage.

  Similarly, in the third stage, the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA3 to the total number of pixels included in the determination area AREA3 is a predetermined threshold Th3. Is continuously determined. If the threshold value Th3 is exceeded, the process proceeds to the fourth stage. In the subsequent fourth stage, whether or not the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA4 to the total number of pixels included in the determination area AREA4 exceeds a predetermined threshold Th4. Is continuously determined, and if the threshold value Th4 is exceeded, the process proceeds to the fifth stage. In the subsequent fifth stage, the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA5 to the total number of pixels included in the determination area AREA5 exceeds a predetermined threshold Th5. Whether the first game is “successful” when the threshold value Th5 is exceeded.

  Note that the thresholds Th1 to Th5 are all preferably set to 50% or more. Further, in the above description, each determination area is exemplified by the ratio of the number of pixels detected as “skin color”. However, the determination is based on the absolute number of pixels detected as “skin color”. You may do it.

  As described above, at each stage, it is determined whether or not the condition is satisfied based on the pixels included in the corresponding determination region. A detection result that is different from the user's expected movement, for example, immediately after the first game is started, even though the number of pixels detected as “skin color” in the inner determination region is small, the outer determination In the case where the number of pixels detected as “skin color” in the region is large, it is possible to avoid an erroneous determination that the first game is “successful” suddenly.

  Further, the game effect is changed according to the stage of the above-described game progress. For example, in the first to third stages, the image object as shown in FIG. 7 is displayed, and the display is switched to the image object as shown in FIG. 8 at the timing of proceeding from the third stage to the fourth stage. Further, when the condition in the fifth stage is satisfied and the first game is determined to be “successful”, the display is switched to an image object as shown in FIG.

  In each game according to the present embodiment, the user's playing appearance is captured at a timing when a predetermined condition is satisfied. In the first game, the image data acquired by the inner camera 23 is captured at the timing when the conditions in the first to fifth stages are satisfied. That is, the captured image data includes the user's appearance around the timing at which the stage of game progress changes.

  Note that the positions of the reference point RP1 and the determination areas AREA1 to AREA5 are appropriately changed depending on the position of the object displayed on the game effect.

  As described above, in the first game, in a plurality of predetermined stages, the game progresses by determining the success of the game based on the success conditions associated with each stage. After the end of the first game, a plurality of image data acquired at each of a plurality of stages during the game and stored in the image storage unit 116 are continuously displayed.

  FIG. 24 is a flowchart showing a processing procedure of the first game according to the embodiment of the present invention. The flowchart shown in FIG. 24 is executed when the user gives an instruction to start the first game.

  Referring to FIG. 24, first, in step S <b> 100, the CPU 31 acquires image data captured by the inner camera 23. In subsequent step S102, the CPU 31 detects a skin color region in the acquired image data. Further, in step S104, the CPU 31 determines that the ratio of the number of pixels detected as “skin color” to the total number of pixels included in the determination area AREA1 among the pixels included in the determination area AREA1 illustrated in FIG. It is determined whether or not it exceeds. If the ratio does not exceed threshold value Th1 (NO in step S104), the process of step S100 is repeated. On the other hand, if the ratio exceeds threshold value Th1 (YES in step S104), the process proceeds to step S106.

  In step S <b> 106, the CPU 31 stores the most recently acquired image data in the main memory 32 or the storage data memory 34. That is, the CPU 31 captures the appearance of the user playing. Then, the process proceeds to step S108. That is, the progress of the first game proceeds from the first stage to the second stage.

  In step S <b> 108, the CPU 31 acquires image data captured by the inner camera 23. In subsequent step S110, CPU 31 detects a skin color region in the acquired image data. Further, in step S112, the CPU 31 determines that the ratio of the number of pixels detected as “skin color” to the total number of pixels included in the determination area AREA2 among the pixels included in the determination area AREA2 illustrated in FIG. It is determined whether or not it exceeds. If the ratio does not exceed threshold value Th2 (NO in step S112), the process of step S108 is repeated. On the other hand, if the ratio exceeds threshold value Th2 (YES in step S112), the process proceeds to step S114.

  In step S <b> 114, the CPU 31 stores the most recently acquired image data in the main memory 32 or the storage data memory 34. That is, the CPU 31 captures the appearance of the user playing. Then, the process proceeds to step S116. That is, the progress of the first game proceeds from the second stage to the third stage.

  In step S <b> 116, the CPU 31 obtains image data captured by the inner camera 23. In subsequent step S118, CPU 31 detects a skin color area in the acquired image data. Further, in step S120, the CPU 31 determines that the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA3 shown in FIG. 23 to the total number of pixels included in the determination area AREA3 is the threshold Th3. It is determined whether or not it exceeds. If the ratio does not exceed threshold value Th3 (NO in step S120), the process of step S116 is repeated. On the other hand, if the ratio exceeds threshold value Th3 (YES in step S120), the process proceeds to step S122.

  In step S 122, the CPU 31 stores the most recently acquired image data in the main memory 32 or the storage data memory 34. That is, the CPU 31 captures the appearance of the user playing. In further subsequent step S124, the CPU 31 switches the image object to be displayed from the image object shown in FIG. 7 to the image object shown in FIG. Then, the process proceeds to step S126. That is, the progress of the first game proceeds from the third stage to the fourth stage.

  In step S <b> 126, the CPU 31 acquires image data captured by the inner camera 23. In subsequent step S128, CPU 31 detects a skin color region in the acquired image data. Further, in step S130, the CPU 31 determines that the ratio of the number of pixels detected as “skin color” among the pixels included in the determination area AREA4 shown in FIG. 23 to the total number of pixels included in the determination area AREA4 is the threshold Th4. It is determined whether or not it exceeds. If the ratio does not exceed threshold value Th4 (NO in step S130), the process of step S126 is repeated. On the other hand, if the ratio exceeds threshold value Th4 (YES in step S130), the process proceeds to step S132.

  In step S <b> 132, the CPU 31 stores the most recently acquired image data in the main memory 32 or the storage data memory 34. That is, the CPU 31 captures the appearance of the user playing. Then, the process proceeds to step S134. That is, the progress of the first game proceeds from the fourth stage to the fifth stage.

  In step S134, the CPU 31 obtains image data captured by the inner camera 23. In subsequent step S136, the CPU 31 detects a skin color region in the acquired image data. Further, in step S138, the CPU 31 determines that the ratio of the number of pixels detected as “skin color” to the total number of pixels included in the determination area AREA5 among the pixels included in the determination area AREA5 illustrated in FIG. It is determined whether or not it exceeds. If the ratio does not exceed threshold value Th5 (NO in step S138), the processes in and after step S134 are repeated. On the other hand, if the ratio exceeds threshold value Th5 (YES in step S138), the process proceeds to step S140.

  In step S <b> 140, the CPU 31 stores the most recently acquired image data in the main memory 32 or the storage data memory 34. That is, the CPU 31 captures the appearance of the user playing. In further subsequent step S142, CPU 31 switches the image object to be displayed from the image object shown in FIG. 8 to the image object shown in FIG. Then, the process proceeds to step S144. That is, the first game is determined to be “successful”, and the progress of the first game ends.

  In addition, when the series of processes from the above-described steps S100 to S142 does not end within a predetermined time, it may be determined that the first game has been “failed”. In this case, a screen indicating that the first game is “failure” is displayed instead of the processing after step S144.

  In step S144, the CPU 31 reads a plurality of captured image data from the main memory 32 or the storage data memory 34, and sequentially switches and displays them on the second LCD 22 in a predetermined order. That is, the CPU 31 replays and displays the appearance of the user who is playing the game. The display order of the image data may be a captured time order or may be a random order independent of the captured time order.

  In subsequent step S146, CPU 31 determines whether or not the user has instructed termination. If termination is not instructed by the user (NO in step S146), the process in step S144 is repeated. On the other hand, when the end is instructed by the user (YES in step S146), the process for the first game ends.

<Processing contents and processing procedure of second game>
As described above, the second game according to the present embodiment includes a plurality of mini games. In each mini game, the user's playing appearance is captured in parallel with the progress of the game. When a series of mini-games is completed, various objects are displayed in an overlapping manner on the image data captured in each mini-game.

(1) Overall Processing FIG. 25 is a flowchart showing a processing procedure of the second game according to the embodiment of the present invention. The flowchart shown in FIG. 25 is executed when the user instructs the start of the second game.

  Referring to FIG. 25, first, in step S20, CPU 31 executes a mini game. In the mini game to be executed, game progress and image data capture are executed in parallel. In the subsequent step S30, the CPU 31 determines whether or not the execution of all the prepared mini games has been completed. If execution of all mini-games has not been completed (NO in step S30), the process of step S20 is repeated. That is, the process of step S20 is repeated for the number of mini games prepared in advance. If execution of all mini-games has been completed (YES in step S30), that is, after the end of the second game, the process proceeds to a replay process in step S32 and subsequent steps.

  In step S32, the CPU 31 reads one image data out of the image data captured in the previous mini game. In subsequent step S34, CPU 31 reads an object corresponding to the read image data. In further succeeding step S36, the CPU 31 displays the object read in step S34 so as to overlap the image data read in step S32. After the elapse of the predetermined time, the process proceeds to step S38.

  In step S38, the CPU 31 determines whether or not the user has instructed the end. If termination is not instructed by the user (NO in step S38), the process in step S32 is repeated. On the other hand, if the end is instructed by the user (YES in step S38), the process for the second game ends.

  Next, processing contents and processing procedures for providing the first to third mini games (FIGS. 11 to 13) included in the second game will be described.

(2) First mini-game In the first mini-game included in the second game, mainly the position of the object displayed in accordance with the game effect and the facial feature amount detected by the face recognition unit 122 (FIG. 22). The game proceeds based on the position. More specifically, it is determined whether or not a condition related to the game progress is satisfied based on the center coordinates of the “pupil” among the facial feature amounts recognized by the face recognition unit 122.

  FIG. 26 is a diagram for describing the game progress conditions of the first mini game included in the second game according to the embodiment of the present invention. Referring to FIG. 26, in the first mini game, the image data acquired by the inner camera 23 in association with the center point of the “glasses” object OBJ10 displayed on the second LCD 22 shown in FIG. A reference point RP2 is set. Then, determination areas DTL and DTR are set symmetrically about the reference point RP2. These determination areas DTL and DTR are set so as to correspond to the lens portion of the object “OBJ10” of “glasses”. In FIG. 26, the determination areas DTL and DTR are rectangular. However, the determination areas are not limited to this and may be circular or elliptical. Further, the degree of inclusion of the value of the center coordinates of the recognized “pupil” in each of the determination areas DTL and DTR is set as the game progress condition.

  More specifically, in the first mini game, for each determination cycle, it is determined whether or not the recognized center coordinates DEP of the “pupil” are included in the determination areas DTL and DTR, respectively. A predetermined score is added, and if not included, the predetermined score is subtracted. As a result, the first mini game becomes “successful” when the cumulative point exceeds a predetermined threshold.

  In the first mini game included in the second game, a plurality of intermediate threshold values are set in advance between the initial value (typically zero) and the threshold value determined as “success”. At the timing when the cumulative point exceeds these intermediate thresholds for the first time, the user's playing appearance is captured. Typically, five threshold values of Th11, Th12, Th13, Th14, and Th15 (where Th11 <Th12 <Th13 <Th14 <Th15) are set in advance, and the cumulative value is the first time that each of the threshold values is set. When the image data is captured at a timing exceeding the threshold value and the accumulated value exceeds the threshold Th5, the first mini game is “successful”. Therefore, the captured image data includes the user's appearance at the timing when the accumulated value exceeds the threshold value.

  The position of the reference point RP2 and the positions and sizes of the determination areas DTL and DTR are appropriately changed depending on the type and position of the object displayed on the game effect.

  FIG. 27 is a flowchart showing a processing procedure of the first mini game included in the second game according to the embodiment of the present invention.

  Referring to FIG. 27, first, in step S200, CPU 31 initializes accumulated value S (sets accumulated value S to “zero”). In subsequent step S202, CPU 31 initializes threshold value ThS (sets threshold value ThS, which is the smallest threshold value Th11).

  In step S <b> 204, the CPU 31 acquires image data captured by the inner camera 23. In the subsequent step S206, the CPU 31 recognizes the center coordinates of the left and right “pupils” in the acquired image data. In the subsequent step S208, it is determined whether or not the recognized center coordinates of the left and right “pupils” are included in the determination regions DTL and DTR (FIG. 26). If the center coordinates of the left and right “pupils” are included in the determination areas DTL and DTR (YES in step S208), a fixed value α is added to the accumulated value S (step S210). On the other hand, when the center coordinates of the left and right “pupils” are not included in the determination areas DTL and DTR (NO in step S208), the constant value β is subtracted from the accumulated value S (step S212). Thereafter, the process proceeds to step S214.

  In step S214, the CPU 31 determines whether or not the accumulated value S exceeds the threshold value ThS. If cumulative value S does not exceed threshold value ThS (NO in step S214), the processes in and after step S202 are repeated. On the other hand, if accumulated value S exceeds threshold value ThS (YES in step S214), the process proceeds to step S216.

  In step S <b> 216, the CPU 31 stores the most recently acquired image data in the main memory 32 or the storage data memory 34. That is, the CPU 31 captures the appearance of the user playing. In subsequent step S218, CPU 31 determines whether or not threshold value Th15 which is the largest threshold value is set to current threshold value ThS. When threshold value Th15 is not set to current threshold value ThS (NO in step S218), CPU 31 determines the next larger threshold value than the threshold value currently set to threshold value ThS. Is set (step S220), and the processes in and after step S202 are repeated.

  On the other hand, when threshold value Th15 is set to current threshold value ThS (YES in step S218), the first mini game is determined to be “successful”, and the progress of the first mini game ends. .

  In addition, when the series of processes from the above-described steps S200 to S220 does not end within a predetermined time, it may be determined that the first mini game has been “failed”. In addition, it is preferable to determine a minimum value (for example, “zero”) of the cumulative value S.

(3) Second Mini Game In the second mini game included in the second game, mainly the position of the object displayed according to the game effect and the skin color area detected by the skin color area detection unit 126 (FIG. 22). The game progresses based on the position and size.

  FIG. 28 is a diagram for describing the game progress conditions of the second mini game included in the second game according to the embodiment of the present invention. Referring to FIG. 28, in the second mini game, the image data acquired by inner camera 23 is associated with the center point of “mosquito” object OBJ11 displayed on second LCD 22 shown in FIG. A reference point RP3 is set. A determination area DT31 including the reference point RP3 is set around the reference point RP3, and a determination area DT32 including the reference point RP3 is set in a wider range than the determination area DT31. Furthermore, determination lines DL1 to DL8 are set in a total of eight directions, starting from the reference point RP3, up and down, left and right, and diagonally. The number of pixels (ratio) detected as “skin color” among the pixels included in the determination regions DT31 and DT31 and the determination lines DL1 to DL8 is set as the game progress condition.

  More specifically, in the second mini game, the game is determined to be “successful” when all of the following conditions (i) to (iii) are satisfied.

(I) The ratio of the number of pixels detected as “skin color” among the pixels included in the determination region DT32 to the total number of pixels included in the determination region DT32 exceeds a predetermined threshold Th32. (Ii) Determination line The ratio of the number of pixels detected as “skin color” among the pixels on DL1 to DL8 to the total number of pixels on the determination lines DL1 to DL8 all exceeds a predetermined threshold Th33. (Iii) Determination region DT31 The ratio of the number of pixels detected as “skin color” to the total number of pixels included in the determination area DT31 does not exceed a predetermined threshold value Th31 among the pixels included in (i) and (ii) Is a condition for determining that the user's “hand” is in the vicinity of the displayed “mosquito” object OBJ11. Further, (iii) is a condition for avoiding erroneous recognition of the user's face or the like as a “hand”.

  Further, in the second mini game included in the second game, image data is periodically acquired and accumulated by a predetermined number from the most recent one. Then, at the timing when the second mini game is “successful”, the acquisition (update) of the image data is stopped. Therefore, it is possible to capture the appearance of the user playing during the predetermined period from the timing when the second mini game is “successful”.

  Note that the position of the reference point RP3 and the sizes of the determination areas DT31 and DT32 are appropriately changed depending on the position and number of objects displayed in the game effect.

  FIG. 29 is a flowchart showing a processing procedure of the second mini game included in the second game according to the embodiment of the present invention.

  Referring to FIG. 29, first, in step S240, CPU 31 obtains image data captured by inner camera 23. In subsequent step S242, CPU 31 determines whether or not a prescribed number of image data has already been stored in main memory 32 or storage data memory 34. If the prescribed number of image data has not been stored (NO in step S242), the process proceeds to step S246. If the prescribed number of image data has already been stored (YES in step S242), the CPU 31 deletes the oldest acquired image data from the stored image data (step S244), and starts a new one. The acquired image data is stored in the main memory 32 or the storage data memory 34 (step S246). Then, the process proceeds to step S248.

  In step S248, the CPU 31 detects a skin color area in the acquired image data. In subsequent step S250, CPU 31 determines whether or not all of the above-described determination conditions (i) to (iii) are satisfied. If any of the above-described determination conditions (i) to (iii) is not satisfied (NO in step S250), the processing in step S240 and subsequent steps is repeated. If all of the above-described determination conditions (i) to (iii) are not satisfied (YES in step S250), the second mini game is determined to be “successful”, and the progress of the second mini game ends. .

(4) Third Mini Game In the third mini game included in the second game, the position of the object displayed according to the game effect and the position of the face area detected by the face recognition unit 122 (FIG. 22). The game proceeds based on the above.

  FIG. 30 is a diagram for describing the conditions for the progress of the third mini game included in the second game according to the embodiment of the present invention. Referring to FIG. 30, in the third mini game, the image is acquired by inner camera 23 in association with the cutout portions of objects “OBJ14” and “OBJ15” of “totem pole” displayed on second LCD 22 shown in FIG. Reference points RP41 and RP42 are set in the image data, respectively. A determination region DT41 including the reference point RP41 is set with the reference point RP41 as the center, and a determination region DT42 including the reference point RP42 is set with the reference point RP42 as the center. Then, it is determined whether or not the recognized face areas DFA1 and DFA2 are included in these determination areas DT41 and DT42, respectively. If they are included, a predetermined score is added. Subtract points. As a result, the third mini game is “successful” when the cumulative point exceeds a predetermined threshold.

  That is, the third mini game is determined to be “successful” when it can be determined that the user's face is located at any of the cutout portions of the objects “OBJ14” and “OBJ15” of the “totem pole”.

  Further, in the third mini game included in the second game, image data is periodically acquired and accumulated by a predetermined number from the latest game. Then, at the timing when the third mini game is “successful”, the acquisition (update) of the image data is stopped. Therefore, it is possible to capture the user's playing appearance before a predetermined period from the timing when the third mini game is “successful”.

  Note that the positions of the reference points RP41 and RP42 and the sizes of the determination areas DT41 and DT42 are appropriately changed depending on the position and number of objects displayed in the game effect.

  Since the processing procedure of the third mini game included in the second game is the same as the processing procedure of the first mini game shown in FIG. 27 described above, detailed description will not be repeated.

<Processing contents and processing procedure of third game>
As described above, the third game according to the present embodiment includes a plurality of mini games executed in series. In each mini game, in parallel with the progress of the game, at least one image of the user playing at the timing when a predetermined condition is satisfied is captured. When a series of mini-games are completed, an object having a specific meaning is superimposed on each captured image data to display a cartoon-like expression according to a prepared scenario. Provided.

  In game device 100 according to the present embodiment, a plurality of scenarios are prepared in advance, and a plurality of mini-games are executed in a predetermined order along one scenario selected every time the third game is executed. The

  In each mini game, it is determined whether or not the game progress condition is satisfied based on the result of the human body recognition process (skin color area detection process and face recognition process). Since these game progress conditions are the same as or similar to the conditions in each mini-game included in the second game described above, detailed description will not be repeated.

  FIG. 31 is a flowchart showing a third game processing procedure according to the embodiment of the present invention. The flowchart shown in FIG. 31 is executed when the start of the third game is instructed by the user.

  Referring to FIG. 31, first, in step S300, CPU 31 randomly selects one scenario from a plurality of prepared scenarios. In subsequent step S302, CPU 31 determines a plurality of mini-games to be executed along with the scenario selected in step S300, and determines the order of execution. Then, the process proceeds to step S304.

  In step S304, the CPU 31 executes the first mini game set. The CPU 31 proceeds with the mini game based on the result of the human body recognition process (skin color area detection process and face recognition process). In subsequent step S306, CPU 31 determines whether or not the first mini-game is “successful”. If the first mini game is “successful” (YES in step S306), the CPU 31 captures image data in the first mini game (step S308). On the other hand, if the first mini game has not been “successful” (NO in step S306), CPU 31 sets a flag indicating that the first mini game has ended in “failure”. (Step S310).

  Thereafter, the same processing as steps S304 to S310 is repeatedly executed for each of the second to fifth mini games. When the series of mini games is completed, the process proceeds to step S312.

  In step S312, the CPU 31 reads and displays a title image corresponding to the selected scenario (see FIG. 19A). In subsequent step S314, CPU 31 reads out and displays the image data acquired in the first mini game and the corresponding image object (see FIG. 19B). When a flag indicating that the mini game has ended in “failure” is set for the corresponding mini game, the CPU 31 displays an image indicating that the game has failed.

  In subsequent step S316, CPU 31 reads out and displays the image data acquired in the second mini game and the corresponding image object (see FIG. 19C). Similarly, in steps S318 to S322, the CPU 31 sequentially reads out and displays the image data and the corresponding image objects acquired in the third to fifth mini games (FIGS. 20A to 20C). )reference).

  In subsequent step S324, CPU 31 determines whether or not the user has instructed termination. If termination is not instructed by the user (NO in step S324), the processes in and after step S312 are repeated. On the other hand, if the end is instructed by the user (YES in step S324), the process for the third game ends.

  As described above, a plurality of image data acquired by capturing images when a predetermined condition is satisfied during the game may be displayed, or a plurality of image data captured during the game may be displayed. Such image data may be sequentially acquired, and after the game, for example, a plurality of acquired image data satisfying a predetermined condition may be extracted and displayed.

  In the above-described embodiment, the information amount included in the image data acquired by the imaging unit is used for display as it is. However, the information amount per pixel of the acquired image data is reduced and the image is Display may be performed.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

DESCRIPTION OF SYMBOLS 11 1st housing, 12 1st LCD, 13 Touch panel, 14A-14K button, 15A 1st LED, 15B 2nd LED, 15C 3rd LED, 16 Microphone hole, 21 2nd housing, 22 2nd LCD, 23 Inside camera, 24 Sound removal Hole, 25 Outside camera, 26 4th LED, 27 Touch pen, 28, 29 Memory card, 32 Main memory, 33 Memory control circuit, 34 Data memory for saving, 35 Memory for preset data, 36, 37 Memory card I / F, 38 Wireless communication module, 39
Local communication module, 41 power supply circuit, 42 interface circuit (I / F circuit), 43 microphone, 44 amplifier, 45 speaker, 100 game device, 102 voice data buffer, 104 voice recognition unit, 110 determination unit, 110a condition group, 112 Image data buffer, 114 capture controller, 116 image storage unit, 120 human body recognition unit, 122 face recognition unit, 124 color coordinate conversion unit, 126 skin color area detection unit, 130 game image controller, 132 game image DB, 134 recognition result reflection unit 136 Replay controller, 138 Replay image DB, 140 rendering engine, AP access point, NW network, SRV distribution server device.

Claims (17)

A game program executed by a computer of a game device capable of using an imaging unit and a display unit, wherein the imaging unit is adapted to image a user, and the game program stores the computer on the imaging unit. game progress means causes the display device display an image of the shadow of the user as a mode different from the image of the user taken to proceed with the game on the basis of the image of the user by, and
Wherein after the end of the game, at least, to function as a display control unit, to be displayed on said display means the images of the user captured by the imaging means when the image of the user is satisfied the predetermined condition, game program. Wherein the display control unit, wherein by the game progress means when the game is not in progress, and displays the images of the user captured by the pre-Symbol image pickup means on the display means, a game program according to claim 1. The game progress means determines whether the game is successful based on the image of the user,
Wherein the display control unit, at least, the game displays the images of the user captured by the imaging means when it is judged to be successful on the display means, a game program according to claim 1 or 2. The game program according to claim 3 , wherein the game progress means determines whether or not the game is successful based on a position of the image of the user and a position of an object displayed in accordance with a game effect. The game program causes the computer to further function as presence detection means for detecting the presence of a user,
It said game progress means, when the presence of the user is detected by the presence detection means, for determining whether the game is successful, a game program according to claim 3 or 4. The display control means includes
Between the game start to the game termination, every time a plurality of different conditions, including conditions that determine the the game was successful is satisfied, stores the images of the user, respectively,
Wherein after the end of the game to the images of the plurality of users stored by switching in a predetermined order to be displayed, a game program according to any one of claims 1-5. The display control means displays superimposed a predetermined object to a user of the image captured by the imaging means, a game program according to any one of claims 1-6. Wherein the display control unit, the object corresponding to the game presentation when the images of the user is captured is displayed over the images of the user, the game program according to claim 7. Wherein the display control unit, the game effect when the images of the user is captured to display the independent object superimposed on the images of the user, the game program according to claim 7. The game program, the computer further during the display of images of the user captured by the progress and the imaging means of the game, to function as the object display means for displaying the object on the display means,
The object display means, the position of the object displayed on the progress of the game, by replacing the object to another object, is displayed during the display of images of the user captured by the imaging unit, according to claim 1 The game program according to any one of to 6 . The game program, the computer further during the display of images of the user captured by the progress and the imaging means of the game, to function as the object display means for displaying the object on the display means,
The object display means, an object not associated with the object displayed on the progress of the game, is displayed during the display of images of the user captured by the imaging means, any one of claims 1 to 6 The game program described in. The game progress means advances the game by determining the success of the game based on the success conditions associated with each stage in a plurality of predetermined stages,
The game program causes the computer to further function as an object display unit that switches an object associated with a success condition corresponding to each stage and displays the object on the display unit according to the progress of the game. The game program according to any one of to 6 . The game progress means advances the game by determining the success of the game based on the success conditions associated with each stage in a plurality of predetermined stages,
Wherein the display control unit, success and images of a plurality of users captured by pre SL imaging means Te respective smell of the plurality of stages of the game progress, images of each user associated with step imaged The game program of any one of Claims 1-6 which displays the image which synthesize | combined with the object linked | related with conditions. The game progress means advances the game by determining the success of the game based on the success conditions associated with each stage in a plurality of predetermined stages,
Said display control means, to continuously display the images of a plurality of users captured by pre SL imaging means Te respective smell of the plurality of stages of the game progress, in any one of claims 1 to 6 The described game program. A game system,
An imaging unit adapted to image a user;
A display unit;
An arithmetic processing unit,
The arithmetic processing unit includes:
Together to display the shadow image of the user as a mode different from the images of the user captured by the imaging unit on the display unit, and the progress of the game based on the image of the user,
After completion of the game, at least, it is capable of executing a display on the display unit of the images of the user captured by the front SL imaging unit when the image of the user is satisfied the predetermined condition, game system. An acquisition unit configured to acquire images from adapted imaging device to image the user,
Causes display on the display device an image of the shadow of the user as a mode different from the images of the acquired user by the acquisition unit, a traveling means for traveling the game based on the image of the user,
After completion of the game, at least, and a display control means for displaying on the display device the images of the user acquired by the previous SL acquisition means when the image of the user is satisfied the predetermined condition, the game apparatus . Acquiring images from adapted imaging device to image the user,
Together causes the display device to display the image of the shadow of the user as a mode different from the images of the user the acquired, a step of proceeding a game based on the image of the user,
Wherein after the end of the game, at least, and a step of displaying the images of the user image of the user is acquired when satisfying the predetermined condition in the display device, a game control method.