JP5066047B2 - Information processing apparatus, information processing method, program, and information storage medium - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, a program, and an information storage medium.

  A technique relating to motion capture that captures the movement of a moving body such as a human being imaged by a video camera or the like into an information processing apparatus is known. For example, there is an information processing apparatus that generates motion data indicating the movement of a moving object based on a series of images generated by imaging the moving object with a video camera. In such an information processing device, a user marks several positions in an image to be generated and specifies the context of each mark to simulate the three-dimensional movement of a moving object. There is something that can generate motion data.

  However, in the conventional information processing apparatus, when the motion data is generated by simulating the three-dimensional movement of the moving body, it is necessary for the user to specify the position in the image, which is troublesome.

  The present invention has been made in view of the above problems, and is an information processing apparatus and information processing that can reduce the user's trouble in generating motion data by simulating the three-dimensional movement of a moving object It is an object to provide a method, a program, and an information storage medium.

  In order to solve the above-described problems, an information processing apparatus according to the present invention is an information processing apparatus that generates motion data indicating a motion of a predetermined moving body, and an imaging unit captures at least one object including the moving body. Basic data for acquiring a plurality of basic data including an image generated by imaging and distance data indicating a distance between the object displayed on the image and the imaging unit based on a measurement result by the distance measuring unit Based on an image included in each of the basic data, an acquisition unit, an in-image position specifying unit that specifies a position in the image where the moving body is displayed, and the in-image position specifying unit A distance specifying means for specifying a distance between the moving body and the imaging means based on a position in the image and the distance data included in the basic data including the image; Position coordinate calculating means for calculating position coordinates in the three-dimensional space of the moving body based on the position in the image specified by the position specifying means in the image and the distance specified by the distance specifying means; And motion data generating means for generating motion data indicating the movement of the movable body in a three-dimensional space based on the position coordinates calculated based on the basic data.

  The information processing method according to the present invention is an information processing method for generating motion data indicating a motion of a predetermined moving body, and is generated when an imaging unit images at least one object including the moving body. A basic data acquisition step for acquiring a plurality of basic data including an image and a distance data indicating a distance between the object displayed in the image and the imaging unit based on a measurement result by the distance measuring unit; Based on the image included in the basic data, an in-image position specifying step for specifying the position in the image where the moving body is displayed; a position in the image specified by the in-image position specifying step; A distance specifying step for specifying a distance between the moving body and the imaging means based on the distance data included in the basic data including the image; A position coordinate calculating step for calculating a position coordinate in the three-dimensional space of the moving body based on the position in the image specified by the inner position specifying step and the distance specified by the distance specifying step; And a motion data generation step of generating motion data indicating movement of the mobile body in a three-dimensional space based on the position coordinates calculated based on the basic data.

  The program according to the present invention is a program that causes a computer to function as an information processing apparatus that generates motion data indicating a motion of a predetermined moving body, and an imaging unit images at least one object including the moving body. Basic data acquisition means for acquiring a plurality of basic data including an image generated by the measurement and distance data indicating a distance between the object displayed in the image and the imaging means based on a measurement result by the distance measurement means A position in the image that is specified by the position specifying means in the image, and a position in the image that is specified by the position specifying means in the image based on the image included in each basic data And the distance specifying the distance between the moving body and the imaging means based on the distance data included in the basic data including the image A position for calculating a position coordinate in the three-dimensional space of the moving body based on a position in the image specified by the determining means and the position specifying means in the image and a distance specified by the distance specifying means Coordinate calculation means, causing the computer to function as motion data generation means for generating motion data indicating movement of the mobile body in a three-dimensional space based on the position coordinates calculated based on the basic data. Features.

  The above program can also be stored in a computer-readable information storage medium.

  According to the present invention, since the position coordinates in the three-dimensional space of the moving object are calculated based on the distance data based on the measurement result by the distance measuring means, the motion data is simulated by simulating the three-dimensional movement of the moving object. It is possible to reduce the user's trouble in generating.

  In one aspect of the present invention, the moving body includes a plurality of elements, and the in-image position specifying unit includes at least a part of the elements included in the moving body. The position is specified, the distance specifying means specifies the distance between at least some of the elements included in the moving body and the imaging means, and the position coordinate calculating means is included in the moving body. The position coordinates in the three-dimensional space of each element of at least some of the elements to be calculated are calculated, and the position coordinate calculation unit is used to calculate a missing element among the plurality of elements included in the moving body based on the basic data If the position coordinates of the missing element are not calculated, the position coordinate calculation means calculates the position coordinates of the missing element from the basic data different from the basic data among the plurality of basic data acquired by the basic data acquisition means. And calculating on the basis of. In this way, when the position coordinates of the missing element in the three-dimensional space are not calculated from the basic data, based on the basic data different from the basic data among the plurality of basic data acquired by the basic data acquisition means. The position coordinates of the missing element in the three-dimensional space can be calculated.

  Further, in this aspect, when the position coordinate calculation unit does not calculate the position coordinate of the missing element continues for a predetermined time, the position coordinate calculation unit sets the position coordinate of the missing element before the time. May be calculated based on the position coordinates of the missing element calculated last in step 1 and the position coordinates of the missing element calculated first after the time. In this way, when the position coordinates of the missing elements in the three-dimensional space cannot be calculated over a predetermined time, the position coordinates of these missing elements in the three-dimensional space are calculated immediately before and after the predetermined time. It can be calculated based on the position coordinates of the missing element.

  Further, in this aspect, when the position coordinate of the missing element is not calculated by the position coordinate calculation means, the position coordinate of the missing element is calculated based on each of a plurality of basic data different from the basic data. You may calculate based on the difference of the position coordinate of a missing element. In this way, when the position coordinates of the missing element in the three-dimensional space are not calculated based on the basic data, the position of the missing element in the three-dimensional space calculated based on a plurality of basic data different from the basic data. Using the coordinate difference information, the position coordinates of the missing element can be calculated.

  In one embodiment of the present invention, the basic data is distance data indicating a distance between an image generated by each of a plurality of imaging units, an imaging unit that captured each of the images, and an object displayed in the image. The position specifying means in the image specifies the position where the moving body is displayed in each image based on each of the plurality of images included in the basic data, and the distance specifying means The distance between the moving body and each imaging means is specified, and the position coordinate calculating means is specified by the distance specifying means and the positions in the plurality of images specified by the in-image position specifying means. The position coordinates are calculated based on a plurality of the distances. In this way, since the blind spot when the object is imaged by the imaging means is reduced, it is possible to reduce the possibility that the position coordinates of the moving body in the three-dimensional space are not calculated based on the basic data.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing system 1 according to the present embodiment. As shown in FIG. 1, the information processing system 1 according to the present embodiment includes an information processing device 2 and a portable game device 3. FIG. 2 is a perspective view showing an example of the portable game apparatus 3 according to the present embodiment.

  The information processing apparatus 2 includes a control unit 21, a storage unit 22, a communication unit 23, and a user interface (UI) unit 24. The control unit 21, the storage unit 22, the communication unit 23, and the UI unit 24 are connected via a bus.

  The control unit 21 of the information processing apparatus 2 is a program control device such as a CPU, and operates according to a program installed in the information processing apparatus 2.

  The storage unit 22 of the information processing apparatus 2 is a storage element such as a RAM or a hard disk. The storage unit 22 of the information processing device 2 stores a program executed by the control unit 21 of the information processing device 2. In addition, the storage unit 22 of the information processing device 2 also operates as a work memory of the control unit 21 of the information processing device 2.

  The communication unit 23 of the information processing apparatus 2 is a network interface, for example, and transmits information according to an instruction input from the control unit 21 of the information processing apparatus 2. The communication unit 23 of the information processing apparatus 2 outputs information received via a network such as a LAN to the control unit 21 of the information processing apparatus 2.

  The UI unit 24 of the information processing apparatus 2 is a display, a microphone, a speaker, a button, and the like, and outputs the content of the operation performed by the user and the voice input by the user to the control unit 21. In addition, the UI unit 24 of the information processing device 2 displays and outputs information according to an instruction input from the control unit 21 of the information processing device 2.

  As shown in FIG. 2, the shape of the housing 30 of the portable game apparatus 3 according to the present embodiment is a box shape. As shown in FIG. 1, the portable game apparatus 3 according to the present embodiment includes a control unit 31, a storage unit 32, a communication unit 33, a display unit 34, an operation unit 35, an imaging unit 36, a distance measurement unit 37, It has.

  The control unit 31 of the portable game apparatus 3 is a program control device such as a CPU, and operates according to a program stored in the storage unit 32 of the portable game apparatus 3.

  The storage unit 32 of the portable game apparatus 3 is a storage element such as a RAM or a hard disk. The storage unit 32 of the portable game device 3 stores a program executed by the control unit 31 of the portable game device 3. Specifically, for example, a game program executed by the control unit 31 of the portable game apparatus 3 is stored. This game program is supplied to the portable game apparatus 3 via an information transmission medium such as a disk, CD-ROM, DVD-ROM or the like, or via a communication network such as the Internet. The storage unit 32 of the portable game apparatus 3 also operates as a work memory of the control unit 31 of the portable game apparatus 3.

  The communication unit 33 of the portable game apparatus 3 is, for example, a network interface or the like (specifically, for example, a wireless LAN module), and the information processing apparatus according to an instruction input from the control unit 31 of the portable game apparatus 3 Send information to 2 etc. In addition, the communication unit 33 of the portable game apparatus 3 outputs the received information to the control unit 31 of the portable game apparatus 3.

  In the present embodiment, the communication unit 23 of the information processing device 2 and the communication unit 33 of the portable game device 3 are connected to a network such as a LAN and can communicate with each other. The network may be wired or wireless.

  The display unit 34 of the portable game device 3 is, for example, a liquid crystal display, and displays an image in accordance with an instruction from the control unit 31 of the portable game device 3. In the present embodiment, the display unit 34 of the portable game apparatus 3 is provided in the center of the front surface of the housing 30 of the portable game apparatus 3. Here, the control unit 31 of the portable game apparatus 3 may instruct the display unit 34 of the portable game apparatus 3 together with attributes such as the resolution and aspect ratio of the displayed image. Further, the control unit 31 of the portable game apparatus 3 displays an image displayed on the display unit 34 of the portable game apparatus 3 in accordance with the contents of attribute information indicating attributes such as image resolution and aspect ratio included in the program to be executed. You may indicate the attribute.

  The operation unit 35 of the portable game apparatus 3 is, for example, an operation member (specifically, for example, a cross key or a button), and output data corresponding to an operation on the operation member is output to the control unit of the portable game apparatus 3. 31 is output. The control unit 31 of the portable game apparatus 3 acquires the output data, and executes processing corresponding to the operation of the player on the operation member by using the output data. In the present embodiment, the operation unit 35 of the portable game apparatus 3 is provided on the left and right of the display unit 34 of the portable game apparatus 3. In other words, the operation unit 35 of the portable game apparatus 3 is arranged separately on the front left side and the front right side of the housing 30 of the portable game apparatus 3.

  The imaging unit 36 of the portable game device 3 includes, for example, a known digital camera, and is a black and white, grayscale, or color image (for example, an image in JPEG format) every predetermined time (for example, 1/60 seconds). ) Is generated. Note that the imaging unit 36 of the portable game apparatus 3 may be a standard camera, a telephoto camera, or a wide-angle camera. In the present embodiment, the imaging unit 36 of the portable game device 3 is provided on the upper side of the display unit 34 of the portable game device 3. That is, the imaging unit 36 of the portable game apparatus 3 is provided on the upper side near the front center of the housing 30.

  The distance measuring unit 37 of the portable game device 3 measures the distance between the portable game device 3 and an external object existing outside the portable game device 3. The distance measuring unit 37 of the portable game apparatus 3 is, for example, a distance between the imaging unit 36 of the portable game apparatus 3 and an external object (more specifically, for example, the surface of the external object) (hereinafter referred to as object distance). The distance data 44 indicating the object distance is generated (see FIG. 3). Details of the distance data 44 will be described later.

  In the present embodiment, the distance measuring unit 37 of the portable game apparatus 3 associates each pixel included in the image generated by the imaging unit 36 of the portable game apparatus 3 with the object distance corresponding to each pixel. Distance data 44 is generated. That is, in the present embodiment, there is a one-to-one correspondence between the pixel position in the image generated by the imaging unit 36 of the portable game device 3 and the object distance measured by the distance measurement unit 37 of the portable game device 3. It is attached.

  Specifically, the distance measuring unit 37 of the portable game apparatus 3 is, for example, a stereo camera composed of a plurality of lenses. Based on the parallax of each lens, the distance between the external object and the portable game apparatus 3 is determined. Measure distance. Of course, the distance measuring unit 37 of the portable game apparatus 3 is not limited to a stereo camera, and the distance measuring unit 37 of the portable game apparatus 3 outputs infrared rays, and the infrared rays are reflected back to the imaging target object. You may measure the time until it comes and measure the distance between the external object and the portable game apparatus 3 based on the measured time. In addition, the distance measuring unit 37 of the portable game apparatus 3 includes a plurality of lenses having different focal lengths, and among these lenses, the external object and the portable type are based on a lens that is in focus with respect to the external object. The distance from the game apparatus 3 may be measured. In the present embodiment, the distance measuring unit 37 of the portable game apparatus 3 is formed integrally with the imaging unit 36 of the portable game apparatus 3 and is provided on the upper side of the display unit 34. That is, the distance measuring unit 37 of the portable game apparatus 3 is provided on the upper side near the front center.

  In the present embodiment, the portable game apparatus 3 generates basic data 40 and transmits it to the information processing apparatus 2 (see FIG. 3). FIG. 3 is a diagram illustrating an example of the data structure of the basic data 40. As shown in FIG. 3, the basic data 40 includes a user image 42 and distance data 44.

  In the present embodiment, the imaging unit 36 of the portable game apparatus 3 captures the movement of the subject user 46 every predetermined time (for example, 1/60 seconds), and generates a series of user images 42 (FIGS. 4A to 4). (See FIG. 4E). Here, the subject user 46 is only shown as an example of a moving object. The imaging unit 36 may capture a moving body other than the subject user 46 and generate an image of the moving body.

  In the present embodiment, specifically, for example, the imaging unit 36 images the subject user 46 and sequentially generates n-th user images 42-n (n = 1, 2, 3,...). 4A shows an example of the first user image 42-1, FIG. 4B shows an example of the second user image 42-2, FIG. 4C shows an example of the third user image 42-3, and FIG. 4D shows the fourth user image 42. -4, and FIG. 4E shows an example of the fifth user image 42-5. 4A to 4E, the user image 42 according to the present embodiment displays not only the subject user 46 but also the ground 48. That is, at least one object including a moving body such as the subject user 46 is displayed in the image generated by the imaging unit 36. The distance measuring unit 37 of the portable game apparatus 3 sequentially measures the distance between the subject user 46 and the portable game apparatus 3, and associates the object distance corresponding to each pixel in the nth user image 42-n. N-th distance data 44-n is generated. Then, the control unit 31 of the portable game apparatus 3 generates the nth basic data 40-n including the nth user image 42-n and the nth distance data 44-n corresponding to the user image 42. .

  Then, the communication unit 33 of the portable game apparatus 3 transmits the basic data 40 to the information processing apparatus 2. Then, the communication unit 23 of the information processing device 2 receives the basic data 40.

  Next, functions realized by the information processing apparatus 2 according to the present embodiment will be described. FIG. 5 is a functional block diagram of the information processing apparatus 2 according to the present embodiment. As shown in FIG. 5, the information processing apparatus 2 according to the present embodiment functionally includes a basic data acquisition unit 50, an in-image position specifying unit 52, a distance specifying unit 54, a position coordinate calculating unit 56, and motion data generation. It functions as including part 58. These elements are realized by executing a program installed in the information processing apparatus 2, which is a computer, by the control unit 21 such as a CPU included in the information processing apparatus 2. Note that this program is supplied to the information processing apparatus 2 via a computer-readable information transmission medium such as a CD-ROM or DVD-ROM or via a communication network such as the Internet.

  The basic data acquisition unit 50 acquires a plurality of basic data 40 including images and distance data 44. In the present embodiment, the basic data acquisition unit 50 includes basic data including user images 42 and distance data 44 that are transmitted by the communication unit 33 of the portable game apparatus 3 and received by the communication unit 23 of the information processing apparatus 2. 40 is acquired sequentially.

  The in-image position specifying unit 52 displays an image (this book) in which a predetermined moving body (subject user 46 in this embodiment) is displayed based on an image (user image 42 in this embodiment) included in the basic data 40. In the embodiment, the position in the user image 42) is specified.

  In the present embodiment, the subject user 46 includes a plurality of elements 60 (see FIG. 6). In the present embodiment, each element 60 corresponds to a body part of the subject user 46 such as a head, a torso, a left knee, a right knee, a left foot, a right foot, a left hand, and a right hand. Then, the in-image position specifying unit 52 specifies the position of each element 60 in the user image 42 using, for example, a pattern recognition technique. FIG. 6 is a diagram illustrating an example of the position of each element 60 including the right hand element 60a indicating the right hand and the torso element 60b indicating the torso in the first user image 42-1 specified by the in-image position specifying unit 52. It is.

  Here, it goes without saying that the in-image position specifying unit 52 may specify the position of each element 60 based on the color of each pixel in the user image 42. The in-image position specifying unit 52 may specify the position of each element 60 in the user image 42 using an existing optical motion capture technique or the like.

  The in-image position specifying unit 52 specifies the elements 60 included in the subject user 46 and the positions of the respective elements 60 in the user image 42, which are stored in advance in the storage unit 22 of the information processing device 2. You may make it identify the position coordinate of each element using the specific method data which show a method.

The in-image position specifying unit 52 may specify the position coordinates of a representative point representing each element 60 (for example, the center of gravity of the area where each element 60 is displayed). In the present embodiment, the in-image position specifying unit 52 includes position coordinates (right hand nth image position coordinates) of the center of gravity (right hand center of gravity) of the area in which the right hand element 60a is displayed in the nth user image 42-n. X _n, specifies the _{Y n).} FIG. 6 also shows the right-hand first image position coordinates (X ₁ , Y ₁ ) specified by the image position specifying unit 52. Here, the position coordinates (X _n , Y _n ) in the right hand n-th image are two-dimensional coordinates with the upper left point of the n-th user image 42-n as the origin, the lower direction as the X axis, and the right direction as the Y axis. It is expressed in position coordinates in the system. Of course, the method of expressing the position coordinates in the image is not limited to the above-described method.

The distance specifying unit 54 determines whether the mobile object and the portable game apparatus 3 are based on the position in the image specified by the in-image position specifying unit 52 and the distance data 44 included in the basic data 40 including the image. The distance from the imaging unit 36 is specified. In the present embodiment, the distance specifying unit 54 indicates the object distance associated with the position coordinates of the representative point of each element 60 in the user image 42 indicated by the distance data 44 included in the basic data 40 including the user image 42. Identify. Specifically, for example, the right hand n th-image position coordinates (X _{n, Y n)} is associated to a pixel of the object distance indicated by the n-th distance data 44-n, specified as the right hand n th object distance Z _n To do. Figure 7 shows a diagram illustrating an example of a right hand first object distance Z _1. The distance specifying unit 54 may specify the object distance by a method different from that described above. Specifically, for example, the average value of the object distance associated with each pixel in the region where the right hand element 60a is displayed in the nth user image 42- _n is specified as the right hand nth object distance Zn. May be.

The position coordinate calculation unit 56 calculates the position coordinates of the moving object in the three-dimensional space based on the position in the image specified by the in-image position specifying unit 52 and the distance specified by the distance specifying unit 54. To do. In the present embodiment, the position coordinate calculation unit 56 is specified based on the position coordinates of the representative point of each element 60 in the nth user image 42-n and the distance specifying unit 54 based on the nth distance data 44-n. The nth three-dimensional position coordinate of the representative point is calculated based on the object distance associated with the position coordinate. More specifically, for example, the position coordinate calculation unit 56 determines the right hand n th three-dimensional position coordinates based on the right hand n th image position coordinates (X _n , Y _n ) and the right hand n th object distance Z _n. (X _n , y _n , z _n ) is calculated (see FIG. 8). Note that the position coordinate calculation unit 56 may calculate the fuselage n-th three-dimensional position coordinates (x _n ′, y _n ′, z _n ′) by the same method. In the present embodiment, the three-dimensional position coordinates are expressed by an orthogonal coordinate system, but it is needless to say that the three-dimensional position coordinates may be expressed by another coordinate system such as a polar coordinate system.

The motion data generation unit 58 generates motion data indicating the movement of the moving object in the three-dimensional space based on the position coordinates calculated based on each basic data 40. In the present embodiment, the motion indicating the movement of the subject user 46 in the three-dimensional space based on the three-dimensional position coordinates of the representative point of each element 60 included in the subject user 46 calculated based on each basic data 40. Generate data. More specifically, for example, a series of right-hand first 3D position coordinates (x ₁ , y ₁ , z ₁ ), right-hand second 3D position coordinates (x ₂ , y ₂ , z ₂ ), etc. n three-dimensional position coordinates (x _n , y _n , z _n ) are combined to generate motion data indicating the movement in the right-hand three-dimensional space.

  For example, the motion data generation unit 58 uses the connection relationship data indicating the connection relationship between the elements 60 stored in advance in the storage unit 22 of the information processing apparatus 2 as illustrated in FIG. 46 wire frame model data 62 may be generated. In this way, by combining motion data indicating the movement of each element, for example, motion data indicating how a person walks is generated.

  In the present embodiment, motion data is generated based on a series of three-dimensional position coordinates. Therefore, according to the information processing apparatus 2 according to the present embodiment, it is possible to generate motion data in which the three-dimensional movement of the moving object is simulated.

Next, on the basis of the nth basic data 40-n acquired by the basic data acquisition unit 50 of the information processing apparatus 2 according to the present embodiment, the nth tertiary in the three-dimensional space of the right hand element 60a included in the subject user 46. An example of the flow of processing for calculating the original position coordinates (x _n , y _n , z _n ) will be described with reference to the flowchart shown in FIG.

First, the in-image position specifying unit 52 specifies the right-hand nth in-image position coordinates (X _n , Y _n ) based on the nth user image 42-n included in the nth basic data 40-n (S101). ).

Then, the distance specifying unit 54 specifies the right-hand n-th object distance Z _n indicated in the distance data 44 included in the basic data 40 corresponding to the position coordinates (X _n , Y _n ) in the right-hand n-th image (S102). ).

Then, the position coordinate calculation unit 56 calculates the right hand n th three-dimensional position coordinates (x _n , y) based on the right hand _n th image position coordinates (X _n , Y _n ) and the right hand n th object distance Z _{n. n} , z _n ) is calculated (S103).

Then, the position coordinate calculating unit 56, by the processing shown in S103, the right hand n th three-dimensional position coordinates _{(x n,} y n, _{z n)} checks whether the calculated (S104). If the right-hand nth three-dimensional position coordinates (x _n , y _n , z _n ) have been calculated (S104: Y), this process ends.

Here, for example, the in-image position specifying unit 52 may not be able to specify the right-hand nth in-image position coordinates (X _n , Y _n ) based on the nth user image 42-n. More specifically, for example, as shown in FIG. 4C, since the right hand element 60a is hidden behind the subject user 46, the in-image position specifying unit 52 is based on the nth user image 42-3. There are cases where the position coordinates (X ₃ , Y ₃ ) in the right-hand third image cannot be specified. The distance specifying unit 54, based on the first n distance data 44-n, it may not determine the right hand n th object distance Z _n. In such a case, in the processing shown in S104, based on the first n basic data 40-n, the right hand n th three-dimensional position coordinates _{(x n,} y n, _{z n)} is not calculated.

As described above, when the right hand n-th three-dimensional position coordinates (x _n , y _n , z _n ) are not calculated based on the n-th basic data 40-n in the process shown in S104 (S104: N). For the right-hand element 60a, the position coordinate calculation unit 56 performs a three-dimensional operation based on basic data 40 different from the nth basic data 40-n among the plurality of basic data 40 acquired by the basic data acquisition unit 50. The position coordinates are calculated (S105).

Specifically, for example, the position coordinate calculation unit 56 performs three-dimensional position coordinates (body n-th three-dimensional position coordinates (x _n ′, y _n ′, z _n ′)) of the element 60 (body element 60 b) indicating the body. ) from the right hand n th three-dimensional position coordinates _{(x n,} y n, _{z n)} using vector _{V n} in the n-th three-dimensional position coordinates of the right hand element 60a, which are not calculated by the processing shown in S104 _{(x n} , Y _n , z _n ) may be calculated. FIG. 10 shows an example of a method in which the position coordinate calculation unit 56 calculates the third three-dimensional position coordinates (x ₃ , y ₃ , z ₃ ) using the vector V _n . For example, the position coordinate calculation unit 56 first calculates the right hand second vector V ₂ and the right hand third vector V ₃ based on the angle θ ₁₂ formed by the right hand first vector V ₁ and the right hand second vector V _2. angle theta ₂₃ becomes equal (θ ₁₂ = θ _23), right norm of the second vector V2 | and the right hand third vector _{V 3} norm | | _{V 2 V 3} | is equal _(| V 2 _| = _| V ₃ |), the right hand first vector _{V 1,} the right hand second vector _{V 2,} so that the right hand third vector _{V 3} a vector coplanar, to calculate the value of the right hand third vector _{V 3.} Then, the position coordinate calculation unit 56 calculates the right-hand first based on the vector sum of the position vector of the body third three-dimensional position coordinates (x ₃ ′, y ₃ ′, z ₃ ′) and the right-hand third vector V ₃ . 3 Three-dimensional position coordinates (x ₃ , y ₃ , z ₃ ) are calculated.

Note that the calculation method of the right-hand n-th three-dimensional position coordinate (x _n , y _n , z _n ) in the process shown in S105 is not limited to the above-described method. For example, the position coordinate calculation unit 56 generates a vector (x ₂ −x) from the _first 3D position coordinates (x ₁ , y ₁ , z ₁ ) to the second 3D position coordinates (x ₂ , y ₂ , z ₂ ). ₁ , y ₂ −y ₁ , z ₂ −z ₁ ) and the position vector of the second 3D position coordinates (x ₂ , y ₂ , z ₂ ), the third 3D position coordinates ( x ₃ , y ₃ , z ₃ ) may be calculated ((x ₃ , y ₃ , z ₃ ) = (2x ₂ −x ₁ , ₂ y ₂ −y ₁ , 2z ₂ −z ₁ )) .

In addition, the position coordinate calculation unit 56 is based on, for example, the right hand second 3D position coordinates (x ₂ , y ₂ , z ₂ ) and the right hand 4th 3D position coordinates (x ₄ , y ₄ , z ₄ ). Thus, the right-hand third three-dimensional position coordinates (x ₃ , y ₃ , z ₃ ) may be calculated. More specifically, for example, the middle point between the right-hand second 3D position coordinates (x ₂ , y ₂ , z ₂ ) and the right-hand fourth 3D position coordinates (x ₄ , y ₄ , z ₄ ) It may be calculated as three three-dimensional position coordinates (x ₃ , y ₃ , z ₃ ).

  As described above, when the position coordinate calculation unit 56 does not calculate the position coordinates of the missing element among the plurality of elements included in the moving body based on the nth basic data 40-n (see S104), the position coordinates The calculation unit 56 calculates the position coordinates of the missing element based on basic data 40 different from the nth basic data 40-n among the plurality of basic data 40 acquired by the basic data acquisition unit 50. It does not matter (see S105). Further, the position coordinate calculation unit 56 may calculate the three-dimensional position coordinates of the missing element based on the difference of the three-dimensional position coordinates.

In the processing shown in S104, the right-hand third 3D position coordinates (x ₃ , y ₃ , z ₃ ) and the right-hand fourth 3D position coordinates (x ₄ , y ₄ , z ₄ ) are not calculated. In the process shown in S105, the position coordinate calculation unit 56 determines that the right hand second 3D position coordinates (x ₂ , y ₂ , z ₂ ) and the right hand fifth 3D position coordinates (x ₅ , Y ₅ , z ₅ ), the right hand third 3D position coordinates (x ₃ , y ₃ , z ₃ ) and the right hand fourth 3D position coordinates (x ₄ , y ₄ , z ₄ ) are calculated. It doesn't matter if you do. Thus, for example, when the state where the position coordinates of the missing element are not calculated continues from time t1 to time t2, the position coordinate calculation unit 56 calculates the position coordinates of the missing element last before time t1. It may be calculated based on the position coordinates of the missing element and the position coordinates of the missing element calculated first after time t2.

  If the three-dimensional position coordinates are not specified even in the process shown in S105, the control unit 21 of the information processing device 2 instructs the UI unit 24 of the information processing device 2 such as a speaker to output an alarm sound. Also good. When the portable game device 3 includes a speaker, the control unit 21 of the information processing device 2 outputs an alarm sound to the speaker of the portable game device 3 via the communication unit 23 of the information processing device 2. You may instruct them to do so.

In the above processing example, the processing example for calculating the right-hand nth three-dimensional position coordinate (x _n , y _n , z _n ) has been shown, but the three-dimensional of the elements 60 other than the right-hand element 60a is calculated in the same manner. Of course, the position coordinates can be calculated.

  As described above, in the present embodiment, based on the user image 42 generated by the imaging unit 36 of the portable game apparatus 3 and the distance data 44 generated by the distance measurement unit 37 of the portable game apparatus 3, The three-dimensional position coordinates of the moving body in each frame are calculated. Since the motion data is generated based on the series of three-dimensional position coordinates, it is possible to reduce the labor of the user when generating the motion data by simulating the three-dimensional movement of the moving body.

  Further, in the case of calculating the n-th three-dimensional position coordinate based on the position coordinates calculated from the other basic data 40, the n-th three-dimensional position coordinate is not calculated only from the n-th basic data 40-n. Since the information related to the three-dimensional position coordinates calculated from the other basic data 40 can be utilized, the accuracy of the calculated n-th three-dimensional position coordinates can be further increased.

  In addition, this invention is not limited to the above-mentioned embodiment.

  For example, as shown in FIG. 11, the imaging units 36 of the plurality of portable game devices 3 (the first portable game device 3-1 and the second portable game device 3-2) are synchronized with each other, The moving body such as the subject user 46 is imaged, and the distance measuring unit 37 of the first portable game apparatus 3-1 measures the distance between the first portable game apparatus 3-1 and the moving body such as the subject user 46. The distance measuring unit 37 of the second portable game apparatus 3-2 may measure the distance between the second portable game apparatus 3-2 and a moving body such as the subject user 46. At this time, the position coordinate calculation unit 56 may calculate the three-dimensional position coordinates based on the combination of the basic data 40 transmitted from each portable game apparatus 3.

  Specifically, for example, first, the user image 42 generated by imaging the subject user 46 in advance by the imaging unit 36 of the first portable game apparatus 3-1 is displayed on the first portable game apparatus 3-1. Displayed on the unit 34. Then, the user designates the position of an element 60 (for example, the right hand element 60a) in the user image 42 by the operation unit 35 of the first portable game apparatus 3-1. And the control part 31 of the 1st portable game device 3-1 acquires the data which show the position designated by the user. And the user image 42 produced | generated by imaging the to-be-photographed user 46 previously with the imaging part 36 of the 2nd portable game device 3-2 is displayed on the display part 34 of the 2nd portable game device 3-2. Then, the user designates the position of the element 60 (for example, the right hand element 60a) in the user image 42 by the operation unit 35 of the second portable game apparatus 3-2. And the control part 31 of the 2nd portable game device 3-2 acquires the data which show the position designated by the user. In this way, in each portable game apparatus 3, the positions of the elements 60 corresponding to each other are designated in advance, and the movement of the subject user 46 is captured by the imaging unit 36 of each portable game apparatus 3. A series of user images 42 is generated, and the distance measuring unit 37 of each portable game apparatus 3 generates a series of distance data 44. Each portable game apparatus 3 generates basic data 40 including the user image 42 and the distance data 44 and transmits the basic data 40 to the information processing apparatus 2. Then, the information processing device 2 receives the basic data 40 transmitted from each portable game device 3, and based on the combination of the basic data 40, the position coordinate calculation unit 56 calculates the three-dimensional position coordinates. To do.

  In this way, for example, the element 60 that is not displayed in the user image 42 generated by one portable game apparatus 3 is displayed in the user image 42 generated by the other portable game apparatus 3. Therefore, the number of scenes where the n-th three-dimensional position coordinates are calculated using the basic data 40 different from the n-th basic data 40-n as shown in S105 of the above processing example can be reduced.

  Each portable game apparatus 3 may transmit data indicating the position where the portable game apparatus 3 is arranged together with the basic data 40 to the information processing apparatus 2. In this way, the three-dimensional position coordinates of each element 60 can be calculated based on the data indicating the position of each portable game apparatus 3.

  For example, the information processing system 1 may include a video camera having an imaging function and a distance measurement function instead of the portable game apparatus 3. That is, instead of the portable game apparatus 3, a video camera having an imaging function and a distance measurement function may be used.

  Further, the information processing device 2 and the portable game device 3 described above may be mounted by a single housing.

  Next, an example of using the motion data generated in this way is shown below.

  For example, the generated motion data may be uploaded from the communication unit 23 of the information processing apparatus 2 to a server (not shown) connected via a network such as an internetwork. At that time, the user associates attribute data such as data indicating the game title using the motion data, data indicating the creator of the motion data, and data indicating the CG character generated based on the motion data. You may be able to upload motion data together. In the server, the motion data and the attribute data may be stored in association with each other. The motion data may be stored in the server after being converted into data described in an existing script language.

  The user may be able to view the motion data stored in the server using, for example, a browser. At this time, the motion data may be displayed after being classified into categories. Moreover, motion data may be iconified. Specifically, for example, the motion data may be expressed by a browser as a movement of a wire frame or a character.

  In addition, a user who uploads motion data may set a user to whom the motion data is disclosed, and only the set user can view the motion data.

  Even if the motion data to be viewed is a hyperlink to the network store of motion data, and the user can click on the hyperlink to purchase motion data Good.

  In addition, the motion data newly uploaded to the server may be sequentially downloaded via the network by the information processing device 2 and cached in the storage unit 22 of the information processing device 2. Further, for example, motion data may be stored in the storage unit 22 of the information processing apparatus 2 together with game save data.

  In addition, for example, the information processing device 2 is stored in the server, and motion data indicating that a person is dancing or a person is performing a golf swing, and the portable game device 3 The motion data generated based on a series of user images 42 captured by the image capturing unit 36 may be compared, and the score based on the degree of correspondence of each motion data may be calculated.

It is a hardware block diagram which shows an example of the hardware configuration of the information processing system which concerns on one Embodiment of this invention. It is a perspective view which shows an example of the portable game device which concerns on one Embodiment of this invention. It is a figure which shows an example of the data structure of basic data. It is a figure which shows an example of a 1st user image. It is a figure which shows an example of a 2nd user image. It is a figure which shows an example of a 3rd user image. It is a figure which shows an example of a 4th user image. It is a figure which shows an example of a 5th user image. It is a functional block diagram which shows an example of the function of the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the position of each element in a 1st user image. It is a figure which shows an example of a right hand 1st object distance. It is a figure which shows an example of wire frame model data. It is a figure which shows an example of the flow of the process performed with the information processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows an example of the calculation method of a 3rd three-dimensional position coordinate. It is a figure which shows an example of a mode that a user is imaged by the imaging part of a some portable game device.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Information processing system, 2 Information processing apparatus, 21 Control part, 22 Storage part, 23 Communication part, 24 User interface (UI) part, 3 Portable game device, 30 Case, 31 Control part, 32 Storage part, 33 Communication Unit, 34 display unit, 35 operation unit, 36 imaging unit, 37 distance measurement unit, 40 basic data, 42 user image, 44 distance data, 46 subject user, 48 ground, 50 basic data acquisition unit, 52 in-image position specifying unit , 54 Distance specifying unit, 56 Position coordinate calculating unit, 58 Motion data generating unit, 60 elements, 60a Right hand element, 60b Body element, 62 Wire frame model data.

Claims (7)

An information processing apparatus that generates motion data indicating the movement of a predetermined moving body,
The mobile body includes a plurality of elements in which connection relationships are defined,
A distance indicating the distance between the object displayed in the image and the imaging unit based on an image generated by the imaging unit imaging at least one object including the moving body and a measurement result by the distance measuring unit Basic data acquisition means for acquiring a plurality of basic data including data,
In-image position specifying means for specifying a position in the image where at least some of the elements included in the moving body are displayed based on an image included in each basic data;
Based on the position in the image specified by the position specifying means in the image and the distance data included in the basic data including the image , each of at least some of the elements included in the moving body Distance specifying means for specifying the distance between the element and the imaging means ;
Based on the position in the image specified by the position specifying means in the image and the distance specified by the distance specifying means, a tertiary of at least some of the elements included in the moving body Position coordinate calculation means for calculating position coordinates in the original space ;
Motion data generating means for generating motion data indicating the movement of the movable body in a three-dimensional space based on the position coordinates calculated based on the basic data ,
When the position coordinate calculation means does not calculate the position coordinates of the missing element among the plurality of elements included in the moving body based on the basic data, the position coordinate calculation means calculates based on the basic data. Calculated from the position coordinates of the connection element connected to the missing element and the basic data different from the basic data, and from the position coordinates of one of the missing element and the connection element to the other Calculate the position coordinates of the missing element based on the vector to the position coordinates,
An information processing apparatus characterized by that. When the state where the position coordinate of the missing element is not calculated by the position coordinate calculation unit continues for a predetermined time, the position coordinate calculation unit calculates the position coordinate of the missing element last before the time. Calculate based on the position coordinates of the missing element and the position coordinates of the missing element calculated first after the time,
The information processing apparatus according to claim 1 . When the position coordinate of the missing element is not calculated by the position coordinate calculation means, the position coordinate of the missing element is calculated based on each of the plurality of basic data different from the basic data. Calculate based on the difference,
The information processing apparatus according to claim 1 or 2 . The basic data includes an image generated by each of a plurality of imaging means, distance data indicating a distance between an imaging means that has captured each image and an object displayed in the image, and
The position specifying means in the image specifies a position where the moving body is displayed in each image based on each of the plurality of images included in the basic data,
The distance specifying means specifies a distance between the moving body and each imaging means;
The position coordinate calculating means calculates the position coordinates based on the positions in the plurality of images specified by the in-image position specifying means and the plurality of distances specified by the distance specifying means. ,
The image processing apparatus according to any one of claims 1 to 3, characterized in that. An information processing method for generating motion data indicating movement of a predetermined moving body,
The mobile body includes a plurality of elements in which connection relationships are defined,
A distance indicating the distance between the object displayed in the image and the imaging unit based on an image generated by the imaging unit imaging at least one object including the moving body and a measurement result by the distance measuring unit A basic data acquisition step for acquiring a plurality of basic data including data,
An in-image position specifying step for specifying a position in the image in which at least some of the elements included in the moving body are displayed based on the image included in the basic data;
Based on the position in the image specified by the position specifying step in the image and the distance data included in the basic data including the image, each of at least some of the elements included in the moving body A distance identifying step for identifying a distance between an element and the imaging means;
Based on the position in the image specified by the position specifying step in the image and the distance specified by the distance specifying step, a tertiary of at least some of the elements included in the moving body A position coordinate calculation step for calculating position coordinates in the original space;
A motion data generating step for generating motion data indicating a movement of the movable body in a three-dimensional space based on the position coordinates calculated based on the basic data;
When the position coordinates of the missing element among the plurality of elements included in the moving body are not calculated based on the basic data by the position coordinate calculating step, the missing element calculated based on the basic data and A position coordinate of the connected connection element and a vector from one position coordinate of the missing element and the connection element to the other position coordinate calculated based on basic data different from the basic data; On the basis of the position coordinates of the missing element,
An information processing method comprising: A program that causes a computer to function as an information processing device that generates motion data indicating the movement of a predetermined moving body,
The mobile body includes a plurality of elements in which connection relationships are defined,
A distance indicating the distance between the object displayed in the image and the imaging unit based on an image generated by the imaging unit imaging at least one object including the moving body and a measurement result by the distance measuring unit Basic data acquisition means for acquiring a plurality of basic data including data,
In-image position specifying means for specifying the position in the image where at least some of the elements included in the moving body are displayed based on the image included in each basic data;
Based on the position in the image specified by the position specifying means in the image and the distance data included in the basic data including the image, each of at least some of the elements included in the moving body Distance specifying means for specifying the distance between the element and the imaging means;
Based on the position in the image specified by the position specifying means in the image and the distance specified by the distance specifying means, a tertiary of at least some of the elements included in the moving body Position coordinate calculation means for calculating position coordinates in the original space;
Based on the position coordinates calculated based on the basic data, the computer is caused to function as motion data generating means for generating motion data indicating the movement of the moving body in a three-dimensional space,
When the position coordinate calculation means does not calculate the position coordinates of the missing element among the plurality of elements included in the moving body based on the basic data, the position coordinate calculation means calculates based on the basic data. Calculated from the position coordinates of the connection element connected to the missing element and the basic data different from the basic data, and from the position coordinates of one of the missing element and the connection element to the other Calculate the position coordinates of the missing element based on the vector to the position coordinates,
A program characterized by that. A computer-readable information storage medium storing the program according to claim 6 .

Images