JP2009048305A - Shape analysis program and shape analysis apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To specify a metamere section of an observation object represented by each voxel of volume data generated by a volume intersection method in real time and with high precision. <P>SOLUTION: An input volume data generation unit 50 generates input volume data that is the volume data of the observation object having the same multiple metamere sections as those of a reference object by the volume intersection method every time a frame is obtained by a camera 11 to 1 m. A search unit 60 projects the input volume data generated by the input volume data generation unit 50 onto learning space, and searches a nearest neighbor reference projection point that is a reference point located nearest to a project point of the input volume data. Based on a label attached to the reference volume corresponding to the nearest neighbor reference projection point searched by the search unit 60, it is specified a metamere section to which each voxel of the input volume data belongs. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a shape analysis program and a shape analysis apparatus that can specify in real time which segmental part each voxel of volume data of an observation object composed of a plurality of body segment parts belongs to.

  In recent years, by acquiring information on the posture and movement of the human body, an interface that uses gestures and other inputs as input, human-robot interaction, active observation that controls the camera for shooting according to the position and posture of the observed person, sports -It is expected to realize various systems such as automatic shooting / learning support system for traditional skills. In order to acquire human body movement information in the environment, identification of body parts including clothing is required as elemental technology.

  As a method for posture estimation including loose clothing and its shape analysis, Non-Patent Document 1 describes the shape of each body segment as a parametric spatial distribution instead of a rigid body. A technique for estimating optimum parameters of each body segment shape by using the spatial continuity of the label as a constraint is disclosed.

Non-Patent Document 2 discloses a technique called a SpaceCarving method that checks the identity of the color of corresponding pixels between different viewpoints and restores the shape of a three-dimensional object.
Takefuji Fujita, Yasuhiro Mukakawa, Ken Shaku Naga, “Acquisition and analysis of dance movements with multi-view camera system” Information Processing Research Report, CVIM-2002-132, pp.95.102, 2002. KN Kutulakos, SM Seitz, “A Theory of Shape by Space Carving,” IJCV, Vol.38, No.3, pp.199.218, 2000.

  However, in the method of Non-Patent Document 1, it is difficult to perform real-time processing because the amount of iterative calculation performed to estimate the optimal parameter is enormous, and the shape representation is limited in the parametric model. The segmental shape cannot be calculated with high accuracy. In the method of Non-Patent Document 2, it is possible to represent a three-dimensional object realistically, but since the amount of calculation is enormous, it is practically impossible to perform real-time processing. On the other hand, using the visual volume intersection method that restores a three-dimensional object by photographing the three-dimensional object with a plurality of cameras, the three-dimensional object can be restored in real time. It is difficult to accurately reproduce the body part including the clothes including the recesses.

  An object of the present invention is to provide a shape analysis apparatus and a shape analysis program capable of accurately identifying in real time which segmental part of an observation object each voxel of volume data generated by the visual volume intersection method represents. Is to provide.

  The shape analysis program according to the present invention is a reference that is volume data of the reference object to which a label indicating which body segment belongs to each voxel from a moving image of the reference object composed of a plurality of body segments. Reference volume data generation means for generating volume data for each frame, and a plurality of reference volume data generated for each frame by the reference volume data generation means, a learning space of a predetermined dimension lower than the number of voxels of the reference volume data Learning data generating means for generating, as learning data, a plurality of reference projection point sequences obtained by projecting onto the reference volume, reference volume data generated by the reference volume data generating means, and learning generated by the learning data generating means Learning data storage means for storing data in advance and the same as the reference object A plurality of photographing means for photographing a moving image of an observation object consisting of a number of body segments, and volume data of the observation object using a visual volume intersection method every time a frame is acquired by the plurality of photographing means Input volume data generating means for generating the input volume data, and projecting the input volume data generated by the input volume data generating means onto the learning space, and being positioned closest to the projection point of the input volume data Search means for searching the nearest reference projection point, which is a reference projection point to be searched, in the learning space, and labels attached to each voxel of reference volume data corresponding to the nearest reference projection point searched by the search means Based on this, the computer functions as a specifying means for specifying to which body segment each voxel of the input volume data belongs. And wherein the Rukoto.

  Further, the shape analysis program according to the present invention is volume data of the reference object to which a label indicating which body segment part each voxel belongs is obtained from a moving image of the reference object composed of a plurality of body part parts. A reference volume data generating means for generating a certain reference volume data for each frame, and a plurality of reference volume data generated for each frame by the reference volume data generating means, having a predetermined dimension lower than the number of voxels of the reference volume data; Learning data generating means for generating a plurality of reference projection point sequences obtained by projecting onto a learning space as learning data, reference volume data generated by the reference volume data generating means, and generated by the learning data generating means Learning data storage means for previously storing the learned data, and the reference object A plurality of photographing means for photographing a moving image of the observation object composed of the same plurality of body segments, and a volume of the observation object using a visual volume intersection method every time a frame is acquired by the plurality of photographing means. Input volume data generating means for generating input volume data as data, and projecting the input volume data generated by the input volume data generating means onto the learning space, and being closest to the projection point of the input volume data Search means for searching the nearest reference projection point, which is a reference projection point located in the learning space, and a label attached to each voxel of the reference volume data corresponding to the nearest reference projection point searched by the search means And a specifying means for specifying to which body segment each voxel of the input volume data belongs. To.

  According to these configurations, the reference volume data generated for each frame from the moving image of the observation object is projected onto the learning space, and learning data including a reference projection point sequence that is a projection point is generated in advance. Further, every time a frame in which the observation object is photographed by the photographing means is acquired, input volume data having the same number of voxels as the reference volume data is generated in the virtual three-dimensional space using the visual volume intersection method. The volume data is projected onto the learning space, the nearest reference projection point that is the nearest reference projection point to the projection point of the input volume data is searched, and each of the reference volume data corresponding to the nearest reference projection point is searched. Based on the label representing the part of the body segment attached to the voxel, it is specified to which body segment each voxel of the input volume data belongs.

  Here, because the number of dimensions in the learning space is lower than the number of voxels in the input volume data, when the input volume data is projected onto the learning space, the number of voxels in the input volume data is compressed, resulting in faster search processing for the nearest reference point. Can be done. Further, since the input volume data is generated by the visual volume intersection method, the input volume data can be generated at high speed from the moving image captured by the imaging means. Therefore, it is possible to perform real-time processing for specifying which segmental section each voxel of the input volume data belongs to.

  In addition, since the reference projection point serving as the learning data is a projection of the reference volume data of the reference object composed of the same body segment as the observation object, the reference volume data corresponding to the nearest reference projection point Is similar to the posture of the input volume data, and can accurately specify which segmental part each voxel of the input volume data represents.

  Furthermore, since the body segment in the input volume data is specified using the label attached to the reference volume data with a similar posture, a person wearing loose clothing as the reference object and observation object was adopted. However, the body segment can be identified with high accuracy.

  The learning data storage means stores in advance a reliability for removing false volume data included in the input volume data generated by the input volume data generation means, and the search means stores the reliability. It is preferable that the input volume data is corrected by using and the corrected input volume data is projected onto the learning space.

  According to this configuration, since the input volume data is corrected using the reliability for removing the false volume, the false volume generated in the input volume data can be reduced as long as the visual volume intersection method is employed. Is possible.

  Further, the reference volume data generation means generates the reference volume data by correcting the volume data generated by the visual volume intersection method, and the reference volume data generated by the reference volume data generation means is virtual three-dimensional. Visible volume intersection method from a two-dimensional image obtained by photographing the reference volume data with the plurality of virtual cameras while changing the positional relationship between the plurality of virtual cameras and the reference volume data. The simplified volume data generating means for generating simplified volume data for each positional relationship using the above, and comparing each simplified volume data with the reference volume data, the occurrence location of false volume data in the simplified volume data is determined. As a reliability generation means for identifying and generating the reliability It is preferable to further function computer.

  According to this configuration, since the volume data generated by the visual volume intersection method is corrected and the reference volume data is generated, the reference volume data represents the reference object without including the false volume. Then, the reference volume data is arranged in the virtual three-dimensional space, photographed by a plurality of virtual cameras, and simplified volume data is generated from the obtained two-dimensional image using the view volume intersection method. The simplified volume data for each positional relationship is generated while the positional relationship with the volume data is changed.

  Here, the false volume data represented by the visual volume intersection method appears more or less depending on the positional relationship between the reference object and the plurality of cameras. On the other hand, since the reference volume data is considered to represent the reference object itself, if the reference volume data and the simplified volume data for each positional relationship are compared, many false volumes are generated in each voxel of the simplified volume data. Voxels that are present and those that do not occur much can be identified. Since the reliability is created based on the fake volume specified in this way, it is possible to indicate which voxels of the input volume data are fake volume data.

  Further, the search means divides the learning space into partial areas of a certain size, searches for the nearest reference projection point in a target partial area to which a projection point of the input volume data belongs, and in the target partial area When the reference projection point does not exist, it is preferable to enlarge the size of the partial region so that the reference projection point is detected.

  According to this configuration, since the search range of the nearest reference projection point is within the target partial area to which the projection point of the input volume data belongs, the search range is narrowed and the nearest reference projection point can be searched at high speed. In addition, when the reference projection point does not exist in the target partial area, the size of the partial area is enlarged so that the reference projection point is detected, so that the reference projection point does not exist in the target partial area. be able to.

  Further, the search means detects a reference projection point located closest to the projection point of the input volume data in the target partial region, and based on the distance between the detected reference projection point and the projection point of the input volume data. Also, the nearest reference projection point is detected by specifying one or a plurality of adjacent partial areas having a short distance to the projection point of the input volume data and searching the specified adjacent partial area and the target partial area. It is preferable.

  According to this configuration, it is possible to search for the nearest reference projection point even when the nearest reference projection point is not in the target partial region but in an adjacent partial region.

  Further, the reference volume data generation means is a whole body reference that is volume data of the entire reference object in which voxels are arranged at a predetermined low resolution and a label indicating which body segment part each voxel belongs to is attached. Volume data and a plurality of body segment reference volume data that are volume data of each segment part of the reference object in which voxels are arranged at a higher resolution than the low resolution are generated as the reference volume data, and the learning data The generation unit projects the whole body reference volume data and the plurality of body segment reference volume data generated by the reference volume data generation unit onto each body learning space and each body part learning space. And learning data of the whole body and each body segment part, and the input volume data generation means includes the low resolution. First whole body input volume data which is volume data of the whole observation object in which voxels are arranged, and second whole body input volume data which is volume data of the whole observation object in which voxels are arranged at the high resolution As the input volume data, and the search means projects the first whole body input volume data onto a whole body learning space, whereby the direction of the first whole body input volume data relative to the whole body reference volume data And the second whole body input volume data is rotated in the estimated direction, and the observation is performed from the rotated second whole body input volume data based on the label attached to each voxel of the whole body reference volume data. Generate multiple body segment input volume data that is the volume data of each body segment of the object. Then, by projecting each body segment input volume data onto the corresponding learning space, the nearest reference projection point for each body segment is searched, and the specifying unit is configured to search for each body segment searched by the search unit. It is preferable to identify to which body segment each voxel of the second whole body input volume data belongs based on the body segment reference volume corresponding to the nearest reference projection point.

  According to this configuration, whole body reference volume data in which voxels are arranged at a low resolution over the entire reference object, and a plurality of body segment reference volume data in which voxels are arranged at a high resolution in each body segment of the reference object And the whole body reference volume data and the plurality of body segment reference volume data are projected on the whole body learning space and the learning space of each body segment, which are the learning spaces, and the learning data of the whole body and each body segment Is generated.

  On the other hand, the first whole-body input volume data in which voxels are arranged at a low resolution and the second whole-body input volume data in which voxels are arranged at a high resolution are generated by photographing the observation object with a plurality of photographing means. Then, by projecting the first whole body input volume data onto the whole body learning space, the direction of the first whole body input volume data with respect to the whole body reference volume data is estimated. Here, since the first whole-body input volume data has voxels arranged at a low resolution, the whole-body reference volume data whose posture is similar to that of the first whole-body input volume data, and the first whole-body reference volume data corresponding to the first whole-body reference volume data The direction of the input volume data can be specified at high speed.

  Then, the second whole body input volume data is rotated in the estimated direction, and based on the labels attached to the respective voxels of the whole body reference volume data, each of the observation object is determined from the rotated second whole body input volume data. The body segment input volume data of the body segment is generated, and each body segment input volume data is projected onto the corresponding learning space, whereby the nearest reference projection point for each body segment is searched. Here, each body segment input volume data is generated based on the label attached to the reference volume data from the high-resolution second whole body input volume data rotated in the same direction as the whole body reference volume data. Therefore, the position of each body segment of the observation object is accurately represented.

  Then, based on the label attached to each voxel of the body segment reference volume corresponding to the nearest reference projection point for each searched body segment part, to which body segment each voxel of the second whole body input volume data is located. Since it is specified whether it belongs, it is possible to specify at high speed and high accuracy to which segmental section each voxel of the second whole body input volume data belongs.

  The search means projects the first whole body input volume data onto the whole body learning space each time the first whole body input volume data is rotated by a predetermined angle rotation with reference to a predetermined direction in the virtual three-dimensional space, and the distance becomes the shortest. It is preferable to estimate the direction of the input volume data by searching for a projection point of one whole body input volume data and a reference projection point.

  According to this configuration, the first whole body input volume data is projected onto the learning space each time it is rotated by a predetermined angle with reference to a predetermined direction in the virtual three-dimensional space, and the first whole body input having the shortest distance is projected. Since the projection point of the volume data and the reference projection point are searched, the whole body reference volume data most similar to the first whole body input volume data is specified, and the direction of the input volume data with respect to this whole body reference volume data is specified. can do.

  The learning data generation means and the search means preferably project the reference volume data and the input volume data onto the learning space using principal component analysis.

  According to this configuration, since the reference volume data and the input volume data are projected onto the learning space using the principal component analysis, the number of voxels can be compressed without impairing the original features.

  Further, the reference volume data generation means specifies a body segment part to which each voxel of the reference volume data belongs using a moving image of a reference object with a different color for each body segment part. Is preferred.

  According to this configuration, since the label attached to the reference volume data is specified using the moving image of the reference object with a different color for each body segment, a reference that accurately represents the position of each body segment Volume data can be generated.

  In addition, it is preferable that the reference object and the observation object are persons wearing comfortable clothes.

  According to this structure, the segment part which comprises the person who wore the clothes with a space can be specified in real time and with high precision.

  According to the present invention, since the number of dimensions in the learning space is lower than the number of voxels in the input volume data, when the input volume data is projected onto the learning space, the number of voxels in the input volume data is compressed, resulting in the search for the nearest reference point. Processing can be performed at high speed. Further, since the input volume data is generated by the visual volume intersection method, the input volume data can be generated at high speed from the moving image captured by the imaging means. Therefore, it is possible to realize in real time a process for specifying which segmental section each voxel of the input volume data belongs to.

  In addition, since the reference projection point serving as the learning data is a projection of the reference volume data of the observation target including the same body segment as the reference target, the reference volume data corresponding to the nearest reference projection point Is similar to the posture of the input volume data, and can accurately specify which segmental part each voxel of the input volume data represents.

  Furthermore, since the body segment in the input volume data is specified using the label attached to the reference volume data with a similar posture, a person wearing loose clothing as the reference object and observation object was adopted. However, the body segment can be identified with high accuracy.

  Hereinafter, a shape analysis apparatus according to an embodiment of the present invention will be described. FIG. 1 is a block diagram showing a hardware configuration of a shape analysis apparatus according to an embodiment of the present invention. This shape analysis apparatus is composed of an ordinary computer or the like, and includes an input device 1, a ROM (read only memory) 2, a CPU (central processing unit) 3, a RAM (random access memory) 4, an external storage device 5, and a display device. 6, a recording medium driving device 7, an interface unit (I / F) unit 9, and m (m is an integer of 2 or more) cameras 11 to 1m. The input device 1, ROM 2, CPU 3, RAM 4, external storage device 5, display device 6, recording medium drive device 7, and I / F unit 9 are connected to an internal bus, and various data etc. are input / output through this bus. Various processes are executed under the control of the CPU 3.

  The input device 1 includes a keyboard, a mouse, and the like, and is used by a user to input various data. The ROM 2 stores a system program such as BIOS (Basic Input / Output System). The external storage device 5 includes a hard disk drive and the like, and stores a predetermined OS (Operating System), a shape analysis program, and the like. The CPU 3 reads the OS and the like from the external storage device 5 and controls the operation of each block. In the present embodiment, for example, OPTERON 1.5 GHz can be adopted as the CPU 3. The RAM 4 is used as a work area for the CPU 3.

  The display device 6 is composed of a liquid crystal display device or the like, and displays various images under the control of the CPU 3. The recording medium driving device 7 includes a CD-ROM drive, a flexible disk drive, and the like.

  The shape analysis program is stored in a computer-readable recording medium 8 such as a CD-ROM and distributed to the market. The user installs the shape analysis program in the computer by causing the recording medium driving device 7 to read the recording medium 8. Alternatively, the shape analysis program may be installed in a computer by storing the shape analysis program in a server on the Internet and downloading it from this server.

  The I / F unit 9 is composed of, for example, a USB interface, and performs an input / output interface between the cameras 1 to 1 m and the shape analysis apparatus. The cameras 1 to 1m are composed of cameras that capture moving images at a predetermined frame rate, and are synchronized so that frames are captured at the same time.

  FIG. 2 is a block diagram showing a functional configuration of the shape analysis apparatus shown in FIG. As shown in FIG. 2, the shape analysis apparatus includes m cameras 11 to 1m (an example of an imaging unit), a reference volume data generation unit 20 (an example of a reference volume data generation unit), and a simplified volume data generation unit 30 (an simplified volume). An example of a data generation unit), a learning data generation unit 40 (an example of learning data generation unit), an input volume data generation unit 50 (an example of input volume data generation unit), a search unit 60 (an example of search unit), and a specification unit 70 (An example of specifying means), a reliability generation unit 80 (an example of reliability generation means), and a learning data storage unit 90 (an example of learning data storage means). The reference volume data generation units 20 to 90 shown in FIG. 2 are realized by the CPU 3 executing the shape analysis program. However, the present invention is not limited to this, and each block may be configured by a dedicated hardware device. Good.

  The cameras 11 to 1m are arranged at predetermined locations in a three-dimensional real space, and shoot moving images used when the reference volume data generation unit 20 and the input volume data generation unit 50 generate volume data.

  The reference volume data generation unit 20 is a reference volume that is volume data of a reference object with a label indicating which body segment each voxel belongs to from a moving image of the reference object composed of a plurality of body segments. Data is generated for each frame.

  Specifically, the reference volume data generation unit 20 generates volume data from a moving image of a reference object photographed by the cameras 11 to 1m by a visual volume intersection method, and the generated volume data is transformed into an elastic mesh or a space. The reference volume data is generated by correcting using a method such as carving (Space Carving). The visual volume intersection method is described in “GKM Cheung, T. Kanade, J.-Y. Bouguet, M. Holler,“ A real time system for robust 3D voxel reconstruction of humanmotions, ”CVPR2000, Vol.2, pp. 714.720, 2000. "or" X. Wu, O. Takizawa, and T. Matsuyama, "Parallel Pipeline Volume Intersection for Real-Time 3D Shape Reconstruction on a PC Cluster," in Proc. Of The 4th IEEE International Conference on Computer Vision Systems (ICVS), 2006. ”can be employed. For elastic mesh deformation, the method described in “S. Nobuhara and T. Matsuyama,“ Deformable Mesh Model for Complex Multi-Object 3D Motion Estimation from Multi-Viewpoint Video, ”3DPVT, 2006.” should be adopted. Can do.

  For space carving, the method described in “KN Kutulakos, SM Seitz,“ A Theory of Shape by Space Carving, ”IJCV, Vol.38, No.3, pp.199.218, 2000.” should be adopted. Can do.

  The reference volume data includes whole body reference volume data, which is volume data of the entire reference object, in which voxels are arranged at a predetermined low resolution, and a label indicating which body segment part each voxel belongs to, A plurality of body segment reference volume data that is volume data of each body segment of a reference object in which voxels are arranged at a resolution higher than the resolution is included.

  As a reference object, a person is adopted, and as a body segment part, a person's head, torso, upper right arm, right lower arm, left upper arm, left lower arm, upper right leg, right lower leg, Ten parts of the upper left leg and the lower left leg are employed. Therefore, each voxel of the whole body reference volume data is labeled with a body segment to which the self belongs, among the 10 body segments. In addition, the said body segment part is an example, 11 or more body segment parts may be employ | adopted, a different body segment part may be employ | adopted, and any body segment among said body segment parts. Part may be omitted.

  FIG. 3 is a diagram showing volume data employed in the present embodiment, where (a) shows whole body reference volume data and (b) shows body segment reference volume data. As shown in FIG. 3A, the whole-body reference volume data includes a plurality of voxels BS1 that divide the inside of a rectangular parallelepiped bounding box B1 arranged so as to surround the entire reference object K1 in a virtual three-dimensional space. Represented by a set of For example, a code “1” is assigned to the voxel BS1 located in the reference object K1, and a code “0” is assigned to the voxel BS1 in which the reference object K1 is not located. Each voxel BS1 has a label indicating the head attached to the voxel BS1 located at the head, and a label indicating the upper right arm attached to the voxel BS1 located at the upper right arm. A label indicating the body segment to which is attached.

  The somite reference volume data shown in FIG. 3 (b) is a grid pattern in 10 bounding boxes B2 arranged so as to surround each of the 10 somite parts of the reference object K1 in the virtual three-dimensional space. It consists of a plurality of voxels BS2 to be partitioned. Similarly to voxel BS1, voxel BS2 is assigned a code of “1” to voxel BS2 located in the body segment, and a code of “0” is assigned to voxel BS2 not located in the body segment. Here, since the whole body reference volume data has a low resolution and the body segment reference volume data has a high resolution, the size of the voxel BS1 is larger than the size of the voxel BS2.

  Returning to FIG. 2, the simplified volume data generation unit 30 arranges the whole body reference volume data generated by the reference volume data generation unit 20 in the virtual three-dimensional space, and determines the positional relationship between the plurality of virtual cameras and the whole body reference volume data. Simplified volume data for each positional relationship is generated by restoring the shape using a visual volume intersection method from a plurality of silhouette images obtained by photographing whole body reference volume data with a plurality of virtual cameras while changing.

  The reliability generation unit 80 compares the simplified volume data for each positional relationship with the whole-body reference volume data, identifies the occurrence location of the false volume in the simplified volume data, and determines the false volume included in the input volume data. A reliability attached to each voxel of the whole-body reference volume data for removal is generated. Here, the reliability is data that probabilistically represents the likelihood of generating a false volume in each voxel of the input volume data generated by the input volume data generation unit 50. For example, in a voxel having a whole body reference volume data of a frame, when the positional relationship between the virtual camera and the whole body reference volume data is changed 10 times, if a false volume occurs twice, the reliability of the voxel is 0. .2.

  The place where the false volume is generated varies greatly depending not only on the target shape but also on the positional relationship with the photographing camera group. Therefore, by comparing the simplified volume data for each positional relationship with the whole-body reference volume data, it is possible to specify the location where the false volume is generated.

  The learning data generation unit 40 projects the plurality of reference volume data generated for each frame by the reference volume data generation unit 20 onto a learning space having a predetermined dimension lower than the number of voxels of the reference volume data, for example, using principal component analysis. A plurality of reference projection point sequences obtained as a result is generated as learning data. Instead of principal component analysis, “ND Lawrence,“ Probabilistic non-linear principal component analysis with Gaussian process latent variable models, ”in Journal of Machine Learning Research, Vol.6, pp.1783. 1816, 2005.” A method using a non-linear mapping using the implicit variable shown in FIG.

  In the present embodiment, since the whole body reference volume data and the body segment reference volume data are included as the reference volume data, the learning space is a whole body that is a learning space of a predetermined dimension lower than the number of voxels of the whole body reference volume data. There exists a learning space and a somite learning space which is a learning space of a predetermined dimension lower than the number of voxels of each somite reference volume data. In this embodiment, since there are 10 types of body segment parts, there are 10 individual section learning spaces.

  The learning data generation unit 40 generates a plurality of reference projection point sequences obtained by projecting the plurality of whole body reference volume data generated for each frame on the whole body learning space as whole body learning data, and for each frame. A plurality of reference projection point sequences obtained by projecting the plurality of body segment reference volume data generated in the above into the corresponding body segment learning space are generated as body segment learning data.

  The learning data storage unit 90 stores the reliability generated by the reliability generation unit 80 and the learning data generated by the learning data generation unit 40 in advance. Note that the processing by the reference volume data generation unit 20 to the learning data generation unit 40 and the reliability generation unit 80 is executed in advance as preprocessing of processing executed in real time by the input volume data generation unit 50 to identification unit 70 described later. The

  The input volume data generation unit 50 is input that is volume data of an observation target having a plurality of body segments similar to the reference target using the visual volume intersection method every time frames are acquired by the cameras 11 to 1m. Generate volume data. The input volume data includes the first whole body input volume data that is the volume data of the entire observation object in which the voxels are arranged at the same low resolution as the whole body reference volume data, and the same high resolution as the body segment reference volume data. 2nd whole body input volume data which is the volume data of the whole observation object arranged is included. Accordingly, the first whole body input volume data is data represented by the voxel BS1 similar to the whole body reference volume data shown in FIG. 3A, and the second whole body input volume data is shown in FIG. It is represented by data in which the voxel BS1 of the whole body reference volume data is the voxel BS2.

  The search unit 60 projects the input volume data generated by the input volume data generation unit 50 onto the learning space, and determines a nearest reference projection point that is a reference projection point located closest to the projection point of the input volume data. Search within the learning space.

  Specifically, the search unit 60 estimates the direction of the first whole body input volume data with respect to the whole body reference volume data by projecting the first whole body input volume data onto the whole body learning space, and sets the first whole body input volume data in the estimated direction. The whole body input volume data of 2 is rotated, and the positions of the ten body segments in the rotated second whole body input volume data are specified based on the labels attached to the voxels of the whole body reference volume data. By setting 10 bounding boxes surrounding each of the body segment parts, 10 body segment input volume data are generated, and 10 body segment input volume data are projected onto the corresponding learning space. The nearest reference projection point for 10 body segment parts is searched.

  The specifying unit 70 determines which voxel of the second whole body input volume data is based on the label attached to the body segment reference volume corresponding to the nearest reference projection point for each body segment searched by the search unit 60. Identify whether it belongs to the body segment.

  Next, learning data generation processing in the shape analysis apparatus will be described. FIG. 4 is a flowchart showing learning data generation processing. First, in step S1, the cameras 11 to 1m capture a moving image of the reference object. FIG. 5 is a diagram illustrating a process in which whole body reference volume data is generated. As shown in FIG. 5A, the cameras 11 to 1 m photograph a person who wears a garment (for example, a yukata) with a space that is given a different color for each of ten body segment parts to be specified.

  Next, in step S2, the reference volume data generation unit 20 generates three-dimensional volume data using the visual volume intersection method from m frames at the same time taken by the cameras 11 to 1m, Whole volume reference volume data is generated by applying elastic mesh deformation or space carving to the generated volume data. Thereby, the shape of the whole body reference volume data is approximated to the shape of the actual reference object shown in FIG. 5B, and the reference object can be reproduced realistically.

  Next, in step S3, the reference volume data generation unit 20 attaches a label (body segment label) indicating the body segment to which each voxel belongs to each voxel of the whole body reference volume data generated in step S2. Specifically, the reference volume data generation unit 20 attaches a somatic label as follows. First, color detection processing is executed on m frames taken at the same time taken by the cameras 11 to 1m, and a multi-view label image labeled for each body segment as shown in FIG. Generate. For color detection processing, the method described in “Toshikazu Wada:“ Color target detection using nearest neighbor classifier ”, Information Processing Society of Japan CVIM Journal, Vol. 44, No. SIG17, pp. 126.135, 2003.” Can be adopted.

  Next, by performing back projection of the body segment label on the whole body reference volume data from the multi-viewpoint label image, the surface voxel of the whole body reference volume data is labeled with the body segment label. Next, the surface voxel located nearest to each internal voxel of the whole body reference volume data is searched, and each internal voxel is labeled with the body segment label of the searched surface voxel. As described above, the whole-body reference volume data having a high-precision shape in which the body segment label is attached to each voxel as shown in FIG. 5D is generated.

  Next, in step S4, the reference volume data generation unit 20 uses the whole body reference volume data to which the body segment label is attached, to generate 10 body segment reference volume data representing 10 body segment parts of the reference object. Generate. Here, the reference volume data generation unit 20 arranges the bounding box in which the voxels are arranged at a high resolution so that each body segment part included in the whole-body reference volume data is covered, so that the high resolution body segment reference is performed. Generate volume data.

  In order to perform principal component analysis on the whole body reference volume data and each body segment reference volume data, it is necessary to match the number of voxels (the number of dimensions) of the whole body reference volume data and each body segment reference volume data in each frame. Therefore, the reference volume data generation unit 20 obtains the three-dimensional centroid of the reference object in each frame, and generates a whole body reference volume data by arranging a bounding box of a certain size so as to surround the reference object based on that. At the same time, a three-dimensional center of gravity of each body segment in each frame is obtained, and a bounding box of a certain size is arranged so as to surround each body segment based on that, thereby generating each body segment reference volume. In addition, it is preferable that the size of the bounding box that surrounds the whole body and the bounding box that surrounds each body segment is a size that can surround the reference object and each body segment in all frames. Further, as shown in FIG. 3B, the body segment reference volume data of the other body segment part is included in the bounding box BS2 of a certain body segment part.

Next, in step S5, the learning data generation unit 40 calculates a set V of volume data in which the whole body reference volume data and the body segment reference volume data in all frames are arranged in time series. Here, d-dimensional (d voxel number d) volume data at a certain time t is represented by v _t = [v _{t, 1} , v _{t, 2} ,..., V _{t, d} ] ^T (v _{t, i} ∈ {0, 1}, 0 is a non-target voxel, and 1 is a target voxel). A set V of volume data of all frames is expressed as follows.

V = [v ₁ −m, v ₂ −m,..., V _T −m] (1)
Here, T is the number of volume data used as learning data (that is, the number of captured frames), and m is the average volume of T volume data.

Next, in step S6, the learning data creation section 40, in each of the whole-body reference volume data and somites reference volume data, to calculate the covariance matrix S = VV ^T from the set V, the set of eigenvectors {e i _| i = 1,..., D}. However, e _i is assumed that corresponding eigenvalues λ _i are arranged in descending order. A space based on this eigenvector is an eigenspace. Note that the number of voxels in the whole body reference volume data and each body segment reference volume data is represented by d for convenience of explanation, but actually, it is a different number of voxels.

Next, in step S7, the learning data generation unit 40 approximates the d-dimensional volume data v _t by d ₁ eigenvectors sufficiently smaller than d in each of the whole body reference volume data and the body segment reference volume data. and, _{d l} number of matrix _{E = [e 1, e 2} , ···, e dl] consisting of eigenvectors is used to generate systemic learning space and 10 somites learning space _{d l} dimensions. The number of dimensions of the whole body learning space and each body segment learning space is represented by _dl for convenience of explanation, but in actuality, it may be a different number of dimensions or the same number of dimensions.

Next, in step S8, the learning data generation unit 40 references the whole body reference volume data and each body segment reference volume data to the _dl- dimensional whole body learning space and the corresponding body segment learning space by linear projection of the following equation. Project as a projection point. Incidentally, _{y t} denotes a reference projection point.

y _t = E ^T (v _t −m) (2)
Next, in step S9, the learning data generation unit 40 connects the reference projection points projected onto the whole body learning space and the body segment learning space in time series, and this reference projection point sequence is used to learn the whole body and each body part. Generate as data. That is, by connecting projection points of continuous volume data in time series, a time series shape change can be expressed as a trajectory in the learning space. Through the above processing, the learning data can be expressed as a manifold on the learning space. This manifold is composed of a set of reference projection points {y ₁ ^L ,..., Y _T ^L }. However, y _t ^L represents a reference projection point at time t. Furthermore, a segment label attached to each voxel is recorded for each point in the manifold (whole body reference volume data of a certain frame). For the body segment reference volume data, the body segment label attached to each voxel may be recorded or may not be recorded.

  FIG. 6 is a diagram illustrating an example of a learning space. In FIG. 6, the whole body learning space is shown in three dimensions. As shown in FIG. 6, each whole body reference volume data VD generated for each frame is represented as one point (reference projection point) in the three-dimensional whole body learning space. Each reference projection point is associated with a body segment label attached to each voxel of the corresponding whole-body reference volume data VD. And the learning data generation part 40 produces | generates learning space as shown in FIG. 6 with respect to each body segment part. In the present embodiment, since there are 10 body segment parts, a total of 11 learning spaces including one whole body learning space and 10 body segment learning spaces are generated.

Next, the nearest neighbor reference projection point search process by the search unit 60 will be described. The search unit 60 searches for the nearest reference projection point with respect to the first whole body input volume data and the ten segment input volume data, but in the following description, for convenience of explanation, these volume data are simply used as the input volume. Data. When the input volume data {v t , v t−1 ,..., V t−n } composed of the current frame and the past n frames is projected onto the whole body learning space, the projection point of the input volume data is {y t I, y t-1 I, ···, represented by trajectory pattern consisting of shading point group represented by y t-n I}. However, y t I represents the projection point of the input volume data at time t.

The search unit 60 then obtains y _t ^I = (y _t ^I , y _t−1 ^I ,..., Y _t−n ^I ) that is a trajectory pattern obtained from the input volume data at time t, and the learning data The nearest neighbor reference projection by comparison with the locus pattern y _s ^L = (y _s ^L , y _s−1 ^L ,..., Y _s−n ^L ) (sε {n + 1,..., T}) The reference volume data similar to the input volume data at the point, that is, time t is searched.

  Therefore, as n, which is the number of past histories to be referred to, increases, the search success rate for the same operation improves, but the search cost increases. On the other hand, if n is too large, it is only possible to analyze an input operation that moves exactly the same as the learning data for a long time. Therefore, it is desirable to determine n in consideration of the applicability to a motion consisting of a combination of processing time and a short similar operation according to a task. In this embodiment, n = 5 is adopted. .

Specifically, the whole body learning data y _s ^L satisfying D (t) shown in Expression (3), which is the evaluation value of the search, is the sum of the distances between the points of the trajectory pattern of the input volume data and the whole body learning data. Is determined as the learning data most similar to y _t ^I , that is, the nearest reference projection point.

To describe Equation 3 a n = 5, first, take a _y ^{6 L} _~y ¹ L out of the T _y ^{S _L,} the distance between each of the ^y 6 ^L _~y ¹ L and _y ^{t I} the total sum, _then, taking the ^{_{y 7 L ~y 2 L, s}} = 6 to that process obtains each total sum of the distance between ^y 7 ^L _~y ² L and ^{_{y t I (= n + 1}} ) to ~T It repeats and specifies y _S ^L that minimizes the sum of distances as the nearest reference projection point for y _t ^I.

From the whole body learning data y _s ^L determined in this way, it is possible to refer to highly accurate whole body reference volume data that has been subjected to processing such as volume removal for the concave portion, and also refer to the body segment label corresponding to the shape. Can do.

However, the input volume data includes a fake volume, and there is a possibility that correct learning data cannot be searched due to the influence of the fake volume. Therefore, the search unit 60 is by using the reliability C _t generated by the reliability determining unit 80 reduces the volume of false included in the input volume data. FIG. 7 shows a false volume. A fake volume is generated in the circled area in FIG. According to the volume intersection method, such false volume occurs, by using the reliability C _t, it is possible to reduce the volume of this fake.

Further, since the evaluation is performed on the learning data y _s ^L = (y _s ^L , y _s−1 ^L ,..., Y _s−n ^L ), the processing is performed in proportion to the number of learned volume data. Time increases. Therefore, the search unit 60 reduces the processing time by improving the efficiency of the search process as described below. 8 and 9 are diagrams for explaining the efficiency of the search process. 8 and 9, for convenience of explanation, the learning space is represented in two dimensions, and each point indicated by a square represents learning data.

  In order to speed up the search process, the learning data in the learning space need not be the search target, but only the learning data in the vicinity of the projection point P of the input volume data may be the search target. Therefore, the search unit 60 divides the entire learning space into partial areas at regular intervals in advance, and examines the learning data existing in each partial area. In the search process, first, the partial area including the projection point P of the input volume data is selected as the target partial area CD, and the search process is performed only within the target partial area CD. If the size of the partial area is reduced, the number of data to be searched decreases, but the possibility that there is no learning data in the target partial area CD increases as shown in FIG. Accordingly, the search unit 60 prepares a plurality of partial areas having a larger size as shown in FIG. 8B, and projects the input volume data in order from a set of partial areas having a smaller size, thereby obtaining the target partial area CD. It is possible to include learning data within.

  However, if only the target partial area CD including the projection point P is searched, there is a possibility that the true nearest reference projection point Px exists outside the target partial area CD as shown in FIG. Therefore, as shown in FIG. 9B, the search unit 60 sets the distance di from the projection point P to the projection point P with the distance dp between the projection point P and the nearest reference projection point P1 in the target partial region CD as a threshold value. Adjacent partial regions CD1 to CD3 satisfying di <dp are included in the search range. As described above, the efficient search in which the search range is set hierarchically can achieve both high speed and stability capable of reliably searching for similar solutions.

  Next, the shape analysis process performed by the input volume data generation unit 50 to the identification unit 70 will be described. FIG. 10 is a flowchart showing the shape analysis process. First, in step S31, the cameras 11 to 1m acquire m frames of the observation object at the same time. Next, in step S32, the input volume data generating unit 50 uses the visual volume intersection method from the m frames acquired in step S31, and the low-resolution first whole body input volume data and the high-resolution second volume data. The whole body input volume data is generated.

  Next, in step S33, the first whole body input volume data is projected onto the whole body learning space using Expression (4).

^{_{y t I = E T C t}} (v t -m) (4)
Here, C _t indicates the reliability calculated in advance by the reliability generation unit 80 and is represented by Expression (5).

Here, _{ct, i} represents the reliability of the i-th voxel of the whole-body reference volume data in the frame at time t that can take a value between 0 and 1. As a result, the false volume included in the first whole-body input volume data is reduced, and a highly accurate search process can be realized.

The reliability C _t cannot be given unless the learning data corresponding to the input volume data at time t is known, and has a relationship between chicken and egg. Therefore, in the present embodiment, it is assumed that the difference between the first whole-body input volume data that is continuous in time series is very small, and the voxel reliability used at time t is the learning searched at time t−1. Determined from the data. That is, the reliability associated with the whole-body reference volume data corresponding to the nearest reference projection point at time t−1 is used as the reliability C _{t at} time t.

Next, in step S34, the search unit 60 calculates D ^θ (t) that is the evaluation value shown in Expression (3), and uses y _S ^L that is learning data that satisfies D ^θ (t) as the nearest neighbor reference projection. Search as a point.

In step S35, when the search unit 60 can search for ^{θ ′} that minimizes the evaluation value D ^θ (t) (YES in step S35), the search unit 60 proceeds to step S37 and minimizes D ^θ (t). If θ ′ to be found cannot be searched (NO in step S35), θ is changed by a predetermined angle (step S36). That is, the search unit 60 repeatedly executes the processes of steps S33 to S36 until θ ′ can be searched.

Next, in step S37, whole body reference volume data corresponding to whole body learning data satisfying D ^{θ ′} (t) is specified. Here, the specified whole body reference volume data is the data most similar to the first whole body input volume data. The searched θ ′ represents the direction of the first whole body input volume data with respect to the whole body reference volume data specified in step S37.

  The whole body reference volume data and the first whole body input volume data are voxel sets restored under the world coordinate system. Therefore, even if the reference object and the observation object have the same shape and posture, if the directions are different, they are regarded as completely different data. Therefore, the search unit 60 specifies the orientation of the first whole body input volume data with respect to the whole body reference data.

  In step S38, the search unit 60 rotates the second whole body input volume data by θ ′. As a result, the orientations of the whole body reference volume specified in step S37 and the second whole body input volume data match. In step S39, the search unit 60 identifies and identifies the position of each body segment in the second whole body input volume data according to the body segment label attached to each voxel of the whole body reference volume data identified in step S37. By setting the high-resolution bounding surrounding each body segment part, the second whole body input volume data is divided into each body segment region to generate body segment input volume data.

In step S40, the search unit 60 projects each body segment input volume data on the corresponding body segment learning space. In step S41, the search unit 60 searches for a plurality of learning data in which D ^L (t), which is the evaluation value shown in Expression (3), satisfies D ^L (t) <threshold. In step S42, the specifying unit 70 adds the body segment attached to each voxel of the second whole body input volume from the body segment label of each voxel of the plurality of body segment reference volume data corresponding to the plurality of learning data searched in step S41. Identify the label. The somatic label attached here is data that probabilistically represents the candidate somatic label of each voxel. For example, there are five learning data satisfying D ^L (t) <threshold, and voxels having four body segment reference volume data out of the five body segment reference volume data corresponding to the five learning data. When “1” is set and one segment reference volume data sets this certain voxel to “0”, the segment label of this voxel is 3/4 = 0.75.

  Since each body segment part exists as one lump, there is a high possibility that the labels of a certain voxel and its neighboring voxels are the same. Therefore, the specifying unit 70 calculates the weighted average of the body segment labels with neighboring voxels (the weight is reduced according to the distance from the target voxel) in each voxel, and performs matching as the whole body segment unit. Plan. In addition, the bounding box of the body segment has an overlap, and voxels belonging to the overlapped region are given a plurality of types of body segment labels. Therefore, the specifying unit 70 determines a body segment label having the highest probability in a certain voxel as a body segment label of the voxel.

  In step S43, if the final frame has been reached (YES in step S43), the process ends. If the final frame has not been reached (NO in step S43), the process returns to step S31, and the same applies to the next frame. Processing is executed. Thus, the object of the shape analysis apparatus is achieved.

  Next, an experiment conducted for confirming the effectiveness of the shape analysis apparatus will be described. In this experiment, the subject of observation was the dance movement of a person wearing a kimono with a large shape change. A time-series image set photographed at 30 fps by seven synchronous photographing cameras (Pointgley Flea: 1024 × 768 pixels, 8-bit Bayer pattern) installed on the ceiling so as to surround the observation target was used. All treatments were performed on an Opteron 1.8 GHz PC.

The dance motion included in the learning data was limited to observation data of only one person. The subjects learned the dance by watching the video. Training data is constituted by time-series volume data consisting of 1000 frames, the voxel size of the low-resolution whole-body volume data and high-resolution each body segment volume data was in the world coordinate system and 60 mm ^3, 20 mm ^3, respectively.

The number of dimensions d ₁ of the learning space is determined based on the cumulative contribution rate a (Equation 6) of eigenvalues obtained by principal component analysis.

  In this experiment, the number of dimensions in which the cumulative contribution rate satisfies 75% as the minimum number of dimensions that can maintain a sufficient shape analysis rate was defined as the number of dimensions of each learning space. FIG. 11 shows a change in the cumulative contribution rate according to the number of dimensions.

  As the learning data, the shape analysis was performed using the above-described data. There were two subjects, one of whom was the same person as the learning data. They both wore a kimono with a shape and material similar to the kimono in the learning data, and performed the dance action that they learned from watching the same video as the learning data. However, since the subject (155 cm) and the other subject (175 cm) included in the learning data are greatly different in diagonal, the latter result of the visual volume intersection method is learned by resizing the overall shape according to the height ratio. Projected to eigenspace.

  An example of the shape analysis result is shown in FIG. In FIG. 12, the 1st to 4th tiers indicate an observed image, an input low resolution volume, and an input high resolution volume, respectively. Throughout the entire frame, low-resolution analysis performs a search ignoring fine shapes such as the hand, while high-resolution analysis correctly restores and identifies the body segment to fine shapes ignored in low-resolution analysis. In addition, it can be confirmed that the body segment is stably divided even for a large deformation of the clothes due to the movement of the human body. Further, as shown in FIG. 13, it can be confirmed that the false volume area included in the input volume data is removed. FIG. 14 shows an experimental result in a subject different from the learning data. As shown in FIG. 14, even if compared with the experimental results of the same person as the learning data, there is no significant inferior point. Furthermore, even when a large error is included in the restoration result of the visual volume intersection method due to the failure of the background difference, an example in which a good restoration result is obtained by comparison with accurate learning data has also been confirmed (FIG. 14). Circled area).

  Experiments were also performed on tight clothing, which is the target of normal posture estimation methods. The experiment was performed under the same conditions as the experiment in the kimono shown above except that the number of frames of the learning data is 4000 frames. An example of the shape analysis result is shown in FIG. As in the case of kimono, it can be confirmed that the processing by the shape analysis apparatus is effective.

The execution speed of the shape analysis result was about 0.0042 seconds per frame. This execution speed does not include the time required for three-dimensional restoration. However, “X. Wu, O. Takizawa, and T. Matsuyama,“ Parallel Pipeline Volume Intersection for Real-Time 3D Shape Reconstruction on a PC Cluster, ”in Proc. Of The 4th IEEE International Conference on Computer Vision Systems (ICVS). , 2006., it is shown that 3D shape analysis can be performed using a VGA-size image set and a visual volume intersection method at 5 mm ³ resolution can be performed at a video rate (0.0033 sec / frame). Restoration with sufficient resolution is now possible in real time. Therefore, even if this shape restoration and the total execution time for shape correction and body segment specification by the method of this shape analysis apparatus are combined, an execution speed close to the video rate can be realized. As a result, it can be said that HCL, robot interaction, active camera control, etc. that require onlineness can be realized.

  As described above, according to this shape analysis apparatus, it is possible to simultaneously perform the removal of the restoration error and the identification of each body segment part based on the online shape analysis of an arbitrary human body / clothing. The time series change of the high-precision shape of the target to be obtained is learned in advance with a body segment label, and the shape analysis is performed by comparing this correct answer data with the input data. As a result of this analysis, the object obtained by the visual volume intersection method A large restoration error area is removed from the three-dimensional volume, and an area corresponding to ten defined body segments (excluding non-target labels) can be obtained from the corrected three-dimensional volume.

It is a block diagram which shows the hardware constitutions of the shape analysis apparatus by embodiment of this invention. It is a block diagram which shows the function structure of the shape analysis apparatus shown in FIG. It is the figure which showed the volume data employ | adopted by this Embodiment. It is a flowchart which shows a learning data generation process. It is the figure which showed the process in which whole body reference | standard volume data is produced | generated. It is the figure which showed an example of learning space. It is the figure which showed the false volume. It is a figure explaining efficiency improvement of search processing. It is a figure explaining efficiency improvement of search processing. It is a flowchart which shows a shape analysis process. It is the graph which showed the change of the accumulation contribution rate according to the number of dimensions. It is a figure which shows an experimental result. It is a figure which shows an experimental result. It is a figure which shows an experimental result. It is a figure which shows an experimental result.

Explanation of symbols

11 to 1 m Camera 20 Reference volume data generation unit 30 Simplified volume data generation unit 40 Learning data generation unit 50 Input volume data generation unit 60 Search unit 70 Identification unit 80 Reliability generation unit 90 Learning data storage unit

Claims (11)

Reference volume data, which is volume data of the reference object with a label indicating to which body segment each voxel belongs, is generated for each frame from a moving image of the reference object composed of a plurality of body segments. Reference volume data generation means;
Learning a plurality of reference projection point sequences obtained by projecting a plurality of reference volume data generated for each frame by the reference volume data generation means onto a learning space of a predetermined dimension lower than the number of voxels of the reference volume data Learning data generating means for generating as data,
Learning data storage means for preliminarily storing the reference volume data generated by the reference volume data generation means and the learning data generated by the learning data generation means;
A plurality of photographing means for photographing a moving image of an observation object composed of a plurality of body segments same as the reference object;
Each time a frame is acquired by the plurality of imaging means, input volume data generation means for generating input volume data that is volume data of the observation object using a visual volume intersection method;
The input volume data generated by the input volume data generation means is projected onto the learning space, and the nearest reference projection point, which is the reference projection point located closest to the projection point of the input volume data, is the learning space. Search means for searching within,
Identification means for identifying to which body segment each voxel of the input volume data belongs based on a label attached to each voxel of the reference volume data corresponding to the nearest reference projection point searched by the search means A shape analysis program characterized by causing a computer to function. The learning data storage means stores in advance a reliability for removing false volume data included in the input volume data generated by the input volume data generation means,
The shape analysis program according to claim 1, wherein the search unit corrects the input volume data using the reliability, and projects the corrected input volume data onto the learning space. The reference volume data generation means generates the reference volume data by correcting the volume data generated by the visual volume intersection method,
The reference volume data generated by the reference volume data generation means is arranged in a virtual three-dimensional space, and the reference volume is changed by the plurality of virtual cameras while changing the positional relationship between the plurality of virtual cameras and the reference volume data. Simplified volume data generating means for generating simplified volume data for each of the positional relationships using a view volume intersection method from a two-dimensional image obtained by photographing data;
By comparing each simplified volume data with the reference volume data, the occurrence location of false volume data in the simplified volume data is specified, and the computer is further functioned as a reliability generation means for generating the reliability. The shape analysis program according to claim 2, wherein:   The search means divides the learning space into partial areas of a certain size, searches for the nearest reference projection point in a target partial area to which a projection point of the input volume data belongs, and the reference projection in the target partial area The shape analysis program according to claim 1, wherein when there is no point, the size of the partial region is enlarged so that the reference projection point is detected.   The search means detects a reference projection point located closest to the projection point of the input volume data within the target partial region, and more than the distance between the detected reference projection point and the projection point of the input volume data, Identifying one or more adjacent partial areas with a short distance to the projection point of the input volume data, and detecting the nearest reference projection point by searching in the identified adjacent partial area and the target partial area The shape analysis program according to claim 4, wherein: The reference volume data generation means is a whole-body reference volume data which is volume data of the whole reference object, in which voxels are arranged at a predetermined low resolution, and a label indicating which body segment part each voxel belongs to is attached And a plurality of body segment reference volume data that are volume data of each body segment of the reference object in which voxels are arranged at a resolution higher than the low resolution, and are generated as the reference volume data,
The learning data generation means includes the whole body reference volume data and the plurality of body segment reference volume data generated by the reference volume data generation means in a whole body learning space and a learning space of each body segment part. By projecting, learning data of the whole body and each body segment is generated,
The input volume data generation means includes first whole body input volume data that is volume data of the entire observation object in which voxels are arranged at the low resolution, and the entire observation object in which voxels are arranged at the high resolution. 2nd whole body input volume data which is volume data of
The searching means estimates the first whole body input volume data relative to the whole body reference volume data by projecting the first whole body input volume data onto a whole body learning space, and the first whole body input volume data is estimated in the estimated direction. The whole body input volume data of 2 is rotated, and based on the label attached to each voxel of the whole body reference volume data, the volume data of each body segment of the observation object is obtained from the rotated second whole body input volume data. A plurality of somite input volume data is generated, and each somite input volume data is projected onto a corresponding learning space to search for the nearest reference projection point for each somite part,
The specifying means is based on the body segment reference volume corresponding to the nearest reference projection point for each body segment searched by the search means, to which body segment each voxel of the second whole body input volume data is located. It is specified whether it belongs or not. The shape analysis program according to any one of claims 1 to 5.   The search means projects the first whole body input volume data to the whole body learning space each time the first whole body input volume data is rotated by a predetermined angle with reference to a predetermined direction in the virtual three-dimensional space, and the first distance becomes the shortest. 7. The shape analysis program according to claim 6, wherein the direction of the input volume data is estimated by searching for a projection point of the whole body input volume data and a reference projection point.   The shape according to claim 1, wherein the learning data generation unit and the search unit project the reference volume data and the input volume data onto the learning space using principal component analysis. Analysis program.   The reference volume data generation means specifies a body segment part to which each voxel of the reference volume data belongs using a moving image of a reference object with a different color for each body segment part. The shape analysis program according to claim 1.   The shape analysis program according to any one of claims 1 to 9, wherein the reference object and the observation object are persons wearing loose clothes. Reference volume data, which is volume data of the reference object with a label indicating to which body segment each voxel belongs, is generated for each frame from a moving image of the reference object composed of a plurality of body segments. Reference volume data generation means;
Learning a plurality of reference projection point sequences obtained by projecting a plurality of reference volume data generated for each frame by the reference volume data generation means onto a learning space of a predetermined dimension lower than the number of voxels of the reference volume data Learning data generating means for generating as data,
Learning data storage means for preliminarily storing the reference volume data generated by the reference volume data generation means and the learning data generated by the learning data generation means;
A plurality of photographing means for photographing a moving image of an observation object composed of a plurality of body segments same as the reference object;
Each time a frame is acquired by the plurality of imaging means, input volume data generation means for generating input volume data that is volume data of the observation object using a visual volume intersection method;
The input volume data generated by the input volume data generation means is projected onto the learning space, and the nearest reference projection point, which is the reference projection point located closest to the projection point of the input volume data, is the learning space. Search means for searching within,
Identification means for identifying to which body segment each voxel of the input volume data belongs based on a label attached to each voxel of the reference volume data corresponding to the nearest reference projection point searched by the search means A shape analysis apparatus comprising: