JP2009288917A - Information processor, information processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate an attitude of an object without imposing a burden on the object such as a person. <P>SOLUTION: In this information processor, a silhouette extraction part 32 extracts a silhouette showing an area wherein a photographic subject appears from an image obtained by imaging the photographic subject, a line thinning part 33 thins the silhouette into a silhouette line simply showing the shape of the silhouette, a matching part 36 calculates a matching degree representing a degree of matching between an attitude model line and the silhouette line in each of a plurality of attitude model lines expressing prepared prescribed attitudes different from each other, and an attitude estimation part 37 estimates the attitude shown by the attitude model line having the maximum matching degree among the matching degrees calculated for each of the plurality of attitude model lines as the attitude of the photographic subject. The information processor can be applied to a personal computer, for example. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information processing device, an information processing method, and a program, and particularly, for example, can easily estimate the posture of an estimation target without imposing a burden on the estimation target that is a target whose posture is estimated. The present invention relates to an information processing apparatus, an information processing method, and a program.

  As a gesture recognition technology, for example, "Eagle & Hawk Digital System" (trademark) by Motion Analysis in the United States, "MX Motion Capture" (trademark) by Vicon Peak in the United States, etc., a marker is added to the human body. Alternatively, a person's gesture can be obtained by wearing a glove with a built-in special sensor on a person's hand, capturing the person with a plurality of cameras, and estimating the posture of the person from the plurality of captured images. A technology for recognizing this has been proposed.

  In addition, for example, like a PlayStation (registered trademark of Sony Computer Entertainment Inc.) eye Toy (registered trademark of Sony Computer Entertainment Europe Limited) system, a person is imaged by a single camera, and a captured image obtained by imaging And a motion region extraction technology that extracts a region in a captured image with a person's movement using the difference between the background image in which only the background not including the person is captured and the difference between frames of the captured image is proposed. ing.

  Further, for example, when an incident image is incident on a holographic element in which a plurality of standard posture images are recorded using reference beams having different incident angles, the intensity of light emitted from the holographic element There is also a technique for detecting whether or not an incident image matches any of a plurality of reference posture images or not according to the detected light intensity and direction (for example, , See Patent Document 1).

JP 09-273920 A

  However, in the gesture recognition technology described above, in a very large studio provided with a plurality of cameras, a marker is added to a person's body or a glove is attached to a person's hand, and a person is detected by a plurality of cameras. Since the image must be taken, it is very burdensome for the person.

  Furthermore, the motion region extraction technique does not require any special setting for adding a marker or the like to a person, but is limited to a function of extracting a moving region in a captured image.

  The present invention has been made in view of such circumstances, and makes it possible to easily estimate the posture of an estimation target without imposing a burden on the estimation target such as a person.

  An information processing apparatus or program according to an aspect of the present invention includes an extraction unit that extracts a silhouette representing a region where the subject appears from a captured image obtained by capturing the subject, and simplifies the shape of the silhouette. The degree of coincidence representing the degree of coincidence between the posture model line and the silhouette line for each of a plurality of posture model lines that are prepared in advance and represent predetermined postures different from each other. And a posture represented by the posture model line corresponding to the degree of coincidence having the maximum degree of coincidence among the degree of coincidence calculated for each of the plurality of posture model lines is estimated as the posture of the subject. An information processing apparatus including an estimation unit that performs the above function or a program for causing a computer to function as the information processing apparatus.

  The calculation unit calculates the distance between the posture model line and the silhouette line for each of the plurality of posture model lines as the degree of coincidence, and the estimation unit calculates the distance for each of the plurality of posture model lines. The posture represented by the posture model line corresponding to the shortest distance among the distances can be estimated as the posture of the subject.

  In the estimation unit, when the shortest distance is less than a predetermined threshold, the posture represented by the posture model line corresponding to the shortest distance can be estimated as the posture of the subject.

  The calculation means calculates, for each of the plurality of posture model lines, the degree of coincidence representing a degree of coincidence between a feature amount representing the feature of the posture model line and a feature amount representing the feature of the silhouette line. it can.

  In the information processing method according to one aspect of the present invention, the extraction unit extracts a silhouette representing an area where the subject appears from a captured image obtained by imaging the subject, and the thinning unit extracts the silhouette and the silhouette. For each of a plurality of posture model lines representing predetermined postures that are different from each other, the calculation unit is configured to match the posture model line and the silhouette line. A degree of coincidence representing a degree is calculated, and the estimation unit represents the posture represented by the posture model line corresponding to the degree of coincidence having the largest degree of coincidence among the degree of coincidence calculated for each of the plurality of posture model lines. And an information processing method including a step of estimating the posture of the subject.

  In one aspect of the present invention, a silhouette representing a region where the subject appears is extracted from a captured image obtained by capturing the subject, the silhouette is thinned into a silhouette line that simplifies the shape of the silhouette, The degree of coincidence representing the degree of coincidence between the posture model line and the silhouette line is calculated for each of a plurality of prepared posture model lines representing predetermined different postures, and calculated for each of the plurality of posture model lines. The posture represented by the posture model line corresponding to the degree of coincidence having the highest degree of coincidence among the obtained degrees of coincidence is estimated as the posture of the subject.

  According to the present invention, the posture of the estimation target can be easily estimated without imposing a burden on the estimation target.

  Hereinafter, the present embodiment will be described with reference to the drawings.

  FIG. 1 shows a configuration example of an information processing apparatus to which the present invention is applied.

  The information processing apparatus 1 includes a camera 31, a silhouette extraction unit 32, a thinning unit 33, a distance calculation unit 34, a posture storage unit 35, a matching unit 36, and a posture estimation unit 37.

  The camera 31 captures, for example, a person as a subject, and supplies a captured image obtained by the imaging to the silhouette extraction unit 32.

  The silhouette extraction unit 32 detects (extracts) a silhouette representing a region in which the person appears in the captured image from the captured image from the camera 31, generates a silhouette image that is an image in which the detected silhouette appears, and To the conversion unit 33.

  FIG. 2 shows an example of a silhouette image generated from the captured image by the silhouette extraction unit 32.

  The silhouette image is, for example, a binarized image in which the pixel value of each pixel is binarized to 0 or 1. The silhouette of the person in the silhouette image is indicated by white pixels (white pixels) whose pixel value is 0, and the background in the silhouette image is indicated by black pixels (black pixels) whose pixel value is 1. ing.

  As a detection method for detecting a silhouette of a person in a captured image, for example, a background image that is captured and held in advance and only a background that does not include a person is captured, and a captured image from the camera 31 are used. By using a background subtraction method that takes a difference, a method of detecting a silhouette of a person in a captured image can be employed. In addition, using the method using Graph Cut and stereo vision ("Bi-Layer segmentation of binocular stereo video" V. Kolmogorov, A. Blake et al. Microsoft Research Ltd., Cambridge, UK) The silhouette of the person inside can be detected.

  Returning to FIG. 1, the thinning unit 33 uses the silhouette of the person in the silhouette image supplied from the silhouette extraction unit 32 as a silhouette line in which the shape of the silhouette is simplified, for example, the line width is one pixel width. A thinning process for thinning the centerline of the silhouette (shown in FIG. 3) is performed, and a thinned image obtained by the thinning process is supplied to the distance calculation unit 34.

  Next, FIG. 3 shows an example of a thinned image obtained from a silhouette image by thinning processing performed by the thinning unit 33.

  The thinned image in FIG. 3 is a binarized image, and a silhouette line obtained by simplifying the shape of a person's silhouette is indicated by white pixels, and a background is indicated by black pixels.

  In the thinning process, the silhouette of a person is thinned into a silhouette line in a form that preserves the topological properties (connectivity and Euler number) of the silhouette of the person. That is, in the thinning process, the connecting parts such as the elbows and knees of the person's silhouette disappear, a plurality of different connecting parts are integrated into one, the hole formed by the person's silhouette disappears, The silhouette of a person is thinned into a silhouette line so that it will not be generated.

  In the thinning process, the center line of the silhouette whose line width is one pixel width is used as the silhouette line. In addition, for example, the silhouette outline is detected from the silhouette of the person, and the detected outline is represented. The contour line can be a silhouette line.

  Returning to FIG. 1, the distance calculation unit 34, for each pixel of the thinned image from the thinning unit 33, for example, the Euclidean distance (Euclidean distance) representing the straight line distance between the closest white pixels that form the silhouette line ) And the like, and a distance image obtained by the distance calculation process is supplied to the matching unit 36. The distance image is composed of, for example, a Euclidean distance calculated for each pixel of the thinned image.

  Next, FIG. 4 shows an example of a distance image obtained from the thinned image by the distance calculation processing performed by the distance calculation unit 34.

  4A shows a thinned image, and FIG. 4B shows a distance image in which the Euclidean distance of each pixel of the thinned image is expressed in gray scale.

  In the distance image of FIG. 4B, the darker the gray (closer to black), the shorter the Euclidean distance, and the lighter the gray (closer to white), the longer the Euclidean distance. .

  Returning to FIG. 1, the posture storage unit 35 uses a plurality of feature points (a head, a chest, and each of characteristic points used to generate a posture model line representing a predetermined posture, which are characteristic portions when indicating the posture of a person. The coordinate values of joints etc. are stored.

  Next, FIG. 5 shows an example of a posture model image in which a posture model line generated from the coordinate values stored in the posture storage unit 35 appears.

  The posture model image in FIG. 5 shows posture model lines in which black circles (●) numbered 0 to 14 are connected by line segments.

  Black circles with numbers 0 to 14 indicate coordinate values of a plurality of feature points in the posture model image.

  That is, for example, the number 0 indicates the position of the person's chest, and the number 1 indicates the position of the person's head. Numbers 2 and 3 indicate the positions of the left shoulder and right shoulder of the person, respectively, and numbers 4 and 5 indicate the positions of the left and right elbows of the person, respectively. Furthermore, numbers 6 and 7 indicate the positions of the left hand and right hand of the person, respectively.

  Number 8 indicates the center position of the person's waist, and numbers 9 and 10 indicate the left and right positions of the person's waist, respectively. Numbers 11 and 12 indicate the positions of the left and right knees of the person, and numbers 13 and 14 indicate the positions of the left and right legs of the person, respectively.

  Returning to FIG. 1, the matching unit 36 reads a set of 15 coordinate values (black circles with numbers 0 to 14) stored in the posture storage unit 35, and sets predetermined coordinate values as line segments. The posture model image in which the predetermined posture model line appears is generated by connecting with.

  Further, the matching unit 36 thins the posture model line in the posture model image for each of the plurality of posture model images based on the plurality of generated posture model images and the distance image supplied from the distance calculation unit 34. A distance matching process is performed to calculate a chamfer distance indicating the degree of coincidence with the silhouette line in the image.

  That is, for example, the matching unit 36 superimposes the posture model image and the distance image so that the centroids of the pixels constituting the posture model line and the white pixels constituting the silhouette line are matched, and the posture model line overlaps. An average value of a plurality of Euclidean distances corresponding to positions on the distance image is calculated as a chamfer distance.

  The distance matching process performed by the matching unit 36 is a general technique that is generally used to calculate the degree of matching between edge images including edges.

  Further, when the circumscribed rectangle circumscribing the silhouette line in the thinned image and the circumscribed rectangle circumscribing the posture model line in the posture model image are different in size, the matching unit 36 determines whether the circumscribed rectangle is If one of the circumscribed rectangles is enlarged or reduced so as to match the circumscribed rectangle of the other, the circumscribed rectangle of the same size is used, and then the distance matching process is performed, so that the accuracy is higher. It is possible to calculate the chamfer distance.

  The matching unit 36 supplies the chamfer distance for each of the plurality of posture model images obtained by the distance matching process to the posture estimation unit 37.

  The posture estimation unit 37 is a posture model image corresponding to the chamfer distance having the maximum degree of matching between the silhouette line and the posture model line among the plurality of posture model images, that is, for example, the plurality of chamfer distances from the matching unit 36. Among them, the posture represented by the posture model line in the posture model image corresponding to the shortest chamfer distance is estimated to be the posture of the person imaged by the camera 31, and the estimation result is output.

  Next, details of the thinning process performed by the thinning unit 33 will be described with reference to FIGS.

  The thinning process performed by the thinning unit 33 in FIG. 1 is a process including a first phase, a second phase, a third phase, and a fourth phase. The first phase, the second phase, By performing the processing in the order of the third phase and the fourth phase, the thinned image of FIG. 3 in which the silhouette of the person in the silhouette image of FIG. 2 is converted into a silhouette line is obtained.

  In the first phase, scanning is performed in which the horizontal lines in the silhouette image are scanned from left to right, and scanning is performed from the upper left to the lower right starting from the uppermost left pixel in the silhouette image. In order (raster scanning order) (hereinafter referred to as the scanning order in the lower right direction), white pixels in the silhouette image sequentially supplied from the silhouette extraction unit 32 to the thinning unit 33 are set as the target pixel. Then, when the target pixel satisfies any one of the three conditions described later in FIG. 6, the target pixel that is a white pixel is converted to a black pixel (pixel value is 0 to 1).

  Next, referring to FIG. 6, when a white pixel in a silhouette image sequentially supplied from the silhouette extraction unit 32 to the thinning unit 33 is sequentially selected in the scanning order in the lower right direction, the target pixel Will be described below (hereinafter referred to as the first condition).

  FIGS. 6A to 6C each show 3 × 3 pixels in the horizontal and vertical directions centering on the pixel of interest, and 0, 1 and X indicate the pixel values of the corresponding pixels. X represents a value of 0 or 1. The same applies to other drawings.

  Further, in the following, each pixel adjacent in the left direction, upper left direction, upper direction, upper right direction, right direction, lower right direction, lower direction, and lower left direction of the target pixel is represented by a left pixel, an upper left pixel, and an upper pixel, respectively. , Upper right pixel, right pixel, lower right pixel, lower pixel, and lower left pixel.

  As shown in FIG. 6A, the first condition is that the lower left pixel, lower pixel, and lower right pixel are black pixels (pixel value is 1), and the upper left pixel, upper pixel, and upper right pixel are white pixels (pixel value is 0). ), The target pixel is converted to a black pixel (pixel value is changed from 0 to 1). Note that the left pixel and the right pixel are either white pixels or black pixels.

  As shown in FIG. 6B, as a first condition, if the lower pixel, the lower right pixel, and the right pixel are black pixels, and the left pixel, the upper left pixel, and the upper pixel are white pixels, the target pixel is black Converted to pixels. The lower left pixel and the upper right pixel are either white pixels or black pixels.

  As shown in FIG. 6C, as a first condition, when the condition that the lower right pixel, the right pixel, and the upper right pixel are black pixels and the upper left pixel, the left pixel, and the lower left pixel are white pixels is satisfied, the target pixel is black Converted to pixels. The lower pixel and the upper pixel are either white pixels or black pixels.

  In the thinning process, the second phase is started after the end of the first phase. In the second phase, the horizontal line in the silhouette image is scanned from the right to the left, and the scanning is performed from the lower right to the upper left starting from the lower right pixel in the silhouette image. In the scanning order (reverse raster scanning order) (hereinafter referred to as the scanning order in the upper left direction), white pixels in the silhouette image after the end of the first phase are sequentially set as the target pixel. Then, when the target pixel satisfies any one of the three conditions described later in FIG. 7, the target pixel that is a white pixel is converted into a black pixel.

  Next, referring to FIG. 7, when a white pixel in the silhouette image after the end of the first phase is sequentially set as a target pixel in the order of scanning in the upper left direction, the target pixel is converted into a black pixel. The condition (hereinafter referred to as the second condition) will be described.

  As shown in FIG. 7A, as a second condition, if the upper left pixel, the upper pixel, and the upper right pixel are black pixels, and the lower left pixel, the lower pixel, and the lower right pixel are white pixels, the target pixel is black Converted to pixels. Note that the left pixel and the right pixel are either white pixels or black pixels.

  As shown in FIG. 7B, as a second condition, if the left pixel, the upper left pixel, and the upper pixel are black pixels, and the lower pixel, the lower right pixel, and the right pixel are white pixels, the target pixel is black Converted to pixels. The lower left pixel and the upper right pixel are either white pixels or black pixels.

  As shown in FIG. 7C, as a second condition, if the upper left pixel, the left pixel, and the lower left pixel are black pixels, and the lower right pixel, the right pixel, and the upper right pixel are white pixels, the target pixel is black Converted to pixels. The lower pixel and the upper pixel are either white pixels or black pixels.

  Further, in the thinning process, the third phase is started after the end of the second phase. In the third phase, the scanning order is to scan the horizontal lines in the silhouette image from right to left, and the scanning order is to scan from the upper right to the lower left starting from the upper right pixel in the silhouette image. The white pixel in the silhouette image after the end of the second phase is sequentially set as the target pixel (hereinafter referred to as the scanning order in the lower left direction). Then, when the target pixel satisfies any one of the three conditions described later in FIG. 8, the target pixel which is a white pixel is converted into a black pixel.

  Next, referring to FIG. 8, when a white pixel in the silhouette image after the end of the second phase is sequentially set as a target pixel in the scanning order in the lower left direction, the target pixel is converted into a black pixel. The condition (hereinafter referred to as the third condition) will be described.

  As shown in FIG. 8A, as a third condition, if the upper left pixel, the left pixel, and the lower left pixel are black pixels, and the lower right pixel, the right pixel, and the upper right pixel are white pixels, the target pixel is black Converted to pixels. The lower pixel and the upper pixel are either white pixels or black pixels.

  As shown in FIG. 8B, as a third condition, if the left pixel, the lower left pixel, and the lower pixel are black pixels, and the right pixel, the upper right pixel, and the upper pixel are white pixels, the target pixel is a black pixel. Is converted to Note that the upper left pixel and the lower right pixel are either white pixels or black pixels.

  As shown in FIG. 8C, as a third condition, if the lower left pixel, the lower pixel, and the lower right pixel are black pixels, and the upper left pixel, the upper pixel, and the upper right pixel are white pixels, the target pixel is black Converted to pixels. Note that the left pixel and the right pixel are either white pixels or black pixels.

  In the thinning process, the fourth phase is started after the end of the third phase. In the fourth phase, the scanning order is to scan the horizontal lines in the silhouette image from the left to the right, and the scanning order is to scan from the bottom left to the top right starting from the bottom left pixel in the silhouette image. The white pixels in the silhouette image after the end of the third phase are sequentially set as target pixels (hereinafter referred to as the scanning order in the upper right direction). Then, when the target pixel satisfies any one of the three conditions described later in FIG. 9, the target pixel that is a white pixel is converted into a black pixel.

  Next, referring to FIG. 9, when a white pixel in the silhouette image after the end of the third phase is sequentially set as a target pixel in the order of scanning in the upper right direction, the target pixel is converted into a black pixel. The condition (hereinafter referred to as the fourth condition) will be described.

  As shown in FIG. 9A, as a fourth condition, when the condition that the lower right pixel, the right pixel, and the upper right pixel are black pixels and the upper left pixel, the left pixel, and the lower left pixel are white pixels is satisfied, the target pixel is black Converted to pixels. The lower pixel and the upper pixel are either white pixels or black pixels.

  As shown in FIG. 9B, as a fourth condition, if the right pixel, the upper right pixel, and the upper pixel are black pixels, and the left pixel, the lower left pixel, and the lower pixel are white pixels, the target pixel is a black pixel. Is converted to Note that the upper left pixel and the lower right pixel are either white pixels or black pixels.

  As shown in FIG. 9C, as a fourth condition, when the upper left pixel, the upper pixel, and the upper right pixel are black pixels, and the lower left pixel, the lower pixel, and the lower right pixel are white pixels, the target pixel is black Converted to pixels. Note that the left pixel and the right pixel are either white pixels or black pixels.

  In the thinning process, the processes of the first to fourth phases are performed, so that the silhouette of the person in the silhouette image is thinned into a silhouette line.

  Next, a distance calculation process for calculating the Euclidean distance of each pixel in the thinned image obtained by the thinning process will be described with reference to FIGS.

  The distance calculation process performed by the distance calculation unit 34 includes three processes: an initialization process, a raster scanning process, and a reverse raster scanning process. The processes are performed in the order of the initialization process, the raster scanning process, and the reverse raster scanning process. Is called.

  In the initialization process, each pixel f (i, j) of the thinned image is converted into an initial distance d ′ (i, j) using the following equation (1).

  The pixel f (i, j) is a pixel existing in the i-th row and j-th column in the thinned image when the upper left pixel in the thinned image is the pixel existing in the 0th row and 0th column. Where d ′ (i, j) represents an initial distance obtained by initializing f (i, j).

  After the initial distance d ′ (i, j) corresponding to each pixel f (i, j) of the thinned image is obtained by the initialization process, the raster scanning process is performed.

  Next, in FIG. 10, the initial distance d ′ (i, j) corresponding to each pixel f (i, j) of the thinned image is converted into a temporary distance d ″ (i, j) by raster scanning processing. An example is shown.

  FIG. 10A shows an initial distance d ′ (i, j) corresponding to each pixel f (i, j) of the thinned image. FIG. 10B shows a temporary distance d ″ (i, j) corresponding to the initial distance d ′ (i, j).

  In the raster scanning process, attention is paid to the initial distance sequentially in the order of scanning in the lower right direction, and the initial distance d ′ (i, j) of interest is converted into the temporary distance d ″ (i, j, Convert to j).

  The temporary distance d "(i, j) is the minimum of d '(i, j), d" (i-1, j) +1, and d "(i, j-1) +1. Represents a value.

  After the provisional distance d ″ (i, j) corresponding to the initial distance d ′ (i, j) is obtained by the raster scanning process, the reverse raster scanning process is performed.

  Next, FIG. 11 shows an example when the temporary distance d ″ (i, j) is converted into the Euclidean distance d (i, j) by the reverse raster scanning process.

  FIG. 11A shows the temporary distance d ″ (i, j) corresponding to the initial distance d ′ (i, j). FIG. 11B shows the Euclidean corresponding to the temporary distance d ″ (i, j). The distance d (i, j) is shown.

  In the reverse raster scanning process, attention is paid to the temporary distance sequentially in the order of scanning in the upper left direction, and using the following equation (3), the temporary distance d "(i, j) of interest is converted into the Euclidean distance d (i, In this way, a distance image composed of the Euclidean distance is generated.

  Note that the Euclidean distance d (i, j) represents the minimum value among d "(i, j), d (i + 1, j) +1, and d (i, j + 1) +1. ing.

  Next, the maximum likelihood posture output process performed by the information processing apparatus 1 of FIG. 1 will be described with reference to the flowchart of FIG.

  In step S31, the camera 31 images, for example, a person as a subject, and supplies a captured image obtained by the imaging to the silhouette extracting unit 32.

  In step S <b> 32, the silhouette extraction unit 32 detects (extracts) a silhouette representing a region in which the person appears in the captured image from the captured image from the camera 31, and a silhouette image that is an image in which the detected silhouette of the person appears. Is generated and supplied to the thinning unit 33.

  In step S33, the thinning unit 33 uses the silhouette of the person in the silhouette image supplied from the silhouette extraction unit 32 as a silhouette line with a simplified shape of the silhouette. A thinning process for thinning the center line of the silhouette is performed, and a thinned image obtained by the thinning process is supplied to the distance calculation unit 34.

  In step S34, the distance calculation unit 34, for each pixel of the thinned image from the thinning unit 33, for example, a distance such as a Euclidean distance that represents a straight line distance from the nearest white pixel that forms the silhouette line. A distance calculation process for calculating the distance image is performed, and the distance image obtained by the distance calculation process is supplied to the matching unit 36.

  In step S35, the matching unit 36 reads a plurality of coordinate values stored in the posture storage unit 35, and connects predetermined coordinate values with line segments, whereby a posture model in which a predetermined posture model line appears. Generate an image.

  Further, the matching unit 36 thins the posture model line in the posture model image for each of the plurality of posture model images based on the plurality of generated posture model images and the distance image supplied from the distance calculation unit 34. A distance matching process is performed to calculate a chamfer distance representing the degree of matching with the silhouette line in the image. Then, the chamfer distance for each of the plurality of posture model images obtained by the distance matching process is supplied to the posture estimation unit 37.

  In step S36, the posture estimation unit 37 is a posture model image corresponding to the chamfer distance having the maximum degree of matching between the silhouette line and the posture model line among the plurality of posture model images, that is, for example, from the matching unit 36. Of the plurality of chamfer distances, the posture represented by the posture model line in the posture model image corresponding to the shortest chamfer distance is estimated to be the posture of the person imaged by the camera 31, and the estimation result is output.

  After the posture of the person estimated as described above is output, the maximum likelihood posture output process ends.

  Next, details of the thinning process in step S33 of FIG. 12 will be described with reference to the flowchart of FIG.

  In step S61, the thinning unit 33 sets the variable n incremented (added) by 1 to 1.

  In steps S62 to S65, the thinning unit 33 performs processing in the nth phase of the thinning process.

  That is, in step S62, the thinning unit 33 sequentially processes the silhouette images that are the processing target in the scanning order when performing the processing in the nth phase (for example, the scanning order in the lower right direction in the first phase). The white pixel inside is set as the target pixel.

  In step S63, the thinning unit 33 determines whether or not the target pixel satisfies the nth condition, and if it is determined that the target pixel satisfies the nth condition, the process proceeds to step S64. The target pixel is converted into a black pixel.

  If the thinning unit 33 determines in step S63 that the pixel of interest does not satisfy the nth condition, the process skips step S64 and proceeds to step S65.

  In step S65, the thinning unit 33 determines whether all the white pixels in the silhouette image to be processed are the target pixels, and if it is determined that all the white pixels are not yet the target pixels, the process is performed. The process returns to step S62, and the same processing is repeated thereafter with the white pixel that has not yet been set as the target pixel as a new target pixel.

  In step S65, the thinning unit 33 determines that all the white pixels in the silhouette image to be processed are the target pixels, that is, if the processing of the nth phase is completed, The process proceeds to step S66.

  In step S66, the thinning unit 33 determines whether the variable n is 4, and if it is determined that the variable n is not 4, the process proceeds to step S67. Then, the thinning unit 33 increments the variable n by 1 to make a new variable n, the process returns to step S62, and the same process is repeated thereafter.

  In step S66, if the thinning unit 33 determines that the variable n is 4, that is, if the processing including the first to fourth phases is finished, the thinning processing is finished and the processing is performed. Return to step S33 of FIG.

  Next, details of the distance calculation processing in step S34 in FIG. 12 will be described with reference to the flowchart in FIG.

  In step S91, the distance calculation unit 34 sets each pixel f (i, j) of the thinned image from the thinning unit 33 to the corresponding initial distance d ′ (i, j) using Expression (1). Perform initialization processing to convert. In step S92, the distance calculation unit 34 sets the variables i and j incremented by 1 to zero.

  In step S93 to step S98, the distance calculation unit 34 pays attention to the initial distance sequentially in the order of scanning in the lower right direction, and uses the equation (2) to determine the initial distance d ′ (i, j) of interest. A raster scanning process for converting to the temporary distance d "(i, j) is performed.

  That is, in step S93, the distance calculation unit 34 calculates the minimum value of d ′ (i, j), d ″ (i−1, j) +1, and d ″ (i, j−1) +1. Then, the temporary distance d ″ (i, j) corresponding to the initial distance d ′ (i, j) of interest is determined. In step S94, the distance calculation unit 34 determines that the variable j is the horizontal line of the thinned image. Whether or not the constant jmax represents the number of pixels in the direction −1 (whether or not the pixel corresponding to the initial distance d ′ (i, j) of interest is the pixel present at the right end of the thinned image) judge.

  In step S94, when the distance calculation unit 34 determines that the variable j is not a constant jmax (a pixel corresponding to the initial distance d ′ (i, j) of interest is not a pixel present at the right end of the thinned image). If so, the process proceeds to step S95. Then, the distance calculation unit 34 increments the variable j by 1 to make a new variable j, the process returns to step S93, and the same process is repeated thereafter.

  In step S94, when the distance calculation unit 34 determines that the variable j is a constant jmax (a pixel corresponding to the initial distance d ′ (i, j) of interest exists at the right end of the thinned image). If it is determined that the pixel is a pixel), the process proceeds to step S96.

  In step S96, the distance calculation unit 34 determines whether or not the variable i is a constant imax indicating the number of pixels -1 in the vertical direction of the thinned image (the initial distance d ′ (i, j) of interest). It is determined whether or not the corresponding pixel is a pixel existing at the lower end of the thinned image.

  In step S96, the distance calculation unit 34 determines that the variable i is not a constant imax (the pixel corresponding to the initial distance d ′ (i, j) of interest is not a pixel existing at the lower end of the thinned image). If so, the process proceeds to step S97. The distance calculation unit 34 sets the variable j to 0, and the process proceeds to step S98. The variable i is incremented by 1 to obtain a new variable i. The process returns to step S93. Repeated.

  In step S96, if the distance calculation unit 34 determines that the variable i is a constant imax (a pixel corresponding to the initial distance d ′ (i, j) of interest exists at the lower end of the thinned image). If it is determined that the pixel is a pixel), the process proceeds to step S99.

  In steps S99 to S104, the distance calculation unit 34 pays attention to the temporary distance sequentially in the order of scanning in the upper left direction, and uses the equation (3) to calculate the temporary distance d ″ (i, j) of interest. A reverse raster scanning process for converting the distance to d (i, j) is performed.

  That is, in step S99, the distance calculation unit 34 pays attention to the minimum value among d "(i, j), d (i + 1, j) +1, and d (i, j + 1) +1. The Euclidean distance d (i, j) corresponding to the provisional distance d "(i, j) is determined. In step S100, the distance calculation unit 34 determines whether or not the variable j is 0 (the pixel corresponding to the temporary distance d ″ (i, j) of interest is a pixel existing at the left end of the thinned image). Whether or not there is).

  In step S100, when the distance calculation unit 34 determines that the variable j is not 0 (determined that the pixel corresponding to the temporary distance d "(i, j) of interest is not a pixel existing at the left end of the thinned image. The distance calculation unit 34 increments the variable j by -1 to obtain a new variable j, the process returns to step S99, and the same process is performed thereafter. Repeated.

  In step S100, when the distance calculation unit 34 determines that the variable j is 0 (a pixel corresponding to the focused temporary distance d "(i, j) is a pixel in which a thinned image exists at the left end. The process proceeds to step S102.

  In step S102, the distance calculation unit 34 determines whether or not the variable i is 0 (the pixel corresponding to the temporary distance d ″ (i, j) of interest is a pixel existing at the upper end of the thinned image). Whether or not there is).

  In step S102, when the distance calculation unit 34 determines that the variable i is not 0 (determined that the pixel corresponding to the temporary distance d ″ (i, j) of interest is not a pixel existing at the upper end of the thinned image. The distance calculation unit 34 sets the variable j as a constant jmax, the process proceeds to step S104, increments the variable i by -1, and sets a new variable i. The process returns to step S99, and the same process is repeated thereafter.

  In step S102, the distance calculation unit 34 determines that the variable i is 0 (a pixel corresponding to the focused temporary distance d "(i, j) is present at the upper end of the thinned image. If it is determined that the distance is calculated, the distance calculation process ends, and the process returns to step S34 in FIG.

  As described above, in the maximum likelihood posture output process of FIG. 12, the posture of the person is estimated from the captured image obtained by capturing the person without adding a marker or the like to the person's body. The posture of the person can be estimated without applying.

  Further, in the maximum likelihood posture output process of FIG. 12, a plurality of posture model lines with respect to the direction in which the line extends, the length, the connection relationship between the line segments, and the like based on the plurality of posture model images and the distance image. Since the posture model line and the silhouette line are directly compared each time, for example, more accurate estimation can be performed as compared with the case where the posture is estimated from the feature amount such as the image moment.

  That is, in order to estimate the posture from the feature quantity such as the image moment, it is necessary to associate the feature quantity with the body structure by mapping such as a neural network, but at the stage of associating the feature quantity with the body structure. In addition, ambiguity is involved in the correspondence between the feature quantity and the body structure. For this reason, for example, when the posture is estimated from the feature amount such as the image moment, the accurate posture may not be estimated.

  However, in the maximum likelihood posture output process of FIG. 12, since the silhouette line and the posture model line are directly compared, it is not necessary to associate the feature quantity with the structure of the body. Can be done.

  In the maximum likelihood posture output process of FIG. 12, in step S34 (distance calculation process), the distance calculation unit 34 calculates the Euclidean distance d (i, j) for each pixel f (i, j) of the thinned image. For example, instead of the Euclidean distance d (i, j), a city block distance (Manhattan distance) or a chess board distance may be calculated.

  In this case, in step S35, the matching unit 36 calculates the chamfer distance based on the plurality of posture model images and the distance image formed by the city block distance or the chess board distance from the distance calculation unit 34. A distance matching process is performed, and in step S36, the attitude estimation unit 37 represents the attitude represented by the attitude model line in the attitude model image corresponding to the shortest chamfer distance among the plurality of chamfer distances obtained by the distance matching process. It is estimated that the posture of the person is captured by the camera 31, and the estimation result is output.

  Next, the Euclidean distance, the city block distance, and the chess board distance will be described with reference to FIG.

  As described above, the Euclidean distance represents a linear distance between two pixels. Therefore, for example, in FIG. 15A, among the 3 × 3 pixels in the horizontal × vertical direction, the center pixel (pixel having a pixel value of 1) existing at the center and the Euclidean distance between the center pixel is 0.0, The Euclidean distance between any four pixels adjacent to the central pixel in the vertical and horizontal directions is 1.0. For example, the Euclidean distance between the central pixel and any four pixels adjacent to the central pixel in the oblique direction is 1.41 (= √2).

  The city block distance is the shortest path from a predetermined pixel to another pixel of the two pixels, and the pixel existing between the predetermined pixel and the other pixel is traced vertically and horizontally. It represents the number of pixels -1 present on the current pass.

  Therefore, for example, in FIG. 15B, among the 3 × 3 pixels in the horizontal and vertical directions, the city block distance between the central pixel present at the center and the central pixel is 0, and the central pixel and the central pixel are vertically and horizontally. The city block distance between any four adjacent pixels is 1. For example, the city block distance between the central pixel and any four pixels adjacent to the central pixel in the oblique direction is 2.

  The chessboard distance is the shortest path from a predetermined pixel to another pixel of the two pixels, and the pixel existing between the predetermined pixel and the other pixel is tilted in addition to the vertical and horizontal directions. This represents the number of pixels -1 present on the path when traced in the direction.

  Therefore, for example, in FIG. 15C, among the 3 × 3 pixels in the horizontal × vertical direction, the chessboard distance between the central pixel present at the center and the central pixel is 0, and the central pixel and the central pixel in the vertical and horizontal directions. The chess board distance between any four adjacent pixels is 1. Further, for example, the chessboard distance between the central pixel and any four pixels adjacent to the central pixel in the oblique direction is 1.

  FIG. 16 shows another configuration example of the information processing apparatus 1 that estimates the posture of a person.

  In the figure, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.

  That is, the information processing apparatus 1 in FIG. 16 replaces the distance calculation unit 34, the posture storage unit 35, the matching unit 36, and the posture estimation unit 37 with a feature amount generation unit 61, a posture storage unit 62, a matching unit 63, and The configuration is the same as in the case of FIG. 1 except that the posture estimation unit 64 is provided.

  A thinned image is supplied from the thinning unit 33 to the feature quantity generation unit 61. The feature quantity generation unit 61 uses a shape context (Shape Context) feature quantity (hereinafter referred to as a silhouette line shape) composed of a plurality of histograms representing the features of silhouette lines from the silhouette lines in the thinned image from the thinning unit 33. A feature amount generation process for generating a context feature amount is performed, and the shape context feature amount of the silhouette line obtained as a result is supplied to the matching unit 63.

  The details of the generation method for generating the shape context feature amount will be described later with reference to FIG. A more detailed method for generating shape context feature values is described in, for example, ““ Matching with Shape Contexts ”(IEEE Workshop on Content-based Access of Image and Video Libraries, 2000)”.

  The posture storage unit 62 stores in advance a shape context feature amount (hereinafter referred to as a shape context feature amount of the posture model line) configured by a plurality of histograms representing the features of the posture model line for each of the plurality of posture model lines. ing. Note that the shape context feature amount of the pose model line is generated by the feature amount generation unit 61 using the same method as that for generating the shape context feature amount of the silhouette line, for example.

  The matching unit 63 compares the shape context feature amount of the silhouette line from the feature amount generation unit 61 with the shape context feature amount of the posture model line stored in the posture storage unit 62 for each of a plurality of posture model lines. By performing the feature amount matching processing for calculating the degree of coincidence (shape context feature amount coincidence) indicating the degree of coincidence between the shape context feature amount of the silhouette line and the shape context feature amount of the pose model line, Through the amount matching process, the degree of coincidence of the shape context feature amounts obtained for each of the plurality of posture model lines is supplied to the posture estimation unit 64.

  The posture estimation unit 64 corresponds to the posture model line corresponding to the degree of coincidence of the shape context feature amount having the maximum degree of matching among the plurality of posture model lines, that is, for example, the plurality of shape context feature amounts from the matching unit 63. Among the matching degrees, the posture represented by the posture model line corresponding to the matching degree of the minimum shape context feature amount is estimated to be the posture of the person imaged by the camera 31, and the estimation result is output.

  Next, a generation method in which the feature amount generation unit 61 generates a shape context feature amount (predetermined histogram constituting the shape context) will be described with reference to FIG.

  On the left side of FIG. 17, a substantially fan-like shape formed by being surrounded by a silhouette line, a plurality of concentric circles centering on a predetermined white pixel constituting the silhouette line, and a line segment extending radially from the predetermined white pixel. Multiple areas are shown.

  As described above, as a silhouette line, in addition to the center line of the silhouette shown in FIG. 3, it is possible to adopt a contour line representing the outline of the silhouette. Accordingly, on the left side of FIG. 17, a contour line representing the contour of the silhouette is shown as a silhouette line.

  On the right side of FIG. 17, the horizontal axis is a Bin number indicating each of a plurality of regions, and the vertical axis is a sample point indicating the number of white pixels constituting the silhouette line existing in the region of the corresponding Bin number. A histogram defined by an axis and a vertical axis is shown.

  The feature quantity generation unit 61 sequentially pays attention to the white pixels constituting the silhouette line from the thinning unit 33. Then, a histogram is generated from a plurality of regions as shown on the left side of FIG. 17 that are formed around the white pixel of interest.

  That is, for example, as shown in the left side of FIG. 17, the matching unit 36 has five sample points corresponding to the Bin number indicating the region A and exists in the region B when the number of the regions A is five. When the number to be calculated is 7, the number of sample points corresponding to the Bin number indicating the region B is set to 7 to generate a histogram as shown on the right side of FIG.

  The feature quantity generating unit 61 supplies a plurality of histograms obtained by the number of white pixels constituting the silhouette line to be noticed to the matching unit 63 as shape context feature quantities of the silhouette line.

  The matching unit 63 matches the shape context feature amount of the silhouette line from the feature amount generation unit 61 with the shape context feature amount of the posture model line stored in the posture storage unit 35 for each of a plurality of posture model lines. A feature amount matching process for calculating (shape context feature amount coincidence) is performed, and by the feature amount matching process, the shape context feature amount coincidence obtained for each of a plurality of posture model lines is transmitted to the posture estimation unit 64. Supply.

  That is, for example, the matching unit 63 rasterizes the histograms obtained for the x-th white pixel when the white pixels constituting the silhouette line are arranged in order, for example, in raster scan order, and the pixels constituting the posture model line. The histogram obtained for the x-th pixel when arranged in the scan order is determined. And the number of sample points of the corresponding Bin number between the histogram obtained for the xth white pixel of the white pixels constituting the silhouette line and the histogram obtained for the xth pixel of the pixels constituting the posture model line The sum of absolute differences is calculated as the degree of coincidence between histograms.

  In addition, as an evaluation method for evaluating the degree of coincidence between histograms, for example, various scales such as a chi-square distance, a KL divergence, a virtual distance, and the like can be used.

  In addition, the matching unit 63 calculates the sum of the degrees of coincidence between the histograms calculated for each determined histogram as the degree of coincidence of the shape context feature values, and supplies the same to the posture estimation unit 64.

  The posture estimation unit 64 corresponds to the posture model line corresponding to the degree of coincidence of the shape context feature amount having the maximum degree of matching among the plurality of posture model lines, that is, for example, the plurality of shape context feature amounts from the matching unit 63. Among the matching degrees, the posture represented by the posture model line corresponding to the matching degree of the minimum shape context feature amount is estimated to be the posture of the person imaged by the camera 31, and the estimation result is output.

  Next, with reference to FIG. 18, it will be described that a predetermined histogram constituting the shape context feature amount uniquely represents a feature of a part of a line such as a silhouette line.

  In the upper left side of FIG. 18 and the upper right side of FIG. 18, contour pixels, which are pixels representing the contour, are shown so as to trace the contour of the Roman letter “A”.

  The “A” area 91 on the upper left side of FIG. 18 and the “A” area 92 on the upper right side of FIG. 18 are both configured by a plurality of contour pixels extending from the upper right direction to the lower left direction. Therefore, the region 91 and the region 92 are similar to each other.

  In this case, as shown in the lower side of FIG. 18, it can be seen that the histogram 91 a obtained from the region 91 and the histogram 92 a obtained from the region 92 are similar to each other.

  Also, the area 93 of “A” on the left side of FIG. 18 is an area where there is a line segment composed of a plurality of contour pixels extending from the left direction to the right direction. It is a different area.

  In this case, as shown in the lower side of FIG. 18, it can be seen that the histogram 93a obtained from the region 93 and the histogram 91a obtained from the region 91 (histogram 92a obtained from the region 92) are different.

  As shown in FIG. 18, when the figures (placement of outline pixels) existing in the area are similar, the histograms obtained from the areas are similar, and when the figures existing in the area are not similar The histograms obtained from the regions are not similar. Therefore, the histogram obtained from the region uniquely represents a graphic existing in the region.

  Next, another maximum likelihood posture output process performed by the information processing apparatus 1 in FIG. 16 will be described with reference to the flowchart in FIG. 19.

  In steps S131 to S133, the same processing as in steps S31 to S33 in FIG. 16 is performed.

  In step S134, the feature quantity generation unit 61 uses the shape context feature quantity (the silhouette context feature of the silhouette line) configured from a plurality of histograms representing the characteristics of the silhouette line from the silhouette lines in the thinned image from the thinning section 33. A feature amount generation process for generating (quantity) is performed, and the shape context feature amount of the silhouette line obtained as a result is supplied to the matching unit 63.

  In step S <b> 135, the matching unit 63 determines the shape context feature of the posture line stored in the posture storage unit 62 for each of the plurality of posture model lines and the silhouette line shape context feature amount from the feature amount generation unit 61. A feature amount matching process for calculating a degree of coincidence (a degree of coincidence of shape context feature amounts) indicating a degree of coincidence between the shape context feature amount of the silhouette line and the shape context feature amount of the pose model line by comparing the amount. And the degree of coincidence of the shape context feature values obtained for each of the plurality of posture model lines is supplied to the posture estimation unit 64 by the feature amount matching processing.

  In step S136, the posture estimation unit 64 selects a posture model line corresponding to the degree of coincidence of the shape context feature quantity having the largest degree of matching among the plurality of posture model lines, that is, for example, a plurality of shapes from the matching unit 63. The posture represented by the posture model line corresponding to the degree of coincidence of the minimum shape context feature amount among the degree of coincidence of the context feature amount is estimated to be the posture of the person imaged by the camera 31, and the estimation result is output. To do.

  Next, details of the feature quantity generation processing in step S134 of FIG. 19 will be described with reference to the flowchart of FIG.

  In step S <b> 161, the feature amount generation unit 61 pays attention to a predetermined pixel among white pixels constituting a silhouette line in the thinned image from the thinning unit 33 and sets it as a target pixel.

  In step S162, the feature quantity generation unit 61 sets a plurality of substantially sector-shaped areas as shown on the left side of FIG. 17 corresponding to the target pixel.

  In step S163, the feature quantity generation unit 61 generates a histogram as shown on the right side of FIG. 17 by detecting white pixels included in each area for each of the plurality of areas set in step S162.

  In step S164, the feature quantity generation unit 61 determines whether or not all the white pixels constituting the silhouette line are the target pixels, and if it is determined that all the white pixels are not yet the target pixels, the process proceeds to step S164. Returning to S161, the process proceeds to step S162 using a white pixel that has not yet been set as the target pixel as a new target pixel, and the same processing is repeated thereafter.

  In step S164, when the feature quantity generation unit 61 determines that all the white pixels constituting the silhouette line are the target pixels, in step S163, the feature amount generation unit 61 determines the number of white pixels constituting the silhouette line to be noticed. The processing is returned to step S134 in FIG. 19 using the obtained plurality of histograms as shape context feature quantities of silhouette lines.

  As described above, in the other maximum likelihood posture output processing in FIG. 19, the posture of the person is estimated from the captured image in which the person is captured without adding a marker or the like to the human body. It is possible to estimate the posture of the person without burdening the user.

  In the maximum likelihood posture output process of FIG. 12, for example, in step S <b> 36, the posture estimation unit 37 includes the posture model in the posture model image corresponding to the shortest chamfer distance among the plurality of chamfer distances from the matching unit 36. The posture represented by the line is estimated to be the posture of the person imaged by the camera 31, and the estimation result is output. For example, when the shortest chamfer distance is less than a predetermined threshold, The posture represented by the posture model line in the posture model image corresponding to the shortest chamfer distance is output as an estimation result, and when the shortest chamfer distance is equal to or greater than a predetermined threshold, the posture estimated as the posture of the person is It may be assumed that it does not exist, and that fact may be output.

  In this case, since only the posture represented by the posture model line in the posture model image corresponding to the shortest chamfer distance that is less than the predetermined threshold is output as an estimation result, the posture model image corresponding to the shortest chamfer distance in the posture model image Compared with the case where the posture represented by the posture model line is always output, the accuracy of estimating the posture of the person can be improved. The same applies to step S136 of other maximum likelihood posture output processing in FIG.

  In addition, the matching unit 63 converts a histogram obtained for the x-th white pixel when the white pixels constituting the silhouette line are arranged in the raster scan order and the pixels constituting the posture model line in the raster scan order. By calculating the sum of absolute difference values of the corresponding Bin number sample points with the histogram obtained for the x-th pixel when arranged in order as the degree of coincidence between the histograms, the degree of coincidence of the context feature amount is calculated. Although it was decided to calculate, it is not limited to this.

  That is, for example, the matching unit 63 matches the context feature amount for every combination when the white pixels constituting the silhouette line and the pixels constituting the posture model line are respectively associated one-to-one. Calculate the degree. Then, the matching unit 63 calculates the minimum value of the matching degree of the context feature amount calculated for every combination using a cost minimizing algorithm such as Hungarian Method, and the final context feature amount is calculated. You may make it employ | adopt as a coincidence degree.

  In this case, for example, more appropriate matching of the context feature amount than in the case where the white pixels constituting the silhouette line and the pixels constituting the posture model line are associated one-to-one in the order of raster scan order or the like. The degree can be calculated.

  The silhouette extraction unit 32 of the information processing apparatus 1 in FIG. 1 uses, for example, a background difference method that obtains a difference between a background image in which only a background not including a person is captured and a captured image from the camera 31. Although the person in the image is detected, it is possible to more accurately detect the person in the captured image if the movement of the person in the captured image is also taken into consideration.

  Further, the method for detecting a person in the captured image is not limited to this. That is, for example, the imaging target distance representing the distance from the camera 31 to the imaging target is obtained by irradiating the imaging target such as a person to the camera 31 and measuring the time until the reflected light of the laser is detected. A laser range finder to be calculated is provided, and a person can be detected based on the imaging target distance calculated by the laser range finder.

  Specifically, for example, the silhouette extraction unit 32 determines whether or not the imaging target distance calculated by the laser range finder is less than a subject determination threshold for determining whether the subject is a subject such as a person or the background. It is possible to detect a subject such as a person by using an imaging target that is less than the subject determination threshold as a subject and using an imaging target that is greater than or equal to the subject determination threshold as a background.

  In addition, for example, in place of the laser range finder for calculating the imaging target distance, in addition to the camera 31, another camera different from the camera 31 is provided, and the imaging target distance is determined by parallax between the camera 31 and the other camera. The imaging target distance may be calculated by the calculated stereo process, and a person may be detected based on the calculated imaging target distance.

  Note that the posture storage unit 35 stores the coordinate values of a plurality of feature points, but instead of the coordinate values, for example, a posture model image in which posture lines appear may be stored. Good. In this case, since the matching unit 36 can perform the distance matching process using the posture model image stored in the posture storage unit 35, the posture model line is generated from the coordinate values, and the generated posture model appears. Compared with the case where the distance matching process is performed using the posture model image, the distance matching process can be performed more quickly.

  In the maximum likelihood posture output processing and other maximum likelihood posture output processing, for example, a person is adopted as a subject, but a certain posture such as a monkey, an animal such as an orangutan, and a mannequin is taken. Anything that can be used can be used as a subject.

  As the information processing apparatus to which the present invention is applied, for example, a personal computer can be employed.

  The series of processes described above can be executed by dedicated hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 21 shows a configuration example of a computer that performs the above-described series of processes by executing a program.

  A CPU (Central Processing Unit) 201 executes various processes according to a program stored in a ROM (Read Only Memory) 202 or a storage unit 208. A RAM (Random Access Memory) 203 appropriately stores programs executed by the CPU 201, data, and the like. These CPU 201, ROM 202, and RAM 203 are connected to each other by a bus 204.

  An input / output interface 205 is also connected to the CPU 201 via the bus 204. Connected to the input / output interface 205 are an input unit 206 composed of a keyboard, a mouse, a microphone, and the like, and an output unit 207 composed of a monitor, a speaker, and the like. The CPU 201 executes various processes in response to commands input from the input unit 206. Then, the CPU 201 outputs the processing result to the output unit 207.

  A storage unit 208 connected to the input / output interface 205 includes, for example, a hard disk, and stores programs executed by the CPU 201 and various data. The communication unit 209 communicates with an external device via a network such as the Internet or a local area network.

  The drive 210 connected to the input / output interface 205 drives a removable medium 211 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives programs and data recorded there. Etc. The acquired program and data are transferred to and stored in the storage unit 208 as necessary.

  As shown in FIG. 21, a recording medium that stores a program that is installed in a computer and can be executed by the computer includes a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only Memory). ), DVD (Digital Versatile Disc), a removable medium 211 that is a package medium made of a magneto-optical disk, a semiconductor memory, or the like, or a ROM 202 or a storage unit 208 in which a program is temporarily or permanently stored. It is composed of a hard disk or the like. The program is stored in the recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 209 that is an interface such as a router or a modem as necessary. Is called.

  In the present specification, the step of describing the program stored in the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a block diagram which shows the structural example of the information processing apparatus to which this invention is applied. It is a figure which shows an example of a silhouette image. It is a figure which shows an example of a thinning image. It is a figure which shows an example of the distance image obtained from the thinning image. It is a figure which shows an example of a posture model image. It is a 1st figure explaining the conditions in which an attention pixel is converted into a black pixel. It is a 2nd figure explaining the conditions in which an attention pixel is converted into a black pixel. It is a 3rd figure explaining the conditions by which an attention pixel is converted into a black pixel. It is a 4th figure explaining the conditions in which an attention pixel is converted into a black pixel. It is a figure which shows an example when an initial distance is converted into a temporary distance. It is a figure which shows an example when a temporary distance is converted into a Euclidean distance. It is a flowchart explaining the detail of maximum likelihood attitude | position output processing. It is a flowchart explaining the detail of a thinning process. It is a flowchart explaining the detail of a distance calculation process. It is a figure explaining a Euclidean distance, a city block distance, and a chess board distance. It is a block diagram which shows the other structural example of the information processing apparatus to which this invention is applied. It is a figure explaining the production | generation method of the predetermined | prescribed histogram which comprises a shape context feature-value. It is a figure explaining that the predetermined histogram which comprises the shape context feature-value uniquely represents the characteristic of a part of line. It is a flowchart explaining the detail of another maximum likelihood attitude | position output process. It is a flowchart explaining the detail of a feature-value production | generation process. It is a block diagram which shows the structural example of a computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 Information processing apparatus, 31 Cameras, 32 Silhouette extraction part, 33 Thinning part, 34 Distance calculation part, 35 Posture memory | storage part, 36 Matching part, 37 Posture estimation part, 61 Feature-value production | generation part, 62 Posture memory | storage part, 63 Matching , 64 Posture estimation unit

Claims (6)

Extraction means for extracting a silhouette representing a region where the subject appears from a captured image obtained by imaging the subject;
Thinning means for thinning the silhouette into a silhouette line obtained by simplifying the shape of the silhouette;
Calculating means for calculating a degree of coincidence representing a degree of coincidence between the posture model line and the silhouette line for each of a plurality of posture model lines that are prepared in advance and represent different predetermined postures;
Information comprising: an estimation unit configured to estimate, as the posture of the subject, a posture represented by the posture model line corresponding to a degree of matching having a maximum degree of matching among the degree of matching calculated for each of the plurality of posture model lines. Processing equipment. The calculation means calculates a distance between the posture model line and the silhouette line for each of the plurality of posture model lines as the degree of coincidence,
The information processing apparatus according to claim 1, wherein the estimation unit estimates a posture represented by the posture model line corresponding to the shortest distance among the distances calculated for each of the plurality of posture model lines as the posture of the subject. . The information processing apparatus according to claim 2, wherein when the shortest distance is less than a predetermined threshold, the estimation unit estimates the posture represented by the posture model line corresponding to the shortest distance as the posture of the subject. The calculation means calculates, for each of the plurality of posture model lines, the degree of coincidence representing a degree of coincidence between a feature amount representing the feature of the posture model line and a feature amount representing the feature of the silhouette line. The information processing apparatus according to 1. In the information processing method of the information processing apparatus for estimating the posture of the subject from the captured image obtained by imaging the subject,
The information processing apparatus includes:
Extraction means;
Thinning means;
A calculation means;
An estimation means, and
The extraction means extracts a silhouette representing a region where the subject appears from a captured image obtained by imaging the subject,
The thinning means thins the silhouette into a silhouette line that simplifies the shape of the silhouette,
The calculation means calculates a degree of coincidence representing a degree of coincidence between the posture model line and the silhouette line for each of a plurality of posture model lines representing predetermined postures prepared in advance,
A step of estimating, as the posture of the subject, the posture represented by the posture model line corresponding to the degree of coincidence having the highest degree of coincidence among the degree of coincidence calculated for each of the plurality of posture model lines; An information processing method including: Computer
Extraction means for extracting a silhouette representing a region where the subject appears from a captured image obtained by imaging the subject;
Thinning means for thinning the silhouette into a silhouette line obtained by simplifying the shape of the silhouette;
Calculating means for calculating a degree of coincidence representing a degree of coincidence between the posture model line and the silhouette line for each of a plurality of posture model lines that are prepared in advance and represent different predetermined postures;
Functions as an estimation unit that estimates the posture represented by the posture model line corresponding to the degree of coincidence having the maximum degree of coincidence among the degree of coincidence calculated for each of the plurality of posture model lines as the posture of the subject. Program to let you.

Images