JP2013120497A - Method for extracting silhouette and silhouette extraction system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method for extracting a silhouette capable of extracting the silhouette of a person in each frame of a moving image.SOLUTION: The method for extracting silhouettes is configured to extract a frame from a moving image obtained by photographing a person performing a swing action along with background, identify each pixel to the person or the background, and extract the silhouette of the person. The method includes: a feature point extraction step S102 of performing image processing for an address frame and extracting a plurality of feature points of the person and a golf club; a step S103 of setting a mask region that is the region including the person and the golf club and including the background of the periphery with a small range; a step S104 of setting a human body zone that has a high probability zone of constituting the silhouette of the person, among the pixels included in the mask region; a step S105 of determining a human body section representing the human body in a histogram on the basis of a histogram of the pixels of the human body zone and color information of the pixels of the human zone of the address frame; and a step S106 of determining the human body section in the histogram of the pixels outside the human body zone on the basis of the determined human body section.

Description

  The present invention is a silhouette extraction capable of extracting a person's silhouette by dividing each pixel of a frame extracted from a moving image obtained by photographing a person performing a swing motion with a golf club together with a background into either a person or a background. The present invention relates to a method and a silhouette extraction system.

  In order to improve the skill of golf, it is important to acquire an appropriate golf swing. For this purpose, a video of a golf swing is taken, and a swing diagnosis is made based on this video. In classic swing diagnosis, teaching professionals point out problems during swing based on video. On the other hand, an attempt has been made to diagnose a swing by performing image processing on the moving image.

  In the case of image processing, a pixel where a person is photographed is distinguished from a pixel where a background is photographed, and a silhouette of the person can be extracted. The present applicant has already proposed the following Patent Document 1 for performing such processing with high accuracy.

JP 2011-78069 A

  In the above patent document, although some accuracy is ensured, several problems are pointed out. That is, in the above-mentioned patent document, when the person and the background have similar colors, there is a problem that the two cannot be distinguished well.

  The present invention has been devised in view of the above problems. A silhouette extraction method and a silhouette extraction system that can distinguish a person and a background even if they have similar colors are provided. The main purpose is to provide.

  According to the first aspect of the present invention, a frame, which is an aggregate of pixels, is extracted from a moving image in which a person who performs a swing motion with a golf club is photographed together with a background, and each pixel of the extracted frame is extracted. A method for extracting a silhouette of the person by specifying either the person or the background, the step of creating an entire frame set consisting of all frames for each pixel, and a frequency for the entire frame set Is a number of frames and a class is a color information histogram, extracting an address frame in which the address state of the person is imaged from the moving image, image processing the address frame, the person and A feature point extraction step for extracting a plurality of predetermined feature points for the golf club. And a step of setting a mask region that is a region that includes the person and the golf club and partially includes a background around the person and the golf club, based on the feature point, and on the mask region based on the feature point. A step of setting a human body zone which is a region having a high probability of constituting the silhouette of the person among the included pixels, a histogram of pixels of the human body zone, and color information of pixels of the human body zone of the address frame Based on the human body section determination step for determining a human body section that is a class representing the human body in the histogram, and based on the determined human body section, the human body section is determined in a histogram of pixels outside the human body zone. A human body segment propagation step, a histogram of pixels in a non-mask area outside the mask area, and the histogram. Based on the color information of the pixels of the non-masked area of the less frame, a background section determining step for determining a background section that is a class representing a background in the histogram, and based on the determined background section, And a background interval propagation step for determining a background interval in a histogram of pixels in the mask area.

  In the invention according to claim 2, the histogram includes a first histogram for all pixels and a class as luminance, a second histogram for chromatic pixels as a class and hue, and an achromatic color A third histogram having a luminance level for pixels is included.

  According to a third aspect of the present invention, the feature points represent the top of the human body in the addressed state, the forehead, the back, the waist, the back of the knee, the heel, the toes, the thighs, the hands, and the golf club head and shoulders. It includes a position.

  According to a fourth aspect of the present invention, the step of setting the mask region includes the step of connecting the feature points to set an initial mask region, and expanding the initial mask region to a predetermined thickness to form the mask. An area is set.

  In the invention according to claim 5, the step of setting the human body zone includes a step of setting a reference point inside the initial mask area and a position that is clearly a human body based on the feature point, and the reference point The human body zone is determined using.

  According to a sixth aspect of the present invention, a strut is displayed based on a step of identifying a pixel for displaying a driving range support post, a pixel for displaying the support post, and a histogram thereof. Determining a strut section which is a certain class, and the background section propagation step skips the strut section.

  The invention according to claim 7 further includes a step of determining consistency between the background section and the human body section after the background section and the human body section are determined.

  The invention according to claim 8 extracts a frame, which is an aggregate of pixels, from a moving image in which a person who performs a swing motion with a golf club is photographed together with a background, and each pixel of the extracted frame is defined as the person or A system for extracting the silhouette of the person by specifying any of the backgrounds, comprising a camera for capturing the moving image, a memory for storing the captured moving image, and a calculation unit, The pixel includes a set creation unit that creates a set of all frames including all frames, and a histogram creation unit that creates a histogram in which the frequency is the number of frames and the class is color information for the entire frame set. An address frame extraction unit for extracting an address frame in which the address state of the person is imaged from the moving image; An image processing is performed on the address frame, and a feature point extracting unit that extracts a plurality of predetermined feature points for the person and the golf club. Based on the feature points, the human body and the golf club are included while surrounding the human body and the golf club. A mask region setting unit that sets a mask region that is a region including a background in a small range, and a region that has a high probability of forming the silhouette of the person from the pixels included in the mask region based on the feature points Based on the human body zone setting unit for setting the human body zone, the pixel histogram of the human body zone, and the color information of the pixel of the human body zone of the address frame, a human body section which is a class representing the human body in the histogram. A human body segment determining unit for determining, based on the determined human body segment, a histogram of pixels outside the human body zone; A human body section propagation unit for determining a human body section in a ram, a histogram of pixels in a non-mask area outside the mask area, and color information of pixels in the non-mask area in the address frame. And a determination unit including a background interval determining unit that determines a background interval that is a class representing a background, and a background interval propagating unit that determines a background interval in a histogram of pixels in the mask region based on the determined background interval It is characterized by having.

  The present invention has been made paying attention to the address state during the swing operation. In other words, the address state has a smaller individual difference than the other states of the swing motion, and all of the persons are in almost the same posture. In the present invention, based on such an empirical rule, an address frame, which is a frame in an address state, is subjected to image processing, and feature points at a plurality of predetermined locations for a person and a golf club are extracted. Based on the feature points, a mask region including a person's silhouette and a human body zone which is a region having a higher probability of forming a human body in the mask region are set. Then, based on the histogram of the pixel of the human body zone and the color information of the pixel of the human body zone of the address frame, a human body section which is a class representing the human body is determined in the histogram, and based on this, the human body zone Propagation of the so-called human body section is performed to determine the human body section in the histogram of the outer pixels.

  In the address frame, the pixels in the non-mask area outside the mask area have a high probability of forming the background. Therefore, in the present invention, based on the histogram of the pixels of the non-mask area and the color information of the pixels of the non-mask area of the address frame, a background section that is a class representing the background in the histogram is determined, Based on this, so-called background interval propagation is performed to determine the background interval in the histogram of the pixels in the mask area.

  According to the method of the present invention including such two propagation processes, even if the person and the background have similar colors, the two can be distinguished well. Therefore, highly accurate silhouette extraction is possible.

It is a conceptual diagram of the silhouette extraction system of this embodiment. FIG. 2 is a conceptual diagram illustrating details of a calculation unit of the system of FIG. 1. It is a flowchart of the silhouette extraction method of this embodiment. It is explanatory drawing in which the camera angle was shown. (A) thru | or (c) are the brightness | luminance histograms about a pixel. (A) to (c) are color histograms for pixels. It is a flowchart of a determination process. It is a conceptual diagram explaining the feature point of a person. It is a difference image between an address frame and a top frame. It is an image explaining the feature point of a heel and a toe. It is an image explaining the feature point of a thigh. It is an image explaining the feature point of a head. It is an image explaining this forehead feature point. It is an image explaining the feature point of the waist. It is an image explaining the feature point of the back of the knee. It is an image explaining the feature point of back and shoulder vicinity. It is an image explaining a mask area. It is an image explaining a human body zone. It is a flowchart which shows an example of a human body area determination step. It is a graph explaining each position of a histogram. It is a flowchart which shows an example of a human body area propagation step. It is a pixel arrangement | sequence figure explaining a distance image. It is a graph explaining each position of a histogram. It is a diagram explaining the concept of the present invention. It is a flowchart which shows an example of a consistency check step. It is explanatory drawing using a histogram. It is the silhouette of the person of the address state obtained by this embodiment. It is the silhouette of the person of the top state obtained by this embodiment. It is the silhouette of the person of the finish state obtained by this embodiment. This is an address frame in which a support is photographed. It is an enlarged array diagram of a pixel explaining a processing method of a background section propagation step. It is a silhouette image which shows the processing result of Example 1 and Comparative Example 1. It is a silhouette image which shows the processing result of Example 2 and Comparative Example 2. It is a silhouette image which shows the processing result of Example 3 and Comparative Example 3. It is a silhouette image which shows the processing result of Example 4 and Comparative Example 4.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 1 shows a configuration diagram of a silhouette extraction system 1 for carrying out the present invention. The system of the present embodiment includes a mobile phone 2 and a server 3 connected to the mobile phone 2 via communication means. Note that the communication means may include not only wireless but also wired communication means in the middle.

  The mobile phone 2 includes a camera 4, a memory 5, and a transmission / reception unit 6. The camera 4 can take a moving image. Examples of the memory 5 include a RAM, an SD card (including a mini SD, a micro SD, and the like) and other storage media.

  The server 3 includes a calculation unit 7, a memory 8, and a transmission / reception unit 9.

  For example, a CPU is used as the arithmetic unit 7, and as shown in FIG. 2, a frame extraction unit 7a, a first set creation unit 7b, a second set creation unit 7c, a luminance histogram creation unit 7d, and a color histogram creation unit. 7e and determination unit 7f.

  FIG. 3 describes a processing procedure of the silhouette extraction method performed by the system 1 of the present embodiment. In the present embodiment, first, a golf swing is photographed by the camera 4 (step S1).

  A golf swing is a series of movements performed by a person with a golf club. In this specification, the golf swing refers to an address state where the golf club is stationary, a takeback where the golf club is gradually swung up, a top where the golf club is most swung up, and the golf club is swung down from the top. It includes a downswing, an impact in which the head of the golf club collides with the golf ball, and a finish after the impact, by swinging the golf club from the front upward.

  FIG. 4 shows a screen when shooting is started with the camera 4 of the mobile phone 2. This screen is displayed on a monitor (not shown) of the mobile phone 2. This screen is taken from the back of the person 11 in the address state where the person 11 stands still holding the golf club 10.

  As a preferred embodiment, the mobile phone 2 shows a first guide frame 12 and a second guide frame 13. These frames 12 and 13 are displayed by software executed on a calculation unit (not shown) of the mobile phone 2. The photographer determines the angle of the camera 4 so that the first guide frame 12 includes the grip 10a of the golf club 10 and the second guide frame 13 includes the head 10b. Therefore, these frames 12 and 13 are useful for determining the distance between the camera 4 and the person 11 and the camera angle.

  In the present embodiment, the photographing is started from the address state of FIG. When shooting is started, the person 11 starts a golf swing, and shooting is continued until the finish. Movie data is obtained by shooting. This data is stored in the memory 5 (step S2). The number of pixels of this moving image is, for example, length 640 × width 320.

  When the photographer or person 11 operates the mobile phone 2, moving image data is transmitted to the server 3 (step S3). The data is transmitted from the transmission / reception unit 6 of the mobile phone 2 to the transmission / reception unit 9 of the server 3 and stored in the memory 8 (step S4).

  Next, the frame extraction unit 7a extracts a large number of frames (that is, still image data) that are aggregates of pixels from the moving image data (step S5). The number of extractions (frames) per second is 30 or 60. For example, in the case of 3 second moving image data with 30 frames, 30 × 3 90 frames are extracted. The extracted frame is subjected to correction processing as necessary. An example of the correction process is a camera shake correction process.

  Next, the first set creation unit 7b creates a whole frame set including all frames for each pixel (step S6). When the image is the above-mentioned moving image of 640 × 320, a set of 640 × 320 × 90 = 184432000 (pixels) is created.

  The second set creation unit 7c determines whether each pixel of each frame is an achromatic color or a chromatic color. For each pixel, a chromatic color frame set and an achromatic color frame set are determined. Create (step S7).

  Further, the luminance histogram creating unit 7d creates a luminance histogram (first histogram) for all frame sets (step S8). In this luminance histogram, for example, as shown in FIGS. 5A to 5C, the frequency is the number of frames, and the class is luminance (color information). A luminance histogram may be created based on other color information. Further, the luminance histogram may be smoothed.

  Further, the color histogram creation unit 7e creates a color histogram (second histogram) for the chromatic color frame set and the achromatic color frame set (step S9). In this color histogram, for example, as shown in FIGS. 6A to 6C, the frequency is the number of frames, the class relating to the chromatic color frame set is the hue (color information), and the class relating to the achromatic color frame set. Is the luminance (color information). The class related to the chromatic color frame set may be color information other than hue, and the class related to the achromatic color frame set may be color information other than luminance.

  The determination unit 7f performs determination processing (labeling) for determining whether each frame of each pixel is a background or a person using these pieces of information (step S10). The procedure of the determination process is shown in FIG.

  In the determination process of the present embodiment, first, the address frame extraction unit of the determination unit 7f extracts an address frame in which the address state of the person 11 is captured from the moving image frame (step S101). For example, when shooting starts from the addressed state, the first frame on the time axis is extracted. In this address frame, a person 11 and a golf club 10 are photographed as shown in FIG.

[Feature point extraction step]
Next, the feature point extraction unit of the determination unit 7f performs image processing on the address frame, and performs a feature point extraction step of extracting a plurality of predetermined feature points for the person 11 and the golf club 10 (step S102).

  In the present embodiment, as shown in FIG. 8, the feature points include the top 11a of the human body 11 in the addressed state, the forehead 11b, the back 11c, the waist 11d, the back 11e, the heel 11f, the toes 11g, the thigh 11h, It includes feature points P1 to P12 which are positions representing the respective parts of the hand 11i, the golf club head 10b and the shoulder 11j. These feature points P1 to P12 can be extracted by calculation by considering a substantially constant posture taken by a person (golfer) in an address state with respect to data obtained from image processing. Examples thereof will be described below.

[Feature point P1 (x1, y1) at hand]
First, the determination unit 7f calculates the difference between the address frame and the frame in the top state. FIG. 9 shows a binary image D1 of the difference image, which is the calculation result, and the portion displayed in black is a pixel that has changed. As is apparent from FIG. 9, by taking the difference between the two frames in a state that can be said to be a game, the hand of the address state and the state of the golf club 10 clearly appear. Next, the determination unit 7f sets a detection area F1 in the difference image D1. This detection area F1 is shown in white and is set in the range of 200 ≦ x ≦ 480 and 250 ≦ y ≦ 530 as pixel coordinates. This is determined in advance as an area that reliably includes the hand and the head based on the camera angle and the like. However, the range of the detection area F1 can be changed in different settings. Next, the determination unit 7f performs Hough transform on the detection area F1 of the difference image D1, and extracts a straight line La from the shaft 10c of the golf club. Both ends La1 and La2 of the detected straight line La are on the boundary of the detection area F1. Further, the determination unit 7f defines a rectangular area at the midpoint La3 of the straight line L1, moves it to the upper left along the straight line La, and the number of difference images in the rectangular area becomes equal to or greater than a predetermined value. The extracted position is extracted as a feature point P1 at hand. This processing is based on an empirical rule that, in the address posture, there is a portion that is swollen by the hand of the person, that is, the hand, on the upper end side of the shaft 10c.

[Feature point P2 of head (x2, y2)]
In FIG. 9, the determination unit 7f moves the rectangular area to the lower right along the straight line La, and sets the position where the number of difference pixels in the rectangular area is equal to or larger than a predetermined value to the position of the golf club head. Extracted as a feature point P2. These processes are based on an empirical rule that there is a large-shaped head on the lower end side of the shaft 10c in the address posture.

[Right foot toe feature point P3 (x3, y3), right foot heel feature point P4 (x4, y4)]
FIG. 10 shows a simplified binary image of an address frame. The general silhouette of the person 11 is displayed in black. In this example, the determination unit 7f sets a detection area F2 in the binary image. The detection area F2 is determined in advance as an area including the heel and toe of the right foot based on the camera angle and the like. The determination unit 7f extracts the black pixel closest to the lower left corner F2L of the detection area F2 as the feature point P4 of the right foot heel. Similarly, the determination unit 7f extracts the black pixel closest to the lower right corner F2R of the detection area F2 as the feature point P3 of the toe of the right foot. This process is based on an empirical rule that when the address state is photographed from the back, the right end is a toe and the left end is a heel.

[Left foot toe feature point P5 (x5, y5)]
The feature point P5 of the toe of the left foot is extracted using the result of the feature point P3 of the toe of the previous right foot. That is, as shown in FIG. 10, the determination unit 7f sets a reference region F3 surrounding the feature point P3 of the right foot toe, and calculates the most matching region using the reference region F3 as a template. This process is based on an empirical rule that when the address state is photographed from the rear, the person's feet have substantially the same shape on the left and right. In the address frame, the toe of the left foot is located in the vicinity of the upper portion of the toe of the right foot, so that the search target area to be matched can be limited.

[Thigh feature point P6 (x6, y6)]
FIG. 11 shows a simplified binary image of an address frame. The determination unit 7f extracts, as a feature point P6 of the thigh, an intersection point between a straight line Lb parallel to the y axis passing through the feature point P5 of the toe of the left foot and a straight line Lc parallel to the x axis passing through the feature point P1 at hand. . This process is also based on an empirical rule that when the address state is taken from behind, the front side of the person's thigh is in the vicinity of the above position.

[Characteristic point P7 (x7, y7)]
The determination unit 7f calculates the difference between the address frame and the frame in the impact state. FIG. 12 shows a binary image D2 of the calculated difference. Next, the determination unit 7f sets a detection area F4. This detection area F4 is shown in white, and the coordinates of x and y are set in the following range. This detection area F4 is determined in advance based on the camera angle or the like. Next, the determination unit 7f extracts the position having the smallest y coordinate among the difference pixels in the detection region F4 as the feature point P7 on the top of the head. In this process, since it is empirically known that the head position is almost the same in the above two states, the position having the smallest y coordinate in any of these states can be regarded as the top of the head. Based on the rule of thumb.
x5-15 pixels ≦ x ≦ x1 + 10 pixels 10 ≦ y ≦ y1
However, the symbols are as follows.
x1: x coordinate of the feature point P1 at hand x5: x coordinate of the feature point P5 of the toe of the left foot y1: y coordinate of the feature point P1 at hand

[Forehead feature point P8 (x8, y8)]
The determination unit 7f calculates the difference between the address frame and the frame in the finish state. FIG. 13 shows a binary image D3 of the calculated difference. Next, the determination unit 7f sets a detection area F5. This detection area F5 is shown in white, and the coordinates of x and y are set in the following range. This detection area F5 is determined in advance based on the camera angle or the like. Next, the determination unit 7f extracts, as the feature point P8, the position where the x coordinate is minimum among the difference pixels in the detection region F5. In this process, since it is empirically known that the head position is approximately the same in the above two states, it is possible to regard the position where the x coordinate is minimum in either of these states as a forehead. It is based on the rule of thumb that you can.
x7 ≦ x ≦ x7 + 70 pixels y7 ≦ y7 + 40 pixels However, the signs are as follows.
x7: x-coordinate of the top feature point P7 y7: y-coordinate of the top feature point P7

[Lumbar feature point P9 (x9, y9)]
The determination unit 7f calculates the difference between the address frame and the frame in the finish state. FIG. 14 shows a binary image D4 of the calculated difference. Next, the determination unit 7f sets a detection area F6. This detection area F6 is shown in white, and the coordinates of x and y are set in the following range. This detection area F6 is determined in advance based on the camera angle or the like. Next, the determination unit 7f extracts a position having the minimum x coordinate as the waist feature point P9 from among the difference pixels in the detection region F6. In this process, since it is empirically known that the position of the waist protrudes most in the above two states, the experience that the position where the x coordinate is the minimum can be regarded as a forehead in any of these states. Based on the law.
x4-70 pixels ≦ x ≦ x4 + 10 pixels y1-50 pixels ≦ y1 + 50 pixels However, the symbols are as follows.
x4: x-coordinate of the right foot heel feature point P4 y1: y-coordinate of the hand feature point P1

[Feature point P10 of the knee back (x10, y10)]
The determination unit 7f calculates the difference between the address frame and the frame in the impact state. FIG. 15 shows a binary image D5 of the calculated difference. Next, the determination unit 7f sets the detection area F7 to the binary image D5. This detection area F7 is shown in white, and the x and y coordinates are set in the following range. This detection area F7 is determined in advance based on the camera angle or the like. Next, the determination unit 7f extracts, as a feature point P10 of the back of the knee, a position having the maximum x coordinate among the difference pixels in the detection region F7. In this process, since it is empirically known that the position of the back of the knee is the most concave in the above two states, the position where the x coordinate is maximum in any of these states can be regarded as the back of the knee. Based on rules of thumb.
x9 ≦ x ≦ x9 + 10 pixels 0.5 * (y9 + y4) −10 ≦ y ≦ 0.5 * (y9 + y4) +10
However, the symbols are as follows.
x9: x-coordinate of the waist feature point P9 y4: y-coordinate of the feature point P4 of the right heel y9: y-coordinate of the waist feature point P9

[Back feature point P11 (x11, y11)]
As illustrated in FIG. 16, the determination unit 7f sets a position 25 pixels outside in the normal direction from the midpoint Lc1 of the straight line Lc connecting the top feature point P7 and the waist feature point P9 as the back feature point P11. Extract. This is derived heuristically based on the average human body shape.

[Feature point P12 near shoulder (x12, y12)]
As shown in FIG. 16, the determination unit 7f determines a position 30 pixels inside the normal direction from the midpoint Ld1 of the straight line Ld connecting the forehead feature point P8 and the feature point P1 at hand. Extract as P12. This is derived heuristically based on the average human body shape.

[Mask area determination step]
Next, the determination unit 7f sets a mask area based on the feature points P1 to P12 (step S103).

  As illustrated in FIG. 17, the determination unit 7f connects feature points P1-P6-P5-P3-P4-P10-P9-P11-P7-P8-P12-P1 to form a closed section. Next, the determination unit 7f sets a first mask region M1 that is a region in which the parallel line Le is closed and separated by a distance d of several pixels outside these connected lines.

  Moreover, the determination part 7f expands the straight line which connects P1-P2 containing the shaft of a golf club, and sets the 2nd mask area | region M2. Further, the determination unit 7f sets a third mask area M3 that includes a feature point P2 of the head and is a predetermined rectangular area larger than the head.

  Further, the determination unit 7 f determines the logical sum of these first to third mask areas as the mask area M and stores it in the memory 8. Based on the feature points P1 to P12, the mask number M is set in advance so that the mask region M is a region that includes the human body 10 and the golf club 11 and includes a surrounding background in a small range. . As an example, the distance d is preferably about 10 pixels. The rectangular area is preferably a rectangle having x of about 30 pixels and y of about 25 pixels. These settings may be appropriately changed according to the camera angle or the like.

[Human zone determination step]
Next, the human body zone setting unit of the determination unit 7f is a human body zone that has a high probability of forming the silhouette of the person 11 from the pixels included in the mask region M based on the feature points P1 to P12. Z is set (step S104).

In this process, as shown in FIG. 18, the next reference points Z1 to Z6 for zone setting are first extracted.
Reference point Z1: intersection of the y-axis passing through the top feature point P7 and the x-axis passing through the forehead feature point P8 Reference point Z2: midpoint of a straight line connecting the top feature point P7 and the waist feature point P9 Reference point Z3: 20 pixels to the right from the waist feature point P9 Reference point Z4: 20 pixels to the right from the knee feature point P10 Reference point Z5: 20 pixels to the right from the feature point P4 of the right foot 20 pixel points above Reference point Z6: Midpoint of straight line connecting feature point P12 near shoulder and reference point Z2 These reference points Z1 to Z2 are characteristic points in consideration of empirical rules and camera angles. It is determined to be inside (the person 11 side).

  Next, the determination unit 7f determines the human body zone Za using the reference points Z1 to Z6 and stores it in the memory 8. In this embodiment, the human body zone Za is a region surrounded by the reference points Z1-Z2-Z3-Z4-Z5-Z4-Z6-Z1. However, in order to increase the accuracy, the section Z4-Z6 includes an extended line of a straight line connecting the reference points Z5-Z4 extending upward from the reference point Z4 and a straight line connecting the reference points Z2-Z3 extending downward from the reference point Z6. And a bend line that is concave on the back side using a straight line parallel to. Also, the section Z1-Z6 is not formed by connecting the reference points Z1 and Z6 with a straight line, but is formed by a line parallel to the y-axis from the reference point Z6 and a straight line connecting the reference points Z1 and Z2. In the straight line portion of the human body zone Za, pixels on the straight line are included in the human body zone Za. The human body zone Za determined in this way is a set of pixels that are apparently likely to constitute the person 11 in the address state based on a rule of thumb.

[Human body segment determination step]
Next, the human body section determining step in which the human body section determining unit of the determination unit 7f determines the human body section in the histogram based on the histogram of the pixels of the human body zone and the color information of the pixels of the human body zone of the address frame. Is performed (step S105).

  Here, the human body section is a range for a class (in this example, luminance and hue) representing the human body in the histogram. A pixel in the human body zone Za of the address frame is likely to display a person as a rule of thumb. Therefore, if the information on the luminance and hue of the pixel is used, the human body section can be accurately determined in the histogram.

FIG. 19 shows an example of the processing procedure of the human body section determination step.
First, the determination unit 7f refers to the memory 8 and selects one pixel of the human body zone Za that has not been processed for an address frame (step S1051).

Next, the determination unit 7f determines _left and _right ends L _left and L _right (in the case of hue, the same _applies to H _left and H _right ) based on the luminance (or hue) of the pixel (step). S1052). Now, an example in which the luminance histogram of the pixel has been smoothed is shown in FIG. 20, and when the luminance of the pixel is represented by L _middle (the same _applies to H _middle in the case of hue), the left end L _“left” is a position where the number of frames becomes zero by _moving the curve of the histogram from L _middle to the left. Similarly, the right end L _right is set to a position where the number of frames becomes zero by _moving the curve of the histogram from L _middle to the right. However, in order to avoid noise, the position where the value becomes zero is assumed to be a place where the third class zero is consecutive. Thereby, for any pixel in the human body zone Za of the address frame, both ends of the peak of the histogram including the luminance (or hue) are known.

Next, the determination unit 7f the absolute value of the difference between the left L _left and right _{L. Right} of the histogram | L _right -L _left | (case of the hue, | H _right- H _left |. The same hereinafter) is given It is determined whether or not it is within a class (step S1053). In this embodiment, it is determined whether or not the absolute value | L _right −L _left | of the difference is 1 or more and 14 or less. This class width can be changed as required.

If the determination unit 7f determines that the absolute value | L _right −L _left | of the difference does not satisfy step S1053, that is, if the width of the histogram is zero or larger than a predetermined width, all the pixels are Are determined as pixels for displaying “background” in the frame (step S1057). Since such a pixel cannot be determined as a human body section, it is determined as “background” in all frames.

On the other hand, if the absolute value | L _right −L _left | of the difference is determined to be true in step S1053, the color distance D is calculated (step S1054). Here, for example, when the color information of a pixel is HSV, the color vector of hue H ₁ , saturation S ₁ , and brightness V ₁ of one pixel is C _1, and the hue H ₂ and saturation S of the other pixel are set. _{2. If} the color vector of lightness V ₂ is C ₂ , the color distance D (C ₁ , C ₂ ) is calculated by the following equation. In this step, the color distance between the HSV average of pixels of the same class as the luminance (hue) of the pixel and the HSV average of pixels of the same class from L _{right to} L _left is calculated.

D (C ₁ , C ₂ ) = aΔH ′ + bΔS ′ + cΔV ′
In the above formula, the symbols are as follows.
a, b, c: constants; in this embodiment, a = 5.1, b = 2.25, and c = 2.65
ΔH ′ = ΔH / 4.0
ΔS ′ = ΔS / 2.0
ΔV ′ = ΔV / 2.0
ΔH = √ {(X ₁ −X ₂ ) ² + (Y ₁ −Y ₂ ) ² }
_{ΔS = | S 1/100-} S 2/100 |
_{ΔV = | V 1/100-} V 2/100 |
X ₁ = S ′ _avg cos (H ₁ × 3.6)
Y ₁ = S ′ _avg sin (H ₁ × 3.6)
X ₂ = S ′ _avg cos (H ₂ × 3.6)
Y ₂ = S ′ _avg sin (H ₂ × 3.6)
S ′ _avg = (S ₁ ′ + S ₂ ′) / 2
S ₁ '= log ₁₀ (S ₁ /100×99+1.0)
S ₂ '= log ₁₀ (S ₂ /100×99+1.0)

Next, the determination unit 7f determines whether or not the color distance is 0.25 or less for all combinations (step S1055). If the determination result in step S1055 is true, the _left and _right ends L _left and L _right of the histogram of the pixel are determined as the range of the human body section (step S1056). The color distance threshold 0.25 can be variously changed.

  If the determination result in step S1055 is false, the pixel is determined as a pixel that displays “background” in all frames (step S1057).

Next, the determination unit 7f determines whether or not the determination is made using both the luminance histogram and the hue histogram. When only the luminance histogram is determined, the processing from step S1052 is similarly performed on the hue histogram. At this time, it is assumed that the luminance is read as hue, and the _left and _right ends L _left and L _right of the histogram are read as _left and _right ends H _left and H _right of the hue histogram, respectively.

  Next, the determination unit 7f determines whether or not a human body section has been determined for all the pixels in the human body zone Za (step S1059). If the result is false, the next pixel is selected (step S1051). ), Step S1052 and subsequent steps are performed again. On the other hand, if the result of step S1059 is true, the process returns to the main routine (return).

Thus, the human body section uses pixels that clearly constitute the human body in the address frame, and the peak of the histogram including the luminance L _middle (or hue H _middle ) of the pixel is a section having a certain width. In addition, it is determined as a condition that the color distances are substantially equal so that the color distances are not more than a certain value in all combinations. Therefore, the human body section setting step of the present embodiment can determine the human body section with high accuracy.

[Human body interval propagation step]
Next, based on the human body section determined in step S105, the human body section propagation unit of the determination unit 7f performs a human body section propagation step of determining the human body section in the histogram of the pixels outside the human body zone Za (step S106). ).

  In general, the colors of adjacent pixels representing a human body are often almost equal. With such two pixels, the range of the human body section of the histogram is approximately equal, and the colors of the corresponding pixels are approximately equal. Using these characteristics, the human body segment propagation step uses pixels of which the human body segment has not been determined (undecided pixels) for pixels in the vicinity of eight pixels (determination completion pixels) for which the human body segment has already been set with high accuracy. It determines the human body section.

An example of a specific processing procedure of the human body section propagation step is shown in FIG. First, the determination unit 7f generates a distance image d _ij (step S1061). As shown in the pixel arrangement diagram in FIG. 22, the distance image _dij is an image representing the distance from the human body zone Za expressed in gray, and is “1” closest to the human body zone Za, and from there The distance increases by 1 as the distance increases.

  Next, the determination unit 7f sets 1 as a parameter t that defines the propagation range as an initial value (step S1062).

  Next, the determination unit 7f refers to the distance image and determines whether or not there is a determination completion pixel in which a human body section is determined in the vicinity of 8 of the undetermined pixels where t ≧ dij (step S1063). . Here, “near 8” means eight pixels located adjacent to the undetermined pixel at the left, upper left, upper, upper right, right, lower right, lower and lower left.

  If there is no determination completion pixel for which the human body section is determined in the vicinity of 8 of the undetermined pixel (N in step S1063), the undetermined pixel is not adjacent (continuous) to the pixel for which the human body section is determined. Means that. In such a case, since the feature is not satisfied, the pixel is determined as a “background section” in all frames (step S1070).

If there is a determination completion pixel in the vicinity of 8 (Y in step S1063), the determination unit 7f determines (sets) the luminance and hue of the undetermined pixel as LB _middle and HB _middle (step S1064).

  Next, the determination unit 7f determines whether or not the propagation condition 1 is satisfied (step S1065). Propagation condition 1 is that the luminance and hue of an undetermined pixel to be determined approximate to the most frequent luminance and hue of the human body section of the determination-completed pixel. The specific calculation is performed under the following conditions.

[Propagation condition 1]
(In the case of brightness)
LA _max −β ≦ LB _middle ≦ LA _max + β
D (CLB _middle , CLA _max ) ≦ ρ
However, the symbols are as follows.
LA _max : the most frequent luminance of the human body section [L _left , L _right ] of the determination completion pixel β: constant, 3 in the present embodiment
ρ: constant, 0.25 in this embodiment
D: Color distance CLB _middle : Color vector of undetermined pixel of address frame CLA _max : Color vector of mode value of determination completed pixel (in case of hue)
HA _max −β ≦ HB _middle ≦ HA _max + β
D (CHB _middle , CHA _max ) ≦ ρ
However, the symbols are as follows.
HA _max : the most frequent luminance of the human body section [H _left , H _right ] of the determination completion pixel β: constant, 3 in the present embodiment
ρ: constant, 0.25 in this embodiment
D: Color distance CHB _middle : Color vector of undetermined pixel of address frame CHA _max : Color vector of mode value of determination completed pixel

Next, when the undetermined pixel satisfies the propagation condition 1 (Y in step S1065), the determination unit 7f determines the _left and _right ends LB _left , LB _right , HB _left , and HB _right of the histogram of the undetermined pixel. (Step S1066).

Now, an example in which the luminance and hue histograms of the undetermined pixel are smoothed is shown in FIG. 23. When the luminance (or hue) of the pixel is represented by LB _middle (HB _middle ), the left end LB Similarly to the above, _left (HB _left ) is a position where the number of frames becomes zero by _moving the curve on the histogram from LB _middle (HB _middle ) to the left. Similarly, the right end LB _right (HB _right ) is set to a position where the number of frames becomes zero by proceeding on the curve of the histogram from LB _middle (HB _middle ) to the right. However, in order to avoid noise, the position where the value becomes zero is assumed to be a place where the third class zero is consecutive. Thereby, the both ends of the peaks of both histograms including the luminance and hue of the pixel are found.

  Next, the determination unit 7f determines whether or not the undetermined pixel satisfies the propagation condition 2 (step S1067). Propagation condition 2 is that the peaks of the luminance histogram and hue histogram of the undetermined pixel to be determined are within a predetermined small width interval. Specifically, it is calculated by the following formula.

[Propagation condition 2]
(In the case of brightness)
1 ≦ | LB _right −LB _left | ≦ 14
LA _left −γ ≦ LB _left ≦ LA _left + γ
LA _right −γ ≦ LB _right ≦ LA _right + γ
Here, the symbols are as follows.
LA _left : _Left end of the human body section of the determination completion pixel LA _right : _Right end of the human body section of the determination completion pixel γ: Constant, which is 4 in this embodiment.
(In the case of hue)
1 ≦ | HB _right −HB _left | ≦ 14
HA _left −γ ≦ HB _left ≦ HA _left + γ
HA _right −γ ≦ HB _right ≦ HA _right + γ
Here, the symbols are as follows.
HA _left : The left end of the human body section of the determination completion pixel HA _right : The _right end of the human body section of the determination completion pixel γ: A constant, which is 4 in this embodiment.

  When the propagation condition 2 is satisfied (Y in step S1067), the determination unit 7f determines whether or not the undetermined pixel satisfies the propagation condition 3 (step S1068).

[Propagation condition 3]
In the propagation condition 3, the luminance is calculated by the color average p of pixels in the same class in the luminance histogram (where pε [LB _left , LB _right ]) and the average of the colors of all the pixels that are LB _middle. It is defined that the value of the color distance D is not more than a certain value (in this embodiment, not more than 0.25). Regarding the hue, the HSV average m (where m∈ [HB _left , HB _right ]) of pixels in the same class in the hue histogram and the color distance D calculated by the average of the colors of all the pixels that are LB _middle Is defined to be not more than a certain value (in this embodiment, not more than 0.25). By verifying the propagation condition 3, it is possible to confirm whether the colors have the same tendency in the histogram with respect to luminance and hue.

  When the determination unit 7f determines that the undetermined pixel satisfies the propagation condition 3, regarding the histogram of this pixel, the section that satisfies all the propagation conditions 1 to 3 is determined as the human body section, and the other sections are determined as the background section. (Step S1069).

  Next, when the undetermined pixel does not satisfy one of the propagation conditions 1, 2, and 3 (N in Steps S1065, 1067, or 1068), the determination unit 7f determines that all frames of the undetermined pixel are the background. (Step S1070).

  Next, the determination unit 7f determines whether or not all undetermined pixels have been processed (step S1071). If the determination result is false (N in step S1071), another undetermined pixel is selected and step is performed. S1063 and subsequent steps are repeated.

  Next, when the determination result is true in step S1071 (Y in step S1071), the determination unit 7f adds 1 to the parameter t that specifies the propagation range (step S1072). Thereby, the area | region of a propagation range is expanded.

  Next, the determination unit 7f determines whether or not the parameter t is equal to or less than the maximum value (100 in the present embodiment) of the propagation range (step S1073). If the result is true, the process returns to the main routine (return). . On the other hand, if the result of step S1073 is false, the propagation range is expanded by one and step S1063 and subsequent steps are repeated.

[Background section determination step]
Next, the background interval determination unit of the determination unit 7f determines the background interval based on the histogram of the pixels in the non-mask area outside the mask area M and the color information of the pixels in the non-mask area of the address frame. Determine (step S107). Here, the background section is a range for a class (in this example, luminance and hue) representing the background in the histogram.

  The pixels in the unmasked area of the address frame are likely to display the background. Therefore, the background interval can be accurately determined in the histogram by using the luminance and hue information of the pixels in the non-mask area.

  The processing procedure of the background section determination step is basically the same as the human body section determination step shown in FIG. 19, with “human body section” in FIG. 19 being “background section” and “background section” being “human body section. ”, The detailed flowchart is not shown, but the determination unit 7f generally performs the following processing.

a) Selection of a pixel in an unmasked area b) Determination of _left and _right ends L _left and L _right (H _left and H _{right in} the case of hue) of the histogram of the pixel based on the luminance (hue) of the pixel c ) Whether the absolute value | L _right −L _left | (| H _right −H _left |) of the difference between the _left end L _left (H _left ) and the right end L _right (H _right ) of the histogram is within a predetermined predetermined class. D) If the absolute value | L _right −L _left | (| H _right −H _left |) of the difference is larger than a predetermined value, the pixel is set as a pixel for displaying “person” in all frames. Determination e) When the absolute value | L _right −L _left | (| H _right −H _left |) of the difference is equal to or less than a predetermined value, the color distance D is calculated. F) The color distance D is a predetermined range in all combinations. G) The result of step f is within a predetermined range For end L _left (H _left) of the right and left of the histogram of the pixel, when L _right (H _right) class to determine h) the result of step f as a range of the background section of is not within the predetermined range, the pixel Determined to display “person” in all frames i) perform the above steps for all pixels in the non-masked area

Thus, using a pixel that will obviously constitute the background in the address frame, the peak of the histogram containing the luminance L _middle (or hue H _middle ) of that pixel is in a narrow width interval, and The background interval is determined as a condition that the color distances are substantially equal so that the color distance is equal to or less than a certain value in all combinations. Therefore, the background interval setting step of this embodiment can also determine the background interval with high accuracy.

[Background interval propagation step]
Next, the background interval propagation unit of the determination unit 7f performs a background interval propagation step of determining a background interval in the histogram of the pixels in the mask region based on the determined background interval (step S108).

  The processing procedure of the background section propagation step is basically the same as the human body section propagation step shown in FIG. 21. In FIG. 21, “human body section” is “background section” and “background section” is “human body section. The detailed description is not shown, but the determination unit 7f generally performs the following processing.

A) Generation of distance image d _ij b) Set 1 to parameter t that defines the propagation range c) Whether there is a determination completion pixel whose background section is determined in the vicinity of 8 undetermined pixels where t ≧ d _ij D) If there are no determination complete pixels whose background interval is determined in the vicinity of 8 undetermined pixels, the pixel is determined to be a “human body interval” in all frames. E) 8 neighborhoods of undetermined pixels If there is a determination completion pixel, the luminance and hue of the undetermined pixel are determined as LB _middle and HB _{middle. F} ) It is determined whether the undetermined pixel satisfies the propagation condition 1, and the undetermined pixel is determined as the propagation condition 1. If it satisfies, the _left and _right ends LB _left , LB _right , HB _left , HB _right of the histogram of the undetermined pixel are determined to determine whether or not the propagation condition 2 is satisfied. If the undetermined pixel satisfies the propagation condition 3, if the undetermined pixel satisfies the propagation condition 3, the interval that satisfies all of the propagation conditions 1 to 3 in the histogram of this pixel is The background section and the other sections are determined as the human body section. I) If the undetermined pixel does not satisfy one of the propagation conditions 1, 2, and 3, all frames of the undetermined pixel are determined as the human body. After executing the above processing for the undetermined pixels, add 1 to the parameter t. The processing continues until the parameter t exceeds the maximum value of the propagation range.

[Consistency judgment step]
Next, after the background section and the human body section are determined, the consistency check section of the determination unit 7f determines the consistency of these (step S109).

FIG. 24 schematically shows the processing performed so far. In the human body zone propagation step, the human body zone is propagated around the human body zone using pixels. In the background interval propagation step, the background interval is propagated inward using pixels in the non-mask area. With these steps, the human body section and the background section are determined for each of the following areas by two methods.
Human body zone: human body zone determination step and background zone propagation step Outside the human body zone in mask area: human body zone propagation step and background zone propagation step Non-masked region: background zone determination step and human body zone propagation step

  However, there is a possibility that contradictions occur in the method of determining the section due to the difference in each method. Therefore, in this embodiment, a consistency determination step is performed in order to solve such a contradiction. An example of the processing procedure of the consistency determination step is shown in FIG.

  First, the determination unit 7f determines whether or not the consistency condition 1 is satisfied (step 1091). In this embodiment, the consistency condition 1 is as follows.

[Consistency condition 1]
D (C _b ; C _h ) <0.3
Pixel values (luminance and hue) of the background section for pixels for which both the background section and the human body section are determined in any of the human body section propagation step, the background section propagation step, the human body section determination step, or the background section determination step. not; _{(C h} C b) is less than a constant value determined in advance (less than 0.3 in this embodiment) of the average _{C b,} the body section pixel value average _{C h} color distance _D (luminance and hue) Or

  Then, when the determination result of the consistency condition 1 is false (N in Step S1091), the determination unit 7f determines that the determined human body section and the background section are correct. The other sections are determined as background sections (step S1095). In the consistency condition 1, it is determined whether the average color of the pixel values of the human body section and the background section is not too close. If the determination unit 7f determines that the consistency condition 1 is not satisfied, the determination unit 7f determines that the colors of the human body section and the background section are sufficiently different, and treats the previous determination result as correct.

  On the other hand, when determining that the consistency condition 1 is satisfied (Y in step S1091), the determination unit 7f determines whether the consistency condition 2 is satisfied (step S1092). In this embodiment, the consistency condition 2 is as follows.

[Consistency condition 2]
D (C _b ; C _back ) <D (C _b ; C _human ) (1)
A> B (2)
Here, the symbols are as follows.
D (C _b ; C _back ): Color distance D (C _b ;) between the average C _b of the pixel values of the target pixel and the color vector C _back of the pixel value of the background section of the non-mask region pixel that is the propagation source. C _human): average pixel value C _b and the color distance between the color vector C _human pixel values of the human body section of the propagation source in which the human body zones pixels a: are common in (F _b and F _back) Number of frames / Number of F _back B: Number of frames common to (F _b and F _human ) / Number of F _human F _b : In the background section at a position satisfying D (C _b ; C _h ) <0.3 Corresponding frame set F _back : Frame set corresponding to the background section of C _back F _human : Frame set corresponding to the _human body section of C _human

  FIG. 26 shows a histogram of a certain pixel as an example. When the color distance between the average color of pixels in the background section of a certain pixel, the background color of the propagation source, and the human body color of the propagation source is calculated, the color distance from the background color may be small. Equation (1) of consistency condition 2 verifies this point. In addition, it is more reliable as a background section to include more background frames. This point is verified in the expression (2) of the consistency condition 2.

  If it is determined that the consistency condition 2 is also satisfied (Y in step S1092), the determination unit 7f determines that the section is a background section (step S1094). If the determination unit 7f determines that the consistency condition 2 is not satisfied, the section is determined as a human body section (step S1093). As described above, by performing the consistency check, it is possible to perform determination with higher accuracy.

  Through the above processing, all the pixels in all the frames are classified into either “background section” or “person section”. For each frame, it is possible to determine whether it is a background or a person using a luminance histogram and a hue histogram.

  27 to 29 show the silhouettes of the person in the address, top, and finish frames obtained by the silhouette extraction method of the present embodiment. By this method, the silhouette of the person is reproduced almost faithfully. A golfer's swing is diagnosed by image processing using such a silhouette.

Various improvements can be added to the present invention.
For example, when shooting of a swing by the camera 4 is performed at a golf driving range, the background of each frame includes one or more support columns 16 that support the net of the golf driving range, as shown in FIG. May be. Such a support column 16 extends vertically in the camera angle. For this reason, the support | pillar 16 blocks | interrupts the horizontal continuity of the color i of the scenery with the similar background in the meantime. This tends to reduce the accuracy of the background interval propagation step. Therefore, if the struts 16 are separately recognized in the background in advance, if the pixels representing the struts 16 are skipped (spread) in the background interval propagation step, the accuracy of the background interval propagation step is increased. be able to.

As a feature of the support, for example, the following regularity can be given.
A) In the upper area of the address frame (for example, 0 ≦ y ≦ 250) B) Almost achromatic C) ± 2 degrees with respect to the vertical line D) Range in the x direction of a predetermined pixel (in this embodiment, 1-11 pixels)

  In the present embodiment, the pixel set is searched using the information of each pixel of the address frame using the regularity. Thereby, in the address frame, it is possible to specify the column section that is the class displaying the column 16. Moreover, since it is thought that the support | pillar 16 does not change during a golf swing, it can be considered as the support | pillar 16 also in another frame.

  In the background section determination step, it is desirable to skip pixels in the support section. FIG. 31 shows an enlarged plan view of the pixel, and the pixel displayed in gray displays the column 16. Now, assume that the pixel 9 is processed in the background interval propagation step. The eight neighboring pixels of pixel 9 become pixels 1-8. However, in this case, since the pixels 3, 5 and 8 are the pixels displaying the pillars, the neighborhood of 8 in this case skips these pixels, and the pixels 1, 2, 4, 6, 7, 3 ′, 5 These shall be referred to as “and 8”. Therefore, for example, even if the pixels 1, 2, 4, 6, and 7 are undetermined pixels, if any of the pixels 3 ′, 5 ′, and 8 ′ is a determination completed pixel, the pixels are referred to these pixels. Nine background segments can be determined.

  As described above, the present invention has been variously described, but the silhouette extraction system of the present invention can be realized by using only the CPU of the mobile phone 2. In this case, the present invention can be implemented using only a mobile phone, and connection with a server is unnecessary. If the photographer brings only the mobile phone 2, the photographer can diagnose the swing on the spot.

[Processing Example 1: FIG. 32]
In the first embodiment, a silhouette extraction process according to the present invention is performed on an original image. In Comparative Example 1, silhouette extraction is performed by performing a background interval propagation step + fixed mask process on an original image as in Patent Document 1 described above. As a result of the processing, in Comparative Example 1, a person's chest part having a color very similar to the background, such as a rectangular framed part, is recognized as the background. On the other hand, in Example 1, there is no such inconvenience and the recognition accuracy is high.

[Processing Example 2: FIG. 33]
In the second embodiment, a silhouette extraction process according to the present invention is performed on an original image. In Comparative Example 2, the silhouette extraction of Patent Document 1 is performed. As a result of the processing, in Comparative Example 1, a waist front portion of a person having a color very similar to the background, such as a rectangular framed portion, is recognized as the background. On the other hand, in Example 2, there is no such a malfunction and recognition accuracy is high.

[Processing Example 3: FIG. 34]
In the third embodiment, silhouette extraction processing according to the present invention is performed on the original image, and strut pixels are skipped in the strut recognition processing and the background interval propagation step. In Comparative Example 3, the silhouette extraction of Patent Document 1 is performed. As a result of the processing, in Comparative Example 3, a part of the support column is recognized as a person like a rectangular framed part. This is because the background pixel cannot be propagated beyond the support column, and the pixel of the support column is considered to be included in the person. On the other hand, in Example 3, there is no such inconvenience and the recognition accuracy is high.

[Processing Example 4: FIG. 35]
In the fourth embodiment, silhouette extraction processing according to the present invention is performed on the original image, and strut pixels are skipped in the strut recognition processing and the background interval propagation step. In Comparative Example 4, the silhouette extraction of Patent Document 1 is performed. As a result of the processing, in Comparative Example 4, a part of the support column is recognized as a person like a rectangular framed portion. On the other hand, in Example 4, there is no such inconvenience and the recognition accuracy is high.

1 Silhouette Extraction System 2 Mobile Phone 3 Server 4 Camera 5 and 8 Memory 7 Calculation Unit

Claims (8)

Extract a frame, which is a collection of pixels, from a moving image of a person who performs a swing motion with a golf club and the background, and specify each pixel of the extracted frame as either the person or the background A method for extracting the silhouette of the person,
Creating a complete frame set of all frames for each pixel;
Creating a histogram in which the frequency is the number of frames and the class is color information for the entire frame set;
Extracting an address frame in which the address state of the person is imaged from the moving image;
A feature point extracting step of performing image processing on the address frame and extracting feature points at a plurality of predetermined locations for the person and the golf club;
Based on the feature points, setting a mask area that includes the person and the golf club while partially including the surrounding background;
Setting a human body zone that is a region having a high probability of forming the silhouette of the person from pixels included in the mask region based on the feature points;
A human body section determining step for determining a human body section that is a class representing a human body in the histogram based on a histogram of pixels of the human body zone and color information of pixels of the human body zone of the address frame;
A human body segment propagation step for determining a human body segment in a histogram of pixels outside the human body zone based on the determined human body segment;
Based on the histogram of the pixels in the non-mask area outside the mask area and the color information of the pixels in the non-mask area in the address frame, a background section that is a class representing the background in the histogram is determined. A background segment determination step;
A silhouette extraction method comprising: a background interval propagation step of determining a background interval in a histogram of pixels in the mask region based on the determined background interval.   The histogram is a first histogram for all pixels and having a class as luminance, a second histogram with a class as hue for chromatic pixels, and a class whose luminance is achromatic pixels. The silhouette extraction method according to claim 1, comprising a third histogram.   3. The feature points include positions representing the top of the human body in the addressed state, forehead, back, waist, back of knees, heels, toes, thighs, hands, golf club heads and shoulders. Silhouette extraction method. The step of setting the mask region includes the step of connecting the feature points to set an initial mask region;
4. The silhouette extraction method according to claim 1, wherein the mask area is set by expanding the initial mask area to a predetermined thickness. Setting the human body zone based on the feature points, setting a reference point at a position that is clearly human body inside the initial mask region; and
The silhouette extraction method according to claim 4, wherein the human body zone is determined using the reference point. Identifying a pixel for displaying a driving range support among the pixels of the frame;
Determining a column section that is a class displaying the column based on the pixel displaying the column and the histogram thereof;
The silhouette extraction method according to claim 1, wherein the background section propagation step skips the strut section.   The silhouette extraction method according to claim 1, further comprising a step of determining consistency between the background section and the human body section after the background section and the human body section are determined. Extract a frame, which is a collection of pixels, from a moving image of a person who performs a swing motion with a golf club and the background, and specify each pixel of the extracted frame as either the person or the background A system for extracting the silhouette of the person,
A camera for capturing the video, a memory for storing the captured video, and a calculation unit;
The computing unit is
For each pixel, a set creation unit that creates a set of all frames including all frames;
A histogram creation unit that creates a histogram in which the frequency is the number of frames and the class is color information for the entire frame set;
An address frame extraction unit that extracts an address frame in which the address state of the person is imaged from the moving image;
A feature point extraction unit that performs image processing on the address frame and extracts feature points at a plurality of predetermined positions for the person and the golf club;
Based on the feature points, a mask area setting unit that sets a mask area that includes the human body and the golf club and includes a background around the human body and the golf club;
Based on the feature points, a human body zone setting unit that sets a human body zone that is a region having a high probability of forming the silhouette of the person from the pixels included in the mask region;
A human body section determination unit that determines a human body section that is a class representing a human body in the histogram based on a histogram of pixels of the human body zone and color information of pixels of the human body zone of the address frame;
Based on the determined human body section, a human body section propagation unit that determines a human body section in a histogram of pixels outside the human body zone,
Based on the histogram of the pixels in the non-mask area outside the mask area and the color information of the pixels in the non-mask area in the address frame, a background section that is a class representing the background in the histogram is determined. A silhouette extraction system comprising: a background section determination section; and a determination section including a background section propagation section that determines a background section in a histogram of pixels in the mask region based on the determined background section.