JP2011078069A - Silhouette extracting method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for highly accurately extracting the silhouette of a subject. <P>SOLUTION: A first aggregate forming section 24 forms a full frame aggregate consisting of all frames. A second aggregate forming section 26 determines whether each pixel per frame is achromatic or chromatic, and forms a chromatic color frame aggregate and an achromatic color frame aggregate for each pixel. A luminance histogram forming section 28 forms, for the full frame aggregate, a luminance histogram in which the frequency is the number of frames and the rank is expressed by luminance. A color histogram forming section 30 forms, for the chromatic frame aggregate and the achromatic frame aggregate, a second histogram in which the frequency is the number of frames, the rank for the chromatic frame aggregate is expressed by hue and the rank for the achromatic color frame aggregate is expressed by luminance. The determining section 32 determines whether each frame of each pixel is a background or the subject on the basis of both the histograms. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a method for extracting a silhouette of a subject performing an action such as sports.

  When a golfer hits a golf ball, the golf ball is addressed so that the line connecting the left and right toes is substantially parallel to the hitting direction. In the address of a right-handed golfer, the left foot is located on the front side in the batting direction and the right foot is located on the rear side in the batting direction. At the address, the golf club head is located near the golf ball. From this state, the golfer starts taking back and swings the head backward and then upwards. The position where the head is most swung up is the top position. A downswing is started from the top position, the head is swung down, and the head collides with the golf ball (impact). After the impact, the golfer swings the golf club forward and then upwards (follow-through), and finishes.

  In order to improve the skill of golfers, it is important to acquire appropriate swing forms. Swing diagnosis is done to help improve the skill. In swing diagnosis, a swing is photographed with a video camera. A swing may be filmed for the purpose of collecting materials useful for developing golf equipment.

  In classic swing diagnosis, teaching professionals watch videos and point out problems during swing. On the other hand, attempts have been made to diagnose swing by image processing. In the case of image processing, it is necessary to distinguish between a pixel where a golfer is photographed and a pixel where a background is photographed. A golfer's silhouette can be extracted by this distinction. For the discrimination, difference processing is usually used. In this process, the background is captured in advance. After that, the golfer is filmed. The golfer is distinguished from the background by the difference between the image in which only the background is photographed and the image in which the background and the golfer are photographed. Specifically, pixels having the same color information in both images are determined to be the background, and pixels other than the background are determined to be golfers (or golf clubs). Such a diagnostic method is disclosed in Japanese Patent Application Laid-Open No. 2005-270534.

JP-A-2005-270534

  The weather may change between shooting the background and shooting the swing. For example, sunlight may be shining when shooting a swing, even though it was cloudy when shooting a background. When a shadow is caused by sunlight, the color information of the pixel of the shadow is different from the color information when the background is photographed. Therefore, the background pixel is erroneously recognized as a golfer pixel by the difference processing. Misrecognition impedes the accuracy of silhouette extraction. Golfers want accurate silhouette extraction. In various sports such as baseball and tennis, accurate silhouette extraction is desired.

  An object of the present invention is to provide a method by which a silhouette of a subject can be extracted with high accuracy.

The silhouette extraction method according to the present invention includes:
Photographing a subject to be actuated with a background to obtain a plurality of frames each containing a number of pixels;
Creating a complete frame set of all frames for each pixel;
Determining whether each pixel of each frame is achromatic or chromatic, and creating a chromatic frame set and an achromatic frame set for each pixel;
Creating a first histogram for all frame sets, wherein the frequency is the number of frames and the class is the first color information;
For the chromatic color frame set and the achromatic color frame set, a second histogram in which the frequency is the number of frames, the class relating to the chromatic color frame set is the second color information, and the class relating to the achromatic color frame set is the third color information. Steps to create,
Determining whether each frame of each pixel is a background or a subject based on the first histogram and the second histogram;

Preferably, the step of determining includes
Based on the first histogram and the second histogram, for each pixel, determining whether the pixel is a background pixel over all frames, or whether the background frame and the subject frame are mixed. Including.

Preferably, the step of determining includes
Based on the first and second histograms, it is determined whether or not the pixels can be classified into the background frame group and the subject frame group based on the first histogram and the second histogram. Steps,
And for pixels that can be divided into a frame group as a background and a frame group as a subject.

Preferably, the step of determining includes
Whether each pixel is determined to be indistinguishable from the background frame group and the captured frame group, whether it is the background or the subject based on the relationship between this pixel and other adjacent pixels. Determining the step.

The silhouette extraction system according to the present invention is:
(A) a camera for photographing an object to be operated together with a background;
(B) A memory for storing captured image data and (C) a calculation unit. This calculator is
(C1) a frame extraction unit that extracts a plurality of frames including a large number of pixels from the image data;
(C2) For each pixel, a first set creation unit that creates a set of all frames including all frames;
(C3) A second set creation unit that determines whether each pixel of each frame is an achromatic color or a chromatic color, and creates a chromatic frame set and an achromatic frame set for each pixel ,
(C4) For all frame sets, a first histogram creating unit that creates a first histogram whose frequency is the number of frames and whose class is the first color information;
(C5) For the chromatic color frame set and the achromatic color frame set, the frequency is the number of frames, the class relating to the chromatic color frame set is the second color information, and the class relating to the achromatic color frame set is the second color information. A second histogram creation unit for creating a histogram,
(C6) includes a determination unit that determines whether each frame of each pixel is a background or a subject based on the first histogram and the second histogram.

  With the method according to the present invention, the silhouette of a subject to be operated is extracted with high accuracy. The accuracy of diagnosis using this silhouette is high.

FIG. 1 is a conceptual diagram illustrating a silhouette extraction system according to an embodiment of the present invention. FIG. 2 is a conceptual diagram showing details of the calculation unit of the system of FIG. FIG. 3 is a flowchart illustrating a silhouette extraction method performed by the system of FIG. FIG. 4 is an explanatory diagram showing the screen of the camera of FIG. FIG. 5 is an explanatory diagram showing a mask for the silhouette extraction method of FIG. FIG. 6 is a flowchart showing details of some steps of the silhouette extraction method of FIG. FIG. 7 is a luminance histogram for a certain pixel. FIG. 8 is a luminance histogram for other pixels. FIG. 9 is a luminance histogram for yet another pixel. FIG. 10 is a color histogram for the pixel of FIG. FIG. 11 is a color histogram for the pixel of FIG. FIG. 12 is a color histogram for the pixel of FIG. FIG. 13 is a flowchart showing the first stage of the determination step of the method of FIG. FIG. 14 is a flowchart showing a second stage of the determination step of the method of FIG. FIG. 15 is a flowchart showing the third stage of the determination step of the method of FIG. FIG. 16 is an explanatory diagram showing a silhouette obtained by the method of FIG. FIG. 17 is a conceptual diagram illustrating a silhouette extraction system according to another embodiment of the present invention.

  Hereinafter, the present invention will be described in detail based on preferred embodiments with appropriate reference to the drawings.

  The system 2 shown in FIG. 1 includes a mobile phone 4 and a server 6. The mobile phone 4 and the server 6 are connected via a communication line 8. The mobile phone 4 includes a camera 10, a memory 12, and a transmission / reception unit 14. Specific examples of the memory 12 include a RAM, an SD card (including a mini SD, a micro SD, and the like) and other storage media. The server 6 includes a calculation unit 16, a memory 18, and a transmission / reception unit 20.

  A typical arithmetic unit 16 is a CPU. FIG. 2 shows the calculation unit 16. The calculation unit 16 includes a frame extraction unit 22, a first set creation unit 24, a second set creation unit 26, a luminance histogram creation unit 28 (first histogram creation unit), and a color histogram creation unit 30 (second histogram creation unit). And a determination unit 32.

  FIG. 3 shows a flowchart of the silhouette extraction method performed by the system 2 of FIG. In this correction method, an image is taken by the camera 10 (STEP 1). FIG. 4 shows a screen before shooting is started. This screen is displayed on a monitor (not shown) of the mobile phone 4. On this screen, the address of the golfer 34 holding the golf club 33 is photographed. On this screen, the golfer 34 is photographed from behind. On this screen, a first frame 36 and a second frame 38 are shown. These frames 36 and 38 are displayed by software executed on the CPU (not shown) of the mobile phone 4. These frames 36 and 38 help the photographer to determine the angle of the camera 10. The photographer determines the angle of the camera 10 so that the first frame 36 includes the grip 40 and the second frame 38 includes the head 42. These frames 36 and 38 also help determine the distance between the camera 10 and the golfer 34.

  Shooting is started from the state shown in FIG. After the start of shooting, the golfer 34 starts swinging. Shooting continues until a golf ball (not shown) is hit and the swing is completed. Movie data can be obtained by shooting. This data is stored in the memory 12 (STEP 2). The number of pixels of this moving image is, for example, 640 × 320.

  When the photographer or the golfer 34 operates the mobile phone 4, moving image data is transmitted to the server 6 (STEP 3). The data is transmitted from the transmission / reception unit 14 of the mobile phone 4 to the transmission / reception unit 20 of the server 6. Transmission is performed via the communication line 8. The data is stored in the memory 18 of the server 6 (STEP 4).

  The frame extraction unit 22 extracts a large number of frames (that is, still image data) from the moving image data (STEP 5). The number of extractions per second is 30 or 60. Correction processing is performed on each frame as necessary. A specific example of the correction process is a camera shake correction process. These frames include a first frame and other frames taken after the first frame.

  The first set creation unit 24 creates an entire frame set including all frames for each pixel (STEP 6). The second set creation unit 26 determines whether each pixel of each frame is an achromatic color or a chromatic color, and creates a chromatic color frame set and an achromatic color frame set for each pixel. (STEP7).

  The luminance histogram creation unit 28 creates a luminance histogram (first histogram) for all frame sets (STEP 8). In this luminance histogram, the frequency is the number of frames, and the class is luminance (first color information). A luminance histogram may be created based on other color information. The color histogram creation unit 30 creates a color histogram (second histogram) for the chromatic color frame set and the achromatic color frame set (STEP 9). In this color histogram, the frequency is the number of frames, the class relating to the chromatic color frame set is hue (second color information), and the class relating to the achromatic color frame set is luminance (third color information). The class related to the chromatic color frame set may be color information other than the hue. The class related to the achromatic color frame set may be color information other than luminance.

  The determination unit 32 determines whether each frame of each pixel is a background or a subject based on the luminance histogram and the color histogram (STEP 10). The main steps are described in detail below.

  In the present embodiment, the mask 44 shown in FIG. 5 is set in the first frame. As is clear from FIG. 5, the mask 44 includes the golfer 34 and the golf club 33 shown in FIG. 4. The outer edge of the mask 44 is outside the outer edge of the golfer 34, and is outside the outer edge of the golf club 33. In determining whether each pixel is achromatic or chromatic, the pixels contained in the mask 44 are not considered for calculation.

  The flowchart of FIG. 6 shows details of the step of determining whether each pixel is an achromatic color or a chromatic color and creating an achromatic color frame set and a chromatic color frame set for each pixel (STEP 7). It is shown.

  In this method, the saturation value sf of the pixel is calculated (STEP 71). For example, when silhouettes are extracted based on 60 frames from the first frame to the 60th frame, the number of luminance values sf per pixel is 60.

  It is determined whether or not each of these 60 luminance values sf is smaller than the threshold value θs. The threshold value θs can be determined as appropriate. The threshold value θs used by the present inventor is 0.15. In other words, the color of the pixel having the luminance value sf less than 0.15 is regarded as an achromatic color or a substantially achromatic color. An initial achromatic color frame set Fm is obtained from the frames whose luminance value sf is smaller than the threshold value θs (STEP 72).

  Regarding the pixel color vector Cf in the frame f not belonging to the achromatic color frame set Fm, the minimum color distance d (Cf) with the set Fm is calculated (STEP 73). The calculation is performed based on the following mathematical formula.

  Based on the above equation, n is searched for when the color distance from the frame f is the smallest in the achromatic color frame set Fm.

  It is determined whether or not the obtained d (Cf) is less than the threshold value θd (STEP 74). The threshold value θd can be determined as appropriate. The threshold value θd used by the present inventor is 3.0. In other words, the color of a pixel with d (Cf) less than 3.0 is considered achromatic or substantially achromatic. If d (Cf) is less than the threshold value θd, the frame is added to the achromatic frame set Fm (STEP 75). With this addition, the achromatic color frame set Fm is updated. If d (Cf) is equal to or greater than the threshold value θd, the frame is classified into a chromatic color frame set (STEP 76). This flow is repeated until the chromatic color and achromatic color classification for all frames is completed.

  The flow shown in FIG. 6 is performed for all pixels except the mask 44. For example, when the number of pixels excluding the mouse is 150,000 and the number of frames is 60, a luminance value sf of 9000000 (15000 × 60) is calculated.

  The luminance histogram creation unit 28 creates a luminance histogram for all frame sets (STEP 8). An example of a luminance histogram for a pixel is shown in FIG. In this luminance histogram, the class is luminance. This histogram includes 100 classes from 1 to 100. In this histogram, the frequency is the number of frames. A smoothing process may be performed at a frequency. FIG. 8 shows an example of a luminance histogram for other pixels. FIG. 9 shows an example of a luminance histogram for still another pixel. In each luminance histogram, the total number of frames is 98.

  The color histogram creation unit 30 creates a color histogram for the chromatic color frame set and the achromatic color frame set (STEP 9). An example of a color histogram for a pixel is shown in FIG. This color histogram is a composite of a chromatic color frame set histogram and an achromatic color frame set histogram. In this color histogram, the class of the chromatic color frame set is hue. The hue class includes 100 classes from 1 to 100. In this color histogram, the class of the achromatic frame set is luminance. The luminance class includes 100 classes from 1 to 100. The total number of classes is 200. In this color histogram, the frequency is the number of frames. A smoothing process may be performed at a frequency. FIG. 11 shows an example of a color histogram for other pixels. FIG. 12 shows an example of a color histogram for still another pixel. In each color histogram, the total number of frames is 98.

  Based on the luminance histogram and the color histogram, it is determined whether each pixel is a background or a subject (STEP 10). The determination unit 32 performs the determination. This determination includes a first stage, a second stage, and a third stage. Each stage will be described in detail below.

FIG. 13 is a flowchart showing the first stage. This first stage is done for each pixel. In the first stage, first, it is determined whether or not the condition 1 is satisfied (STEP 1011). Condition 1 is as follows.
Condition 1: In the luminance histogram, all frames are included in a range where the class width is 20 or less. Note that a value other than “20” may be used as the class width.

  In the luminance histogram of FIG. 7, all frames are included in the range of luminance from 12 to 19 (that is, the width is 8). Therefore, this luminance histogram satisfies the above condition 1. In the luminance histogram of FIG. 8, the minimum value of the class is 12 and the maximum value is 72. Therefore, this luminance histogram does not satisfy the above condition 1. In the luminance histogram of FIG. 9, the minimum value of the class is 13 and the maximum value is 77. Therefore, this luminance histogram does not satisfy the above condition 1.

Next, it is determined whether or not the condition 2 is satisfied (STEP 1012). Condition 2 is as follows.
Condition 2: In the color histogram, all frames are included in a range where the class width is 20 or less. Note that a value other than “20” may be used as the class width.

  FIG. 10 is a color histogram for the pixel of FIG. FIG. 11 is a color histogram for the pixel of FIG. FIG. 12 is a color histogram for the pixel of FIG. In the color histogram of FIG. 10, all the frames are included in the range of hues from 59 to 66 (that is, the width is 7). Therefore, this color histogram satisfies the above condition 2. In the color histogram of FIG. 11, the minimum value of the hue class is 40 and the maximum value is 65. Furthermore, in the histogram of FIG. 11, the luminance class has a frequency. Therefore, this color histogram does not satisfy the above condition 2. In the color histogram of FIG. 12, the minimum value of the hue class is 16 and the maximum value is 64. Furthermore, in the histogram of FIG. 12, the luminance class has a frequency. Therefore, this color histogram does not satisfy the above condition 2.

  7 and 10, the luminance histogram satisfies the above condition 1 and the color histogram satisfies the above condition 2. When swinging, the golfer 34 moves. Due to this movement, both the golfer 34 and the background can be photographed on the pixel. If both the golfer 34 and the background are photographed, the brightness or hue of the pixel will vary significantly. A pixel satisfying both of the above conditions 1 and 2 is a pixel with small variations in luminance and hue. In other words, it is considered that the golfer 34 is not photographed for this pixel from the first frame to the last frame. The pixels satisfying the above conditions 1 and 2 are determined as “background” in all frames (STEP 1013).

  A chromatic color and an achromatic color having the same luminance cannot be distinguished from each other in the luminance histogram, but can be distinguished from each other in the color histogram. Two chromatic colors having the same hue and different luminance cannot be distinguished in the color histogram, but can be distinguished in the luminance histogram. In the silhouette extraction method according to the present invention, if both of the conditions 1 and 2 are satisfied, the pixel is determined as “background” in all frames. In other words, the determination is made in consideration of both the luminance histogram and the color histogram. Thus, pixels that are not backgrounds are rarely misidentified as backgrounds.

  It should be noted that the above conditions 1 and 2 can be satisfied even with pixels in which only the golfer 34 is photographed between the first frame and the final frame. However, since the golfer 34 is masked by the mask 44 as described above, the pixels satisfying the above conditions 1 and 2 can be determined as “background” in all frames.

  A pixel in which both the golfer 34 and the background are photographed between the first frame and the final frame does not satisfy the above condition 1 or 2. For pixels that do not meet condition 1 or 2, the decision is carried over to the second stage.

Hereinafter, the second stage will be described in detail. Further consideration is given in the second stage for pixels that are determined to be “both golfer 34 and background are being photographed” in the first stage. FIG. 14 is a flowchart showing the second stage. This second stage is done on a pixel-by-pixel basis. In the second stage, it is first determined whether or not the condition 3 is satisfied (STEP 1021). Condition 3 is as follows.
Condition 3: In the luminance histogram, 60% or more of all frames are included in the range where the class width is 20 or less, and a value other than “20” may be used as the class width. A value other than “60%” may be used as the ratio.

  In the luminance histogram of FIG. 8, 80 (that is, 81.6%) frames are included in the range of luminance from 12 to 19 (that is, the width is 8). Therefore, the above condition 3 is satisfied. In the luminance histogram of FIG. 9, the condition 3 is not satisfied.

Next, it is determined whether or not the condition 4 is satisfied (STEP 1022). Condition 4 is as follows.
Condition 4: In the color histogram, 60% or more of all frames are included in the range where the class width is 20 or less. A value other than “20” may be used as the class width. A value other than “60%” may be used as the ratio.

  In the color histogram of FIG. 11, 72 (that is, 73.5%) frames are included in the range of luminance from 59 to 65 (that is, the width is 7). Therefore, the above condition 4 is satisfied. In the luminance histogram of FIG. 12, the above condition 4 is not satisfied.

  In the pixels shown in FIGS. 8 and 11, the luminance histogram satisfies the condition 3 and the color histogram satisfies the condition 4. When 60% or more of all the frames are included in the range of the class width of 20 or less, it is considered that the luminance or hue variation is small in the pixel of the frame group within the class width. On the other hand, it is considered that the luminance or hue of the pixel in the frame group outside the class width is significantly different from the luminance or hue of the pixel in the frame within the class width. From this phenomenon, it is considered that the background was mainly photographed in the pixel from the first frame to the final frame, and the human body of the golfer 34 was temporarily photographed. For pixels satisfying the above conditions 3 and 4, the frame within the class width is determined as “background”, and the other frames are determined as “subject” (STEP 1023).

  A chromatic color and an achromatic color having the same luminance cannot be distinguished from each other in the luminance histogram, but can be distinguished from each other in the color histogram. Two chromatic colors having the same hue and different luminance cannot be distinguished in the color histogram, but can be distinguished in the luminance histogram. In the silhouette extraction method according to the present invention, the determination is made based on both conditions 3 and 4. In other words, the determination is made in consideration of both the luminance histogram and the color histogram. Accordingly, erroneous recognition is suppressed.

  The determination of pixels exhibiting histograms as shown in FIGS. 9 and 12 is carried over to the third stage.

  Hereinafter, the third stage will be described in detail. Further consideration is given in the third stage with respect to pixels that have been carried over in the second stage and pixels corresponding to the mask 44. Hereinafter, a pixel for which “background” or “subject” has already been determined is referred to as a “determination completed pixel”. On the other hand, a pixel that has not yet been determined as “background” or “subject” is referred to as a “determination pixel”.

  FIG. 15 is a flowchart showing the third stage. In the third stage, a distance image dxy is generated for the pixels that have not been determined (STEP 1031). The distance image dxy is obtained by adding depth data to two-dimensional data. Here, the depth data is a distance to the boundary.

  When the initial value of the threshold θd is 1, it is examined whether or not there is a determination completion pixel in the vicinity of eight of the determination incomplete pixels whose dxy is less than θd (STEP 1032). Here, the vicinity of 8 means eight pixels located at the left, upper left, upper, upper right, right, lower right, lower and lower left of the determination incomplete pixel.

  If there is no determination completion pixel in the vicinity of 8, the pixel is determined to be a “subject” in all frames (STEP 1033). When one or more determination completion pixels exist in the vicinity of 8, it is determined whether or not the following condition 5 is satisfied (STEP 1034). Condition 5 is as follows.

    Condition 5: In the luminance histogram, there exists a frame group that satisfies the following formula:

min (LQ)> min (LB) −θw
max (LQ) <max (LB) + θw
In these equations, min (LQ) is the minimum class width value of the frame group in the luminance histogram of the incomplete determination pixel, and max (LQ) is the maximum class width value of the frame group in the luminance histogram of the incomplete determination pixel. Yes, min (LB) is the minimum class width of the frame group that is the background in the luminance histogram of one determination completion pixel existing in the vicinity of 8, and max (LB) is one determination completion pixel existing in the vicinity of 8. This is the maximum value of the class width of the frame group as the background in the luminance histogram. θw is set as appropriate. The inventor used 6 as θw.

  When one or more determination completion pixels exist in the vicinity of 8, it is further determined whether or not the following condition 6 is satisfied (STEP 1035). Condition 6 is as follows.

    Condition 6: In the color histogram, there is a frame group that satisfies the following mathematical expression.

min (CQ)> min (CB) −θw
max (CQ) <max (CB) + θw
In these formulas, min (CQ) is the minimum class width value of the frame group in the color histogram of undecided pixels, and max (CQ) is the maximum class width value of the frame group in the color histogram of undecided pixels. Yes, min (CB) is the minimum class width of the frame group that is the background in the color histogram of one determination completion pixel existing in the vicinity of 8, and max (CB) is one determination completion pixel existing in the vicinity of 8. This is the maximum value of the class width of the frame group as the background in the color histogram. θw is set as appropriate. The inventor used 6 as θw.

  The pixel in the frame group that satisfies the above conditions 5 and 6 is determined as “background”, and the pixel in the frame group that does not satisfy the conditions 5 and 6 is determined as “subject” (STEP 1036). If any of the above conditions 5 and 6 is not satisfied in the relationship with the determination completion pixel and another determination completion pixel exists in the vicinity of 8, the above conditions 5 and 6 are in the relationship with the other determination completion pixel. It is determined whether it is satisfied.

  After the examination of the conditions 5 and 6 is completed for all the incomplete determination pixels, “1” is added to θd (STEP 1037). Then, the flow from examination (STEP 1032) on whether or not there is a judgment completion pixel in the vicinity of eight judgment incomplete pixels to the judgment (STEP 1036) is repeated. This is repeated until θd reaches θdmax. θdmax is the maximum value in the distance image.

  Through the above flow, all the pixels in all the frames are classified into either “background” or “subject”. In each frame, a set of pixels set as a subject is a silhouette of the subject. FIG. 16 shows the silhouette of one frame. In FIG. 1, the pixel of the subject is shown in black, and the other pixels are shown in white. As is apparent from FIG. 16, the silhouette of the subject (golfer 34) is almost faithfully reproduced by this method. Using such a silhouette, the swing of the golfer 34 can be diagnosed by image processing. Since the time from the start to the end of the shooting is short, the weather rarely changes during this time. Accordingly, misrecognition due to weather changes is unlikely to occur.

  FIG. 17 is a conceptual diagram showing a silhouette extraction system 46 according to another embodiment of the present invention. The system 46 includes a mobile phone 48. The cellular phone 48 includes a camera 50, a memory 52, and a calculation unit 54. Although not shown, this calculation unit 54 is similar to the calculation unit 16 shown in FIG. 2, and includes a frame extraction unit, a first set creation unit, a second set creation unit, a luminance histogram creation unit, a color histogram creation unit, and Includes a determination unit. The calculation unit 54 performs the same function as the calculation unit 16 shown in FIG. That is, the calculation unit 54 extracts silhouettes. Therefore, connection between the cellular phone 48 and the server is not necessary. If the photographer brings only the mobile phone 48, the photographer can diagnose the swing on the spot.

  The silhouette extraction method described above is useful in swing diagnosis, golf club development, golf shoe development, and the like. This extraction method can be used for form diagnosis of various sports such as tennis and baseball.

2 ... Silhouette extraction system 4 ... Mobile phone 6 ... Server 8 ... Communication line 10 ... Camera 12 ... Memory 16 ... Calculation unit 22 ... Frame extraction unit 24 ... First set creation unit 26 ... second set creation unit 28 ... luminance histogram creation unit 30 ... color histogram creation unit 32 ... determination unit

Claims (5)

Photographing a subject to be actuated with a background to obtain a plurality of frames each containing a number of pixels;
Creating a complete frame set of all frames for each pixel;
Determining whether each pixel of each frame is achromatic or chromatic, and creating a chromatic frame set and an achromatic frame set for each pixel;
Creating a first histogram for all frame sets, wherein the frequency is the number of frames and the class is the first color information;
For the chromatic color frame set and the achromatic color frame set, a second histogram in which the frequency is the number of frames, the class relating to the chromatic color frame set is the second color information, and the class relating to the achromatic color frame set is the third color information. Steps to create,
A silhouette extraction method including a step of determining whether each frame of each pixel is a background or a subject based on the first histogram and the second histogram. The step of determining
A step of determining, based on the first histogram and the second histogram, whether each pixel is a background pixel over all frames or a pixel in which a background frame and a subject frame are mixed. The silhouette extraction method according to claim 1, comprising: The step of determining
Regarding the pixel in which the background frame and the subject frame are mixed, based on the first histogram and the second histogram, whether or not the pixel can be classified into the background frame group and the subject frame group. Steps to be determined,
The method according to claim 2, further comprising: a step of performing classification on pixels that can be classified into a frame group that is a background and a frame group that is a subject. The step of determining
For each frame of a pixel determined to be indistinguishable from the background frame group and the captured frame group, based on the relationship between this pixel and other adjacent pixels, it is the background or the subject. 4. The silhouette extraction method according to claim 3, further comprising the step of determining whether or not. (A) a camera for photographing an object to be operated together with a background;
(B) a memory for storing captured image data, and (C) a calculation unit,
This computing unit is
(C1) a frame extraction unit that extracts a plurality of frames including a large number of pixels from the image data;
(C2) For each pixel, a first set creation unit that creates a set of all frames including all frames;
(C3) A second set creation unit that determines whether each pixel of each frame is an achromatic color or a chromatic color, and creates a chromatic frame set and an achromatic frame set for each pixel ,
(C4) For all frame sets, a first histogram creating unit that creates a first histogram whose frequency is the number of frames and whose class is the first color information;
(C5) For the chromatic color frame set and the achromatic color frame set, the frequency is the number of frames, the class relating to the chromatic color frame set is the second color information, and the class relating to the achromatic color frame set is the second color information. A second histogram creation unit for creating a histogram,
(C6) A silhouette extraction system including a determination unit that determines whether each frame of each pixel is a background or a subject based on the first histogram and the second histogram.

Images