JP5884759B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program for detecting a region of a measurement object.

Conventionally, a user's form in a ball game such as golf or tennis is captured as a continuous image, the continuous image is analyzed, and advice of the user's form is performed based on the analysis result.
As one of such continuous image analyses, a technique for detecting a moving ball from a continuous image is known (see, for example, Patent Documents 1 and 2).

JP 2008-60974 A JP 2008-60982 A

However, in the conventional techniques including Patent Document 1, when the moving object included in the continuous image is only a ball, the ball can be detected with a certain accuracy. When the object exists separately from the ball, there is a possibility that the accuracy of detecting the ball deteriorates or the ball cannot be detected well.
Therefore, there is a demand for detecting an object to be measured with high accuracy even when moving objects other than the object to be measured are present in the image.

  The present invention has been made in view of such a situation, and an object of the present invention is to enable detection of an object to be measured with high accuracy even when a moving object other than the object to be measured exists in an image.

In order to achieve the above object, an image processing apparatus according to an aspect of the present invention includes:
A degree-of-separation calculating means for calculating the degree of separation of a plurality of images captured continuously;
A degree-of-separation difference value calculating means for calculating a difference value between the plurality of images of the degree of separation calculated by the degree-of-separation calculating means ;
A specifying unit for specifying an image immediately before the frame-out of the object to be measured based on the separation degree difference value calculated by the separation degree difference value calculating unit from the plurality of images captured continuously;
Detecting means for detecting a region of the object to be measured from each of the plurality of images taken continuously with reference to the image immediately before the frame-out specified by the specifying means;
It is characterized by providing.

  According to the present invention, it is possible to accurately detect an object to be measured even when a moving object other than the object to be measured exists in the image.

It is a block diagram which shows the hardware constitutions of the imaging device which concerns on one Embodiment of the image processing apparatus of this invention. It is a functional block diagram which shows the functional structure for performing a ball speed estimation process among the functional structures of the imaging device of FIG. FIG. 3 is a diagram for explaining a technique for calculating an index value used when calculating the trajectory and speed of a ball by the imaging apparatus of FIG. 2, and shows an example of a difference resolution map corresponding to a predetermined frame to be processed. FIG. It is a figure which shows an example of the termination | terminus image for demonstrating the detection method of the ball | bowl used when the locus | trajectory and speed | velocity | rate of a ball | bowl are calculated with the imaging device of FIG. It is a figure for demonstrating the detection method of the ball | bowl used when the locus | trajectory and speed of a ball | bowl are calculated with the imaging device of FIG. 2, Comprising: It is a figure which shows an example of a detection area. 3 is a flowchart for explaining a flow of ball speed estimation processing executed by the imaging apparatus of FIG. 1 having the functional configuration of FIG. 2. It is a flowchart explaining the detailed flow of the speed calculation process of the part before a shot among the ball speed estimation processes of FIG. It is a flowchart explaining the detailed flow of the ball detection process of the part after a shot among the ball speed estimation processes of FIG. FIG. 9 is a flowchart for explaining a detailed flow of a ball detection process in reverse chronological order among the ball detection processes in the post-shot portion of FIG. 8.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing a hardware configuration of an imaging apparatus according to an embodiment of the image processing apparatus of the present invention.
The imaging device 1 is configured as a digital camera, for example.

  The imaging device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, an image processing unit 14, a bus 15, an input / output interface 16, and an imaging unit. 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22.

The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 20 to the RAM 13.
The RAM 13 appropriately stores data necessary for the CPU 11 to execute various processes.

The image processing unit 14 is configured by a DSP (Digital Signal Processor), a VRAM (Video Random Access Memory), and the like, and performs various image processing on image data in cooperation with the CPU 11.
For example, the image processing unit 41 performs image processing such as noise reduction, white balance adjustment, and camera shake correction on the image data output from the imaging unit 17.

  The CPU 11, ROM 12, RAM 13, and image processing unit 14 are connected to each other via a bus 15. An input / output interface 16 is also connected to the bus 15. An imaging unit 17, an input unit 18, an output unit 19, a storage unit 20, a communication unit 21, and a drive 22 are connected to the input / output interface 16.

  Although not shown, the imaging unit 17 includes an optical lens unit and an image sensor.

The optical lens unit is configured by a lens that collects light, for example, a focus lens or a zoom lens, in order to photograph a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. The zoom lens is a lens that freely changes the focal length within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance as necessary.

The image sensor includes a photoelectric conversion element, AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (captures) the subject image, accumulates the image signal for a predetermined time, and sequentially supplies the accumulated image signal as an analog signal to the AFE.
The AFE performs various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing and output as an output signal of the imaging unit 17.
Such an output signal of the imaging unit 17 is hereinafter referred to as “captured image data”. The captured image data is appropriately supplied to the CPU 11, the image processing unit 14, and the like.

The input unit 18 includes various buttons and the like, and inputs various types of information according to user instruction operations.
The output unit 19 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 20 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores various image data.
The communication unit 21 controls communication with other devices (not shown) via a network including the Internet.

  A removable medium 31 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately attached to the drive 22. The program read from the removable medium 31 by the drive 22 is installed in the storage unit 20 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 20 in the same manner as the storage unit 20.

  FIG. 2 is a functional block diagram illustrating a functional configuration for executing the ball speed estimation process among the functional configurations of the imaging apparatus 1 as described above.

Here, the ball speed estimation process is, for example, a continuous image (for example, a moving image composed of a plurality of frames in the present embodiment) obtained by shooting a golf swing motion at a high speed and an image to be processed. It means a series of processes executed until the trajectory and speed of the moving (flighting) ball in the image to be processed are calculated.
In the present embodiment, in order to enable accurate tracking of the ball, the imaging apparatus 1 uses a frame (hereinafter referred to as “frame”) in which the moving ball is no longer imaged out of the imaging range of the camera. (Referred to as “out image”). Then, the imaging apparatus 1 calculates the trajectory and speed of the ball by applying the ball speed estimation algorithm in reverse time series based on the frame-out image.
Here, the ball speed estimation algorithm generates a separability map (an image having the same resolution as the processing target and using the separability as each pixel value) from the luminance image of the continuous image to be processed. An algorithm for calculating the trajectory of the ball and calculating the velocity of the ball by connecting points (pixels) with a high degree of separation under the assumption that the ball is moving at a constant linear velocity. . Since details of the ball speed estimation algorithm including the degree of separation have already been disclosed in Patent Document 1, the description thereof is omitted here.
Although details will be described later, in the present embodiment, when the ball speed estimation algorithm is applied, various threshold values are automatically set and used adaptively from the relationship between the golf club and the ball. The accuracy of estimation is improved compared to the case.

When such a ball speed calculation process is executed, as shown in FIG. 2, in the CPU 11 or the image processing unit 14, an image reading unit 51, a separation degree calculation unit 52, a difference separation degree calculation unit 53, A club speed calculation unit 54, an index value calculation unit 55, a frame-out image specifying unit 56, a threshold setting unit 57, a ball detection unit 58, and a ball speed calculation unit 59 function.
As one area of the storage unit 20, a moving image storage unit 61 and a target image storage unit 62 are provided.
The moving image storage unit 61 stores in advance data of one or more moving images in which the state of the golf swing operation is shown. The moving image data stored in the moving image storage unit 61 may be data of a captured image captured by the imaging unit 17, or may be transmitted from another device (not shown) and received by the communication unit 21. The image data may also be used.
The target image storage unit 62 stores data to be subjected to ball speed calculation processing among the data of one or more moving images stored in the moving image storage unit 61, that is, moving images to be subjected to ball trajectory and speed calculation. Are stored.

  The image reading unit 51 reads the moving image data to be subjected to the ball speed calculation process from the moving image storage unit 61 and stores it in the target image storage unit 62.

Here, the ball speed calculation process is roughly divided into the following two processes before and after the scene of the ball shot.
That is, as the process before the shot scene of the ball, since the ball is stationary and the speed is 0 before the shot, the process of calculating the speed of the golf club immediately before the shot is executed. Hereinafter, such processing is referred to as “speed calculation processing before shot”.
On the other hand, as a process after the ball shot scene, a process of calculating a trajectory of the ball that moves (flys) after the shot (hereinafter referred to as a “ball detection process after the shot”) is executed. Processing for calculating the speed is executed.

  When the speed calculation process for the pre-shot portion is executed, among the functional blocks in FIG. 2, in particular, the image reading unit 51, the separation degree calculation unit 52, the difference separation degree calculation unit 53, and the club speed calculation unit 54 function. .

The image reading unit 51 uses moving image data stored in the target image storage unit 62 as a processing target, such as moving image data in a predetermined section in which a state before a shot is reflected, for example, data of four frames immediately before the shot frame. Read.
Here, the shot frame is a frame in which the moment of the shot is shown, and is assumed to be specified by a known algorithm. It is also assumed that the position of the ball in the shot frame has already been specified.

  The degree-of-separation calculation unit 52 generates a degree-of-separation map by calculating the degree of separation in units of pixels for each data of each frame to be processed. That is, in the speed calculation process for the pre-shot portion, a separability map is generated for each of the four frames immediately before the shot frame.

なお、画素値としては、輝度値に代えて、例えば特定の色成分や色相等を利用してもよい。
式（１）の分離度ηは、全体領域Ａ０全体の変動に占める第１領域Ａ１と第２領域Ａ２との領域間の変動の割合を示しており、領域間を最良に分離するときに最大値をとるようになっている。
ここで、式（１）の分離度ηは、０〜１の範囲に正規化された値であるため、異なる大きさや形状の領域に適用しても単純に比較することができる点で有用である。
なお、当該手法のさらなる詳細については、特許文献１を参照するとよい。 The method of calculating the degree of separation is not particularly limited, but in the present embodiment, the method described in Patent Document 1 is adopted.
The outline of the method for calculating the degree of separation described in Patent Document 1 will be described below.
For example, a pixel located at a predetermined coordinate (x, y) in a predetermined frame (image) is set as the target pixel. When the radius of the moving object is “ro” and the swing range of the moving object is “r”, the coordinates (x−2r, y−2r0) and the coordinates (x−2r, y + 2r0) are centered on the target pixel. ), Coordinates (x + 2r, y−2r0), and coordinates (x + 2r, y + 2r0), a rectangular area having apexes at four points is set as the entire area A0. Further, in the entire area A0, a rectangular area having coordinates (x−r, y−r0), coordinates (x−r, y + r0), coordinates (x + r, y−r0), and coordinates (x + r, y + r0) as vertices. Is set as the first area A1, and the area other than the first area A1 is set as the second area A2.
Here, for the entire area A0, the number of pixels is described as “N”, each pixel value (luminance value) is “Pi”, and the average pixel value is “Pmave”, and the number of pixels for the first area A1. When “n1” is described as “P1ave” and the pixel value average is described as “P1ave”, and for the second area A2, the number of pixels is described as “n2” and the pixel value average is “P2ave”. The pixel separation η is calculated by the following equation (1).

As the pixel value, for example, a specific color component or hue may be used instead of the luminance value.
The degree of separation η in Expression (1) indicates the ratio of the variation between the first region A1 and the second region A2 in the variation of the entire region A0, and is the maximum when the regions are best separated. It is supposed to take a value.
Here, the degree of separation η in equation (1) is a value normalized to a range of 0 to 1, and is useful in that it can be simply compared even when applied to regions of different sizes and shapes. is there.
For further details of the method, Patent Document 1 may be referred to.

  For each frame to be processed, all the pixels are sequentially set as the target pixel, and the degree of separation η is calculated each time, thereby generating a degree of separation map. That is, image data having each degree of separation η as each pixel value is generated as a degree of separation map corresponding to one frame.

For each frame data to be processed, the difference separation degree calculation unit 53 generates a so-called interframe difference value (correspondence between a separation degree map corresponding to a predetermined frame and a separation degree map corresponding to the immediately preceding frame). Difference of each degree of separation at the pixel position to be calculated). Note that image data obtained as a result of this calculation and having a difference value of the degree of separation as each pixel value is hereinafter referred to as a “difference degree of separation map”.
The reason why the difference separation degree map is calculated in this way is that the influence of a stationary object can be removed. In other words, when there is a stationary object (or an object having a design pattern, etc.) that is approximately the same size as the moving object in the background, the degree of separation in the area of the stationary object is high as in the area of the moving object. When the position of the ball is calculated based on the degree of separation, the position of the stationary object may be mistaken for the position of the ball. Therefore, by calculating the difference between the respective separability maps corresponding to adjacent frames, that is, by generating the differential separability map, the difference value in the area of the stationary object becomes approximately 0. It becomes possible to remove the influence of.
Note that Patent Document 1 may be referred to for further details of the difference in the degree of separation.

The club speed calculation unit 54 calculates the speed of the golf club using the difference separation degree map corresponding to each frame to be processed using the golf club as a moving object.
That is, the difference in the degree of separation at each pixel position represented by the difference degree of separation map relates to the probability that the moving object (here, a golf club) at each pixel position from which the influence of the stationary object in the background image has been removed exists. Represents an evaluation value.
Therefore, the club speed calculation unit 54 accumulates the evaluation value within the region where the movement of the golf club is predicted, and calculates the golf club speed based on the accumulated value.
For further details of the method of calculating the speed of the moving object (here, the golf club), Patent Document 1 may be referred to.

  In this way, when the speed calculation process for the pre-shot portion is executed by the image reading unit 51, the separation degree calculation unit 52, the difference separation degree calculation unit 53, and the club speed calculation unit 54, As the processing, as described above, the ball detection processing after the shot is executed, and the processing for calculating the velocity of the ball is executed.

  When the ball detection process after a shot is executed, among the functional blocks in FIG. 2, in particular, the image reading unit 51, the separation degree calculation unit 52, the difference separation degree calculation unit 53, the index value calculation unit 55, and the frame-out image specification The unit 56, the threshold setting unit 57, and the ball detection unit 58 function.

The image reading unit 51 processes moving image data in a predetermined section in which the state after the shot is reflected, for example, data of 15 frames immediately after the shot frame, among the moving image data stored in the target image storage unit 62. Read.
Thereafter, the separation degree calculation unit 52 calculates a separation degree map for each processing target frame data, and the difference separation degree calculation unit 53 calculates a difference separation degree map.

  The index value calculation unit 55 uses a value obtained by accumulating the difference separation degree (evaluation value) in the region where the golf ball is expected to move in the difference separation degree map corresponding to each frame to be processed. Is calculated as a value indicating an accuracy index (hereinafter referred to as “index value”).

式（２）において、Ｄ［ｘｉ，ｙｊ］は、処理対象の所定フレームにおける座標（ｘｉ，ｙｊ）に位置する画素についての、差分分離度を示している。
即ち、処理対象の所定フレームについて、ｎ個（領域ＣＡのＸ方向の画素数分）の指標値Ｔ［ｘ１］乃至Ｔ［ｘｎ］が算出される。つまり、本実施形態では、処理対象の１５フレーム毎に、ｎ個の指標値Ｔ［ｘ１］乃至Ｔ［ｘｎ］が夫々算出される。 FIG. 3 shows an example of a difference separation degree map corresponding to a predetermined frame to be processed for explaining a technique for calculating an index value.
For example, in the difference separation degree map FT corresponding to a predetermined frame to be processed, the lower right area CA is an area where a golf ball is expected to move. This area CA is a rectangular area having apexes at four points of coordinates (x1, y1), coordinates (xn, y1), coordinates (x1, ym), and coordinates (xn, ym). Here, n is an integer value of 1 or more. That is, the length of the long side of the area CA (the length in the X-axis direction) is a length corresponding to n pixels. Specifically, for example, in this embodiment, n is assumed to be 4/5 of the total number of pixels in the horizontal line. On the other hand, m is an integer value of 1 or more and is an integer value independent of n. That is, the length of the short side (the length in the Y-axis direction) of the area CA is set to m pixels. Specifically, for example, in the present embodiment, m is assumed to be 1/2 of the total number of pixels in the vertical line.
In this case, the index value calculation unit 55 calculates the cumulative addition value of the difference separation degree in the Y-axis direction for each X coordinate of the area CA in the difference separation degree map FT corresponding to the predetermined frame to be processed. Calculate as a value.
That is, the index value T [xi] at the X coordinate xi (i is any integer value from 1 to n) is calculated by the following equation (2).

In Expression (2), D [xi, yj] represents the difference separation degree for the pixel located at the coordinates (xi, yj) in the predetermined frame to be processed.
That is, n index values T [x1] to T [xn] (for the number of pixels in the X direction of the area CA) are calculated for a predetermined frame to be processed. That is, in this embodiment, n index values T [x1] to T [xn] are calculated for every 15 frames to be processed.

Returning to FIG. 2, based on the calculation result of the index value calculation unit 55, the frame-out image specifying unit 56 results from the movement (flight) of the golf ball from the 15 frames to be processed to the outside of the imaging range of the camera. The frame in which the ball is no longer included is specified as a frame-out image.
Specifically, in the present embodiment, the frame-out image specifying unit 56 specifies a frame in which index values T [x1] to T [xn] are greater than or equal to a threshold value from 15 frames to be processed. To do.
Here, the threshold value is arbitrarily set based on an index value with which the accuracy of the presence of the ball becomes a certain level or more. The threshold value is referred to as a “first threshold value” in order to clearly distinguish it from other threshold values described later. That is, when there is an index value T [xi] that is greater than or equal to the first threshold in the frame of interest, the position of the X coordinate xi of the area CA in the frame of interest (any of Y coordinates y1 to ym in the Y-axis direction). Position) means that the ball is positioned with a certain degree of accuracy.
Therefore, if a golf ball moves relatively from left to right in a moving image, the ball is moving within the imaging range of the camera (the frames in which the state is shown are continuous). The x coordinate xi of the index value T [xi] equal to or greater than the first threshold value should have shifted to the right with respect to the immediately preceding frame. Then, when it becomes a frame at a stage where the ball has moved out of the imaging range of the camera, that is, a frame-out image, the index value T [xi] greater than or equal to the first threshold value should not suddenly exist.
Therefore, when the frame-out image specifying unit 56 specifies a frame in which the index value T [xi] is equal to or greater than the first threshold from the 15 frames to be processed, the index equal to or greater than the first threshold is set for the subsequent frames. When the value T [xi] is tracked and moves to the right and suddenly disappears, the frame at that time is identified as a frame-out image.
The specific detailed processing of the frame-out image will be described later with reference to the flowchart of FIG.

The threshold value setting unit 57 variably sets the first threshold value described above and notifies the frame-out image specifying unit 56 as appropriate. In addition, the threshold value setting unit 57 uses a threshold value used in a ball detection unit 58 described later (to be clearly distinguished from the first threshold value below. The second detection threshold value ”is variably set and notified to the ball detection unit 58 as appropriate.
The method for variably setting the threshold is not particularly limited, but in the present embodiment, a method is adopted in which the threshold is set based on the brightness of each data of the moving image stored in the target image storage unit 62. . Accordingly, it is possible to set an appropriate threshold value according to the moving image stored in the target image storage unit 62, that is, according to the processing target.

The ball detection unit 58 detects the balls in reverse chronological order using the frame immediately before the frame-out image, that is, the frame in which the ball is considered to be reflected last (hereinafter referred to as “terminal image”) as a base point. .
Hereinafter, an example of a ball detection method will be described with reference to FIGS. 4 and 5 as appropriate.

FIG. 4 is a diagram for explaining a ball detection technique, and shows an example of a terminal image.
The ball detection unit 58 detects pixels having a difference separation degree equal to or higher than the second threshold in the area CA of the terminal image FF.
Here, the second threshold value is arbitrarily set independently from the first threshold value by the threshold value setting unit 57 based on the degree of separation at which the accuracy of the presence of the ball becomes a certain level or more. That is, a pixel having a difference separation degree equal to or greater than the second threshold in the area CA of the terminal image FF is a pixel that shows a ball (at least a part thereof) with a certain accuracy or more, that is, the last image in the moving image. ) Means a pixel candidate for the ball.
Therefore, the ball detection unit 58 firstly selects each of one or more pixels having a difference separation degree equal to or greater than the second threshold from the area CA of the terminal image FF (which is the last in the moving image). Is detected as a pixel candidate. In the example of FIG. 4, the white area in the area CA has a difference separation degree equal to or greater than the second threshold value, so that each pixel in the white area is detected.

Next, the ball detection unit 58 traces each frame to be processed in the reverse time series starting from the candidate for the pixel of the ball in the terminal image FF (shown last in the moving image) in the reverse time series. By tracking pixel candidates, ball trajectory candidates are detected.
In this case, the range of detection by the ball detection unit 58 may be the entire frame, but the processing amount increases as the range increases, requiring processing time. Therefore, in the present embodiment, for the purpose of reducing the processing amount and the processing time, as shown in FIG. 5, the parallelogram region that is likely to include the trajectory of the ball movement is a ball. It is adopted as an area of a detection range by the detection unit 58 (hereinafter referred to as “detection area”).
FIG. 5 is a diagram for explaining a ball detection method and illustrates an example of a detection region.
In the example of FIG. 5, the candidate for the pixel of the ball in the terminal image FF (shown last in the moving image) is taken as the starting point and extended toward the initial position of the ball (the position of the ball in the shot frame). The parallelogram area is set as the detection area. Here, the length a is set by adding the distance from the starting point to the initial position and the size of the ball (relative size in the frame). The length b is set according to the size of the ball.
The reason why the parallelogram is adopted as the detection region in the present embodiment is as follows. That is, since the ball is predicted to move (fly) in a straight line from the point (initial position) where it was initially placed, the point designated as having the ball in the end image (the candidate point of the ball pixel) ) To the initial position of the ball, the ball is detected. However, since it is rare that the trajectory of the ball is in a straight line without any deviation in practice, a parallelogram having a length a (height a) about the diameter of the ball is adopted as the detection region. It is.

Next, the ball detection unit 58 considers a combination of a plurality of line segments passing through the range of the parallelogram detection area, and sequentially sets each of the plurality of line segments belonging to the combination as a target line segment. The ball detection unit 58 cumulatively adds the difference separation degrees so as to be spatially equidistant while tracing each frame to be processed in reverse time series on the attention line segment, and obtains the cumulative addition value. .
That is, one cumulative addition value is obtained for each of a plurality of line segments belonging to the combination within the parallelogram detection area.
Here, cumulative addition of the difference separation degree so as to be spatially equidistant means the following calculation. That is, for example, the first difference separation degree for the first frame is added, and the second difference separation for the second frame (the frame immediately before the first frame in the reproduction order) is added. It is assumed that the degree is added, and further, the third difference separation degree for the third frame (one frame before the second frame in the reproduction order) is added. In this case, the first pixel (within the first frame) having the first difference separation degree, the second pixel (within the second frame) having the second difference separation degree, and the third pixel having the third difference separation degree. (In the third frame) is points arranged at equal intervals on the line of interest. That is, the corresponding point of the point (pixel in the previous frame) on the attention line segment in which the difference separation degree is cumulatively added in the previous frame (the next frame in the reproduction order) is the pixel in the current frame. In the current frame, the difference separation degree is cumulatively added at points (pixels in the current frame) that are spaced to the lower left side at equal intervals with respect to the corresponding point.
In this case, the number divided at equal intervals on the line of interest is variably set according to the number of images from the terminal image to the shot image, and the width thereof is a shot executed by the club speed calculation unit 54 in advance. It is set according to the result of the previous speed calculation process. That is, the width is variably set according to the speed of the ball calculated from the speed (movement) of the golf club.
The ball detection unit 58 identifies a line segment that is the maximum cumulative addition value among a plurality of line segments within the parallelogram detection area as a line segment that indicates a candidate for the trajectory of the ball.
Here, there may be a plurality of pixels that are candidates for the ball in the terminal image. In this case, one parallelogram detection area is set for each of the plurality of pixels (for each candidate for the ball). Is done. Therefore, one line segment indicating a candidate for the trajectory of the ball is specified for each of the plurality of parallelogram detection areas.
Therefore, the ball detection unit 58 identifies the line segment that is the maximum cumulative addition value among the line segments that indicate candidate ball trajectories for each of the plurality of parallelogram detection areas as the ball trajectory.

When the ball speed calculation process is executed in this way, in the CPU 11 or the image processing unit 14 shown in FIG. 2, the image reading unit 51, the separation degree calculation unit 52, the difference separation degree calculation unit 53, and the club The velocity calculation unit 54, the index value calculation unit 55, the frame-out image specifying unit 56, the threshold setting unit 57, the ball detection unit 58, and the ball speed calculation unit 59 function to calculate the ball trajectory. The
When the trajectory of the ball is calculated, the ball speed calculation unit 59 functions in the CPU 11 or the image processing unit 14 shown in FIG.
The ball speed calculation unit 59 calculates the speed of the ball based on the calculation result of the ball trajectory.

Next, with reference to FIG. 6, a ball speed calculation process will be described as details of the operation of the imaging apparatus 1 having the functional configuration of FIG.
FIG. 6 is a flowchart for explaining the flow of the ball speed calculation process executed by the image processing apparatus 1 having the functional configuration of FIG.
The ball speed calculation process is started when a predetermined condition is satisfied in a state in which one or more moving image data of a golf form is recorded in the moving image storage unit 61 in FIG. 2, and the following series of processes is executed. Is done.

  In step S <b> 1, the image reading unit 51 determines and reads a processing target from the moving image data of the golf form stored in the moving image storage unit 61, and stores the processing target in the target image storage unit 62.

In step S <b> 2, the image reading unit 51 through the club speed calculation unit 54 execute the speed calculation process for the pre-shot portion of the moving image data to be processed.
Details of the speed calculation process for the pre-shot portion will be described later with reference to FIG.

In step S <b> 3, the image reading unit 51 to the difference separation degree calculation unit 53 and the index value calculation unit 55 to the ball detection unit 58 execute a ball detection process for the post-shot portion on the moving image data to be processed.
Details of the ball detection process for the post-shot portion will be described later with reference to FIGS.

In step S4, the ball speed calculation unit 59 calculates the speed of the ball appearing in the moving image to be processed based on the result of the ball detection process for the post-shot portion.
As a result, the ball speed calculation process ends.

Next, with reference to FIG. 7, the details of the speed calculation process of the pre-shot portion in step S2 in the ball speed calculation process of FIG. 6 will be described.
FIG. 7 is a flowchart for explaining the detailed flow of the speed calculation process for the pre-shot portion in step S2 in the ball speed calculation process of FIG.
The speed calculation processing of the pre-shot portion in step S2 is started when the moving image data to be processed is determined and stored in the target image storage unit 62 in step S1, and the following series of processing is executed. .

  In step 21, the image reading unit 51 reads each data from the target image storage unit 62 with the four frames immediately before the shot frame as processing targets.

  In step 22, the degree-of-separation calculating unit 52 generates a degree-of-separation map from the luminance image to be processed. That is, one separation degree map is generated for every four frames immediately before the shot frame.

  In step 23, the difference separation degree calculation unit 53 generates a difference separation degree map from the separation degree map. That is, one difference separation degree map is generated every four frames immediately before the shot frame.

In step 24, the club speed calculation unit 54 calculates the speed of the golf club immediately before the shot, as described above, from the difference separation degree map.
As a result, the speed calculation process for the pre-shot portion ends, that is, the process of step S2 in FIG. 6 ends, and the process proceeds to step S3.

Next, details of the ball detection process for the post-shot portion in step S3 in the ball speed calculation process of FIG. 6 will be described.
FIG. 8 is a flowchart for explaining the detailed flow of the ball detection process for the post-shot portion in step S3 in the ball speed calculation process of FIG.
The ball detection process for the post-shot part in step S3 is started when the speed calculation process for the pre-shot part in step S2 is completed as described above, and the following series of processes is executed.

  In step 41, the image reading unit 51 reads each data from the target image storage unit 62 with 15 frames immediately after the shot frame as a processing target.

  In step 42, the separation degree calculation unit 52 generates a separation degree map from the luminance image to be processed. That is, one separation degree map is generated every 15 frames immediately after the shot frame.

  In step 43, the difference separation degree calculation unit 53 generates a difference separation degree map from the separation degree map. That is, one difference separation degree map is generated every 15 frames immediately after the shot frame.

  In step 44, the index value calculation unit 55 performs the process in the Y-axis direction for each of the 15 frames to be processed for each X coordinate of the predetermined area (in the example of FIG. 3, for each of the X coordinates x1 to Xn of the area CA). The cumulative addition value of the difference separation degree is calculated as an index value. For example, in the example of FIG. 3, the above equation (2) is calculated, and n index values T [x1] to T [xn] are calculated for each frame.

  In step 45, the frame-out image specifying unit 56 sets the first frame among the 15 frames to be processed, that is, the frame next to the shot frame, as the frame of interest.

In step 46, the frame-out image specifying unit 56 determines whether or not there are n index values in the frame of interest that are greater than or equal to the first threshold value.
If none of the n index values of the frame of interest is greater than or equal to the first threshold value, it is determined that the moving ball is not yet included in the predetermined area of the frame of interest, and in step S46 If it is determined as NO, the process proceeds to step S47. In step S47, the frame-out image specifying unit 56 sets the next frame as the frame of interest. Thereby, the process returns to step S46, and the subsequent processes are repeated.
That is, each of the 15 frames to be processed immediately after the shot frame is sequentially set to the frame of interest in the chronological order (forward playback order), and it is sequentially determined whether or not an index value equal to or greater than the first threshold exists. It will be done.
When a frame having an index value equal to or greater than the first threshold value becomes a target frame, it is determined that the moving ball of the target frame is included, it is determined as YES in step S46, and the process proceeds to step S48. .

  In step 46, the frame-out image specifying unit 56 determines whether or not the n index values of the next frame after the frame of interest are greater than or equal to the first threshold value.

If there is no n index value in the next frame of the frame of interest that is greater than or equal to the first threshold value, the predetermined area of the frame of interest contains a moving ball. In the predetermined area of the next frame, it is determined that the moving ball is not included (has gone out), it is determined NO in step S48, and the process proceeds to step S51.
In step S51, the frame-out image specifying unit 56 specifies a frame next to the frame of interest as a frame-out image.
As a result, the process proceeds to the ball detection process in the reverse chronological order in step S52. However, the processing after step S52 will be described later with reference to FIG.

On the other hand, when at least one of the n index values in the frame following the frame of interest has a value equal to or greater than the first threshold, the moving ball is included in the frame following the frame of interest. If it is determined that the answer is YES in step S48, the process proceeds to step S49.
In step S49, the frame-out image specifying unit 56 determines whether or not the X frame of the index value equal to or greater than the first threshold is on the right side of the next frame relative to the frame of interest.
When the X coordinate of the index value equal to or greater than the first threshold is on the right side of the next frame relative to the frame of interest, it is shot in the right direction when shot with a golf club in both the frame of interest and the next frame. Is included when the ball continues to move, that is, the next frame is not yet a frame-out image.
In such a case, it is determined as YES in Step S49, and the process proceeds to Step S50. In step S50, the frame-out image specifying unit 56 sets the next frame as the frame of interest. Thereby, the process returns to step S48, and the subsequent processes are repeated.
That is, each of the processing targets after it is determined that a ball is included in a predetermined area (an index value greater than or equal to the first threshold value) is sequentially set to the frame of interest in chronological order (forward playback order). Then, it is sequentially determined whether or not there is an index value greater than or equal to the first threshold value. If the X coordinate of the index value equal to or greater than the first threshold (the X coordinate calculated when the ball is present) continues to the right of the next frame compared to the frame of interest, the frame of interest and the next It is determined that any of the frames includes a ball that has been shot with a golf club and continues to move in the right direction, and the loop processing of steps S48 to S50 is repeatedly executed.

If the index value equal to or greater than the first threshold no longer exists in the frame following the frame of interest, or if the index value equal to or greater than the first threshold exists but does not exist to the right of the frame of interest, step S48. Or it determines with it being NO in step S49.
Here, in the case where it is determined as NO in step S48, as described above, in the predetermined area of the target frame, the moving ball was included, whereas in the predetermined area of the next frame, , Which means that it is determined that the moving ball is not included (being out of the outside). Further, when it is determined as NO in step S49, the index value equal to or higher than the first threshold included in the predetermined area of the next frame of the frame of interest is not for the moving ball (the ball is abruptly It is about noise and other moving objects, and as a result, it is judged that the moving ball is no longer included in the predetermined area of the next frame. Means when
Therefore, if it is determined as NO in step S48 or step S49, the process proceeds to step S51.
In step S51, the frame-out image specifying unit 56 specifies a frame next to the frame of interest as a frame-out image.

  In step S52, the ball detection unit 58 performs a ball detection process in reverse time series.

Next, details of the ball detection processing in the reverse time series of step S52 in the ball detection processing in the post-shot portion of FIG. 8 will be described.
FIG. 9 is a flowchart for explaining the detailed flow of the ball detection process in the reverse time series of step S52 in the ball detection process for the post-shot part of FIG.
The reverse time-series ball detection process in step S52 is started when a frame-out image is specified in step S51 as described above, and the following series of processes is executed.

In step S <b> 61, the ball detection unit 58 determines whether there is a pixel that has a difference separation degree higher than the second threshold in the terminal image and is not set as the target pixel.
As described above, a pixel whose degree of separation is higher than the second threshold in the terminal image is a pixel candidate indicating a ball (at least a part thereof) in the terminal image (hereinafter referred to as a “ball pixel candidate”). It means that.
Therefore, the case where there is no pixel having a difference separation degree higher than the second threshold in the terminal image means a case where there is no ball pixel candidate. In this case, it is determined as NO in Step S61, and the ball detection process in the reverse chronological order ends. That is, the ball detection process itself after the shot in FIG. 8 is also terminated, and the process proceeds from step S3 to S4 in FIG.

On the other hand, if there is one or more pixels having a difference separation degree higher than the second threshold in the terminal image, it is determined as YES in Step S61, and the process proceeds to Step S62.
In step S <b> 62, the ball detection unit 58 sets a pixel that has not been set as a target pixel among pixels whose difference separation degree is higher than the second threshold in the terminal image as a target pixel. That is, a predetermined one ball pixel candidate is set as a target pixel.

In step S63, the ball detection unit 58 sets a parallelogram detection region starting from the target pixel.
In step S64, the ball detection unit 58 sequentially traces a plurality of frames to be processed in reverse time series, and finds a combination of a line segment and an interval that maximizes the cumulative addition value of the equidistant separation within the detection region. calculate.
That is, the combination calculated in step S64 means a combination having the highest accuracy as a ball trajectory candidate when the target pixel is a ball pixel candidate.

  In step S65, the ball detection unit 58 determines whether or not the cumulative addition value of the combination calculated in the previous step S64 is larger than the cumulative addition value of the stored combination.

The case where the cumulative addition value of the combination calculated in the immediately preceding step S64 is smaller than the cumulative addition value of the stored combination is the ball obtained when other pixels in the terminal image are ball pixel candidates. The trajectory candidate (the ball trajectory candidate specified by the stored combination) is the ball trajectory obtained in the immediately preceding step S64 when the current pixel of interest is the ball pixel candidate. This means that the accuracy is higher than the candidate (candidate for the locus of the ball specified by the combination calculated in the immediately preceding step S64).
Therefore, in this case, since the combination that has already been stored is adopted as the candidate for the trajectory of the ball, it is determined as NO in step S65, the process returns to step S61, and the subsequent processes are repeated. . That is, the process of steps S61 to S66 is executed for another ball pixel candidate in the terminal image, and a ball locus candidate is obtained.

On the other hand, the case where the cumulative addition value of the combination calculated in the immediately preceding step S64 is smaller than the cumulative addition value of the stored combination is the case where the current pixel of interest is the ball pixel candidate. In the ball trajectory candidate obtained in the process of step S64 (candidate of the ball trajectory specified by the combination calculated in the immediately preceding step S64), other pixels in the terminal image are set as ball pixel candidates. This means a case where the accuracy is higher than the ball trajectory candidate (candidate ball trajectory specified by the stored combination) obtained at the time.
Therefore, in this case, as a candidate for the trajectory of the ball, in order to be updated to the combination calculated in step S64 immediately before this time, it is determined as YES in step S65, and the process proceeds to step S66.
In step S65, the ball detection unit 58 stores (overwrites) the combination calculated in the immediately preceding step S64 in association with the accumulated addition value.
Thereby, a process is returned to step S61 and the process after it is repeated.

As described above, the imaging apparatus 1 according to the present embodiment includes the frame-out image specifying unit 56 and the ball detection unit 58.
The frame-out image specifying unit 56 specifies an image immediately before the frame-out image (for example, the above-described end image) from which the ball, which is an example of the object to be measured, does not exist, from each of the continuously captured images.
The ball detection unit 58 detects the region of the measurement target object from each of the continuously captured images with reference to the image immediately before the frame-out image.
In this way, since the measurement target object is detected based on the frame-out image in which the measurement target object does not exist, the measurement target object can be accurately obtained even when moving objects other than the measurement target object exist in the image. The effect that can be detected can be achieved.
In particular, this effect becomes significant when the present invention is applied under the following conditions.
For example, when there is another object that moves at the same speed, size, or separation as a golf club, when a variety of lighting conditions are imposed such as a dim night or a bright day, a golf driving range, etc. The above-mentioned effect becomes remarkable when the shot analysis is performed with the objective.
Also, for example, when the golf club and ball velocities vary from shot to shot and cannot be separated simply by a threshold, etc., the golf club trajectory is an ellipse, but the movement of several frames after the shot is almost linear. When the circular motion and the linear motion cannot be separated, the above-described effect becomes remarkable.
Furthermore, the following effects are also achieved. In other words, the ball trajectory, which was difficult with existing algorithms and simple threshold settings, can be accurately tracked with a single algorithm, so the user does not need to make settings in advance and can easily know the speed and trajectory. be able to. Further, since the same image as that used for form analysis or the like can be used, space saving and cost saving of the entire system can be realized.

Further, the ball detection unit 58 can detect the region of the object to be measured from the image before the frame out immediately before the region of the object to be measured in reverse order of the time series.
As described above, the objects to be measured are detected in reverse time-series order, so that the detection accuracy of the objects to be measured is improved.
That is, if shot frames are detected in time series as a reference, the ball and the golf club are erroneously recognized, and the ball detection itself often fails. In particular, as the time elapses after the ball is hit by the golf club, the distance between the golf club and the ball increases, so the degree of erroneous recognition increases. On the other hand, in the present embodiment, conversely, the object candidate to be measured is detected from a predetermined image before the frame-out image in which the distance between the golf club and the ball is the longest, and the golf club Since the ball is detected in the direction in which the distance to the ball is shortened, the degree of erroneous recognition of the measurement target object is reduced, and as a result, the detection accuracy of the measurement target object is improved.

In addition, the ball detection unit 58 of the imaging apparatus 1 according to the present embodiment generates a predetermined area (the detection area in FIG. 5 in the above example) in which the locus of the object can be included based on the detected object area. It has a generation means. And the imaging device 1 of this embodiment calculates the locus | trajectory of an object and / or the speed of the said object based on the predetermined area | region produced | generated by the production | generation means.
As a result, the detection accuracy of the object to be measured is further improved, and the detection range is limited, so that the processing load thereafter is reduced and the processing time is shortened.

In addition, the imaging apparatus 1 according to the present embodiment further includes a degree-of-separation calculation unit 52 that calculates the degree of separation of each image captured continuously. Further, the frame-out image specifying unit 56 includes a determination unit that determines whether or not the calculated degree of separation is greater than or equal to a threshold value (second threshold value in the above example).
The ball detection unit 58 detects an area determined by the determination unit as being equal to or greater than a predetermined threshold as an object area.
Thereby, the detection accuracy of the object to be measured is further improved. That is, the detection accuracy when detection is performed in reverse time-series order depends on the detection accuracy of the object candidate to be measured from the predetermined image before the reference frame-out image. That is, in order to improve the detection accuracy of the object to be measured, the key is how to accurately detect the target object candidate from the predetermined image. In this respect, in the predetermined image, an area having a degree of separation equal to or greater than the threshold is detected as a candidate for the object to be measured, so that the detection accuracy of the candidate is improved, and as a result, the detection accuracy of the object to be measured Is further improved.

In addition, the imaging apparatus 1 of the present embodiment can further include a threshold setting unit 57 that variably sets the threshold (second threshold in the above example) based on the images that are continuously captured.
As a result, if an appropriate threshold value is set, the detection accuracy of the object to be measured is further improved from the predetermined image before the reference frame-out image, and as a result, the object to be measured is detected. The accuracy is further improved.
Furthermore, if the threshold value is set strictly, the number of pixels detected as a candidate for the object to be measured is reduced by that amount, reducing the processing load thereafter and shortening the processing time. Connected.

Further, the threshold setting unit 57 can variably set the threshold based on the luminance of each image captured continuously.
Thereby, a more appropriate threshold value can be set.

Further, the ball detection unit 58 of the imaging apparatus 1 of the present embodiment includes an area setting unit that sets a partial area common to each image (the detection area in FIG. 5 in the above example). The ball detection unit 58 detects the area of the object in the partial area set by the area setting means.
As a result, the detection accuracy of the object to be measured is further improved, and the detection range is limited, so that the processing load thereafter is reduced and the processing time is shortened.

  In addition, this invention is not limited to the above-mentioned embodiment, The deformation | transformation in the range which can achieve the objective of this invention, improvement, etc. are included in this invention.

In the above-described embodiment, the second threshold value for detecting the candidate object to be measured (the ball in the above example) in the terminal image is set based on the luminance of the moving image to be processed. It is not limited to this.
For example, the second threshold value may be set based on the luminance of a moving image within a predetermined section instead of the luminance of the entire moving image, or set based on the luminance of a partial area of the frame instead of the entire frame. May be.
Further, for example, since the speed of the golf club immediately before the shot is calculated by the speed calculation process of the portion before the shot (see FIG. 7), the second threshold value may be set based on the speed of the golf club.
Further, for example, the second threshold value is set based on the degree of separation having a distribution of the upper few percent (about the number of pixels of the ball in the frame) in the whole degree separation map (or the difference separation degree map). May be.

In the above-described embodiment, the detection area in which the balls are detected in the order of the reverse time series is the parallelogram area generated based on the ball pixel candidates of the terminal image, but is not particularly limited thereto.
For example, since the speed of the golf club immediately before the shot is calculated by the speed calculation process of the portion before the shot (see FIG. 7), an area that varies according to the speed of the golf club is employed as the detection area. May be. Specifically, for example, an area that is variably set for each image (frame) from each speed of the golf club and the ball after the shot assumed from the speed of the club immediately before the shot may be adopted as the detection area. Good.
In addition, for example, since the ball is considered to move (fly) in a straight line from the point at which it was initially placed, the line from the point (ball pixel candidate) where the ball appears to be in the terminal image to the initial position of the ball The minute itself may be adopted as the detection area. However, since it is rare that the ball is actually on the line segment without any deviation at all, a parallelogram having a height about the diameter of the ball (length b in FIG. 5) as in the above-described embodiment. It is preferable that the above shape is adopted as the detection region.

In the above-described embodiment, the frame-out image is specified during the execution of the speed calculation process for the pre-shot portion and during the ball detection process for the post-shot portion, but the specific timing is not particularly limited.
For example, the frame-out image may be specified every time the difference separation degree map is generated. In this case, it contributes to speeding up of processing and saving of used memory.

In the above-described embodiment, the degree of separation is used as the evaluation value for estimating the presence of the ball in the speed calculation process for the pre-shot part and the ball detection process for the post-shot part. However, the present invention is not particularly limited to this.
For example, not the calculated degree of separation itself but a degree of separation processed as necessary may be used. Specifically, for example, when the ball passes through an area where the background is overexposed, the brightness of the overexposed portion is higher than the color of the ball, so the sign of the degree of separation is reversed. In such a region, a degree of separation with the sign reversed (processed as such) may be used.
As an example other than the degree of separation, for example, a matching ratio obtained by normalizing a sum of squared differences by the number of area pixels by template matching or the like may be used.

  In the above-described embodiment, the measurement target object is a ball, but the measurement amount is a speed, but is not particularly limited thereto. As the measurement target, any object can be adopted as long as it is a moving object reflected in a moving image, and any measurement object can be adopted as long as it is an amount related to movement.

In the above-described embodiment, the image processing apparatus to which the present invention is applied has been described using a digital camera as an example, but is not particularly limited thereto.
For example, the present invention can be applied to general electronic devices having an image processing function. Specifically, for example, the present invention can be applied to a notebook personal computer, a printer, a television receiver, a video camera, a portable navigation device, a mobile phone, a smartphone, a portable game machine, and the like.

The series of processes described above can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 2 is merely an example and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the above-described series of processing as a whole, and what functional blocks are used to realize this function is not particularly limited to the example of FIG.
In addition, one functional block may be constituted by hardware alone, software alone, or a combination thereof.

When a series of processing is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer incorporated in dedicated hardware. The computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose personal computer.

  The recording medium including such a program is not only constituted by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body in order to provide the program to the user, but also in a state of being incorporated in the apparatus main body in advance. It is comprised with the recording medium etc. which are provided in this. The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Read Only Memory), a DVD (Digital Versatile Disk), or the like. The magneto-optical disk is configured by an MD (Mini-Disk) or the like. In addition, the recording medium provided to the user in a state of being preliminarily incorporated in the apparatus main body includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 20 in FIG.

In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series along the order, but is not necessarily performed in time series, either in parallel or individually. The process to be executed is also included.
Further, in the present specification, the term “system” means an overall apparatus configured by a plurality of devices, a plurality of means, and the like.

  As mentioned above, although several embodiment of this invention was described, these embodiment is only an illustration and does not limit the technical scope of this invention. The present invention can take other various embodiments, and various modifications such as omission and replacement can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are included in the invention described in the claims and the equivalents thereof.

The invention described in the scope of claims at the beginning of the filing of the present application will be appended.
[Appendix 1]
A specifying means for specifying an image immediately before the frame out of the object to be measured from each of the continuously captured images;
Detecting means for detecting an area of the object to be measured from each of the continuously captured images with reference to the image immediately before the frame-out specified by the specifying means;
An image processing apparatus comprising:
[Appendix 2]
The detection means includes
The image processing apparatus according to appendix 1, wherein an area of the object to be measured is detected in an order reverse to time series from an image before the image immediately before the frame out.
[Appendix 3]
Generating means for generating a predetermined area that can include a trajectory of the object based on the area of the object detected by the detecting means;
The image processing apparatus according to appendix 2, further comprising: a calculation unit that calculates a trajectory of the object and / or a velocity of the object based on the predetermined region generated by the generation unit.
[Appendix 4]
A degree-of-separation calculating means for calculating the degree of separation of each of the continuously captured images;
Determination means for determining whether the degree of separation calculated by the degree of separation calculation means is equal to or greater than a threshold;
Further comprising
The detection means detects an area determined by the determination means as being equal to or greater than a predetermined threshold as the area of the object.
The image processing apparatus according to any one of supplementary notes 1 to 3, wherein:
[Appendix 5]
Threshold setting means for variably setting the threshold value based on the continuously captured images.
The image processing apparatus according to appendix 4, wherein
[Appendix 6]
The threshold value setting means variably sets the threshold value based on the luminance of each of the continuously captured images;
The image processing apparatus according to appendix 5, characterized in that:
[Appendix 7]
An area setting means for setting a partial area common to the images;
The detection means detects the area of the object in the partial area set by the area setting means;
The image processing apparatus according to any one of supplementary notes 1 to 6, wherein:
[Appendix 8]
An image processing method executed by an image processing apparatus,
A specific step of identifying an image immediately before the frame out of the object to be measured from each of the continuously captured images;
A detection step of detecting a region of the object to be measured from each of the images with reference to the image immediately before the frame-out specified in the specifying step;
An image processing method comprising:
[Appendix 9]
Computer
A specifying means for specifying an image immediately before the frame-out of the object to be measured from each of the continuously captured images;
Detecting means for detecting a region of the object to be measured from each image based on the image immediately before the frame-out specified by the specifying means;
A program characterized by functioning as

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Image processing part, 17 ... Imaging part, 18 ... Input part, 19 ... -Output unit, 20 ... Storage unit, 21 ... Communication unit, 22 ... Drive, 31 ... Removable media, 51 ... Image reading unit, 52 ... Separation degree calculation unit, 53 .. Difference separation degree calculation unit, 54... Club speed calculation unit, 55... Index value calculation unit, 56... Frame-out image specifying unit, 57. Unit, 59... Ball speed calculation unit, 61... Moving image storage unit, 62.

Claims (9)

A degree-of-separation calculating means for calculating the degree of separation of a plurality of images captured continuously;
A degree-of-separation difference value calculating means for calculating a difference value between the plurality of images of the degree of separation calculated by the degree-of-separation calculating means ;
A specifying unit for specifying an image immediately before the frame-out of the object to be measured based on the separation degree difference value calculated by the separation degree difference value calculating unit from the plurality of images captured continuously;
Detecting means for detecting a region of the object to be measured from each of the plurality of images taken continuously with reference to the image immediately before the frame-out specified by the specifying means;
An image processing apparatus comprising: The detection means includes
From the image before the image immediately before the frame out, the region of the object to be measured is detected in the reverse order of the time series of the region of the object to be measured.
The image processing apparatus according to claim 1. Generating means for generating a predetermined area that can include a trajectory of the object based on the area of the object detected by the detecting means;
Calculation means for calculating the trajectory of the object and / or the speed of the object based on the predetermined region generated by the generation means;
The image processing apparatus according to claim 2, further comprising: Determination means for determining whether the degree of separation calculated by the degree of separation calculation means is equal to or greater than a threshold;
Further comprising
The detection means detects an area determined by the determination means as being equal to or greater than a predetermined threshold as the area of the object.
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. Threshold setting means for variably setting the threshold value based on the continuously captured images.
The image processing apparatus according to claim 4. The threshold value setting means variably sets the threshold value based on the luminance of each of the continuously captured images;
The image processing apparatus according to claim 5. An area setting means for setting a partial area common to the images;
The detection means detects the area of the object in the partial area set by the area setting means;
The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus. An image processing method executed by an image processing apparatus,
A degree-of-separation calculating step for calculating the degree of separation of a plurality of images captured continuously;
A separation degree difference value calculation step of calculating a difference value between the plurality of images of the separation degree calculated by the separation calculating step,
A specifying step for identifying an image immediately before the frame-out of the object to be measured based on the separation degree difference value calculated by the separation degree difference value calculation step from the plurality of images captured continuously;
A detection step of detecting a region of the object to be measured from each of the plurality of images taken continuously with reference to the image immediately before the frame-out specified in the specifying step;
An image processing method comprising: Computer
A degree-of-separation calculating means for calculating the degree of separation of a plurality of images captured continuously;
A degree-of-separation difference value calculating unit that calculates a difference value between the plurality of images of the degree of separation calculated by the degree-of-separation calculating unit;
A specifying unit for specifying an image immediately before the frame-out of the object to be measured based on the separation degree difference value calculated by the separation degree difference value calculation unit from the plurality of continuously captured images;
Detecting means for detecting an area of the object to be measured from each of the plurality of images taken continuously with reference to the image immediately before the frame-out specified by the specifying means;
A program characterized by functioning as

Images