WO2012128572A2

WO2012128572A2 - Virtual golf simulation device, and sensing device and method used in same

Info

Publication number: WO2012128572A2
Application number: PCT/KR2012/002076
Authority: WO
Inventors: 장형욱; 박현진; 윤형식; 정현담
Original assignee: (주) 골프존
Priority date: 2011-03-22
Filing date: 2012-03-22
Publication date: 2012-09-27
Also published as: TWI453056B; WO2012128572A3; TW201244790A; KR101078954B1

Abstract

The purpose of the present invention is to provide a virtual golf simulation device, and a sensing device and method used in same, wherein the virtual golf simulation device can present an image of a simulated trajectory of a ball, so as to sense a ball hit with a golf club by a user using a virtual golf simulation, by acquiring an image of the ball hit and extracting a trajectory of motion for the ball found in the image, and more specifically, can solve a problem of a difficult extraction of a ball due to the changing size of a ball on an image obtained from a camera, and thus easily and accurately extract the ball from the image, thereby accurately extracting a trajectory of the ball.

Description

Virtual golf simulation device, sensing device and sensing method used therein

The present invention relates to a virtual golf simulation apparatus and method, and a sensing apparatus and a sensing method used therein. More specifically, the trajectory of the golf ball hit by the user is simulated by the virtual golf course is simulated by the sensing device The present invention relates to a virtual golf simulation apparatus and method, and a sensing apparatus and a sensing method used therein, which are sensed and simulated in the virtual golf course.

Recently, various developments have been actively developed to enable popular sports games such as baseball, soccer, basketball, and golf to be enjoyed in the form of interactive sports games through simulation indoors or in specific places. It is done.

In particular, a so-called screen golf system has recently emerged. As a user swings a golf club with a golf club, the sensing device senses the physical characteristics of the moving golf ball by sensing the moving ball. By tracking the trajectory of the ball on the golf course, a technology has been developed to allow a user to enjoy golf in virtual reality.

In this interactive sports game, research and development on various sensing systems for accurately sensing the physical information about the movement of the ball in order to simulate the sports using the ball, such as golf balls, is very active. There is a situation.

For example, various sensing methods such as a sensing device using an infrared sensor, a sensing device using a laser sensor, a sensing device using an acoustic sensor, and a sensing device using a camera sensor are emerging. Research into a sensing device of a camera sensor type that acquires and analyzes an image of a ball has been actively conducted.

However, since the shooting area of the camera is fixed and the image of the ball moving in it is acquired, the size of the ball on the image is inevitably changed as the ball moves, so it is very difficult to distinguish the ball from other noises. When using a camera that does not have a high performance, it is difficult to find the ball trajectory on the image, which makes it difficult to accurately calculate the ball trajectory.

The present invention simulates the trajectory of the ball by acquiring an image of the user hitting the ball and finding the ball from the ball to calculate the motion trajectory in order to sense that the user using the virtual golf simulation hits the ball with the golf club. In order to solve the problem that it is difficult to extract the ball due to the change in the size of the ball on the image acquired from the camera, it is possible to extract the ball easily and accurately from the image so that the trajectory of the ball can be accurately calculated. It is to provide a golf simulation device, a sensing device and a sensing method used therein.

The sensing device used in the virtual golf simulation apparatus according to an embodiment of the present invention, the camera unit for obtaining a plurality of images of the frame hitting the ball by the user's swing; And preprocessing means for extracting a ball candidate by performing a predetermined process on the image acquired by the camera unit, and presetting and storing a ball template as a reference image for the ball, and the size of the ball candidate for the extracted ball candidate. And a sensing processing unit including a template matching means for extracting ball information by variably matching the ball template.

On the other hand, the virtual golf simulation apparatus according to an embodiment of the present invention, the camera unit including a plurality of cameras for acquiring a plurality of frames each for hitting the ball by the user's swing; And preprocessing means for extracting a ball candidate by performing a predetermined process on the images acquired by the respective cameras, and presetting and storing a ball template as a reference image for the ball, and storing the extracted ball candidate in the size of the ball candidate. And a sensing device including a template matching means for extracting ball information by variably matching the ball template, and a simulation image of a ball trajectory according to the information extracted from the sensing processing unit. And an image implementing unit.

On the other hand, the sensing method for a virtual golf simulation according to an embodiment of the present invention, the step of acquiring a plurality of images of the frame hitting the ball by the user's swing; Extracting a ball candidate by performing a predetermined process on the acquired image; And extracting ball information from the extracted ball candidates by variably matching a stored ball template, which is previously set as a reference image for the ball according to the size of the ball candidate.

1 is a diagram illustrating an example of a screen golf system to which a virtual golf simulation apparatus according to an embodiment of the present invention is applied.

2 is a block diagram showing the configuration of a virtual golf simulation apparatus according to an embodiment of the present invention.

FIG. 3 is a diagram schematically illustrating a more detailed configuration of the sensing device illustrated in FIG. 2 and functions of each component.

FIG. 4A illustrates an image acquired by a camera unit, and FIG. 4B illustrates a source image obtained by dividing a region of a predetermined section from the image illustrated in FIG.

FIG. 5A illustrates an image of a source image, FIG. 5B illustrates an image showing a stationary background image, and FIG. 5C illustrates an image in which a background image is removed from a source image. .

FIG. 6A illustrates the ball candidates (C1-1, etc.) of several frames in one image by combining several frames processed as shown in FIGS. 4 and 5, in which ball candidates are extracted. (B) shows the ball template T which is preset and stored as a reference image for the ball.

7 is a flowchart illustrating a sensing method for a virtual golf simulation according to an embodiment of the present invention.

An embodiment of a virtual golf simulation apparatus according to the present invention, a sensing apparatus and a sensing method used therein will be described in more detail with reference to the accompanying drawings.

First, a virtual golf simulation apparatus and a sensing apparatus used therein will be described with reference to FIGS. 1 and 2.

1 is a view showing an example of a screen golf system to which the virtual golf simulation apparatus according to an embodiment of the present invention, Figure 2 is a block showing the configuration of the virtual golf simulation apparatus applied to the screen golf system shown in FIG. It is also.

1 and 2, the virtual golf simulation apparatus according to an embodiment of the present invention is a sensing device (S) for sensing the user hitting the ball (B), and the image of the virtual golf course It is configured to include a simulator (1) for performing a virtual golf simulation by providing a simulation image of the trajectory of the ball in the virtual golf course according to the sensing result of the sensing device (S).

As shown in FIG. 1, a screen golf system to which a virtual golf simulation apparatus is applied according to an embodiment of the present invention has a batter that allows a user to swing a golf on the bottom of a golf booth 2 that provides a predetermined size of space. 110 is provided, the batting mat 120 is provided on one side of the plate at bat 110 so that the user can hit the ball (B) placed on the batting mat 120 in the plate at bat 110 by the golf swing In front of the screen 3 is provided a device for outputting the image information received from the simulator 1 (image output unit 30 shown in Figure 2, for example, a beam project, etc.) on the screen (3) And to project an image relating to the virtual golf simulation.

The batter 110 and the hitting mat 120 may be provided on the bottom surface of the golf booth 2, but may be provided on the swing plate 100 as shown in FIG.

The swing plate 100 may be inclined at a predetermined angle in front, rear, left, and right directions, and may be connected to the simulator 1 to form a slope corresponding to the terrain of the virtual golf course that is currently implemented.

Meanwhile, as shown in FIG. 1, a sensing device S is provided in the golf booth 2 to sense that the user strikes the ball B. FIG.

That is, as shown in FIG. 2, the sensing device S includes a camera unit 50 including a plurality of

cameras

51 and 52 and a physical characteristic of a ball moving by processing an image acquired by the camera unit 50. It may be configured to include a sensing processing unit 60 for extracting.

The camera unit 50 may be configured by a camera for one image sensing or may be configured by two or more cameras. The camera unit 50 acquires an image of a moving ball B and adjusts the coordinates of the ball in three-dimensional space. In order to extract, as shown in FIG. 1, two or

more cameras

51 and 52 may be driven in conjunction with each other to configure a stereo camera unit.

Here, the physical characteristics of the ball moving may include the speed of the ball, the direction of movement of the ball (horizontal direction of movement), the angle of height of the ball (direction of movement in the vertical direction), the spin applied to the ball, and the like.

The sensing processing unit 60 receives a grabber 71 that sequentially obtains and collects the images acquired through the camera unit 50 in units of frames, and receives and processes an image transmitted from the grabber 71. Shot detector 72 for detecting whether the preparation is completed and whether the hit by the user has been made, and when the shot is detected by the shot detector 72 receives the image related to the hit from the shot detector 72 The ball image processor 80 extracts the physical property information of the moving ball by performing the image processing of the ball, and the spin of the ball by analyzing the motion locus of the golf club from the image received from the shot detector 72. It may be configured to include a club image processing unit 90 for extracting information.

In addition, the sensing processing unit 60 matches the images acquired from each of the plurality of

cameras

51 and 52 with each other to obtain two-dimensional information (information extracted from one camera is two-dimensional information) for each camera image. It is preferable to include conversion means (73) which is provided to convert the three-dimensional information extracted by matching the plurality of camera images with each other, or vice versa, into two-dimensional information.

Meanwhile, the simulator 1 constituting the virtual golf simulation apparatus according to an embodiment of the present invention includes a controller M, a database 10, an image processor 20, an image output unit 30, and the like. This is preferable.

The database 10 stores all data necessary for the virtual golf simulation. For example, data necessary for driving the system, data about the image implementation of the virtual golf course, data about the simulation image implementation for the ball trajectory, and the like are stored.

The image processor 20 is a part where predetermined image processing is performed to implement an image of a virtual golf course or a simulation image of a trajectory of a ball unfolding in a virtual golf course.

The image output unit 30 is for outputting the image information received from the image processing unit 20 on the screen for the user to see.

The controller M receives information according to the sensing result from the sensing device S so as to control operations of all components such as the database 10, the image processor 20, and the image output unit 30. It is provided. That is, the control unit M, the database 10, the image processing unit 20 and the like function as image implementing means for implementing a simulation image of the trajectory of the ball according to the sensing result of the sensing device S.

On the other hand, with reference to Figure 3 will be described in more detail the configuration of the sensing device (S) shown in Figure 2 and the function of each component.

As shown in FIG. 3, the flow of information transmission in the sensing device according to the present invention includes a camera unit 50-> grabber 71-> shot detector 72-> ball image processing unit 80 and a club image processing unit ( 90)-> Simulator (1) in the order.

The camera unit 50 acquires an image of several hundred frames per second while photographing a predetermined range including a portion at which a turn at bat and a hit mat are provided.

The obtained image is transmitted to the grabber 71, and the grabber 71 receives the image from the camera unit 50 and stores the image, and transfers the image to the shot detector 72 to perform image processing.

The shot detector 80 detects whether the ball is ready to hit by finding a ball in the image received from the grabber and determining whether the ball meets a predetermined condition, and whether the ball moves from the position where the ball is ready to hit. Determining whether or not the blow was made by the user by determining whether or not.

Since the method for detecting the hitting preparation and the hitting detection of the shot detector 72 is not essential to the present invention, a detailed description thereof will be omitted.

If the hit is made, the shot detector 72 stores both the image at the time of hitting and the image of a plurality of frames thereafter, which is transmitted to the ball image processing unit 80 to perform image processing.

In addition, when a shot is made, the shot detector 72 also stores images of a plurality of frames before the hitting point, and transmits the shot detector 72 to the club image processing unit 90, and the club image processing unit 90 receives an image received from the shot detector 72. And extracting and processing the golf club from the image processed by the ball image processor 80 to derive the motion trajectory of the golf club head to estimate the spin of the ball.

The physical property information of the ball extracted by the ball image processing unit 80 and the club image processing unit 90 is transmitted to the simulator 1, and the simulator 1 simulates the movement of the ball in the virtual golf course based on the same. The virtual golf simulation is performed by outputting an image.

Here, the ball image processing unit 80 performs preprocessing means 81 for extracting a ball candidate by performing a predetermined process on the images acquired by the respective cameras, and sets and stores a ball template as a reference image for the ball in advance. And a template matching means 82 for extracting ball information by variably matching the ball template with respect to the ball candidate according to the size of the ball candidate.

In other words, the image acquired by the camera includes the image of the ball, but because there are various noises similar to the shape of the ball (for example, a specific part of the surrounding terrain, a part of the user's body, a head of the golf club, etc.) In order to extract the ball accurately from the pre-processing means 81 performs a predetermined process on the image and selects and extracts what can be regarded as a ball primarily as a ball candidate, and the template matching means is selected from among the extracted ball candidates. The most similar to the ball template is recognized as a ball, and the coordinates of the recognized ball are extracted to calculate the ball trajectory information.

However, it is not easy to compare the ball candidate and the ball template by the template matching means, because the size of the ball candidate may be different for each frame, and the size of the ball may be different for each frame. Because the shooting area of the camera is fixed and the distance to the camera changes as the ball moves in the shooting area, the image of the moving ball must appear in different sizes for each frame.

In this regard, the ball template has a fixed size and shape, so if you compare the ball candidate with the ball template as it is, it becomes impossible to determine how similar the ball candidate is to the ball template. do.

Accordingly, the template matching means estimates the size of the ball candidate and variably matches the ball candidate and the ball template according to the estimated size to extract the similarity, thereby enabling accurate ball recognition.

First, image processing by the preprocessing means will be described with reference to FIGS. 3 to 5.

First, the preprocessing means 81 (see FIG. 3) separates the source image 220 as shown in FIG. 4B from the image 200 acquired from the camera as shown in FIG. 4A. .

That is, since the image 200 acquired from the camera includes all images of a wide shooting area, it takes a long time to process all of them, so that only the necessary parts are extracted and only the extracted image is processed, thereby shortening the image processing time. It was made to be.

The source image 220 shown in (b) of FIG. 4 corresponds to a section from the image shown in (a) of FIG. 4 to a point P2 separated by a predetermined distance forward from the point P1. It can be defined as an image.

That is, when the ball is hit, it flies toward the front screen 3 (see FIG. 1), so that information such as the speed, direction, and height angle of the ball is analyzed by analyzing the movement of the ball from the point where the hit was made to a predetermined distance forward. Can be sufficiently calculated.

Here, the point P2 illustrated in FIG. 4A may be a position corresponding to the screen 3 (refer to FIG. 1) or may be a preset point located a predetermined distance further from the screen.

By specifying the P1 point and the P2 point in the image acquired from the camera and separating the image corresponding to the section, the source image as shown in FIG. 4B can be obtained.

On the other hand, the source image obtained as described above can be obtained only the image for the ball candidate by removing the background image through the difference operation, as shown in Figure 5, the source image 220 shown in (a) of FIG. In FIG. 5B, only the moving image may be extracted as illustrated in FIG. 5C by removing the background image 230 illustrated in FIG. 5B through the difference operation.

Not all moving parts in the image extracted through the difference operation are seen as balls, among which there are parts corresponding to noise, such as a part of the head of the golf club or a part of the user's body.

Therefore, the preprocessing means designates the portions of the image 240 that are calculated as the ball candidates based on the geometric characteristics of the balls. For example, if the aspect ratio is too large, it cannot be viewed as a 'ball', so it is preferable to exclude it, and because it is too small to display as one or two pixels, it is excluded.

As described above, the remaining parts removed based on the geometrical characteristics of the ball are designated as ball candidates. In FIG. 5C, two ball candidates C1-1 and C1-2 are extracted.

4 and 5 show the processing of the image of one frame, and the ball image processing unit according to the present invention obtains each of the images obtained by each camera at the time of hitting and thereafter. The image candidates are extracted by performing image processing on the images of all the frames of the camera.

Meanwhile, the image processing by the template matching means described above will be described with reference to FIGS. 3 and 6. 6 (a) shows the ball candidates (C1-1, etc.) of several frames in one image by combining several frames processed as shown in FIGS. 4 and 5 to which ball candidates are extracted. 6 (b) shows the ball template T which is preset and stored as a reference image for the ball.

In Fig. 6A, C1-1 and C1-2 are ball candidates extracted from the source image of the first frame, C2-1 is in the second frame, C3-1 is in the third frame, and C4- is 1 represents the ball candidates extracted in the fourth frame, respectively.

The template matching means 82 includes estimating means 82a for estimating the size of the ball candidate extracted by the preprocessing means 81 (see FIG. 5C), and the ball template corresponding to the ball candidate. Normalizing means 82b for normalizing the image of the ball candidate to a size that matches the ball candidate and the ball template with the normalized ball candidate are extracted to extract a similarity to find a ball among the ball candidates. And matching means 82c for extracting coordinate information.

Here, the estimating means 82a estimates the size of each extracted ball candidate in the three-dimensional space. Since the images acquired by each of the plurality of cameras contain two-dimensional information, the size of the ball candidate in the three-dimensional space is determined. In order to know, the 3D coordinate information of each ball candidate is extracted through the conversion means 73 (see FIGS. 2 and 3), and the extracted 3D coordinate information is used to extract the camera and the camera in the 3D space of the ball candidate. The distance information and the like are estimated to estimate the size of each ball candidate therefrom.

That is, the estimation means 82a estimates the size in the three-dimensional space by using the conversion means 73 for each of the ball candidates (C1-1, etc.) of the various frames shown in FIG.

Here, it can be seen from FIG. 6A that the ball candidates of the respective frames have different sizes, which is caused by the distance between the camera and the ball as the ball moves in the photographing area. Therefore, by estimating the size of the ball candidates of each frame using coordinate values in three-dimensional space, the size of each ball candidate is modified to fit the ball template, or conversely, the size of the ball template is modified to fit each ball candidate. By comparing, it is possible to determine exactly how similar each ball candidate is compared with the ball template, so that it is easy to find which ball candidate among the ball candidates is the real ball.

The normalizing means 82b normalizes the ball template T to a size corresponding to the ball template T in consideration of the size of each ball candidate estimated by the estimating means 82a, or individually processes the ball template T itself. Means for normalizing to correspond to the size of the ball candidate.

Here, the normalizing process refers to deforming the image so as to correspond to the size of the ball template T so that the ball candidate image can be easily compared with the ball template T while maintaining the shape, pixel ratio, and the like of the ball candidate image, and the ball template T Normalizing the conversely means deforming the image to correspond to the size of the ball candidate so that the ball template T can be easily compared with the ball candidate while maintaining the shape, pixel ratio, and the like of the ball template T image.

When the normalization process for the ball candidate or the ball template is completed by the normalization means 82b, the matching means 82c compares each ball candidate and the ball template with each other and extracts each similarity.

The similarity is derived by converting the result determined by various criteria such as the overall shape, pixel ratio, and pixel value distribution into numerical values. When the similarity value is higher than or equal to a preset value, the ball candidate may be recognized as a 'ball'. have.

According to the matching result, coordinate values of the ball candidates recognized as balls may be extracted, and physical characteristic information such as speed, direction, and height angle of the ball may be calculated by changing the extracted coordinate values.

Meanwhile, a sensing method for a virtual golf simulation according to an embodiment of the present invention will be described with reference to FIG. 7.

First, when the virtual golf simulation begins, the user prepares to hit the ball on the hitting mat. At this time, the sensing device acquires an image of the ball placed on the hitting mat and finds the ball in the acquired image (S10). The ball finding process is performed in the shot detector.

After the ball is found in the acquired image and the coordinate information on which the ball is placed is checked, if the predetermined requirement is satisfied, the preparation for the hit is completed (S11).

After the preparation for hitting is completed, the shot detector starts to detect whether the user hits the ball (S12), and if the hit is made (S20), accurately grasps the hitting time point and the image of the hitting time point and subsequent frames. Obtain and pass to the image processing unit for viewing. If the blow is not made, the flow returns to the blow detection step S12.

After the hit detection, when the shot detector acquires a plurality of frames of images and transmits them to the ball image processor in real time, the ball image processor extracts a source image by separating a predetermined area from the received image (S32).

The background image is removed from the extracted source image (S33), and the ball candidate is extracted (S34).

Three-dimensional coordinate information is obtained by matching a plurality of camera images with respect to the extracted ball candidates, and information about the distance to the camera of the ball candidate is extracted therefrom to estimate the size of the ball candidate (S40).

The normalization process for the ball candidate or the normalization process for the ball template is performed in consideration of the estimated size of the ball candidate (S50).

After the normalization process is performed, the ball candidate and the ball template are matched with each other (S61).

The degree of similarity of the ball candidate with the ball template is calculated by converting the similarity into a numerical value (S62).

If the calculated similarity is less than the preset value, the ball candidate is determined to be non-ball and excluded (S63). If the calculated similarity is equal to or greater than a preset value, the ball candidate is recognized as a ball and a coordinate value is extracted (S64).

In this manner, a ball candidate having a predetermined value or more among the ball candidates on each frame is recognized as a ball, a coordinate value of the corresponding ball is extracted, and a ball trajectory is calculated by connecting the coordinate values of the extracted ball to each other (S65).

The calculated ball trajectory is implemented as a simulation image through a simulator (S70).

The best mode for carrying out the invention is described in detail.

The virtual golf simulation device, the sensing device and the sensing method used according to the present invention is a golf game or a so-called screen golf industry that can allow a user to enjoy a virtual golf game by performing a golf simulation based on virtual reality It can be used.

Claims

In the sensing device used for the virtual golf simulation device,

A camera unit for acquiring a plurality of frames of images for hitting the ball by the swing of the user; And

Preprocessing means for extracting a ball candidate by performing a predetermined process on the image acquired by the camera unit, and presetting and storing a ball template as a reference image for the ball, and storing the extracted ball candidate in the size of the ball candidate. A sensing processing unit including a template matching means for extracting ball information by variably matching the ball template accordingly;

Sensing device comprising a.
The method according to claim 1, wherein the pretreatment means,

And a ball candidate is extracted from a source image prepared by separating an image for a predetermined section based on a point where a hit is made from the images acquired by the respective cameras.
The method of claim 1, wherein the template matching means,

And estimating the size of the ball candidate and processing the image to correspond to the size of the ball template to match each other.
The method of claim 1, wherein the template matching means,

And estimating the size of the ball candidate to image the ball template to match the size of the ball candidate to match each other.
The method of claim 1,

The camera unit is driven in conjunction with each other and consists of a plurality of cameras, each of which acquires a plurality of frames of images

The sensing processing unit,

And a converting means adapted to convert two-dimensional information into three-dimensional information or inversely convert the three-dimensional information into two-dimensional information by matching images acquired from each of the plurality of cameras with each other.
The method of claim 5, wherein the template matching means,

Estimation means for estimating the size of the ball candidate by extracting three-dimensional information about the extracted ball candidate through the conversion means;

Normalizing means for normalizing an image for the ball candidate with a size corresponding to the ball template with respect to the ball candidate;

And matching means for extracting coordinate information by finding a ball among the ball candidates by extracting a similarity by matching the normalized ball candidate and the ball template with each other.
The method of claim 5, wherein the template matching means,

Estimation means for estimating the size of the ball candidate by extracting three-dimensional information about the extracted ball candidate through the conversion means;

Normalizing means for normalizing the ball template to correspond to the estimated size of the ball candidate;

And matching means for extracting coordinate information by finding a ball among the ball candidates by extracting similarity by matching the normalized ball template with the ball candidate.
A camera unit including a plurality of cameras each obtaining a plurality of frames of images for striking the ball by a user's swing; And preprocessing means for extracting a ball candidate by performing a predetermined process on the images acquired by the respective cameras, and presetting and storing a ball template as a reference image for the ball, and storing the extracted ball candidate in the size of the ball candidate. And a sensing processing unit including a template matching means for extracting ball information by variably matching the ball template.

And a video implementation unit for implementing a simulation image of a ball trajectory according to the information extracted by the sensing processing unit.
In the sensing method for the virtual golf simulation,

Acquiring a plurality of frames of images about hitting the ball by the user's swing;

Extracting a ball candidate by performing a predetermined process on the acquired image; And

Extracting ball information by variably matching the stored ball template with the ball candidate pre-set as a reference image for the ball according to the size of the ball candidate;

Sensing method comprising a.
The method of claim 9, wherein extracting the ball candidates comprises:

Preparing a source image by separating an image for a preset section based on a point where a hit is made from the images acquired by the cameras;

Removing a background image through the difference operation on the source image;

And removing an image determined to be non-ball based on geometrical characteristics from the difference- computed image.
The method of claim 9,

In the acquiring of the image, a plurality of cameras respectively acquire images of a plurality of frames.

Variablely matching the ball template,

Estimating the size of the ball candidate by matching images acquired by the plurality of cameras with respect to the extracted ball candidates, respectively;

Normalizing the ball candidate having the estimated size to a size corresponding to the ball template;

And matching the ball candidate with the normalized ball candidate with the ball template to extract similarity.
The method of claim 9,

In the acquiring of the image, a plurality of cameras respectively acquire images of a plurality of frames.

Variablely matching the ball template,

Estimating the size of the ball candidate by matching images acquired by the plurality of cameras with respect to the extracted ball candidates, respectively;

Normalizing the ball template to a size corresponding to the estimated size of the ball candidate;

And matching the normalized ball template with the ball candidate to extract a similarity.