WO2012002732A2 - Sensing processing device and method for moving ball and virtual golf simulation device using the same - Google Patents

Sensing processing device and method for moving ball and virtual golf simulation device using the same Download PDF

Info

Publication number
WO2012002732A2
WO2012002732A2 PCT/KR2011/004759 KR2011004759W WO2012002732A2 WO 2012002732 A2 WO2012002732 A2 WO 2012002732A2 KR 2011004759 W KR2011004759 W KR 2011004759W WO 2012002732 A2 WO2012002732 A2 WO 2012002732A2
Authority
WO
WIPO (PCT)
Prior art keywords
image
ball
point
ball image
half circle
Prior art date
Application number
PCT/KR2011/004759
Other languages
French (fr)
Other versions
WO2012002732A3 (en
Inventor
Won Il Kim
Young Chan Kim
Chang Heon Woo
Min Soo Hahn
Tae Wha Lee
Myung Sun Choi
Jae Wook Jung
Shin Kang
Seung Woo Lee
Han Bit Park
Myung Woo Kim
Yong Woo Kim
Seong Kook Heo
Original Assignee
Golfzon Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from KR1020100062207A external-priority patent/KR101019829B1/en
Priority claimed from KR1020100062200A external-priority patent/KR101019902B1/en
Application filed by Golfzon Co., Ltd. filed Critical Golfzon Co., Ltd.
Priority to CN201180041833.7A priority Critical patent/CN103079651B/en
Priority to JP2013518250A priority patent/JP5626939B2/en
Publication of WO2012002732A2 publication Critical patent/WO2012002732A2/en
Publication of WO2012002732A3 publication Critical patent/WO2012002732A3/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00—Training appliances or apparatus for special sports
    • A63B69/36—Training appliances or apparatus for special sports for golf
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • A63B2071/0636—3D visualisation
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • A63B2071/0638—Displaying moving images of recorded environment, e.g. virtual environment
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B2071/0647—Visualisation of executed movements
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/05—Image processing for measuring physical parameters
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/10—Positions
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/30—Speed
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/806—Video cameras
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Definitions

  • the present invention relates to a sensing processing device and method for a
  • moving ball and a virtual golf simulation device using the same and, more particularly, to a sensing processing device and method for a moving ball that acquires and analyzes an image of a moving ball, such as a golf ball, to obtain physical information thereof and a virtual golf simulation device using the same.
  • sensing systems such as a sensing system using an infrared sensor, a sensing system using a laser sensor, a sensing system using an acoustic sensor and a sensing system using a camera sensor, have come onto the market.
  • a sensing processing device to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing device including a light direction estimating means to analyze a ball image from the acquired image and to estimate a direction of the light toward the moving ball and an image processing means to fit the ball image into the shape of a ball based on an irradiated side of the ball image along the estimated light direction and to extract coordinates of a center point of the fitted image.
  • the sensing processing device to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing device including a half circle fitting means to form a half circle curve of a predetermined diameter or a diameter measured through the ball image at a specific position of the ball image based on the light direction estimated by the light direction estimating means and to fit a half circle opposite to the half circle curve based on the diameter and a coordinate extracting means to extract coordinates of a center point of the half circle fitted ball image.
  • sensing processing method to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing method including analyzing a ball image from the acquired image to estimate the direction of the light toward the moving ball, fitting the ball image into the shape of a ball based on an irradiated side of the ball image along the estimated light direction, and extracting coordinates of a center point of the fitted image.
  • virtual golf simulation device including a sensor unit including a camera device and a strobe flashlight device to acquire a multiple exposure image of a moving golf ball hit by a golfer, a sensing processing unit including a light direction estimating means to extract and analyze an image of the golf ball from the acquired image and to estimate a direction of light emitted from the strobe flashlight device toward the golf ball and an image processing means to fit the golf ball image into the shape of a golf ball based on an irradiated side of the golf ball image along the estimated light direction and to extract coordinates of a center point of the fitted image, and a simulator to calculate the change of the coordinates of the center point and to obtain physical information on the moving golf ball, thereby simulating an orbit of the golf ball.
  • a sensing processing unit including a light direction estimating means to extract and analyze an image of the golf ball from the acquired image and to estimate a direction of light emitted from the strobe flashlight device toward the golf ball and an image processing means to fit the golf ball image into the shape of a
  • FIG. 1 is a block diagram showing a sensing processing device or a virtual golf
  • FIG. 2 is a view showing an example of a screen golf system to which the sensing processing device or the virtual golf simulation device shown in FIG. 1 is applied;
  • FIG. 3 is a view showing an operational signal scheme of a camera device and a strobe flashlight device to acquire an image to be processed by the sensing processing device according to the embodiment of the present invention;
  • FIGS. 4(a) and 4(b) are views showing image patterns of a ball acquired by the
  • FIG. 5(a) is a view showing an example of an actually acquired image of the image having the pattern shown in FIG. 4(a) and FIG. 5(b) is a view showing a preliminarily processed image of the image shown in FIG. 5(a);
  • FIG. 6 is an enlarged view of the image shown in FIG. 5(b) arranged in a state in which a plurality of frames overlaps, showing a ball image extracting process;
  • FIGS. 7(a) to 7(c) are views illustrating a principle of fitting a ball image wherein
  • FIG. 7(a) is a view showing a ball image
  • FIG. 7(b) is a view showing an incorrect fitting example of the ball image
  • FIG. 7(c) is a view showing a fitting example of the ball image using a fitting means of the present invention
  • FIG. 8(a) is a view showing an image from which a background is removed and FIG.
  • FIG. 8(b) is an enlarged view showing region A of FIG. 8(a);
  • FIG. 9(a) is a view showing an image acquired by image processing the image shown in FIG. 8(a) at a low level and FIG. 9(b) is an enlarged view showing region B of FIG.
  • FIG. 10 is a view showing an image acquired by image processing the image shown in FIG. 8(a) at a high level and FIG. 10(b) is an enlarged view showing region C of FIG. 10(a);
  • FIG. 11(a) is a view showing an image acquired by extracting the maximum
  • FIG. 10(b) and FIG. 11(b) is a view showing an image acquired by applying the maximum brightness point to the image shown in FIG. 9(a);
  • FIG. 12(a) is a view showing a line scanning process performed about the maximum brightness point of a ball image
  • FIG. 12(b) is a view showing an image acquired as the result of the line scan
  • FIG. 12(c) is a view showing an image acquired by applying an average filter to the line scan result shown in FIG. 12(b);
  • FIG. 13 is a view illustrating estimation of a direction in which light is directed to a ball image
  • FIG. 14 is a view showing an axis of the light direction estimated by a light direction estimating means of the sensing processing device according to the present invention displayed on the ball image;
  • FIG. 15 is a view showing a process of measuring a diameter of the ball image
  • FIGS. 16, 17(a) and 17(b) are views showing a process of fitting a ball image in consideration of the estimated light direction;
  • FIG. 18 is a view showing an image acquired by extracting a center point from the ball image fitted by fitting process shown in FIGS. 17(a) and 17(b);
  • FIGS. 19 and 20 are flow charts showing respective embodiments of a sensing
  • FIG. 21 is a flow chart showing the ball image fitting process shown in FIGS. 19 and 20 in more detail.
  • a sensing processing device for a moving ball is applicable to all systems that perform sensing by acquiring and analyzing an image of the state of a moving ball in sports, such as golf, using a ball.
  • the sensing processing device may be applied to a so-called screen golf system to which a virtual golf simulation device is applied.
  • FIGS. 1 and 2 schematically show the construction of a sensing processing device for a moving ball according to an embodiment of the present invention and a virtual golf simulation device using the same.
  • the present invention relates to a sensing processing device 220 which processes an image acquired by a sensor unit including camera devices 310 and 320, a strobe flashlight device 330 and a signal generating unit 210 to process an image of a ball, thereby extracting coordinates of a center point.
  • a sensing processing device 220 according to an embodiment of the present invention includes a preliminary processing means 230, an image extracting means 240, a light direction estimating means 250 and an image processing means 260.
  • the camera devices 310 and 320 are provided to respectively capture a plurality of frames on the movement of a ball moving from an initial position in a moving direction of the ball.
  • two camera devices 310 and 320 are provided, to which, however, the present invention is not limited.
  • a camera device or more than two camera devices may be provided.
  • the strobe flashlight device 330 is a lighting device using light emitting diodes
  • the strobe flashlight device 330 is used as a light source for capturing of the camera devices.
  • the strobe flashlight device 330 generates strobe flashlight at a predetermined time interval (that is, a plurality of flashes is generated at a predetermined time interval) so that the camera devices 310 and 320 can capture a multiple exposure image.
  • a ball is present on a frame captured by the camera devices in correspondence to the number of flashes generated from the strobe flashlight device 330 to provide a multiple exposure image thereof.
  • a trigger signal to operate the camera devices 310 and 320 and the strobe flashlight device 330 is generated by the signal generating unit 210.
  • the multiple exposure image captured by the camera devices 310 and 320 and the strobe flashlight device 330 is processed by the sensing processing device 220 according to the present invention and is transmitted to a simulator 100.
  • the light direction estimating means 250 includes a brightness analyzing means (not shown) and a scanning means (not shown), which will be described below in more detail.
  • the image processing means 260 is provided to fit the ball image into the shape of a ball based on the contour of the side of the ball image to which light is irradiated according to the light direction estimated by the light direction estimating means 250, thereby extracting the coordinates of a center point of the fitted image.
  • the image processing means 260 preferably includes a half circle fitting
  • a coordinate extracting means 280 to extract the coordinates of a center point of the ball image to which the half circle is fitted, which will be described below in detail.
  • the sensing processing device preferably further includes a diameter measuring means to measure the diameter of the ball image.
  • the preliminary processing means 230 is provided to remove a
  • the image extracting means 240 is provided to extract a ball image from the image preliminarily processed by the preliminary processing means 230 (more specifically, to extract an image estimated as a ball, i.e. a ball image candidate).
  • the estimation of the light direction in the ball image performed by the light direction estimating means 250 and the fitting of the ball image performed by the image processing means 260 are performed with respect to the ball image extracted by the image extracting means 240.
  • the simulator 100 preferably includes a controller M, a database 110, an image processing unit 120 and an image output unit 130.
  • the controller M receives coordinate information of a moving ball which has been image processed and acquired by the sensing processing device 220, converts the coordinate information into three-dimensional coordinates, calculates the change of the coordinates of the moving ball in a three-dimensional space, obtains predetermined physical information for moving orbit simulation of the ball and transmits the obtained physical information to the image processing unit 120.
  • Predetermined data for moving orbit simulation of the ball may be extracted from the database 110, and the image processing of the moving orbit simulation of the ball by the image processing unit 120 may be performed by extracting image data stored in the database 110.
  • a converting means to covert coordinate information of the moving ball transmitted from the sensing processing device 220 into three-dimensional coordinates may be provided separately from the controller M.
  • the preliminary processing means 230, the image extracting means 240, the light direction estimating means 250 and the image processing means 260 may be realized as a single controller configured to perform the functions of the above means or separate controllers configured to respectively performed the functions of the above means.
  • the preliminary processing means 230, the image extracting means 240, the light direction estimating means 250 and the image processing means 260 may be realized as a single program configured to perform the functions of the above means or separate programs configured to respectively performed the functions of the above means.
  • FIG. 2 An example of a screen golf system, to which the sensing processing device or the virtual golf simulation device with the above-stated construction is applied, is shown in FIG. 2.
  • a hitting mat 30 having a golf tee, on which a golf ball is placed, and a ground (preferably including at least one selected from among a fairway mat, a rough mat and a bunker mat) and a swing plate 20, on which a golfer hits a golf ball, are provided at one side of a golf booth B.
  • a screen 40 on which a virtual golf simulation image realized by the image output unit 130 is displayed, is provided at the front of the golf booth B.
  • the camera devices 310 and 320 and the strobe flashlight device 330 are provided at the ceiling of the golf booth B.
  • the camera devices 310 and 320 are provided at the ceiling and wall of the golf booth B, respectively, to which, however, the present invention is not limited.
  • the camera devices 310 and 320 can be installed at any positions of the golf booth B so long as the camera devices 310 and 320 do not disturb the swing of the golfer and are prevented from colliding with a golf ball hit by the golfer while the camera devices 310 and 320 can effectively capture an image of a moving state of the golf ball 10.
  • the strobe flashlight device 330 is installed at the ceiling of the golf booth B to provide strobe flashlight substantially perpendicular to a hitting point, to which, however, the present invention is not limited.
  • the strobe flashlight device 330 can be installed at any position of the golf booth B so long as the strobe flashlight device 330 can effectively provide strobe flashlight.
  • the camera devices 310 and 320 capturing predetermined regions in which the hitting is performed capture a plurality of frames, respectively.
  • the strobe flashlight device 330 generates several flashes per frame to acquire a multiple exposure image of the moving golf ball.
  • FIG. 3 is a view showing a trigger signal generation scheme of a camera device and a strobe flashlight device by the signal generating unit 210 (see FIG. 1).
  • trigger signals of the camera device have a time interval of tc.
  • trigger signals are generated at a time interval of tc for each frame.
  • exposure time for each frame is te (at this time, preferably tc > te).
  • data on an image acquired for a time of tc - te is preferably transmitted to the sensing processing device 220 (see FIG. 1).
  • the camera device is triggered at an interval between the trigger signal generating times for each frame, and data on the acquired multiple exposure image is transmitted to the sensing processing device.
  • strobe flashlight is generated by the strobe flashlight device several times. In FIG. 3, strobe flashlight trigger signals are generated three times at a time interval of tsl.
  • strobe flashlight is generated three times at the same time interval of tsl.
  • the camera device and the strobe flashlight device are preferably synchronized for the signal generating unit 210 (see FIG. 1) to generate a first trigger signal.
  • a time interval of ts2 is preferably provided between the trigger signal of the last one of the three strobe flashlights and a first strobe flashlight of the next frame.
  • the time interval of tsl and the time interval of ts2 may be set to be equal to each other. As shown in FIG. 3, however, the time interval of tsl and the time interval of ts2 are preferably set to be different from each other. In this case, the time interval of ts2 may be set to be longer than the time interval of tsl.
  • FIGS. 4(a) and 4(b) show an image of a moving ball (golf ball) when the ball is hit by the golfer using a golf club.
  • FIGS. 4(a) and 4(b) are views showing two patterns of the multiple exposure image acquired by the camera device and the strobe flashlight device based on the signal generation scheme shown in FIG. 3, respectively.
  • An image II shown in FIG. 4(a) is acquired through multiple exposure of golf club images CI, C2 and C3 and ball images 1 la, 1 lb and 11c.
  • the ball images 11a, l ib and 1 lc are spaced apart from each other by a predetermined interval.
  • An image 12 of FIG. 4(b) shows a case in which ball images overlap to provide an image region 12 of a predetermined size.
  • the sensing processing device and method according to the present invention are provided to accurately acquire the coordinates of a center point of the ball when the ball images are separated from each other on the multiple exposure image as the ball is moved at a speed equal to or greater than a predetermined speed based on the fixed interval of the strobe flashlight as shown in FIG. 4(a).
  • the sensing processing method according to the present invention the case in which the ball images overlap as shown in FIG. 4(b) is excluded.
  • FIG. 5(a) is a view showing the original copy of a multiple exposure image including separate ball images.
  • the multiple exposure image is acquired using a camera device having a low resolution of 320 x 240 and operated at a low speed of 75 fps and a strobe flashlight device operating at 330 Hz.
  • the preliminary processing means 230 removes a stationary image, i.e. a background image, from the original image shown in FIG. 5(a) through subtraction and performs a predetermined preliminary processing operation such as Gaussian blur. An image acquired through the preliminary processing operation is shown in FIG. 5(b).
  • the image shown in FIG. 6 is obtained by overlapping a plurality of frames captured by the camera device into a single image.
  • an image set SO indicates an image of a zeroth frame
  • an image set S 1 indicates an image of a first frame
  • an image set S2 indicates an image of a second frame.
  • the image extracting means 240 may extract an image estimated by the ball image through a contour check or a check window. The processing may be carried out so that the final image estimated as the ball is kept while images estimated not to be the ball are discarded.
  • the ball has a predetermined diameter. Therefore, the contours of the respective images on the multiple exposure image are checked in consideration of the diameter of the ball, and the checked contours may be excluded if they are too large or too small to be the ball. [81] Also, as shown in FIG. 6, check windows W corresponding to the size of the respective images 11 of the multiple exposure image are formed so as to include the respective images 11, and an aspect ratio of the respective check windows W is checked.
  • the respective images 11 on the multiple exposure image are considered not to be the ball and thus may be discarded.
  • the images 11 extracted as the ball images are fitted into the shape of a ball as shown in FIGS. 7(a), 7(b) and 7(c) to extract the coordinates of a center point of each of the images.
  • FIG. 7(a) is a view showing an example of a ball image acquired under strobe
  • FIG. 7(b) is a view showing an incorrect fitting result of the ball
  • FIG. 7(c) is a view showing a result of the ball correctly fitted using the half circle fitting means 270 (see FIG. 1) of the image processing means according to the present invention.
  • the ball image 11 is captured while strobe flashlight is irradiated. Consequently, an irradiated region Rl clearly appears at one side of the ball image 11 , and a shaded region appears at the region opposite to the irradiated region Rl. That is, a shaded region R2 covering a portion of the ball image 11 is present at the region opposite to the irradiated region Rl.
  • a shaded region may appear on only a portion of the ball image or may cover more than the half of the ball image.
  • the fitting of the ball image acquired under strobe flashlight is performed in consideration of the shaded region present at the ball image.
  • the light direction of the strobe flashlight irradiated to the ball may be estimated and the fitting may be performed according to the light direction so that the shaded region is included in a fitted curve.
  • FIG. 7(c) shows fitting of the ball image in consideration of the light direction. As shown in FIG. 7(c), a fitted curve FC is formed at the irradiated region Rl of the ball image 11 so that the shaded region R2 is included in the fitted curve FC.
  • the ball image may be damaged as the result of the preliminary processing to extract specific information from the image.
  • images are preferably formed in stages so that specific information can be extracted from the image in each of the stages.
  • FIGS. 8(a) to 10(b) show images formed in stages according to the preliminary
  • FIG. 8(a) shows an image acquired by removing a background from an initially
  • FIG. 8(b) is an enlarged view of region A of FIG. 8(a), showing the original ball image 11-0.
  • the ball has a plurality of dimples with the result that the effect of the light is irregular. Consequently, the distribution of pixel values of the original ball image 11-0 is very irregular. Also, there exists a plurality of salt and pepper noises having irregular pixel values in one or more small pixel units at the periphery thereof.
  • FIG. 9(a) shows an image acquired by almost completely removing various peripheral noises from the image shown in FIG. 8(a) through Gaussian blur processing and threshold processing at a proper level at which the original ball image is not greatly damaged
  • FIG. 9(b) is an enlarged view of region B of FIG. 9(a), showing a first processed image 11-1.
  • FIG. 10(a) shows an image acquired by performing Gaussian blur processing and threshold processing at a level higher than the first processed image 11-1 with respect to the image shown in FIG. 8(a) and performing normalization processing of the pixel values of the ball image
  • FIG. 10(b) is an enlarged view of region C of FIG. 10(a), showing a second processed image 11-2.
  • the light direction is estimated using the image shown in FIG. 10(c), i.e. the second processed image 11-2.
  • the pixel values are properly distributed and the pixel values are clearly divided based on sizes thereof, whereby it is possible to easily estimate the light direction by finding a region in which the pixel values are high.
  • the image is a grayscale image, and therefore, the pixel values only have information on brightness values.
  • FIG. 11(a) shows the center point of the region R3, which is the brightest point, i.e. the maximum brightness point PI.
  • the pixel may be designated as the maximum brightness point PI.
  • the center of mass of the region may be extracted through a predetermined process, and the corresponding point may be designated as the maximum brightness point PI.
  • the light direction may be estimated by selecting the maximum
  • the brightness point PI on the ball image as a first point and another point as a second point, forming a line connecting the first point and the second point and setting the line as the axis of the light direction.
  • the first point may be extracted by the brightness analyzing means of the light direction estimating means 250 (see FIG. 1) and the second point may be extracted by the scanning means of the light direction estimating means 250 (see FIG. 1).
  • the brightness analyzing means may find the center point, i.e. the maximum
  • the scanning means may extract the second point through the process as shown in FIGS. 11(b), 12(a) and 12(b).
  • the maximum brightness point PI shown in FIG. 11(a) is applied to the ball image 11-1 of the first processed image.
  • the coordinate values of the maximum brightness point PI on the ball image 1 1-2 of the second processed image may be applied to the ball image 11-1 of the first processed image or the ball image 11-2 of the second processed image may be overlapped with the ball image 11-1 of the first processed image so that the maximum brightness point PI can be displayed on the ball image 11-1 of the first processed image.
  • FIG. 12(b) shows the line scanning results of the ball image shown in FIG. 12(a).
  • processed image formed by the line scanning has a very irregular shape.
  • the contour line is adjusted to be a fitted contour line L2 using an average filter.
  • Dmin is selected as the second point. This will hereinafter be explained with reference to FIG. 13.
  • strobe flashlight is irradiated to the ball 11 during movement of the ball 11 includes a portion 11-1 captured while light is irradiated and a shaded portion R2.
  • the captured portion 11-1 has a portion Rl appearing brighter under the influence of the light.
  • a portion R3 having the maximum pixel brightness value is included in the brighter portion Rl.
  • maximum brightness point PI may be designated as the second point P2.
  • the second point P2 may be decided as the point P2 on the contour line forming the shortest distance as the result of the line scanning process shown in FIGS. 12(a) to 12(c).
  • a line connecting the first point PI, i.e. the maximum brightness point, and the second point P2 obtained using the above method may be set as the axis AX of the light direction.
  • FIG. 14 is a view showing an image acquired by obtaining the first point PI and the second point P2 on the ball image 11-1 in a state as shown in FIG. 12(b) and extracting the axis AX of the light direction connecting the two points.
  • the fitting is not performed using the second processed image 11-2 but using the first processed image 11-1 in order to improve the accuracy in fitting of the ball image.
  • the sensing processing device includes a diameter measuring means to measure the diameter of the ball image.
  • a predetermined value may be used as the diameter for fitting of the ball image.
  • the diameter of the ball on the ball image may be measured so that the fitting is performed using the measured diameter.
  • the shape of the ball may be somewhat damaged due to the influence of the shaded portions on the initially acquired image, the limit of the resolution and the performance of several preliminary processing operations.
  • the fitting is performed using the diameter measured from the ball image rather than using the predetermined diameter value to more accurately extract the center point.
  • FIG. 15 is a view showing a process of measuring the diameter of the ball image 11-1 using the diameter measuring means.
  • the axis of the light direction is preferably vertically arranged so that the irradiated region is present at the upper part of the ball image and the shaded region is present at the lower part of the ball image.
  • line scanning is performed through horizontal lines L along the axis of the light direction to extract the longest one of the sections in which the pixel value is changed to a predetermined value or more as a diameter De.
  • the ball image is fitted by the half circle fitting means 270 (see FIG. 1).
  • the fitting operation is performed as shown in FIGS. 16, 17(a) and 17(b).
  • the half circle fitting means fits the ball image so that the shaded region is included when the shaded region is present on the ball image by light and therefore, the coordinates of the center point can be accurately extracted.
  • the half circle fitting means preferably includes a pixel checking means to check a pixel value on the ball image along the direction estimated as the light direction and a fitting processing means to form a half circle curve at a portion wt which the checked pixel value is changed to a predetermined value or more and fit the other half circle.
  • a half circle curve FC-1 is prepared as shown in FIG. 16 based on the diameter De measured by the diameter measuring means
  • the pixel checking means checks the pixel value (that is, the brightness value) of the image on the half circle curve FC-1. More preferably, the pixel checking means calculates the sum of the brightness values of the image on the half circle curve FC-1 along the axis AX of the light direction to detect the change.
  • the position at which the brightness value checked along the axis AX of the light direction or the sum of the brightness values of the image on the half circle curve is abruptly changed may be determined as the contour of the ball image 11.
  • the fitting processing means locates the half circle curve FC-1 at the position.
  • the fitting processing means covers the contour of the irradiated region Rl of the ball image 11 with the half circle curve FC-1.
  • the fitting processing means fits the other half circle curve FC-2 according to the measured diameter De to achieve correct fitting including the shaded region R2 of the ball image 11.
  • the coordinate extracting means 280 can accurately extract the coordinates of a center point CP of the ball image through the fitted curve FC of the ball image as shown in FIG. 18.
  • a trigger signal is applied by the signal generating unit so that the camera device and the strobe flashlight device capture a multiple exposure image of the moving state of a ball at a predetermined interval (S10).
  • a ball image is extracted from the preliminarily processed image (S30) (when the ball image is extracted, noise may also be extracted, and therefore, more strictly speaking, a ball image candidate is extracted) and pixel values, i.e. brightness values, of the extracted ball image are checked (S40) to extract a first point having the highest brightness value (S50).
  • a point on the contour line of the ball image which is connected to the first point at the shortest distance is extracted as a second point (S80).
  • a line passing through the first point and the second point is formed and the line is set as an axis of the light direction (S90).
  • an angle between the axis of the light direction set as described above and the vertical line or the horizontal line may deviate from a predetermined range.
  • the axis of the light direction is preferably used.
  • the light direction estimated through the steps S60 to S80 is corrected, and the corrected axis is set as the axis of the light direction (SI 10).
  • the sensing processing method according to this embodiment is basically identical to the sensing processing method according to the embodiment shown in FIG. 19 except that images are separately formed at the respective preliminary processing steps and the light direction is estimated based on the separately formed images. Therefore, a description of the duplicated portions will be omitted, and only the difference will be described in detail.
  • a preliminary processing step is carried out with respect to the acquired multiple exposure image (S20), and a first processed image and a second processed image are formed based on the preliminary processing level of the image (S21 and S22).
  • a ball image is extracted from the second processed image (S30) and pixel values, i.e. brightness values, of the extracted ball image are checked (S40) to extract a first point having the highest brightness value (S50).
  • 360 degree line scanning is performed on the ball image of the first processed image about the first point (S60).
  • the average filter is applied to the contour line of the ball image of the extracted line scanning result (S70).
  • FIG. 21 shows the step S120 shown in FIGS. 19 and 20, i.e. the step of fitting the ball image in consideration of the light direction, in more detail.
  • the diameter of the ball image is preferably measured for fitting (S123).
  • the diameter of the ball image may be measured as illustrated in FIG. 12 in a state in which the ball image is rotated so that the axis of the light direction is vertically arranged.
  • a pixel value (brightness value) is checked along the axis of the light direction and the half circle curve is located at the contour of the irradiated portion of the ball image (S125).
  • the other half circle based on the measured diameter is formed at the side opposite to the half circle to fit the ball image (SI 26) and the coordinates of a center point of the fitted ball image are extracted (SI 30).
  • a sensing processing device and method for a moving ball according to the present invention as described above and a virtual golf simulation device using the same although a low quality image is acquired by a low resolution and low speed camera device and strobe flashlight device, it is possible to obtain coordinates of a center point of a ball image in the acquired image at a sub pixel level, thereby achieving high sensing processing performance and sensing accuracy at low cost and a virtual golf simulation device using the same. Consequently, the present invention can be widely used in industries related to a sensing processing device and method for a moving ball and a virtual golf simulation device using the same.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Geometry (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Analysis (AREA)

Abstract

Disclosed herein are a sensing processing device and method for a moving ball that, although a low quality image is acquired by a low resolution and low speed camera device and strobe flashlight device, is capable of accurately obtaining coordinates of a center point of a ball image in the acquired image at a sub pixel level, thereby achieving high sensing processing performance and sensing accuracy at low cost and a virtual golf simulation device using the same.

Description

Description
Title of Invention: SENSING PROCESSING DEVICE AND METHOD FOR MOVING BALL AND VIRTUAL GOLF SIMULATION DEVICE USING THE SAME
Technical Field
[1] The present invention relates to a sensing processing device and method for a
moving ball and a virtual golf simulation device using the same, and, more particularly, to a sensing processing device and method for a moving ball that acquires and analyzes an image of a moving ball, such as a golf ball, to obtain physical information thereof and a virtual golf simulation device using the same.
Background Art
[2] In recent years, various devices have been developed which allow users to enjoy popular sports games, such as baseball, soccer, basketball and golf, in rooms or in specific places through simulation in the form of interactive sports games.
[3] In order to simulate sports using balls, such as baseballs, soccer balls, basketballs and golf balls in such interactive sports games, much research has been conducted into various sensing systems to accurately sense physical information on a moving ball, i.e. movement of the ball.
[4] For example, various sensing systems, such as a sensing system using an infrared sensor, a sensing system using a laser sensor, a sensing system using an acoustic sensor and a sensing system using a camera sensor, have come onto the market.
Disclosure of Invention
Technical Problem
[5] It is an object of the present invention to provide a sensing processing device and method for a moving ball that, although a low quality image is acquired by a low resolution and low speed camera device and strobe flashlight device, is capable of accurately obtaining coordinates of a center point of a ball image in the acquired image at a sub pixel level, thereby achieving high sensing processing performance and sensing accuracy at low cost and a virtual golf simulation device using the same.
Solution to Problem
[6] In accordance with one aspect of the present invention, the above and other objects can be accomplished by the provision of a sensing processing device to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing device including a light direction estimating means to analyze a ball image from the acquired image and to estimate a direction of the light toward the moving ball and an image processing means to fit the ball image into the shape of a ball based on an irradiated side of the ball image along the estimated light direction and to extract coordinates of a center point of the fitted image.
[7] In accordance with another aspect of the present invention, there is provided a
sensing processing device to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing device including a half circle fitting means to form a half circle curve of a predetermined diameter or a diameter measured through the ball image at a specific position of the ball image based on the light direction estimated by the light direction estimating means and to fit a half circle opposite to the half circle curve based on the diameter and a coordinate extracting means to extract coordinates of a center point of the half circle fitted ball image.
[8] In accordance with another aspect of the present invention, there is provided a
sensing processing method to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing method including analyzing a ball image from the acquired image to estimate the direction of the light toward the moving ball, fitting the ball image into the shape of a ball based on an irradiated side of the ball image along the estimated light direction, and extracting coordinates of a center point of the fitted image.
[9] In accordance with a further aspect of the present invention, there is provided a
virtual golf simulation device including a sensor unit including a camera device and a strobe flashlight device to acquire a multiple exposure image of a moving golf ball hit by a golfer, a sensing processing unit including a light direction estimating means to extract and analyze an image of the golf ball from the acquired image and to estimate a direction of light emitted from the strobe flashlight device toward the golf ball and an image processing means to fit the golf ball image into the shape of a golf ball based on an irradiated side of the golf ball image along the estimated light direction and to extract coordinates of a center point of the fitted image, and a simulator to calculate the change of the coordinates of the center point and to obtain physical information on the moving golf ball, thereby simulating an orbit of the golf ball.
Advantageous Effects of Invention
[10] In a sensing processing device and method for a moving ball according to the present invention as described above and a virtual golf simulation device using the same, although a low quality image is acquired by a low resolution and low speed camera device and strobe flashlight device, it is possible to obtain coordinates of a center point of a ball image in the acquired image at a sub pixel level. Consequently, the present invention has the effect of achieving high sensing processing performance and sensing accuracy at low cost and a virtual golf simulation device using the same. Brief Description of Drawings
] The above and other objects, features and other advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
] FIG. 1 is a block diagram showing a sensing processing device or a virtual golf
simulation device according to an embodiment of the present invention;
] FIG. 2 is a view showing an example of a screen golf system to which the sensing processing device or the virtual golf simulation device shown in FIG. 1 is applied;] FIG. 3 is a view showing an operational signal scheme of a camera device and a strobe flashlight device to acquire an image to be processed by the sensing processing device according to the embodiment of the present invention;
] FIGS. 4(a) and 4(b) are views showing image patterns of a ball acquired by the
screen golf system shown in FIG. 2;
] FIG. 5(a) is a view showing an example of an actually acquired image of the image having the pattern shown in FIG. 4(a) and FIG. 5(b) is a view showing a preliminarily processed image of the image shown in FIG. 5(a);
] FIG. 6 is an enlarged view of the image shown in FIG. 5(b) arranged in a state in which a plurality of frames overlaps, showing a ball image extracting process;
] FIGS. 7(a) to 7(c) are views illustrating a principle of fitting a ball image wherein
FIG. 7(a) is a view showing a ball image, FIG. 7(b) is a view showing an incorrect fitting example of the ball image and FIG. 7(c) is a view showing a fitting example of the ball image using a fitting means of the present invention;
] FIG. 8(a) is a view showing an image from which a background is removed and FIG.
8(b) is an enlarged view showing region A of FIG. 8(a);
] FIG. 9(a) is a view showing an image acquired by image processing the image shown in FIG. 8(a) at a low level and FIG. 9(b) is an enlarged view showing region B of FIG.
9(a);
] FIG. 10 is a view showing an image acquired by image processing the image shown in FIG. 8(a) at a high level and FIG. 10(b) is an enlarged view showing region C of FIG. 10(a);
] FIG. 11(a) is a view showing an image acquired by extracting the maximum
brightness point from the image shown in FIG. 10(b) and FIG. 11(b) is a view showing an image acquired by applying the maximum brightness point to the image shown in FIG. 9(a);
] FIG. 12(a) is a view showing a line scanning process performed about the maximum brightness point of a ball image, FIG. 12(b) is a view showing an image acquired as the result of the line scan and FIG. 12(c) is a view showing an image acquired by applying an average filter to the line scan result shown in FIG. 12(b);
[24] FIG. 13 is a view illustrating estimation of a direction in which light is directed to a ball image;
[25] FIG. 14 is a view showing an axis of the light direction estimated by a light direction estimating means of the sensing processing device according to the present invention displayed on the ball image;
[26] FIG. 15 is a view showing a process of measuring a diameter of the ball image
shown in FIG. 9(b);
[27] FIGS. 16, 17(a) and 17(b) are views showing a process of fitting a ball image in consideration of the estimated light direction;
[28] FIG. 18 is a view showing an image acquired by extracting a center point from the ball image fitted by fitting process shown in FIGS. 17(a) and 17(b);
[29] FIGS. 19 and 20 are flow charts showing respective embodiments of a sensing
processing method according to the present invention; and
[30] FIG. 21 is a flow chart showing the ball image fitting process shown in FIGS. 19 and 20 in more detail.
Best Mode for Carrying out the Invention
[31] Now, exemplary embodiments of a sensing processing device and method for a
moving ball according to the present invention and a virtual golf simulation device using the same will be described in detail with reference to the accompanying drawings.
[32] A sensing processing device for a moving ball according to the present invention is applicable to all systems that perform sensing by acquiring and analyzing an image of the state of a moving ball in sports, such as golf, using a ball. In an example, the sensing processing device may be applied to a so-called screen golf system to which a virtual golf simulation device is applied.
[33] FIGS. 1 and 2 schematically show the construction of a sensing processing device for a moving ball according to an embodiment of the present invention and a virtual golf simulation device using the same.
[34] First, a sensing processing device for a moving ball according to an embodiment of the present invention and a virtual golf simulation device using the same will be described with reference to FIGS. 1 and 2.
[35] As shown in FIG. 1, the present invention relates to a sensing processing device 220 which processes an image acquired by a sensor unit including camera devices 310 and 320, a strobe flashlight device 330 and a signal generating unit 210 to process an image of a ball, thereby extracting coordinates of a center point. Preferably, a sensing processing device 220 according to an embodiment of the present invention includes a preliminary processing means 230, an image extracting means 240, a light direction estimating means 250 and an image processing means 260.
[36] First, the sensor unit will be described. The camera devices 310 and 320 are provided to respectively capture a plurality of frames on the movement of a ball moving from an initial position in a moving direction of the ball.
[37] In FIG. 1, two camera devices 310 and 320 are provided, to which, however, the present invention is not limited. For example, a camera device or more than two camera devices may be provided.
[38] The strobe flashlight device 330 is a lighting device using light emitting diodes
(LED). The strobe flashlight device 330 is used as a light source for capturing of the camera devices. The strobe flashlight device 330 generates strobe flashlight at a predetermined time interval (that is, a plurality of flashes is generated at a predetermined time interval) so that the camera devices 310 and 320 can capture a multiple exposure image.
[39] That is, a ball is present on a frame captured by the camera devices in correspondence to the number of flashes generated from the strobe flashlight device 330 to provide a multiple exposure image thereof.
[40] The operation of the camera devices 310 and 320 and the strobe flashlight device 330 will be described below in detail.
[41] Meanwhile, a trigger signal to operate the camera devices 310 and 320 and the strobe flashlight device 330 is generated by the signal generating unit 210. The multiple exposure image captured by the camera devices 310 and 320 and the strobe flashlight device 330 is processed by the sensing processing device 220 according to the present invention and is transmitted to a simulator 100.
[42] The light direction estimating means 250 of the sensing processing device 220
according to the embodiment of the present invention is provided to extract the maximum brightness point of the ball image to estimate the light direction of the strobe flashlight passing the maximum brightness point. The light direction estimating means 250 includes a brightness analyzing means (not shown) and a scanning means (not shown), which will be described below in more detail.
[43] Meanwhile, the image processing means 260 is provided to fit the ball image into the shape of a ball based on the contour of the side of the ball image to which light is irradiated according to the light direction estimated by the light direction estimating means 250, thereby extracting the coordinates of a center point of the fitted image.
[44] That is, the image processing means 260 preferably includes a half circle fitting
means 270 to form a half circle curve of a predetermined diameter or a diameter measured through the ball image on a specific position of the ball image and to fit the half circle opposite to the half circle curve according to the diameter and a coordinate extracting means 280 to extract the coordinates of a center point of the ball image to which the half circle is fitted, which will be described below in detail.
[45] In a case in which a value measured through the ball image is used as the diameter of the half circle curve, the sensing processing device preferably further includes a diameter measuring means to measure the diameter of the ball image.
[46] Meanwhile, the preliminary processing means 230 is provided to remove a
background image and noise excluding a ball image from the multiple exposure image acquired by the sensor unit through a predetermined process.
[47] The image extracting means 240 is provided to extract a ball image from the image preliminarily processed by the preliminary processing means 230 (more specifically, to extract an image estimated as a ball, i.e. a ball image candidate). The estimation of the light direction in the ball image performed by the light direction estimating means 250 and the fitting of the ball image performed by the image processing means 260 are performed with respect to the ball image extracted by the image extracting means 240.
[48] Meanwhile, the simulator 100 preferably includes a controller M, a database 110, an image processing unit 120 and an image output unit 130.
[49] The controller M receives coordinate information of a moving ball which has been image processed and acquired by the sensing processing device 220, converts the coordinate information into three-dimensional coordinates, calculates the change of the coordinates of the moving ball in a three-dimensional space, obtains predetermined physical information for moving orbit simulation of the ball and transmits the obtained physical information to the image processing unit 120.
[50] Predetermined data for moving orbit simulation of the ball may be extracted from the database 110, and the image processing of the moving orbit simulation of the ball by the image processing unit 120 may be performed by extracting image data stored in the database 110.
[51] Meanwhile, a converting means to covert coordinate information of the moving ball transmitted from the sensing processing device 220 into three-dimensional coordinates may be provided separately from the controller M.
[52] In a hardware aspect, the preliminary processing means 230, the image extracting means 240, the light direction estimating means 250 and the image processing means 260 may be realized as a single controller configured to perform the functions of the above means or separate controllers configured to respectively performed the functions of the above means. In a software aspect, the preliminary processing means 230, the image extracting means 240, the light direction estimating means 250 and the image processing means 260 may be realized as a single program configured to perform the functions of the above means or separate programs configured to respectively performed the functions of the above means. [53] An example of a screen golf system, to which the sensing processing device or the virtual golf simulation device with the above-stated construction is applied, is shown in FIG. 2.
[54] As shown in FIG. 2, a hitting mat 30 having a golf tee, on which a golf ball is placed, and a ground (preferably including at least one selected from among a fairway mat, a rough mat and a bunker mat) and a swing plate 20, on which a golfer hits a golf ball, are provided at one side of a golf booth B.
[55] A screen 40, on which a virtual golf simulation image realized by the image output unit 130 is displayed, is provided at the front of the golf booth B. The camera devices 310 and 320 and the strobe flashlight device 330 are provided at the ceiling of the golf booth B.
[56] In FIG. 2, the camera devices 310 and 320 are provided at the ceiling and wall of the golf booth B, respectively, to which, however, the present invention is not limited. The camera devices 310 and 320 can be installed at any positions of the golf booth B so long as the camera devices 310 and 320 do not disturb the swing of the golfer and are prevented from colliding with a golf ball hit by the golfer while the camera devices 310 and 320 can effectively capture an image of a moving state of the golf ball 10.
[57] Also, in FIG. 2, the strobe flashlight device 330 is installed at the ceiling of the golf booth B to provide strobe flashlight substantially perpendicular to a hitting point, to which, however, the present invention is not limited. The strobe flashlight device 330 can be installed at any position of the golf booth B so long as the strobe flashlight device 330 can effectively provide strobe flashlight.
[58] When the golfer on the swing plate 20 hits a golf ball 10 on the hitting mat 30 toward the screen 40 in the system with the above-stated construction, as shown in FIG. 2, the camera devices 310 and 320 capturing predetermined regions in which the hitting is performed capture a plurality of frames, respectively. At this time, the strobe flashlight device 330 generates several flashes per frame to acquire a multiple exposure image of the moving golf ball.
[59] FIG. 3 is a view showing a trigger signal generation scheme of a camera device and a strobe flashlight device by the signal generating unit 210 (see FIG. 1).
[60] As shown in FIG. 3, trigger signals of the camera device have a time interval of tc.
That is, trigger signals are generated at a time interval of tc for each frame. At this time, exposure time for each frame is te (at this time, preferably tc > te). Also, data on an image acquired for a time of tc - te is preferably transmitted to the sensing processing device 220 (see FIG. 1).
[61] That is, as shown in FIG. 3, the camera device is triggered at an interval between the trigger signal generating times for each frame, and data on the acquired multiple exposure image is transmitted to the sensing processing device. [62] During the exposure time (time of te) of the camera device, strobe flashlight is generated by the strobe flashlight device several times. In FIG. 3, strobe flashlight trigger signals are generated three times at a time interval of tsl.
[63] That is, during the exposure time (time of te) of the camera device, strobe flashlight is generated three times at the same time interval of tsl. At this time, the camera device and the strobe flashlight device are preferably synchronized for the signal generating unit 210 (see FIG. 1) to generate a first trigger signal.
[64] Also, as shown in FIG. 3, a time interval of ts2 is preferably provided between the trigger signal of the last one of the three strobe flashlights and a first strobe flashlight of the next frame. The time interval of tsl and the time interval of ts2 may be set to be equal to each other. As shown in FIG. 3, however, the time interval of tsl and the time interval of ts2 are preferably set to be different from each other. In this case, the time interval of ts2 may be set to be longer than the time interval of tsl.
[65] Since the trigger signal intervals of the camera device and the strobe flashlight device are uniform as described above, the interval between the ball images on the multiple exposure image is uniform.
[66] When the multiple exposure image is processed by the sensing processing device, therefore, it is possible to effectively separate a correct ball image from the ball image present on the multiple exposure image and various noises based on the characteristics of the uniformly fixed interval of the trigger signal.
[67] Also, since the interval of the strobe flashlight is uniformly fixed for each frame of the camera device, images having various patterns may be generated from the multiple exposure image according to the moving speed of the ball.
[68] FIGS. 4(a) and 4(b) show an image of a moving ball (golf ball) when the ball is hit by the golfer using a golf club. FIGS. 4(a) and 4(b) are views showing two patterns of the multiple exposure image acquired by the camera device and the strobe flashlight device based on the signal generation scheme shown in FIG. 3, respectively.
[69] An image II shown in FIG. 4(a) is acquired through multiple exposure of golf club images CI, C2 and C3 and ball images 1 la, 1 lb and 11c. In this case, the ball images 11a, l ib and 1 lc are spaced apart from each other by a predetermined interval.
[70] An image 12 of FIG. 4(b) shows a case in which ball images overlap to provide an image region 12 of a predetermined size.
[71] That is, in the image shown in FIG. 4(a), the ball is moved at high speed, and
therefore, the image is formed in a state in which the ball images are spaced apart from each other by the predetermined interval when strobe flashlight is triggered, and, in the image shown in FIG. 4(b), the ball is moved at low speed, and therefore, strobe flashlight is triggered before the ball moves far away with the result that the ball images overlap each other. [72] The sensing processing device and method according to the present invention are provided to accurately acquire the coordinates of a center point of the ball when the ball images are separated from each other on the multiple exposure image as the ball is moved at a speed equal to or greater than a predetermined speed based on the fixed interval of the strobe flashlight as shown in FIG. 4(a). In the sensing processing method according to the present invention, the case in which the ball images overlap as shown in FIG. 4(b) is excluded.
[73] The overlapped ball images are processed by an additional process, which deviates from the scope of the present invention, and therefore, a detailed description thereof will be omitted.
[74] Hereinafter, processing of the separate ball images as shown in FIG. 4(a) by the sensing processing device for the moving ball according to the present invention will be described with reference to FIGS. 5 to 18.
[75] FIG. 5(a) is a view showing the original copy of a multiple exposure image including separate ball images. The multiple exposure image is acquired using a camera device having a low resolution of 320 x 240 and operated at a low speed of 75 fps and a strobe flashlight device operating at 330 Hz.
[76] The preliminary processing means 230 (see FIG. 1) removes a stationary image, i.e. a background image, from the original image shown in FIG. 5(a) through subtraction and performs a predetermined preliminary processing operation such as Gaussian blur. An image acquired through the preliminary processing operation is shown in FIG. 5(b).
[77] When the background image and noise are properly removed from the multiple
exposure image by the preliminary processing means as described above, it is possible to easily extract a ball image, which will hereinafter be described with reference to FIG. 6.
[78] The image shown in FIG. 6 is obtained by overlapping a plurality of frames captured by the camera device into a single image. On the assumption that three ball images formed by generating three flashes constitute an image set, an image set SO indicates an image of a zeroth frame, an image set S 1 indicates an image of a first frame, and an image set S2 indicates an image of a second frame.
[79] The image extracting means 240 (see FIG. 1) may extract an image estimated by the ball image through a contour check or a check window. The processing may be carried out so that the final image estimated as the ball is kept while images estimated not to be the ball are discarded.
[80] That is, the ball has a predetermined diameter. Therefore, the contours of the respective images on the multiple exposure image are checked in consideration of the diameter of the ball, and the checked contours may be excluded if they are too large or too small to be the ball. [81] Also, as shown in FIG. 6, check windows W corresponding to the size of the respective images 11 of the multiple exposure image are formed so as to include the respective images 11, and an aspect ratio of the respective check windows W is checked.
[82] Since the ball is circular, the aspect ratio of the ball must be almost equal. If the
aspect ratio of the check windows W corresponding to the respective images 11 on the multiple exposure image is too large with the result that each of the check windows W has a large width or is too small with the result that each of the check windows W has a small width, the respective images 11 on the multiple exposure image are considered not to be the ball and thus may be discarded.
[83] Consequently, it is possible to extract images 11 estimated as the ball from the
multiple exposure image using the above illustrated method.
[84] Meanwhile, the images 11 extracted as the ball images are fitted into the shape of a ball as shown in FIGS. 7(a), 7(b) and 7(c) to extract the coordinates of a center point of each of the images.
[85] FIG. 7(a) is a view showing an example of a ball image acquired under strobe
flashlight, FIG. 7(b) is a view showing an incorrect fitting result of the ball and FIG. 7(c) is a view showing a result of the ball correctly fitted using the half circle fitting means 270 (see FIG. 1) of the image processing means according to the present invention.
[86] As shown in FIG. 7(a), the ball image 11 is captured while strobe flashlight is irradiated. Consequently, an irradiated region Rl clearly appears at one side of the ball image 11 , and a shaded region appears at the region opposite to the irradiated region Rl. That is, a shaded region R2 covering a portion of the ball image 11 is present at the region opposite to the irradiated region Rl.
[87] According to light angle, a shaded region may appear on only a portion of the ball image or may cover more than the half of the ball image.
[88] If the ball image is fitted into the shape of a ball as shown in FIG. 7(b) when the ball image acquired under the strobe flashlight is shaded, the center point of the ball image is distorted with the result that coordinate information of the center point contains an error, and therefore, the accuracy in moving simulation of the ball is greatly lowered.
[89] Preferably, therefore, the fitting of the ball image acquired under strobe flashlight is performed in consideration of the shaded region present at the ball image.
[90] As an example of the fitting of the ball image in consideration of the shaded region, the light direction of the strobe flashlight irradiated to the ball may be estimated and the fitting may be performed according to the light direction so that the shaded region is included in a fitted curve.
[91] FIG. 7(c) shows fitting of the ball image in consideration of the light direction. As shown in FIG. 7(c), a fitted curve FC is formed at the irradiated region Rl of the ball image 11 so that the shaded region R2 is included in the fitted curve FC.
[92] In this way, the ball image is fitted, and therefore, the center point CP of the ball image is accurately extracted.
[93] In order to accurately extract the center point of the ball image as described above, it is necessary to perform a preliminary processing operation so that the background image and noise are properly removed from the initially acquired multiple exposure image, and therefore, only the ball image remains. For estimation of the light direction and fitting of the ball image, it is necessary to perform a preliminary processing operation, such as proper Gaussian blur with respect to the image, threshold processing with respect to a specific pixel value and normalization of the pixel value of the image.
[94] Also, the ball image may be damaged as the result of the preliminary processing to extract specific information from the image. For this reason, images are preferably formed in stages so that specific information can be extracted from the image in each of the stages.
[95] FIGS. 8(a) to 10(b) show images formed in stages according to the preliminary
processing operation performed by the preliminary processing means 230 (see FIG. 1).
[96] FIG. 8(a) shows an image acquired by removing a background from an initially
acquired multiple exposure image through subtraction, and FIG. 8(b) is an enlarged view of region A of FIG. 8(a), showing the original ball image 11-0.
[97] As shown in FIG. 8(b), the ball has a plurality of dimples with the result that the effect of the light is irregular. Consequently, the distribution of pixel values of the original ball image 11-0 is very irregular. Also, there exists a plurality of salt and pepper noises having irregular pixel values in one or more small pixel units at the periphery thereof.
[98] It is very difficult to obtain various information necessary to accurately extract the center point from an image in such a state and the extracted information may contain an error.
[99] Therefore, it is necessary to remove all peripheral noises and to properly adjust the distribution of the irregular pixel values of the original ball image 11-0.
[100] FIG. 9(a) shows an image acquired by almost completely removing various peripheral noises from the image shown in FIG. 8(a) through Gaussian blur processing and threshold processing at a proper level at which the original ball image is not greatly damaged, and FIG. 9(b) is an enlarged view of region B of FIG. 9(a), showing a first processed image 11-1.
[101] FIG. 10(a) shows an image acquired by performing Gaussian blur processing and threshold processing at a level higher than the first processed image 11-1 with respect to the image shown in FIG. 8(a) and performing normalization processing of the pixel values of the ball image, and FIG. 10(b) is an enlarged view of region C of FIG. 10(a), showing a second processed image 11-2.
[102] In the second processed image 11-2, the shape of the original ball image 11-0 is greatly damaged. However, it can be seen that the effect caused by the irregular distribution of the pixel values due to the dimples of the ball is removed and the irradiated region is clearly divided.
[103] Preferably, therefore, as shown in FIG. 11, the light direction is estimated using the image shown in FIG. 10(c), i.e. the second processed image 11-2.
[104] In the second processed image 11-2, the pixel values are properly distributed and the pixel values are clearly divided based on sizes thereof, whereby it is possible to easily estimate the light direction by finding a region in which the pixel values are high. Here, the image is a grayscale image, and therefore, the pixel values only have information on brightness values.
[105] That is, it can be seen that in the second processed image 11-2 shown in FIG. 11(a), light is concentrated on a region R3 at which the brightness is relatively high. The center of the region R3 may be considered to be the center point of the light direction. That is, a brightest point is a point through which an axis of the light direction passes. FIG. 11(a) shows the center point of the region R3, which is the brightest point, i.e. the maximum brightness point PI.
[106] If only one of the pixels on the ball image 11-2 of the second processed image is the brightest point, the pixel may be designated as the maximum brightness point PI. However, in an irregularly shaped region in which several pixels has the brightness points, the center of mass of the region may be extracted through a predetermined process, and the corresponding point may be designated as the maximum brightness point PI.
[107] Meanwhile, the light direction may be estimated by selecting the maximum
brightness point PI on the ball image as a first point and another point as a second point, forming a line connecting the first point and the second point and setting the line as the axis of the light direction. The first point may be extracted by the brightness analyzing means of the light direction estimating means 250 (see FIG. 1) and the second point may be extracted by the scanning means of the light direction estimating means 250 (see FIG. 1).
[108] The brightness analyzing means may find the center point, i.e. the maximum
brightness point PI, of the region R3 on the ball image shown in FIG. 11(a) to extract the first point.
[109] The scanning means may extract the second point through the process as shown in FIGS. 11(b), 12(a) and 12(b). First, as shown in FIG. 11(b), the maximum brightness point PI shown in FIG. 11(a) is applied to the ball image 11-1 of the first processed image. [110] That is, the coordinate values of the maximum brightness point PI on the ball image 1 1-2 of the second processed image may be applied to the ball image 11-1 of the first processed image or the ball image 11-2 of the second processed image may be overlapped with the ball image 11-1 of the first processed image so that the maximum brightness point PI can be displayed on the ball image 11-1 of the first processed image.
[111] As shown in FIG. 12(a), 360 degree line scanning is performed on the ball image
1 1-1 of the first processed image about the maximum brightness point PI.
[112] FIG. 12(b) shows the line scanning results of the ball image shown in FIG. 12(a).
[113] As shown in FIG. 12(b), a contour line LI of the ball image 1 1-1 of the first
processed image formed by the line scanning has a very irregular shape.
[114] Preferably, therefore, as shown in FIG. 12(c), the contour line is adjusted to be a fitted contour line L2 using an average filter.
[115] As shown in FIG. 12(c), it can be seen from the line scanning result of the ball image that a point forming the shortest distance Dmin and a point forming the longest distance Dmax clearly appear.
[116] Preferably, in the line scanning result, the point P2 forming the shortest distance
Dmin is selected as the second point. This will hereinafter be explained with reference to FIG. 13.
[117] As shown in FIG. 13, the ball image 11 acquired by capturing the ball 11 while
strobe flashlight is irradiated to the ball 11 during movement of the ball 11 includes a portion 11-1 captured while light is irradiated and a shaded portion R2. The captured portion 11-1 has a portion Rl appearing brighter under the influence of the light. Also, a portion R3 having the maximum pixel brightness value is included in the brighter portion Rl.
[118] It can be seen in the direction of the strobe flashlight that the brightest portion, i.e. the maximum brightness point PI, on the ball image becomes the center of the axis AX of the light direction.
[119] As shown in Fig. 13, it can be estimated that the axis AX of the light direction passes through the point, i.e. the second point P2, on the contour line on which the distance between the maximum brightness point PI and the contour of the ball image 1 1-1 is the shortest.
[120] Consequently, the point on the contour line of the ball image 11-1 nearest the
maximum brightness point PI may be designated as the second point P2.
[121] The second point P2 may be decided as the point P2 on the contour line forming the shortest distance as the result of the line scanning process shown in FIGS. 12(a) to 12(c).
[122] Consequently, a line connecting the first point PI, i.e. the maximum brightness point, and the second point P2 obtained using the above method may be set as the axis AX of the light direction.
[123] FIG. 14 is a view showing an image acquired by obtaining the first point PI and the second point P2 on the ball image 11-1 in a state as shown in FIG. 12(b) and extracting the axis AX of the light direction connecting the two points.
[124] Meanwhile, after the axis AX of the light direction in the ball image is extracted as shown in FIG. 14, fitting of the ball image and coordinates extraction of the center point are performed as shown in FIGS. 15 to 17 using the image processing means 260 (see FIG. 1).
[125] Preferably, the fitting is not performed using the second processed image 11-2 but using the first processed image 11-1 in order to improve the accuracy in fitting of the ball image.
[126] This is because in the first processed image 11-1, the shape of the ball image is not greatly damaged, and therefore, it is possible to more accurately extract the center point thereof upon fitting.
[127] Meanwhile, for fitting of the ball image, the sensing processing device according to the embodiment of the present invention includes a diameter measuring means to measure the diameter of the ball image.
[128] A predetermined value may be used as the diameter for fitting of the ball image. Alternatively, the diameter of the ball on the ball image may be measured so that the fitting is performed using the measured diameter.
[129] However, the shape of the ball may be somewhat damaged due to the influence of the shaded portions on the initially acquired image, the limit of the resolution and the performance of several preliminary processing operations. Preferably, therefore, the fitting is performed using the diameter measured from the ball image rather than using the predetermined diameter value to more accurately extract the center point.
[130] FIG. 15 is a view showing a process of measuring the diameter of the ball image 11-1 using the diameter measuring means.
[131] First, in order to easily measure the diameter, as shown in FIG. 15, the axis of the light direction is preferably vertically arranged so that the irradiated region is present at the upper part of the ball image and the shaded region is present at the lower part of the ball image.
[132] Here, as shown in FIG. 15, line scanning is performed through horizontal lines L along the axis of the light direction to extract the longest one of the sections in which the pixel value is changed to a predetermined value or more as a diameter De.
[133] When the diameter is measured by the diameter measuring means as described
above, the ball image is fitted by the half circle fitting means 270 (see FIG. 1). The fitting operation is performed as shown in FIGS. 16, 17(a) and 17(b). [134] The half circle fitting means fits the ball image so that the shaded region is included when the shaded region is present on the ball image by light and therefore, the coordinates of the center point can be accurately extracted.
[135] More specifically, the half circle fitting means preferably includes a pixel checking means to check a pixel value on the ball image along the direction estimated as the light direction and a fitting processing means to form a half circle curve at a portion wt which the checked pixel value is changed to a predetermined value or more and fit the other half circle.
[136] In a case in which the axis AX of the light direction based on the estimated light direction is inclined at an arbitrary angle as shown in FIG. 14, it is preferable to rotate the entire image so that the axis AX of the light direction is vertically arranged as shown in FIG. 16 in order to more easily fit the half circle.
[137] For half circle fitting, first, a half circle curve FC-1 is prepared as shown in FIG. 16 based on the diameter De measured by the diameter measuring means
[138] The half circle curve FC-1 is moved toward the ball image 11 along the axis AX of the light direction which is vertically arranged as shown in FIG. 16.
[139] At this time, the pixel checking means checks the pixel value (that is, the brightness value) of the image on the half circle curve FC-1. More preferably, the pixel checking means calculates the sum of the brightness values of the image on the half circle curve FC-1 along the axis AX of the light direction to detect the change.
[140] When the brightness value is calculated while the half circle curve FC-1 is moved along the axis AX of the light direction or the sum of the brightness values of the image on the half circle curve is calculated and the upper end contour of the ball image is reached, the brightness value or the sum of the brightness values of the image on the half circle curve are abruptly increased.
[141] The position at which the brightness value checked along the axis AX of the light direction or the sum of the brightness values of the image on the half circle curve is abruptly changed may be determined as the contour of the ball image 11. The fitting processing means locates the half circle curve FC-1 at the position.
[142] That is, as shown in FIG. 17(a), the fitting processing means covers the contour of the irradiated region Rl of the ball image 11 with the half circle curve FC-1.
[143] As shown in FIG. 17(b), the fitting processing means fits the other half circle curve FC-2 according to the measured diameter De to achieve correct fitting including the shaded region R2 of the ball image 11.
[144] When the ball image is fitted through the above process, the coordinate extracting means 280 (see FIG. 1) can accurately extract the coordinates of a center point CP of the ball image through the fitted curve FC of the ball image as shown in FIG. 18.
[145] Therefore, it is possible to accurately extract the coordinates of the center point of the ball although the low quality image acquired by the low resolution and low speed camera device and strobe flashlight device is used.
[146] The coordinates of the center point of the ball extracted as described above are
converted into three-dimensional coordinates by the converting means, and physical information of the moving orbit of the ball to be simulated is obtained based on the three-dimensional coordinates.
[147] Hereinafter, embodiments of a sensing processing method for a moving ball
according to the present invention will be described with reference to FIGS. 19 and 20.
[148] As shown in FIG. 19, first, a trigger signal is applied by the signal generating unit so that the camera device and the strobe flashlight device capture a multiple exposure image of the moving state of a ball at a predetermined interval (S10).
[149] Various preliminary processing steps are carried out to remove a background image and various noises from the acquired multiple exposure image (S20).
[150] A ball image is extracted from the preliminarily processed image (S30) (when the ball image is extracted, noise may also be extracted, and therefore, more strictly speaking, a ball image candidate is extracted) and pixel values, i.e. brightness values, of the extracted ball image are checked (S40) to extract a first point having the highest brightness value (S50).
[151] 360 degree line scanning is performed on the ball image about the extracted first point (S60) and the average filter is applied to the contour line of the ball image of the extracted line scanning result to smooth the contour line (S70).
[152] A point on the contour line of the ball image which is connected to the first point at the shortest distance is extracted as a second point (S80).
[153] A line passing through the first point and the second point is formed and the line is set as an axis of the light direction (S90).
[154] However, an angle between the axis of the light direction set as described above and the vertical line or the horizontal line may deviate from a predetermined range.
[155] In the above description, it is assumed that the axis of the light direction passes
through a point forming the shortest distance between the maximum brightness point and the contour line to extract the second point. However, when the angle between the light direction and the vertical line exceeds the predetermined range, the above assumption may not be satisfied.
[156] In this case, a step of adjusting the angle of the axis of the light direction using a predetermined process may be necessary.
[157] Therefore, it is determined whether the angle defined by the axis of the light direction is within the predetermined range (SlOO). When it is determined that the angle between the axis of the light direction and the vertical line is within the predetermined range, the axis of the light direction is preferably used. On the other hand, when it is de- termined that the angle between the axis of the light direction and the vertical line is not within the predetermined range, the light direction estimated through the steps S60 to S80 is corrected, and the corrected axis is set as the axis of the light direction (SI 10).
[158] When the axis of the light direction is finally decided, the ball image is fitted in consideration of the light direction estimated as described above (S120), and a center point of the fitted image is extracted (S130).
[159] In a sensing processing method according to an embodiment shown in FIG. 20, on the other hand, a center point of the ball is extracted separately using the images formed respectively at the preliminary processing steps shown in FIGS. 8 to 11.
[160] The sensing processing method according to this embodiment is basically identical to the sensing processing method according to the embodiment shown in FIG. 19 except that images are separately formed at the respective preliminary processing steps and the light direction is estimated based on the separately formed images. Therefore, a description of the duplicated portions will be omitted, and only the difference will be described in detail.
[161] A preliminary processing step is carried out with respect to the acquired multiple exposure image (S20), and a first processed image and a second processed image are formed based on the preliminary processing level of the image (S21 and S22).
[162] The first processed image is shown in FIG. 9, and the second processed image is shown in FIG. 10.
[163] A ball image is extracted from the second processed image (S30) and pixel values, i.e. brightness values, of the extracted ball image are checked (S40) to extract a first point having the highest brightness value (S50).
[164] On the other hand, the extracted first point is applied to the first processed image and
360 degree line scanning is performed on the ball image of the first processed image about the first point (S60). The average filter is applied to the contour line of the ball image of the extracted line scanning result (S70).
[165] Subsequent steps are identical to those shown in FIG. 19, and therefore, a detailed description thereof will be omitted.
[166] Meanwhile, FIG. 21 shows the step S120 shown in FIGS. 19 and 20, i.e. the step of fitting the ball image in consideration of the light direction, in more detail. As shown in FIG. 21, the diameter of the ball image is preferably measured for fitting (S123).
[167] Here, the diameter of the ball image may be measured as illustrated in FIG. 12 in a state in which the ball image is rotated so that the axis of the light direction is vertically arranged.
[168] Subsequently, a half circle curve based on the measured diameter is formed (S124).
A pixel value (brightness value) is checked along the axis of the light direction and the half circle curve is located at the contour of the irradiated portion of the ball image (S125).
[169] Subsequently, the other half circle based on the measured diameter is formed at the side opposite to the half circle to fit the ball image (SI 26) and the coordinates of a center point of the fitted ball image are extracted (SI 30).
[170] The change of the extracted coordinates of the center point of the ball image is
calculated to obtain physical property information of the moving ball.
Mode for the Invention
[171] Various embodiments of a sensing processing device and method for a moving ball and a virtual golf simulation device using the same have been described in the best mode for carrying out the invention.
Industrial Applicability
[172] In a sensing processing device and method for a moving ball according to the present invention as described above and a virtual golf simulation device using the same, although a low quality image is acquired by a low resolution and low speed camera device and strobe flashlight device, it is possible to obtain coordinates of a center point of a ball image in the acquired image at a sub pixel level, thereby achieving high sensing processing performance and sensing accuracy at low cost and a virtual golf simulation device using the same. Consequently, the present invention can be widely used in industries related to a sensing processing device and method for a moving ball and a virtual golf simulation device using the same.

Claims

Claims
A sensing processing device to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing device comprising:
a light direction estimating means to analyze a ball image from the acquired image and to estimate a direction of the light toward the moving ball; and
an image processing means to fit the ball image into the shape of a ball based on an irradiated side of the ball image along the estimated light direction and to extract coordinates of a center point of the fitted image. The sensing processing device according to claim 1, wherein the light direction estimating means is configured to extract the maximum brightness point of the ball image and to estimate the direction of the light toward the moving ball, and
the image processing means is configured to fit the ball image including a shaded region of the ball image opposite to the maximum brightness point.
The sensing processing device according to claim 1, wherein the light direction estimating means comprises:
a brightness analyzing means to extract a first point having the maximum brightness value among all pixels of the ball image; and a scanning means to extract a second point on the contour of the ball image nearest the first point, wherein
a line connecting the first point and the second point is set as an axis of the light direction.
The sensing processing device according to claim 3, wherein the brightness analyzing means is configured to check a pixel having the maximum brightness value among all the pixels of the ball image and to calculate the center of mass of the at least one checked pixel to obtain the first point.
The sensing processing device according to claim 3, wherein the scanning means is configured to perform 360 degree line scanning with respect to the ball image about the first point to extract the second point.
The sensing processing device according to claim 1, further comprising a preliminary processing means configured to perform preliminary processing to remove a background image and noise from the acquired image and to form a first processed image and a second processed
image, which is preliminarily processed at a higher level than the first processed image, according to a preliminary processing degree, wherein
the light direction estimating means comprises a brightness analyzing means to extract a first point having the maximum brightness value among all pixels of the second processed image and a scanning means to scan the shortest distance from a point on the first processed image corresponding to the first point to the contour of the first processed image to extract a second point on the contour corresponding to the distance, wherein
a line connecting the first point and the second point is set as an axis of the light direction.
[Claim 7] The sensing processing device according to claim 1, wherein the image processing means comprises a half circle fitting means to form a half circle curve of a predetermined diameter at a specific position of the ball image based on the light direction estimated by the light direction estimating means and to fit a half circle opposite to the half circle curve based on the diameter.
[Claim 8] A sensing processing device to acquire and analyze an image of a
moving ball under predetermined lighting conditions, the sensing processing device comprising:
a half circle fitting means to form a half circle curve of a predetermined diameter or a diameter measured through the ball image at a specific position of the ball image based on the light direction estimated by the light direction estimating means and to fit a half circle opposite to the half circle curve based on the diameter; and
a coordinate extracting means to extract coordinates of a center point of the half circle fitted ball image.
[Claim 9] The sensing processing device according to claim 8, wherein the half circle fitting means is configured to form the half circle curve at the contour of the ball image on the maximum brightness point side and fit a half circle opposite to the half circle curve so that the ball image is fitted including a shaded region of the ball image.
[Claim 10] The sensing processing device according to claim 8, further comprising a diameter measuring means to analyze the ball image to measure the diameter of the ball image, wherein
the half circle fitting means is configured to perform half circle fitting based on the measured diameter.
[Claim 11] The sensing processing device according to claim 8, wherein the half circle fitting means comprises:
a pixel checking means to check a pixel of the ball image along a direction estimated as the direction of the light toward the moving ball; and
a fitting processing means to form the half circle curve at a position at which the checked pixel value is changed to a predetermined value or more and fit a half circle opposite to the half circle curve.
[Claim 12] A sensing processing method to acquire and analyze an image of a moving ball under predetermined lighting conditions, the sensing processing method comprising:
analyzing a ball image from the acquired image to estimate the direction of the light toward the moving ball;
fitting the ball image into the shape of a ball based on an irradiated side of the ball image along the estimated light direction; and
extracting coordinates of a center point of the fitted image.
[Claim 13] The sensing processing method according to claim 12, wherein the step of estimating the light direction comprises:
extracting a first point having the maximum brightness value among all pixels of the ball image;
extracting a second point on the contour of the ball image nearest the first point; and
setting a line connecting the first point and the second point as an axis of the direction of the light toward the ball image.
[Claim 14] The sensing processing method according to claim 13, wherein the step of extracting the first point comprises:
checking a pixel having the maximum brightness value among all the pixels of the ball image; and
calculating the center of mass of the at least one checked pixel to obtain the first point.
[Claim 15] The sensing processing method according to claim 13, wherein the step of extracting the second point comprises:
performing 360 degree line scanning with respect to the ball image about the first point; and
extracting a point corresponding to the shortest distance between the first point and the contour of the ball image as the second point.
[Claim 16] The sensing processing method according to claim 13, wherein the step of estimating the light direction comprises:
determining whether an angle defined by the axis of the estimated light direction deviates from a predetermined range; and
correcting the estimated light direction upon determining that the angle defined by the axis of the estimated light direction deviates from the predetermined range.
[Claim 17] The sensing processing method according to claim 12, wherein the fitting step comprises:
forming a half circle curve of a predetermined diameter or a diameter measured through the ball image at a specific position of the ball image based on the estimated light direction; and
fitting a half circle opposite to the half circle curve based on the diameter.
[Claim 18] The sensing processing method according to claim 17, wherein the step of forming the half circle curve comprises:
analyzing the ball image to extracting a shaded region of the ball image generated by the light; and
forming the half circle curve on the ball image at a side opposite to the shaded region.
[Claim 19] The sensing processing method according to claim 17, wherein the step of forming the half circle curve comprises:
extracting an axis of the light direction based on the estimated light direction;
rotating the ball image so that the axis of the light direction is vertically arranged;
checking the sum of pixel values on the half circle curve while moving the half circle curve to the ball image along the axis of the light direction; and
locating the half circle curve at a position at which the checked sum of the pixel values is changed to a predetermined value or more.
[Claim 20] The sensing processing method according to claim 17, wherein
the step of forming the half circle curve comprises analyzing the ball image to measure a diameter of the ball image and forming a half circle curve based on the measured diameter, and
the fitting step comprises fitting a half circle opposite to the half circle curve formed based on the measured diameter.
[Claim 21] The sensing processing method according to claim 20, wherein the step of measuring the diameter of the ball image comprises: extracting an axis of the light direction based on the estimated light direction;
rotating the ball image so that the axis of the light direction is vertically arranged;
performing horizontal line scanning with respect to the ball image along the vertically rotated axis of the light direction; and
estimating the length between two points of a horizontal line having the longest length between the two points meeting the contour of the ball image as the diameter of the ball image.
[Claim 22] The sensing processing method according to claim 21, wherein the step of measuring the diameter of the ball image comprises:
determining whether a difference between the length estimated as the diameter at the step of estimating the diameter of the ball image and the predetermined diameter of the ball is within a predetermined range; and setting the estimated length as the measured diameter upon determining that the difference is within the predetermined range.
[Claim 23] The sensing processing method according to claim 21, wherein the step of measuring the diameter of the ball image comprises analyzing a shade degree of the ball image based on the light and correcting the length estimated as the diameter at the step of estimating the diameter of the ball image according to a result of the shade degree analysis.
[Claim 24] A virtual golf simulation device comprising:
a sensor unit comprising a camera device and a strobe flashlight device to acquire a multiple exposure image of a moving golf ball hit by a golfer;
a sensing processing unit comprising a light direction estimating means to extract and analyze an image of the golf ball from the acquired image and to estimate a direction of light emitted from the strobe flashlight device toward the golf ball and an image processing means to fit the golf ball image into the shape of a golf ball based on an irradiated side of the golf ball image along the estimated light direction and to extract coordinates of a center point of the fitted image; and a simulator to calculate the change of the coordinates of the center point and to obtain physical information on the moving golf ball, thereby simulating an orbit of the golf ball.
PCT/KR2011/004759 2010-06-29 2011-06-29 Sensing processing device and method for moving ball and virtual golf simulation device using the same WO2012002732A2 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201180041833.7A CN103079651B (en) 2010-06-29 2011-06-29 The sensing treatment device of moving ball and method and use the virtual golf ball analogue means of these apparatus and method
JP2013518250A JP5626939B2 (en) 2010-06-29 2011-06-29 Sensing processing device for moving ball, sensing processing method, and virtual golf simulation device using the same

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
KR10-2010-0062200 2010-06-29
KR1020100062207A KR101019829B1 (en) 2010-06-29 2010-06-29 Sensing processing device and method for moving object, and virtual golf simulation device using the same
KR10-2010-0062207 2010-06-29
KR1020100062200A KR101019902B1 (en) 2010-06-29 2010-06-29 Sensing processing device and method for moving object, and virtual golf simulation device using the same

Publications (2)

Publication Number Publication Date
WO2012002732A2 true WO2012002732A2 (en) 2012-01-05
WO2012002732A3 WO2012002732A3 (en) 2012-04-26

Family

ID=45402567

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2011/004759 WO2012002732A2 (en) 2010-06-29 2011-06-29 Sensing processing device and method for moving ball and virtual golf simulation device using the same

Country Status (3)

Country Link
JP (1) JP5626939B2 (en)
CN (1) CN103079651B (en)
WO (1) WO2012002732A2 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105288982A (en) * 2014-07-23 2016-02-03 黄详龙 Motion state testing device for golf
CN111816579A (en) * 2020-07-10 2020-10-23 英特尔产品（成都）有限公司 Method and apparatus for detecting chips contaminated with flux residue

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101723432B1 (en) * 2015-06-12 2017-04-18 주식회사 골프존 Device for sensing moving ball and method for the same
JP6606930B2 (en) * 2015-09-04 2019-11-20 株式会社プロギア Imaging device
US10224149B2 (en) 2015-12-09 2019-03-05 Kemet Electronics Corporation Bulk MLCC capacitor module
KR101850139B1 (en) * 2016-06-29 2018-04-19 주식회사 크리에이츠 Method, system and non-transitory computer-readable recording medium for compensating brightness of ball images
CN111355898B (en) * 2019-12-02 2021-12-24 联想(北京)有限公司 Method for obtaining photo by photographing and electronic equipment
JP7130118B2 (en) 2020-01-16 2022-09-02 クリーツ インク Method, system and non-transitory computer readable medium for measuring ball rotation
WO2021147228A1 (en) * 2020-01-21 2021-07-29 神盾股份有限公司 Image scanning device and image scanning method

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001264016A (en) * 2000-03-15 2001-09-26 Sumitomo Rubber Ind Ltd Motion-measuring instrument for ball
JP2003117044A (en) * 2001-04-26 2003-04-22 Photron Ltd Hit ball diagnosing device
US20040030527A1 (en) * 2002-02-07 2004-02-12 Accu-Sport International, Inc. Methods, apparatus and computer program products for processing images of a golf ball

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11304589A (en) * 1998-04-22 1999-11-05 Toppan Printing Co Ltd Ambient light measuring system
JP4213807B2 (en) * 1999-03-19 2009-01-21 Ｓｒｉスポーツ株式会社 Ball trajectory measuring device
JP4466133B2 (en) * 2004-03-09 2010-05-26 横浜ゴム株式会社 Moving body measuring device
JP2005291824A (en) * 2004-03-31 2005-10-20 Yokohama National Univ Flying behavior measuring apparatus of flying object, and flying behavior measuring method of flying object

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001264016A (en) * 2000-03-15 2001-09-26 Sumitomo Rubber Ind Ltd Motion-measuring instrument for ball
JP2003117044A (en) * 2001-04-26 2003-04-22 Photron Ltd Hit ball diagnosing device
US20040030527A1 (en) * 2002-02-07 2004-02-12 Accu-Sport International, Inc. Methods, apparatus and computer program products for processing images of a golf ball

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105288982A (en) * 2014-07-23 2016-02-03 黄详龙 Motion state testing device for golf
CN105288982B (en) * 2014-07-23 2018-02-23 黄详龙 The motion state measure device of golf
CN111816579A (en) * 2020-07-10 2020-10-23 英特尔产品（成都）有限公司 Method and apparatus for detecting chips contaminated with flux residue
CN111816579B (en) * 2020-07-10 2023-04-07 英特尔产品（成都）有限公司 Method and apparatus for detecting chips contaminated with flux residue

Also Published As

Publication number Publication date
CN103079651B (en) 2016-01-20
JP2013535715A (en) 2013-09-12
WO2012002732A3 (en) 2012-04-26
JP5626939B2 (en) 2014-11-19
CN103079651A (en) 2013-05-01

Similar Documents

Publication Publication Date Title
WO2012002732A2 (en) Sensing processing device and method for moving ball and virtual golf simulation device using the same
US11033826B2 (en) Methods and systems for sports simulation
KR101019823B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
US9514379B2 (en) Sensing device and method used for virtual golf simulation apparatus
CA2830497C (en) Virtual golf simulation apparatus and sensing device and method used for the same
KR101019829B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
JP5572853B2 (en) Sensing device for moving object, sensing processing method, and virtual golf simulation device using the same
KR101019801B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
KR101019902B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
KR101019782B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
CA3042375C (en) Sensing device for calculating information on golf shot of user and sensing method using the same
KR101019824B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
KR101019847B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
WO2012002733A2 (en) Sensing device and sensing processing method for moving object and virtual golf simulation device using the same
KR101019798B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
KR101078954B1 (en) Apparatus for virtual golf simulation, and sensing device and method used to the same

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 201180041833.7

Country of ref document: CN

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11801138

Country of ref document: EP

Kind code of ref document: A2

ENP Entry into the national phase

Ref document number: 2013518250

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11801138

Country of ref document: EP

Kind code of ref document: A2