WO2012002731A2 - Sensing device and method for moving object and virtual golf simulation device using the same - Google Patents

Sensing device and method for moving object and virtual golf simulation device using the same Download PDF

Info

Publication number
WO2012002731A2
WO2012002731A2 PCT/KR2011/004758 KR2011004758W WO2012002731A2 WO 2012002731 A2 WO2012002731 A2 WO 2012002731A2 KR 2011004758 W KR2011004758 W KR 2011004758W WO 2012002731 A2 WO2012002731 A2 WO 2012002731A2
Authority
WO
WIPO (PCT)
Prior art keywords
images
image
multiple exposure
sensing
exposure image
Prior art date
Application number
PCT/KR2011/004758
Other languages
French (fr)
Other versions
WO2012002731A3 (en
Inventor
Won Il Kim
Young Chan Kim
Chang Heon Woo
Min Soo Hahn
Tae Wha Lee
Myung Sun Choi
Jae Wook Jung
Shin Kang
Seung Woo Lee
Han Bit Park
Myung Woo Kim
Yong Woo Kim
Seong Kook Heo
Original Assignee
Golfzon Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Golfzon Co., Ltd. filed Critical Golfzon Co., Ltd.
Publication of WO2012002731A2 publication Critical patent/WO2012002731A2/en
Publication of WO2012002731A3 publication Critical patent/WO2012002731A3/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06—Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619—Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622—Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G09B19/003—Repetitive work cycles; Sequence of movements
    • G09B19/0038—Sports
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0021—Tracking a path or terminating locations
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B69/00—Training appliances or apparatus for special sports
    • A63B69/36—Training appliances or apparatus for special sports for golf
    • A63B69/3658—Means associated with the ball for indicating or measuring, e.g. speed, direction
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00—Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/04—Games or sports accessories not covered in groups A63B1/00 - A63B69/00 for small-room or indoor sporting games
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/16—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/56—Cameras or camera modules comprising electronic image sensors; Control thereof provided with illuminating means
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0021—Tracking a path or terminating locations
    • A63B2024/0028—Tracking the path of an object, e.g. a ball inside a soccer pitch
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0021—Tracking a path or terminating locations
    • A63B2024/0028—Tracking the path of an object, e.g. a ball inside a soccer pitch
    • A63B2024/0031—Tracking the path of an object, e.g. a ball inside a soccer pitch at the starting point
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B24/00—Electric or electronic controls for exercising apparatus of preceding groups; Controlling or monitoring of exercises, sportive games, training or athletic performances
    • A63B24/0021—Tracking a path or terminating locations
    • A63B2024/0028—Tracking the path of an object, e.g. a ball inside a soccer pitch
    • A63B2024/0034—Tracking the path of an object, e.g. a ball inside a soccer pitch during flight
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/05—Image processing for measuring physical parameters
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/20—Distances or displacements
    • A63B2220/24—Angular displacement
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/30—Speed
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/806—Video cameras
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B2220/00—Measuring of physical parameters relating to sporting activity
    • A63B2220/80—Special sensors, transducers or devices therefor
    • A63B2220/807—Photo cameras

Definitions

  • the present invention relates to a sensing device and method and a virtual golf
  • simulation device using the same, and, more particularly, to a sensing device and method for a moving object that acquires and analyzes an image of a moving object, such as a golf ball, to obtain physical information thereof and a virtual golf simulation device using the same.
  • a sensing device and method for a moving object that acquires and analyzes an image of a moving object, such as a golf ball, to obtain physical information thereof and a virtual golf simulation device using the same.
  • sensing systems such as a sensing system using an infrared sensor, a sensing system using a laser sensor, a sensing system using an acoustic sensor and a sensing system using a camera sensor, have come onto the market.
  • a sensing device for a moving object including at least one camera device to acquire plural frames of images with respect to a moving state of an object, a strobe flashlight device to generate a plurality of flashes per frame to acquire a multiple exposure image, and a sensing processing unit including a first processing means to extract coordinates of the object in separate object images of the acquired multiple exposure image and a second processing means to extract coordinates of the object in overlapping object images of the acquired multiple exposure image.
  • sensing device for a moving object including at least one camera device to acquire plural frames of images with respect to a moving state of an object, a strobe flashlight device to generate a plurality of flashes at a uniformly fixed cycle per frame to acquire a multiple exposure image, and a sensing processing unit including a first processing means to extract coordinates of the object from images of the object moving at a velocity equal to or greater than a predetermined velocity based on the fixed flash cycle of the strobe flashlight device and a second processing means to extract coordinates of the object from images of the object moving at a velocity less than the predetermined velocity based on the fixed flash cycle of the strobe flashlight device.
  • sensing device for a moving object including a sensor unit to acquire a multiple exposure image of a moving object and a sensing processing unit including a plurality of processing means to extract coordinates of the object through image processing based on different processes according to patterns of object images in the acquired multiple exposure image.
  • sensing device for a moving object including a sensor unit to acquire a multiple exposure image of a first moving object and a second object moved by the first object, a first sensing processing means to process an image of the first object in the acquired multiple exposure image to extract coordinates of the first object, and a second sensing processing means to process an image of the second object in the acquired multiple exposure image to extract coordinates of the second object.
  • sensing method for a moving object including acquiring a multiple exposure image of a moving object, extracting images estimated as the object from the acquired multiple exposure image, and processing the extracted object images through a first process if the extracted object images are separate object images and processing the extracted object images through a second process if the extracted object images are overlapping object images.
  • a sensing method for a moving object including acquiring a multiple exposure image of a first moving object and a second object moved by the first object, processing images of the first object in the acquired multiple exposure image to extract coordinates of the first object, and processing images of the second object in the acquired multiple exposure image through a first process if the images of the second object are separated from each other and processing images of the second object in the acquired multiple exposure image through a second process if the images of the second object overlap each other to extract coordinates of the second object.
  • a virtual golf simulation device including a sensor unit to acquire a multiple exposure image of a golf ball hit by a golfer, a sensing processing unit including a plurality of processing means to extract coordinates of the golf ball through image processing based on different processes according to image patterns of the golf ball in the multiple exposure image acquired by the sensor unit, and a simulator to convert the coordinates extracted by the sensing processing unit into three-dimensional coordinates and to calculate physical property information on the moving golf ball based on the three-dimensional coordinates, thereby simulating an orbit of the golf ball.
  • virtual golf simulation device including a sensor unit to acquire a multiple exposure image of a golf club swung by a golfer and a golf ball hit by the golf club, a sensing processing unit including a first sensing processing means to extract coordinates of the golf club by processing an image of the golf club in the multiple exposure image acquired by the sensor unit and a second sensing processing means including a plurality of processing means to extract coordinates of the golf ball through image processing based on different processes according to image patterns of the golf ball in the acquired multiple exposure image, and a simulator to convert the coordinates extracted by the sensing processing unit into three-dimensional coordinates and to calculate physical information on the golf club and the golf ball based on the three- dimensional coordinates, thereby simulating an orbit of the golf ball.
  • FIG. 1 is a block diagram showing a sensing device or a virtual golf simulation
  • FIG. 2 is a view showing an example of a screen golf system to which the sensing device or the virtual golf simulation device shown in FIG. 1 is applied;
  • FIG. 3 is a view showing an operational signal scheme of a camera device and a strobe flashlight device of the sensing device according to the embodiment of the present invention
  • FIGS. 4(a) and 4(b) are views showing image patterns of an object acquired by the screen golf system shown in FIG. 2;
  • FIG. 5(a) is a view showing an example of an actually acquired image having the pattern shown in FIG. 4(a) and FIG. 5(b) is a view showing a preliminarily processed image of the image shown in FIG. 5(a);
  • FIG. 6 is a view of the image having the pattern shown in FIG. 4(a) in a state in
  • FIG. 7(a) is a view showing a process of fitting an object in the image having the pattern shown in FIG. 4(a) and FIG. 7(b) is a view showing a state in which the object is fitted using the object fitting process shown in FIG. 7(a) and a center point of the object is extracted;
  • FIG. 8 is a view showing matching of the object, the center point of which has been extracted as shown in FIGS. 7(a) and 7(b), with a strobe flashlight interval;
  • FIG. 9(a) is a view showing an example of an actually acquired image having the pattern shown in FIG. 4(b) and FIG. 9(b) is a view showing a preliminarily processed image of the image shown in FIG. 9(a);
  • FIG. 10(a) is a view showing an example of an image having the pattern shown in
  • FIG. 4(b) with noise and FIG. 10(b) is a graph showing pixel values on an axis of the image shown in FIG. 10(a);
  • FIG. 11 is a view showing a process of fitting an object image region, in which
  • FIGS. 12(a) and 12(b) are views showing a process of extracting center points of the fitted object image region shown in FIG. 11;
  • FIG. 13 is a flow chart showing a sensing method according to an embodiment of the present invention.
  • FIG. 14 and 15 are respectively a part and the other part of a flow chart showing a sensing method in a virtual golf simulation device according to an embodiment of the present invention.
  • a sensing device senses motion of a moving
  • the moving object may include a ball hit by a golfer in sports, such as golf, using a ball.
  • the sensing device according to the present invention is applicable to all systems that acquire and analyze an image of the movement of the object to sense the object.
  • the sensing device may be applied to a so-called screen golf system to which a virtual golf simulation device is applied.
  • FIGS. 1 and 2 schematically show the construction of a sensing device according to an embodiment of the present invention and a virtual golf simulation device using the same.
  • the sensing device includes a sensor unit including camera devices 310 and 320, a strobe flashlight device 330 and a signal generating unit 210 and a sensing processing unit 220 to process an image acquired by the sensor unit and to process an image of a moving object, thereby extracting coordinates of a center point thereof.
  • the camera devices 310 and 320 are provided to respectively capture a plurality of frames with respect to the movement of a moving object.
  • the strobe flashlight device 330 generates a plurality of flashes per frame to acquire a multiple exposure image having the same number of object images as the number of flashes per frame.
  • the sensor unit acquires an image of a moving state of the object.
  • the sensor unit acquires an image of a moving state of each of the objects.
  • the sensor unit acquires a multiple exposure image of the moved first object and the second object moved by the first object.
  • the sensor unit acquires a multiple exposure image of the moved golf club and the golf ball moved by the golf club.
  • the sensing processing unit 220 may include a first sensing processing means 230 and a second sensing processing means 240.
  • the first sensing processing means 230 processes an image of the first object in the acquired multiple exposure image to extract the coordinates of the first object.
  • the second sensing processing means 240 processes an image of the second object in the acquired multiple exposure image to extract the coordinates of the second object.
  • the first sensing processing means 230 calculates the change of the extracted coordinates of the first object to extract physical property information, such as velocity, a direction angle and moving orbit, of the first object.
  • the second sensing processing means 240 calculates the change of the extracted coordinates of the second object to extract physical property information, such as velocity, a direction angle and moving orbit, of the second object.
  • the first object may be a golf club and the second object may be a golf ball.
  • the sensor unit acquires a multiple exposure image of the golf club and the golf ball.
  • the first sensing processing means 230 of the sensing processing unit 220 processes an image of the golf club in the acquired multiple exposure image to extract the coordinates of the golf club and the second sensing processing means 240 of the sensing processing unit 220 processes an image of the golf ball in the acquired multiple exposure image to extract the coordinates of the golf ball.
  • Either the first sensing processing means 230 or the second sensing processing means 240 or both the first sensing processing means 230 and the second sensing processing means 240 may be provided based on objects to be sensed and sensing accuracy.
  • images of the object may be separated from each other or overlap each other depending upon whether the object moved at a velocity greater than or less than a predetermined level.
  • the second sensing processing means 240 includes a first processing
  • a second processing means 260 to extract the coordinates of the overlapping object images in the acquired multiple exposure image.
  • the first processing means 250 preferably includes a first image extracting means
  • the second processing means 260 preferably includes a second image extracting means 251 to extract overlapping object images estimated as the object in the acquired multiple exposure image and a first coordinate extracting means 252 to obtain the coordinates of center points of the overlapping object images using the extracted images.
  • the camera devices 310 and 320 are provided to capture a plurality of frames on the movement of an object moving from an initial position in a moving direction of the object.
  • two camera devices 310 and 320 are provided, to which, however, the present invention is not limited.
  • a camera device or more than two camera devices may be provided.
  • the strobe flashlight device 330 is a lighting device using light emitting diodes
  • the strobe flashlight device 330 is used as a light source for the camera devices to capture images.
  • the strobe flashlight device 330 generates strobe flashlight at a predetermined time interval (that is, a plurality of flashes is generated at a predetermined time interval) so that the camera devices 310 and 320 can capture a multiple exposure image.
  • an object is present on a frame captured by the camera devices in correspondence to the number of flashes generated by the strobe flashlight device 330 to provide a multiple exposure image thereof.
  • a trigger signal to operate the camera devices 310 and 320 and the strobe flashlight device 330 is generated by the signal generating unit 210.
  • the multiple exposure image captured by the camera devices 310 and 320 and the strobe flashlight device 330 is processed by the sensing processing unit 220 and is transmitted to a simulator 100.
  • the simulator 100 preferably includes a controller M, a database 110, an image
  • processing unit 120 and an image output unit 130 are processing units 120 and an image output unit 130.
  • the controller M receives coordinate information of an object which has been image processed and acquired by the sensing processing unit 220, converts the coordinate in- formation into three-dimensional coordinates, obtains predetermined physical information for moving orbit simulation of the object and transmits the obtained physical information to the image processing unit 120.
  • Predetermined data for moving orbit simulation of the object may be extracted from the database 110, and the image processing of the moving orbit simulation of the object by the image processing unit 120 may be performed by extracting image data stored in the database 110.
  • the first sensing processing means 230, the second sensing processing means 240, the first processing means 250 and the second processing means 260 may be realized as a single controller configured to perform the functions of the above means or separate controllers configured to respectively performed the functions of the above means.
  • the first sensing processing means 230, the second sensing processing means 240, the first processing means 250 and the second processing means 260 may be realized as a single program configured to perform the functions of the above means or separate programs configured to respectively performed the functions of the above means.
  • FIG. 2 An example of a screen golf system, to which the sensing device or the virtual golf simulation device with the above-stated construction is applied, is shown in FIG. 2.
  • a hitting mat 30 having a golf tee, on which a golf ball is placed, and a ground (preferably including at least one selected from among a fairway mat, a rough mat and a bunker mat) and a swing plate 20, on which a golfer hits a golf ball, are provided at one side of a golf booth B.
  • a screen 40 on which a virtual golf simulation image realized by the image output unit 130 is displayed, is provided at the front of the golf booth B.
  • the camera devices 310 and 320 and the strobe flashlight device 330 are provided at the ceiling of the golf booth B.
  • the camera devices 310 and 320 are provided at the ceiling and wall of the golf booth B, respectively, to which, however, the present invention is not limited.
  • the camera devices 310 and 320 can be installed at any position of the golf booth B so long as the camera devices 310 and 320 do not disturb the swing of the golfer and are prevented from colliding with a golf ball hit by the golfer while the camera devices 310 and 320 can effectively capture an image of a moving state of the golf ball 10.
  • the strobe flashlight device 330 is installed at the ceiling of the golf booth B to provide strobe flashlight substantially perpendicular to a hitting point, to which, however, the present invention is not limited.
  • the strobe flashlight device 330 can be installed at any position of the golf booth B so long as the strobe flashlight device 330 can effectively provide strobe flashlight.
  • the camera devices 310 and 320 capturing predetermined regions in which the hitting is performed capture a plurality of frames, respectively.
  • the strobe flashlight device 330 generates several flashes per frame to acquire a multiple exposure image of the moving golf ball.
  • FIG. 3 is a view showing a trigger signal generation scheme of a camera device and a strobe flashlight device by the signal generating unit 210 (see FIG. 1).
  • trigger signals of the camera device have a time interval of tc.
  • trigger signals are generated at a time interval of tc for each frame.
  • exposure time for each frame is te (at this time, preferably tc > te).
  • data on an image acquired for a time of tc - te is preferably transmitted to the sensing processing unit 220 (see FIG. 1).
  • the camera device is triggered at an interval between the trigger signal generating times for each frame, and data on the acquired multiple exposure image is transmitted to the sensing processing unit.
  • strobe flashlight trigger signals are generated three times at a time interval of tsl .
  • strobe flashlight is generated three times at the same time interval of tsl.
  • the camera device and the strobe flashlight device are preferably synchronized for the signal generating unit 210 (see FIG. 1) to generate a first trigger signal.
  • a time interval of ts2 is preferably provided between the trigger signal of the last one of the three strobe flashlights and a first strobe flashlight of the next frame.
  • the time interval of tsl and the time interval of ts2 may be set to be equal to each other. As shown in FIG. 3, however, the time interval of tsl and the time interval of ts2 are preferably set to be different from each other. The time interval of ts2 may be set to be longer than the time interval of tsl .
  • FIGS. 4(a) and 4(b) are views showing two patterns of the multiple exposure image acquired by the camera device and the strobe flashlight device based on the signal generation scheme shown in FIG. 3, respectively.
  • FIGS. 4(a) and 4(b) a moving state of a golf ball hit by a golfer using a golf club is shown using two image patterns acquired based on the trigger signal scheme shown in FIG. 3 (II and 12).
  • An image II shown in FIG. 4(a) is acquired through multiple exposure of golf club images CI, C2 and C3 and golf ball images 11a, l ib and 1 lc.
  • the object images 11a, 1 lb and 1 lc are spaced apart from each other by a predetermined interval.
  • An image 12 of FIG. 4(b) shows a case in which golf ball images overlap to provide an image region 12 of a predetermined size.
  • the sensing processing unit of the sensing device for the moving object according to the present invention is characterized in that, in a case in which the object is moved at a speed higher than a predetermined speed based on the fixed flash cycle of the strobe flashlight device with the result that the object images are spaced apart from each other in the multiple exposure image as described above, the coordinates of the center points of the object images are extracted by the first processing means 250 (see FIG. 1) and, in a case in which the object is moved at a speed lower than the predetermined speed based on the fixed flash cycle of the strobe flashlight device with the result that the object images overlap each other in the multiple exposure image as described above, the coordinates of the center points of the object image region are extracted by the second processing means 260 (see FIG. 1).
  • inventions may include the first sensing processing means 230 (see FIG. 1) to analyze an image of the swung golf club in the multiple exposure image when the golf ball is moved by the golf club as shown in FIGS. 4(a) and 4(b) to extract the coordinates of the center point of the golf club and the second sensing processing means 240 (see FIG. 1) to analyze an image of the golf ball in the multiple exposure image when the golf ball is moved by the swung golf club as shown in FIGS. 4(a) and 4(b) to extract the coordinates of the center point of the golf ball, which will be hereinafter described in detail.
  • FIG. 5(a) is a view showing the original copy of a multiple exposure image including separate object images.
  • the multiple exposure image is acquired using a camera device having a low resolution of 320 x 240 and operated at a low speed of 75 fps and a strobe flashlight device operating at 330 Hz.
  • FIG. 5(b) An image acquired through removal of a stationary image, i.e. a background image, from the original image shown in FIG. 5(a) through subtraction and a predetermined preliminary processing operation, such as Gaussian blur, is shown in FIG. 5(b).
  • a predetermined preliminary processing operation such as Gaussian blur
  • 5(b) contains noise in addition to the images of the object.
  • an image estimated as the object is extracted as shown in FIG. 6.
  • the image shown in FIG. 6 is obtained by overlapping a plurality of frames captured by the camera device into a single image.
  • an image set SO indicates an image of a zeroth frame
  • an image set S 1 indicates an image of a first frame
  • an image set S2 indicates an image of a second frame.
  • an image estimated by the object image may be extracted through a contour check or a check window.
  • the processing may be carried out so that an image estimated as the object is kept while images estimated not to be the object are discarded.
  • the object has a predetermined size (a predetermined diameter for a golf ball). Therefore, the contours of the respective images of the multiple exposure image are checked in consideration of the size of the object, and the checked contours may be excluded if they are too large or too small to be the object.
  • check windows W corresponding to the size of the respective images 11 in the multiple exposure image are formed so as to include the respective images 11 , and an aspect ratio of the respective check windows W is checked.
  • the aspect ratio of the golf ball must be almost equal. If the aspect ratio of the check windows W corresponding to the respective images 1 1 in the multiple exposure image is too large with the result that each of the check windows W has a large width or is too small with the result that each of the check windows W has a small width, the respective images 11 in the multiple exposure image are considered not to be the object and thus may be discarded.
  • the images 11 extracted as the object images as described above are fitted into the shape of an object as shown in FIGS. 7(a) and 7(b) by the first coordinate extracting means 252 (see FIG. 1) of the first processing means to extract the coordinates of a center point of each of the images.
  • FIG. 7(a) is an enlarged view of an image set extracted from the multiple exposure image shown in FIG. 6 and FIG. 7(b) is a view showing a fitted result of the object.
  • the object image is partially damaged as the result of the preliminary processing.
  • an image perfectly appears at one side at which the strobe flashlight is irradiated to the object, whereas a shaded region is formed at the other side at which the strobe flashlight is not irradiated to the object with the result that the shape of the object is distorted depending upon the size of the shaded region.
  • the object image 11 has an irradiated region Rl and a shaded region R2.
  • the fitted curve may be formed based on information on the size of the object which is previously stored or based on the measured size of the object image 11.
  • a fitted object image 11 as shown in FIG. 7(b) may be obtained through the fitting as described above, and the coordinates of the center point CP of the object may be accurately extracted based on the shape of the fitted object image.
  • the first coordinate extracting means 252 (see FIG. 1) to accurately extract only object images based on specific conditions as shown in FIG. 8.
  • the object must have three images per frame and the interval between the object images must be kept at the same ratio as the strobe flashlight cycle, i.e. the trigger time intervals tsl and ts2 (see FIG. 3).
  • a ratio of c to a or c to b must be substantially equal to a ratio of a strobe flashlight trigger time interval ts2 to strobe flashlight trigger time interval tsl (see FIG. 3).
  • the coordinates of the center points of the finally confirmed objects are converted into three-dimensional coordinates by the converting means, and physical information of the moving orbit of the object to be simulated is obtained based on the three- dimensional coordinates.
  • FIG. 9(a) is a view showing the original copy of a multiple exposure image including overlapping object images.
  • the multiple exposure image is acquired using a camera device having a low resolution of 320 x 240 and operated at a low speed of 75 fps and a strobe flashlight device operating at 330 Hz.
  • the second image extracting means 261 (see FIG. 1) of the second processing means extracts an object image region constituted by the overlapping object image as shown in FIG. 9(a) (see FIGS. 9(a) and 10), and the second coordinate extracting means 262 (see FIG. 1) of the second processing means extracts the coordinates of at least two center points of the extracted object image region (see FIGS. 11 and 12).
  • FIG. 9(b) An image acquired through removal of a stationary image, i.e. a background image, from the original image shown in FIG. 9(a) through subtraction and a predetermined preliminary processing operation, such as Gaussian blur, is shown in FIG. 9(b).
  • a predetermined preliminary processing operation such as Gaussian blur
  • a moving direction of a moving object is estimated using a tool or program to estimate the moving direction of the moving object.
  • a predetermined image region arranged on an axis 13 of the estimated moving direction is extracted as an object image region 12 in which object images overlap each other.
  • the extracted object image region may contain noise Nl as shown in FIG. 10(a).
  • pixel intensity of the image on the axis 13 of the moving direction may be checked to determine whether the portion l in the image is an image of a real object or noise.
  • FIG. 10(b) is a graph showing the distribution of pixel intensity of the images Nl and 12 shown in FIG. 10(a) on the axis 13 of the estimated moving direction.
  • the object images overlap each other to constitute an image region of a predetermined size. Consequently, in a case in which a region having the distribution of the pixel intensity equal to or greater than a critical value corresponds to a predetermined range, the corresponding image may be extracted as an image region of the
  • this region does not correspond to the overlapping object images, and therefore, this region may be excluded.
  • the noise image Nl has a pixel intensity section of al to a2
  • the object image region 12 has a pixel intensity section of a3 to a4. Since the section of a3 to a4 is long sufficient to be estimated as the object image region, in which the object images overlap each other, the section of a3 to a4 may be extracted as the object image region. On the other hand, since the section of al to a2 is too short to be estimated as the object image region, in which the object images overlap each other, the section of al to a2 may be excluded.
  • the image region extracted as described above may be fitted to obtain the coordinates of center points of the image region.
  • fitted curves HC1 and HC2 having a shape of the object are formed at the contour parts of opposite ends of the image region 12 about the axis 13 of the moving direction.
  • estimated center points EP1 and EP2 estimated as the center points of the fitted objects located at opposite ends of the image region are set and the positions of the estimated center pints EP1 and EP2 are corrected to obtain correct center points CP1 and CP2 as shown in FIG. 12(b).
  • the respective center points EP1 and EP2 estimated as shown in FIG. 12(a) may be corrected to be correct center points by checking three directional lines rl, r2 and r3 of the fitted curve HC1 and three directional lines r4, r5 and r6 of the fitted curve HC2.
  • the estimated center points EP1 and EP2 are corrected so that the three directional lines shown in FIG. 12(a), i.e. the three directional lines rl, r2 and r3 of the fitted curve HC1 and three directional lines r4, r5 and r6 of the fitted curve HC2, must substantially have the same length.
  • a trigger signal is applied by the signal generating unit so that the camera device and the strobe flashlight device continuously capture an image of a predetermined region in which an object is placed at a predetermined interval
  • the camera device senses an initial position of the object while continuously
  • the initial position of the object is sensed because the initial position of the object is specified in a multiple exposure image acquired by the movement of the object and a moving direction of the object is estimated based on the specified initial position of the object. Preferably, therefore, coordinate values of the initial position of the object is confirmed at the step of sensing the initial position of the object.
  • triggering of the object is sensed (S30). For example, the time when the object starts to move is sensed as if impact time of a golf ball at which the golf ball is hit by a golf club is sensed.
  • the camera device acquires and stores a multiple exposure image having a plurality of frames under strobe flashlight before and after the triggering time (S40).
  • a second object for example, a golf ball
  • a first moving object for example, a golf club
  • a first object process to extract the coordinates of a center point of the first object from the acquired image through image processing of the first object
  • a second object process to extract the coordinates of a center point of the second object from the acquired image through image processing of the second object is carried out.
  • the first object process and the second object process may be simultaneously carried out. Alternatively, either the first object process or the second object process may be carried out.
  • the first process (SI 10) and the second process (SI 20) may be simultaneously or selectively carried out.
  • an overlapping object image, if any, in the multiple exposure image during execution of the first process is processed to be excluded.
  • a separate object image, if any, in the multiple exposure image during execution of the second process is processed to be excluded.
  • both the first process and the second process are carried out. In this case, however, any separate object image is excluded during execution of the second process, and therefore, the coordinates of the center points are acquired based on the execution result of the first process.
  • both the first process and the second process are carried out. In this case, however, any overlapping object image is excluded during execution of the first process, and therefore, the coordinates of the center points are acquired based on the execution result of the second process.
  • Step S40 acquired at Step S40 is recognized (S200), and a first object process is carried out by the first sensing processing means (S210).
  • both an image of the golf club and an image of the golf ball are included in the multiple exposure image.
  • the image processing of the hit golf ball is carried out at Steps S100 to S140, and the image processing of the swung golf club is carried out at Steps S200 to S220.
  • FIG. 14 and 15 show an example of a sensing method of a sensing
  • FIG. 14 and 15 are divided a flow chart of a sensing method into two parts. "A” described in FIG. 14 is connected to “A” described in FIG. 15, and “B” described in FIG. 14 is connected to “B” described in FIG. 15.
  • a simulator determines whether a golfer will perform a tee shot (S2), an iron shot (S3), an approach shot (S4) or a putting (S5).
  • the sensing method shown in FIG. 13 is preferably used.
  • the sensing method shown in FIG. 13 excluding the image processing method based on the first process is preferably used.
  • the first process (Si l l) and the second process (S 121 ) are simultaneously carried out.
  • the coordinates of the center points of the separate golf ball images in the multiple exposure image are extracted by the first process (S131) and the coordinates of the center points of the overlapping golf ball images in the multiple exposure image are extracted by the second process (S141).
  • the golf ball is moved at high speed since the golf ball is strongly hit by a number one wood driver at the tee shot.
  • golf ball images are separated from each other so as to correspond to a flash interval of the strobe flashlight device.
  • the first process and the second process are carried out. Since the second process is programmed to process the overlapping golf ball images, however, the execution of the second process is stopped with the respect to the separate golf ball images, and the separate golf ball images is processed by the first process to extract the coordinates of the center points of the separate golf ball images.
  • the golfer may strongly or weakly hit the golf ball in consideration of the flight distance of the golf ball.
  • a multiple exposure image of the golf ball hit by the golfer is acquired and the first process and the second process are simultaneously carried out, it is possible to acquire the coordinates of the center point of the golf ball by one of the first and second processes regardless of whether the golf ball images are separated from each other or overlap each other.
  • a trigger signal of the camera device and the strobe flashlight device is
  • FIG. 14 shows the process of extracting only the image of the golf ball using the camera device and the strobe flashlight device to extract the center point of the golf ball.
  • the image of the golf club is also included in the multiple exposure image acquired by the camera device and the strobe flashlight device. Consequently, it is possible to extract the coordinates of the center point of the golf club through image processing of the golf club at Step S200 to S220 of FIG. 13 and to use the extracted coordinate information as a basis of calculating the physical information of the golf ball.
  • the present invention can be widely used in industries related to a sensing device and method for a moving object and a virtual golf simulation device using the same.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physical Education & Sports Medicine (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Biophysics (AREA)
  • Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Educational Administration (AREA)
  • Electromagnetism (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Educational Technology (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Image Analysis (AREA)
  • Studio Devices (AREA)
  • Image Processing (AREA)

Abstract

Disclosed herein are a sensing device and method for a moving object that is capable of acquiring a multiple exposure image of a moving object using a low resolution and low speed camera device and strobe flashlight device and accurately extracting an image of the object from the acquired image to obtain coordinates of a center point of the object and, in particular, that is capable of processing a moving object through separate processes based on the velocity of the object or based on patterns of object images in a multiple exposure image to expand a sensing processing ability and range and achieve accurate sensing although a low resolution and low speed camera device is used and a virtual golf simulation device using the same.

Description

Description
Title of Invention: SENSING DEVICE AND METHOD FOR MOVING OBJECT AND VIRTUAL GOLF SIMULATION DEVICE USING THE SAME
Technical Field
[1] The present invention relates to a sensing device and method and a virtual golf
simulation device using the same, and, more particularly, to a sensing device and method for a moving object that acquires and analyzes an image of a moving object, such as a golf ball, to obtain physical information thereof and a virtual golf simulation device using the same.
Background Art
[2] In recent years, various devices have been developed which allow users to enjoy popular sports games, such as baseball, soccer, basketball and golf, in rooms or in specific places through simulation in the form of interactive sports games.
[3] In order to simulate sports using balls, such as baseballs, soccer balls, basketballs and golf balls in such interactive sports games, much research has been conducted into various sensing systems to accurately sense physical information on a moving object, i.e. movement of a ball.
[4] For example, various sensing systems, such as a sensing system using an infrared sensor, a sensing system using a laser sensor, a sensing system using an acoustic sensor and a sensing system using a camera sensor, have come onto the market.
Disclosure of Invention
Technical Problem
[5] It is an object of the present invention to provide a sensing device and method for a moving object that is capable of acquiring a multiple exposure image of a moving object using a low resolution and low speed camera device and strobe flashlight device and accurately extracting an image of the object from the acquired image to obtain coordinates of a center point of the object and, in particular, that is capable of processing a moving object through separate processes based on the velocity of the object or based on patterns of object images in a multiple exposure image to expand sensing a processing ability and range and achieve accurate sensing although a low resolution and low speed camera device is used and a virtual golf simulation device using the same.
Solution to Problem
[6] In accordance with one aspect of the present invention, the above and other objects can be accomplished by the provision of a sensing device for a moving object including at least one camera device to acquire plural frames of images with respect to a moving state of an object, a strobe flashlight device to generate a plurality of flashes per frame to acquire a multiple exposure image, and a sensing processing unit including a first processing means to extract coordinates of the object in separate object images of the acquired multiple exposure image and a second processing means to extract coordinates of the object in overlapping object images of the acquired multiple exposure image.
[7] In accordance with another aspect of the present invention, there is provided a
sensing device for a moving object including at least one camera device to acquire plural frames of images with respect to a moving state of an object, a strobe flashlight device to generate a plurality of flashes at a uniformly fixed cycle per frame to acquire a multiple exposure image, and a sensing processing unit including a first processing means to extract coordinates of the object from images of the object moving at a velocity equal to or greater than a predetermined velocity based on the fixed flash cycle of the strobe flashlight device and a second processing means to extract coordinates of the object from images of the object moving at a velocity less than the predetermined velocity based on the fixed flash cycle of the strobe flashlight device.
[8] In accordance with another aspect of the present invention, there is provided a
sensing device for a moving object including a sensor unit to acquire a multiple exposure image of a moving object and a sensing processing unit including a plurality of processing means to extract coordinates of the object through image processing based on different processes according to patterns of object images in the acquired multiple exposure image.
[9] In accordance with another aspect of the present invention, there is provided a
sensing device for a moving object including a sensor unit to acquire a multiple exposure image of a first moving object and a second object moved by the first object, a first sensing processing means to process an image of the first object in the acquired multiple exposure image to extract coordinates of the first object, and a second sensing processing means to process an image of the second object in the acquired multiple exposure image to extract coordinates of the second object.
[10] In accordance with another aspect of the present invention, there is provided a
sensing method for a moving object including acquiring a multiple exposure image of a moving object, extracting images estimated as the object from the acquired multiple exposure image, and processing the extracted object images through a first process if the extracted object images are separate object images and processing the extracted object images through a second process if the extracted object images are overlapping object images. [11] In accordance with another aspect of the present invention, there is provided a sensing method for a moving object including acquiring a multiple exposure image of a first moving object and a second object moved by the first object, processing images of the first object in the acquired multiple exposure image to extract coordinates of the first object, and processing images of the second object in the acquired multiple exposure image through a first process if the images of the second object are separated from each other and processing images of the second object in the acquired multiple exposure image through a second process if the images of the second object overlap each other to extract coordinates of the second object.
[12] In accordance with another aspect of the present invention, there is provided a virtual golf simulation device including a sensor unit to acquire a multiple exposure image of a golf ball hit by a golfer, a sensing processing unit including a plurality of processing means to extract coordinates of the golf ball through image processing based on different processes according to image patterns of the golf ball in the multiple exposure image acquired by the sensor unit, and a simulator to convert the coordinates extracted by the sensing processing unit into three-dimensional coordinates and to calculate physical property information on the moving golf ball based on the three-dimensional coordinates, thereby simulating an orbit of the golf ball.
[13] In accordance with a further aspect of the present invention, there is provided a
virtual golf simulation device including a sensor unit to acquire a multiple exposure image of a golf club swung by a golfer and a golf ball hit by the golf club, a sensing processing unit including a first sensing processing means to extract coordinates of the golf club by processing an image of the golf club in the multiple exposure image acquired by the sensor unit and a second sensing processing means including a plurality of processing means to extract coordinates of the golf ball through image processing based on different processes according to image patterns of the golf ball in the acquired multiple exposure image, and a simulator to convert the coordinates extracted by the sensing processing unit into three-dimensional coordinates and to calculate physical information on the golf club and the golf ball based on the three- dimensional coordinates, thereby simulating an orbit of the golf ball.
Advantageous Effects of Invention
[14] In a sensing device and method for a moving object according to the present
invention as described above and a virtual golf simulation device using the same, it is possible to acquire a multiple exposure image of a moving object using a low resolution and low speed camera device and strobe flashlight device and accurately extract an image of the object from the acquired image to obtain coordinates of a center point of the object and, in particular, to process a moving object through separate processes based on the velocity of the object or based on patterns of object images in a multiple exposure image to expand a sensing processing ability and range and achieve accurate sensing although a low resolution and low speed camera device is used.
Brief Description of Drawings
[15] The above and other objects, features and other advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
[16] FIG. 1 is a block diagram showing a sensing device or a virtual golf simulation
device according to an embodiment of the present invention;
[17] FIG. 2 is a view showing an example of a screen golf system to which the sensing device or the virtual golf simulation device shown in FIG. 1 is applied;
[18] FIG. 3 is a view showing an operational signal scheme of a camera device and a strobe flashlight device of the sensing device according to the embodiment of the present invention;
[19] FIGS. 4(a) and 4(b) are views showing image patterns of an object acquired by the screen golf system shown in FIG. 2;
[20] FIG. 5(a) is a view showing an example of an actually acquired image having the pattern shown in FIG. 4(a) and FIG. 5(b) is a view showing a preliminarily processed image of the image shown in FIG. 5(a);
[21] FIG. 6 is a view of the image having the pattern shown in FIG. 4(a) in a state in
which a plurality of frames overlaps, showing an object image extracting process;
[22] FIG. 7(a) is a view showing a process of fitting an object in the image having the pattern shown in FIG. 4(a) and FIG. 7(b) is a view showing a state in which the object is fitted using the object fitting process shown in FIG. 7(a) and a center point of the object is extracted;
[23] FIG. 8 is a view showing matching of the object, the center point of which has been extracted as shown in FIGS. 7(a) and 7(b), with a strobe flashlight interval;
[24] FIG. 9(a) is a view showing an example of an actually acquired image having the pattern shown in FIG. 4(b) and FIG. 9(b) is a view showing a preliminarily processed image of the image shown in FIG. 9(a);
[25] FIG. 10(a) is a view showing an example of an image having the pattern shown in
FIG. 4(b) with noise and FIG. 10(b) is a graph showing pixel values on an axis of the image shown in FIG. 10(a);
[26] FIG. 11 is a view showing a process of fitting an object image region, in which
object images overlap each other, in the image having the pattern shown in FIG. 4(b);
[27] FIGS. 12(a) and 12(b) are views showing a process of extracting center points of the fitted object image region shown in FIG. 11;
[28] FIG. 13 is a flow chart showing a sensing method according to an embodiment of the present invention; and
[29] FIG. 14 and 15 are respectively a part and the other part of a flow chart showing a sensing method in a virtual golf simulation device according to an embodiment of the present invention.
Best Mode for Carrying out the Invention
[30] Now, exemplary embodiments of a sensing device and method for a moving object according to the present invention and a virtual golf simulation device using the same will be described in detail with reference to the accompanying drawings.
[31] A sensing device according to the present invention senses motion of a moving
object to obtain physical information thereof. Here, the moving object may include a ball hit by a golfer in sports, such as golf, using a ball. The sensing device according to the present invention is applicable to all systems that acquire and analyze an image of the movement of the object to sense the object. In an example, the sensing device may be applied to a so-called screen golf system to which a virtual golf simulation device is applied.
[32] FIGS. 1 and 2 schematically show the construction of a sensing device according to an embodiment of the present invention and a virtual golf simulation device using the same.
[33] First, a sensing device for a moving object according to an embodiment of the
present invention and a virtual golf simulation device using the same will be described with reference to FIGS. 1 and 2.
[34] As shown in FIG. 1, the sensing device according to the embodiment of the present invention includes a sensor unit including camera devices 310 and 320, a strobe flashlight device 330 and a signal generating unit 210 and a sensing processing unit 220 to process an image acquired by the sensor unit and to process an image of a moving object, thereby extracting coordinates of a center point thereof.
[35] The camera devices 310 and 320 are provided to respectively capture a plurality of frames with respect to the movement of a moving object. At this time, the strobe flashlight device 330 generates a plurality of flashes per frame to acquire a multiple exposure image having the same number of object images as the number of flashes per frame.
[36] During movement of an object, the sensor unit acquires an image of a moving state of the object. During movement of two different objects, the sensor unit acquires an image of a moving state of each of the objects.
[37] That is, when a first object is moved and a second object is moved by the first object, the sensor unit acquires a multiple exposure image of the moved first object and the second object moved by the first object.
[38] For example, when a golfer hits a golf ball using a golf club, the golf club is moved as the result of swing of the golfer and the golf ball is hit by the golf club with the result that the golf ball is moved. At this time, the sensor unit acquires a multiple exposure image of the moved golf club and the golf ball moved by the golf club.
[39] Meanwhile, the sensing processing unit 220 may include a first sensing processing means 230 and a second sensing processing means 240. The first sensing processing means 230 processes an image of the first object in the acquired multiple exposure image to extract the coordinates of the first object. The second sensing processing means 240 processes an image of the second object in the acquired multiple exposure image to extract the coordinates of the second object.
[40] The first sensing processing means 230 calculates the change of the extracted coordinates of the first object to extract physical property information, such as velocity, a direction angle and moving orbit, of the first object. The second sensing processing means 240 calculates the change of the extracted coordinates of the second object to extract physical property information, such as velocity, a direction angle and moving orbit, of the second object.
[41] For example, the first object may be a golf club and the second object may be a golf ball. When a user swings to hit the gall ball using the golf club, the golf ball is moved. At this time, the sensor unit acquires a multiple exposure image of the golf club and the golf ball. The first sensing processing means 230 of the sensing processing unit 220 processes an image of the golf club in the acquired multiple exposure image to extract the coordinates of the golf club and the second sensing processing means 240 of the sensing processing unit 220 processes an image of the golf ball in the acquired multiple exposure image to extract the coordinates of the golf ball.
[42] Either the first sensing processing means 230 or the second sensing processing means 240 or both the first sensing processing means 230 and the second sensing processing means 240 may be provided based on objects to be sensed and sensing accuracy.
[43] Meanwhile, in a multiple exposure image of a moving object, such as golf ball hit and moved by a golf club, images of the object may be separated from each other or overlap each other depending upon whether the object moved at a velocity greater than or less than a predetermined level.
[44] Preferably, the second sensing processing means 240 includes a first processing
means 250 to extract the coordinates of the separate object images in the acquired multiple exposure image and a second processing means 260 to extract the coordinates of the overlapping object images in the acquired multiple exposure image.
[45] That is, image processing is carried out using different processes depending upon whether the object images are separated from each other or overlap each other in the acquired multiple exposure image.
[46] The first processing means 250 preferably includes a first image extracting means
251 to extract object images separated at a predetermined interval estimated as the object in the acquired multiple exposure image and a first coordinate extracting means
252 to obtain the coordinates of center points of the respective object images using the extracted images. The second processing means 260 preferably includes a second image extracting means 251 to extract overlapping object images estimated as the object in the acquired multiple exposure image and a first coordinate extracting means 252 to obtain the coordinates of center points of the overlapping object images using the extracted images.
[47] The above components will be described below in more detail.
[48] Meanwhile, the camera devices 310 and 320 are provided to capture a plurality of frames on the movement of an object moving from an initial position in a moving direction of the object.
[49] In FIG. 1, two camera devices 310 and 320 are provided, to which, however, the present invention is not limited. For example, a camera device or more than two camera devices may be provided.
[50] The strobe flashlight device 330 is a lighting device using light emitting diodes
(LED). The strobe flashlight device 330 is used as a light source for the camera devices to capture images. The strobe flashlight device 330 generates strobe flashlight at a predetermined time interval (that is, a plurality of flashes is generated at a predetermined time interval) so that the camera devices 310 and 320 can capture a multiple exposure image.
[51] That is, an object is present on a frame captured by the camera devices in correspondence to the number of flashes generated by the strobe flashlight device 330 to provide a multiple exposure image thereof.
[52] The operation of the camera devices 310 and 320 and the strobe flashlight device 330 will be described below in detail.
[53] Meanwhile, a trigger signal to operate the camera devices 310 and 320 and the strobe flashlight device 330 is generated by the signal generating unit 210. The multiple exposure image captured by the camera devices 310 and 320 and the strobe flashlight device 330 is processed by the sensing processing unit 220 and is transmitted to a simulator 100.
[54] The simulator 100 preferably includes a controller M, a database 110, an image
processing unit 120 and an image output unit 130.
[55] The controller M receives coordinate information of an object which has been image processed and acquired by the sensing processing unit 220, converts the coordinate in- formation into three-dimensional coordinates, obtains predetermined physical information for moving orbit simulation of the object and transmits the obtained physical information to the image processing unit 120.
[56] Predetermined data for moving orbit simulation of the object may be extracted from the database 110, and the image processing of the moving orbit simulation of the object by the image processing unit 120 may be performed by extracting image data stored in the database 110.
[57] Meanwhile, a converting means to convert coordinate information of the object
transmitted from the sensing processing unit 220 into three-dimensional coordinates may be provided separately from the controller M.
[58] In a hardware aspect, the first sensing processing means 230, the second sensing processing means 240, the first processing means 250 and the second processing means 260 may be realized as a single controller configured to perform the functions of the above means or separate controllers configured to respectively performed the functions of the above means. In a software aspect, the first sensing processing means 230, the second sensing processing means 240, the first processing means 250 and the second processing means 260 may be realized as a single program configured to perform the functions of the above means or separate programs configured to respectively performed the functions of the above means.
[59] An example of a screen golf system, to which the sensing device or the virtual golf simulation device with the above-stated construction is applied, is shown in FIG. 2.
[60] As shown in FIG. 2, a hitting mat 30 having a golf tee, on which a golf ball is placed, and a ground (preferably including at least one selected from among a fairway mat, a rough mat and a bunker mat) and a swing plate 20, on which a golfer hits a golf ball, are provided at one side of a golf booth B.
[61] A screen 40, on which a virtual golf simulation image realized by the image output unit 130 is displayed, is provided at the front of the golf booth B. The camera devices 310 and 320 and the strobe flashlight device 330 are provided at the ceiling of the golf booth B.
[62] In FIG. 2, the camera devices 310 and 320 are provided at the ceiling and wall of the golf booth B, respectively, to which, however, the present invention is not limited. The camera devices 310 and 320 can be installed at any position of the golf booth B so long as the camera devices 310 and 320 do not disturb the swing of the golfer and are prevented from colliding with a golf ball hit by the golfer while the camera devices 310 and 320 can effectively capture an image of a moving state of the golf ball 10.
[63] Also, in FIG. 2, the strobe flashlight device 330 is installed at the ceiling of the golf booth B to provide strobe flashlight substantially perpendicular to a hitting point, to which, however, the present invention is not limited. The strobe flashlight device 330 can be installed at any position of the golf booth B so long as the strobe flashlight device 330 can effectively provide strobe flashlight.
[64] When the golfer on the swing plate 20 hits a golf ball 10 on the hitting mat 30 toward the screen 40 in the system with the above-stated construction, as shown in FIG. 2, the camera devices 310 and 320 capturing predetermined regions in which the hitting is performed capture a plurality of frames, respectively. At this time, the strobe flashlight device 330 generates several flashes per frame to acquire a multiple exposure image of the moving golf ball.
[65] FIG. 3 is a view showing a trigger signal generation scheme of a camera device and a strobe flashlight device by the signal generating unit 210 (see FIG. 1).
[66] As shown in FIG. 3, trigger signals of the camera device have a time interval of tc.
That is, trigger signals are generated at a time interval of tc for each frame. At this time, exposure time for each frame is te (at this time, preferably tc > te). Also, data on an image acquired for a time of tc - te is preferably transmitted to the sensing processing unit 220 (see FIG. 1).
[67] That is, as shown in FIG. 3, the camera device is triggered at an interval between the trigger signal generating times for each frame, and data on the acquired multiple exposure image is transmitted to the sensing processing unit.
[68] During the exposure time (time of te) of the camera device, strobe flashlight is
generated by the strobe flashlight device several times. In FIG. 3, strobe flashlight trigger signals are generated three times at a time interval of tsl .
[69] That is, during the exposure time (time of te) of the camera device, strobe flashlight is generated three times at the same time interval of tsl. At this time, the camera device and the strobe flashlight device are preferably synchronized for the signal generating unit 210 (see FIG. 1) to generate a first trigger signal.
[70] Also, as shown in FIG. 3, a time interval of ts2 is preferably provided between the trigger signal of the last one of the three strobe flashlights and a first strobe flashlight of the next frame. The time interval of tsl and the time interval of ts2 may be set to be equal to each other. As shown in FIG. 3, however, the time interval of tsl and the time interval of ts2 are preferably set to be different from each other. The time interval of ts2 may be set to be longer than the time interval of tsl .
[71] Since the trigger signal intervals of the camera device and the strobe flashlight device are uniform as described above, the interval between the object images on the multiple exposure image is uniform.
[72] When the multiple exposure image is processed by the sensing processing unit,
therefore, it is possible to effectively separate a correct object image from the object image present on the multiple exposure image and various noises based on the characteristics of the uniformly fixed interval of the trigger signal. [73] Also, since the interval of the strobe flashlight is uniformly fixed for each frame of the camera device, images having various patterns may be generated in the multiple exposure image according to the moving speed of the object.
[74] FIGS. 4(a) and 4(b) are views showing two patterns of the multiple exposure image acquired by the camera device and the strobe flashlight device based on the signal generation scheme shown in FIG. 3, respectively.
[75] That is, in FIGS. 4(a) and 4(b), a moving state of a golf ball hit by a golfer using a golf club is shown using two image patterns acquired based on the trigger signal scheme shown in FIG. 3 (II and 12).
[76] An image II shown in FIG. 4(a) is acquired through multiple exposure of golf club images CI, C2 and C3 and golf ball images 11a, l ib and 1 lc. In this case, the object images 11a, 1 lb and 1 lc are spaced apart from each other by a predetermined interval.
[77] An image 12 of FIG. 4(b) shows a case in which golf ball images overlap to provide an image region 12 of a predetermined size.
[78] That is, in the image shown in FIG. 4(a), the golf ball is moved at high speed, and therefore, the image is formed in a state in which the golf ball images are spaced apart from each other by the predetermined interval when strobe flashlight is triggered, and, in the image shown in FIG. 4(b), the golf ball is moved at low speed, and therefore, strobe flashlight is triggered before the ball moves far away with the result that the golf ball images overlap each other.
[79] The sensing processing unit of the sensing device for the moving object according to the present invention is characterized in that, in a case in which the object is moved at a speed higher than a predetermined speed based on the fixed flash cycle of the strobe flashlight device with the result that the object images are spaced apart from each other in the multiple exposure image as described above, the coordinates of the center points of the object images are extracted by the first processing means 250 (see FIG. 1) and, in a case in which the object is moved at a speed lower than the predetermined speed based on the fixed flash cycle of the strobe flashlight device with the result that the object images overlap each other in the multiple exposure image as described above, the coordinates of the center points of the object image region are extracted by the second processing means 260 (see FIG. 1).
[80] Also, the sensing processing unit of the sensing device according to the present
invention may include the first sensing processing means 230 (see FIG. 1) to analyze an image of the swung golf club in the multiple exposure image when the golf ball is moved by the golf club as shown in FIGS. 4(a) and 4(b) to extract the coordinates of the center point of the golf club and the second sensing processing means 240 (see FIG. 1) to analyze an image of the golf ball in the multiple exposure image when the golf ball is moved by the swung golf club as shown in FIGS. 4(a) and 4(b) to extract the coordinates of the center point of the golf ball, which will be hereinafter described in detail.
[81] First, image processing of the separate object images as shown in FIG. 4(a) by the first processing means will be described with reference to FIGS. 5 to 8.
[82] FIG. 5(a) is a view showing the original copy of a multiple exposure image including separate object images. The multiple exposure image is acquired using a camera device having a low resolution of 320 x 240 and operated at a low speed of 75 fps and a strobe flashlight device operating at 330 Hz.
[83] An image acquired through removal of a stationary image, i.e. a background image, from the original image shown in FIG. 5(a) through subtraction and a predetermined preliminary processing operation, such as Gaussian blur, is shown in FIG. 5(b).
[84] The image acquired through the preliminary processing operation as shown in FIG.
5(b) contains noise in addition to the images of the object.
[85] Preferably, an image estimated as the object is extracted as shown in FIG. 6.
[86] The image shown in FIG. 6 is obtained by overlapping a plurality of frames captured by the camera device into a single image. On the assumption that three object images formed by generating three flashes constitute an image set, an image set SO indicates an image of a zeroth frame, an image set S 1 indicates an image of a first frame, and an image set S2 indicates an image of a second frame.
[87] Here, an image estimated by the object image may be extracted through a contour check or a check window. The processing may be carried out so that an image estimated as the object is kept while images estimated not to be the object are discarded.
[88] That is, the object has a predetermined size (a predetermined diameter for a golf ball). Therefore, the contours of the respective images of the multiple exposure image are checked in consideration of the size of the object, and the checked contours may be excluded if they are too large or too small to be the object.
[89] Also, as shown in FIG. 6, check windows W corresponding to the size of the respective images 11 in the multiple exposure image are formed so as to include the respective images 11 , and an aspect ratio of the respective check windows W is checked.
[90] If the object is a golf ball, the aspect ratio of the golf ball must be almost equal. If the aspect ratio of the check windows W corresponding to the respective images 1 1 in the multiple exposure image is too large with the result that each of the check windows W has a large width or is too small with the result that each of the check windows W has a small width, the respective images 11 in the multiple exposure image are considered not to be the object and thus may be discarded.
[91] Consequently, it is possible for the first image extracting means 251 (see FIG. 1) of the first processing means to extract images 11 estimated as the object from the multiple exposure image using the above illustrated method.
[92] Meanwhile, the images 11 extracted as the object images as described above are fitted into the shape of an object as shown in FIGS. 7(a) and 7(b) by the first coordinate extracting means 252 (see FIG. 1) of the first processing means to extract the coordinates of a center point of each of the images.
[93] FIG. 7(a) is an enlarged view of an image set extracted from the multiple exposure image shown in FIG. 6 and FIG. 7(b) is a view showing a fitted result of the object.
[94] As shown in FIG. 7(a), the object image is partially damaged as the result of the preliminary processing. In particular, an image perfectly appears at one side at which the strobe flashlight is irradiated to the object, whereas a shaded region is formed at the other side at which the strobe flashlight is not irradiated to the object with the result that the shape of the object is distorted depending upon the size of the shaded region.
[95] In FIG. 7(a), the object image 11 has an irradiated region Rl and a shaded region R2.
[96] In this case, if the damaged object image 11 is fitted without consideration of the shaded region R2, the coordinates of the center point of the object image 11 is greatly distorted, which is not preferable.
[97] Preferably, therefore, as shown in FIG. 7(a), fitting is performed so that the shaded region R2 is properly included by a fitted curve FC on the basis of the irradiated region Rl of the object image 11.
[98] Here, the fitted curve may be formed based on information on the size of the object which is previously stored or based on the measured size of the object image 11.
[99] A fitted object image 11 as shown in FIG. 7(b) may be obtained through the fitting as described above, and the coordinates of the center point CP of the object may be accurately extracted based on the shape of the fitted object image.
[100] The center points of all extracted object images are extracted using the above method to accurately obtain information on two-dimensional coordinates of the object from the acquired image.
[101] However, images which are not object may be included in all images, the center points of which have been extracted.
[102] In this case, it is possible for the first coordinate extracting means 252 (see FIG. 1) to accurately extract only object images based on specific conditions as shown in FIG. 8.
[103] That is, according to the trigger signal scheme of the camera device and the strobe flashlight device shown in FIG. 3, the object must have three images per frame and the interval between the object images must be kept at the same ratio as the strobe flashlight cycle, i.e. the trigger time intervals tsl and ts2 (see FIG. 3).
[104] As shown in FIG. 8, in an image set SO or SI, intervals a and b or d and e
[105] between object images 1 la, 1 lb and 1 lc must be substantially equal to each other.
Also, a ratio of c to a or c to b must be substantially equal to a ratio of a strobe flashlight trigger time interval ts2 to strobe flashlight trigger time interval tsl (see FIG. 3).
[106] It is determined whether the object images in each of the image sets have been accurately extracted based on the above specific conditions. The accurately extracted images are confirmed as the final object images. If the number of object images in each of the image sets is insufficient or excessive, the object images are corrected so as to satisfy the specific conditions and confirmed as the final object images.
[107] The coordinates of the center points of the object images finally confirmed as
described above are confirmed as the final coordinates.
[108] Therefore, it is possible to accurately extract the coordinates of the center points of the objects although the low quality image acquired by the low resolution and low speed camera device and strobe flashlight device is used.
[109] The coordinates of the center points of the finally confirmed objects are converted into three-dimensional coordinates by the converting means, and physical information of the moving orbit of the object to be simulated is obtained based on the three- dimensional coordinates.
[110] Next, image processing of the overlapping object images as shown in FIG. 4(b) by the second processing means will be described with reference to FIGS. 9 to 12.
[I l l] FIG. 9(a) is a view showing the original copy of a multiple exposure image including overlapping object images. The multiple exposure image is acquired using a camera device having a low resolution of 320 x 240 and operated at a low speed of 75 fps and a strobe flashlight device operating at 330 Hz.
[112] The second image extracting means 261 (see FIG. 1) of the second processing means extracts an object image region constituted by the overlapping object image as shown in FIG. 9(a) (see FIGS. 9(a) and 10), and the second coordinate extracting means 262 (see FIG. 1) of the second processing means extracts the coordinates of at least two center points of the extracted object image region (see FIGS. 11 and 12).
[113] An image acquired through removal of a stationary image, i.e. a background image, from the original image shown in FIG. 9(a) through subtraction and a predetermined preliminary processing operation, such as Gaussian blur, is shown in FIG. 9(b).
[114] The image acquired through the preliminary processing operation as shown in FIG.
9(b) contains noise in addition to the images of the object.
[115] Preferably, an image estimated as the object image region, in which the object
images overlap each other, is extracted as shown in FIG. 10.
[116] First, a moving direction of a moving object is estimated using a tool or program to estimate the moving direction of the moving object. As shown in FIG. 10(a), a predetermined image region arranged on an axis 13 of the estimated moving direction is extracted as an object image region 12 in which object images overlap each other. [117] The extracted object image region may contain noise Nl as shown in FIG. 10(a). As shown in FIG. 10(b), pixel intensity of the image on the axis 13 of the moving direction may be checked to determine whether the portion l in the image is an image of a real object or noise.
[118] FIG. 10(b) is a graph showing the distribution of pixel intensity of the images Nl and 12 shown in FIG. 10(a) on the axis 13 of the estimated moving direction.
[119] The object images overlap each other to constitute an image region of a predetermined size. Consequently, in a case in which a region having the distribution of the pixel intensity equal to or greater than a critical value corresponds to a predetermined range, the corresponding image may be extracted as an image region of the
overlapping object images. On the other hand, in a case in which the length of a region having the distribution of the pixel intensity equal to or greater than the critical value is very short, it is determined that this region does not correspond to the overlapping object images, and therefore, this region may be excluded.
[120] In FIG. 10(b), the noise image Nl has a pixel intensity section of al to a2, and the object image region 12 has a pixel intensity section of a3 to a4. Since the section of a3 to a4 is long sufficient to be estimated as the object image region, in which the object images overlap each other, the section of a3 to a4 may be extracted as the object image region. On the other hand, since the section of al to a2 is too short to be estimated as the object image region, in which the object images overlap each other, the section of al to a2 may be excluded.
[121] In this way, the image estimated as the object image region, in which the object images overlap each other, in the multiple exposure image can be extracted.
[122] Meanwhile, the image region extracted as described above may be fitted to obtain the coordinates of center points of the image region. As shown in FIG. 11, fitted curves HC1 and HC2 having a shape of the object are formed at the contour parts of opposite ends of the image region 12 about the axis 13 of the moving direction.
[123] Pixel intensity on the fitted curves HC1 and HC2 is checked while moving from the center to the contour parts of opposite ends of the image region 12 along the axis 13 of the moving direction. The fitted curves HC1 and HC2 are located at positions at which the pixel intensity starts to be abruptly lowered.
[124] As shown in FIGS. 12(a) and 12(b), the coordinates of two center points of the image region 12 fitted as described above may be extracted.
[125] That is, as shown in FIG. 12(a), estimated center points EP1 and EP2 estimated as the center points of the fitted objects located at opposite ends of the image region are set and the positions of the estimated center pints EP1 and EP2 are corrected to obtain correct center points CP1 and CP2 as shown in FIG. 12(b).
[126] The respective center points EP1 and EP2 estimated as shown in FIG. 12(a) may be corrected to be correct center points by checking three directional lines rl, r2 and r3 of the fitted curve HC1 and three directional lines r4, r5 and r6 of the fitted curve HC2.
[127] That is, if the object is a golf ball, the object fitted as shown in FIG. 12(a) is fitted by the half circle curves HC1 and HC2. Since the object is circular, the three directional length of the object from the center thereof must be equal.
[128] Consequently, the estimated center points EP1 and EP2 are corrected so that the three directional lines shown in FIG. 12(a), i.e. the three directional lines rl, r2 and r3 of the fitted curve HC1 and three directional lines r4, r5 and r6 of the fitted curve HC2, must substantially have the same length.
[129] In this way, two center points CP1 and CP2 of the object image region 12, in which the object images overlap each other, are confirmed as shown in FIG. 12(b).
[130] Consequently, it is possible to accurately extract the coordinates of the center points of the overlapping object images even based on the overlapping images, the shape of which cannot be confirmed, acquired by the low resolution and low speed camera devices and strobe flashlight device.
[131] The coordinates of the center points of the object image region finally confirmed as described above are converted into three-dimensional coordinates by the converting means, and physical information of the moving orbit of the object to be simulated is obtained based on the three-dimensional coordinates.
[132] Hereinafter, embodiments of a sensing method for a moving object according to the present invention will be described with reference to FIGS. 13 to 14.
[133] As shown in FIG. 13, first, a trigger signal is applied by the signal generating unit so that the camera device and the strobe flashlight device continuously capture an image of a predetermined region in which an object is placed at a predetermined interval
(S10).
[134] The camera device senses an initial position of the object while continuously
capturing the image of the predetermined region as described above (S20). The initial position of the object is sensed because the initial position of the object is specified in a multiple exposure image acquired by the movement of the object and a moving direction of the object is estimated based on the specified initial position of the object. Preferably, therefore, coordinate values of the initial position of the object is confirmed at the step of sensing the initial position of the object.
[135] Subsequently, triggering of the object is sensed (S30). For example, the time when the object starts to move is sensed as if impact time of a golf ball at which the golf ball is hit by a golf club is sensed.
[136] When the movement of the object is triggered, the camera device acquires and stores a multiple exposure image having a plurality of frames under strobe flashlight before and after the triggering time (S40). [137] At this time, in a case in which a second object (for example, a golf ball) moved by a first moving object (for example, a golf club) is acquired and stored, a first object process to extract the coordinates of a center point of the first object from the acquired image through image processing of the first object is carried out, and a second object process to extract the coordinates of a center point of the second object from the acquired image through image processing of the second object is carried out.
[138] The first object process and the second object process may be simultaneously carried out. Alternatively, either the first object process or the second object process may be carried out.
[139] First, in a case in which the second object process is carried out, an image pattern of the second object in the acquired image is recognized (SI 00), the first process is carried out by the first processing means (SI 10) and the second process is carried out by the second processing means (S120).
[140] The first process (SI 10) and the second process (SI 20) may be simultaneously or selectively carried out.
[141] The coordinates of center points of separated object images are extracted by the first processing means to carry out the first process (S130) (see FIGS. 5 to 8).
[142] The coordinates of a plurality of center points of overlapping object images are
extracted by the second processing means to carry out the second process (SI 40) (see FIGS. 9 to 12).
[143] Here, an overlapping object image, if any, in the multiple exposure image during execution of the first process is processed to be excluded. Also, a separate object image, if any, in the multiple exposure image during execution of the second process is processed to be excluded.
[144] That is, in a case in which the object is move at high speed (the object images are separated from each other), both the first process and the second process are carried out. In this case, however, any separate object image is excluded during execution of the second process, and therefore, the coordinates of the center points are acquired based on the execution result of the first process. On the other hand, in a case in which the object is move at low speed (the object images overlap each other), both the first process and the second process are carried out. In this case, however, any overlapping object image is excluded during execution of the first process, and therefore, the coordinates of the center points are acquired based on the execution result of the second process.
[145] Meanwhile, an image pattern of the first object in the multiple exposure image
acquired at Step S40 is recognized (S200), and a first object process is carried out by the first sensing processing means (S210).
[146] Subsequently, the coordinates of a center point of an image of the first object in the multiple exposure image are extracted (S220).
[147] For example, in a case in which a golf club is swung to hit a golf ball, both an image of the golf club and an image of the golf ball are included in the multiple exposure image. The image processing of the hit golf ball is carried out at Steps S100 to S140, and the image processing of the swung golf club is carried out at Steps S200 to S220.
[148] When the coordinates of the center points of the first object and the second object are acquired as described above, the acquired coordinates of the center points of the first object and the second object are converted into three-dimensional coordinates (S300).
[149] Subsequently, physical information on the moving objects (the first object and the second object) is calculated based on the converted three-dimensional coordinates so that moving orbits of the moving objects are simulated (S310).
[150] Meanwhile, FIG. 14 and 15 show an example of a sensing method of a sensing
device for a moving object applied to a virtual golf simulation device according to the present invention.
[151] FIG. 14 and 15 are divided a flow chart of a sensing method into two parts. "A" described in FIG. 14 is connected to "A" described in FIG. 15, and "B" described in FIG. 14 is connected to "B" described in FIG. 15.
[152] When a virtual golf game is performed (SI), a simulator determines whether a golfer will perform a tee shot (S2), an iron shot (S3), an approach shot (S4) or a putting (S5).
[153] In a case in which the golfer performs the tee shot, the iron shot or the approach shot, the sensing method shown in FIG. 13 is preferably used. On the other hand, in a case in which the golfer performs the putting, the sensing method shown in FIG. 13 excluding the image processing method based on the first process is preferably used.
[154] That is, when the golfer performs the tee shot, the iron shot or the approach shot, a trigger signal of the camera device and the strobe flashlight device is generated by the signal generating unit (SI 1), an initial position of the golf ball is sensed (S21) and impact time of the golf ball is sensed (S31).
[155] When the golf ball is impacted, a multiple exposure image having a plurality of
frames with respect to the moving golf ball is acquired and stored by the camera device and the strobe flashlight device (S41).
[156] The first process (Si l l) and the second process (S 121 ) are simultaneously carried out. The coordinates of the center points of the separate golf ball images in the multiple exposure image are extracted by the first process (S131) and the coordinates of the center points of the overlapping golf ball images in the multiple exposure image are extracted by the second process (S141).
[157] For example, when the golfer performs the tee shot, the golf ball is moved at high speed since the golf ball is strongly hit by a number one wood driver at the tee shot. When a multiple exposure image of the golf ball moved at high speed is acquired, golf ball images are separated from each other so as to correspond to a flash interval of the strobe flashlight device. In a case in which the golf ball images are separated from each other as described above, the first process and the second process are carried out. Since the second process is programmed to process the overlapping golf ball images, however, the execution of the second process is stopped with the respect to the separate golf ball images, and the separate golf ball images is processed by the first process to extract the coordinates of the center points of the separate golf ball images.
[158] If the golfer performs the approach shot, the golfer may strongly or weakly hit the golf ball in consideration of the flight distance of the golf ball. When a multiple exposure image of the golf ball hit by the golfer is acquired and the first process and the second process are simultaneously carried out, it is possible to acquire the coordinates of the center point of the golf ball by one of the first and second processes regardless of whether the golf ball images are separated from each other or overlap each other.
[159] In a case in which the golfer performs the putting, on the other hand, only the second process is preferably carried out without execution of the first process.
[160] That is, a trigger signal of the camera device and the strobe flashlight device is
generated by the signal generating unit (S12), an initial position of the golf ball is sensed (S22) and impact time of the golf ball by the putter is sensed (S32).
[161] When the golf ball is putted, a multiple exposure image having a plurality of frames with respect to the moving golf ball is acquired and stored by the camera device and the strobe flashlight device (S42) and the second process (S 121) is carried out to extract the coordinates of the center points of the overlapping golf ball images in the multiple exposure image (S141).
[162] For all of the shots and the putting as described above, coordinates information of the center points of the golf ball images extracted from the multiple exposure image are converted into three-dimensional coordinates by a predetermined tool (S301), physical information of the golf ball is calculated based on the three-dimensional coordinates of the golf ball to simulate an orbit of the golf ball, thereby achieving virtual golf simulation (S311).
[163] Meanwhile, FIG. 14 shows the process of extracting only the image of the golf ball using the camera device and the strobe flashlight device to extract the center point of the golf ball. However, the image of the golf club is also included in the multiple exposure image acquired by the camera device and the strobe flashlight device. Consequently, it is possible to extract the coordinates of the center point of the golf club through image processing of the golf club at Step S200 to S220 of FIG. 13 and to use the extracted coordinate information as a basis of calculating the physical information of the golf ball. Mode for the Invention
[164] Various embodiments of a sensing device and method and a virtual golf simulation device using the same have been described in the best mode for carrying out the invention.
Industrial Applicability
[165] In a sensing device and method for a moving object according to the present
invention as described above and a virtual golf simulation device using the same, it is possible to acquire a multiple exposure image of a moving object using a low resolution and low speed camera device and strobe flashlight device and accurately extract an image of the object from the acquired image to obtain coordinates of a center point of the object and, in particular, to process a moving object through separate processes based on the velocity of the object or based on patterns of object images in a multiple exposure image to expand a sensing processing ability and range and achieve accurate sensing although a low resolution and low speed camera device is used. Consequently, the present invention can be widely used in industries related to a sensing device and method for a moving object and a virtual golf simulation device using the same.

Claims

Claims
A sensing device for a moving object comprising:
at least one camera device to acquire plural frames of images with respect to a moving state of an object;
a strobe flashlight device to generate a plurality of flashes per frame to acquire a multiple exposure image; and
a sensing processing unit comprising a first processing means to extract coordinates of the object in separate object images of the acquired multiple exposure image and, a second processing means to extract coordinates of the object in overlapping object images of the acquired multiple exposure image.
The sensing device according to claim 1, further comprising a signal generating unit to generate a trigger signal having a uniformly fixed cycle with respect to the camera device and the strobe flashlight device. The sensing device according to claim 2, wherein the signal generating unit is configured to generate the trigger signal so that a time interval between flashes per frame of the strobe flashlight device is substantially uniform.
The sensing device according to claim 3, wherein the signal generating unit is configured to generate the trigger signal so that a time interval between flashes at one frame is different from a time interval between the last flash at that frame and a first flash of the next frame.
The sensing device according to claim 1 , wherein
the first processing means comprises a first image extracting means to extract object images separated at a predetermined interval estimated as the object in the acquired multiple exposure image and, a first coordinate extracting means to obtain coordinates of a center point of each of the object images using the extracted images, and
the second processing means comprises a second image extracting means to extract overlapping object images estimated as the object in the acquired multiple exposure image and, a second coordinate extracting means to obtain coordinates of a plurality of center points of the overlapping object images using the extracted images.
A sensing device for a moving object comprising:
at least one camera device to acquire plural frames of images with respect to a moving state of an object;
a strobe flashlight device to generate a plurality of flashes at a uniformly fixed cycle per frame to acquire a multiple exposure image;
and
a sensing processing unit comprising a first processing means to extract coordinates of the object from images of the object moving at a velocity equal to or greater than a predetermined velocity based on the fixed flash cycle of the strobe flashlight device and, a second processing means to extract coordinates of the object from images of the object moving at a velocity less than the predetermined velocity based on the fixed flash cycle of the strobe flashlight device.
[Claim 7] A sensing device for a moving object comprising:
a sensor unit to acquire a multiple exposure image of a moving object; and
a sensing processing unit comprising a plurality of processing means to extract coordinates of the object through image processing based on different processes according to patterns of object images in the acquired multiple exposure image.
[Claim 8] The sensing device according to claim 7, wherein the sensor unit is configured so that the acquired multiple exposure image includes at least one of object images separated from each other at a predetermined interval and object images overlapping each other based on velocity of the moving object.
[Claim 9] A sensing device for a moving object comprising:
a sensor unit to acquire a multiple exposure image of a first moving object and a second object moved by the first object;
a first sensing processing means to process an image of the first object in the acquired multiple exposure image to extract coordinates of the first object; and
a second sensing processing means to process an image of the second object in the acquired multiple exposure image to extract coordinates of the second object.
[Claim 10] The sensing device according to claim 9, wherein the second sensing processing means comprises:
a first processing means to process separate images of the second object in the acquire multiple exposure image to extract coordinates of the second object; and
a second processing means to process overlapping images of the second object in the acquire multiple exposure image to extract coordinates of the second object. [Claim 11] A sensing method for a moving object comprising:
acquiring a multiple exposure image of a moving object;
extracting images estimated as the object from the acquired multiple exposure image; and
processing the extracted object images through a first process if the extracted object images are separate object images and processing the extracted object images through a second process if the extracted object images are overlapping object images.
[Claim 12] The sensing method according to claim 11, wherein the step of
acquiring the multiple exposure image comprises acquiring plural frames of images with respect to the moving object through at least one camera device and generating a plurality of flashes per frame through a strobe flashlight device.
[Claim 13] The sensing method according to claim 11, wherein the step of
acquiring the multiple exposure image comprises:
synchronizing at least one camera device and a strobe flashlight device; continuously capturing an image through the camera device;
confirming an initial position of the object from the captured image; generating a trigger signal with respect to the camera device and the strobe flashlight device; and
acquiring plural frames of images with respect to the moving object through the camera device and generating a plurality of flashes per frame through the strobe flashlight device.
[Claim 14] The sensing method according to claim 12 or 13, wherein a cycle
between the frames acquired by the camera device is uniformly fixed and a time interval between flashes per frame of the strobe flashlight device is substantially uniform.
[Claim 15] The sensing method according to claim 11, wherein
the first process comprises fitting the separate object images and acquiring coordinates of a center point of each of the fitted object images, and
the second process comprises fitting the overlapping object images and acquiring coordinates of plurality of center points of the fitted overlapping object images.
[Claim 16] The sensing method according to claim 11, wherein the first process comprises:
fitting each of the separate object images into the shape of the object in consideration of light direction so that a shaded region is included in each of the separate object images; and
acquiring coordinates of a center point of each of the fitted object images.
[Claim 17] The sensing method according to claim 11, wherein the second process comprises:
extracting an axis of a moving direction of the object from an initial position of the object in consideration of the moving direction of the object; and
checking the distribution of pixel intensity of the respective images passing through the axis of the moving direction along the axis of the moving direction and, when a region having pixel intensity equal to or greater than a critical value corresponds to a predetermined range, extracting the corresponding image as an image region of the overlapping object images.
[Claim 18] A sensing method for a moving object comprising:
acquiring a multiple exposure image of a first moving object and a second object moved by the first object;
processing images of the first object in the acquired multiple exposure image to extract coordinates of the first object; and
processing images of the second object in the acquired multiple exposure image through a first process if the images of the second object are separated from each other and processing images of the second object in the acquired multiple exposure image through a second process if the images of the second object overlap each other to extract coordinates of the second object.
[Claim 19] A virtual golf simulation device comprising:
a sensor unit to acquire a multiple exposure image of a golf ball hit by a golfer;
a sensing processing unit comprising a plurality of processing means to extract coordinates of the golf ball through image processing based on different processes according to image patterns of the golf ball in the multiple exposure image acquired by the sensor unit; and
a simulator to convert the coordinates extracted by the sensing processing unit into three-dimensional coordinates and to calculate physical property information on the moving golf ball based on the three-dimensional coordinates, thereby simulating an orbit of the golf ball.
[Claim 20] The virtual golf simulation device according to claim 19, wherein the plurality of processing means comprises:
a first processing means to process a plurality of golf ball images in the acquired multiple exposure image if the golf ball images are separated from each other to extract coordinates of the golf ball; and
a second processing means to process a plurality of golf ball images in the acquired multiple exposure image if the golf ball images overlap each other to extract coordinates of the golf ball.
[Claim 21] The virtual golf simulation device according to claim 19, wherein the plurality of processing means is configured so that a first processing means or a second processing means to extract coordinates of the golf ball is selectively applied depending upon kind of swings performed by the golfer or a flight distance of the golf ball during virtual golf simulation using the simulator.
[Claim 22] A virtual golf simulation device comprising:
a sensor unit to acquire a multiple exposure image of a golf club swung by a golfer and a golf ball hit by the golf club;
a sensing processing unit comprising a first sensing processing means to extract coordinates of the golf club by processing an image of the golf club in the multiple exposure image acquired by the sensor unit and a second sensing processing means comprising a plurality of processing means to extract coordinates of the golf ball through image processing based on different processes according to image patterns of the golf ball in the acquired multiple exposure image; and
a simulator to convert the coordinates extracted by the sensing processing unit into three-dimensional coordinates and to calculate physical information on the golf club and the golf ball based on the three-dimensional coordinates, thereby simulating an orbit of the golf ball.
PCT/KR2011/004758 2010-06-29 2011-06-29 Sensing device and method for moving object and virtual golf simulation device using the same WO2012002731A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR1020100062205A KR101019823B1 (en) 2010-06-29 2010-06-29 Apparatus and method for sensing moving object and virtual golf simulation device using the same
KR10-2010-0062205 2010-06-29

Publications (2)

Publication Number Publication Date
WO2012002731A2 true WO2012002731A2 (en) 2012-01-05
WO2012002731A3 WO2012002731A3 (en) 2012-04-26

Family

ID=43938477

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2011/004758 WO2012002731A2 (en) 2010-06-29 2011-06-29 Sensing device and method for moving object and virtual golf simulation device using the same

Country Status (2)

Country Link
KR (1) KR101019823B1 (en)
WO (1) WO2012002731A2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021156560A1 (en) * 2020-02-06 2021-08-12 Centre National d'Études Spatiales Method and device for optical detection

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101048091B1 (en) 2011-03-22 2011-07-08 (주) 골프존 Apparatus and method for virtual golf simulation, and sensing device and method used to the same
KR101276054B1 (en) * 2011-06-27 2013-06-17 동서대학교산학협력단 Ball mark tracking system
KR101649181B1 (en) 2015-01-22 2016-08-18 주식회사 인피니전 Flight information estimator and estimation method of the flying objects
KR101645693B1 (en) * 2015-02-26 2016-08-04 주식회사 파온테크 Apparatus and method for simulation golf putting using virtual cross stripes
KR101602222B1 (en) * 2015-04-29 2016-03-10 (주)레드골프 Golf balls recognition of golf simulation
KR101865576B1 (en) * 2016-01-22 2018-06-08 주식회사 골프존뉴딘홀딩스 Apparatus for base-ball practice, sensing device and sensing method used to the same and control method for the same
KR101995098B1 (en) * 2016-05-18 2019-07-01 이미화 Golf balls recognition of golf simulation
KR102084858B1 (en) 2017-11-27 2020-03-04 최은정 A method of recognizing only golf ball objects in photograph using cameras in a golf simulator

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001264016A (en) * 2000-03-15 2001-09-26 Sumitomo Rubber Ind Ltd Motion-measuring instrument for ball
JP2003117044A (en) * 2001-04-26 2003-04-22 Photron Ltd Hit ball diagnosing device
US20050026710A1 (en) * 2003-08-01 2005-02-03 Yi-Ching Pao Apparatus and method for monitoring launch parameters of a launched sports object
KR20090021407A (en) * 2007-08-27 2009-03-04 마이크로 인스펙션 주식회사 Apparatus for measuring motion of golf ball and control method thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2001264016A (en) * 2000-03-15 2001-09-26 Sumitomo Rubber Ind Ltd Motion-measuring instrument for ball
JP2003117044A (en) * 2001-04-26 2003-04-22 Photron Ltd Hit ball diagnosing device
US20050026710A1 (en) * 2003-08-01 2005-02-03 Yi-Ching Pao Apparatus and method for monitoring launch parameters of a launched sports object
KR20090021407A (en) * 2007-08-27 2009-03-04 마이크로 인스펙션 주식회사 Apparatus for measuring motion of golf ball and control method thereof

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021156560A1 (en) * 2020-02-06 2021-08-12 Centre National d'Études Spatiales Method and device for optical detection
FR3107123A1 (en) * 2020-02-06 2021-08-13 Centre National d'Études Spatiales Optical detection method and device.

Also Published As

Publication number Publication date
KR101019823B1 (en) 2011-03-04
WO2012002731A3 (en) 2012-04-26

Similar Documents

Publication Publication Date Title
WO2012002731A2 (en) Sensing device and method for moving object and virtual golf simulation device using the same
US11033826B2 (en) Methods and systems for sports simulation
CA2830488C (en) Sensing device and method used for virtual golf simulation apparatus
CN108697922B (en) Sensing device and sensing method for baseball practice device, baseball practice device using the same, and control method thereof
US9162132B2 (en) Virtual golf simulation apparatus and sensing device and method used for the same
JP5626939B2 (en) Sensing processing device for moving ball, sensing processing method, and virtual golf simulation device using the same
CA3086676C (en) Golf ball tracking system
US20130281223A1 (en) Virtual golf simulation apparatus and method capable of compensation ball flight distance decreasing rate
WO2009023598A1 (en) Sports simulator and simulation method
JP5572853B2 (en) Sensing device for moving object, sensing processing method, and virtual golf simulation device using the same
KR101019801B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
KR101019902B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
KR101019782B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
KR101019829B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
KR101019847B1 (en) Sensing processing device and method for moving object, and virtual golf simulation device using the same
KR101019824B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
GB2570847A (en) Sensing device for calculating information on user's golf shot and sensing method using same
KR101019798B1 (en) Apparatus and method for sensing moving object and virtual golf simulation device using the same
WO2012002733A2 (en) Sensing device and sensing processing method for moving object and virtual golf simulation device using the same

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11801137

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 11801137

Country of ref document: EP

Kind code of ref document: A2