TWI448319B - Sensing device and method used for virtual golf simulation apparatus - Google Patents

Description

Sensing device and sensing method for virtual golf simulation device

The present invention relates to a sensing device and a sensing method for a virtual golf simulation device, and more particularly to a trajectory in which a virtual golf course is imaged and simulated and the golf ball hit by the user is sensed by the sensing device and A sensing device and a sensing method used in a virtual golf simulation device simulated on a virtual golf course.

In recent years, many devices have been developed to allow users to enjoy popular sports games, such as baseball, soccer, basketball, and golf, in the form of interactive sports games, either indoors or in specific places.

In particular, in recent years, a so-called screen golf system has been developed in which when a user swings a golf club to strike a golf ball placed on a ball striking pad, the sensor senses the tapped golf ball to extract the movement The physical information on the golf ball enables the trajectory of the golf ball to be simulated on the virtual golf course, so that the user can enjoy the golf in the virtual reality.

In order to simulate the use of sports such as golf balls in such interactive sports games, many studies have been introduced into many sensing systems for accurately sensing the physical information on a moving golf ball, ie the movement of the golf ball. .

For example, many sensing devices, such as a sensing device using an infrared sensor, a sensing device using a laser sensor, a sensing device using an acoustic wave sensor, and a sensing device using a photo sensor, They have all been introduced to the market. Many studies have been introduced into photosensor type sensing devices for obtaining and analyzing images of moving golf balls in order to accurately sense the state of the moving golf ball.

However, in the photographic sensing type sensing device, a super-high speed camera having a high resolution is necessary for greatly increasing the sensing of the cost of the sensing device. On the other hand, if a camera with very low resolution and very low speed is used, the image quality obtained by the camera will be low, and as a result, it is difficult to accurately extract and analyze the golf ball from the obtained golf club. It is difficult to derive information on the physical characteristics of mobile golf balls. In particular, it is difficult to measure the rotation of the golf ball.

Accordingly, the present invention has been made in view of the above problems, and an object of the present invention is to provide a sensing device and a sensing method for a virtual golf simulation device in which a photo with very low resolution and speed is inexpensively analyzed The images obtained by the machine are very accurate in extracting information on physical characteristics, such as the speed, direction and elevation angle of the moving golf ball, especially the moving trajectory of the golf club, which is calculated very accurately from the obtained image, so as to be very accurate estimation. The rotation of the golf ball and the estimated rotation of the golf ball in the golf simulation, thereby constructing a virtual golf simulation device with high accuracy and reliability at a low cost, and further improving the realism of the virtual golf.

In accordance with features of the present invention, the above and other objects are achieved by providing a virtual golf simulation device comprising: a camera unit for obtaining a user swinging a golf club and tapping the golf ball a plurality of frame images of the golf ball; and a sensing processing unit including a golf ball image processing device and a golf club image processing device for each of the image frames from the frames The frame image extracts a golf ball to obtain a three-dimensional coordinate of the golf ball, and the golf club image processing device is configured to extract an effective moving object from each obtained image of the frame to calculate a golf club head from the effective object. The movement trajectory to calculate the information on the physical characteristics of the mobile golf ball.

According to another feature of the present invention, a sensing device for a virtual golf simulation device is provided, comprising: a photographic unit for obtaining a plurality of golf balls that a user swings a golf club to strike a golf ball a frame image; and a sensing processing unit including a golf ball image processing device and a golf club image processing device for performing two-dimensional analysis of each frame from the frame images The image extracts a trajectory of the golf ball, thereby extracting a three-dimensional coordinate of the golf ball on the trajectory of the golf ball, and the golf club image processing device is configured to extract one of the moving object from each frame image obtained through two-dimensional analysis. The lines are matched and used to extract the movement trajectory of the golf club head through the three-dimensional matching of the matching lines, thereby calculating the information on the plurality of physical characteristics of the moving golf ball.

According to another feature of the present invention, a sensing device for a virtual golf simulation device is provided, comprising: a photographic unit for obtaining a plurality of golf balls that a user swings a golf club to strike a golf ball a frame image; and a sensing processing unit comprising a first processing device and a second processing device, the first processing device for extracting and aggregating individual frame images The difference between the images to extract a valid moving object and extract a matching line from the extracted effective object, and the second processing device is used to calculate the trajectory of a specific portion on the matching line as the movement of the golf club head Track.

According to another feature of the present invention, a sensing device for a virtual golf simulation device is provided, comprising: a camera unit comprising a plurality of cameras for obtaining a user to swing a golf club and tapping the golf ball a plurality of frame images of the golf ball; and a sensing processing unit for extracting matching lines relative to the valid moving object from each image obtained by the individual camera through two-dimensional analysis, and for The three-dimensional matching of the matching lines of the two-dimensional analysis extracts the trajectory of a specific part of the matching line as the moving trajectory of the golf club head.

According to another feature of the present invention, a sensing method for a sensing device of a virtual golf simulation device includes: continuously collecting a plurality of images of a preset shooting range by a shot; and analyzing the obtained images To sense whether the user has hit the golf ball; to extract and analyze the golf ball from a plurality of frame images after the golf ball has been hit by the user, to calculate multiple physical characteristics of the mobile golf ball Information on the top; and extracting and analyzing the movement trajectory of the golf club head to estimate the rotation of the moving golf ball.

According to another feature of the present invention, a sensing method for a sensing device of a virtual golf simulation device is provided, comprising: obtaining a plurality of frame images of a golf ball that a user swings a golf club to strike a golf ball Extracting valid moving objects from each image obtained; matching straight lines relative to the extracted valid objects; calculating the trajectory of a particular portion of the matching line; and determining the calculated trajectory as the movement trajectory of the golf club head In order to estimate the rotation of the moving golf ball.

Exemplary embodiments of a virtual golf simulation apparatus and method and a sensing apparatus and sensing method therefor according to the present invention will now be described in detail with reference to the accompanying drawings.

First, a virtual golf simulation apparatus and a sensing apparatus used therefor according to an embodiment of the present invention will be described with reference to FIGS. 1 and 2.

1 is a schematic diagram showing an example of a screen golf system to which a virtual golf simulation apparatus according to an embodiment of the present invention is applied, and FIG. 2 is a block diagram showing a structure of a virtual golf simulation apparatus to which the screen golf system of FIG. 1 is applied.

As shown in FIGS. 1 and 2, a virtual golf simulation apparatus according to an embodiment of the present invention includes a sensing device S for sensing a golf ball B tapped by a user, and an image for realizing a virtual golf course. And based on the sensing result of the sensing device S, the simulator 1 of the simulated image of the trajectory of the golf ball B is provided on the virtual golf course, so that the virtual golf simulation can be performed.

As shown in FIG. 1, a screen golf system to which a virtual golf simulation apparatus according to an embodiment of the present invention is applied may be configured to have a structure in which a hitting table 110 is disposed on the ground of a golf room 2 that provides a space of a preset size. And the user hits the golf ball B on the hitting table 110, the hitting pad 120 is disposed on one side of the hitting table 110, and the golf ball B is placed on the hitting pad 120 so that the user can be on the hitting table The golf ball B is tapped on the 110, and the screen 3 is placed in front of the golf room 2, and the virtual golf simulation image output from the image output device (the image output unit 30 shown in Fig. 2, such as a beam projector) is projected. On the screen 3, the image information is transmitted from the simulator 1 to the image output device.

The ball table 110 and the ball striking pad 120 are provided on the blade 100 shown in Fig. 1. Alternatively, the hitting table 110 and the ball striking pad 120 may be disposed on the ground of the golf room 2.

The blade 100 can be set such that the blade 100 can be tilted by a predetermined angle in the forward and backward directions as well as in the left and right directions. Moreover, the blade 100 can be coupled to the simulator 1 such that the blade 100 can be tilted corresponding to the terrain of the currently implemented virtual golf course.

The ball striking pad 120 may be formed of artificial turf. As shown in Fig. 1, the ball striking pad 120 can be divided into a fairway area 121, a long grass area 122, and a bunker area 123, which individually correspond to the fairway, long grass, and bunker of the real golf course. Preferably, the fairway area 121, the long grass area 122, and the bunker area 123 are configured such that the fairway area 121, the long grass area 122, and the bunker area 123 have artificial turf of different lengths, or the fairway area 121, the long grass area 122, and The bunker zone 123 is formed from a different material than a real golf course. 1 shows that the ball striking pad 120 includes a fairway area 121, a long grass area 122, and a bunker area 123, however, the present invention is not limited thereto. For example, the ball striking pad 120 may include only the fairway area 121, or may include a long grass area 122 and a bunker area 123 together with the fairway area 121.

Meanwhile, as shown in Fig. 1, the sensing device S is disposed in the golf room 2 to sense the golf ball B hit by the user.

That is, as shown in FIG. 2, the sensing device S includes a photographic unit 50 and a sensing processing unit 60, and the photographic unit 50 includes a plurality of cameras 51 and 52, and the sensing processing unit 60 It is used to process the image obtained by the photographing unit 50, thereby extracting the physical characteristics of the moving golf ball.

The photographing unit 50 can be constituted by a single camera or more cameras for image sensing. In order to obtain an image of the mobile golf ball B and extract the coordinates of the golf ball in three-dimensional space, it is preferable to configure a stereoscopic image unit using two or more cameras 51 and 52 that are cooperatively coupled to each other, such as the first The figure shows.

The physical characteristics of the moving golf ball may include the speed of the golf ball, the moving direction of the golf ball (the moving angle of the golf ball in the horizontal direction), the height of the golf ball (the moving angle of the golf ball in the vertical direction), and the rotation of the golf ball.

The sensing processing unit 60 may include a gripper 65 (please refer to FIG. 3), a shot detector 70, a golf ball image processing device 80, and a golf club image processing device 90, wherein the gripper 65 has each map The image obtained by the photographic unit 50 is sequentially collected on the basis of the frame, and the shot detector 70 is configured to receive and process the image of the frame collected by the gripper 65, thereby sensing whether the preparation for the shot has been completed and whether the shot is The golf image processing device 80 is configured to process the image of the golf ball received from the shot detector 70 when the shot detector 70 senses that the golf ball has been tapped by the user. The information on the physical characteristics of the mobile golf ball is extracted, and the golf club image processing device 90 is used to analyze the movement trajectory of the golf club from the image received by the shot detector 70, thereby extracting information on the rotation of the golf ball.

Moreover, the sensing processing unit 60 further includes a converting device 73 for matching the images obtained by the cameras 51 and 52, thereby converting the two-dimensional information of each camera image (the information extracted from each camera is Two-dimensional information is three-dimensional information, or conversely, three-dimensional information extracted from a plurality of camera images is converted into two-dimensional information.

Meanwhile, the simulator 1 constituting the virtual golf simulation apparatus according to the embodiment of the present invention preferably includes a controller M, a data library 10, a material processing unit 20, and an image output unit 30.

The database 10 stores all the materials necessary for the virtual golf simulation. For example, the database 10 stores the data necessary for the drive system, the materials necessary to realize the image of the virtual golf course, and the materials necessary for realizing the simulated image of the golf course.

The data processing unit 20 is a component that performs preset image processing to implement an image of a virtual golf course or a simulated image of a golf course on a virtual golf course.

The video output unit 30 outputs the image information received from the data processing unit 20 to the screen so that the user can see the image.

The controller M receives information according to the sensing result of the sensing device S to control the operations of all components, such as the database 10 of the simulator 1, the data processing unit 20, and the image output unit 30. That is, the controller M, the data library 10, and the data processing unit 20 function as an image realizing unit for realizing a simulated image of the golf ball trajectory according to the sensing result of the sensing device S.

Hereinafter, a more detailed structure of the sensing device S shown in FIG. 2 and functions of the individual elements constituting the sensing device S will be described with reference to FIG.

As shown in FIG. 3, the transmission of information in the sensing device according to the present invention is a photographic unit 50 → a gripper 65 → a shot detector 70 → a golf ball image processing device 80 and a golf club image processing device 90 → The order of the simulator 1 is performed.

The photographing unit 50 photographs a preset range including a portion where the hitting table and the hitting pad are provided to obtain hundreds of frame images per second.

The obtained frame image is transmitted to the gripper 65. The gripper 65 stores the image received from the photographing unit 50 and simultaneously transmits the image to the shot detector 70 for image processing.

Preferably, the shot detector 70 searches for a golf ball from the image received by the gripper 65 and determines whether the golf ball satisfies a preset condition to sense whether the shot preparation has been completed. And the shot detector 70 determines whether the golf ball has moved from the position where the shot preparation has been completed, thereby sensing whether the shot has been made by the user.

4(a) to 4(c) show that the shot detector 70 senses whether the shot has been processed by the user.

Fig. 4(a) is a schematic diagram showing one of several frame source images transmitted from the gripper 65 to the shot detector 70.

The shot detector 70 sets the shot area 220 in the source image 200, separates the shot area 220 from the source image 200, and finds the golf ball from the shot area 220. At this time, as shown in FIG. 4(a), an area corresponding to a part or the entire hitting pad 120 may be set as the hitting area 220. That is, a predetermined area in which the golf ball is located near the position of the ball striking pad 120 can be set as the ball area 220.

In the source image 200 shown in FIG. 4(a), the golf ball is placed on the fairway area 121 of the ball striking pad 120, and the shot detector 70 sets the area corresponding to the fairway area 121 as the hitting area. 220. If the golf is on the long grass area 122, the area corresponding to the long grass area 122 can be set as the hitting area. On the other hand, if the golf is on the bunker area 123, the area corresponding to the bunker area 123 can be set as the hitting area.

In this manner, the hitting zone 220 is separated and the golf ball is only found from the hitting zone 220, which is faster and more accurate than when the golf ball is found from the entire source image.

The process of finding a golf ball from the separate hitting area 220 can be performed using a number of methods. For example, the specific object appearing in the hitting area 220 and the degree of similarity between the golf samples that are preset and stored as reference images of the golf ball may be analyzed, and if the degree of similarity between them is equal to or greater than a preset degree, the specific object may be It was decided to become a golf ball.

Moreover, as shown in FIG. 4(b), the sensing area R of a predetermined size near the golf ball is set in the image of the hitting area 220. The sensing region R is preferably set to have a size small enough to contain a golf ball. Whether the user taps the golf ball can be easily determined by sensing whether the golf ball is placed in the sensing area R.

As a result, it is possible to sense whether or not the hitting has been performed by sensing whether or not the golf ball is placed in the sensing region R from the state shown in FIG. 4(b) to the state shown in FIG. 4(c).

When it is sensed that the hitting has been performed as described above, the shot detector 70 checks a plurality of frame images received before the hitting has been performed to confirm when the golf ball begins to move, that is, the hitting time, and the The frame image of the hitting time is designated as the trigger frame.

The shot detector 70 stores a plurality of frame images before the hitting on the basis of the trigger frame, and transmits the stored frame images to the golf club image processing device 90 so that the necessary images can be performed. The images transmitted to the golf club image processing device 90 may include an image of the frame of the shot time, that is, the trigger frame.

Moreover, the shot detector 70 stores a plurality of frame images after the hitting on the basis of the trigger frame, and simultaneously transfers the stored frame images to the golf ball image processing device 80 at the same time, so that necessary The image can be performed. The images transmitted to the golf ball image processing device 80 are transmitted to the golf club image processing device 90 so that the image can be performed. As a result, the golf club image processing device 90 receives and processes a plurality of frame images before the shot and a plurality of frame images after the shot.

The golf image processing device 80 processes the image received from the shot detector 70 to extract information on the three-dimensional coordinates of the golf ball, and uses the extracted information to calculate information on the physical characteristics, such as the speed and direction of the moving golf ball. And height angle.

The golf club image processing device 90 extracts information on the movement trajectory of the golf club head and estimates the rotation of the golf ball using the extracted information.

Information on the physical characteristics of the golf ball extracted by the golf ball image processing device 80 and the golf club image processing device 90 is transmitted to the simulator 1. The simulator 1 outputs a simulated image in which the golf ball moves on the virtual golf course in accordance with the received information, so that virtual golf simulation can be performed.

The golf ball image processing device 80 basically extracts a plurality of golf ball candidates from the images in the individual frames obtained by the photographing unit 50, converts the three-dimensional coordinates of each golf ball candidate into two-dimensional coordinates, and analyzes each golf ball. The two-dimensional trajectory of the ball candidate is used to extract the final trajectory of the golf ball, thereby calculating the information on the physical characteristics of the moving golf ball.

That is, as shown in FIG. 5, the golf ball image processing apparatus 80 extracts a plurality of golf candidates from the images obtained by the individual frames with respect to the photographing unit 50 (the golf ball in the first frame in Fig. 5) The ball candidates 1_1 and 1_2, the golf ball candidates 2_1 and 2_2 in the second frame, etc.), and the golf track list are generated to link the individual golf ball candidates with respect to the continuous frame image. Moreover, as shown in FIG. 6, the golf ball image processing apparatus 80 performs a quadratic polynomial regression analysis with respect to the golf ball track list, and displays the golf ball in the form of a two-dimensional trajectory (curve 1 to curve 3) on a two-dimensional plane. a track list, extracting a final trajectory of the golf ball from the two-dimensional trajectory according to a preset condition, and extracting coordinates of the individual golf balls on the trajectory of the extracted golf ball (coordinates of the golf ball candidate on the finally selected two-dimensional trajectory), In order to calculate the information on the physical characteristics of the mobile golf ball.

Figure 6 is a diagram showing a two-dimensional trajectory according to a list of golf ball tracks displayed on a two-dimensional plane via quadratic polynomial regression analysis.

Some of the two-dimensional trajectories extracted as described above may contain a variety of noises and are therefore incorrect. To this end, the incorrect two-dimensional trajectory is removed and the trajectory of the golf ball is finally selected.

For example, one of the two-dimensional trajectories obtained by the quadratic polynomial regression analysis connected to the ground as shown in Fig. 6 cannot be regarded as the trajectory of the golf ball (this is because when the golf ball is waved by the user to the golf ball When the golf club hits the golf ball, the golf ball does not fly through the ground, and the trajectory that is connected to the ground is because a part of the golf club is selected as a golf ball candidate. Therefore, the final trajectory of the golf ball excludes the above trajectory. Select.

If there are multiple two-dimensional trajectories left, even after a two-dimensional trajectory that cannot be considered a golf trajectory, the general physics that occurs when the golf ball moves is the final trajectory that can be applied to the golf ball.

For example, when the golf ball is struck toward the screen, the golf ball moving in a short section of about 5 m to 10 m moves at approximately uniform speed in the horizontal direction and at approximately uniform acceleration in the vertical direction. As a result, a two-dimensional trajectory conforming to this movement rule can be selected as the final trajectory of the golf ball.

Hereinafter, the configuration and processing of the golf club image processing apparatus shown in Fig. 3 will be described with reference to Figs. 7 to 14 .

As shown in FIG. 3, the golf club image processing apparatus may include a pre-processing apparatus 91, a first processing apparatus 92, and a second processing apparatus 93.

The pre-processing device 91 performs a variety of image processing to easily extract and process images of the golf club from the images received by the ball sensing device 70 or the golf ball image processing device 80. The image processed by the pre-processing device 91 will be described below with reference to FIGS. 7 to 10.

The first processing device 92 extracts and aggregates the differences between the individual frame images to extract a plurality of valid moving objects, and extracts a plurality of matching straight lines from the extracted effective objects. The second processing means 93 calculates the trajectory of a particular portion of each matching straight line as a movement trajectory of the golf club head, thereby estimating the rotation of the moving golf ball.

More specifically, the first processing device 92 can include an object extracting device 92a and a line matching device 92b, wherein the object extracting device 92a is configured to extract and gather a certain frame image and another with respect to all the frame images to be processed. The difference between the image of one frame, while extracting a portion of the golf club as an effective object, and the line matching device 92b is used to match the straight line with respect to the extracted effective object.

The second processing device 93 may include a trajectory calculating device 93a for checking a coordinate movement of a specific portion of the straight line matched by the line matching device 92b, and a rotation estimating device 93b for calculating a movement trajectory, and the rotation estimating device 93b is used to determine the calculated movement trajectory as a movement trajectory of the golf club head for extracting the moving direction of the golf club head from the movement trajectory and to thereby estimate the rotation of the golf ball.

The image processing performed by the first processing device 92 and the second processing device 93 will be described with reference to Figs. 11 to 14 .

First, the image processing performed by the preprocessing apparatus of the golf club image processing apparatus will be described with reference to Figs. 7 to 10 .

The golf club image processing device processes a plurality of frame images before and after the shot, The trajectory of the golf club head is counted, and thus the rotation of the golf ball is estimated. The number of frame images can be changed depending on the settings. In the seventh and subsequent figures, the image processing is performed using five frame images before and after the shot, that is, five images; however, the present invention is not limited thereto. Can handle less than five frame images or more than five frame images.

The image of the first to fifth frames shown in Fig. 7 is obtained by the camera of the photographing unit to obtain the golf ball hit by the user. Get five images before and after the shot.

The images from the first frame to the fifth frame shown in Fig. 7 are the source images 301 to 305 obtained by the camera. The images include a portion of the user's body (represented by the body), a golf club (represented by the golf club), and a ball striking pad (represented by the ball striking pad).

The main purpose of the golf club image processing apparatus is to extract only the golf club portion from the source images 301 to 305 of the individual frames.

To this end, the background image BI is provided as shown in Fig. 7, and the difference operation is performed between the source images 301 to 305 of the individual frames and the background image BI. Preferably, the background image BI is obtained at a different time than when the individual frames were obtained.

The image of the first frame to the fifth frame shown in FIG. 8 is operated by the difference between the source images 301 to 305 of the first frame to the fifth frame shown in FIG. 7 and the background image BI. obtain.

That is, the image 311 of the first frame shown in FIG. 8 can be obtained by the difference operation between the source image 301 of the first frame and the background image BI. In this way, the images 311 to 315 of the first to fifth frames shown in Fig. 8 are obtained. The images 311 to 315 of the first to fifth frames shown in Fig. 8 are used as difference images.

As shown in Fig. 8, the stationary portion (e.g., the background portion and the golfing mat portion) is removed from the difference image 311 of the No. 1 frame to the difference image 315 of the No. 5 frame, and the result leaves a moving portion ( For example, the user's body and a portion of the golf club; although the portion of the user's body does not move significantly, there is considerable movement so that a significant portion of the user's body cannot be removed via differential operations).

The extraction of the difference image described above is performed in the same manner as the image obtained by another camera of the photographing unit.

It can be seen from the difference images 311 to 315 of the individual frames shown in Fig. 8 if the image is The brightness is reduced during the difference operation, and the brightness of the image must be increased. That is, it is necessary to increase the brightness of the moving portion while keeping the brightness of the stationary portion unchanged.

An example of a method of increasing the brightness of the moving portion while keeping the brightness of the stationary portion constant is as shown in FIG.

That is, a preset number of pixel values (for a black and white image, the pixel value is expressed as a luminance value) of a plurality of pixels in the vicinity of each pixel of a certain image is aggregated to the corresponding pixel. For example, as shown in FIGS. 9(b) and 9(c), pixel values corresponding to peripheral pixels of the 3x3 array are aggregated to correct the luminance of the center pixel.

Assuming that the image shown in FIG. 9(a) includes 35 pixels ranging from the first pixel to the 35th pixel, the pixel values corresponding to the peripheral pixels of the 3×3 array are gathered with respect to each pixel of the image to correct the center. The brightness of the pixel.

That is, it is assumed that the pixel value of the 16th pixel or the 17th pixel is corrected as shown in FIG. 9(b) or FIG. 9(c), and is arranged in the 3x3 array at the 16th pixel or the 17th. The pixel values of a plurality of pixels in the vicinity of the pixel are aggregated to correct the pixel value of the 16th pixel (Fig. 9(b)) or the 17th pixel (Fig. 9(c)), so that the 16th pixel or the 17th The brightness of the number of pixels increases. In this way, a plurality of pixel values corresponding to the 3x3 array are aggregated with respect to all pixels ranging from the first pixel to the 35th pixel to correct the pixel values of the individual pixels.

Through the correction of each of the above pixels, the brightness of the moving portion can be increased while the pixel value of the black background portion is kept almost unchanged.

The corrected images 321 to 325 of the first to fifth frames shown in FIG. 10 are corrected by the pixel values corresponding to the plurality of peripheral pixels of the 3x3 array as shown in FIG. 9 to the opposite of FIG. Obtained from the central pixels of the difference images 311 to 315.

It can be seen from the corrected images 321 to 325 of the first frame to the fifth frame shown in Fig. 10 that the brightness of the moving portion, that is, the user's body and the golf club portion, has increased.

The correction of each pixel of each of the above images is performed with respect to an image obtained by the other camera of the photographing unit in the same manner.

However, the corrected images 321 to 325 shown in Fig. 10 still contain noise, such as a part of the user's body. The effective portion of the image to be processed by the golf club image processing device, i.e., the active object, is the golf club portion. The rest is noise.

To do this, you must remove unnecessary parts, such as a part of the user's body, and extract only An effective item, that is, a golf club part.

As shown in Fig. 11, the first object extraction image (331) to the fifth object extraction image (335) are corrected images 321 to 325 from the first frame to the fifth frame shown in Fig. 10. Obtained by extracting valid objects OB1 to OB5.

That is, the No. 1 object extracting image (331) is generated by extracting and merging the corrected image 321 of the No. 1 frame and the corrected images 322 to 325 of the remaining frames, and the No. 2 object extracting image (332) is extracted. The corrected image 322 of the No. 2 frame and the corrected image 321 of the remaining frames and 323 to 325 are generated. In this manner, the No. 3 object extracted image (333) is generated to the No. 5 object extracted image (335).

As shown in Fig. 11, the individual object extracted images 331 to 335 are obtained by extracting the effective objects OB1 to OB5 from the moving portions of the individual frame images.

That is, the difference between the individual frame images to be processed is gathered as described above, and as a result, the image of a portion of the body of the smaller user is preferably removed, and the portion that is significantly larger is moved. For example, a golf club is extracted as an effective item.

The above-mentioned effective drawing of the object is performed in the same manner as the image obtained by the other camera of the photographing unit.

At the same time, as shown in Fig. 11, the effective objects OB1 to OB5 extracted by the difference between the images of the corresponding frames and the images of the remaining frames are unclear, and it is difficult to treat the effective objects OB1 to OB5. The extent of the golf club is thus insufficient to confirm the movement of the golf club.

For this purpose, the straight strips must be matched with respect to the active objects OB1 to OB5 so that the movement of the golf club can be more clearly confirmed.

Although the individual effective objects OB1 to OB5 must be displayed in the form of straight bars like golf clubs, the individual effective objects OB1 to OB5 are not displayed in the form of clear straight lines during image processing. For this reason, the matching of straight lines is necessary.

The random sample consensus (RANSAC) algorithm can be used as an example of a straight line matching method with respect to valid objects OB1 to OB5.

That is, the pixels constituting the individual effective objects OB1 to OB5 may have information on the golf club (inner layer) or incorrect information (outer layer) not related to the golf club. A plurality of factors having information on a golf club are relative to a group of pixels including pixels having such information, via The preset mathematical model is repeated and predicted and corrected to match the straight line.

Fig. 12 is a view showing an image obtained by the result of straight line matching (application of the RANSAC algorithm) of the objects OB1 to OB5 which are effective on the individual object extracted images 331 to 335.

That is, in Fig. 12, the line L1 is obtained by line matching of the effective object OB1, the line L2 is obtained by line matching of the effective object OB2, and the line L3 is obtained by line matching of the effective object OB3. Line L4 is obtained by line matching of the effective object OB4, and line L5 is obtained by line matching of the effective object OB5.

The matching of the above straight line is performed with respect to the image obtained by the other camera of the photographing unit in the same manner.

Meanwhile, as described above, when the matching line is extracted with respect to the image obtained by the plurality of cameras constituting the photographic unit, it is necessary to check along which trajectory the golf club head is along with the result of the extraction. Move in 3D space.

Fig. 13 is a view showing the principle of matching the images obtained by the first camera 51 and the second camera 52 to calculate the movement trajectory of the golf club head.

In Fig. 13, it is assumed that the image obtained by the first camera 51 is the image I51, the matching line of the image I51 is the matching line L51, and the matching line L51 is matched to the image I51 of the first camera 51 in the three-dimensional space. The plane is the plane DP. Further, it is assumed that the image obtained by the second camera 52 is the image I52, and the straight line matching the line L52 on the image I52 corresponding to the second camera 52 in the three-dimensional space is the straight line DL.

As shown in FIG. 13, a corresponding point P51 on the matching line L51 is found, and the matching line L51 is an intersection corresponding to the three-dimensional spatial plane DP and the three-dimensional spatial straight line DL, wherein the plane DP corresponds to the first camera. The matching line L51 in the image I51, and the straight line DL is the matching line L52 in the image I52 corresponding to the second camera.

As a result, the point P52 on the matching line L52 and the corresponding point P51 on the matching line L51 constitute a pair of corresponding points.

Such corresponding points are examined, and a portion corresponding to the golf club head, that is, the tail of each matching line, is found from the individual matching lines shown in FIG.

If the tail of each matching line, that is, the head of the golf club head, is found, the coordinate information of the head in the three-dimensional space can be extracted, and the movement trajectory of the golf club head can be calculated therefrom.

After calculating the movement trajectory of the golf club head as described above, it is necessary to estimate the moving gaer The rotation of the ball.

Fig. 14 is a schematic view showing an example of the principle of estimating the rotation of the moving golf ball from the movement trajectory of the golf club head. Fig. 14 shows the moving position of the golf club head CH of each frame, and the movement trajectory TR of the golf club head according to this. In Fig. 14, the symbol B indicates a golf ball.

It is not necessary to check the movement trajectory of the golf club head from the beginning to the end to estimate the rotation of the moving golf ball. That is, as shown in Fig. 14, a region 400 of a predetermined size can be set in the vicinity of the golf ball B, and the movement of the golf club head CH in the effective region 400 can be observed.

If the size of the active area 400 is too small, a quick check is possible, but the accuracy will decrease. On the other hand, if the size of the effective area 400 is too large, the accuracy is improved, but the inspection speed is lowered. Therefore, it is preferable that the size of the effective area 400 is set to an appropriate size in consideration of the above factors.

The moving direction HD of the golf club head CH can be extracted from the movement trajectory TR of the golf club head CH in the effective area 400. It is preferable to extract the moving direction HD of the golf club head CH at the time of hitting the ball.

As a result, after the golf ball is struck, it is easy to estimate how much side or back rotation the golf ball has from the moving direction HD of the golf club head CH.

For convenience of explanation, Fig. 14 shows the moving direction of the golf club head on a plane. In fact, the direction of movement of the golf club head in three-dimensional space can be extracted, and thus the side or back spin can be easily estimated.

Hereinafter, a sensing method for a sensing device of a virtual golf simulation device according to an embodiment of the present invention will be described with reference to FIGS. 15 and 16.

First, reference will be made to Fig. 15 to illustrate the sensing method. When the sensing device is operated, the sensing device continuously collects images of the preset projection range in which the shot is made (S10). The collection process can be performed by the gripper 65 shown in Fig. 3.

The collected image is transmitted to the shot sensing device 70 (see FIG. 3), and the received image is analyzed to sense whether the user has hit the golf ball (S11).

When the result of the image analysis performed by the shot sensing device 70 is that the user has sensed that the golf ball has been tapped, the hitting sensing device 70 checks that the user has sensed that the golf ball has been tapped. The plurality of front frame images are used to check the actual hitting time when the golf club hits the golf ball (S20).

Next, the shot sensing device 70 stores a plurality of frame images (including images at the time of hitting) before hitting the ball (S21).

Next, the ball sensing device 70 sequentially collects a plurality of frame images after the hitting from the gripper 65, and immediately transmits the collected frame images to the golf ball image processing device (S22).

The golf ball image processing apparatus performs golf ball processing with respect to the received image (S30). If the information such as the speed, direction and elevation angle of the golf ball is calculated as a result of the golf ball processing, the golf ball image processing device transmits the calculated information to the simulator without waiting for the result processed by the golf club image processing device. (S50).

The golf club image processing apparatus receives the image stored in step S21 and the image processed by the golf ball image processing apparatus, and performs golf club processing (S40). If the rotation information of the golf ball is calculated as a result of the golf club processing, the golf club image processing apparatus transmits the calculated rotation information of the golf ball to the simulator (S50).

The simulator uses the information to start the simulation based on the golf processing results, and the golf club processing results are used to reflect the golf ball information during the simulation to complete the simulation.

Hereinafter, the golf club process shown in Fig. 15 will be explained with reference to Fig. 16.

When the golf club process is started, a plurality of frame images before and after the shot are received from the hitting sensing device and the golf ball image processing device (S41).

In order to remove the still portion, that is, the background image, from the collected image, the difference image is extracted via a difference operation with respect to the image collected at another time (S42).

In order to increase the brightness of the moving portion in the image extracted by the difference operation, the pixel values of the peripheral pixels of each pixel are aggregated with respect to all the pixels of each image, thereby generating the image previously described with reference to FIG. 9 (S43) And thus its detailed description is omitted.

After the image whose brightness has been corrected is generated, the difference between the image of one frame and the image of the other frame is extracted and aggregated, thereby extracting valid objects (portions including golf clubs) from each frame image. . This processing is performed with respect to the frame image to be processed (S44).

The valid object is extracted from each frame image via step S44. The straight line bar is matched with respect to the extracted effective object (S45).

Then, a plurality of corresponding points of the same frame image in the three-dimensional space with respect to the plurality of cameras Checked (S46) to find a specific portion of the matching line (preferably, the portion corresponding to the golf club head, which may be part of the matching line), thereby identifying the portion of the matching line as the golf club head, and checking The coordinates of the portion are moved to calculate the movement trajectory of the golf club head (S47).

Next, the moving direction of the golf club head is calculated from the calculated movement trajectory of the golf club head (S48), and the rotation of the moving golf ball is thereby estimated (S49).

It is apparent from the above description that in the sensing device and the sensing method for the virtual golf simulation device according to the present invention, the image obtained by the inexpensive camera having low resolution and low speed is analyzed to be very accurate. Extracting information on the physical characteristics of the moving golf ball, such as speed, direction and elevation angle, especially where the moving trajectory of the golf club is calculated very accurately from the obtained image to accurately estimate the rotation of the golf ball, and The golf simulation reflects the estimated rotation of the golf ball, thereby constructing a virtual golf simulation device with high accuracy and reliability at a low cost, and further improving the realism of the virtual golf.

The above is only a preferred embodiment for explaining the present invention, and is not intended to limit the present invention in any way, and any modifications or alterations to the present invention made in the spirit of the same invention. All should still be included in the scope of the intention of the present invention.

1‧‧‧ Simulator

2‧‧‧ Golf Room

3‧‧‧ screen

10‧‧‧Database

20‧‧‧Data Processing Department

30‧‧‧Image output department

50‧‧‧Photographic unit

51‧‧‧Photo Camera

52‧‧‧Photo Camera

60‧‧‧Sensing processing unit

65‧‧‧ Grabber

70‧‧‧Ball sensing device

73‧‧‧ Conversion device

80‧‧‧Golf image processing device

90‧‧‧Golf club image processing device

91‧‧‧Pre-processing device

92‧‧‧First treatment unit

92a‧‧‧Object extraction device

92b‧‧‧Line matching device

93‧‧‧Second treatment unit

93a‧‧‧Track computing device

93b‧‧‧Rotation Estimation Device

100‧‧‧Battering board

110‧‧‧ hitting table

120‧‧‧Battering pad

121‧‧‧ fairway area

122‧‧‧Long Grass Area

123‧‧‧Bankeng District

200‧‧‧ source image

220‧‧‧Ball area

301~305‧‧‧ source image

311~315‧‧‧ images

321~325‧‧‧Correct image

331~335‧‧‧ Object extraction image

400‧‧‧effective area

B‧‧‧ Golf

BI‧‧‧ background image

CH‧‧‧Golf club head

DL‧‧‧Lines

DP‧‧‧ plane

HD‧‧‧ moving direction

I51‧‧‧ images

I52‧‧‧ images

L1~L5‧‧‧(match) lines

M‧‧‧ controller

OB1~OB5‧‧‧ valid objects

P51‧‧ points

P52‧‧ points

R‧‧‧Sensing area

S‧‧‧Sensing device

S10~S50‧‧‧Steps

S41~S49‧‧‧Steps

TR‧‧‧Moving track

The above and other objects, features and other advantages of the present invention will become more apparent from 2 is a block diagram showing the structure of a virtual golf simulation device according to an embodiment of the present invention; and FIG. 3 is a view showing the detailed structure of the sensing device of FIG. 2 and the functions of components constituting the sensing device. Fig. 4(a) is a schematic view showing an image obtained by the photographing unit, and Fig. 4(b) is a view showing a sensing area separated from the image shown in Fig. 4(a), and Fig. 4 (Fig. 4) c) is a schematic diagram showing whether the sensing golf ball has been tapped via the separate sensing area; FIG. 5 is a view showing the golfer candidates in the individual frame extracted by the golf ball image processing apparatus according to the embodiment of the present invention Schematic; Figure 6 is a diagram showing the golf ball candidates in combination with the individual frames to display golf trajectory candidates Schematic diagram of a two-dimensional plane; FIG. 7 is a schematic diagram showing a plurality of frame images collected by a golf club image processing apparatus of a sensing device and a background image for difference operation according to an embodiment of the present invention; 8 is a schematic diagram showing an image obtained by the difference operation between the individual frame image and the background image shown in FIG. 7; FIG. 9 is a schematic diagram showing the pixel compensation method, wherein FIG. 9(a) is a display A schematic view of the image, and FIGS. 9(b) and 9(c) are schematic views showing portions separated from the image, and each portion corresponds to a 3×3 array; FIG. 10 is a view according to FIG. A schematic diagram of an image obtained by pixel compensation of an individual frame image shown in FIG. 8; FIG. 11 is a view showing an image obtained by extracting an effective object from the image of the individual frame shown in FIG. FIG. 12 is a schematic diagram showing an image in which a plurality of straight lines are matched with respect to individual objects shown in FIG. 11; and FIG. 13 is a schematic view showing a principle of three-dimensional matching with respect to individual matching lines shown in FIG. 12; Figure 14 Is a schematic diagram showing the rotation of the golf ball estimated by the movement trajectory of the golf club head calculated according to the principle shown in FIG. 13; FIG. 15 is a view showing the sensing method for the virtual golf simulation according to the embodiment of the present invention. The flowchart; and Fig. 16 is a more detailed flowchart showing the golf club processing steps shown in Fig. 15.

1‧‧‧ Simulator

50‧‧‧Photographic unit

65‧‧‧ Grabber

70‧‧‧Ball sensing device

73‧‧‧ Conversion device

80‧‧‧Golf image processing device

90‧‧‧Golf club image processing device

91‧‧‧Pre-processing device

92‧‧‧First treatment unit

92a‧‧‧Object extraction device

92b‧‧‧Line matching device

93‧‧‧Second treatment unit

93a‧‧‧Track computing device

93b‧‧‧Rotation Estimation Device

Claims (20)

A sensing device for a virtual golf simulation device, comprising: a photographic unit for obtaining a plurality of frame images of a golf ball that a user swings a golf club to strike a golf ball; and a sensing process The unit includes: a golf ball image processing device, configured to extract a golf ball from each frame image of the frame image to obtain a three-dimensional coordinate of the golf ball, and a golf club image processing device, including: a processing device for extracting and aggregating differences between individual frame images to extract valid moving objects and matching straight lines with respect to the extracted effective objects; and a second processing device for calculating matching lines The trajectory of the upper part is used as the movement trajectory of the golf club head to calculate the information on the physical characteristics of the moving golf ball. A sensing device for a virtual golf simulation device according to claim 1, wherein the golf club image processing device is configured to receive an image from the golf ball image processing device and process the received image, The golf ball extracted from each image processed by the golf image processing device has been removed from the received image. The sensing device for a virtual golf simulation device according to claim 1, further comprising a ball striking sensing device for processing each image received from the image capturing unit to sense the hitting ball Whether it has been performed by the user to confirm the hitting time. A sensing device for a virtual golf simulation device according to claim 3, wherein the golf ball image processing device is configured to receive a plurality of frame images from the ball striking sensing device after hitting the ball, And processing the received images, and the golf club image processing device is configured to receive a plurality of frame images from the ball sensing device prior to hitting the ball, and the golf image processing device processes the same Image and process the received images. A sensing device for a virtual golf simulation device according to claim 1 of the patent application scope, The sensing processing unit is configured to calculate information on a plurality of physical characteristics of the mobile golf ball using the three-dimensional coordinates of the golf ball extracted by the golf ball image processing device, and the physical characteristics include speed, direction, and height And a moving image of the golf club head calculated from the golf club image processing apparatus, and calculating information on a plurality of physical characteristics including the rotation of the moving golf ball. A sensing device for a virtual golf simulation device according to claim 1, wherein the golf club image processing device further comprises a rotation estimating device for setting a portion of the golf ball in a stopped state A predetermined size effective area for checking the movement trajectory of the golf club head in the effective area to extract the moving direction of the golf club head, and for estimating the rotation of the golf ball therefrom. A sensing device for a virtual golf simulation device, comprising: a photographic unit for obtaining a plurality of frame images of a golf ball that a user swings a golf club to strike a golf ball; and a sensing process The unit includes a golf ball image processing device and a golf club image processing device for extracting a golf ball from each frame image of the frame images by two-dimensional analysis, thereby extracting a golf ball a three-dimensional coordinate of the golf ball on the trajectory, and the golf club image processing device is configured to extract a matching line from the moving object from the obtained image of each frame through two-dimensional analysis, and use the matching line through the matching line The three-dimensional matching extracts the movement trajectory of the golf club head to calculate information on multiple physical characteristics of the mobile golf ball. The sensing device for a virtual golf simulation device according to claim 7, wherein the photographic unit comprises a plurality of stereoscopic driving cameras for obtaining a plurality of frame images, and the golf ball The image processing device is configured to find a golf ball candidate for each image of the images obtained from the cameras, thereby extracting a three-dimensional coordinate of the golf candidate, and polynomial regression analysis of the golf candidate on the two-dimensional plane And extract and analyze golf candidates A two-dimensional trajectory for extracting the trajectory of the golf ball and converting the golf ball coordinates on the trajectory of the extracted golf ball into three-dimensional coordinates. The sensing device for a virtual golf simulation device according to claim 7, wherein the photographic unit comprises a plurality of stereoscopic driving cameras for obtaining a plurality of frame images, and the golf ball The bar image processing device includes: a first processing device for extracting and aggregating differences between the individual frame images obtained by the cameras to extract valid moving objects and relative to the extracted effective objects Matching a straight line; and a second processing device for checking a plurality of corresponding points of the same frame image in the three-dimensional space with respect to the cameras, thereby calculating a three-dimensional trajectory of a specific portion on the matching line. A sensing device for a virtual golf simulation device, comprising: a photographic unit for obtaining a plurality of frame images of a golf ball that a user swings a golf club to strike a golf ball; and a sensing process The unit includes a first processing device and a second processing device for extracting and aggregating differences between the individual frame images to extract valid moving objects and relative to the extracted valid objects The matching line is matched, and the second processing device is used to calculate a trajectory of a specific portion on the matching line as a moving trajectory of the golf club head. The sensing device for a virtual golf simulation device according to claim 10, wherein the sensing processing unit further comprises a pre-processing device for removing one image from each obtained image via a difference operation. a still portion, and a predetermined number of pixel values of the plurality of peripheral pixels are aggregated with respect to all pixels on the difference operation image to generate an image, and the first processing device and the second processing device process using the pre-processing device The image is image processed. A sensing device for a virtual golf simulation device according to claim 10, wherein the first processing device comprises: An object extracting device for extracting and merging a difference between a certain frame image and another frame image with respect to all frame images to be processed, thereby extracting a portion corresponding to the golf club as effective An object; and a line matching device for matching a line of the object that is valid relative to the extracted object. A sensing device for a virtual golf simulation device according to claim 12, wherein the second processing device comprises: a trajectory calculating device for checking a coordinate movement of a specific portion on the matching line to calculate a movement trajectory And a rotation estimating device for determining the calculated movement trajectory as a movement trajectory of the golf club head for extracting a moving direction of the golf club head from the movement trajectory, and thereby estimating the rotation of the golf ball. A sensing device for a virtual golf simulation device, comprising: a photographic unit comprising a plurality of cameras for obtaining a plurality of frames of a golf ball that a user swings a golf club to strike a golf ball And a sensing processing unit for extracting a matching line with respect to an effective moving object through the two-dimensional analysis of the images obtained from the individual cameras, and for using the matching line that has been two-dimensionally analyzed The three-dimensional matching extracts the trajectory of a specific part on the matching line as the moving trajectory of the golf club head. A sensing method for a sensing device of a virtual golf simulation device, comprising: continuously collecting a plurality of images of a preset shooting range by a shot; analyzing the obtained images to sense whether the user has knocked Shooting a golf ball; extracting and analyzing the golf ball from a plurality of frame images after the hitting time that the golf ball has been tapped by the user, thereby calculating information on a plurality of physical characteristics of the moving golf ball (golf processing step); And extracting and analyzing a movement trajectory of the golf club head to estimate a rotation of the moving golf ball (golf club processing step), wherein the golf club processing step includes: Extracting and aggregating differences between individual frame images to extract valid moving objects; matching straight lines relative to the extracted objects; and calculating the trajectory of a particular portion of the matching line as the movement trajectory of the golf club head . A sensing method for a sensing device for a virtual golf simulation device according to claim 15, wherein the golf ball processing step comprises: extracting a golf ball candidate for each frame with respect to the frame images Converting the three-dimensional coordinates of the golf candidate into two-dimensional coordinates to extract at least one two-dimensional trajectory obtained by the golf ball candidates connected to the individual frames; and analyzing the two-dimensional trajectory to extract the golf ball The trajectory is borrowed to obtain the three-dimensional coordinates of the golf ball on the trajectory of the golf ball. A sensing method for a sensing device of a virtual golf simulation device, comprising: obtaining a plurality of frame images of a golf ball that a user swings a golf club to strike a golf ball; and extracting each image obtained from the image An effective moving object; matching a straight line with respect to the extracted effective object; calculating a trajectory of a specific portion on the matching line; and determining the calculated trajectory as a moving trajectory of the golf club head, thereby estimating the rotation of the moving golf ball. A sensing method for a sensing device for a virtual golf simulation device according to claim 17, wherein the step of extracting the effective moving object comprises: removing a still from each of the obtained images via a difference operation Portioning a predetermined number of pixel values of a plurality of surrounding pixels to generate an image relative to all pixels on the difference operation image; and extracting and aggregating a certain frame image and other with respect to all frame images to be processed The difference between the image of a frame to extract a portion corresponding to the golf club as an effective object. Sensing device for virtual golf simulation device according to claim 17 of the patent application scope The sensing method, wherein the obtaining the image comprises obtaining the frame images via a plurality of stereoscopic driving cameras; and the step of calculating the trajectory of the specific portion comprises: checking the three-dimensional images relative to the cameras A plurality of corresponding points of the same frame image in the space; and finding the tail of the matching line, identifying the tail as the golf club head, and checking the coordinate movement of the golf club head to calculate its three-dimensional trajectory. A sensing method for a sensing device for a virtual golf simulation device according to claim 17, wherein the step of estimating the rotation of the golf ball comprises: setting a preset in the vicinity of a position of the golf ball in a stopped state An effective area of the size; checking the movement trajectory of the golf club head in the effective area to extract the moving direction of the golf club head; and estimating the rotation of the moving golf ball from the moving direction of the golf club head.