WO2008052605A1

WO2008052605A1 - Data processing system and golf diagnosis apparatus

Info

Publication number: WO2008052605A1
Application number: PCT/EP2007/006107
Authority: WO
Inventors: Kristian Hohla; Georg Korn; Ernst Hegels; Yaroslav Cherednikov
Original assignee: Friend For Golfers Gmbh
Priority date: 2006-10-31
Filing date: 2007-07-10
Publication date: 2008-05-08

Abstract

A data processing device comprising an image processing unit adapted for determining a position of at least one marker on at least two images of a movable object, a mirror plane determining unit adapted for determining a plurality of mirror planes each mapping one of the at least one marker from one of the at least two images to another one of the at least two images of the movable object, and a spin determining unit adapted for determining a spin direction of the movable object using at least one straight line obtained by intersecting at least two of the plurality of mirror planes.

Description

Data processing system and golf diagnosis apparatus

This application claims the benefit of the filing date of United

States Provisional Patent Application No. 60/863,730 filed October 31, 2006, of United States Provisional Application No. 60/885,850 filed January 19, 2007, and of European Patent Application No. 07001184.6 filed January 19, 2007 the disclosure of which is hereby incorporated herein by reference.

The invention relates to a data processing device. Moreover, the invention relates to a golf diagnosis apparatus. The invention further relates to a data processing method. Moreover, the invention relates to a program element.

Further, the invention relates to a computer-readable medium.

US 2005/0026710 Al discloses a video image acquisition apparatus having one or multiple digital cameras taking images of a flying golf ball created by at least two flashes or strobes of light on continuous video mode at a predetermined frame rate. Each image frame is then subtracted from the background and compared to determine the existence of the ball image in flight. Furthermore, another video image acquisition apparatus is also disclosed in US 2005/0026710 Al that consists of at least two video cameras taking images of flying golf balls created by at least two flashes or strobes of light at predetermined time intervals. The apparatus then applies triangulate calculation of the two camera images to determine the exact physical locations of the flying golf balls in space at a given time of flight. US 2004/0030527 discloses a system for capturing and thereafter analyzing the images of a golf ball in a manner that permits various parameters associated with a golf shot, including the backspin and sidespiπ of the golf ball, to be reliably determined. Further, the system may be configured to separately calibrate each image that is captured in a timely manner.

However, conventional golf diagnosis systems suffer from the fact that they are inappropriate for an accurate determination of a spin of a golf ball hit by a golf club.

It is an object of the invention to provide an accurate spin determination system. In order to achieve the object defined above, a data processing device, a golf diagnosis apparatus, a method of data processing, a program element and a computer readable medium according to the independent claims are provided.

According to an exemplary embodiment of the invention, a data processing device is provided comprising an image processing unit adapted for determining a position of at least one marker on (each of) at least two images of a movable object (i.e., the marker may be visible on each of the images of the movable object), a mirror plane determining unit adapted for determining a plurality of mirror planes each mapping one of the at least one marker from one of the at least two images to another one of the at least two images of the movable object, and a spin determining unit adapted for determining a spin direction of the movable object using at least one straight line obtained by intersecting at least two of the plurality of mirror planes. According to another exemplary embodiment of the invention, a golf diagnosis apparatus for evaluating a performance, particularly a stroke, of a golf player is provided, the golf diagnosis apparatus comprising a data processing device having the above mentioned features and being adapted to process images of a golf ball as the movable object. According to another exemplary embodiment of the invention, a method of data processing is provided, the method comprising determining a position of at least one marker on at least two images of a movable object, determining a plurality of mirror planes each mapping one of the at least one marker from one of the at least two images to another one of the at least two images of the movable object, and determining a spin direction of the movable object using at least one straight line obtained by intersecting at least two of the plurality of mirror planes. According to still another exemplary embodiment of the invention, a program element is provided, which, when being executed by a processor, is adapted to control or carry out a method of data processing having the above mentioned features.

According to yet another exemplary embodiment of the invention, a computer-readable medium is provided, in which a computer program is stored which, when being executed by a processor, is adapted to control or carry out a method of data processing having the above mentioned features.

The data processing scheme according to embodiments of the invention can be realized by a computer program, that is by software, or by using one or more special electronic optimization circuits, that is in hardware, or in hybrid form, that is by means of software components and hardware components.

In the context of this application, the term "movable object" may particularly denote a physical structure which is adapted, designed or configured to be operated in a fluidic (particularly a gas, but possibly also a liquid) environment in which it shall move, for instance fly. Examples for movable objects which may also rest in a static operation state are sports devices such as balls, particularly golf balls, tennis balls, table tennis balls, squash balls, or soccer balls. According to embodiments of the invention, it is possible to map one marker on the movable object related to an image of the movable object at a first point of time to the position of the marker on the movable object at a second point of time. For this purpose, the two images of the movable object including the marker may be projected onto one another in a manner that the centers of the movable object are coincident on the projected images, so that the marker on the image at the first point of time is mapped, by the mirror plane, onto the marker on the movable object at the second point of time. The term "spin vector" may particularly denote a vector indicative of direction and absolute value of the angular momentum of the movable object, that is to say a vector having a specific direction and a specific absolute value. The spin vector of a hit golf ball is indicative of a back spin transferred from a golf club to a golf ball when the latter is hit by the golf club.

The term "straight line" may particularly denote a direction/line which is obtained by an intersection of the individual mirror planes determined in accordance with a method of an exemplary embodiment of the invention. The term "performance" of a golf player may particularly denote any action a golf player takes before, during or after carrying out a stroke or a putt. This may particularly include the behavior directly before the stroke, for instance when the golf player stands in front of the tee and concentrates before carrying out the stroke. It may particularly include the behavior during the stroke, for instance when the golf player swings the golf club and hits the golf ball. It may particularly include the behavior after the stroke, for instance when the golf ball has left the tee/golf club and flies or rolls in the direction of the goal.

The term "stroke" may particularly denote the entire procedure or a part of the procedure including a swing with the golf club, a hit between golf club and golf ball, and the flight of the golf ball until the ball rests. A stroke may be at least a part of the performance. Thus, a golf stroke may denote a stroke which makes the golf ball fly (for instance when launching the ball from a tee), and may also denote a stroke which makes the golf ball roll (for instance when putting the ball towards a hole on a green). Therefore, a stroke may cover any motion of the golf ball with a distance of hundreds of meters to several centimeters.

The term "stroke distance" may particularly denote the distance between a resting position of the golf ball before a stroke and after the stroke.

The term "hit" may particularly denote the short time interval in which an interaction between the golf club and the golf ball occurs.

The term "electromagnetic radiation" may particularly light, but other wavelengths (for instance infrared and/or UV light) are possible as well.

The term "golf diagnosis apparatus" may particularly denote an apparatus which may monitor the performance of a golf player and may carry out calculations in correspondence with this performance. Also golf simulators may be covered by the term "golf diagnosis apparatus". For instance, such a golf diagnosis apparatus may comprise one or more cameras making one or more pictures of a golf ball and/or a golf club and/or a golf player in order to derive therefrom information allowing to perform a diagnosis of a golf stroke.

For instance, a stroboscope may define different points of time at which an image is taken, and the individual images may be evaluated using image recognition methods so as to analyze a stroke of a golf player. For instance, such a golf diagnosis apparatus may calculate parameters like velocity, angle, acceleration, spin, stroke distance, etc. in accordance with a stroke. Such a system may be implemented also in combination with a self-adaptive golf analysis feature, allowing to determine which body positions, or other stroke parameters statistically yield good results, and which not. Thus, such a golf diagnosis system may provide a golfer with suggestions as to how to improve the performance or provide information which parameters have been successful in the past.

In the context of such a golf diagnosis apparatus, a golfer may position a golf ball on the tee, may select a golf club and may carry out a stroke. In the vicinity of the tee (for instance at a distance of 40 cm from the golf diagnosis apparatus), the user (for instance positioned at a distance of 120 cm from the golf diagnosis apparatus) may position the golf diagnosis apparatus which may comprise a camera or another image acquisition device so that one or more images can be captured before, during and/or after hitting the ball. Such images may then be evaluated, with respect to ball, golf club, and/or body position of the golfer so as to derive parameters allowing to perform a diagnosis of a stroke so as to evaluate the quality of the stroke.

In the context of golf diagnosis, it may be advantageous to determine the spin of the golf ball because this may have a significant influence on the motion characteristic of the golf ball. According to an exemplary embodiment of the invention, several images of a movable object captured during the motion of the movable object are evaluated regarding a position of one or more markers provided on the movable object. Several images of the movable object at different points of time may be projected to one another (or overlaid) so that a center of gravity of the (essentially homogeneous) movable object (for instance a center of a ball) coincide in the overlaid images. Then, mirror planes are determined mapping each marker at different points of time onto one another. These planes all include the center of the ball. Intersecting the plurality of obtained mirror planes allows to derive the direction of the spin of the golf ball hit by a human golf player as the intersection line. The direction of the spin may thus be determined on the basis of the assumption that no external force acts on the golf ball during the flight, which is a proper approximation in many cases. Furthermore, after having determined the spin direction, the rotation angle of the marker on the movable object is determinable on the basis of an analysis of the positions of the markers on the golf ball at the different points of time. This may be used to determine the absolute value of the spin. Thus, a combination of these two procedures allows to determine the three-dimensional characteristics of the spin regarding absolute value and direction.

In the frame of reference (coordinate system) of the ball, the mirror plane goes through the center of the ball. In case of a homogeneous movable object such as a ball, the spin vector goes through the center of the ball. At least one marker may be evaluated on at least three photos of the movable object, or at least two markers may be evaluated on at least two photos of the movable object so as to determine at least two mirror planes. Multiple mirror planes may also be determined by analyzing different portions of one extended marker. An intersection of these mirror planes then provides for the spin direction. Particularly, it may be advantageous to provide four distinguishable

(by colour, shape, etc.) markers on a golf ball, for instance four markers on corners of a rectangle on the golf ball. Providing four markers has turned out to allow for a reliable estimation of the spin, since it may happen that on one or more images of the movable object, one or more of the markers may be hidden or may be invisible since they are on the back surface of the golf ball. Thus, one or more of the markers may rotate out of the images due to the angular momentum or may be invisible due to reflections or other disturbances so that the redundant provision of four markers allows for a reliable determination of the spin vector. In case that all four markers are visible on an image, the spin vector determination procedure is over-determined allowing for a more accurate determination of the spin vector, by implementing averaging procedures or statistical procedures.

Typically, the golf ball may be imaged at a distance of approximately 10 cm directly after launching the ball. At typical golf ball launch velocities, two flashes illuminating the golf ball may be generated with a time distance of approximately 1 ms. For typical spin values of a hit golf ball, the golf ball may perform 1/4 rotations in the first 10 cm. Such a proper adjustment of the flash times may allow to rule out that uncertainties are involved due to multiple rotations of the ball on the different images. Furthermore, the system may be programmed with the a priori information that a golf ball is usually hit by a golf club with a back spin, since it is almost impossible in golf play to provide the ball with a top spin. Therefore, the rotation direction of the golf ball may be used which may be a valuable additional information during the determination of the golf ball spin.

In the case of elongated patterns, for instance an inscription or a complex geometrical structure, on a golf ball, it is possible, implementing image processing routines, to individually evaluate a plurality of portions of such an inscription separately. When determining the plurality of portions (for instance letters of an inscription), they may be non- distinguished at the beginning. Particularly, it may be unknown which portion is correlated with which other portion. Using statistical methods (such as a Hough transformation) it is possible to derive additional information which correlates the different portions thereby making the portions distinguishable. When a plurality of portions (for instance 100) are used for determining mirror planes which each other one of the portions, a huge number of mirror planes may be found (for instance approximately 10000). Only a small percentage of this number (for instance 100 out of 10000) of the mirror planes include meaningful information regarding spin direction. These meaningful mirror planes may be determined by a statistical analysis. For instance, one word can be separated into 100 components, for each of which the corresponding mirror plane may be determined. This may involve a huge number of intersection straight lines, statistical methods or methods using elements of artificial intelligence such as a Fuzzy algorithm may be implemented to determine the most frequent intersection straight line and thus the most likely spin direction.

In this context, a Hough transformation may be carried out. Such a Hough transformation may allow to determine points or positions of the largest occurrence or frequency. For instance, in the context of such a Hough transformation, each mirror plane may be considered as a set of concurrent lines going through the center of gravity of the movable object. If such a set of concurrent lines is determined for each of the mirror planes, the most frequently occurring straight line may be determined which may then be considered as the spin axis. Such a procedure may be similar to a procedure for determining great circles in the field of air traffic.

As an alternative to a Hough transformation, it may also be advantageous to implement Quaternion algebra for the estimation of mirror planes. Quaternion algebra may generalize complex numbers into four dimensions and may serve as a valuable tool for deriving the mirror planes and the spin vector thereof with reasonable computational burden and numerical effort. Although it may be sufficient to use information of a golf ball for determining its spin, it may be advantageous to also include information regarding the golf club shortly before and/or after the hit. This may allow to independently and complementarily derive spin information using conservation of spin, etc. The total spin of a rotating golf ball may be determined using spectroscopic photography using symmetry planes between the position of one or more points or markers on the surface of the ball at at least two points of time. Using a golf launch monitor such as the Golf Optimizer of the

Friend for Golfers GmbH, the parameters of the starting ball may be measured, and the trajectory of the ball may be determined on the basis of this information. For this purpose, the spin of the ball is an important parameter with which the determination of the trajectory is possible in an accurate manner. Using spectroscopic photography, the rotation of the golf ball may be determined at at least two points of time.

Using patterns or markers on the golf ball, an unambiguous assignment of positions on the golf ball surface, for instance by shape and/or colour, is possible. In this context, the following considerations may be performed:

The spatial coordinates of the found positions may be determined using methods of photogrammetry. Considering now two positions of an unambiguous identical position on the ball at different points of time, the rotation axis running through the center of the ball is aligned in the mirror plane of these two positions. When a third position of this portion on the ball is known or if two positions of another unique identical position are determined, several of these mirror planes may be known. These mirror planes intersect in one straight line, namely the rotational axis. Based on the rotation angle around this axis, the absolute value of the spin may be determined. It is also possible to determine the position of extended or elongated patterns (such as markers or imprints). This may allow a determination of colours and/or shapes.

In such a scenario, it is possible to map the mirror planes of all points which are assigned to the pattern in the first position of the ball with all points which are assigned to the pattern in the second position of the ball. What is obtained is a large number of intersection lines of these mirror planes. The desired rotation axis may thus be found by an analysis of the frequentness of the occurrence of the individual section lines. Using statistical methods, the rotation axis and subsequently the amount of the spin may be determined.

In order to save computational time during the numerical analysis of the above type, a transformation of the resulting planes similarly to a

Hough transformation may be applied. Each plane may be considered as a group of lines all going through the center of the ball. A lab coordinate system may be selected in a manner as it is similarly known when analyzing the earth sphere. All these straight lines may be described by two angular parameters (longitude and latitude). For each group of these straight lines, the following function may be determined:

longitude = f(latitude)

This function f may be plotted in a diagram (longitude versus latitude). This diagram corresponds to the Hough space. Now, further transformed mirror planes may be added. The position of the rotation axis may be determined based on the coordinates of the point in the Hough space with the largest value. On the basis of the rotation angle around this found axis, the absolute value of the spin may be determined using statistical methods.

A launch monitor may measure the motion of the hit golf ball and the motion of the golf club before and/or after the point of time of the hit. The launch monitor may be provided with optional additional devices like sensors, additional cameras or additional flashes for detecting parameters of the motion of the golfer, the ball and/or the equipment. The communication with the additional devices can be carried out using cables or a wireless communication path. Particularly, it is possible to use 06107

- 12 -

Bluetooth for such a communication. It is also possible to use infrared communication, radio frequency communication, a (mobile) telecommunication network, wireless LAN (WLAN), etc.

In the following, further exemplary embodiments of the golf diagnosis apparatus will be explained. However, these embodiments also apply to the data processing device, the method of data processing, the program element and the computer readable medium.

The golf diagnosis apparatus may comprise at least one of the group consisting of a power supply unit for supplying at least a part of the golf diagnosis apparatus with electrical energy, an optical display unit for displaying golf diagnosis related information, a user interface unit for allowing a user to communicate with at least a part of the golf diagnosis apparatus, a sensor unit for sensing at least one golf diagnosis related sensor parameter, a stroboscope unit for generating pulses of electromagnetic radiation (for instance infrared or visible or ultraviolet light flashes), and a data evaluation unit for evaluating golf diagnosis related data.

The image acquisition device may be a camera, for instance a CCD camera or a CMOS camera. It is also possible to provide a plurality of cameras.

The power supply unit may be a battery, an accumulator, solar cells, etc.

The optical display unit may be a monitor, like an LCD monitor, a TFT monitor, an OLED (organic LED) based display, a plasma monitor or a conventional cathode ray tube.

The user interface unit may comprise input elements like a keypad, a joystick, a trackball, or may even comprise a voice recognition system. The user interface unit may also include a touch screen. A sensor unit may be any kind of sensor, like a sensor of acoustic waves (for instance for detecting a point of time at which the golf club hits the golf ball), an optical sensor, a position sensor, a pressure sensor for detecting the weight distribution within the shoes of the golfer, a pressure sensitive platform or mat (pad), etc. One or more flashlight units, for instance strobes, may be provided so as to define different points of time at which the golf ball and the golf club shall be visible at an image of the camera. Therefore, by taking a plurality of images of the golf ball and/or the golf club and/or the golf player, it is possible to derive motion parameters from the captured images.

The data evaluation unit may be a CPU (centra! processing unit) and may include also a storage device, an input/output unit, etc. Such a data evaluation unit may carry out calculations in accordance with pre- stored algorithms so as to derive golf analysis related parameters from the captured information.

The golf diagnosis apparatus may comprise a plurality of image acquisition devices positioned to capture images of a golf player carrying out a stroke from different viewing directions. Thus, the amount of information provided and usable for assessing a stroke and the quality thereof may be increased and refined. Particularly, complementary information from different viewing directions may be obtained.

In the following, further exemplary embodiments of the data processing device will be explained. However, these embodiments also apply for the data processing method, for the golf diagnosis apparatus, for the program element and for the computer readable medium.

The image processing unit may be adapted for determining a position of at least two markers on the at least two images of the movable object. The mirror plane determining unit may then be adapted for determining a plurality of mirror planes for the at least two markers on pairs of the at least two images of the movable object. According to this embodiment, several markers on two images may be sufficient to unambiguously determine the spin direction. For this purpose, two (or more) images may be projected onto one another. The mirror planes going through the center of gravity of the ball and mapping one marker to the corresponding position on the other image may be generated or determined. An intersection of these individual planes then derives the spin direction.

Alternatively, the image processing device may be adapted for determining a position of the at least one marker on at least three images of the movable object, and the mirror plane determining unit may be adapted for determining a plurality of mirror planes for the at least one marker on pairs of the at least three images of the movable object. One marker and at least three images may also be sufficient for unambiguously determining the spin direction. A first mirror plane may map the marker at the first point of time to the marker at the second point of time. A second mirror plane may map the marker from the first point of time to the third point of time or from the second point of time to the third point of time. These two or more mirror planes may then be intersected where an intersection line corresponds to a spin direction direction.

The image processing unit may be adapted for determining at least two positions of at least two portions of the at least one marker on the at least two images of the movable object. Furthermore, the mirror plane determining unit may be adapted for determining a plurality of mirror planes for the at least two portions of the at least one marker on pairs of the at least two images of the movable object. Therefore, an elongated object having a one or two-dimensional extension may be evaluated by image processing techniques and may be interpreted as a plurality of individual marker portions. A mapping of each marker portion at one point of time to another position at another point of time may then be performed, and a plurality of mirror planes may be defined which (theoretically) all intersect one another in one straight line. Therefore, the evaluation of one complex structure at several points of time allows to derive the spin direction.

The image processing unit may further be adapted for distinguishing different ones of the plurality of markers based on at least one surface property of the group consisting of a colour, a shape, a reflectivity, a fluorescence, and an emission characteristic of different ones of the plurality of markers. In other words, the individual markers may differ regarding at least one property which is capable to be automatically recognized by image processing routines.

The image processing unit may further be adapted for determining a position of four markers on the movable object on at least two images of the movable object, the four markers having different colors (or differ regarding another surface property) and being arranged on corners of a rectangle. The mirror plane determining unit may be adapted for determining a plurality of mirror planes for the four markers on pairs of the at least two images of the movable object. Using four markers which may be provided on corners of a rectangle, a redundant or over- determined system may be obtained which allows to derive the spin direction with high accuracy using averaging or statistical methods. It may also happen that one or more markers are not visible on one or more images since they are on the back surface of the ball which is not visible at specific images due to the fast rotation of the ball. In this context, the provision of four markers may ensure that a sufficient number of markers is always visible, allowing a determination of the spin direction even when others of the markers are temporarily not visible. The image processing unit may be adapted for determining the position of the at least one marker on at least two images of a ball, particularly of a golf ball, as the movable object. A (golf) ball as a rotationally symmetric structure may be particularly suitable as a basis for the data processing, since such a spherical structure with a homogeneous weight distribution allows to simplify the evaluation procedure. Therefore, the data processing device may be advantageously implemented in a golf diagnosis apparatus.

The image processing unit may be adapted for determining a three-dimensional position (that is a position in coordinates of a three- dimensional body) of the at least one marker on at least two two- dimensional images (that is a position in coordinates of a two- dimensional projection of the three-dimensional body as obtained on a planar image) of the movable object. Therefore, the planar information derived from the image (captured by a CCD camera or the like) may be transferred into the three-dimensional space by evaluating parameters such as the size, the rotation, the distance of the golf ball on the different images, etc. Photogrammetry methods may be implemented for such a dimension transformation.

The mirror plane determining unit may be adapted for determining each of the plurality of mirror planes as a mirror plane mapping the at least one marker on one of the at least two images of the movable object on the at least one marker on another one of the at least two images of the movable object. For enabling such a mapping, the images of the movable object may be projected on one another, for instance by size adaptation and/or shifting of the images. The mirror plane determining unit may be adapted for determining the plurality of mirror planes in a manner such that each of the plurality of mirror planes include a center of gravity of the movable object. Without wishing to be bound to a specific theory, it is presently believed to be a physical frame condition that the spin axis goes through the center of gravity of the (homogeneous) movable object. Therefore, this frame condition may be used for deriving spin properties.

The spin determining unit may further be adapted for determining the spin direction of the movable object using a center of gravity of the movable object. In other words, the spin direction may be defined to necessarily go through the center of gravity of the essentially homogeneous movable object such as a golf ball.

The spin determining unit may be adapted for determining the spin direction of the movable object using a plurality of straight lines obtained by intersecting the plurality of mirror planes. Calculating such an intersection allows to unambiguously derive the direction of the spin.

The spin determining unit may be adapted for determining the spin direction of the movable object by averaging over the plurality of straight lines. In the absence of any artifacts or miscalculations or measurement errors, all straight lines should be identical. However, in reality, there may be a statistical distribution of the individually estimated straight lines around a maximum representing a most likely spin direction.

The spin determining unit may be adapted for determining the spin direction of the movable object by a statistical evaluation of the plurality of straight lines. Such a statistical evaluation may include the generation of a histogram in which the individual straight lines may be plotted. A maximum value of this distribution may then be selected as a likely true spin direction axis.

The spin determining unit may be adapted for determining an absolute value of the spin direction of the movable object by an angle of rotation of the at least one marker of the at least two images of the movable object. After having determined the spin axis, the absolute value of the spin may be calculated on the basis of the rotation angle of the ball on the individual images. A time distance between the two or more capturing times may be derived based on the times of flash illumination. On the basis of this time information and this angle information, the absolute value of the spin may be derived mathematically.

The spin determining unit may be adapted for determining the spin direction of the movable object based on a Hough transformation of the plurality of mirror planes. The Hough transformation is a feature extraction technique used in digital image processing. Such a transform identifies lines in an image. An underlying principle of the Hough transform is that there are an infinite number of lines that pass through any point, each at a different orientation. The purpose of the transform may be to determine which of these theoretical lines passes through the most features in an image - that is, which lines fit most closely to the data in the image. By performing a Hough transform, the mirror planes may be evaluated to determine the true spin axis with low numerical effort and therefore in a fast time. Additionally or alternatively, the spin determining unit may be adapted for determining the spin direction of the movable object based on a Quaternion evaluation. Such a Quaternion evaluation (see "Edward Pervin and Jon A. Webb, "Quaternions in Computer Vision and Robotics", published 1982) may allow to determine the spin direction based on the inclusion of mirror planes. Quaternions may be considered as an extension of complex numbers into four dimensions. Quaternions may be used for a geometrical analysis of the individual images and markers including rotations. Thus, the implementation of Quaternion algebra may accelerate the spin determination procedure. The spin determining unit may be adapted for determining a spin vector of the movable object using the at least one straight line obtained by intersecting the at least two of the plurality of mirror planes. Thus, a spin vector including a spin direction and an absolute value of the spin may be determined, the direction being derivable from the intersection line(s), and the absolute value being derivable from the rotation angle of the marker(s) on the different images.

The aspects defined above and further aspects of the invention are apparent from the examples of embodiment to be described hereinafter and are explained with reference to these examples of embodiment.

The invention will be described in more detail hereinafter with reference to examples of embodiment but to which the invention is not limited.

Figure 1 illustrates a golf diagnosis system according to an exemplary embodiment of the invention.

Figure 2 to Figure 5 schematically illustrate schemes of deriving a spin direction from images of markers of a golf ball according to exemplary embodiments of the invention. Figure 6 illustrates an image of a golf ball captured and evaluated by a golf diagnosis apparatus according to an exemplary embodiment of the invention.

Figure 7 illustrates an image of a golf ball and of a golf club captured and evaluated by a golf diagnosis apparatus according to an exemplary embodiment of the invention.

The illustration in the drawing is schematically. In different drawings, similar or identical elements are provided with the same reference signs. In the following, referring to Fig. 1, a golf analysis system 100 according to an exemplary embodiment of the invention will be described.

As shown in Fig. 1, a golf player 101 is in a position to carry a golf club 102 including a shaft 103 and a club head 104. A golf ball 105 is positioned on a tee (not shown). The golf diagnosis apparatus 100 comprises a central processing unit (CPU) 113 (which may, in another embodiment, be a microprocessor) which includes processing resources and storage resources. The CPU 113 may serve as a control system for the entire golf diagnosis apparatus 100. The CPU 113 is electrically coupled (in a bidirectional manner or in a unidirectional manner) with a CCD (charge coupled device) camera 114. Instead of providing a single CCD camera 114, it is also possible to provide two or more cameras. It may be particularly advantageous to provide only a single camera, since this may allow to manufacture the device 100 with low costs and in a small size. When a plurality of CCD cameras 114 are provided, the device 100 may be adapted to monitor the golf player 101 from different viewing directions/viewing angles so as to derive complementary information for evaluating a stroke of the golfer 101. Furthermore, a first flash 116 and a second flash 117 are provided.

The flashes 116, 117 can be positioned at any desired position of the golf diagnosis apparatus 100, particularly attached to a casing of the golf diagnosis apparatus 100. The flashes 116, 117 may emit light flashes so as to define points of time at which images of the golf club 102, of the golf ball 105 and/or of the golf player 101 are captured by the camera 114. As an alternative for the flashes 116, 117, strobes may be provided. It is possible to implement such light flash sources using LEDs, particularly OLEDs. Instead of using two flashes 116, 117, it is possible to use only one flash or at least three flashes. For example, each of the flashes 116, 117 can emit a single flash, or a single flash 116 or 117 may emit two or more flashes. Also the number of light pulses may vary, and can be larger or equal than two.

Furthermore, the CPU 113 is coupled to an LCD display 118 as an optical display unit for displaying results of the golf diagnosis. Moreover, the CPU 113 is coupled to an input/output device 119 like a keypad, a joystick, a touch screen or the like so as to provide the CPU 113 with control information. For instance, the golfer 101 may input, via the input/output device 119, information indicating a club 102 which shall be used for the strike, so as to provide the system 100 with the required information needed to evaluate the stroke.

As further shown in Fig. 1, a microphone 124 is provided for detecting acoustic waves resulting from a hit between the golf club head 104 and the ball 105. Furthermore, a wireless communication interface 125 is provided at the golf diagnosis apparatus 100, and is coupled to the CPU 113. Via the wireless communication interface 125 (which may communicate via Bluetooth or an RFID protocol), communication with optional sensors 128, 129 located in both shoes 126, 127 of the golfer 101 is possible. Furthermore, wireless communication with the sensor 130 provided in the golf club head 104 and with the sensor 131 provided in the golf ball 105 is possible.

Furthermore, the golf ball 105 comprises a marker 150, which may be a text or a symbol having optical properties differing from those of the surrounding of the generally white golf ball 105. In a similar manner, a marker 151 may be provided at the golf club 104, and a marker 152 may be provided at the shaft 103 of the club 102.

In the following, the functionality of the system 100 will be explained in more detail. When the golf player 101 has operated the golf club 102 so that the golf head 104 hits the ball 105, acoustic waves are generated. These are detected - with a corresponding delay - by the microphone 124. Consequently, the flashes 116, 117 are triggered to emit light pulses, particularly two light pulses having a length of 20 μs and having a time distance of 2 ms. Correspondingly, points of time are defined by these flashes 116, 117 at which the camera 114 detects images of the hit ball 105, the moving club 102, and/or the moving golf player 101 (essentially) during or after the hit.

Furthermore, sensor information from the sensors 128 to 131 are transmitted to the communication interface 125. All these items of information may be used by the CPU 113 to derive golf diagnosis information, like spin information, angle information, velocity information, distance information, etc. A result of such an evaluation may be output via the display unit 118. A light barrier 140 may be provided for optically detecting the time of hitting the ball 105 or for detecting a swinging golf club 102.

More particularly, the golf diagnosis apparatus 100 comprises an imaging apparatus formed by the illumination arrangement (namely the flashes 116, 117) adapted for illuminating the moving golf ball 105 during two or more timely spaced intervals, defined by the duration of the flashes and the time distance between subsequent flashes. The CCD camera 114 (alternatively a CMOS camera) is provided to capture an image of the moving golf ball 105 and of the swinging golf club 102. The CPU 113 serves as a control unit for coordinating the flashes 116, 117 and the CCD camera 114 in a manner that the CCD camera 114 captures the image of the illuminated golf ball 105 and of illuminated swinging golf club 102 during the two or more timely spaced time intervals and that the CCD camera 114 is deactivated during at least a portion of the time distance between the at least two timely spaced time intervals. In other words, the camera 114 will be activated only during specific points of time which correlate at least partially with the illuminating times of the flashes 116, 117.

However, the CCD camera 114 adds the images of the flying golf ball 105 and of the swinging golf club 102 captured during the multiple flashes of the flash units 116, 117 to thereby form a single image illustrating the flying golf ball 105 and the swinging golf club 102 during the flash intervals. However, a shutter mechanism, more particularly an electronic shutter mechanism, of the CCD camera 114 deactivates, under the control of the CPU 113, the CCD camera 114 during the major part of the time distance between the light pulses emitted by the flashes 116, 117. According to the described embodiment, the flashes 116, 117 emit the light pulses simultaneously. Alternatively, the different flashes 116, 117 may be used to generate flashes at different points of time.

The. CPU 113 also serves as an evaluation unit for evaluating motion characteristics of the flying golf ball 105 and of the swinging golf club 102 based on an analysis of the images captured by the CCD camera 114. On this image, the golf ball 105 and the golf club 102 are displayed in an illuminated fashion at different times during the golf ball 105 flight and the golf club 102 swing. Since the flashes 116, 117 are positioned so that the CCD camera 114 is located between the flashes 116, 117, the camera 114 is positioned essentially symmetrically and detects a bright centre of the ball 105 surrounded by a dark circular edge of the ball 105. An image processing software running on the CPU 113 recognizes particularly a shoulder between the edge of the ball 105 and a (grey) background. Due to the deactivation of the camera 114 between the flashes generated by the flash units 116, 117, the contrast between the bright ball and the dark background is improved or enhanced, thereby allowing the image processing routines to be performed with improved accuracy, providing more meaningful golf diagnosis results. In a similar manner, the golf club 102 head 104 having a characteristic shape and design may be reliably recognized on the images. Moreover, the markers 150 to 152 may be identified by image processing techniques.

When images of the golf club 102 before the hit shall be evaluated, a swinging golf club 102 may be detected using a light barrier unit 140 detecting when the golf club 102 is elongated by more than a predetermined angle, as indicated by a dotted line in Fig. 1.

When images of the golf ball 105 directly after the hit shall be evaluated, the point of time of the hit may be detected by the microphone 124.

After having captured the individual images of the golf ball, the CPU 113 also serves for a data processing to derive one or more parameters indicative of a golf stroke. In this context, the data provided by the camera 114 may be processed to derive the total spin, more particularly back spin, exerted by the golf club 104 onto the ball 105. For this purpose, the CPU 113 performs image processing and determines a position of the marker 150 on a plurality of images of the golf ball 105. The CPU 113 may be further adapted for determining a plurality of mirror planes for the marker 150 on pairs of the multiple images of the golf ball 105. Beyond this, the CPU 113 determines a spin direction of the golf ball 105 using at least one straight line obtained by intersecting the several mirror planes.

This procedure will be explained in more detail in Fig. 2 to Fig. 5. Fig. 2 shows the golf ball 105 at three points of time after the launching.

The three points of time correspond to three flashes generated by the units 116, 117. A center of gravity 200 of the essentially homogeneous golf ball 105 is indicated as well. Furthermore, the marker 150 is shown in each of three positions of the rotating ball 105. A rotation direction is defined by the back spin transferred onto the golf ball 105 by the interaction with the golf club 104.

Thus, Fig. 2 illustrates the golf ball 105 at a first point of time indicated by the number "1", a second point of time indicated by the number "2" and a third point of time indicated by the number "3". Fig. 3 shows a (virtual) image of a superposition of the golf ball 105 at the three points of time referring to Fig. 2.

An image processing unit of the CPU 113 first projects the three images at the times "1", "2" and "3" of Fig. 2 on one another so that the ball size equals to one another and that the center of gravity 200 coincides for all three images.

Then, the image processing unit of the CPU 113 determines the position of the single marker 150 on the three images of the golf ball 105. This is indicated in Fig. 3 schematically by the numbers "1", "2" and "3" on the marker 150 corresponding to the position of the marker 150 in the three images "1", "2" and "3" of Fig. 2.

Furthermore, the CPU 113 includes a mirror plane determining unit for determining a first mirror plane 300 (extending perpendicular to the paper plane of Fig. 3) mapping the marker 150 at the first position "1" to the third position "3". The first mirror plane 300 goes through the center 200 of the golf ball 105.

Moreover, a second mirror plane 301 (extending perpendicular to the paper plane of Fig. 3) is determined in a manner that the marker 150 is mapped to the third marker 150. Also the second mirror plane 301 goes through the center of gravity 200. It is also possible to calculate a mirror plane mapping the marker 150 at the second position "2" to the third position "3". Also this mirror plane (not shown in Fig. 3) would go through the center of gravity 200 of the golf ball 105.

Furthermore, the CPU 113 includes a spin determining unit determining the spin direction of the moving ball 105 using a straight line obtained by intersecting the mirror planes 300 and 301. In the example of Fig. 3, the spin direction is oriented perpendicular to the paper plane which is a direction included in both of the planes 300, 301 which are also oriented perpendicular to the paper plane. This allows to unambiguously determine the spin vector of the golf ball 105. Thus, the spin vector goes through the center of gravity 200 of the golf ball 105 and is oriented perpendicular to the paper plane of Fig. 3. Furthermore, the amount, that is to say the absolute value, of the spin vector may be determined based on an angular rotation of the ball 105 indicated by the rotation of the marker 150 at the different points of time of the images "1", "2" and "3" which can be defined by points of time at which the flashes 116, 117 emit their light pulses. Thus, the absolute value and the direction of the spin vector of the golf ball 105 are determined, thereby allowing to determine the entire spin vector. Fig. 4 shows another exemplary embodiment on which two markers 150, 400 are provided in different surface portions of the golf ball 105.

Therefore, since two markers 150, 400 are evaluated, a first image taken at a first point of time (indicated by number "1") and a second image captured at a second point of time (indicated by a number "2") are sufficient to derive the spin vector.

As can be taken from Fig. 5, a first mirror plane 500 (extending perpendicular to the paper plane of Fig. 5) maps the first image of the first marker 150 onto the second image of the first marker 150.

Furthermore, a second mirror plane 501 (extending perpendicular to the paper plane of Fig. 5) maps the first image of the second marker 400 onto the second image of the second marker 400.

The cross-section line of the planes 500, 501 then again equals to the spin vector which is oriented perpendicular to the paper plane of Fig. 5, as the orientation of the planes 500, 501 are.

Fig. 6 shows an image 600 of a golf ball 105 at two different points of time.

Four markers 601 to 604 which may be distinguished by the different colours, are provided on the golf ball 105. The four markers 601 to 604 are positioned on corners of a rectangle, more particularly a square on the golf ball 105. This redundant opportunity to determine the spin vector (since four mirror planes are determined) allows to determine the spin vector with higher accuracy, since four intersection lines are obtained, the average of which may be a good measure for the true spin vector axis.

Even when a ball 105 has such a fast rotation that one or more of the markers 601 to 604 is not visible on a second or a third image, the other markers may still serve as a basis for determining the spin vector. In the following, referring to Fig. 7, an image 700 illustrating a golf ball 105 and a golf club 102 captured and evaluated by the golf diagnosis apparatus 100 according to an exemplary embodiment of the invention will be explained.

Fig. 7 shows a first image 701, a second image 702, a third image 703 and a fourth image 704 of the club 102. Furthermore, a first image 705, a second image 706 and a third image 707 of the golf ball hit by the club 102 is shown.

Two images 701, 702 of the club 102 before the hit between the club 102 and the golf ball 107 are shown so that the images 701, 702 of the club 102 correspond to the static image of the golf ball 107 resting on a tee 708. Directly after the hit, the third image 703 of the club 102 and the second image 706 of the golf ball 107 are detected and may be correlated. Moreover, at a further later point of time, the fourth image 704 of the club 102 and the third image 707 of the golf ball 107 are correlated and grouped in a pairwise manner.

The position and angular orientation of the club 102 on the different images 701 to 704 allows to derive information regarding the kinematics of the golf stroke and of the hit between the club 102 and the ball 107. Size, distance and position of the markers of the golf ball 107 provide further information about the kinematics of the hit. Therefore, the spin of the ball 105 as well as other motion parameters may be analyzed qualitatively and quantitatively using the image of Fig. 7.

It should be noted that the term "comprising" does not exclude other elements or features and the "a" or "an" does not exclude a plurality. Also elements described in association with different embodiments may be combined.

It should also be noted that reference signs in the claims shall not be construed as limiting the scope of the claims.

Claims

C l a i m s

1. A data processing device, comprising an image processing unit adapted for determining a position of at least one marker on at least two images of a movable object; a mirror plane determining unit adapted for determining a plurality of mirror planes each mapping one of the at least one marker from one of the at least two images to another one of the at least two images of the movable object; a spin determining unit adapted for determining a spin direction of the movable object using at least one straight line obtained by intersecting at least two of the plurality of mirror planes.

2. The data processing device of claim 1, wherein the image processing unit is adapted for determining a position of at least two markers on the at least two images of the movable object; wherein the mirror plane determining unit is adapted for determining a plurality of mirror planes each mapping one of the at least two markers from one of the at least two images to another one of the at least two images of the movable object.

3. The data processing device of claim 1, wherein the image processing unit is adapted for determining a position of the at least one marker on at least three images of the movable object; wherein the mirror plane determining unit is adapted for determining a plurality of mirror planes each mapping one of the at least one marker from one of the at least three images to another one of the at least three images of the movable object.

4. The data processing device of claim 1, wherein the image processing unit is adapted for determining at least two positions of at least two portions of the at least one marker on the at least two images of the movable object; wherein the mirror plane determining unit is adapted for determining a plurality of mirror planes each mapping one of the at least two portions of the at least one marker from one of the at least two images to another one of the at least two images of the movable object.

5. The data processing device of any one of claims 1 to 4, wherein the image processing unit is adapted for distinguishing different ones of a plurality of markers based one at least one surface property of the group consisting of a colour, a shape, a reflectivity, a fluorescence, and an emission characteristic of different ones of the plurality of markers.

6. The data processing device of any one of claims 1 to 5, wherein the image processing unit is adapted for determining a position of four markers on the movable object on at least two images of the movable object, the four markers being distinguishable by at least one surface property and being arranged on corners of a rectangle; wherein the mirror plane determining unit is adapted for determining a plurality of mirror planes each mapping one of the four markers from one of the at least two images to another one of the at least two images of the movable object.

7. The data processing device of any one of claims 1 to 6, wherein the image processing unit is adapted for determining the position of the at least one marker on at least two images of a ball, particularly of a golf ball, as the movable object.

8. The data processing device of any one of claims 1 to 7, wherein the image processing unit is adapted for determining a three-dimensional position of the at least one marker on at least two two-dimensional images of the movable object.

9. The data processing device of any one of claims 1 to 8, wherein the mirror plane determining unit is adapted for determining the plurality of mirror planes in such a manner that each of the plurality of mirror planes includes a center of gravity of the movable object.

10. The data processing device of any one of claims 1 to 9, wherein the spin determining unit is adapted for determining the spin direction of the movable object using a center of gravity of the movable object.

11. The data processing device of any one of claims 1 to 10, wherein the spin determining unit is adapted for determining the spin direction of the movable object using a plurality of straight lines obtained by intersecting the plurality of mirror planes.

12. The data processing device of claim 11, wherein the spin determining unit is adapted for determining the spin direction of the movable object by averaging over the plurality of straight lines.

13. The data processing device of claim 11, wherein the spin determining unit is adapted for determining the spin direction of the movable object by a statistical evaluation of the plurality of straight lines.

14. The data processing device of any one of claims 1 to 13, wherein the spin determining unit is adapted for determining an absolute value of the spin vector of the movable object by an angle of rotation of the at least one marker on the at least two images of the movable object.

15. The data processing device of any one of claims 1 to 14, wherein the spin determining unit is adapted for determining the spin direction of the movable object based on a Hough transformation of the plurality of mirror planes.

16. The data processing device of any one of claims 1 to 15, wherein the spin determining unit is adapted for determining the spin direction of the movable object based on a Quaternion evaluation.

17. The data processing device of any one of claims 1 to 16, wherein the spin determining unit is adapted for determining a spin vector of the movable object using the at least one straight line obtained by intersecting the at least two of the plurality of mirror planes.

18. A golf diagnosis apparatus for evaluating a performance, particularly a stroke, of a golf player, the golf diagnosis apparatus comprising a data processing device according to any one of claims 1 to 17 adapted to process images of a golf ball as the movable object.

19. The golf diagnosis apparatus according to claim 18, comprising at least one of the group consisting of a power supply unit for supplying at least a part of the golf diagnosis apparatus with electrical energy, an optical display unit for displaying golf diagnosis related information, a user interface unit for allowing a user to communicate with at least a part of the golf diagnosis apparatus, a sensor unit for sensing at least one golf diagnosis related sensor parameter, and a data evaluation unit for evaluating golf diagnosis related data.

20. A method of data processing, the method comprising determining a position of at least one marker on at least two images of a movable object; determining a plurality of mirror planes each mapping one of the at least one marker from one of the at least two images to another one of the at least two images of the movable object; determining a spin direction of the movable object using at least one straight line obtained by intersecting at least two of the plurality of mirror planes.

21. A program element, which, when being executed by a processor, is adapted to control or carry out a method of claim 20 of data processing.

22. A computer-readable medium, in which a computer program is stored which, when being executed by a processor, is adapted to control or carry out a method of claim 20 of data processing.