WO2007087385A2

WO2007087385A2 - Imaging system and method including multiple, sequentially exposed image sensors

Info

Publication number: WO2007087385A2
Application number: PCT/US2007/002008
Authority: WO
Inventors: David Benjamin Rankin
Original assignee: Accu-Sport International, Inc.
Priority date: 2006-01-23
Filing date: 2007-01-23
Publication date: 2007-08-02
Also published as: WO2007087385A3; US20070202959A1

Abstract

An imaging system and method are provided that include one or more image sensors, a light source for illuminating the field(s) of view of the image sensor(s), a control system for directing operation of the light source and the image sensor(s) in order to capture image(s) of a ball and an image processor for analyzing the image(s) of the ball to determine at least one parameter associated with movement of the ball through the field(s) of view of the image sensor(s). An imaging system and method of one aspect are capable of determining a plurality of parameters associated with movement of a ball based upon single exposure images captured by a plurality of image sensors, while an imaging system and method of another aspect include an accelerometer and are capable of automatically compensating for changes in orientation of an image sensor from a predefined orientation.

Description

IMAGING SYSTEM AND METHOD INCLUDING MULTIPLE, SEQUENTIALLY EXPOSED IMAGE SENSORS

FIELD OF THE INVENTION

Embodiments of the present invention relate generally to techniques for measuring the launch conditions of a golf ball or other object in motion and, more particularly, to techniques for capturing successive images of a golf ball or other object from which a variety of launch parameters can be determined.

BACKGROUND OF THE INVENTION

Many golfers work consistently to improve their game. One tool that has been developed to assist golfers in analyzing their golf shots is a golf ball launch monitor. Golf ball launch monitors are used extensively to provide detailed information regarding the velocity of the ball, the launch angle of the ball, the side spin of the ball and the like. As such, golf ball launch monitors are quite useful in training sessions to evidence the manner in which swing changes affect the resulting golf shots. Similarly, launch monitors are quite useful in club fitting to provide feedback on the golf shots hit by a golfer with different clubs such that the golfer can more intelligently select the most suitable set of clubs for the game. Further details regarding launch monitors are provided by U.S. Patent Application No. 10/360,196 filed February 7, 2003, entitled "Methods Apparatus and Computer Program Products for Processing Images of Golf Balls", the contents of which are incorporated herein in their entirety.

Various types of launch monitors have been developed that rely upon different technologies to capture information regarding the launch conditions of a golf ball. One type of launch monitor captures two, successive images of a golf ball shortly after the golf ball has been struck. By comparing the images, parameters relating to the velocity of the ball, launch angle of the ball_* side spin of the ball and the like can be determined as described by the ' 196 application. In order to capture two successive images of the golf ball in a cost effective manner;, a single camera has typically been utilized. The single camera can capture two successive images of the golf ball in different manners. In a double exposure approach, the shutter of the camera is opened and the strobe lights are pulsed on two successive occasions such that two representations of the golf ball are captured in a single, double exposed image while the shutter remains open. Thus, within a single image, two different representations of the golf ball at two slightly different points in time are depicted. While this technique is quite workable and provides accurate results, the image captured by the camera can be adversely affected by the ambient lighting conditions since the shutter remains open for relatively long period of time while the strobes are pulsed on two occasions. The ambient light that is received while the shutter is open tends to decrease the resolution and the contrast of the images of the golf ball that are otherwise captured by the camera. This decreased resolution can adversely affect the accuracy with which the resulting golf ball launch parameters are determined.

In order to at least partially overcome the adverse affects introduced by the ambient lighting conditions, filters can be employed. While filters reduce the deleterious effects of the ambient light, the filters increase the illumination required to be provided by the strobe lights can be utilized. However, the cost of the resulting launch monitor is increased as more powerful strobe lights are utilized. Additionally, the power consumption of the launch monitor is increased as more powerful strobe lights are employed.

Another technique for capturing two successive images of a golf ball following launch with a single camera utilizes a double shutter method. In this technique, the shutter of the camera is opened and an image of the golf ball is captured. The shutter is then closed and the frame containing the image of the golf ball is moved out, such as by transferring the image from the charged coupled device (CCD) array to memory. The shutter is then reopened and a second image of the golf ball is captured. The shutter is then closed again and the image is moved from the CCD array to another location in memory. While this technique overcomes at least some of the issues associated with the double exposure technique described above by closing the shutter between those points in time at which images of the golf ball are obtained, the camera and its associated control circuitry must operate extremely quickly in order to capture the first image of the golf ball and then transfer the image to memory prior to capturing a second image of the golf ball. While cameras are available that can capture the successive images such cameras are typically quite expensive so as to adversely affect the cost of the resulting golf ball launch monitor.

As such, it would be advantageous to develop an improved launch monitor for capturing successive images of an object, such as a golf ball, such that various parameters that define the path of motion of the object and its launch conditions can be accurately determined in a cost effective manner.

Launch monitors are generally portable. As such, launch monitors may be used in the field, such as on a driving range, where the terrain on which the launch monitor will be set is uneven. In order to utilize the launch monitor, the launch monitor is placed adjacent the tee or other location from the golf ball will be struck and a configuration process is carried out in order to ensure that the launch monitor is prepared to capture images of the golf ball. Since a number of the parameters captured by the launch monitor are dependent upon the orientation of the launch monitor, it is desirable to orient the launch monitor in a repeatable manner. However, it is generally quite difficult to ensure that the launch monitor is disposed in the same orientation with the orientation of the launch monitor generally changing by at least a degree or two from one location to the next. In order to take into account the differences in orientation, launch monitors have provided users with the ability to input an offset from a level orientation. However, users may have difficulty in accurately determining the offset to be entered or users may simply forget to enter the offset. In instances in which the user forgets to enter the offset or enters an incorrect offset, some of the resulting parameters, such as the launch angle of the golf ball, that are determined by the launch monitor will be correspondingly inaccurate.

Accordingly, it would also be desirable to provide a launch monitor that takes into account any offset in the orientation of the launch monitor without requiring user input.

SUMMARY OF THE INVENTION

An imaging system and method are therefore provided to address at least some of the issues of conventional imaging systems. For example, an imaging system and method of one embodiment are capable of determining a plurality of parameters associated with movement of a ball based upon single exposure images captured by a plurality of image sensors, thereby potentially reducing the power required to operate the system and improving the resolution of the images and, in turn, the accuracy with which the parameters are determined. By way of another example, an imaging system and method of another embodiment are capable of automatically compensating for changes in orientation of an image sensor from a predefined orientation such that the resulting parameters that define the movement of the ball are potentially determined in a more accurate fashion.

According to one aspect, an imaging system is provided that includes a plurality of image sensors spaced apart from one another by a predetermined distance and having respective fields of view. In one embodiment, the plurality of image sensors may have different respective fields of view. For example, the plurality of image sensors may have respective image planes, and the plurality of image sensors may be positioned such that reference vectors normal to the respective image planes of the plurality of image sensors are parallel to one another.

In addition to the plurality of image sensors, the imaging system can also include a light source, such as .an array of light emitting diodes (LEDs), for illuminating at least portions of the fields of view of the plurality of image sensors and a control system configured to direct the light source to illuminate the respective fields of view of the image sensors and also configured to direct the image sensors to capture images, such as single exposure images, of a ball at different times. Further, the imaging system may include an image processor for analyzing the images of the ball in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors. Advantageously, the image processor may be configured to take into account the predetermined distance by which the image sensors are spaced apart in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors. If desired, the imaging system can also include a trigger mechanism for providing a trigger signal to the control system for initiating actuation of the light source.

In one embodiment in which the imaging system includes first and second image sensors, the control system is configured to direct the light source to illuminate the field of view of the first image sensor and also configured to direct the first image sensor to capture an image of the ball at a first time. Additionally, the control system of this embodiment is further configured to direct the light source to illuminate the field of view of the second image sensor and also configured to direct the second image sensor to capture an image of the ball at a second time. In this regard, the second time can be separated timewise from the first time by a predefined time interval.

According to another aspect, an imaging system, such as a portable, battery-powered unit, is provided that includes an image sensor having a predefined field of view, a light source for illuminating at least a portion of the field of view of the image sensor, an accelerometer for providing a signal indicative of the orientation of the image sensor, a control system configured to direct the light source to illuminate the field of view of the image sensor and also configured to direct the image sensor to capture an image of a ball while the field of view of the image sensor is illuminated, and an image processor for analyzing the image of the ball while taking into account the orientation of the image sensor that is automatically provided by the accelerometer without manual input in order to determine at least one parameter associated with movement of the ball. In one embodiment, the image processor is configured to compensate for the orientation of the image sensor based on the signal provided by the accelerometer in order to determine an angle of flight, such as a launch angle, associated with movement of the ball.

According to yet another aspect, a method is provided in which a plurality of image sensors are provided that are spaced apart from one another by a predetermined distance and that have respective fields of view. In this regard, the plurality of image sensors that are provided may have different respective fields of view. For example, the plurality of image sensors may have respective image planes with the plurality of image sensors being positioned such that reference vectors normal to the respective image planes of the plurality of image sensors are parallel to one another.

The method also illuminates at least portions of the fields of view of the plurality of image sensors, captures images, such as single exposure images, of a ball at different times with the image sensors while the respective fields of view of the image sensors are illuminated, and analyzes the images of the ball in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors. In regards to the analysis of the images of the ball, the method of one embodiment may take into account the predetermined distance by which the image sensors are spaced apart from one another. If desired, a trigger signal may also be provided to initiate illumination of at least portions of the fields of view of the plurality of image sensors and capture of images of the ball at different times.

In one embodiment in which the plurality of image sensors that are provided include first and second image sensors, the field of view of the first image sensor is illuminated and an image of the ball is captured by the first image sensor at a first time. Then, the field of view of the second image sensor is illuminated and an image of the ball is captured at a second time. In this regard, the second time is separated timewise from the first time by a predefined time interval.

BRIEF DESCRIPTION OF THE DRAWINGS Having thus described the invention in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:

Figure 1 is a block diagram of a imaging system according to one embodiment of the present invention; and

Figures 2A and 2B are exemplary images of a golf ball captured by first and second image sensors of an imaging system of one embodiment of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION The present inventions now will be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all embodiments of the inventions are shown. Indeed, these inventions may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like numbers refer to like elements throughout.

As shown in Figure 1, the imaging system 10 of one advantageous embodiment of the present invention includes at least two image sensors 12 operating under the direction of a control system 14, such as a microcontroller, microprocessor, personal computer or other computing system. While various types of image sensors may be utilized, the imaging system of one embodiment includes image sensors that are of an accumulator type, such as image sensors that are each comprised of a plurality of CCDs arranged in an array, such as a rectangular array. Alternatively, the image sensors may be formed of CMOS imaging sensors or film-based photographic film. In any event, the image sensors are advantageously of an accumulator type which includes elements or pixels that receive photons while the shutter is open and that generate and accumulate electrons in response to the received photons with the electrons building up until the frame is moved out of the image sensor.

As will be apparent, the image sensors 12 cannot be co-located and therefore must be positioned offset from one another by a predetermined distance and in a predetermined direction. Each image sensor is positioned, however, such that the field of view of the image sensor includes a different portion of the anticipated path of the object to be imaged, such as the anticipated flight path of a golf ball following launch of the golf ball. In this regard, although the imaging system 10 will be described as a launch monitor in conjunction with the capture of a pair of images of a golf ball at two different points in time following launch in order to determine various launch parameters, the imaging system can be utilized to capture successive images of a number of different objects including, for example, baseballs, softballs, tennis balls or the like, without departing from the spirit and scope of the present invention.

Notably, the pair of image sensors 12 do not capture a stereo image of the golf ball. In this regard, the image sensors are not adapted to simultaneously capture an image of the golf ball from two different points of view from which a stereo image could be constructed. Instead, the control system 14 drives the image sensors to capture respective images of the golf ball at two different points in time separated by a predetermined interval, such as a millisecond or so. Since the image sensors are exposed at different points in time, the image sensors are not focused on the same point, i.e., the image sensors do not have the same field of view. Instead, the vectors normal to the plane of each image sensor, such as the plane defined by the respective CCD arrays, are generally parallel for the two different image sensors.

In operation, the imaging system 10 can either be triggered or can be free- running. In a free-running system, such as might be employed to capture video images of a putt from which the speed of the putt could be determined, the image sensors 12 alternatively capture images and those images are analyzed by an image processor 16, typically comprised of a personal computer or other computing system, to identify the golf ball. As described by the '196 patent, the image processor can detect the appearance of a golf ball within the image as a relatively circular object that typically has certain predetermined markings, such as a pair of dots, a logo or the like imprinted thereupon. If triggered, the imaging system can include a triggering mechanism 18, such as an optical trigger mechanism which detects movement of a golf club past a particular point or an audible trigger mechanism which detects the sound created upon contact between the golf club and the golf ball. The triggering mechanism provides a trigger signal to the control system 14 which directs the image sensors to capture respective images. In this regard, the control system can actuate the first image sensor to capture an image at a first time and can then actuate the second image sensor to capture an image at a second time (later than the first time) in order to capture successive images of the golf ball following its launch.

Regardless of whether the imaging system 10 is sensor free-running or triggered, the shutter of each image sensor 12 is generally only open for a relatively short period of time so as to capture the image without being significantly adversely affected by ambient light. In this regard, in response to actuation by the control system, the shutter of a respective image sensor is opened and an image is captured, typically while the field of view is illuminated by a light source 20, such as described by U.S. Provisional Application No. 60/761,118, filed

January 23, 2006 and, in turn, U.S. Patent Application No. filed January 23, 2007, both entitled "Imaging System and Method Including Adaptive Lighting". The shutter then closes following a predefined exposure time, such as 100 μsec in one embodiment. As a result of the shutter only being opened for the relatively short period of time, the image captured by the image sensor is generally more precise and has a higher contrast than images captured by the double exposure technique, thereby leading to improved accuracy in the eventual determination of the golf shot parameters.

As the image sensor 12 is exposed in such a manner as to reduce the effects of the ambient light, the illumination which is required to capture an image of the golf ball is less than is required by a double exposure system, thereby correspondingly requiring less power. As such, the light source 20 of the imaging system 10 of the present invention can include less powerful strobe lights than a corresponding double exposure system, thereby potentially decreasing both the cost and the energy requirements of the imaging system. Alternatively, instead of strobe lights, the light source can, instead, include arrays of light-emitting diodes (LEDs) in order to illuminate the scene captured by the image sensor, thereby further reducing the power requirements of the imaging system. While reducing the power requirements of the imaging system is advantageous in general, the reduction in the power requirements is particularly advantageous in instances in which a portable battery operated imaging system is desired, thereby facilitating use of the imaging system on a golf range or other location at which power is not readily available.

After capturing an image with the first image sensor 12, the control system 14 waits a predefined time period and then actuates the second image sensor in order to capture a second image of the object. The actuation of the second image sensor is generally performed in the same manner as described above in conjunction with the first image sensor and may utilize the same or a different light source 20.

In instances in which the image sensors 12 are comprised of CCD arrays, CMOS imaging sensors or film-based photographic film, the control system 14 moves the images from the image sensors to respective memory 22 locations. While the control system may move the image captured by the first image sensor to a respective memory location prior to actuation of the second image sensor or while the second image sensor is capturing its respective image, the control system can wait until both image sensors have captured their respective images prior to moving either of the images to their respective memory locations. Once the respective images have been moved to memory, the image processor 16 retrieves and analyzes the images in order to determine various parameters associated with the object. In instances in which the object is a golf ball, the image processor may determine various launch parameters including the velocity of the ball, the launch angle of the ball, the side spin of the ball and the like. While the image processor can analyze the respective images in various manners, the image processor of one embodiment analyzes the images of the golf balls in the manner described by the ' 196 application in order to determine the various launch parameters. Moreover, the image processor can process the images immediately or at least soon after the transfer of the images to memory or the image processor can process the images at some later point in time if so desired.

It is noted that unlike the double exposure technique described by the ' 196 application in which the change in the XY position of the golf ball was readily determinable from the single frame captured by the image sensor, the image processor 16 of the imaging system 10 of one embodiment of the present invention must also take into account the offset between the two image sensors 12 and, in particular, between the respective field of view for each image sensor. In this regard, for a pair of image sensors having respective fields of view which are offset in the X direction, such as the horizontal direction, by a predefined amount, x_Ofrset» but are not offset in the Y direction, such as the vertical direction, the relative positions of the golf ball within the images captured by each image sensor must take into account x_Ofr_Set- In this regard, for an image of a golf ball centered at position Xi, Yi within the image captured by the first image sensor as shown in Figure 2 A and centered at a position of X₂, Y₂ within the image captured by the second image sensor as shown in Figure 2B, the change in XY position of the golf ball between the times at which the first and second images are captured are as follows: ΔX = X2 — Xi + Xoffset, ΔY = Y₂ - Yi- Based upon the change in the X₅ Y position of the golf ball and the time between the instances at which the two images are captured, the velocity of the golf ball can be determined as described by the ' 196 application. Similarly, the other parameters associated with the golf shot can be determined as described by the ' 196 application.

In order to improve the resulting accuracy, the imaging system 10 can also include an accelerometer 24 that provides input to the image processor 16 that is indicative of the orientation of the image sensors 12. While described in conjunction with the imaging system of the present invention, the accelerometer of this embodiment of the present invention can be utilized in conjunction with other types of launch monitors, including the launch monitor described by the '196 application and, as such, is not limited to a launch monitor that include a pair of sequentially exposed image sensors. Additionally, the imaging system described above that includes the sequentially exposed image sensors does not necessarily need to include an accelerometer as described herein as the imaging system can be oriented in other manners including manually. The imaging system is generally designed to be disposed in a predefined manner relative to the gravitational field, such as in a level and plumb orientation, in order to appropriately detect the gravitational effects upon the golf ball or other object in motion. As such, the accelerometer of this embodiment will provide data indicative of the relative position of the imaging system with respect to the gravitational field. The image processor 16 can then take into account this data provided by the accelerometer and can correspondingly adjust the parameters that are calculated based upon the captured images to also take into account this offset. As a result, the user need not manually enter or provide the offset, but the resulting measurements of the imaging system will automatically adjust the measurements to take into account any misorientation. For example, in instances in which the forward portion of the launch monitor is tipped downwardly by two degrees relative to the rearward portion of the launch monitor, the accelerometer will provide a signal indicative of the two degree offset to the image processor which will, in turn, add two degrees to the launch angle that is determined for the golf shot so as to take into account the misorientation of the launch monitor.

Accordingly, the imaging system 10 of various embodiments of the present invention that employ the sequentially exposed image sensors 12 can capture precise images having relatively high contrast in order to obtain more accurate measurements of the parameters associated with a moving object, such as a golf ball. Additionally, the imaging system of various embodiments of the present invention is structured in such a manner as to reduce the lighting requirements, both in terms of capital costs and power requirements so as to facilitate the affordability and portability of the resulting imaging system. Additionally, the imaging system of one embodiment can incorporate an accelerometer that can work in conjunction with the image processor to take into account and to correct for any misalignments of the imaging system.

Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

THAT WHICH IS CLAIMED:

1. An imaging system comprising: a plurality of image sensors spaced apart from one another by a predetermined distance and having respective fields of view; a light source for illuminating at least portions of the fields of view of the plurality of image sensors; a control system configured to direct the light source to illuminate the respective fields of view of the image sensors and also configured to direct the image sensors to capture images of a ball at different times; and an image processor for analyzing the images of the ball in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors.

2. An imaging system according to Claim 1 wherein the plurality of image sensors comprises first and second image sensors, wherein the control system is configured to direct the light source to illuminate the field of view of the first image sensor and also configured to direct the first image sensor to capture an image of the ball at a first time, and wherein the control system is further configured to direct the light source to illuminate the field of view of the second image sensor and also configured to direct the second image sensor to capture an image of the ball at a second time, the second time being separated timewise from the first time by a predefined time interval.

3. An imaging system according to Claim 1 wherein the control system is configured to direct the image sensors to capture single exposure images of the ball.

4. An imaging system according to Claim 1 wherein the plurality of image sensors have different respective fields of view.

5. An imaging system according to Claim 4 wherein the plurality of image sensors have respective image planes, and wherein the plurality of image sensors are positioned such that reference vectors normal to the respective image planes of the plurality of image sensors are parallel to one another.

6. An imaging system according to Claim 1 wherein the image processor is configured to take into account the predetermined distance hy which the image sensors are spaced apart in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors.

7. An imaging system according to Claim 1 wherein the light source comprises an array of light emitting diodes (LEDs).

^'8. An imaging system according to Claim 1 further comprising a trigger mechanism for providing a trigger signal to the control system for initiating actuation of the light source.

9. An imaging system comprising: an image sensor having a predefined field of view; a light source for illuminating at least a portion of the field of view of the image sensor; an accelerometer for providing a signal indicative of orientation of the image sensor; a control system configured to direct the light source to illuminate the field of view of the image sensor and also configured to direct the image sensor to capture an image of a ball while the field of view of the image sensor is illuminated; and an image processor for analyzing the image of the ball while taking into account the orientation of the image sensor that is automatically provided by the accelerometer without manual input in order to determine at least one parameter associated with movement of the ball.

10. An imaging system according to Claim 9 wherein the image processor is configured to compensate for the orientation of the image sensor based on the signal provided by the accelerometer in order to determine an angle of flight associated with movement of the ball.

11. An imaging system according to Claim 10 wherein the image processor is configured to compensate for the orientation of the image sensor based on the signal provided by the accelerometer in order to determine a launch angle of the ball.

12. An imaging system according to Claim 9 which is a portable, battery-powered unit.

13. An imaging system according to Claim 9 wherein the image sensor comprises a plurality of image sensors spaced apart from one another by a predefined distance, and wherein the control system is configured to direct the image sensors to capture images of a ball at different times.

14. A method comprising: providing a plurality of image sensors spaced apart from one another by a predetermined distance and having respective fields of view; illuminating at least portions of the fields of view of the plurality of image sensors; capturing images of a ball at different times with the image sensors while the respective fields of view of the image sensors are illuminated; and analyzing the images of the ball in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors.

15. A method according to Claim 14 wherein providing the plurality of image sensors comprises providing first and second image sensors, wherein illuminating at least portions of the fields of view of the plurality of image sensors and capturing images of the ball at different times comprise: illuminating the field of view of the first image sensor and capturing an image of the ball at a first time, and illuminating the field of view of the second image sensor and capturing an image of the ball at a second time, the second time being separated timewise from the first time by a predefined time interval.

16. A method according to Claim 14 wherein capturing images of the ball at different times comprises capturing single exposure images of the ball.

17. A method according to Claim 14 wherein providing the plurality of image sensors comprises providing the plurality of image sensors having different respective fields of view.

.

18. A method according to Claim 17 wherein the plurality of image sensors have respective image planes, and wherein the plurality of image sensors are positioned such that reference vectors normal to the respective image planes of the plurality of image sensors are parallel to one another.

19. A method according to Claim 14 further comprising providing a trigger signal to initiate illumination of at least portions of the fields of view of the plurality of image sensors and capture of images of the ball at different times.

20. A method according to Claim 14 wherein analyzing the images of the ball in order to determine at least one parameter associated with movement of the ball through the respective fields of view of the image sensors comprises taking into account the predetermined distance by which the image sensors are spaced apart from one another.