JP2002248188A - Multimedia analyzing system and its usage - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a multimedia analyzing system to obtain and compare a player's exercise performance. SOLUTION: This system includes at least one video camera to collect a plurality of video data sets of the player performance which is recorded with at least one video camera placed almost the same position. This system also contains at least one memory device to store a plurality of video data sets, at least one processing unit to mix at least two video data sets and a display unit to display at least two mixed video data sets. The processing unit just almost synchronizes and superimposes at least video data sets to form an aggregate video data set. Also a shot-monitoring unit can be included to collect a plurality of flying data of the material hit by the player.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an apparatus and method for evaluating athletic performance, and more particularly to an improved multimedia analysis system and method for analyzing sportsman performance, and an apparatus used therefor. In particular, the present invention relates to multimedia analysis systems and methods for evaluating golf player skills. The invention also relates to the generation of a library of data sets on subject performance in a set of known conditions. Furthermore, in particular, the invention allows for the generation, archiving and comparison of data sets on athlete performance during test sessions under various test conditions.

[0002]

2. Description of the Related Art There is known a technique for capturing an image of a series of actions used to evaluate the performance of an athlete. Such captured images can be used on educational equipment to teach and demonstrate play characteristics. Athletes and coaches are also caught to evaluate and compare athlete skills with the preferred skills of the skilled person and to indicate points of development therefrom to improve overall performance. Use the image. Sports that benefit from analysis of a series of actions include, but are not limited to, golf, tennis, baseball, skiing, and bowling. An example of the use of visual recording in sports is disclosed in US Pat. No. 5,797,805. The partially pre-recorded golf lesson videotape includes a blank at a predetermined location into which the player's golf swing and the full motion video of the selected still frame are inserted. In order to be able to create a split screen showing the positions of the player and the professional golfer at various positions along the golf swing, and a side-by-side view, the stationary frame should be aligned with the position of the professional golfer in the corresponding stationary frame. Selected. Another exercise technique video training device is disclosed in US Pat. No. 5,249,967. The device allows a student athlete to acquire dynamic skills by rigorously modeling recognized experts. A set of video cameras sends the student's live view from the direction that shows the most skill. The video overlay generator receives live images and is still in video signal format,
The images are combined with a corresponding set of self-generated template images that display the expert's dynamic skills in the preferred exercise situation in the form of static contours. For golf, contours are used to delineate the trajectory of each tip of the expert during the performance of the technique; for example, the trajectory of the expert's hand during a swing is shown as a continuous or intermittent contour. Different contours are used to indicate the trajectory of the golf club head during a backswing, and further contours are used to indicate the position of the golf club head on a downswing.
U.S. Patent No. 5,486,001 further discloses a personal education aid that allows and assists a person to follow predetermined movements, such as a golf swing. A video camera is used to capture the movement of a person performing a golf swing, and biomechanical information can be obtained from mechanical devices attached to the person or a golf club moved by the person, or weight or load sensing pads, monitors, electron beam devices, etc. Collected from the detector. The computer database includes selected pre-recorded digital form video signals, such as a swing selection, indicating the movement of another professional golfer to emulate. The computer will either display a split screen format with images of the preferred technology displayed alongside the current technology, or in the form of line drawings or detailed drawings, to show the viewer the difference between his stroke and the preferred stroke For example, an audio-video display is generated in a format in which a suitable technology is superimposed on the current technology.

[0003]

Despite these developments, there is a need for improved multimedia analysis systems and methods for use by sportsmen. In particular, there is a need for a real-time video-based system for collecting and archiving relevant impact data corresponding to individual golf swings, golf clubs and golf balls. Further, there is a need for a system that not only allows equipment adjustments and equipment test comparisons, but also allows an individual to compare personal performance to different practice sessions. There is also a need for a system that can also collect data on various equipment and practice conditions, as well as other personal characteristics that change over time. An object of the present invention is to provide an apparatus and a method capable of providing these improvements.

[0004]

Thus, the present invention relates to a method for mixing data relating to a player's athletic performance. The method includes: defining a first set of test conditions applied to the player; collecting a first set of video data relating to a first performance of the player; the first set of video data being a first set of test conditions. Storing a first set of test conditions and a video data set; defining a second set of test conditions adapted to the player; the first and second sets of test conditions are at least one common Collecting a first video data set for a second player performance; a second video data set is collected during a second test condition set; a second test condition and video data Storing the set; a first and a second video data set to form a superimposed set. It mixes at least a portion of; and displaying a superimposed been set. The first and second video data sets can each have a single video frame, or alternatively, each have at least two video frames. Preferably, the superimposed video data sets are generally synchronized from a start time to an end time. A video camera with a capture rate of at least about 250 frames per second can be used to collect the first and second video data sets. The mixing of video data consists of the first and second
At a first percentage of pixels in each frame selected from one of the first and second video data sets, and a second percentage of pixels in each frame selected from the other of the first and second video data sets. It is possible. Preferably, the first percentage is between about 20 to 40 percent, the second percentage is between about 60 to 80 percent, and the first and second percentages are between about 60 to 80 percent.
The sum of the percentages is 100.

[0005] The method further comprises: collecting a first non-video data set for the first time player performance; the first non-video data set is collected during a first test condition set; Storing a video data set; collecting a second non-video data set for a second player performance;
Non-video data sets are collected during a second set of test conditions; storing the second non-video data set; forming first and second non-video data sets to form a superimposed non-video data set. Mixing at least a portion of the non-video data set; and displaying or playing at least a portion of the superimposed non-video data set. Preferably, the superimposed non-video dataset is substantially synchronized from the start time to the end time, and the superimposed non-video dataset is substantially superimposed from the start time to the end time. Synchronized with the paused video dataset. The non-video data is at least one of audio data and launch monitor data.
It is possible to include one. In a preferred embodiment, the first
And the second video data set is collected using at least one analog or digital video camera, and the first and second non-video data sets are collected using at least one of a microphone and a launch monitor. Collected. Each set of test conditions may include at least one of a player's equipment, environmental conditions, and personal characteristics of the player. Mixing the video data may include using at least one key effect, and preferably the video data is from at least two video cameras located at approximately the same angle and / or approximately in the same plane. Including data.

[0006] The present invention further relates to a method of overlaying a data set of athletic performance of a player. The method includes: defining a plurality of test conditions applied to the player; collecting a plurality of video data sets relating to the performance of the player; each video data set has at least one common test condition;
Storing a plurality of test conditions and a video data set; mixing at least two, at least a portion, of the video data set to form a superimposed set; and displaying the superimposed set. including. The method further comprises: collecting a plurality of non-video data sets for player performance, each non-video data set having at least one common test condition; storing a plurality of non-video data sets; Mixing at least a portion of at least two of the non-video data sets to form a paused non-video data set; and displaying or playing the superimposed non-video data set. is there. Also, the method can include synchronizing the plurality of test condition sets, the video data set and the non-video data set.

[0007] The present invention also relates to a multimedia analysis system for collecting and comparing athletic performance of players. The system includes at least one video camera for collecting a plurality of video datasets of the player performance, at least one storage device for storing the plurality of video datasets, and at least two of the video datasets. At least one processing device for mixing and at least one display device for displaying at least two mixed data sets. The processing unit substantially synchronizes and superimposes at least two video data sets to form an aggregate video data set. Preferably, the position of the at least one video camera is approximately the same when collecting each video data set. The system may further include a launch monitor for collecting a plurality of flight data sets of an object struck by the player, and at least one processing unit for mixing at least two of the flight data. It is.
The at least one processing unit is configured to substantially synchronize the at least two flight data sets to form an aggregate flight data set, the aggregate flight data set also being synchronized with the aggregate video data set, and to superimpose the at least two flight data sets. Let it. The plurality of video data sets can be collected by at least two video cameras aimed at the player positioned at substantially the same angle and / or substantially at the same plane.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIGS. 1-3, a multimedia analysis system 110 constructed in accordance with the present invention is shown in a preferred form with respect to a golfer 110. Although the preferred embodiment described herein refers to the sport of golf, the present invention is useful for examining deviations in technical approaches and other performance indicators.
It is possible to adapt to analysis of player performance and equipment handling in other sports or activities. Preferably, the motion analysis system 100 optionally includes four high-speed video cameras 120, 122, 12 located at various locations around the golfer 110 in the space 105.
4, 126. Preferably, the golfer 110 takes a golf swing, standing on a mat or other surface 108 facing the golf ball 114 on the tee gripping the club 112 and fixing the tee relative to the mat 108. Positioned. The club 112 is shown in FIG. 1 as a wood type golf club,
The present invention can also be used with other clubs, such as irons and putters. Motion analysis system 10
0 is adaptable for use by golfer 110 under various test conditions, such as golfers with different physical characteristics, and will be discussed briefly. The location of the mat 108 and the space defined thereby generally defines a central focal region in which the various components of the motion analysis system 100 are located. A computer system 130 and a display 163 are included, which will also be described briefly.

Data collection for golf ball trajectory and flight characteristics is preferably facilitated by the use of a golf ball launch monitor 118, as shown in FIG. The launch monitor 118 is located adjacent to the mat 108 in front of the golfer 110. The launch monitoring device measures golf ball flight characteristics and club head swing characteristics. Examples of launch monitors suitable for use in the present invention include:
US Patent No. 5,501,463 and US Patent No. 5,5
No. 75,719, both issued to Gobush et al., Directed to "Apparatus and Methods for Determining the State of Movement of Object Striking Tools", and to Gobush et al., "Launch Monitoring with Scale Fittings." Apparatus and method of using the same "
No. 09 / 537,295, which is incorporated herein by reference. All three references are assigned to Akshnet Company, the disclosures of which are incorporated herein by reference.

[0010] The data collection further includes audio data regarding the golf ball impact. Such data is useful, for example, for synchronizing video data and can also be an indicator of ball hit quality. Audio and video data sets from a data session can be initially synchronized with each other and can be synchronized with audio and video data from other test sessions. For example, a particular audio frequency "signature" can be used to determine the moment a golf club first impacts a golf ball, and multiple data sets can be synchronized in time. A microphone 116 is provided very close to the golf ball on the tee to receive sound. Various audio processing technologies
U.S. Patent No. 6,124,895 to Filder in Dolby Laboratories Licensing, Inc.
ratories Licensing Corporation) and can be used to synchronize audio and visual data, including those disclosed in "Frame-Based Audio Coding for Audio / Visual Data Synchronization with Dynamic Audio Frame Alignment".

Preferably, the cameras 120, 122, 12
4, 126 are located with respect to golfer 110 to facilitate video collection of golf swing, ball impact and ball flight. To this end, a preferred embodiment of the motion analysis system 100 is shown, for example, with respect to a right-handed golfer 110, although the present invention is readily adaptable to left-handed golfers. When the golfer 110 addresses the ball 116 on the tee, the front camera 122 has a line where the front of the golfer 110 can be seen, the rear camera 120 has a line where the back of the golfer 110 can be seen, and the rear camera 12
4 has a line in which the side of the golfer 110 can be seen, and the upper camera 126 has a line in which the overhead of the golfer 110 can be seen. In an alternative embodiment, only one camera may be provided for video data collection, and preferably, the motion analysis system 100 includes at least one camera positioned at a location having a line through which the golfer 110 can see. Including camera.

In a preferred embodiment, the cameras 120, 12
The positioning of 2, 124, 126 allows simultaneous image acquisition from various body fluoroscopy, thereby allowing more data acquisition and, at the same time, more detailed analysis. Preferably, the cameras 120, 122, 124, 126
Are coupled to a permanent or semi-permanent mounting that ensures repeatability of data collection conditions from one session to another. More preferably, the cameras 120, 122, 12
4, 126 are connected to permanent or semi-permanent mounting fixtures located towards the peripheral area of the space at least partially enclosed, such as the space 105. More preferably, the camera 12
0, 122, 124, 126 are connected to permanent or anti-permanent brackets 128 mounted on the walls and ceiling of the space. Ball 1 put on the tee by golfer 110
When hitting 14, use the net to get the ball 1
Fourteen flights are unrestricted and unrestricted. Beneficially, the preferred configuration of the present development allows golfer 110 to hit the ball on a fairway, for example, golfer 110 can see the ball fly. In another embodiment, a net is used to allow positioning of the camera on the opposite side of camera 124, but it may not be feasible to see the cleanest lines.

With particular reference to FIGS. 2 and 3, in a preferred configuration, cameras 120, 122, 124, 126
It is aligned with 40 and straight line. Preferably, the cameras 120, 122, 124 are generally coplanar. In addition, camera 1
20 and 122 are parallel vertical axes 132 and 134, respectively.
And are coaxial along the axis 144 and aim at each other. In the embodiment shown graphically in FIGS. 2 and 3, the cameras 124, 126
It is located along a central plane containing 42, 143 and perpendicular to the plane of the page. Preferably, the golf ball 114 on the tee is parallel to the axis 142 and the camera 120
Positioned along axis 146 closer to camera 122 (eg, a right-handed player is discussed herein), and positioned along axis 148 parallel to axis 144 and closer to mat tip 150 than camera 124. . Further, preferably, the placement of the ball 114 on the tee is about a distance d1 of about 0 to 6 inches from the axis 144, and the axis 142
At a relative position (offset) with a distance d2 of about 24 to 48 inches. Further, preferably, the ball 114 on the tee is in a relative position (offset) with a distance d1 of about 2 to 4 inches from the axis 144 and a distance d2 of about 36 to 48 inches from the axis 142.

Preferably, the cameras 120, 122, 124
Is mounted at the top of the mat 108, at a height d3 of about 2 to 6 feet vertical, and more preferably the golfer 11
Install it no higher than the waist of 0 and higher than the sternum of the golfer 110. Most preferably, the cameras 120, 12
2, 124 are mounted at a height d3 of about 4 feet. Further, the cameras 120 and 122 are respectively coaxial
42, 143 and horizontal distances d5, d6 from the parallel axes 132, 134, respectively, and the camera 124
Is mounted at a horizontal distance d7 from the axis 144, preferably the distances d5, d6, d7 are each about 8 to 14 feet, more preferably about 10 to 12 feet.

Preferably, the camera 126 is a golfer 11
It is mounted at a vertical height of at least 8 feet above zero, and more preferably about 10 feet above the golfer 110. Most preferably, camera 126 is mounted on mat 108 at a vertical height d4 of about 16 feet. Cameras 120, 122, 124, 126
Is preferably a high-speed camera. In a preferred embodiment, the camera is capable of collecting video data in black and white as well as color in digital form. Suitable cameras include those commercially available from the Motion Analysis Systems Division of Eastman Kodak Company, which can collect data that meets a variety of standards, including image type, resolution, shutter time, capture rate and communication standards. To Preferably, black and white image acquisition is accomplished by using a Kodak Model SR-1000 motion coder analyzer digital camera with a recording speed of 1000 frames per second and a 658 (H) x 496 (V) pixel sensor array. Such CCD cameras provide progressive scanning (non-interlaced), enable electronic zoom, and provide digital data output of 8-bit monochrome TIFF files carried by a SCSI-2 port to a personal computer.

Most preferably, color image acquisition is achieved with at least one of the four cameras. A suitable device for collecting color images is a Kodak EKTAPRO HG-2000 color digital camera with a recording speed of 2000 frames per second and a resolution of 512 (H) x 384 (V) photosensitive pixels at 24-bit color (this type). Or provide an 8-bit black and white image). The HG-2000 camera is a compact (Bayer) or 24 PCMCIA hard drive
Digital images can be stored in bit color TIFF format or during download to a solid state storage card. In addition to serial communication, the device uses the UDP / IP protocol to communicate with a dedicated 100 Base-T physical link.
Enables 00 Mbps Ethernet communication. The video tape recorder (not shown) preferably also stores the output from cameras 120, 122, 124, 126. Suitable equipment is Sony UVW-1800, which can use L-size metal beta cam SP type cassette.
An editing recorder / player, but most other recorders / players and formats are also available. Lighting systems similar to those used for functional film manufacturing (not shown) can be used to further enhance the image acquisition of multimedia analysis system 100. In particular, the choice of lighting can have a significant effect on the quality of the mixing of the data set. Thus, in accordance with the principles of the present invention, some powerful crystal halogen bulbs can be precisely positioned and aimed at the background or wall so that the background is uniformly illuminated with light, avoiding shadows. It is possible.

Preferably, the multimedia analysis system 1
10 further includes at least one computer system 130 used, in particular, for data collection, archiving and comparison tasks. Computer 130 typically includes a processing unit such as a central processing unit or a CPU. It is possible to utilize parallel processing such as multiple or distributed processing devices. Hard disk, CD-ROM, magnetic force optics (optical)
A storage device such as a drive, tape drive or other suitable storage device is also included and is typically coupled to the processing device. Also included is a primary storage device, having a random access storage device (RAM) for storing program instructions and data for processing operations of the central processing unit. The primary storage further includes a permanent storage (ROM) for storing basic operating instructions, data, and objects used by the computer to perform its functions. Computer 130
Typically also includes one or more input sources, including a keyboard, mouse, stylus, and / or
Or as another suitable device. Furthermore, it is possible to provide a network connection. Those of ordinary skill in the art can also anticipate other configurations and related techniques of the computer 130 and use it immediately for the present development. Referring to FIG. 4, in one embodiment of the present invention, the cameras 120, 122, 124, 1
26 transmit signals 152, 154, 156, 158, respectively, to a signal / data processor 160, which can perform various functions including data storage, data distribution, and data filtering. The processing unit 160 is effectively associated with the computer system 130.
May direct and / or perform the primary functions of data collection, archiving, and comparison, and may effectively be associated with removable media storage 162. Preferably, computer system 130 includes one or more displays 163 and removable media storage 162 includes one or more video cassette recorders / players. Computer system 1
30 may also be more effectively associated with mass storage system 164. Each of the components or systems may also be effectively associated with a network 166. For example, network 166
May include servers, routers, etc., and additional computers that allow data, instructions and / or messages to be sent to networked components or systems. Additional computer system 13
0 allows connection to the network 166. Independent network connections are also provided for one or more components or systems. The design, configuration and implementation of a computer network suitable for the present invention can be accomplished in a number of ways known in the art.

In a preferred embodiment, the multimedia analysis system 110 is configured to enable the generation of a dataset library related to the performance of the golfer 110 in a known set of conditions, and
In particular, it allows for the generation, archiving and comparison of data sets regarding the performance of golfer 110 between test sessions based on various test conditions. Test conditions provide a protocol used to generate comparable data from one session to another. The focus of each golf session includes, in part, data collection on these test conditions. It is possible to collect a wide variety of data on test conditions. For example, the equipment used by golfer 110 is particularly relevant to the analysis of test sessions. The range of tools selected by the subject matter includes the type of club used (eg, wood or iron), the structure of the club (eg, material, weight, grip structure, shaft type, overall length, hosel configuration), and use. It also includes the type of ball (eg, wound, multi-layer, liquid-filled, dimple type), and even the type of shoe used by golfer 110 (no spike, spike).
Environmental conditions can include temperature, humidity, time of day, season, and lighting. The personal characteristics of the golfer 110 may include age, height, weight, posture, visual acuity, health condition, degree of self-confidence, and degree of stress. Since some test conditions, such as the weight and eyesight of the golfer 110, may change over time, recording these variables may allow the instructor or other professional to use the golfer's 110 skills or equipment in handling the equipment. Many factors contributing to change can be evaluated.

[0019] Advantageously, the archiving of the data set showing the golfer's performance between a series of individual test sessions is performed by the golfer during different times and possibly at different stages of the golfer's career. And the resulting firing states can be compared. This can be useful, for example, for analyzing deviations in performance resulting from changes in swing speed or body positioning. Furthermore, as the golfer's health varies over the course of his career, it is possible to follow the skill deviations resulting from medical procedures and prescribe corrective procedures. For example, if golfer 110 has undergone shoulder or ankle surgery, the golfer's 110 swing may be affected. Golfer 1 by comparison before and after
It is possible to adjust the swing method so that 10 can be adapted to physiological conditions and potentially return to the pre-operative form. For purposes of the present invention, a data set includes, but is not limited to, a series of individual video frames or portions of individual video frames, as well as audio data, test condition data, and data from devices such as ball launch monitors. It is not something that can be done. Suitable video capture software and hardware are available from Avi Technologies, Inc. of Chukesbury, Mass. (Av.
id Technology, Inc.). In particular, Avi
d Xpress hardware and software enable real-time video editing. The Avid Xpress system uses Avi
d Xpress editing software and hardware, Avid Mer
idien video, audio, effects, display and drive controller hardware, and analog (component, S-video, composite) and digital (SMPTE-259) video I / O and analog, S / PDI
Includes Meridien input / output (I / O) hardware for connecting F-digital and AES / EBU digital audio I / O. Output options include web video server, DVD and CD formats.

Preferably, computer system 130
Are Power Macintosh G3 or G4 with Avid PCI Extender or IBM IntelliStation MPro running Windows NT®. The video features provided by the Avid system are as follows:
ITU R-601 broadcast industry standard image output (720 × 486 N
TSC or 720x576 PAL), image compression, 4 video tracks, offline (draft mode) and online image quality, Sony RS-422 serial and VLAN deck control, batch digitizing, waveform monitor and vectorscope, SMPTE timecode support, integration EDL
16: 9 editing and 1: 1 uncompressed options for import / export, widescreen and DVT. For audio features, Avid Xpress offers: 8-track, real-time audio mixing, 48kHz
z, 44.1 kHz sampling rate, audio punch-in, real-time rubber band in timeline
For gain adjustment, real-time audio input / output level adjustment, volume and pan control between channels by clip or track, display of audio waveform and scan audio, 2-channel or 8-channel audio I / O, and export of CD-ROM 22k of audio
Downsampling to Hz / 11kHz and other audio mixer functions.

With respect to editing capabilities, Avid Xpress is suitable for the present invention, in part because Avid Xpress provides: tight timeline editing, single track ton transition, snap to transition,
Slip and slide editing, splice, overwrite, extract, replace, extend, lift and split editing, sync clock / sync break detection, dynamic storyboard, accurate timecode editing, pop-up source monitor, match frame, shift, consolidation Date and collapse, built-in logging, timeline view function, VTR playback emulation, and remote / offline logging. The present invention
Enhanced Superimpose Data Set and Title
Utilizes Avid Xpress with modifications to allow communication with eist launch monitoring technology.

As will be described next, the present invention makes it possible to utilize a data-rich archive of a golfer's past performance to provide a real-time comparison with the current performance. Generally, individual sources can be combined by such mixing. Of current and past performance,
Simultaneous or near-simultaneous mixing is achieved using the present invention. Also, two or more data sets of the golfer's past performance are mixed, allowing comparison of the archive-only test sessions. Preferably, the aforementioned hard disk drive, CD-ROM, laser disk (registered trademark), DV
Data sets are archived in analog or digital form using storage media such as D and videotape. In particular, data collection from video, audio, and other ball launching devices requires two per golf club swing.
Typically, a sub-second time frame is required. The average golfer makes a swing that takes about 0.9 to 1.5 seconds to complete, and most golfers make a full swing in about 1 second. Therefore, the image acquisition of the swing selection is preferably achieved by using a high-speed camera that shoots at a speed of 2000 frames per second. This data collection rate allows the aggregate dataset to be analyzed immediately on a subset of the aggregate with a sufficient number of frames to evaluate a particular range in time or the golfer's swing phase. , It is possible to compile to a sufficient size. Various fast or relatively fast capture rates, such as 500 frames per second, are used in accordance with the principles of the present invention. Multimedia analysis system 10
Some embodiments of 0 include cameras operating at different capture speeds simultaneously, such as one camera operating at 500 frames per second and another camera operating at 250 frames per second. In some embodiments, a capture rate of at least about 250 frames per second is used. One data collection session can be considered as a single swing, or alternatively, can be defined as a single swing or as part of multiple swings. Session times may occur close together (eg, on the same day) or separated by long periods (eg, different years).

In accordance with the principles of the present invention, multimedia analysis system 100 may include a video, audio, and other video data collected during a golfer's performance session, preferably based on at least partially repeated test conditions. Enables mixing of data sets. Mixing is typically accomplished by using one or more specific effects known in the art and incorporated into the present invention. For example, a mask can be cut off to edit one or more portions of an image. The effect is that some parts of the image are protected from modification. Another effect, matte in the form of a video or image, can be used to create a separate area where additional video or images appear. For video data sets, several "key" effects are known in the art (e.g., placing one image on or inside another image), which facilitate mixing of the data sets in the present invention. I do. Although the naming of key effects varies by device manufacturer, commonly used key effects include chroma keys, luminance keys, external keys, and download keys. In essence, each key effect allows at least two video sources to be combined. A first object can be defined in a first video source and can be used, for example, to define in or around an area where a second video source is located.

Most well-known is the chroma key effect, where a reporter is used for a television weather forecast standing uniformly in front of a blue or green wall that the computer replaces with a television viewer weather map. Is typical.
In this method of combining multiple video images, one video is superimposed on top of another video, which is a specific hue separated by a computer. Each key effect is generally characterized by its ability to provide a superimposition of one video over another. As used herein, the terms superimpose, superimpose, superimposition, etc., include but are not limited to effects, known as layers, overlays, ghosts or source synthesis, and, preferably, Mentions a range of key effects that have been standardized by the industry to allow the combination of multiple image / video sources in a synchronized format. Also, the terms superimpose,
Super imposing, super imposition, etc.
It can accommodate non-video data correlations such as audio data or launch monitor data from multiple data collection sessions. Furthermore, the terms are applicable to frame-by-frame analysis, or analysis of parts of several frames. For example, a frame representing a single still image from a video stream can be superimposed over another frame to identify differences between frames.

The present invention utilizes the principle of A / B role editing, which is more commonly referred to as two-source or multi-source editing. Video mixers and video images can be combined with sophisticated transition and synchronization controls. Mixers are commonly used to synchronize multiple sources, for example, multiple video sources, with each video frame of a first source starting at the same moment as each video frame from another source. Thus, in the present invention, when superimposing a data set, synchronization can be used so that the data set has the same start and end times "in step".

The graphics, title creation and digital effects used in the present invention are as follows:
They are also real-time effects: transitions, dissolves and horizontal, vertical, and box wipes, motion effects, superimposed fast rendering, diagonal lines, matrices, saws and shape warps, matte keys, chroma keys and luminance keys, flips, flops. Resize, peel, push and spin, conceal, L conceal and squeeze, page curling, mask, picture-in-picture (2D or 3D), compressed keyframes, limited or unlimited titles to motion graphics and effects, Nesting (composite into multiple video tracks and single track), 2-stream color effects with keyframes, fast rendering of masks Grayed, title (three-dimensional), AVX (Avi
dVisual eXtention) Plug-in, customizable 2D and 3D DVEs, x, y, z position and rotation, fresh frame search and black hole function. Synchronization is particularly important to ensure that each video frame from the first source starts at the same time as each video frame from the second source. For example, a video consists of a number of frames that are shown sequentially at high speed. Each frame in turn contains hundreds of horizontal lines. Two general standards for frames are North American NTSC (NT
SC method) and European PAL (PAL method). NTSC frames each have 525 scan lines and a refresh rate of 30 frames per second, while each frame in the PAL scheme has 625 scan lines and has a refresh rate of 25 frames per second. Further, each frame has two fields, and scan line 1 (eg, 1, 3, 5,.
7) and a second field representing every other scan line starting with scan line 2 (eg, 2, 4, 6, 8,...). It is.
The difficulty that the combined independent video sources may face is that the first source starts at, for example, scan line 82,
At the same time, the second source starts at scan line 5. The frame synchronizer electronics and / or software can delay one of the two sources of synchronizing the sources, thus avoiding picture shifts or wavers. Further, a time base collection is performed. Synchronization of selecting a common starting point is also particularly important when trying to view multiple sessions together. For example, the impact point of the golf ball and the club can be used as a time "zero" for each of the two data sets to synchronize the data.

The multimedia analysis system 100 uses the BM
Use standard video graphic output types such as P, TIF, GIF, PIC, JPG, SGI and TGA. In addition, the multimedia analysis system 100 supports not only standard digital sound formats such as PCM, but also AVI, JPEG, MPEG, JMPEG.
Use a standard motion video format such as. 2
With respect to the superimposition of one video set, the superimposition takes various forms. For example, golfer 110 swings in two separate cases. Advantageously, the video cameras 120, 1
Since the positions of 22, 124 and 126 are fixed, 2
Data collection between two sessions can be done in a repeatable manner.

In one embodiment of the present invention, a first data set is collected during a first golf swing session,
Stored for later use. A second session then occurs while the exemplary golf swing is selected for replay from the first data set. Preferably, the computer system 130 is used to store each data set, select the desired data set and, if necessary, a subset thereof, and process and play back the data set. Video data from one or more viewpoints can be displayed on one or more display devices 163, such as a computer monitor, television or projection system, and non-video data can also be displayed. . Non-video data includes an estimate of where the golf ball will land (carry), how far the golf ball will roll (total shot distance), and whether the ball will hook or slice, and how much line the ball will have. Including information about the trajectory of the golf ball, as well as other flight information such as whether it moves off,
Contains information from the golf ball launch monitor 118. Preferably, one or more figures include superimposed data.

In one embodiment, shown in FIG. 5 (a), two views of golfer 110 captured by rear camera 124 during separate sessions are displayed on screen portion 170,
172 is shown on the display device 163 in a split screen format. Although display 163 is shown as a computer monitor, other displays such as televisions or projection screen systems can be used. The first video dataset 180 of the golfer 110, captured during the first session, is
70, while the second video data set 190 of the golfer 110 captured during the second session is displayed in the right portion 172. Turning to FIG. 5 (b), the first and second video data sets 18
If 0 and 190 are superimposed respectively,
The composite video data 200 is shown on the display device 163.
The synthesized video data 200 is a video data 18 with an outline.
0, 190, which is achieved by suitable data filtering described herein using mixing techniques. Alternatively, as shown in FIGS. 5 (c) and 5 (d), the first and second video data sets 180, 190 are each provided with a solid color, so that each data set of the golfer 110 can be easily obtained. Can be distinguished from others. In FIG. 5 (c), a first video data set 180 is shown in the foreground,
On the other hand, in FIG. 5D, the second video data set 1
90 is shown in the foreground. In a mutual embodiment,
One of the displayed data sets may be a data set showing the golfer 110 seen by the camera without contouring or other data filtering. Similarly, the representation of the data set is partially or completely shown in color rather than black and white. Further, the entire background around golfer 110 is shown without removing that portion of the data from the dataset. Superimposition of images from at least two data sets occurs at various percentages. For example, in one embodiment, 30 percent of the pixels of each image in the first data set are superimposed on 70 percent of the pixels of each image in the second data set. In another embodiment, 50 percent of the pixels of each image in the first and second data sets are superimposed. In yet another embodiment, about 20 to 4 pixels of each image in the first data set.
Zero percent is superimposed on about 60 to 80 percent of the pixels of each image in the second data set.

The present invention further includes displaying a portion of golfer 110 from one or more data sets. For example, if the body movement of the golfer 110 is analyzed, the reminder of the golfer 110 is selectively removed from the masked or otherwise captured image, thereby focusing on the processing area. Becomes possible. If only the foot placement is analyzed, a zoomed view of the lower limb and foot placement of the golfer 110 during various stages of the swing can be selectively displayed. In each example, as before, the superimposition of the dataset occurs at various percentages. in this way,
The present invention provides an instrument that focuses on analysis for particular aspects of the golfer's method. As is apparent from FIGS. 5A to 5D, the golfer 110 can change the swing method from one data collection session to another data collection session. For example, as shown in the figure, golfer 110 may move foot to video data set 18.
0 and 190 are in the same position, but golfer 1
No. 10 changed the posture and the grip position of the club. The composite video data 200, 210, 220 can be viewed in various ways. In one embodiment, the first and second video data sets 180, 190 are collected during separate sessions and analyzed following completion of both sessions. In another embodiment, the first and second video data sets 18
0, 190 are shown at the same time as the actual acquisition of the second video data set. In some implementations, the golfer 110 includes the first and second video data sets 18.
Synthetic video data 20 until after data collection of 0 and 190
0, 210, 220 cannot be seen, while in other embodiments, the first and second video data sets are within the field of view of golfer 110, such as on a projection screen television positioned behind the net. It can be shown on the arranged display device 163.

In another embodiment, images captured from multiple cameras can be displayed simultaneously. As shown in FIG. 6, a composite view 220 of the golfer 110 captured by the rear camera 124 is shown in the screen portion 226 of the display device 163, while an additional composite view of the golfer 110 captured by the front camera 122. 23
0 is shown in the screen portion 227. Another screen part 22
8 is provided for displaying additional data collection for golfer 110, such as the test conditions described above. The present invention
It can also be used to obtain a video data set that contains only two colors, but does not include details such as clothing or facial expressions. Although software filtering and effects can be used for this purpose, most particular attention to the setup of certain instruments included in the multimedia analysis system 100 also facilitates such data collection. For example, by filling the surroundings of a golfer 110 with light, one or more cameras 12
0, 122, 124 and 126 can capture only black and white images, mainly only non-grayscale images. In other words, individuals show a solid black body on a white background. Computer 163 and associated software can be used to invert the colors of the image, if desired.

[0032] Preferably, data captured by the camera is received and processed by computer system 130. The following components are included in the multimedia analysis system 100 to facilitate data collection and use: digital capture and overlay board; receive some video signals and select for display screen, storage and processing components Video screen switching device that allows the video signal to be transmitted; video contrast enhancers to modify the video signal, such as by using shading compensation, brightness compensation or preselected gray scale to introduce picture contrast; Split screen generator for displaying several images from different video sources simultaneously on the same monitor screen; video time base collector for stabilizing the playback of video tape recordings and synchronizing different video sources; two images Among And physical movement of the time-exposure image generated video peak storage device; video subtractor for displaying are. In another aspect, a non-video data set is mixed. For example, a flight dataset collected by the launch monitoring device can be displayed on the display screen 163, such as by generating a graph or chart of another dataset.

Referring now to FIG. 7, according to the present invention,
An operator powers on various components of the multimedia analysis system 100. Next, the computer processing unit 130 starts up, thereby erasing the computer memory, setting up the device such as a terminal, loading software information into the memory of the computer system, and loading the operating system into the memory (step). 300). After startup of the computer system 130, video capture, audio capture and other data capture and mixing software are executed so that the operator can subsequently perform data collection, manipulation and analysis (step 310). Cameras 120, 1 and 2 associated with the multimedia analysis system 100
22, 124, 126 and the lights are then prepared for image capture, examining and / or setting the appropriate device settings (step 320). In addition, the launch monitoring device 1
18 is calibrated by checking and / or setting the appropriate launch monitor settings (step 330). Optionally, the video splitter box is inspected and / or configured,
One or more images or motion streams can be displayed in one or more quadrants of a display device, such as monitor 163 (step 340). With the various components of the multimedia analysis system 100 in the operating mode, the operator of the multimedia analysis system 100 may collect data into the computer system 130, such as by starting a data collection operation and pressing a start key associated with the mixing software. To start a data collection session (step 350). Preferably, the software is Avid Xpress real-time video editing software. The operator
The golfer 110 is instructed to start the swing flow (step 360), and the golfer performs the swing flow (step 370). Cameras 120, 12
The images captured by 2, 124, 126 are digitized, the sound captured by microphone 116 is digitized, and the ball launch condition and flight characteristics observed by launch monitor 118 are resolved (step 380). . Data from the session, such as video data, audio data, and launch monitor data, are then consolidated and correlated by data capture and mixing software as needed. Historical data, such as an image or a series of images, can be retrieved from the mass storage device (step 400), and a data capture and mixing software routine can be invoked to enable mixing (step 410). A software routine is invoked to correlate the aggregate history dataset with two or more datasets, such as the aggregate dataset just collected (step 420). The term aggregate dataset preferably refers to a dataset containing video, audio and launch monitor data from one session. The correlated data set is processed using mixing software routines to achieve appropriate effects, such as superimposing the data set on the video output. The correlated data is displayed in one or more formats, such as a comparison of test condition data from a plurality of correlated video image streams, a plurality of correlated audio streams and / or data sets (step 430). ).

Referring to FIG. 8, according to the present invention, the data set is superimposed using mixing software. The software operator first selects and marks the “in” point of the first data set,
The point is typically selected to be one frame before the beginning of the swing flow (step 50).
0). The first dataset, or a portion thereof, is then stored as a timeline-based stream such that playback of the stream of the first data set occurs for a period tracked by the timeline (step 5).
10). The software operator then selects and marks the "in" point of the second data set, which is also typically selected to be one frame before the start of the swing flow. (Step 520). Second
, Or a portion thereof, is then stored as a timeline based stream (step 53).
0). One or more mixing software devices are then selected by the operator to apply the desired effect to the first and second data sets (step 540). The data set is then displayed in one or more formats, such as a comparison of a plurality of correlated video image streams, a plurality of correlated audio streams and / or test condition data from the data set (step 550). . Preferably, when analyzing a golf swing from start to finish, the selection of the "in" point starts at least about 0.75 seconds before impact of the golf ball, since this is a golf swing and This is because it is a typical time frame of the contact between the golf club and the golf ball. In addition, the first and second data sets preferably end in a minimum of about 0.25 seconds after the impact of the golf ball, which means that the first and second data sets have the impact of the golf club and the golf ball and the completion of the golf swing. Because it is a typical time frame.

While various descriptions of the invention have been described above, it should be understood that various features can be used alone or in combination. Accordingly, the present invention is not limited to the particular preferred embodiments described and indicated herein. Many variations or modifications fall within the scope of the invention. For example, the required number of cameras can be used to compile the three-dimensional data set so that the golfer 110 can be viewed from any angle, and by using visualization techniques known in the art, the Mixing of dimensional video data sets is possible. In another embodiment, multimedia analysis system 100 can be configured to be configurable. Calibration devices can be provided to accurately position cameras, mats and tee-mounted balls for data collection, and can include devices capable of laser distance measurement. A level sensor is also provided. With such a device and a portable computer, the multimedia analysis system 100 can be easily transported and begin collecting golfer swing data at locations such as golf course fairways, prospots or trade shows.

The present invention is particularly suitable for athletic training, but other uses are contemplated, especially in areas where emulation or precise motion flow is desired. For example, the present invention may be used to train physicians to perform smaller surgical procedures and / or to perform certain surgical techniques.
It is possible to design on a smaller scale for more restricted fields. Physicians pioneering a particular medical procedure can use the devices and methodologies of the present invention to practice the various stages of the procedure without requiring the presence of a living subject. Alternatively, work techniques such as welding can be improved by practicing the present invention. Police officers can also use the present invention for small arms training, for example, a first-class shooter. Therefore, it should be understood that modifications and variations within the spirit and scope of the invention will occur to those skilled in the art to which the invention pertains.

[0037]

The present invention is embodied in the form described above and has the following effects. The present invention is directed to a sequence of motion analysis systems that captures a set of real-time images of the sequence and correlates such images to a set of archived images. Each captured real-time image set can be correlated with one or more archived image sets, each in real-time or a time delay. Also, each captured real-time image set can be added to an archive and compared to other archived sets. 2
Or more archived data sets can be compared. In the preferred embodiment, video technologies such as ghosts, overlays, and telestrator outlines can be used. It is possible to capture and analyze the player's swing method from the start of the swing to the ball impact. Data about ball launch conditions for a particular swing flow, including ball speed, launch angle, back spin, side spin rate, rifle spin rate, and flight distance can also be collected. Other flight conditions and shot patterns can also be monitored. The use of other non-video data also forms part of this development. The audio data is used to analyze the impact of the club and the ball, and can compile data regarding the golfer's personal characteristics or the golfer's equipment characteristics. Since golf swings of the same player can be compared at different times during the course of the career, it is possible to understand not only how to handle the equipment, but also the similarities and differences in one's own technique. Performances can be archived on storage devices, including videotapes, CDs or DVDs, and the captured or recorded images can be reviewed by a third party, such as a mentor or professional golf professional for evaluation purposes. Can be used for

[Brief description of the drawings]

FIG. 1 is a perspective view of one embodiment of a series of motion analysis systems according to the present invention, illustrating a golfer addressing a golf ball.

FIG. 2 is a circuit plan view of the system of FIG. 1;

FIG. 3 is a circuit front view of the system of FIG. 1;

FIG. 4 is a block diagram showing the operation of the entire system according to one embodiment of the present invention.

FIG. 5 (a) shows one embodiment of the present invention having two views of a golfer 110 captured by a camera at different times, the views being displayed on a display device in a split screen format.

FIG. 5 (b) shows the two figures displayed in FIG. 5 (a) superimposed on each other.

5 (c)-(d) show the superimposed views of FIG. 5 (b), each figure being painted in the same color.

FIG. 6 shows a display device 3 including two composite diagrams and an area for test conditions to be listed or otherwise displayed.
1 shows an embodiment of the invention having two regions.

FIG. 7 is a flowchart illustrating steps in a preferred embodiment of the present invention.

FIG. 8 is a flowchart illustrating the steps in another preferred embodiment of the present invention.

[Description of sign]

 REFERENCE SIGNS LIST 100 motion analysis system 110 golfer 120 rear camera 122 front camera 124 rear camera 126 upper camera 128 bracket 130 computer 160 signal / data processor 162 media storage 164 mass storage system 166 network

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification FI FI Theme Court ゛ (Reference) // G06T 7/20 G06T 7/20 C (72) Inventor Robert Campbell Dickinson 92083 Vista Willow Ridge Dora, California, USA Eve 1922 F term (reference) 5B057 AA20 CA12 CB12 CE08 CH01 CH12 5C054 AA01 AA05 CA04 CC02 CE01 CE16 CH01 EA01 EA05 EA07 FA09 FE13 FE21 GB02 GB11 GB15 HA05 HA16 5L096 BA20 CA05 DA01 HA02 LA05 LA12

Claims (25)

[Claims] 1. A method of mixing data relating to athletic performance of a player, comprising: defining a first test condition applied to the player; collecting a first video data set relating to a first player performance; The first video data set is collected during a first test condition, stores the first test condition and the video data, defines a second video data set adapted to a player, and includes a first and a second video data set. The test condition set has at least one common test condition, and collects a second video data set for a second player performance, the second video data set is collected during the second test condition; Storing the second test condition and the video data and forming the first and second A method comprising mixing at least a portion of a video data set and displaying a superimposed set. 2. The method of claim 1, wherein the first and second video data sets each comprise a single video frame. 3. The method of claim 1, wherein the first and second video data sets each comprise at least two video frames. 4. The method of claim 3, wherein the superimposed video data sets are substantially synchronized from a start time to an end time. 5. The method of claim 4, wherein the first and second video data sets are acquired using a video camera having a capture rate of at least about 250 frames per second. 6. The method of claim 4, wherein the mixing of the video data sets comprises: a first percentage of pixels in each frame selected from one of the first and second video data sets; A method performed on a second percentage of pixels in each frame selected from another of the two video data sets. 7. The method of claim 6, wherein the sum of the first and second percentages is 100. 8. The method of claim 7, wherein the first percentage is between about 20 to 40 percent and the second percentage is about 6%.
A method that is between 0 and 80 percent. 9. Collecting a first non-video data set for a first player performance, wherein the first non-video data set is collected during a first set of test conditions; Storing, collecting a second non-video data set for a second player performance, wherein the second non-video data set is collected during a second test condition set, and storing the second non-video data set; Mixing at least a portion of the first and second non-video data sets to form a superimposed non-video data set, and displaying or playing at least a portion of the superimposed non-video data set. The method of claim 4, wherein: 10. The method of claim 9, wherein the superimposed non-video data set is substantially synchronized from a start time to an end time, and the superimposed non-video data set is from a start time. A method that is nearly synchronized with the superimposed video dataset until the end time. 11. The method of claim 10, wherein the first and second non-video data sets include at least one of audio data and launch monitor data. 12. The method of claim 10, wherein the first and second video data sets are collected using at least one analog or digital video camera.
And a method wherein the second non-video data set is collected using at least one of a microphone and a launch monitor. 13. The method of claim 4, wherein each set of test conditions includes at least one of a player's equipment, environmental conditions, and a player's personal characteristics. 14. The method of claim 13, wherein the mixing of the video data includes at least one key effect. 15. The method of claim 4, wherein the first and second
The method wherein each video data set comprises data from at least two video cameras. 16. The method of claim 15, wherein the at least two video cameras are positioned at approximately the same angle. 17. The method of claim 15, wherein at least two video cameras are located on substantially the same plane. 18. A method for overlaying a data set of athletic performance of a player, comprising defining a plurality of test conditions adapted to the player, collecting a plurality of video data sets related to the performance of the player, each video data set comprising at least 1
Storing a plurality of test conditions and a video data set; mixing at least a portion of at least two of the video data sets to form a superimposed set; A method comprising displaying a paused set. 19. Collecting a plurality of non-video data sets relating to the performance of the player, each non-video data set having at least one common test condition, storing the plurality of non-video data sets, and being superimposed. 19. The method of claim 18, wherein mixing at least a portion of at least two of the non-video data sets to form or display the superimposed non-video data sets to form a non-video data set. Method. 20. The method of claim 19, wherein a plurality of test conditions, video data and non-video data sets are synchronized. 21. At least one video camera for collecting a plurality of video data sets of the performance of the player; at least one storage device for storing the plurality of video data sets; and at least two of the video data sets. And at least one display for displaying at least two mixed data sets, the processing apparatus comprising: at least two videos for forming an aggregate video data set. A multimedia analysis system for capturing and comparing athletic performance of players, which is characterized by substantially synchronizing and superimposing datasets. 22. The multimedia analysis system according to claim 21, wherein the position of at least one video camera is substantially the same when collecting each video data set. 23. A launch monitor for collecting a plurality of flight data sets of an object struck by a player, and at least one processing unit for mixing at least two of the flight data sets, wherein at least one process is provided. The apparatus substantially synchronizes and superimposes the at least two flight data sets to form an aggregate flight data set, wherein the aggregate flight data set is also synchronized with the aggregate video data set. 23. The multimedia analysis system according to claim 22, wherein: 24. The multimedia analysis system of claim 23, wherein a plurality of video data sets are collected by at least two video cameras positioned at substantially the same angle and aiming at the player. 25. The multimedia analysis system according to claim 23, wherein a plurality of video data sets are collected by at least two video cameras positioned substantially in the same plane and aiming at the player.