KR20200112350A - device, method, system for inserting time index to sports videos and storage of computer program therefor - Google Patents

Abstract

The present invention relates to a device, method, and system for indexing a sports video and a storage medium in which a computer program therefor is stored. More specifically, the present invention relates to technology for imaging a video that a player or a strike means such as a club of the player strikes a target such as a ball, analyzing a frequency change pattern of a radar signal reflected from the target to generate index corresponding to a strike moment, and reflecting the index to the video.

Description

[Device, method, system for inserting time index to sports videos and storage of computer program therefor}

The present invention relates to an apparatus, method, and system for indexing sports images, and a recording medium storing a computer program therefor. More specifically, the present invention captures an image in which a player or a player's club-like striking means hits a target such as a ball, but analyzes the frequency change pattern of the radar signal reflected from the striking means and the target, It relates to a technology for creating an index corresponding to and reflecting the index on an image.

Until now, radar has been mainly used for military purposes, but now it is used in many parts of the civilian sector. It is used in various fields such as communication technology and autonomous vehicle driving technology, and in particular, it has been used in various fields including golf in recent years.

Examples of the prior art that radar is used in the sports field are as follows.

The golf swing analysis apparatus and method disclosed in Korean Patent Registration No. 10-1716360 uses a radar that generates an output signal by detecting a frequency deviation of a reflected wave by radiating a transmission wave to a golf club and a golf ball. And based on the output signal, the speed of the golf club and the golf ball is calculated. However, this technology using radar has the disadvantage that it is difficult for the user to monitor his or her own posture only by analyzing the speed of the golf club and golf ball.

As disclosed in Korean Patent Application Publication No. 2011-0122148, the radar and the camera derive data related to golf balls and golf clubs, and radar data (ball/club/club head trajectory, direction, and ball) The hitting can be shown in the image provided by the camera), such as the angle of the riding surface of the golf club. Therefore, the user can visually see his or her posture and various information. However, when trying to check a specific point in time, that is, the posture at the time of strike, among the images captured by the radar and camera, the user cannot quickly move to the corresponding part, and there are restrictions on the installation location other than the field environment, so the user cannot easily use it.

Korean Patent Registration No. 10-1716360 (2017. 03. 08) Korean Patent Publication 10-2011-0122148 (2011. 11. 09)

It is an object of the present invention to create an index on the time point at which the striking means strikes a target, and insert the index into an image to easily check an image at a specific point in time, and a storage storing a computer program therefor It is to provide the medium.

The sports image indexing apparatus 1 of the present invention includes: a Doppler radar 10 sensing movement of the object and the target when an object strikes a target; The sports image indexing apparatus of the present invention comprises: a radar detecting movement of a striking means and a target; A digital camera for photographing the movement of the striking means and the target; And an index generator that inserts an index of an event occurrence point into an image captured by the digital camera when the frequency change pattern of the signal detected by the radar matches a preset event pattern.

The radar, a transmitter for transmitting a transmission wave to the striking means and a target, a receiver for receiving the reflected wave of the striking means and the target, and a bit receiving signal based on a frequency difference between the transmitted wave and the reflected wave. It may include a signal generating unit to generate and a digital conversion unit (ADC) for converting an analog bit reception signal into a digital signal.

The index generator includes a filtering unit for selecting a signal having a threshold value or more from the digital converted signal provided by the radar, a windowing unit for dividing the filtered signal into a predetermined unit time, and a frequency for the divided unit time signal. A high-speed Fourier transform unit for converting a signal in the domain, a frequency estimating unit for estimating an effective frequency from the converted signal in the frequency domain, and a change pattern for a frequency equal to or higher than a threshold value among the estimated effective frequencies coincide with a preset event pattern An analysis module including an event detection unit for outputting an event signal indicating occurrence of an event; And an indexing module for inserting an index into an image captured by the digital camera according to the event signal.

The analysis module may further include a speed calculation unit for calculating the instantaneous speed of the striking means and the target based on a frequency corresponding to the striking means and the target in a frequency pattern matching the event pattern and the magnitude of the frequency. have.

In the frequency pattern coincident with the event pattern, the last frequency peak of the frequency change region representing the motion of the striking means may be the instantaneous frequency of the striking means, and the single frequency peak representing the motion of the target may be the instantaneous frequency of striking of the target.

The index generator includes a filtering unit that selects a signal greater than or equal to a threshold value from the digital converted signal provided by the radar, a windowing unit that divides the filtered signal into a predetermined unit time, and included in the divided unit time signal. A fringe counter for converting the changed interference fringe into a signal in a frequency domain, a frequency estimation unit for estimating an effective frequency from the converted signal in the frequency domain, a temporary storage unit for storing the estimated frequency information, and the estimated An analysis module including an event detection unit for outputting an event signal notifying occurrence of an event when a change pattern for a frequency equal to or greater than a threshold value among the effective frequencies matches a preset event pattern; And

It may include an indexing module for inserting an index into the image captured by the digital camera according to the event signal.

The analysis module may further include a speed calculation unit for calculating the instantaneous speed of the striking means and the target based on a frequency corresponding to the striking means and the target in a frequency pattern matching the event pattern and the magnitude of the frequency. have.

In the frequency pattern coincident with the event pattern, the last frequency peak of the frequency change region representing the motion of the striking means may be the instantaneous frequency of the striking means, and the single frequency peak representing the motion of the target may be the instantaneous frequency of striking of the target.

The sports image indexing system of the present invention comprises: a radar module for detecting movement of a hitting means and a target; And a camera module for photographing the movement of the striking means and the target, and when the frequency change pattern of the signal detected by the radar module matches a preset event pattern, an event signal is output and the camera module determines the index of the event occurrence time. Mobile communication terminal including an index generation module to be inserted into the captured image

It may include.

The index generation module includes a filtering unit for selecting a signal having a threshold value or more from the digital converted signal detected by the radar module, a windowing unit for dividing the filtered signal into a predetermined unit time, and a signal of the divided unit time A fast Fourier transform unit for converting a signal in a frequency domain, a frequency estimator for estimating an effective frequency from the converted signal in the frequency domain, and an event pattern in which a change pattern for a frequency equal to or higher than a threshold value among the estimated effective frequencies is preset If matched with, an event detection unit for outputting an event signal indicating occurrence of an event, and an image editing unit for inserting an index into an image captured by the camera module according to the event signal.

The index generation module further comprises a speed calculation unit for calculating the instantaneous speed of the striking means and the target based on a frequency corresponding to the striking means and the target in a frequency pattern matching the event pattern, and the magnitude of the frequency. I can.

In the frequency pattern coincident with the event pattern, the last frequency peak of the frequency change region representing the motion of the striking means may be the instantaneous frequency of the striking means, and the single frequency peak representing the motion of the target may be the instantaneous frequency of striking of the target.

The indexing method of a sports image of the present invention includes the steps of converting a bit receiving signal of a radar to a hitting means and a target into a digital signal; Selecting a signal greater than or equal to a threshold value from the digitally converted bit received signal; Dividing the selected bit reception signal into a predetermined unit time; Converting the divided unit time signal into a frequency domain; Estimating an effective frequency from the frequency domain transformed signal; And outputting an event signal when the estimated effective frequency change pattern matches a preset event pattern.

When the output of the event signal is detected while photographing the striking means and the target, inserting an index indicating an event occurrence time to the digital image being photographed may be further included.

It may further include calculating the instantaneous speed of the striking means and the target based on the frequency corresponding to the striking means and the target in the effective frequency change pattern matching the event pattern, and the magnitude of the frequency.

In the frequency pattern coincident with the event pattern, the last frequency peak of the frequency change region representing the motion of the striking means may be the instantaneous frequency of the striking means, and the single frequency peak representing the motion of the target may be the instantaneous frequency of striking of the target.

The event may be that the hitting means hits the target.

It may include a recording medium storing a computer program that implements each step of the sports image indexing method of the present invention as an algorithm.

Embodiments of the present invention use a simple principle of finding a specific pattern in the frequency domain for the reflected signal of an ultrasonic wave or radar, and thus, an expensive device or an expensive device to measure the instantaneous speed at the time when a specific event occurs or There is no need to use complex analysis algorithms.

In addition, according to embodiments of the present invention, by accurately grasping the occurrence time of an event that is considered meaningful in sports at the same time as the video is captured, and including it as an index in the corresponding video, the user can quickly retrieve the video at a desired specific time without searching the entire video You can look for it.

In addition, according to embodiments of the present invention, by analyzing the reflected signal of the ultrasonic wave or the radar, it is possible to find out the instantaneous speed of the striking means and/or the target in addition to the specific time point at which the event occurs, so that comprehensive information on the event image can be provided.

1 is a block diagram showing an indexing apparatus according to a first embodiment.
2 is a block diagram showing a detailed configuration of a radar in the indexing apparatus of the first embodiment.
3 is a block diagram showing an example of a detailed configuration of the analysis module according to the first embodiment.
4 and 5 are graphs showing exemplary experimental results in the frequency domain and the time domain, respectively.
6 is a graph showing the experimental results of another example in the frequency domain, respectively.
7 shows various video play-related functions that can be output through a display.
8 is a block diagram showing an indexing apparatus according to the second embodiment.
9 is a block diagram showing a detailed configuration of a radar in the indexing apparatus of the second embodiment.
10 is a block diagram showing an example of a detailed configuration of an analysis module according to a second embodiment.
11 is a conceptual diagram of an indexing device according to the first or second embodiment.
12 is a conceptual diagram of an indexing system according to the third embodiment.
13 is a conceptual diagram of an indexing system according to the fourth embodiment.
14 is a flow chart showing a sports image indexing method step by step according to an embodiment.
15 is a flowchart illustrating a sports image indexing method according to another embodiment step by step.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it is directly connected to or may be connected to the other component, but other components may exist in the middle. Should be.

On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

The terms used in the present specification are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, processes, operations, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the possibility of addition or presence of elements or numbers, processes, operations, components, parts, or combinations thereof is not preliminarily excluded.

Unless otherwise defined, all terms including technical or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

The term "MODULE" described herein refers to a unit that processes a specific function or operation, and may mean hardware or software, or a combination of hardware and software.

Terms and words used in the present specification and claims are not limited to their usual or dictionary meanings and should not be interpreted, and the inventor may appropriately define the concept of terms in order to describe his own invention in the best way. Based on the principle that there is, it should be interpreted as meaning and concept consistent with the technical idea of the present invention. In addition, unless there are other definitions in the technical terms and scientific terms used, they have the meanings commonly understood by those of ordinary skill in the art to which this invention belongs, and the gist of the present invention in the following description and accompanying drawings Descriptions of known functions and configurations that may be unnecessarily obscure will be omitted. The drawings introduced below are provided as examples in order to sufficiently convey the spirit of the present invention to those skilled in the art. Accordingly, the present invention is not limited to the drawings presented below and may be embodied in other forms. In addition, the same reference numbers throughout the specification indicate the same elements. It should be noted that the same components in the drawings are indicated by the same reference numerals wherever possible.

In the specification of the present invention, the term "hitter" refers to an exercise device called a club, a stick, or a racket for each sport category, and a separate exercise device is used. In sports that do not play soccer, basketball, volleyball, boxing, etc., it can refer to the player's hand or foot or the player itself.

In addition, in the specification of the present invention, the term "target" refers to a ball, a puck, etc. as a target for hitting by using the exercise equipment listed above.

Hereinafter, for convenience of explanation, golf is used as a main example. In other words, “iron” and “strike means” used by golfers can be used interchangeably, and both are understood to have the same meaning. Also, “golf ball” and “target” may be used interchangeably, and both are understood to have the same meaning.

The apparatus for indexing a sports image according to an embodiment of the present invention uses a radar to find a time when a meaningful event occurs. Here, the term “event” may be variously defined according to the characteristics of a sporting event. For example, a meaningful event in a golf event may be defined as a moment when a golfer strikes a golf ball using an iron. Similarly, in football, the moment a player strikes a soccer ball with his foot near the goal (the moment he shoots a valid shot), in volleyball the moment the player strikes a volleyball ball with his hand near the net, in hockey, the player strikes the puck with a stick towards the goal The moment of hitting, in tennis, can be defined as the moment when the player strikes the tennis ball with the racket at the serving point, and it can be defined in various ways.

In the present specification, the indexing device may employ a radar using at least one or one of laser, infrared, ultrasonic, and electromagnetic waves. In addition, an appropriate modulation method may be used depending on the characteristics of the signal used. For example, a continuous wave method that measures the difference in frequency between transmission and reception by sweeping a frequency, and a pulse Doppler method that measures time delay by sending and receiving pulses ( Pulse Doppler) method, amplitude modulation method of measuring a phase difference between transmission and reception signals by amplitude modulation of a carrier wave may be used. More specific examples, continuous wave modulation methods such as CW (Continuous Wave) and FMCW (Frequency Modulation Continuous Wave), Pulse Amplitude Modulation, Pulse Position Modulation, and Pulse Width Modulation , Pulse Number Modulation, Pulse Code Modulation, a pulse modulation method such as delta modulation, and an amplitude modulation method may be used. However, the modulation methods listed are only examples and need not be limited to a specific modulation method.

Hereinafter, for convenience of explanation, embodiments in which a radar using a continuous wave modulation method for electromagnetic waves and ultrasonic waves will be described.

<Example 1>

Embodiment 1 relates to an apparatus for indexing sports images using electromagnetic waves as radar signals. 1 is a block diagram showing an indexing apparatus according to a first embodiment.

Referring to FIG. 1, the indexing apparatus according to the first embodiment includes a radar 100, a digital camera 200, and an index generator 300, and a display 400 may be further included therein.

When the hitting means hits the target, the radar 100 detects the movement of the hitting means and the target using the reflected wave of the radar. In the example of golf, when the iron corresponding to the striking means strikes the golf ball corresponding to the target, the radar 100 receives the reflected wave reflected by the iron and the golf ball with respect to the electromagnetic wave transmitted by itself, and the analog It converts the received bit signal into a digital signal for easy analysis.

2 is a block diagram showing a detailed configuration of a radar in the indexing apparatus of the first embodiment.

The radar 100 includes a transmitting unit 110 for transmitting an electromagnetic wave, a receiving unit 120 for receiving a reflected wave that is reflected by the transmitting wave by the striking means and the target, and a bit receiving signal based on the frequency difference between the transmitted wave and the reflected wave. A signal generation unit 130 for generating a signal and a digital conversion unit (ADC) 140 for converting an analog bit reception signal into a digital signal.

The digital camera 200 generates digital image data by photographing a situation in which the striking means hits the target.

The index generator 300 includes an analysis module 310 that determines when an event occurs, and an indexing module 320 that inserts an index indicating when an event occurs in an image captured by the digital camera 200.

3 is a block diagram showing an example of a detailed configuration constituting the analysis module 310.

The analysis module 310 identifies a preset pattern of wavelength change from the signal detected by the radar 100, and calculates a time point of occurrence of an event in which the striking means strikes the target based on the identified pattern of wavelength change. 2 is a block diagram showing an example of a detailed configuration of the analysis module 310.

Referring to FIG. 3, the analysis module 310 includes a filtering unit 311 that selects a signal higher than a predetermined threshold value from the digital converted signal transmitted from the radar 100, and divides the selected signal into a predetermined unit time. From the windowing unit 312 and the signal converted by the fast Fourier transform unit 313 that converts the signal of the unit time divided by the windowing unit 312 into a signal in the frequency domain, and the fast Fourier transform unit 313 A frequency estimating unit 314 that estimates a spectrum of an effective frequency and a magnitude of a frequency for each effective frequency, and a change pattern for a frequency signal having a preset threshold magnitude or higher among the estimated frequency signals. It includes an event detection unit 315 for outputting an event signal notifying that the unit time is the moment of hitting if it matches a preset event pattern. It is preferable that the wing doing unit 312 divides the selected digital signal into a unit time short enough to be used as an index.

The windowing unit 312, the fast Fourier transform unit 313, and the frequency estimating unit 314 are repeatedly performed on the received signal selected by the filtering unit 311 until the event detection unit 315 finds an event pattern. do.

Analysis module 310 corresponds to each hitting means (golf iron, tennis racket, hockey stick, etc.) and target (golf ball, tennis ball, hockey puck, etc.) in the pattern when the event detection unit 315 detects the event pattern. It may further include a speed calculation unit 316 for identifying the frequency (frequency)-magnitude value of the point, and setting the time domain conversion value of the points as the speed of the hitting means and the target. That is, when the analysis module 31 includes the event detection unit 315 and the speed calculation unit 316 together, when the event pattern is detected from the reflected wave, the event occurrence time and the speed of the hitting means and the target at the time are simultaneously You will be able to grasp.

The operation of the analysis module 310 will be described with a specific experimental example as follows.

4 and 5 are graphs showing experimental results in the frequency domain and the time domain, respectively, assuming the seventh iron club and the golf ball of golf as striking means and target, respectively.

In more detail, FIG. 4 is a frequency-magnitude graph derived as a result of estimating the frequency by the frequency estimating unit 314 from the signal converted into the frequency domain by the fast Fourier transform unit 313. In the graph of FIG. 4, a pattern in which the frequency amplitude changes abruptly over several times in a frequency region relatively wider than a golf ball in a region of about 3500 Hz to about 5800 Hz is shown. This pattern is a pattern that shows the movement of the iron club through repeated experiments. It can be predefined as a pattern to represent. In addition, in the frequency domain showing such a pattern, a specific frequency can be predefined as the frequency of the iron at the moment of hitting through repeated experiments. In FIG. 3, the frequency peak at about 5000 Hz indicated by a square is the frequency of the iron at the moment of hitting. That is, it can be defined as the last frequency peak indicating the iron at the moment of hitting among the frequency change patterns indicating the movement of the iron club based on the results of repeated experiments.

In the range of about 6000Hz to about 6500Hz, a single frequency shows a sudden change in size in a frequency range that is relatively narrower than that of an iron.This pattern can be predefined as a pattern representing the movement of the moment the golf ball is hit through repeated experiments. . In addition, in the frequency domain showing such a pattern, a specific frequency may be predefined as the frequency of the golf ball at the moment of hitting through repeated experiments. The frequency peak at a point of about 6500 Hz indicated by a circle in FIG. 3 is the frequency of the golf ball at the moment of hitting. In other words, a single frequency peak occurring with a certain frequency gap near the frequency change pattern indicating the movement of the iron club indicates the iron at the moment of hitting.

In addition, the frequency change pattern for the iron and the golf ball as described above may be preset as an event pattern.

The event detection unit 315 analyzes whether a change pattern for a frequency signal equal to or greater than a preset threshold magnitude in the estimated frequency signal of an actual received signal matches a preset event pattern as described above, and if both match, the unit It generates an event signal indicating that time is the moment of hitting.

5 is a graph of a time-frequency after inversely transforming a signal in the frequency domain including the event pattern (ie, the signal in FIG. 3) into the time domain.

In FIG. 4, when looking at the values of the time domain corresponding to the frequency of the iron (square display point) and the frequency of the golf ball (circle display point), it can be confirmed that the hit occurred at approximately 0.55 sec (Sec). In addition, at that time, it can be seen that the frequency spectrum of the golf club is relatively wide up and down in the range of about 3500 Hz to about 5800 Hz, and the frequency spectrum of the golf ball is relatively narrowly spread up and down in the range of about 6000 Hz to about 6500 Hz. .

6 is a graph showing the results of an experiment in the frequency domain, assuming a tennis racket and a tennis ball, respectively, as another example of a hitting means and a target.

In the case of tennis, it can be seen that the frequency pointing to the tennis racket is determined at about 4800Hz and the frequency pointing to the tennis ball at about 6200Hz. In addition, when the threshold of the frequency size is set to 2, the frequency size change pattern of FIG. 5 derived through a number of experiments may be determined as the event pattern.

Finally, the display 400 shows an image in which the index is inserted and shows the instantaneous speed of the target at the moment of hitting on a part of the screen. The display 400 may be a part included in the indexing device of the first embodiment, or may be a separate display device provided separately from the indexing device. In the latter case, the indexing apparatus further includes an interface module (not shown in FIG. 1) for transmitting an image including the index to the display device through a predetermined communication network or communication line.

The display 400 may output an image in various ways by using the index. 7 shows various types of video play-related functions that can be output through the display 400. That is, for the index image generated according to Example 1, a function to move to a point in time to which indexing is applied, a function to play repeatedly at a point in time to which indexing is applied, a function to play slowly, a function to stop, a function to show only the hitting speed data, An image may be output with at least one of a display function, a function to simultaneously display a hitting scene and speed data at the time of a hit, and a function to divide and display a plurality of recent hitting scenes on one screen.

<Example 2>

Embodiment 2 relates to an apparatus for indexing sports images using ultrasonic waves as radar signals. 8 is a block diagram showing an indexing apparatus according to the second embodiment.

Referring to FIG. 8, the indexing apparatus according to the second embodiment includes a radar 1000, a digital camera 2000, and an index generator 3000, and a display 4000 may be further included therein.

When the hitting means hits the target, the radar 1000 senses the movement of the hitting means and the target using the reflected wave of ultrasonic waves. In the example of tennis, when the racket corresponding to the striking means hits the tennis ball corresponding to the target, the radar 1000 receives the reflected wave reflected from the racket and the tennis ball with respect to the ultrasonic wave transmitted by itself, and the analog It converts the received bit signal into a digital signal for easy analysis.

9 is a block diagram showing a detailed configuration of a radar 1000 in the indexing apparatus of the second embodiment.

The radar 1000 includes a transmission unit 1100 that transmits a transmission wave, a reception unit 1200 that receives a reflected wave that is reflected by the hitting means and a target and returns, and a bit receiving signal based on a frequency difference between the transmitted wave and the reflected wave. A signal generation unit 1300 for generating a signal and a digital conversion unit (ADC) 1400 for converting a bit received signal into a digital signal may be included.

The index generator 3000 includes an analysis module 3100 that calculates an event occurrence time point and an indexing module 3200 that inserts an index indicating the event occurrence time into an image photographed by the digital camera 2000.

The analysis module 3100 identifies a preset pattern of wavelength change from a signal detected by the radar 1000, and calculates a time point of occurrence of an event in which the striking means strikes the target based on the identified pattern of wavelength change. 10 is a block diagram showing an example of a detailed configuration of the analysis module 3100.

Referring to FIG. 10, the analysis module 3100 includes a filtering unit 3110 that selects a signal higher than a predetermined threshold value from the digital converted signal transmitted from the radar 1000, and divides the selected signal into a predetermined unit time. A windowing unit 3120, a fringe counter 3130 for converting a change in interference fringe included in a signal of a unit time divided by the windowing unit 3120 into a frequency domain, and a signal converted by the fringe counter 3130 A frequency estimating unit 3140 for estimating a frequency from, a temporary storage unit 3150 for storing the estimated frequency information, and a frequency change pattern for a frequency signal having a preset threshold magnitude or higher among the temporarily stored signals. It includes an event detection unit 3160 that outputs an event signal when it matches a preset event pattern.

The fringe counter 3130 measures the number of changes in the interference fringe and differentiates the intensity of the interference fringe with respect to time to calculate the magnitude and waveform of the frequency. In addition, it is preferable that the wing doing unit 3120 divides the selected digital signal into a short unit time enough to be used as an index.

The windowing unit 3120, the fringe counter 3130, and the frequency estimation unit 3140 may be repeatedly performed for the received signal selected by the filtering unit 3110 until the event detection unit 3160 finds an event pattern. have.

Since the speed calculation unit 3170 and the display 4000 are the same as the speed calculation unit 316 and the display 400 of the first embodiment, duplicate descriptions are omitted.

Since the indexing apparatus of the second embodiment uses ultrasonic waves as a transmission wave, it is inevitably sensitive to sound wave noise such as cheering sounds in a stadium. Therefore, the indexing device of the second embodiment can be mainly used in a quiet environment such as an indoor golf driving range.

<Example 3>

In the indexing apparatus of the first and second embodiments described above, the radars 100 and 1000, the digital cameras 200 and 2000, and the index generators 300 and 3000 are mounted together in the same housing (not shown in the drawing). It can be implemented as an independent device (E1). In this case, the radars 100 and 1000, the digital cameras 200 and 2000, and the index generator 300 and 3000 are connected through at least one substrate (PCB board, not shown in the drawing), and a bus on the substrate or Communication can be performed through electronic circuits. 11 illustrates the concept of an independent indexing device according to the first or second embodiment.

In contrast, Embodiment 3 relates to a case in which the radars 100 and 1000, the digital cameras 200 and 2000, and the index generators 300 and 3000 are implemented as separate devices. In this case, the radars 100 and 1000, the digital cameras 200 and 2000, and the index generators 300 and 3000 may be connected to each other through a wired communication network or a short-range wireless communication network.

12 is a configuration diagram showing an example of the indexing system of the third embodiment.

Considering the case where the indexing device of Example 3 is installed in an indoor golf practice room, radars 100 and 1000 and digital cameras 200 and 2000 are installed on the ceiling or side wall of the practice room, and radars 100 and 1000 and digital The video or radar signal output from the cameras (200, 2000) is an index generator that is connected to a short-range wireless communication network such as Bluetooth, Zigbee, or Wi-Fi, or through a wired communication network such as Ethernet or LAN. It can be transmitted to (300, 3000).

At this time, the index generators 300 and 3000 may be implemented in the form of a PC or a server, and the analysis modules 310 and 3100 and the indexing modules 320 and 3200 constituting the index generator are installed in the form of a computer program on a PC or server. Can be. In addition, the displays 400 and 4000 may be implemented as separate monitors or beam projectors.

<Example 4>

The fourth embodiment relates to a case where the radars 100 and 1000 are implemented as separate devices, and the digital cameras 200 and 2000 and the index generators 300 and 3000 are implemented as one device. In this case, the first device (100, 1000) including a radar, and the second device (E2) including the digital camera (200, 2000) and the index generator (300, 3000) are connected to each other, such as RS-232C or USB. It may be connected through a wired serial communication or parallel communication, a wired network such as Ethernet, a short-range wireless communication network such as Bluetooth, ZigBee, or Wi-Fi, or a broadband mobile communication network.

In particular, mobile communication terminals such as today's smart phones are equipped with high-performance digital cameras as a standard, but when the functions of the index generators 300 and 3000 are implemented and installed as an application program, it can be a preferred example of a second device. .

13 is a configuration diagram showing an example of the indexing system of the fourth embodiment.

In the indoor golf practice room, the user connects a small radar module implemented in a mobile form to a smartphone through a USB cable, and then mounts the smartphone and the radar module in a position where the swing posture can be viewed. The user starts practicing golf after running the index generator app installed on the smartphone. The index generator app operates the camera module installed on the smartphone, while operating the small radar module connected via USB. Here, the small radar module corresponds to the radars 100 and 1000, the camera module corresponds to the digital cameras 200 and 2000, and the index generator app corresponds to the index generators 300 and 3000. In addition, the liquid crystal of the smartphone corresponds to the displays 400 and 4000.

Since the radars 100 and 1000, digital cameras 200 and 2000, and the index generators 300 and 3000 have been described in detail above, the small radar modules, camera modules, index generator apps and liquid crystals of smartphones corresponding to each of these are described in detail. Redundant explanations for this are omitted.

In the fourth embodiment, the small radar module may be implemented in a form detachable from the smartphone through a predetermined interface provided in the smartphone.

<Example 5>

Embodiment 5 relates to a sports image indexing method using an electromagnetic wave radar, and FIG. 14 is a flowchart showing the indexing method of the fifth embodiment step by step.

First, the analog bit reception signal is converted into a digital signal for easy analysis (S101).

Since it is wasteful to perform the analysis for event detection on all of the bit reception signals provided by the radar, it is desirable to selectively analyze only the bit reception signals related to the state in which the movement of the striking means or the target has occurred once. To this end, the bit reception signal is filtered by selecting a signal having a predetermined threshold or higher from the digitally converted bit reception signal (S102).

Next, the filtered bit reception signal is divided into a predetermined unit time (S103). It is preferable to set the predetermined unit time to a short interval so as to correspond to the index indicating the occurrence of the event.

Next, the divided unit time signal is converted into a frequency domain signal using a fast Fourier transform (FFT) or a similar algorithm in order to detect an event using a change in frequency (S104).

Next, the spectrum of the effective frequency and the magnitude of the frequency for each effective frequency are estimated from the signal converted into the frequency domain (S105).

Next, among the estimated frequency signals, a change pattern for a signal having a preset threshold magnitude or more is compared with a preset event pattern (S106), and if both match, an event signal indicating that the unit time is the moment of hitting is generated. Output (S107).

When the output of the event signal is detected while photographing the striking means and the target, an index indicating the event occurrence time is inserted into the digital image being photographed (S108).

The steps S103 to S105 are repeatedly performed until an event pattern is found.

<Example 6>

Embodiment 6 relates to a sports image indexing method using an ultrasonic radar, and FIG. 15 is a flowchart showing the indexing method of Embodiment 6 step by step.

First, the analog bit reception signal is converted into a digital signal for easy analysis (S1001). Further, for efficient event analysis, the bit reception signal is filtered by selecting a signal higher than a predetermined threshold value from the digitally converted bit reception signal (S1002).

Next, the filtered bit reception signal is partitioned into a predetermined unit time (S1003). It is preferable to set the predetermined unit time to a short interval so as to correspond to the index indicating the occurrence of the event.

Next, a change in interference fringe included in the divided unit time signal is converted into a frequency domain signal using a fringe counter (S1004). More specifically, the number of changes in the interference fringe included in the divided unit time signal is measured, and the intensity of the interference fringe is differentiated with respect to time to calculate the magnitude and waveform of the frequency.

Next, the spectrum of the effective frequency and the magnitude of the frequency for each effective frequency are estimated from the signal converted into the frequency domain (S1005). Then, the estimated frequency information is temporarily stored (S1006).

Next, among the temporarily stored frequency signals, a change pattern for a signal having a preset threshold magnitude or higher is compared with a preset event pattern, and if both match, an event signal indicating that the unit time is the moment of hitting is output ( S1007).

The steps S1003 to S1005 are repeatedly performed until an event pattern is found.

Those skilled in the art can easily understand that the partial functions of the sports image indexing apparatus and/or method described above may be included in a recording medium that can be read through a computer by tangibly implementing a program of instructions for implementing the same. There will be. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and constructed for the present invention, or may be known and usable to those skilled in computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and floptical disks. Magneto-optical media and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, USB memory, and the like. The computer-readable recording medium may be a transmission medium such as an optical or metal wire or a waveguide including a carrier wave for transmitting a signal specifying a program command or a data structure. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter or the like, in addition to machine language codes such as those produced by a compiler. The hardware device may be configured to operate as one or more software modules to perform the operation of the present invention and vice versa.

In addition, the present invention is not limited to the above-described embodiments, and of course, various modifications can be implemented without departing from the gist of the present invention as claimed in the claims, as well as various application ranges.

100, 1000: radar
110, 1100: transmitter 120, 1200: receiver
130, 1300: signal generation unit 140, 1400: digital conversion unit (ADC)
200, 2000: digital camera
300, 3000: index generator
311, 3110: filtering unit 312, 3120: windowing unit
313: high-speed Fourier transform unit 3130: fringe counter
314, 3140: frequency estimation unit 315, 3160: event detection unit
3150: temporary storage unit 316, 3170: speed calculation unit
400, 4000: display

Claims (18)

A radar detecting movement of the hitting means and the target;
A digital camera for photographing the movement of the striking means and the target; And
An index generator that inserts an index of the event occurrence time into the image captured by the digital camera when the frequency change pattern of the signal detected by the radar matches a preset event pattern
Sports image indexing device comprising a. The method of claim 1,
The radar,
Signal generation for generating a bit receiving signal based on the frequency difference between the striking means and the reflected wave, and a transmitter for transmitting a transmission wave to the striking means and the target, a receiver for receiving the reflected wave of the striking means and the target And a digital conversion unit (ADC) for converting an analog bit reception signal into a digital signal. The method of claim 2,
The index generator,
A filtering unit that selects a signal greater than or equal to a threshold value from the digital converted signal provided by the radar, a windowing unit that divides the filtered signal into a predetermined unit time, and converts the divided unit time signal into a frequency domain signal A high-speed Fourier transform unit that estimates an effective frequency from the transformed signal in the frequency domain; and an event notifying that an event occurs when a change pattern for a frequency equal to or greater than a threshold value among the estimated effective frequencies matches a preset event pattern. An analysis module including an event detection unit outputting an event signal; And
Indexing module for inserting an index into an image captured by the digital camera according to the event signal
Sports image indexing apparatus comprising a. The method of claim 3,
The analysis module,
Sports video, characterized in that it further comprises a speed calculation unit for calculating the instantaneous speed of the hitting means and the target based on the frequency corresponding to the hitting means and the target in a frequency pattern matching the event pattern Indexing device. The method of claim 4,
In the frequency pattern coinciding with the event pattern, the last frequency peak in the frequency change region representing the movement of the striking means is the instantaneous frequency of the striking means, and the single frequency peak representing the movement of the target is the instantaneous frequency of striking of the target. Sports video indexing device. The method of claim 2,
The index generator,
A filtering unit for selecting a signal greater than or equal to a threshold value from the digital converted signal provided by the radar, a windowing unit for dividing the filtered signal into a predetermined unit time, and a change in interference fringes included in the divided unit time signal A fringe counter for converting a signal in a frequency domain, a frequency estimating unit for estimating an effective frequency from the converted signal in the frequency domain, a temporary storage unit for storing the estimated frequency information, and a threshold value among the estimated effective frequencies An analysis module including an event detection unit for outputting an event signal notifying occurrence of an event when the change pattern for the above frequency matches a preset event pattern; And
Indexing module for inserting an index into an image captured by the digital camera according to the event signal
Sports image indexing apparatus comprising a. The method of claim 6,
The analysis module,
Sports video, characterized in that it further comprises a speed calculation unit for calculating the instantaneous speed of the hitting means and the target based on the frequency corresponding to the hitting means and the target in a frequency pattern matching the event pattern Indexing device. The method of claim 7,
In the frequency pattern coinciding with the event pattern, the last frequency peak in the frequency change region representing the movement of the striking means is the instantaneous frequency of the striking means, and the single frequency peak representing the movement of the target is the instantaneous frequency of striking of the target. Sports video indexing device. A radar module for detecting movement of the striking means and the target; And
When the frequency change pattern of the signal detected by the hitting means and the target is matched with a preset event pattern, an event signal is output and the camera module captures an index of the event occurrence time point. Mobile communication terminal including index generation module to be inserted into one image
Sports image indexing system comprising a. The method of claim 9,
The index generation module,
A filtering unit that selects a signal greater than or equal to a threshold value from the digital converted signal detected by the radar module, a windowing unit that divides the filtered signal into a predetermined unit time, and converts the divided unit time signal into a frequency domain signal. A high-speed Fourier transform unit for converting, a frequency estimating unit for estimating an effective frequency from the transformed signal in the frequency domain, and occurrence of an event when a change pattern for a frequency equal to or greater than a threshold value among the estimated effective frequencies coincides with a preset event pattern. And an event detection unit for outputting an event signal notifying, and an image editing unit for inserting an index into an image captured by the camera module according to the event signal. The method of claim 10,
The index generation module,
Sports video, characterized in that it further comprises a speed calculation unit for calculating the instantaneous speed of the hitting means and the target based on the frequency corresponding to the hitting means and the target in a frequency pattern matching the event pattern Indexing system. The method of claim 11,
In the frequency pattern coinciding with the event pattern, the last frequency peak in the frequency change region representing the movement of the striking means is the instantaneous frequency of the striking means, and the single frequency peak representing the movement of the target is the instantaneous frequency of striking of the target. Sports video indexing system. Converting the bit receiving signal of the radar to the striking means and the target into a digital signal;
Selecting a signal greater than or equal to a threshold value from the digitally converted bit received signal;
Dividing the selected bit reception signal into a predetermined unit time;
Converting the divided unit time signal into a frequency domain;
Estimating an effective frequency from the frequency domain transformed signal; And
Outputting an event signal when the estimated effective frequency change pattern matches a preset event pattern
Indexing method of a sports image comprising a. The method of claim 13,
And when the output of the event signal is detected while photographing the striking means and the target, inserting an index indicating an event occurrence time to the digital image being photographed. The method of claim 13,
And calculating the instantaneous speed of the striking means and the target based on the frequency corresponding to the striking means and the target in the effective frequency change pattern matching the event pattern and the magnitude of the frequency. How to index the video. The method of claim 13,
In the frequency pattern coinciding with the event pattern, the last frequency peak in the frequency change region representing the movement of the striking means is the instantaneous frequency of the striking means, and the single frequency peak representing the movement of the target is the instantaneous frequency of striking of the target. How to index sports videos. The method of claim 13,
The event is a sports video indexing method, characterized in that the hitting means hits the target. A recording medium storing a computer program in which each step of the sports image indexing method of claim 13 is implemented as an algorithm.

Images