KR101848864B1 - Apparatus and method for tracking trajectory of target using image sensor and radar sensor - Google Patents

Abstract

There is provided a method of tracking a trajectory of a target using an image sensor and a radar sensor by means of an operation, the method comprising the steps of acquiring, via an image sensor, an image including a moving state of at least one object Detecting a first velocity of the target from the acquired image, detecting a second velocity of the target through the radar sensor, detecting a first velocity of the target Correcting the first velocity using the second velocity by mapping the time of detecting the second velocity of the target with the time of detecting the second velocity of the target and tracking the trajectory of the target using the first velocity of the corrected target have.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and a method for tracking a trajectory of a target using an image sensor and a radar sensor,

It is an object of the present invention to provide an apparatus and a method for tracing a trajectory by analyzing a motion of a moving target. More particularly, the present invention relates to an apparatus and an invention for tracing an orbit by analyzing a motion of a target using an image sensor and a radar sensor.

In the early 1990s, golf simulation systems developed by applying simulation technology to golf in the US, Germany, and Japan were used for research for analyzing distance and ballistics when golf club manufacturers used their products. Since then, the golf game market has been growing rapidly in Korea in the early 2000s, and golf simulator technology and products have been developed and developed using various sensors in Korea.

It is possible to trace basic trajectory by processing image of motion of golf ball with image processing technology. However, due to the problem of resetting the focus of an object moving on the principle of the camera, blurring occurs in the case of an image out of focus. It is not difficult to confirm the direction of the moving target due to such a problem, but it may cause an error in measuring the accurate instantaneous speed.

According to an embodiment of the present invention, there is provided a method and apparatus for measuring accurate trajectory information of a moving target by correcting the velocity information of a target using a radar sensor in image processing through a camera.

There is provided a method of tracking a trajectory of a target using an image sensor and a radar sensor by means of an operation, the method comprising the steps of acquiring, via an image sensor, an image including a moving state of at least one object Detecting a first velocity of the target from the acquired image, detecting a second velocity of the target through the radar sensor, detecting a first velocity of the target Correcting the first velocity using the second velocity by mapping the time of detecting the second velocity of the target with the time of detecting the second velocity of the target and tracking the trajectory of the target using the first velocity of the corrected target have.

Wherein the step of determining a target comprises the steps of: recognizing at least one object moving in the air from a predetermined firing point in a received three-dimensional image; and determining, in at least one or more objects, And setting an object moving to the target as a target.

The step of detecting the first speed may include the steps of removing speed information of the remaining objects excluding the speed information of the target among the speed information of at least one object, measuring the speed of the target by intervals, Determining a velocity as a first velocity, and detecting a second velocity of the target may include sensing movement of at least one or more objects in the signal received via the radar.

Setting at least one object, which is sensed as a stationary state at a launching point, as an object moving in the air as a target; setting velocity information of the remaining objects excluding velocity information of the target among at least one velocity information of the at least one object; Determining a velocity of the target by interval, and determining the velocity of the target measured at intervals to a second velocity.

The step of detecting the first speed of the target may further include detecting the first speed of the target using the number-of-pixels data of the image photographed at a predetermined time interval, and detecting the second speed of the target Acquiring information on the time of emission of the radiation wave emitted from the radar and the information on the reception time of the reflected wave; calculating distance information of the target from the radar based on the acquired information; .

Comparing the detected first speed and the second speed, determining for each predetermined time interval whether a difference between the first speed and the second speed corresponds to a predetermined error range, determining a first speed and a second speed Replacing the first velocity with a second velocity for a time interval out of the error range if the difference between the first velocity and the second velocity is outside a predetermined error range, calculating an average velocity of the target based on the first velocity replaced by the second velocity And outputting the locus of the target based on the calculated average speed of the target.

The step of detecting the second velocity may further include the steps of receiving a radar sensing signal for the target from the radar sensor, signal processing the radar sensing signal to generate a frequency spectrum for the bit signal, Comparing the peak value of the corresponding frequency band to the threshold value, determining the state of the target, and detecting the second speed of the target based on the frequency spectrum for the bit signal

In addition, the image sensor includes a depth camera module for acquiring a depth image of a launch point at which a putt is performed with respect to a target, and the background image from the depth camera module and the end of putting from a point at which the target is positioned at the launch point A step of detecting a target by comparing a screen of the depth image with a depth value of the background image, and extracting a pointer from the target, extracting a pointer from the target based on the depth value of the extracted pointer position, Acquiring a three-dimensional coordinate value of the pointer according to a predetermined time interval, detecting the velocity of the pointer through analysis of the three-dimensional coordinate value, and obtaining an initial velocity of the target from the velocity of the pointer, Correcting the first velocity using the actually measured distance and the correction factor and correcting the first velocity of the target using the second velocity It may include the step.

(여기에서, v₀ 는 포인터의 초기 속도, t는 타겟이 퍼팅된 이후 경과한 시간, a는 가속도를 의미함)Further, the position of the target at a specific time point and the position at which the target finally arrives can be calculated by the following equation.

(Where v ₀ is the initial velocity of the pointer, t is the time elapsed since the target was put, and a is the acceleration)

The step of outputting the trajectory of the target may further include the steps of: receiving environment setting information including the inclination degree and the speed of the putting grass by the user; locating the locus including the direction, angle and distance of the target in accordance with the environment setting information; And reproducing the trajectory of the target in a different velocity mode when displaying the target on the screen.

According to one embodiment, there is provided an apparatus for tracking a trajectory of a target, the apparatus comprising: an image sensor for acquiring an image including an image in which at least one or more objects are moving; A second velocity detector for detecting a velocity of the target by using a radar sensor, a second velocity detector for detecting a velocity of the target by using a radar sensor, A control unit that corrects the first velocity using the second velocity and maps the corrected target locus using the first velocity of the corrected target by mapping the velocity detected time, and a display unit that outputs the locus of the target .

The first speed detector detects the first speed of the target using the pixel number data of the image photographed at a predetermined time interval, and the second speed detector detects the output speed of the radiation wave emitted from the radar sensor And receiving the received time information of the reflected wave, calculating distance information of the target from the radar based on the obtained information, and detecting the second velocity from the distance information.

Here, the radar sensor may be a continuous wave (CW) radar or an FMCW radar. The second speed detector receives a radar sensing signal for the target from the radar sensor, processes the radar sensing signal, The peak value of the frequency band corresponding to the moving speed of the radar sensor is compared with the threshold value to determine the state of the target and to detect the second speed of the target based on the frequency spectrum for the bit signal have.

According to one embodiment, a computer-readable recording medium having recorded thereon a program for executing the method of the present invention can be provided.

According to an embodiment, it is possible to improve the error in the speed measurement caused by the blurring due to the environment occurring when the trajectory is traced using only the image sensor, and the blurring due to the dirt and lens contamination.

In addition, by using one radar, it is possible to supplement the speed measurement of the target with a low-cost device.

FIG. 1 and FIG. 2 are functional diagrams showing a configuration of an apparatus for measuring a trajectory of a target using an image sensor and a radar sensor according to an embodiment.
3 is a flow chart of a method for measuring the trajectory of a target using an image sensor and a radar sensor according to an embodiment.
4 is a diagram for explaining an embodiment for measuring the trajectory of a target according to an embodiment.
5 is an exemplary diagram for explaining detection of an object using a radar, according to an embodiment.
6 is a diagram illustrating a method of tracking a movement of a target and storing a center point according to an embodiment.
7 is a view for explaining an embodiment in which the directional angle of the target is obtained according to a predetermined section according to an embodiment.

Embodiments of the disclosure are capable of various modifications and may have various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that it is not intended to limit the scope of the specific embodiments but includes all transformations, equivalents, and alternatives falling within the spirit and scope of the disclosure disclosed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description of the embodiments of the present invention,

The terms first, second, etc. may be used to describe various elements, but the elements should not be limited by terms. Terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of the claims. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Referring to the accompanying drawings, the same or corresponding elements are denoted by the same reference numerals, and a duplicate description thereof will be omitted.

According to one embodiment, "target" herein may refer to a golf ball or golf ball and club head struck by the user's club.

According to one embodiment, the term "user" in the present specification may mean a person who hits a golf ball using a golf club, an administrator, a machine installer, and the like, but is not limited thereto.

FIG. 1 and FIG. 2 are functional diagrams illustrating a configuration of a trajectory tracking device for measuring a trajectory of a target using an image sensor and a radar sensor according to an embodiment.

1, the trajectory tracking apparatus 100 according to an embodiment includes an image sensor 200, a first velocity detector 210, a radar sensor 220, a second velocity detector 230, a controller (not shown) 300, and a display unit 400. [ However, not all of the components shown in Fig. 1 are essential components of the trajectory tracking device 100. [ The trajectory tracking device 100 may be implemented by more components than the components shown in FIG. 1, or the trajectory tracking device 100 may be implemented by fewer components than the components shown in FIG.

The image sensor 200 can acquire images of the target and the user. The image sensor 200 can capture the movement trajectory of the target. The image sensor 200 may include at least one camera. The image sensor 200 may be used to photograph a plurality of targets. The image sensor 200 can proceed shooting with a shooting angle of 360 degrees.

The image sensor 200 may be installed at the same position as the radar sensor 200. In addition, the image sensor 200 may be installed in a direction in which the user's head is directed. The image sensor 200 is installed in a ceiling scene corresponding to the bottom surface, so that the trajectory of the moving target can be photographed widely.

The first speed detection unit 210 may receive an image including a moving image of at least one or more objects from the image sensor 200. The first velocity detector 210 may determine a target from at least one or more objects and detect a first velocity of the target. In this case, a plurality of targets may be determined. For example, the object included in the golf swing image may be a person swinging a golf swing, a head of a golf club, and a golf ball. Of these, a golf ball may be set as a sole target, The head of the golf club may be set as a target together with the golf ball so as to reflect the moving speed of the head of the golf club at the time of correcting the moving speed of the golf ball.

The first speed may mean the moving speed of the target measured through the image sensor 200.

The first velocity detection unit 210 recognizes at least one object moving from the predetermined launch point to the air in the received three-dimensional image and sets the object recognized as a stationary state at the launch point as an object moving in the air have.

The first speed detector 210 can detect the first speed of the target using the pixel count data of the image taken at a predetermined time interval with respect to the target position.

The first speed detector 210 may remove the speed information of the remaining objects excluding the speed information of the target among the speed information of at least one object. The first speed detecting unit 210 can detect the first speed of the target by setting the golf ball as a target from the image of the user's putting operation.

The first speed detecting unit 210 can recognize the club head and the golf ball from the images that the club head and the golf ball move. The first speed detecting unit 210 may set a golf ball that is moved at a preset firing point as a target after the movement of the club head. The first speed detector 210 may remove the speed information of the remaining object (for example, the club head) except for the golf ball set as the target. That is, by removing the unnecessary speed information, the speed of only the golf ball set as the target can be detected at the first speed. In some cases, if the golf ball and the club head are set as targets, the respective first speeds for the golf ball and the club head can be detected. In this case, the following process is performed for each target, and the velocity of the golf ball can be corrected using the values calculated for the club head in the subsequent trajectory prediction and velocity prediction processes.

The first speed detector 210 may measure the speed of the target by interval and may determine the speed of the target measured in each interval as the first speed.

The first velocity detector 210 can detect the first velocity using the depth information of the moment when the target is hit and the background. This will be described in detail below.

The radar sensor 220 may measure the second speed of the target.

The velocity measurement of the target through the radar sensor 220 can be analyzed through frequency analysis.

의 시간 딜레이 이후 레이더에 수신된다. 여기서 R은 타겟과의 거리이고 c는 빛의 속도(

)이다. 이때 송신한 신호와 수신된 신호를 서로 믹싱하면 그 차이 주파수를 얻어낼 수 있는데 그 주파수는 아래의 수학식 1과 같다.For example, referring to FIG. 5, when an object remote from the radar by a distance R is assumed to be stationary, a frequency waveform according to time is shown. First, when a linearly frequency-modulated signal is transmitted as in the first waveform, it is reflected on an object at a distance R

Lt; RTI ID = 0.0 > delay < / RTI > Where R is the distance from the target and c is the speed of light

)to be. At this time, if the transmitted signal and the received signal are mixed with each other, the difference frequency can be obtained. The frequency is expressed by Equation 1 below.

Based on the difference frequency information calculated in Equation (1), the distance R can be determined by substituting into Equation (2) below.

의 신호와 움직이는 타겟의 상대 속도

가 0이 아니라면, 수신된 신호는 딜레이된 주파수

에 도플러 주파수

가 쉬프트된 주파수

또는

를 가질 수 있다. Further, the radar sensor 220 may be a continuous wave (CW) radar or an FM-CW radar. The radar sensor 200 may use a fixed transmission frequency to measure the speed of the moving target. The radar sensor 200 can measure the velocity of a moving target based on the Doppler frequency shift. If the frequency transmitted by the radar sensor 200

Of the target and the relative speed of the moving target

Is not zero, the received signal is a delayed frequency

Doppler frequency

Shifted frequency

or

Lt; / RTI >

는 도플러 주파수 쉬프트로써 아래의 수학식 3과 같이 결정될 수 있다. here,

Is a Doppler frequency shift and can be determined as shown in Equation (3) below.

는 레이더의 LOS(line of sight)에 따른 속도 요소로 결정될 수 있다. 아래의 수학식 4는 타겟의 상대 속도를 나타내는 수학식이다. Where c is the speed of light and the relative speed of the target

Can be determined as a speed factor according to the line of sight (LOS) of the radar. Equation (4) below is a mathematical expression representing the relative speed of the target.

는 타겟의 실제 속도이고,

는 타겟 궤적과 LOS 사이의 각도를 나타낼 수 있다.here

Is the actual speed of the target,

Can represent the angle between the target trajectory and the LOS.

That is, the method of measuring the velocity based on the radar sensor 200 uses a modulation frequency of the Doppler frequency generated when the radar is fired on a moving object. For example, if a 1024 GHz continuous wave is launched toward an object while the sender of the sensing signal is stationary, a reflected wave with a frequency slightly different from the oscillation frequency returns. The frequency difference (Doppler Frequency) is proportional to the speed of the object. Therefore, knowing the difference of the frequency, the speed of the object can be calculated. Therefore, the radar sensor 200 can be usefully used to accurately measure the speed of the target.

The second speed detector 230 receives a radar sensing signal for the target from the radar sensor 200 and performs signal processing on the radar sensing signal to generate a frequency spectrum for the bit signal, The peak value of the frequency band may be compared to determine the state of the target, and the second speed of the target may be detected based on the frequency spectrum for the bit signal.

The second speed detector 230 senses the movement of at least one object in the signal received through the radar sensor 220 and detects the movement of the at least one object, Can be set as a target.

The second speed detector 230 may remove the speed information of the remaining objects excluding the speed information of the target among the speed information of at least one object. Removing the speed information of the remaining objects can be performed through a noise elimination filter.

The second speed detector 230 may measure the speed of the target for each section and determine the speed of the target measured for each section at a second speed. The second speed detection unit 230 includes a step of obtaining the emission time point of the radiation wave radiated from the radar and the reception time information of the reflected wave, the step of calculating the distance information of the target from the radar based on the acquired information, The second speed can be detected.

The control unit 300 corrects the first velocity using the second velocity by mapping the time of detecting the first velocity of the target to the time of detecting the second velocity of the target, Can be used to track the trajectory of the corrected target.

The control unit 300 controls the first speed detecting unit 230 and the second speed detecting unit 230 so that the time of detecting the first speed of the target received from the first speed detecting unit 210 matches the time of detecting the second speed of the target received from the second speed detecting beam 230, Can be mapped.

The control unit 300 may compare the detected first speed and the second speed. The controller 300 may determine whether the difference between the first speed and the second speed corresponds to a predetermined error range according to a predetermined time interval.

If the difference between the first speed and the second speed deviates from the predetermined error range, the controller 300 may replace the first speed with the second speed for a time interval out of the error range. That is, the first speed may be lowered in speed accuracy depending on the focus, performance, image quality, etc. of the image sensor 200. The controller 300 may correct the first speed using the second speed.

The controller 300 compares the first speed and the second speed at the mapped time and determines the difference between the first speed and the second speed according to a predetermined time interval that corresponds to a predetermined error range. If the difference between the first speed and the second speed is out of the predetermined error range, the controller 300 may replace the first speed with the second speed for a time interval out of the error range.

Further, the control unit 300 can calculate the average speed of the target based on the first speed replaced at the second speed, and the control unit 300 can track the target's trajectory based on the calculated average speed of the target .

The display unit 400 can output the trajectory of the target. The display unit 400 may display user information, a batting image of a user, a trajectory of a target, environment setting information, and the like.

According to one embodiment, the display unit 1210 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display ), A three-dimensional display (3D display), and an electrophoretic display (electrophoretic display). According to the implementation of the trajectory tracking device 100, the trajectory tracking device 100 may include two or more display portions 400. At this time, the two or more display units 400 may be arranged to face each other using a hinge.

2, the trajectory tracking apparatus 1000 according to some embodiments includes, in addition to the user input unit 1100, the output unit 1200, the control unit 1300, and the communication unit 1500, A sensing unit 1400, an A / V input unit 1600, and a memory 1700. [

The user input unit 1100 means means for the user to input data for controlling the trajectory tracking device 1000. For example, the user input unit 1100 may include a key pad, a dome switch, a touch pad (contact type capacitance type, pressure type resistive type, infrared ray detection type, surface ultrasonic wave conduction type, A tension measuring method, a piezo effect method, etc.), a jog wheel, a jog switch, and the like, but is not limited thereto.

The user input unit 1100 can receive a user input for environment setting of the trajectory tracking device 1000. [ Also, the user input unit 1100 may receive a user input for inputting a trajectory view menu such as viewing, fast viewing, and tracing a trajectory.

The output unit 1200 may output an audio signal or a video signal or a vibration signal and the output unit 1200 may include a display unit 1210, an acoustic output unit 1220, and a vibration motor 1230 have.

The display unit 1210 displays and outputs the information processed by the trajectory tracking apparatus 1000. For example, the display portion 1210 can display the speed of the target corrected by the first speed and the second speed. In addition, the display unit 1210 can display the traced target trajectory using the corrected target speed.

The audio output unit 1220 outputs audio data received from the communication unit 1500 or stored in the memory 1700. The sound output unit 1220 outputs sound signals related to the functions performed by the trajectory tracking apparatus 1000 (for example, a call signal reception tone, a message reception tone, a notification sound). The sound output unit 1220 may include a speaker, a buzzer, and the like.

The vibration motor 1230 can output a vibration signal. For example, the vibration motor 1230 may output a vibration signal corresponding to an output of audio data or video data (e.g., a call signal reception tone, a message reception tone, etc.). In addition, the vibration motor 1230 may output a vibration signal when a touch is input to the touch screen.

The control unit 1300 typically controls the overall operation of the trajectory tracking device 1000. For example, the control unit 1300 may include a user input unit 1100, an output unit 1200, a sensing unit 1400, a communication unit 1500, an A / V input unit 1600 ) Can be generally controlled.

The sensing unit 1400 may sense the state of the trajectory tracking device 1000 or the state around the trajectory tracking device 1000 and may transmit the sensed information to the control unit 1300. [

The sensing unit 1400 includes a magnetism sensor 1410, an acceleration sensor 1420, an on / humidity sensor 1430, an infrared sensor 1440, a gyroscope sensor 1450, (GPS) 1460, an air pressure sensor 1470, a proximity sensor 1480, and an RGB sensor (illuminance sensor) 1490, for example. The function of each sensor can be intuitively deduced from the name by those skilled in the art, so a detailed description will be omitted.

The communication unit 1500 may include one or more components that enable communication between the trajectory tracking device 1000 and a user terminal, another server, or the like. For example, the communication unit 1500 may include a local communication unit 1510, a mobile communication unit 1520, and a broadcast receiving unit 1530.

The short-range wireless communication unit 151 includes a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a Near Field Communication unit, a WLAN communication unit, a Zigbee communication unit, IrDA, an infrared data association) communication unit, a WFD (Wi-Fi Direct) communication unit, an UWB (ultra wideband) communication unit, an Ant + communication unit, and the like.

The mobile communication unit 1520 transmits and receives radio signals to and from at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data depending on a voice call signal, a video call signal, or a text / multimedia message transmission / reception.

The broadcast receiving unit 1530 receives broadcast signals and / or broadcast-related information from outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. The locus tracking apparatus 1000 may not include the broadcast receiving unit 1530 according to the embodiment.

The A / V (Audio / Video) input unit 1600 is for inputting an audio signal or a video signal, and may include a camera 1610, a microphone 1620, and the like. The camera 1610 can obtain image frames such as still images or moving images through the image sensor in the video communication mode or the photographing mode. The image captured through the image sensor can be processed through the control unit 1300 or a separate image processing unit (not shown).

The image frame processed by the camera 1610 may be stored in the memory 1700 or may be transmitted to the outside via the communication unit 1500. More than one camera 1610 may be provided according to the configuration of the terminal.

The microphone 1620 receives an external acoustic signal and processes it as electrical voice data. For example, the microphone 1620 may receive acoustic signals from an external device or speaker. The microphone 1620 may use various noise reduction algorithms to remove noise generated in receiving an external sound signal.

The memory 1700 may store a program for processing and control of the control unit 1300 and may store data input to or output from the trajectory tracking apparatus 1000. [

The memory 1700 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory), a RAM (Random Access Memory) SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory) , An optical disc, and the like.

Programs stored in the memory 1700 can be classified into a plurality of modules according to their functions, for example, a UI module 1710, a touch screen module 1720, a notification module 1730, .

The UI module 1710 can provide a specialized UI, a GUI, and the like that are interlocked with the trajectory tracking device 1000 for each application. The touch screen module 1720 senses a touch gesture on the user's touch screen and can transmit information on the touch gesture to the control unit 1300. [ The touch screen module 1720 according to some embodiments may recognize and analyze the touch code. The touch screen module 1720 may be configured as separate hardware including a controller.

Various sensors may be provided in or near the touch screen to sense the touch or near touch of the touch screen. An example of a sensor for sensing the touch of the touch screen is a tactile sensor. A tactile sensor is a sensor that detects the contact of a specific object with a degree or more that a person feels. The tactile sensor can detect various information such as the roughness of the contact surface, the rigidity of the contact object, and the temperature of the contact point.

In addition, a proximity sensor is an example of a sensor for sensing the touch of the touch screen.

The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. Examples of proximity sensors include a transmission type photoelectric sensor, a direct reflection type photoelectric sensor, a mirror reflection type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor. The user's touch gestures can include tap, touch & hold, double tap, drag, panning, flick, drag and drop, swipe, and the like.

The notification module 1730 may generate a signal for notifying the occurrence of an event of the trajectory tracking device 1000. [ Examples of events generated in the trajectory tracking apparatus 1000 include call signal reception, message reception, key signal input, schedule notification, and the like. The notification module 1730 may output a notification signal in the form of a video signal through the display unit 1210 or may output a notification signal in the form of an audio signal through the sound output unit 1220, It is possible to output a notification signal in the form of a vibration signal.

3 shows a flow chart of a method for tracking a target trajectory using an image sensor and a radar sensor, according to an embodiment.

In step S310, the trajectory tracking device 100 can acquire an image including the moving state of at least one or more objects through the image sensor. The trajectory tracking device 100 can acquire an image including the moving state of the target through an image sensor installed around the target to photograph the target. More specifically, the trajectory tracking device 100 can acquire an image in which the stopped golf ball moves to the air when the golf ball is struck by the user's club head. Also, the image may include a scene in which the club head is moved.

In step S320, the trajectory tracking device 100 may determine a target from at least one or more objects.

That is, the trajectory tracking device 100 can recognize a club head and a golf ball moved by a user as objects. The trajectory tracking device 100 can determine a golf ball as a target.

More specifically, the trajectory tracking device 100 can recognize at least one object moving from the predetermined firing point to the air in the image acquired from the image sensor. The trajectory tracking device 100 can target an object that is recognized as a stationary state at a predetermined launch point to the air. The la-la determined launch point, can mean the bottom side, which leaves the golf ball to strike the golf ball.

In step S330, the trajectory tracking device 100 can detect the first speed of the target from the acquired image. The speed of the target can be detected by analyzing the time at which the image was taken and the distance the target moves in the image.

In step S340, the trajectory tracking device 100 can detect the second speed of the target via the radar sensor. The trajectory tracking apparatus 100 can acquire the emission time point of the radiation wave radiated from the radar sensor and the reception time point information of the reflected wave.

The trajectory tracking device 100 can calculate the distance information of the target from the radar based on the acquired information. The trajectory tracking device 100 may include detecting a second velocity from the calculated distance information.

In step S350, the trajectory tracking device 100 can correct the first velocity using the second velocity by mapping the time of detecting the first velocity of the target to the time of detecting the second velocity of the target.

In step S360, the trajectory tracking device 100 can track the trajectory of the target using the first speed of the corrected target. That is, when the velocity obtained from the image sensor is not clear, the trajectory tracking apparatus 100 can track the trajectory by correcting the velocity of the target using the velocity measured using the radar sensor.

In more detail, the trajectory tracking device 100 can compare the detected first speed and second speed. The trajectory tracking device 100 can determine whether the difference between the first speed and the second speed corresponds to a predetermined error range by a predetermined time interval. The trajectory tracking apparatus 100 may replace the first speed with the second speed for a time interval out of the error range when the difference between the first speed and the second speed deviates from a predetermined error range.

Further, the trajectory tracking device 100 can calculate the average speed of the target based on the first speed replaced at the second speed, and the trajectory tracking device 100 can calculate the average speed of the target Trajectory can be tracked.

4 is a diagram for explaining an embodiment for measuring the trajectory of a target according to an embodiment.

The trajectory tracking device 100 can track the trajectory of the target 6 using the image sensor 200 and the radar sensor 220 when the target 6 moves from the firing point to the air due to the impact of the user have.

The trajectory tracking device 100 can acquire images in which the club head 4 and the golf ball 6 are moved when the golf ball is hit by the user.

Referring to FIG. 4, it can be seen that the club head 4 is moved close to the launch point and then moved together with the movement of the golf ball 6. The club head 4 is fixed by the user and can move within the movement radius of the user by the force after striking the golf ball 6. In FIG. 4, it can be confirmed that the club head 4 moves in the proceeding direction 10 → 12 → 14.

It can be confirmed that the golf ball 6 is positioned in a stationary state at the firing point and is moved by 16 → 18 → 20 after being hit by the club head 4. [ The trajectory tracking device 100 can confirm the movement of the golf ball 6 and the club head 4 and can target the golf ball 6 that has been stationary at the firing point and moved to the air. The trajectory tracking device 100 can determine the vertical and horizontal angles of the golf ball 6 through sequence analysis of the images before and after launching the golf ball 6 at the launch point. That is, the trajectory tracking device 100 can detect the first speed through analysis of the moving image of the golf ball 6 set as the target.

In addition, the trajectory tracking device 100 can detect the second speed of the golf ball 6 via the radar sensor 220. The trajectory tracking device 100 can extract the Doppler frequency corresponding to the velocity component of the target using the CW radar. The trajectory tracking device 100 can detect the second speed of the target by analyzing the transmission signal and the reflection signal for the target.

The trajectory tracking device 100 can find the error of the velocity by mapping the first velocity and the second velocity at the same time. The locus tracking apparatus 100 can correct the first speed information using the second speed with respect to the time period in which the speed error occurs. That is, the trajectory tracking device 100 can compensate for the error of the first speed due to the error of the image sensor 200 to the second speed measured through the radar sensor 220. Thus, the trajectory tracking device 100 can track the accurate trajectory of the golf ball 6 with improved accuracy.

According to another embodiment, the trajectory tracking apparatus 100 can detect the trajectory of the target through the depth information of the image. The trajectory tracking apparatus 100 judges from the moving trajectory of the head whether the head passes from the outside to the inside and passes from the inside to the outside, using the depth information of the image, and determines the putting direction and angle, And the distance the ball is rolled from the position where the ball is hit.

According to one embodiment, the image sensor may include a depth camera module capable of acquiring depth information of the image.

The trajectory tracking apparatus 100 can acquire a background image from the depth camera module and a depth image from a point at which the target is positioned at the launch point to a point at which the putting ends. The trajectory tracking device 100 can acquire a depth image from the time when the putter is positioned behind the ball to the time when the putting is finished.

The trajectory tracking apparatus 100 can detect a target by comparing the screen of the depth image with the depth value in the background image. Here, the target may mean a hit golf ball.

The trajectory tracking device 100 can extract a pointer from the target. The trajectory tracking device 100 can extract the center point of the target as a pointer.

6 is a diagram illustrating a method of tracking a movement of a target and storing a center point according to an embodiment.

As shown in FIG. 6, if the club head 4 collides with the golf ball 6, two objects collide with each other. The trajectory tracking device 100 can track the movement of the golf ball 6 from the point of collision and store the center point. The trajectory tracking device 100 can know the initial velocity by continuously keeping the ball center point and analyzing it. In addition, since the trajectory tracking device 100 may have different frictional forces between the golf course and the putting place, the trajectory tracking device 100 may correct the actually measured distance S and the calculated correction value.

Referring to Equation (5), let v _{0 be} a value obtained by correcting the initial velocity, let t be the time after putting the golf ball (6), and let the acceleration taking the friction coefficient into consideration into a be a It is possible to calculate the position of the ball.

The trajectory tracking apparatus 100 can calculate the position of the ball at a specific time point and the position at which the ball finally arrives by using Equation (5). The trajectory tracking device 100 can also calculate the velocity of the golf ball 6 based on the calculated position and the measured time.

The trajectory tracking apparatus 100 can calculate the spatial coordinates in the image based on the depth value of the extracted pointer position. In addition, the trajectory tracking apparatus 100 can obtain the three-dimensional coordinate value of the pointer according to a predetermined time period, and can detect the speed of the pointer through analysis of the three-dimensional coordinate value.

7 is a view for explaining an embodiment in which the directional angle of the target is obtained according to a predetermined section according to an embodiment.

As shown in FIG. 7, the trajectory tracking apparatus 100 can obtain the direction angle? From the center to the right or left based on the calculated distance S of the ball and the K value measured every frame. The trajectory tracking device 100 can finally calculate the distance K, which is the distance S between the ball and the direction angle?, And the distance K to the right or left with respect to the center line. The trajectory tracking device 100 can track the trajectory of the target by determining the distance and direction. The trajectory tracking device 100 can analyze the trajectory of the target from three-dimensional coordinates.

The trajectory tracking device 100 can obtain the initial velocity of the target from the velocity of the pointer and correct the first velocity using the actually measured distance and the correction coefficient.

According to another embodiment, the trajectory tracking device 100 can display the direction, angle, distance, etc. of the putting ball according to the setting environment when the user sets the speed, inclination, etc. of the putting grass. In addition, the trajectory tracking device 100 can reproduce a moving image of a putting swing in various speed modes from the viewpoint of a camera view. In addition, all screens can be combined and displayed in overlapping mode. The trajectory tracking device 100 can reproduce a moving image of a putting swing in a variety of speed modes from the viewpoint of a direction in which the golf ball rolls or rolls.

The functions of the various elements shown in the figures may be provided through use of dedicated hardware as well as hardware capable of executing the software in association with the appropriate software. When provided by a processor, such functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of individual processors, some of which may be shared. Also, the explicit use of the term " control portion "or" portion "should not be construed to refer exclusively to hardware capable of executing software and includes, without limitation, digital signal processor (DSP) hardware, (ROM), a random access memory (RAM), and a non-volatile storage device.

In the claims hereof, the elements depicted as means for performing a particular function encompass any way of performing a particular function, such elements being intended to encompass a combination of circuit elements that perform a particular function, Or any form of software, including firmware, microcode, etc., in combination with circuitry suitable for carrying out the software for the processor.

Reference throughout this specification to one embodiment and various modifications of the phrase, means that a particular feature, structure, characteristic, etc., is included in at least one embodiment of the principles of the invention in connection with the embodiment. Thus, the appearances of the phrase "in one embodiment" and any other variation disclosed throughout this specification are not necessarily all referring to the same embodiment.

In this specification, the expression 'at least one of' in the case of 'at least one of A and B' means that only the selection of the first option (A) or only the selection of the second listed option (B) It is used to encompass the selection of options (A and B). As an additional example, in the case of 'at least one of A, B and C', only the selection of the first enumerated option (A) or only the selection of the second enumerated option (B) Only the selection of the first and second listed options A and B or only the selection of the second and third listed options B and C or the selection of all three options A, B, and C). Even if more items are listed, they can be clearly extended to those skilled in the art.

An apparatus according to the present invention may include a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port for communicating with an external device, a user interface such as a touch panel, a key, Devices, and the like. Methods implemented with software modules or algorithms may be stored on a computer readable recording medium as computer readable codes or program instructions executable on the processor. Here, the computer-readable recording medium may be a magnetic storage medium such as a read-only memory (ROM), a random-access memory (RAM), a floppy disk, a hard disk, ), And a DVD (Digital Versatile Disc). The computer-readable recording medium may be distributed over networked computer systems so that computer readable code can be stored and executed in a distributed manner. The medium is readable by a computer, stored in a memory, and executable on a processor.

It is to be understood that all embodiments and conditional statements disclosed herein are intended to assist those of ordinary skill in the art in understanding the principles and concepts of the embodiments disclosed herein without departing from the essential characteristics of the embodiments It is to be understood that the invention may be embodied in various other forms without departing from the spirit or scope of the invention. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the embodiments is set forth in the appended claims rather than the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the embodiments.

Claims (13)

A method of tracking a trajectory of a target using an image sensor and a radar sensor,
Obtaining an image including a moving state of at least one object through the image sensor;
Determining the target from the at least one or more objects;
Detecting a first velocity of the target from the acquired image;
Detecting a second velocity of the target via the radar sensor;
Correcting the first velocity using the second velocity by mapping a time of detecting the first velocity of the target to a time of detecting the second velocity of the target;
Tracking the trajectory of the target using the first speed of the corrected target,
Wherein the determining the target comprises:
Recognizing at least one object moving in the air from a predetermined firing point in the acquired image;
Setting an object recognized as a stationary state at the launch point as a target to be moved to the air in the at least one object,
Wherein detecting the first velocity comprises:
Removing velocity information of a remaining object excluding velocity information of the target among velocity information of the at least one object;
Measuring the velocity of the target by interval; and
Determining a velocity of the target measured at each of the intervals as a first velocity,
Wherein detecting the second speed of the target comprises:
Detecting movement of at least one object in the signal received through the radar;
Setting, in the at least one object, an object detected as a stationary state at the launch point as an object moving to the air;
Removing velocity information of a remaining object excluding velocity information of the target among velocity information of the at least one object;
Measuring the velocity of the target by interval; and
And determining a velocity of the target measured for each of the intervals as a second velocity. delete A method of tracking a trajectory of a target using an image sensor and a radar sensor,
Obtaining an image including a moving state of at least one object through the image sensor;
Determining the target from the at least one or more objects;
Detecting a first velocity of the target from the acquired image;
Detecting a second velocity of the target via the radar sensor;
Correcting the first velocity using the second velocity by mapping a time of detecting the first velocity of the target to a time of detecting the second velocity of the target;
Tracking the trajectory of the target using the first speed of the corrected target,
Wherein detecting the first velocity of the target comprises:
Detecting a first speed of the target using pixel count data of the image captured at a predetermined time interval of the position of the target,
Wherein detecting the second speed of the target comprises:
Obtaining information on the time of emission of the radiated wave from the radar and the time of receiving the reflected wave;
Calculating distance information of the target from the radar based on the obtained information;
And detecting the second velocity from the distance information,
Wherein the step of calibrating the first velocity using the second velocity comprises:
Comparing the first rate and the second rate at the mapped time;
Determining whether a difference between the first speed and the second speed corresponds to a predetermined error range for each predetermined time interval; And
And replacing the first velocity with the second velocity for a time interval outside the error range if the difference between the first velocity and the second velocity is outside a predetermined error range. delete The method according to claim 1,
Wherein the step of detecting the second velocity comprises:
Receiving a radar sensing signal for the target from the radar sensor;
Signal processing the radar sensing signal to generate a frequency spectrum for the bit signal;
Comparing a peak value of a frequency band corresponding to a moving speed of the radar sensor with a threshold value to determine a state of the target;
And detecting a second rate of the target based on the frequency spectrum for the bit signal.
The method according to claim 1,
Wherein the image sensor includes a depth camera module for acquiring a depth image of a launch point at which the putting is performed on the target,
Acquiring a background image from the depth camera module and a depth image from a time point at which the target is located at the launch point to a time point at which the putting ends;
Comparing the screen of the depth image with a depth value of the background image to detect the target, and extracting a pointer from the target;
Calculating spatial coordinates in the image based on a depth value of the extracted pointer position;
Acquiring a three-dimensional coordinate value of the pointer according to a predetermined time interval, and detecting the velocity of the pointer through analysis of the three-dimensional coordinate value;
Obtaining an initial velocity of the target from the velocity of the pointer and correcting the first velocity using an actually measured distance and a correction factor;
And correcting the first velocity of the target using the second velocity.
The method according to claim 6,
Calculating a position of the target at a specific point in time and a position at which the target finally arrives,

(Where v ₀ is the initial velocity of the pointer, t is the time elapsed since the target was put, and a is the acceleration).
The method according to claim 1,
The step of outputting the locus of the target may include:
Receiving environment setting information including a slope and a speed of a putting grass by a user;
Outputting the locus including the direction, angle and distance of the target on the screen according to the environment setting information;
And reproducing the trajectory of the target in a different velocity mode when displaying the target on the screen.
An apparatus for tracking a trajectory of a target,
An image sensor for acquiring an image including an image in which at least one object moves;
A first velocity detector for determining the target from the at least one or more objects and detecting a first velocity of the target;
A second velocity detector for detecting a velocity of the target using a radar sensor;
Correcting the first velocity using the second velocity by mapping the time of detecting the first velocity of the target to the time of detecting the second velocity of the target and using the first velocity of the corrected target A control unit for tracking the corrected target trajectory; And
And a display unit for outputting a locus of the target,
Wherein the first speed detecting unit detects the first speed of the target using the pixel number data of the image taken at a predetermined time interval of the position of the target,
Wherein the second velocity detection unit obtains the emission time of the radiation wave emitted from the radar sensor and the reception time information of the reflected wave and calculates distance information of the target from the radar based on the obtained information, Detecting the second velocity from the information,
Wherein the controller compares the first speed and the second speed at a mapped time and determines for each predetermined time interval whether a difference between the first speed and the second speed corresponds to a predetermined error range, And if the difference in the second velocity deviates from a predetermined error range, replacing the first velocity with the second velocity for a time interval out of the error range, and based on the first velocity replaced with the second velocity And calculates an average velocity of the target. delete delete 10. The method of claim 9,
The radar sensor is a continuous wave (CW) radar,
Wherein the second speed detecting unit comprises:
Receiving a radar sensing signal for the target from the CW radar and signal processing the radar sensing signal to generate a frequency spectrum for the bit signal,
And comparing the peak value of the frequency band corresponding to the movement speed of the CW radar with a threshold value to determine a state of the target and to detect a second speed of the target based on a frequency spectrum of the bit signal .
delete

Images