WO2021256764A1

WO2021256764A1 - Video storage device and video storage method

Info

Publication number: WO2021256764A1
Application number: PCT/KR2021/007164
Authority: WO
Inventors: 최명철; 이호형; 김준오; 백성주
Original assignee: 주식회사 브이씨
Priority date: 2020-06-16
Filing date: 2021-06-08
Publication date: 2021-12-23
Also published as: KR20210155564A

Abstract

A video storage device and a video storage method are provided. The video storage device may comprise: a motion determination module which determines the motion of a golfer by using a first camera, and extracts a time stamp associated with the motion; a storage section configuration module which configures, on the basis of the time stamp, multiple storage sections of a swing video filmed using a second camera; and a video storage module which stores a swing video in a first storage section among the multiple storage sections at a first frame rate, and stores a swing video in a second storage section among the multiple storage sections at a second frame rate different from the first frame rate.

Description

Image storage device and image storage method

The present invention relates to an image storage device and an image storage method.

The golfer determines a target point in consideration of the current position of the golf ball and the position of the hole, selects an appropriate golf club and hits the golf ball so that the golf ball moves to the target point. A golfer's swing motion is very important in order to move the golf ball to a desired target point. A golfer wants to shoot a swing video to analyze his or her swing motion or to refer to the swing motion of another golfer, and accordingly, a technique for recording and reproducing a swing video is attracting attention.

Meanwhile, due to the recent increase in camera resolution, image data occupies a large amount of capacity. For example, even when an image is captured by a non-professional photographing device such as a smart phone, a high-resolution image such as 4k resolution is being introduced as a standard resolution. Therefore, unless a special setting is changed, the user naturally takes high-resolution photos in many cases. Accordingly, even an image captured for a short period of time, such as a swing image, occupies a considerable size of capacity, which not only consumes storage resources of the photographing device or server, but also consumes a lot of network resources for transmitting the captured image. will do Accordingly, there is a need for a method capable of reducing the size of the storage capacity while completely storing the captured swing image.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image storage device and an image storage method capable of reducing storage capacity while providing good reproduction quality to a user with respect to a photographed swing image.

An image storage device according to an embodiment of the present invention includes: a motion determination module for determining a golfer's motion using a first camera and extracting a time stamp related to the motion; a storage section setting module for setting a plurality of storage sections in the swing image captured by the second camera based on the time stamp; and storing the swing image at a first frame rate for a first storage section among the plurality of storage sections, and storing the swing image at a second frame rate different from the first frame rate for a second storage section among the plurality of storage sections. It may include an image storage module.

In some embodiments of the present disclosure, the motion determination module may determine the motion of the golfer by performing image recognition of a golf ball or a club from an image captured by the first camera.

In some embodiments of the present invention, the motion determination module detects at least one of a recognition result for a golf ball or club obtained by performing image recognition, an impact sound detection sensor, an infrared sensor, and a MEMS (Micro Electro Mechanical System) sensor. A golfer's motion can be determined by analyzing the recognition result of the golf ball or club obtained by using the same.

In some embodiments of the present disclosure, the capturing frame rate of the first camera may be higher than the capturing frame rate of the second camera.

In some embodiments of the present disclosure, the storage section setting module may set a plurality of storage sections by matching the time stamp extracted by the motion determination module to the swing image.

In some embodiments of the present invention, the storage interval setting module receives a preset adjustment parameter from the outside, and generates a new time stamp by performing an operation that reflects the adjustment parameter to the time stamp extracted by the operation determination module, and , it is possible to set a plurality of storage sections in the swing image by using the new time stamp.

In some embodiments of the present disclosure, the storage interval setting module may determine a storage start time and a storage end time based on the time stamp extracted by the operation determination module.

An image storage device according to an embodiment of the present invention includes: a motion determination module for determining when a golfer's motion occurs using a sensor and extracting a time stamp for the time; a storage section setting module for setting a plurality of storage sections in a swing image captured using a camera, based on the time stamp; and storing the swing image at a first frame rate for a first storage section among the plurality of storage sections, and storing the swing image at a second frame rate different from the first frame rate for a second storage section among the plurality of storage sections. It may include an image storage module.

In some embodiments of the present invention, the sensor includes an impact sound detection sensor, and the operation determination module determines an impact point by sensing the impact sound of a golf ball and a club using the impact sound detection sensor, and a storage interval setting module may set a plurality of storage sections by matching the time stamp for the impact point to the swing image.

In some embodiments of the present invention, the sensor includes an infrared sensor, the motion determination module determines the impact point by determining the starting point of the golf ball using the infrared sensor, and the storage section setting module is at the impact point A plurality of storage sections can be set by matching the time stamp for the swing to the swing image.

In some embodiments of the present invention, the sensor includes a MEMS sensor, the motion determination module determines the impact point by sensing the amount of impact on the golf ball and the club using the MEMS sensor, and the storage interval setting module is the impact point A plurality of storage sections can be set by matching the time stamp for the swing image with the swing image.

An image storage device according to an embodiment of the present invention receives a swing image captured by using a camera, performs image recognition on a golf ball or club from the swing image to determine the golfer's motion, and time associated with the motion an operation determination module for extracting a stamp; a storage section setting module configured to set a plurality of storage sections in the swing image based on the time stamp; and storing the swing image at a first frame rate for a first storage section among the plurality of storage sections, and storing the swing image at a second frame rate different from the first frame rate for a second storage section among the plurality of storage sections. It may include an image storage module.

In some embodiments of the present invention, the motion determination module, a golf ball obtained by using at least one of a recognition result for a golf ball or club obtained by performing image recognition, an impact sound detection sensor, an infrared sensor, and a MEMS sensor Alternatively, the golfer's motion may be determined by analyzing the recognition result of the club together.

An image storage method according to an embodiment of the present invention includes the steps of: determining an operation of a golfer and extracting a time stamp associated with the operation; setting a plurality of storage sections in the swing image based on the time stamp; and storing the swing image at a first frame rate for a first storage section among the plurality of storage sections, and storing the swing image at a second frame rate different from the first frame rate for a second storage section among the plurality of storage sections. may include steps.

In some embodiments of the present invention, the step of extracting the time stamp includes determining the motion of the golfer by performing image recognition on a golf ball or club from an image captured by a first camera, The step of setting the storage section of may include setting a plurality of storage sections in the swing image captured by the second camera.

In some embodiments of the present disclosure, the step of extracting the time stamp includes a recognition result for a golf ball or club obtained by performing image recognition, and at least one of an impact sound detection sensor, an infrared sensor, and a MEMS sensor. It may include analyzing the recognition result of one golf ball or club together to determine the golfer's motion.

In some embodiments of the present disclosure, the setting of the plurality of storage sections may include setting the plurality of storage sections by matching the extracted time stamp to the swing image.

In some embodiments of the present disclosure, the setting of the plurality of storage sections may include: receiving a preset adjustment parameter from the outside; generating a new time stamp by performing an operation to reflect the adjustment parameter to the extracted time stamp; and setting a plurality of storage sections in the swing image by using the new time stamp.

According to embodiments of the present invention, in consideration of the user's interest in the swing image, by storing the swing image at a different frame rate for each swing motion, it is possible to efficiently use the storage resource of the device in which the swing image is stored, as well as It is possible to minimize the network resources required to transmit the swing image.

In addition, according to embodiments of the present invention, by automating the setting of the storage section of the swing image by reflecting the user's interest in the swing image, the accuracy of the storage section set in the swing image can be increased as well as the swing image It is possible to set a storage section immediately after it is taken.

In addition, according to embodiments of the present invention, by reflecting the user's interest in the swing image to customize the section setting of the swing image, the playback quality of the swing image meets the criteria of the user who wants to analyze the swing image You can provide a swing video to do this.

1 is a block diagram illustrating an image storage system according to an embodiment of the present invention.

2 is a diagram for explaining an operation of an image storage device according to an embodiment of the present invention.

3 is a diagram for explaining an operation of an image storage device according to an embodiment of the present invention.

4 is a flowchart illustrating an image storage method according to an embodiment of the present invention.

5 is a block diagram illustrating an image storage system according to an embodiment of the present invention.

6 is a block diagram illustrating an image storage system according to an embodiment of the present invention.

7 is a block diagram illustrating a computing device for implementing an image storage device and an image storage method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail so that those of ordinary skill in the art can easily implement them with reference to the accompanying drawings. However, the present invention may be implemented in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification and claims, when a part "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. In addition, terms such as "...unit", "...group", and "module" described in the specification mean a unit that processes at least one function or operation, which is hardware or software or a combination of hardware and software. can be implemented as In addition, at least some of the image storage device and the image storage method according to the embodiments described below may be implemented as a program or software, and the program or software may be stored in a computer-readable medium.

Referring to FIG. 1 , an image storage system 1 according to an embodiment of the present invention includes a first camera 3 , a second camera 5 , an image storage device 10 , a memory 20 , and a storage 30 . ) may be included. However, the configuration of the image storage system 1 according to embodiments of the present invention is not limited to that shown in FIG. 1 , and some elements may be changed or omitted in FIG. 1 according to specific implementation purposes. Other elements not shown may be added.

The image storage system 1 may take a swing image by using these elements, and store the captured image at a different frame rate for each swing motion. Specifically, the image storage device 10 classifies the golfer's swing motion based on the image data SD provided from the first camera 3 , and provides it from the second camera 5 according to the divided swing motion After setting a plurality of storage sections for the received image data VD, when storing the image data VD in the memory 20 or the storage 30 , different frame rates may be applied to each of the plurality of storage sections. To this end, the image storage device 10 may include an operation determination module 100 , a storage section setting module 110 , an image storage module 120 , and an image buffer 130 .

The first camera 3 may be disposed near the golfer to photograph the golfer's motion. The first camera 3, for example, may be disposed in front of the golfer or may be disposed on the top of the golfer, but the present invention does not limit the location where the first camera 3 is disposed to a specific location. . The first camera 3 may provide the image data SD of the golfer's motion to the motion determination module 100 .

In some embodiments of the present disclosure, the first camera 3 may include a (second) high-speed camera sensor, a radar sensor, and the like. (Second) high-speed camera sensor is a camera sensor that can shoot high-speed objects that are very fast for the human eye, that is, resolution, shooting frame rate, sensitivity, exposure time, etc. It refers to a sensor implemented as a range. On the other hand, the radar sensor refers to a sensor that detects and measures an object by emitting electromagnetic waves and detecting waves that are reflected from a target and return, for example, pulse radar, continuous wave radar (CW Radar). can be implemented as a Doppler radar, a frequency modulated CW radar (FMCW radar), etc. Therefore, in the present specification, the description thereof will be omitted. However, the scope of the present invention is not limited thereto, and the first camera 3 may be implemented to include a sensor other than a (second) high-speed camera sensor and a radar sensor.

In some embodiments of the present invention, the first camera 3 may be connected to the image storage device 10 through an Ethernet interface, but the scope of the present invention is not limited thereto.

The second camera 5 may generate a swing image by photographing the motion of the golfer. Unlike the first camera 3 , the second camera 5 may be a camera capable of photographing at a normal speed. The second camera 5 may provide the image data VD obtained by photographing the golfer's motion at a normal speed to the image buffer 130 .

In some embodiments of the present disclosure, the capturing frame rate of the first camera 3 may be higher than the capturing frame rate of the second camera 5, which will be described later. For example, the shooting frame rate of the first camera 3 may be 300 fps (frames per second), and the shooting frame rate of the second camera 5 may be 120 fps, but the scope of the present invention is limited thereto. no.

Meanwhile, in some embodiments of the present invention, the second camera 5 may be connected to the image storage device 10 through a USB (Universal Serial Bus) interface, but the scope of the present invention is not limited thereto.

The image buffer 130 may temporarily store the image data VD provided from the second camera 5 . The image data VD temporarily stored in the image buffer 130 is provided to the image storage module 120 and stored in the memory 20 or the storage 30 before being stored in the memory 20 or the storage 30 , as will be described later, the storage interval setting module 120 . A plurality of storage sections may be set by The image data VD for which a plurality of storage sections are set may be stored in the memory 20 or the storage 30 at different frame rates for each of the plurality of storage sections.

Here, the memory 20 may be implemented as a volatile memory including random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), etc., and non-volatile memory including flash memory. It may be implemented as a memory, and may be implemented as any type of memory device not listed. Meanwhile, the storage 30 may be implemented as any type of device, including a solid state drive (SSD) and a hard disk drive (HDD).

The motion determination module 100 may determine the golfer's motion using the first camera 3 and extract a time stamp TS associated with the corresponding motion. Specifically, the motion determination module 100 may determine the motion of the golfer from the image data SD obtained by photographing the motion of the golfer using the first camera 3 . To this end, the motion determination module 100 may perform recognition of the golf ball or club, for example, measure the position and speed of the golf ball or club. The motion determination module 100 may distinguish the motion of the golfer from the image data SD provided from the first camera 3 by tracking changes in the measured position and speed.

Here, the operation of the golfer is, for example, an address operation, a take back operation, an operation in a top state, a down swing operation, an operation at the time of impact, and follow. It may be divided into a follow through operation, a finish operation, etc., but the scope of the present invention is not limited thereto. Arbitrary operations that can be defined according to can be determined.

For convenience of description, in the present specification, the description will be continued on the assumption that the operation determination module 100 mainly determines the operation of the top state, the down swing operation, and the operation at the point in time when the impact occurs.

The motion determination module 100 may determine the golfer's motion by performing image recognition of a golf ball or a club from an image captured by the first camera 3 . Here, the image recognition is through software executing a tracking algorithm for tracking the movement of an object, for example, a golf ball or a club, present in the image when a continuous digital image taken with the first camera 3 is first started. can be performed. Such image recognition is a technology that detects objects by paying attention to the characteristics of objects such as SIFT (Scale Invariant Feature Transform), SURF (Speeded-up Robust Features), Haar, and HOG (Histogram of Oriented Gradients), DPM (Deformable Part- Based Model), a technology that divides and analyzes an object into several parts and connects each part with a machine learning method such as SVM (Support Vector Machine), CNN (Convolutional Neural Network), R-CNN (Region-based Convolutional Neural Network) , Fast R-CNN, R-FCN, YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), etc. can be implemented using techniques using deep learning or artificial intelligence, etc. It is not limited. The motion determination module 100 may acquire information about the position and speed of a golf ball or a club through image recognition.

For example, the motion determination module 100 may determine a point in time when the club speed before impact is 0 as a top-state motion, or from a point in time when the speed of the club is maximum and when the position of the golf ball changes. The time of impact can be determined. As such, the motion determination module 100 may subdivide and determine the motion of the golfer from information on the position and speed of the golf ball or club.

In some embodiments of the present disclosure, the motion determination module 100 includes a recognition result for a golf ball or a club obtained by performing image recognition, an impact sound detection sensor, an infrared sensor, and a MEMS (Micro Electro Mechanical System) sensor A golfer's motion may be determined by analyzing a recognition result of a golf ball or a club acquired using at least one of other sensors such as .

Here, the impact sound detection sensor may be used, for example, to detect the impact sound exceeding a predetermined threshold to detect the time point at which the impact occurs, and the infrared sensor may be used to detect the starting point of the golf ball, The MEMS sensor may be implemented in the form of, for example, a wearable device worn by a golfer, and may be used to detect a time point at which an impact occurs by detecting an amount of impact exceeding a predetermined threshold.

From this, the motion determination module 100, for example, the first impact time obtained by performing image recognition, and the second impact time detected using the impact sound detection sensor, infrared sensor, and MEMS sensor There is a difference In this case, a more accurate impact time may be derived by giving different weights to each of the first impact time and the second impact time.

Next, the operation determination module 100 may extract a time stamp TS associated with the determined operation. For example, when the operation determined by the operation determination module 100 from the image data SD provided from the first camera 3 is an operation in the top state, the operation determination module 100 may include the first camera 3 ), a time stamp TS corresponding to a time point at which an operation of the top state is detected may be extracted based on the system time. In addition, when the action determination module 100 determines the impact from the image data SD provided from the first camera 3 , the action determination module 100 determines the impact based on the system time of the first camera 3 . A time stamp (TS) corresponding to a viewpoint may be extracted. The operation determination module 100 may provide the extracted time stamp TS to the storage interval setting module 110 .

The storage section setting module 110 may set a plurality of storage sections in the swing image VD captured by the second camera 5 based on the time stamp TS. Specifically, the storage section setting module 110 matches the time stamp TS extracted by the operation determination module 100 to the swing image VD based on the system time of the second camera 5 to multiple You can set the storage section of

For example, the storage section setting module 110, based on the time stamp (TS) corresponding to the top state and impact point extracted by the operation determination module 100, the down swing section in the swing image (VD) It can be set as one storage section. Specifically, the storage section setting module 110, by matching the time stamp TS corresponding to the top state to the swing image VD as the system time reference of the second camera 5, the swing image VD standard You can set the starting point of the downswing section. In addition, the storage section setting module 110, by matching the time stamp (TS) corresponding to the impact point to the swing image (VD) as the system time reference of the second camera (5), the swing image (VD) as a standard You can set the end point of the down swing section.

On the other hand, the storage section setting module 110 receives a preset adjustment parameter from the outside, and performs an operation to reflect the adjustment parameter to the time stamp (TS) extracted by the operation determination module 100 to obtain a new time stamp created, and a plurality of storage sections may be set in the swing image VD using a new time stamp.

For example, the storage section setting module 110, based on the time stamp (TS) corresponding to the impact point extracted by the operation determination module 100, and a new time stamp generated by reflecting the adjustment parameter, swing An impact section in the video VD can be set as one storage section.

Specifically, the adjustment parameter includes a first adjustment parameter and a second adjustment parameter, the first adjustment parameter indicates a first predetermined time before the impact time, and the second adjustment parameter indicates a second predetermined time after the impact time , the storage section setting module 110 sets a time stamp corresponding to a time point before the first time from the time stamp corresponding to the impact time point, and the swing image based on the system time of the second camera 5 By matching (VD), the viewpoint of the impact section can be set based on the swing image (VD). In addition, the storage section setting module 110 sets a time stamp corresponding to a time point after a second time from the time stamp TS corresponding to the time of impact, the swing image VD as the system time reference of the second camera 5 ), it is possible to set the end point of the impact section based on the swing image (VD). Accordingly, by using such an adjustment parameter, it is possible to customize the setting of the storage section according to the specific needs of the user.

The storage section setting module 110 may provide the information SI on the set storage section to the image storage module 120 . In some embodiments of the present invention, the information SI on the storage section includes, for example, time stamp information on the start and end points of the storage section, which is matched with the system time reference of the second camera 5 . However, the scope of the present invention is not limited thereto, and the method of providing the information SI on the storage section set by the storage section setting module 110 to the image storage module 120 is specific. Depending on the purpose of the implementation, it can be changed as much as possible.

Meanwhile, in some embodiments of the present disclosure, the storage interval setting module 110 may determine a storage start time and a storage end time based on the time stamp TS extracted by the operation determination module 100 . . Here, the storage start time is time stamp information matched based on the system time of the second camera 5 , and the image storage module 120 actually starts storage among the swing images VD temporarily stored in the image buffer 130 . The storage end time is the time stamp information matched with the system time reference of the second camera 5, and the image storage module 120 of the swing images (VD) temporarily stored in the image buffer 130 is This is the point at which saving actually ends.

The storage start time and storage end time, which mean that the image storage module 120 actually starts or ends the storage, may be different from the start time and end time of the swing image VD. Since the storage start time and the storage end time at which the image storage module 120 actually starts or ends the storage may correspond to the starting point or the ending point of the storage section set in the swing image VD, information about the storage section ( SI) and may be transmitted to the image storage module 120 .

The image storage module 120 stores the swing image at a first frame rate for a first storage section among a plurality of storage sections based on the information SI on the storage section provided from the storage section setting module 110 . and the swing image may be stored at a second frame rate different from the first frame rate for the second storage period among the plurality of storage periods.

For example, the image storage module 120 may store at 60 fps for the down swing section set in the swing image VD, and store it at 120 fps for the impact section set in the swing image VD, and other For the section of , it can be saved at 30 fps.

In the swing of golf, the importance may be different for each swing motion, and some golfers, for example, may be mainly interested in the motion at the point in time when an impact occurs, and other golfers, for example, the downswing You may be mainly interested in the motion and the motion at the time of impact.

The image storage device 10 stores the swing image at a different frame rate for each swing motion in consideration of the user's interest in the swing image, so that it is possible to efficiently use the storage resource of the device in which the swing image is stored, as well as It is possible to minimize the network resources required to transmit the swing image. In addition, as the image storage device 10 automates the setting of the storage section of the swing image by reflecting the user's interest in the swing image, the accuracy of the storage section set in the swing image can be increased as well as the swing image It is possible to set a storage section immediately after it is taken. In addition, the image storage device 10 reflects the user's interest in the swing image to customize the section setting of the swing image, so that the playback quality of the swing image meets the criteria of the user who wants to analyze the swing image You can provide a swing video to do this.

The swing video (VD) stored in this way can be played back at a playback speed corresponding to the stored frame rate, so simply playing the stored video analyzes the swing motion in a concise way that the playback speed is slowed down in the section of interest It is possible to provide a user with a swing image that is convenient to perform.

Referring to FIG. 2 , the image storage device 10 according to an embodiment of the present invention may set a first storage period I to a fourth storage period IV in the swing image VD.

The operation determination module 100 may determine the operation of the golfer in the top state and the operation at the point in time when the impact occurs, and provide a time stamp TS associated with the operation to the storage interval setting module 110 . The operation of the golfer's top state and the operation at the point in time when the impact occurs are performed by performing image recognition on the image data SD photographed by the first camera 3 or by combining methods other than image recognition as described above. Thus, it can be determined by detecting a change in the position and speed of the golf ball and the club.

The storage section setting module 110, based on the time stamp (TS), stores the first storage section (I) to the fourth storage section (IV) in the swing image VD photographed using the second camera 5 . can be set.

To this end, the storage interval setting module 110 may determine a storage start time t1 and a storage end time t4. The storage start time t1 and the storage end time t4 are within a range to prevent omission of the address operation and the takeback operation, the follow-through operation, and the finish operation, but are within the range of the start and end time of the swing image VD. It may be appropriately determined within a range that does not include unnecessary images (eg, other actions unrelated to the swing before the address or after the finish operation).

Also, the storage interval setting module 110 may determine a time point t2. The viewpoint t2 refers to a viewpoint of the top state matched to the swing image VD based on the system time of the second camera 5 .

Also, the storage interval setting module 110 may determine a time point t3. The time point t3 refers to an impact time point matched to the swing image VD based on the system time of the second camera 5 . Furthermore, the storage interval setting module 110 may determine a time point t31 and a time point t32 . The time point t31 reflects the preset adjustment parameter, and refers to a time point corresponding to a time before the first time d1 from the time point t3 corresponding to the impact time point, and the time point t32 is a preset adjustment By reflecting the parameters, it means a time point corresponding to a time point after the second time d2 from the time point t3 corresponding to the impact point.

From this, the storage section setting module 110 corresponds to the first storage section I ('up swing section') corresponding to the time point (t1) to the time point (t2), the time point (t2) to the time point (t31). The second storage period (II) ('down swing period'), the third storage period (III) ('impact period') corresponding to the time point t31 to the time point t32, and the time point t32 to the time point t4) A fourth storage period IV ('follow swing period') corresponding to may be set.

The storage section setting module 110 provides such setting information SI to the image storage module 120, and accordingly, the image storage module 120 provides the swing image VD, the first storage section I and For the fourth storage period (IV), a first frame rate, for example, 30 fps, is stored in the memory 20 or storage 30, and for the second storage period (II), a second frame rate, for example, 60 fps. It may be stored in the memory 20 or the storage 30 , and may be stored in the memory 20 or the storage 30 at a third frame rate, for example, 120 fps for the third storage period III.

Referring to FIG. 3 , the image storage device 10 according to an embodiment of the present invention may set a first storage period I to a fifth storage period V in the swing image VD.

The operation determination module 100 determines the operation of the golfer in the top state, the operation at the point in time when the down swing is in progress between the impacts in the top state, and the operation at the point in time when the impact occurs, and a time stamp (TS) associated with the operation may be provided to the storage interval setting module 110 . The operation of the golfer in the top state, the operation at the point in time when the down swing is in progress between the top state and the impact, and the operation at the point in time when the impact occurs, the image recognition is performed for the image data SD captured by the first camera 3 . Or, as described above, by combining a method other than image recognition, it can be determined by detecting a change in the position and speed of the golf ball and the club.

The storage section setting module 110, on the basis of the time stamp (TS), the first storage section (I) to the fifth storage section (V) in the swing image (VD) photographed using the second camera (5) can be set.

To this end, as described above with reference to FIG. 2 , the storage interval setting module 110 may determine a storage start time t1 and a storage end time t5 .

Also, the storage interval setting module 110 may determine a time point t3. The time point t3 refers to a time point at which the down swing progresses in the middle between impacts in the top state, which is matched to the swing image VD based on the system time of the second camera 5 .

Also, the storage interval setting module 110 may determine a time point t4. The time point t4 refers to an impact time point matched to the swing image VD based on the system time of the second camera 5 . Furthermore, the storage interval setting module 110 may determine the time points t41 and t42, as described above with respect to the time points t31 and t32 of FIG. 2 .

From this, the storage section setting module 110 is a first storage section (I) ('up swing section') corresponding to the time point (t1) to the time point (t2), the time point (t2) to the time point (t3) corresponding to The second storage period (II) ('first downswing period'), the third storage period (III) corresponding to the time point (t3) to the time point (t41) ('second downswing period'), the time point (t41) A fourth storage section IV ('impact section') corresponding to time t42 to time t42 and a fifth storage section V corresponding to time point t42 to time t5 ('follow swing section') are set. can

The storage section setting module 110 provides such setting information SI to the image storage module 120, and accordingly, the image storage module 120 provides the swing image VD, the first storage section I and For the fifth storage period (V), a first frame rate, for example, 24 fps, is stored in the memory 20 or storage 30, and for the second storage period (II), a second frame rate, for example, 30 fps. Stored in the memory 20 or storage 30, and stored in the memory 20 or storage 30 at a third frame rate, for example, 60 fps for the third storage period (III), and the fourth storage period ( IV) may be stored in the memory 20 or the storage 30 at a fourth frame rate, for example, 120 fps.

Referring to FIG. 4 , an image storage method according to an embodiment of the present invention includes determining the golfer's operation (S401), extracting a time stamp related to the golfer's operation (S403), and reflecting the adjustment parameter Generating a new time stamp by performing an operation (S405), setting a plurality of storage sections in the swing image based on the time stamp (S407), a first frame for a first storage section among the plurality of storage sections The method may include storing the swing image at a rate ( S409 ) and storing the swing image at a second frame rate different from the first frame rate for a second storage period among a plurality of storage periods ( S411 ).

In some embodiments of the present invention, step S401 includes determining the motion of a golfer by performing image recognition on a golf ball or club from an image captured by the first camera, step S407 ) may include setting a plurality of storage sections in the swing image captured by the second camera.

In some embodiments of the present invention, step S401 is a golf ball or club obtained by performing image recognition, and a golf ball or club obtained by using at least one of an impact sound detection sensor, an infrared sensor, and a MEMS sensor. It may include analyzing the recognition result of the ball or the club together to determine the golfer's motion.

In some embodiments of the present disclosure, step S407 may include setting a plurality of storage sections by matching the extracted time stamp to the swing image.

In some embodiments of the present invention, step S405 includes receiving a preset adjustment parameter from the outside, generating a new time stamp by performing an operation to reflect the adjustment parameter to the extracted time stamp, and , step S407 may include setting a plurality of storage sections in the swing image by using a new time stamp.

For more detailed information on the method for storing an image according to the present embodiment, reference may be made to the contents described with reference to FIGS. 1 to 3 , and thus a description of overlapping contents will be omitted here.

Referring to FIG. 5 , in the image storage system 2 according to an embodiment of the present invention, the operation determination module 100 uses the sensor 7 to determine when the golfer's motion occurs, as shown in FIG. 1 . It differs from the image storage system 1 described above in relation to this. That is, the image storage system 2 may determine when the golfer's motion occurs using only a technique other than image recognition.

In some embodiments of the present invention, the sensor includes an impact sound detection sensor, and the operation determination module 100 determines the impact point by sensing the impact sound of a golf ball and a club using the impact sound detection sensor, and a storage interval setting module ( 110 ) may set a plurality of storage sections by matching the time stamp for the impact point to the swing image.

In some embodiments of the present invention, the sensor includes an infrared sensor, and the motion determination module 100 determines the impact point by determining the starting point of the golf ball using the infrared sensor, and the storage section setting module 110 ) may set a plurality of storage sections by matching the time stamp for the impact point to the swing image.

In some embodiments of the present invention, the sensor includes a MEMS sensor, and the operation determination module 100 determines the impact point by sensing the amount of impact on the golf ball and the club using the MEMS sensor, and a storage interval setting module ( 110) may set a plurality of storage sections by matching the time stamp for the impact point to the swing image.

Of course, the motion determination module 100 may determine the impact point by combining at least two of the impact sound detection sensor, the infrared sensor, and the MEMS sensor.

Referring to FIG. 6 , the image storage system 3 according to an embodiment of the present invention receives a swing image captured by the operation determination module 100 using the camera 9 and receives the golf ball from the swing image. Alternatively, it is different from the image storage system 1 described above with reference to FIG. 1 and the image storage system 1 described with reference to FIG. 5 in that the golfer's motion is determined by performing image recognition on the club. That is, the image storage system 3 directly performs image recognition on the image data VD photographing the golfer's motion at a normal speed corresponding to the second camera 5 of the image storage system 1, so that golf Changes in position and speed of the ball or club can be detected. To this end, as described above with reference to FIG. 1 , image recognition is when a continuous digital image captured by the first camera 3 is first started, the movement of an object, such as a golf ball or club, existing in the image This may be done through software running a tracking algorithm for tracking. Such image recognition is a technology that detects objects by paying attention to the characteristics of objects such as SIFT, SURF, Haar, and HOG, and a technology that divides and analyzes an object into several parts such as DPM and connects each part with a machine learning method such as SVM , CNN, R-CNN, Fast R-CNN, R-FCN, YOLO, may be implemented using techniques using deep learning or artificial intelligence, such as SSD, but the scope of the present invention is not limited thereto.

Referring to FIG. 7 , an image storage device and an image storage method according to an embodiment of the present invention may be implemented using a computing device 50 .

The computing device 50 includes at least one of a processor 510 , a memory 530 , a user interface input device 540 , a user interface output device 550 , and a storage device 560 in communication via a bus 520 . can do. Computing device 50 may also include network interface 570 electrically connected to network 40 , such as a wireless network. The network interface 570 may transmit or receive signals with other entities through the network 40 .

The processor 510 may be implemented in various types such as an application processor (AP), a central processing unit (CPU), a graphic processing unit (GPU), and the like, and executes a command stored in the memory 530 or the storage device 560 . It may be any semiconductor device that does The processor 510 may be configured to implement the functions and methods described with reference to FIGS. 1 to 6 .

The memory 530 and the storage device 560 may include various types of volatile or non-volatile storage media. For example, the memory may include a read-only memory (ROM) 531 and a random access memory (RAM) 532 . In an embodiment of the present invention, the memory 530 may be located inside or outside the processor 510 , and the memory 530 may be connected to the processor 510 through various known means.

In addition, at least some of the image storage device and the image storage method according to the embodiments of the present invention may be implemented as a program or software executed in the computing device 50, and the program or software may be stored in a computer-readable medium. have.

In addition, at least some of the image storage device and the image storage method according to embodiments of the present invention may be implemented as hardware capable of being electrically connected to the computing device 50 .

According to the embodiments of the present invention described so far, by storing the swing image at a different frame rate for each swing motion in consideration of the user's interest in the swing image, the storage resource of the device in which the swing image is stored can be efficiently used. In addition, it is possible to minimize the network resources required to transmit the swing video.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto. Various modifications and improvements by those with knowledge also fall within the scope of the present invention.

Claims

a motion determination module for determining the motion of the golfer using the first camera and extracting a time stamp associated with the motion;

a storage section setting module configured to set a plurality of storage sections in a swing image captured by a second camera based on the time stamp; and

The swing image is stored at a first frame rate for a first storage section among the plurality of storage sections, and the swing image is stored at a second frame rate different from the first frame rate for a second storage section among the plurality of storage sections. comprising an image storage module for storing images

video storage device.
According to claim 1,

The operation determination module, the image storage device for determining the operation of the golfer by performing image recognition of the golf ball or club in the image taken using the first camera.
3. The method of claim 2,

The motion determination module may include a recognition result of the golf ball or the club obtained by performing the image recognition, and the obtained using at least one of an impact sound detection sensor, an infrared sensor, and a MEMS (Micro Electro Mechanical System) sensor. An image storage device for determining the motion of the golfer by analyzing the recognition result of the golf ball or the club together.
According to claim 1,

The image storage device of claim 1, wherein the shooting frame rate of the first camera is higher than the shooting frame rate of the second camera.
The method of claim 1,

The storage section setting module is configured to match the time stamp extracted by the operation determination module to the swing image to set the plurality of storage sections.
6. The method of claim 5,

The storage section setting module,

Receive a preset adjustment parameter from the outside,

generating a new time stamp by performing an operation reflecting the adjustment parameter to the time stamp extracted by the operation determination module;

An image storage device for setting the plurality of storage sections in the swing image by using the new time stamp.
According to claim 1,

The storage interval setting module is configured to determine a storage start time and a storage end time based on the time stamp extracted by the operation determination module.
an operation determination module for determining a time when a golfer's motion occurs using a sensor and extracting a time stamp for the time;

a storage section setting module for setting a plurality of storage sections in a swing image captured using a camera based on the time stamp; and

The swing image is stored at a first frame rate for a first storage section of the plurality of storage sections, and the swing image is stored at a second frame rate different from the first frame rate for a second storage section of the plurality of storage sections comprising an image storage module for storing images

video storage device.
9. The method of claim 8,

The sensor includes an impact sound detection sensor,

The operation determination module determines an impact point by sensing the impact sound of a golf ball and a club using the impact sound detection sensor,

The storage section setting module is configured to set the plurality of storage sections by matching the time stamp for the impact point to the swing image.
9. The method of claim 8,

The sensor includes an infrared sensor,

The action determination module determines the impact point by using the infrared sensor to determine the starting point of the golf ball,

The storage section setting module is configured to set the plurality of storage sections by matching the time stamp for the impact point to the swing image.
9. The method of claim 8,

The sensor comprises a MEMS sensor,

The motion determination module determines the impact point by sensing the amount of impact on the golf ball and the club using the MEMS sensor,

The storage section setting module is configured to set the plurality of storage sections by matching the time stamp for the impact point to the swing image.
a motion determination module configured to receive a swing image photographed using a camera, perform image recognition for a golf ball or club in the swing image, determine a golfer's motion, and extract a time stamp associated with the motion;

a storage section setting module configured to set a plurality of storage sections in the swing image based on the time stamp; and

The swing image is stored at a first frame rate for a first storage section of the plurality of storage sections, and the swing image is stored at a second frame rate different from the first frame rate for a second storage section of the plurality of storage sections comprising an image storage module for storing images

video storage device.
13. The method of claim 12,

The motion determination module may include a recognition result of the golf ball or the club obtained by performing the image recognition, and the golf ball or the club obtained using at least one of an impact sound detection sensor, an infrared sensor, and a MEMS sensor. An image storage device for determining the operation of the golfer by analyzing the recognition result for the same.
13. The method of claim 12,

The storage section setting module is configured to match the time stamp extracted by the operation determination module to the swing image to set the plurality of storage sections.
15. The method of claim 14,

The storage section setting module,

Receive a preset adjustment parameter from the outside,

generating a new time stamp by performing an operation reflecting the adjustment parameter to the time stamp extracted by the operation determination module;

An image storage device for setting the plurality of storage sections in the swing image by using the new time stamp.
determining the motion of the golfer and extracting a time stamp associated with the motion;

setting a plurality of storage sections in the swing image based on the time stamp; and

The swing image is stored at a first frame rate for a first storage section of the plurality of storage sections, and the swing image is stored at a second frame rate different from the first frame rate for a second storage section of the plurality of storage sections saving the image

How to save video.
17. The method of claim 16,

The step of extracting the time stamp,

Comprising the step of determining the motion of the golfer by performing image recognition of the golf ball or club in the image taken using the first camera,

The setting of the plurality of storage sections includes:

An image storage method comprising the step of setting a plurality of storage sections in a swing image captured by a second camera.
18. The method of claim 17,

The step of extracting the time stamp,

The recognition result of the golf ball or the club obtained by performing the image recognition and the recognition result of the golf ball or the club obtained using at least one of an impact sound detection sensor, an infrared sensor, and a MEMS sensor are analyzed together To, comprising the step of determining the operation of the golfer, an image storage method.
17. The method of claim 16,

The setting of the plurality of storage sections includes:

and setting the plurality of storage sections by matching the extracted time stamp to the swing image.
20. The method of claim 19,

The setting of the plurality of storage sections includes:

receiving a preset adjustment parameter from the outside;

generating a new time stamp by performing an operation to reflect the adjustment parameter to the extracted time stamp; and

and setting the plurality of storage sections in the swing image by using the new time stamp.