TWI647424B - Sensing device for calculating information on position of moving object and sensing method using the same - Google Patents
Sensing device for calculating information on position of moving object and sensing method using the same Download PDFInfo
- Publication number
- TWI647424B TWI647424B TW107101649A TW107101649A TWI647424B TW I647424 B TWI647424 B TW I647424B TW 107101649 A TW107101649 A TW 107101649A TW 107101649 A TW107101649 A TW 107101649A TW I647424 B TWI647424 B TW I647424B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- difference
- difference operation
- images
- primitive
- Prior art date
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B69/00—Training appliances or apparatus for special sports
- A63B69/36—Training appliances or apparatus for special sports for golf
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/254—Analysis of motion involving subtraction of images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B2220/00—Measuring of physical parameters relating to sporting activity
- A63B2220/10—Positions
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63B—APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
- A63B2220/00—Measuring of physical parameters relating to sporting activity
- A63B2220/80—Special sensors, transducers or devices therefor
- A63B2220/806—Video cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
- G06T2207/10012—Stereo images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Physical Education & Sports Medicine (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
本發明提供一種感測裝置以及感測方法,能夠在針對如高爾夫球桿那樣具有無法確定的形狀、大小、材質、反射度等的客體運動進行攝影而得到的圖像中用簡單的方法非常準確且穩定地提取運動的客體,本發明所關係的用於計算運動的客體的位置資訊的感測裝置的感測方法包括:從觀察該運動的客體的視角連續地獲取圖像的步驟;針對基準圖像相對該連續地獲取的各個圖像的各個差分運算圖像,分別形成連續的兩個差分運算圖像的差分運算圖像的步驟;以及根據該連續的兩個差分運算圖像的各個差分運算圖像,計算該運動的客體的位置資訊的步驟。 The present invention provides a sensing device and a sensing method capable of being accurately performed by a simple method in an image obtained by photographing an object motion such as a golf club having an indeterminate shape, size, material, reflectance, or the like. And stably extracting the object of motion, the sensing method of the sensing device for calculating the position information of the object of the motion according to the present invention includes the step of continuously acquiring an image from the perspective of observing the object of the motion; a step of forming a difference operation image of two consecutive difference operation images with respect to each difference operation image of each of the images acquired continuously; and a difference between the two consecutive difference operation images The operation image, the step of calculating the position information of the object of the motion.
Description
本發明屬於能夠連續獲取關於運動的客體的圖像並對其進行分析而從各個圖像提取該客體並據此計算該客體的位置資訊的感測(sensing)裝置以及利用該裝置的感測方法,例如關於在使用者手持高爾夫球桿進行高爾夫揮桿來擊打高爾夫球時通過獲取並分析關於該高爾夫球桿以及高爾夫球的圖像來計算關於該高爾夫球和該高爾夫球桿的運動的感測資料的感測裝置以及利用該裝置的感測方法。 The present invention pertains to a sensing device capable of continuously acquiring an image of a moving object and analyzing the object, extracting the object from each image, and calculating position information of the object, and a sensing method using the same For example, regarding the calculation of the motion of the golf ball and the golf club by acquiring and analyzing an image about the golf club and the golf ball when the user holds the golf club to perform the golf swing to hit the golf ball A sensing device for measuring data and a sensing method using the device.
分析對運動的客體進行攝影而得到的圖像來計算該客體的位置資訊的技術主要以感測裝置的形態利用於在室內或者特定場所通過模擬以交互體育模擬(Interactive Sports Simulation)的形態進行如棒球、足球、籃球以及高爾夫等的熱門體育比賽的多種多樣的模擬器以及用於其的裝置。 The technique of analyzing the image obtained by photographing the moving object to calculate the position information of the object is mainly used in the form of a sensing device for performing an interactive sports simulation in a room or a specific place by simulation. A variety of simulators for popular sports events such as baseball, soccer, basketball, and golf, and devices for them.
如果能夠針對對運動的客體例如在使用者用高爾夫球桿進行高爾夫揮桿來擊打高爾夫球時運動的高爾夫球桿和高爾夫球進行攝影而得到的每幀,計算處於哪個位置,則能夠利用針對每幀計算出的位置資訊 來計算高爾夫球桿和高爾夫球的軌跡、速度、方向角、高度角等各種運動學上的資訊,還能夠利用該資訊來計算使用者針對高爾夫揮桿的分析資訊、或者在如所謂螢幕高爾夫(screen golf)的虛擬高爾夫類比系統中通過影像實現高爾夫球飛行的類比影像。 If it is possible to calculate the position of each of the frames obtained by photographing the moving object such as the golf club and the golf ball when the user performs the golf swing with the golf club, the golf club can be utilized for The position information calculated in each frame is used to calculate various kinematic information such as the trajectory, speed, direction angle, and elevation angle of the golf club and the golf ball, and the information can be used to calculate the analysis information of the user for the golf swing, Or, an analog image of a golf ball flight can be realized by images in a virtual golf analog system such as a so-called screen golf.
通常,在針對某個運動的客體連續獲取的圖像中,運動的客體針對每幀在相互不同的位置出現,另一方面,其背景部分針對每幀不改變而大致固定,所以如果針對一幀的圖像和其他幀的圖像,通過關於各個圖元的圖元值(亮度值)的差分運算(Difference Calculation)形成差值影像(Difference Image),則在該差值影像中背景部分被去除而僅運動的客體部分留下,在圖像中提取運動的客體時廣泛利用差值影像法。 Generally, in an image continuously acquired for a certain moving object, the moving object appears at mutually different positions for each frame, and on the other hand, the background portion thereof is substantially fixed for each frame without change, so if for one frame And the image of the other frame, the difference image is formed by a difference calculation on the primitive value (luminance value) of each primitive, and the background portion is removed in the difference image. Only the part of the moving object remains, and the difference image method is widely used when extracting the object of motion in the image.
圖1是用於說明如上述的差值影像法的附圖,表示在時刻ta時間點攝影的圖像(frame_ta)和在時刻tb時間點攝影的圖像(frame_tb)的差值影像Diff(ta,tb)。 BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a view for explaining a difference image method as described above, showing a difference image Diff (ta) of an image (frame_ta) photographed at a time point ta and an image (frame_tb) photographed at a time point tb. , tb).
差分運算是根據同一位置的圖元之間的圖元值(亮度值)的差而求出的運算,在兩個圖像的同一位置的各圖元的圖元值的差大於0情況下,該差值成為相應圖元的圖元值(亮度值),在兩個圖像的同一位置的各圖元的圖元值的差是小於0的負數的情況下,用0代替來製作差值影像。 The difference calculation is an operation obtained from the difference between the primitive values (luminance values) between the primitives at the same position, and when the difference between the primitive values of the primitives at the same position of the two images is greater than 0, The difference becomes the primitive value (luminance value) of the corresponding primitive. When the difference between the primitive values of the primitives at the same position of the two images is a negative number less than 0, the difference is made with 0 instead. image.
在圖1中,例示針對在x軸方向上14個、在y軸方向上13個的14x13大小的圖像進行差分運算的情況。 FIG. 1 illustrates a case where a difference calculation is performed for 14 images of 14×13 sizes in the x-axis direction and 13 in the y-axis direction.
這樣的差分運算的目的在於,在frame_ta圖像中提取運動的客體(A部分)。如果為此利用時刻tb時間點的圖像frame_tb針對兩個圖像的所有圖元進行差分運算,則如圖1所示,形成Diff(ta,tb)的差值影 像。 The purpose of such a difference operation is to extract the object of motion (Part A) from the frame_ta image. If the image frame_tb at the time point tb is used for the difference operation for all the primitives of the two images, as shown in Fig. 1, a difference image of Diff(ta, tb) is formed.
如從圖1可知,在例如frame_ta圖像的第(1,1)圖元和frame_tb圖像的第(1,1)圖元分別具有20的圖元值時,通過差分運算,圖元值成為0而差值影像Diff(ta,tb)上的第(1,1)圖元以黑色顯示。 As can be seen from FIG. 1, when, for example, the (1st, 1st) element of the frame_ta image and the (1st, 1st) element of the frame_tb image respectively have a primitive value of 20, the difference between the primitive values becomes 0 and the (1, 1)th element on the difference image Diff(ta, tb) is displayed in black.
例如,在frame_ta圖像的第(2,7)圖元的圖元值是50且frame_tb圖像的第(2,7)圖元的圖元值是20時,通過差分運算,圖元值成為50-20=30而差值影像Diff(ta,tb)上的第(2,7)圖元以具有30的圖元值的圖元顯示。 For example, when the primitive value of the (2, 7)th element of the frame_ta image is 50 and the primitive value of the (2, 7)th picture of the frame_tb image is 20, the difference between the primitive values becomes 50-20=30 and the (2,7)th element on the difference image Diff(ta,tb) is displayed as a primitive having a primitive value of 30.
例如,在frame_ta圖像的第(6,10)圖元的圖元值是40且frame_tb圖像的第(6,10)圖元的圖元值是100時,通過差分運算,成為40-100=-60而具有負數值,從而該-60被0代替,圖元值成為0而差值影像Diff(ta,tb)上的第(6,10)圖元以黑色顯示。 For example, when the primitive value of the (6th, 10th) primitive of the frame_ta image is 40 and the primitive value of the (6th, 10th) primitive of the frame_tb image is 100, the differential operation becomes 40-100. =-60 has a negative value, so that -60 is replaced by 0, the primitive value becomes 0, and the (6,10)th element on the difference image Diff(ta, tb) is displayed in black.
在用這樣的方式求出frame_ta圖像相對frame_tb圖像的差值影像時,如果基於照明的運動的客體的反射度等始終恒定且在周邊不存其他變數,則能夠通過差值影像非常準確地提取客體A。 When the difference image of the frame_ta image with respect to the frame_tb image is obtained in such a manner, if the reflectance of the object based on the illumination motion is always constant and there are no other variables in the periphery, the difference image can be very accurately Extract object A.
但是,由於客體運動而針對照明的反射度等可能針對每幀不同,所以形成客體的圖元的圖元值針對每幀可能不同,而且,由於其他物體進入到中間或者運動的某個物體被捕捉到圖像等各種變數,無法通過差值影像始終準確地提取客體A的情況較多。如果在圖1中觀察frame_ta圖像相對frame_tb圖像的差值影像Diff(ta,tb),則觀察到客體A由於差分運算,相當部分受到損傷,而以A'部分顯示。 However, since the reflectance for illumination, etc., may be different for each frame due to object motion, the primitive values of the primitives forming the object may be different for each frame, and an object that is caught in the middle or moved due to other objects is captured. It is often impossible to accurately extract the object A from the difference image by various variables such as images. If the difference image Diff(ta, tb) of the frame_ta image with respect to the frame_tb image is observed in FIG. 1, it is observed that the object A is partially damaged due to the difference operation, and is displayed as the A' portion.
特別,由於客體的色相或者針對照明的反射度、周邊照度的 差異、針對照明的陰影(Shade)、客體的材質等的影響,還有在圖像上客體與背景部分幾乎相似或者反而具有更暗的圖元值的情況等,存在非常難以在圖像上用差值影像法提取高爾夫球桿,即便提取但據此計算出的位置資訊的準確度非常低的問題。 In particular, due to the hue of the object or the reflection of illumination, the difference in ambient illumination, the shadow of the illumination, the material of the object, etc., the object on the image is almost similar to the background part or is darker. In the case of the primitive value, there is a problem that it is very difficult to extract the golf club by the difference image method on the image, and the accuracy of the position information calculated based on the extraction is very low.
與高爾夫球桿不同,在高爾夫球的情況下,由於形狀和大小等被整形化,而且大部分是亮的色相,所以能夠通過差值影像法,非常準確地提取圖像,而且在差值影像過程中,即使高爾夫球部分的一部分圖元損失,由於形狀和大小被整形化,所以也能夠非常準確地擬合(fitting)輪廓線。 Unlike the golf club, in the case of a golf ball, since the shape and size are shaped, and most of them are bright hue, the image can be extracted very accurately by the difference image method, and the difference image is In the process, even if a part of the picture element of the golf ball portion is lost, since the shape and size are shaped, it is possible to fit the outline very accurately.
但是,高爾夫球桿不僅非常難以用差值影像法提取,而且即便提取但用如上述的在高爾夫球的提取中利用的影像處理法也非常難以提取。 However, the golf club is not only very difficult to extract by the difference image method, but even if it is extracted, the image processing method used in the extraction of the golf ball as described above is very difficult to extract.
為了解決這樣的問題,提出了針對高爾夫球桿的桿身以及桿頭等附加特定標記並在圖像中尋找該特定標記來確定高爾夫球桿的位置的方法所關係的技術,但存在在使用者進行高爾夫練習或者進行虛擬的高爾夫比賽時,必須使用附加有特定標記的特定高爾夫球桿這樣的致命的問題,即便附加有特定標記但根據高爾夫揮桿也有相應標記在圖像上未完整地出現或者被遮擋的情況,所以存在難以準確地確定高爾夫球桿的準確的位置的問題。 In order to solve such a problem, a technique relating to a method of determining a position of a golf club by attaching a specific mark to a shaft of a golf club and a head or the like and finding the specific mark in the image has been proposed, but exists in the user. When performing golf practice or performing a virtual golf game, it is necessary to use a fatal problem such as a specific golf club with a specific mark attached, even if a specific mark is attached, the corresponding mark on the golf swing does not appear completely on the image or In the case of being blocked, there is a problem that it is difficult to accurately determine the accurate position of the golf club.
作為與本發明關聯的以往的現有技術文獻,有韓國專利申請第10-2011-0025149號、韓國專利申請第10-2011-0111875、日本公開專利第2005-210666號等。 The prior art documents related to the present invention include Korean Patent Application No. 10-2011-0025149, Korean Patent Application No. 10-2011-0111875, Japanese Patent Publication No. 2005-210666, and the like.
本發明是為了解決如上述的以往的問題而完成的,提供一種用於計算運動的客體的位置資訊的感測裝置以及利用該裝置的感測方法,能夠在針對如高爾夫球桿那樣具有無法確定的形狀、大小、材質、反射度等的客體運動進行攝影而得到的圖像中用簡單的方法非常準確且穩定地提取運動的客體。 The present invention has been made to solve the above conventional problems, and provides a sensing device for calculating position information of a moving object and a sensing method using the same, which can be determined not for a golf club. In the image obtained by photographing the object motion of the shape, size, material, and reflectance, the object of motion is extracted very accurately and stably by a simple method.
本發明的一個實施例所關係的用於計算運動的客體的位置資訊的感測裝置的感測方法的特徵在於,包括:從觀察該運動的客體的視角連續地獲取圖像的步驟;針對基準圖像相對該連續地獲取的各個圖像的各個差分運算圖像,分別形成連續的兩個差分運算圖像的差分運算圖像的步驟;以及根據該連續的兩個差分運算圖像的各個差分運算圖像,計算該運動的客體的位置資訊的步驟。 A sensing method of a sensing device for calculating position information of a moving object related to an embodiment of the present invention is characterized by comprising: a step of continuously acquiring an image from a viewing angle of an object observing the motion; a step of forming a difference operation image of two consecutive difference operation images with respect to each difference operation image of each of the images acquired continuously; and a difference between the two consecutive difference operation images The operation image, the step of calculating the position information of the object of the motion.
另外,優選,本發明的用於計算運動的客體的位置資訊的感測裝置的感測方法的特徵在於,該分別形成連續的兩個差分運算圖像的差分運算圖像的步驟包括:將該連續地獲取的圖像中的一幀的圖像決定為該基準圖像的步驟。 Further, preferably, the sensing method of the sensing device for calculating the position information of the moving object of the present invention is characterized in that the step of respectively forming the difference operation image of the two consecutive difference operation images includes: The image of one frame in the continuously acquired image is determined as the step of the reference image.
另外,優選,本發明的用於計算運動的客體的位置資訊的感測裝置的感測方法的特徵在於,該分別形成連續的兩個差分運算圖像的差分運算圖像的步驟包括:根據基準圖像相對該連續地獲取的各個圖像的差分運算,分別形成包括具有比各圖像上的背景部分更亮的圖元值的圖元及具有比該背景部分更暗的圖元值的圖元這全部的差分運算圖像的步驟;以 及針對該形成的差分運算圖像中的連續的兩個差分運算圖像分別進行差分運算處理,形成差分-差分運算圖像的步驟。 Further, preferably, the sensing method of the sensing device for calculating the position information of the moving object of the present invention is characterized in that the step of respectively forming the difference operation image of the two consecutive difference operation images includes: a difference operation of the image with respect to the successively acquired images, respectively forming a primitive including a primitive value having a brighter background portion than each of the images and a map having a darker primitive value than the background portion a step of calculating a difference image of all of the elements; and performing a difference operation process on the two consecutive difference operation images in the formed difference operation image to form a difference-difference operation image.
另外,優選,本發明的用於計算運動的客體的位置資訊的感測裝置的感測方法的特徵在於,該分別形成連續的兩個差分運算圖像的差分運算圖像的步驟包括:在該連續地獲取的各個圖像中,形成將各個圖元的圖元值和該基準圖像的對應的位置的各圖元的圖元值的差的絕對值作為圖元值的絕對值差分運算圖像的步驟;以及針對連續的兩個該絕對值差分運算圖像,進行如果各個對應圖元的圖元值的差大於0則將該值作為圖元值且如果小於0則將0作為圖元值的差分運算處理,形成差分-差分運算圖像的步驟。 Further, preferably, the sensing method of the sensing device for calculating the position information of the moving object of the present invention is characterized in that the step of respectively forming the difference operation image of the two consecutive difference operation images includes: In each of the images successively acquired, an absolute value of the difference between the primitive values of the primitive values of the respective primitives and the primitive values of the corresponding positions of the reference image is formed as an absolute value difference operation graph of the primitive values. a step of the image; and for two consecutive images of the absolute value difference operation, if the difference between the primitive values of the respective corresponding primitives is greater than 0, the value is taken as the primitive value and if less than 0, 0 is used as the primitive The differential operation of the values is performed to form a differential-differential operation image.
另一方面,本發明的另一實施例所關係的用於計算運動的客體的位置資訊的感測裝置的感測方法的特徵在於,包括:從觀察該運動的客體的視角連續地獲取圖像的步驟;在該連續地獲取的各個圖像中包括背景部分、針對每幀在新的位置出現的位置移動客體、及在連續的圖像上活動變換位置重疊地出現的運動位置重疊部分,以在該連續地獲取的各個圖像中,提取該位置移動客體及運動位置重疊部分且去除該背景部分的方式,執行影像處理的步驟;針對該影像處理後的圖像,通過連續的兩個圖像的差分運算,去除該運動位置重疊部分而提取與該運動的客體相關的位置移動客體的步驟;以及計算該提取的運動的客體的位置資訊的步驟。 In another aspect, a sensing method of a sensing device for calculating position information of a moving object related to another embodiment of the present invention is characterized by comprising: continuously acquiring an image from a viewing angle of an object observing the motion a step of including a background portion, a position moving object appearing at a new position for each frame, and a moving position overlapping portion appearing overlappingly on the continuous image on the successive images, to In each of the continuously acquired images, the step of extracting the position moving object and the moving position overlapping portion and removing the background portion, performing the image processing step; for the image processed image, passing the two consecutive images a difference operation of the image, a step of removing the overlapping portion of the motion position to extract a position moving object related to the object of the motion; and a step of calculating position information of the object of the extracted motion.
另外,優選,本發明的用於計算運動的客體的位置資訊的感測裝置的感測方法的特徵在於,該以去除背景部分的方式執行影像處理的步驟包括:通過基準圖像相對該連續地獲取的各個圖像的差分運算的絕對 值,以包括具有比各圖像的背景部分更亮的圖元值的圖元以及具有比該背景部分更暗的圖元值的圖元這全部的方式,進行影像處理的步驟。 Further, preferably, the sensing method of the sensing device for calculating the position information of the moving object of the present invention is characterized in that the step of performing image processing in such a manner as to remove the background portion includes: continuously, by the reference image, the image Obtaining the absolute value of the difference operation of each image, including all the primitives having the primitive values that are brighter than the background portion of each image and the primitives having the primitive values that are darker than the background portion , the steps of image processing.
另一方面,本發明的一個實施例所關係的用於計算根據使用者的高爾夫揮桿運動的高爾夫球桿的位置資訊的感測裝置的感測方法的特徵在於,包括:從觀察該使用者的高爾夫揮桿的視角連續地獲取圖像的步驟;獲取基準圖像相對該連續地獲取的各個圖像的差分運算的絕對值來形成絕對值差分運算圖像的步驟;針對該形成的絕對值差分運算圖像中的連續的兩個絕對值差分運算圖像,分別進行差分運算處理,形成差分-差分運算圖像的步驟;以及根據該形成的各個差分-差分運算圖像,計算該運動的高爾夫球桿的位置資訊的步驟。 In another aspect, a sensing method of a sensing device for calculating position information of a golf club according to a user's golf swing motion according to an embodiment of the present invention is characterized by comprising: observing the user a step of continuously acquiring an image from a viewing angle of the golf swing; a step of acquiring an absolute value of a difference operation of the reference image with respect to the successively acquired respective images; forming an absolute value for the absolute value a sequence of two consecutive absolute value difference operation images in the difference operation image, respectively performing differential operation processing to form a difference-difference operation image; and calculating the motion based on the formed difference-difference operation image The step of the location information of the golf club.
另一方面,本發明的一個實施例所關係的用於計算運動的客體的位置資訊的感測裝置的特徵在於,包括:攝像機部,從觀察該運動的客體的視角連續地獲取圖像;影像處理部,針對基準圖像相對該連續地獲取的各個圖像的各個差分運算圖像,為了分別提取連續的兩個差分運算圖像的差分運算圖像而執行影像處理;以及資訊計算部,根據由該影像處理部提取的該連續的兩個差分運算圖像的各個差分運算圖像,計算該運動的客體的位置資訊。 In another aspect, a sensing device for calculating position information of a moving object related to an embodiment of the present invention is characterized by comprising: a camera portion that continuously acquires an image from a viewing angle of an object observing the motion; The processing unit performs image processing on the difference image of the reference image for each of the continuously acquired images, in order to extract the difference operation images of the two consecutive difference operation images, respectively; and the information calculation unit according to The difference calculation image of the two consecutive difference operation images extracted by the image processing unit calculates the position information of the object of the motion.
另外,優選,本發明的用於計算運動的客體的位置資訊的感測裝置的特徵在於,該影像處理部構成為將該連續地獲取的圖像中的一幀的圖像決定為該基準圖像,形成將通過該基準圖像相對該連續地獲取的各個圖像的差分運算得到的值的絕對值作為圖元值的絕對值差分運算圖像,通過連續的兩個該絕對值差分運算圖像的差分運算,形成差分-差分運算圖 像。 Further, preferably, the sensing device for calculating the position information of the moving object of the present invention is characterized in that the image processing unit is configured to determine an image of one frame of the continuously acquired images as the reference map. For example, an absolute value difference image obtained by performing a difference calculation between the reference image and each of the continuously acquired images is used as an absolute value difference operation image of the primitive value, and two consecutive absolute difference difference operation maps are formed. The difference operation of the image forms a differential-difference operation image.
另一方面,本發明的一個實施例所關係的用於計算根據使用者的高爾夫揮桿運動的高爾夫球桿的位置資訊的感測裝置的特徵在於,包括:攝像機部,從觀察該使用者的高爾夫揮桿的視角連續地獲取圖像;影像處理部,以獲取基準圖像相對該連續地獲取的各個圖像的差分運算的絕對值而形成絕對值差分運算圖像,並針對該形成的絕對值差分運算圖像中的連續的兩個絕對值差分運算圖像分別進行差分運算處理來形成差分-差分運算圖像的方式,執行影像處理;以及資訊計算部,根據該形成的各個差分-差分運算圖像,計算該運動的高爾夫球桿的位置資訊。 In another aspect, a sensing device for calculating position information of a golf club according to a user's golf swing motion according to an embodiment of the present invention is characterized by comprising: a camera portion from which the user is observed The angle of view of the golf swing continuously acquires an image; the image processing unit acquires an absolute value difference calculation image by acquiring an absolute value of a difference calculation of the reference image with respect to each of the continuously acquired images, and forms an absolute image for the absolute The two consecutive absolute value difference operation images in the value difference operation image are subjected to differential operation processing to form a difference-difference operation image, and image processing is performed; and the information calculation unit performs each differential-difference based on the difference The image is calculated to calculate the position information of the golf club of the movement.
本發明所關係的用於計算運動的客體的位置資訊的感測裝置以及利用該裝置的感測方法具有如下效果,即無需附加特別的標記或者準備另外的裝備,而用針對基準圖像相對連續地獲取的各個圖像的各個差分運算圖像,形成連續的兩個差分運算圖像的差分運算圖像的方式,非常準確且穩定地提取運動的客體並據此計算準確的位置資訊,其中,該連續地獲取的各個圖像是針對如高爾夫球桿那樣具有無法確定的形狀、大小、材質、反射度等的客體運動進行攝影而得到的圖像。 The sensing device for calculating position information of a moving object related to the present invention and the sensing method using the same have the effect of being relatively continuous with respect to the reference image without attaching a special mark or preparing additional equipment. The respective difference operation images of the respective images obtained by the ground form a difference operation image of two consecutive difference operation images, and the moving objects are extracted very accurately and stably, and accurate position information is calculated accordingly, wherein Each of the images acquired continuously is an image obtained by photographing an object motion such as a golf club having an undeterminable shape, size, material, reflectance, or the like.
S10~S14‧‧‧步驟 S10~S14‧‧‧Steps
M1~Mn‧‧‧物件客體 M1~Mn‧‧‧ object object
V1~Vn‧‧‧運動位置重疊部分 V1~Vn‧‧‧ overlapping position of motion position
VO1~Von‧‧‧運動位置重疊部分 VO1~Von‧‧‧ overlap of sports positions
U‧‧‧使用者 U‧‧‧Users
GC‧‧‧高爾夫球桿 GC‧‧‧ golf clubs
100‧‧‧攝像機部 100‧‧‧ camera department
110‧‧‧攝像機 110‧‧‧Camera
120‧‧‧攝像機 120‧‧‧Camera
200‧‧‧感測處理部 200‧‧‧Sensing and Processing Department
210‧‧‧影像處理部 210‧‧‧Image Processing Department
220‧‧‧資訊計算部 220‧‧‧Information Computing Department
300‧‧‧用戶端 300‧‧‧ Client
圖1是用於說明一般在影像處理技術領域中利用的差值影像法的附圖;圖2是表示本發明的一個實施例所關係的感測裝置的結構的框圖; 圖3是表示本發明的一個實施例所關係的感測裝置的感測方法所關係的工序的流程圖;圖4是表示在本發明的一個實施例所關係的感測裝置的攝像機部中連續地獲取的圖像的例子的附圖;圖5是表示利用在圖4中圖示的圖像來形成絕對值差分運算圖像的過程的附圖;以及圖6是表示利用在圖5中圖示的絕對值差分運算圖像來形成差分-差分運算圖像的過程的附圖。 1 is a view for explaining a difference image method generally used in the field of image processing technology; FIG. 2 is a block diagram showing a configuration of a sensing device according to an embodiment of the present invention; A flowchart of a process related to a sensing method of a sensing device according to an embodiment of the present invention; and FIG. 4 is a view showing an image continuously acquired in a camera portion of a sensing device according to an embodiment of the present invention; FIG. 5 is a diagram showing a process of forming an absolute value difference operation image using the image illustrated in FIG. 4; and FIG. 6 is a view showing an absolute value difference illustrated in FIG. 5. A drawing of a process of computing an image to form a differential-difference computed image.
參照附圖,說明關於本發明所關係的用於計算運動的客體的位置資訊的感測裝置以及利用該裝置的感測方法的更具體的內容。 A more specific content of a sensing device for calculating position information of a moving object and a sensing method using the same will be described with reference to the accompanying drawings.
首先,參照圖2,說明本發明的一個實施例所關係的感測裝置的結構。圖2是表示本發明的一個實施例所關係的感測裝置的結構的框圖。 First, the configuration of a sensing device according to an embodiment of the present invention will be described with reference to FIG. Fig. 2 is a block diagram showing the configuration of a sensing device according to an embodiment of the present invention.
本發明作為在分析當使用者用高爾夫球桿進行高爾夫揮桿來擊打高爾夫球時對其進行攝影而得到的圖像並在各個圖像中提取高爾夫球和高爾夫球桿來計算其位置資訊的感測裝置的開發過程中完成的發明,特別提供準確且穩定地提取如高爾夫球桿那樣用以往的影像處理法非常難以在圖像中提取的客體的方法,由於能夠高效地提取如高爾夫球桿那樣難以在圖像中提取的客體,所以不僅是如高爾夫球那樣整形化的客體,而且針對任何不特定的客體,都能夠利用本發明所關係的感測裝置以及感測方 法,通過影像處理,高效地提取基於運動的客體來準確地計算其位置資訊。 The present invention is used as an image obtained by photographing a golf ball when it is hit by a golf club to hit a golf ball, and extracting a golf ball and a golf club in each image to calculate position information thereof. The invention completed in the development of the sensing device particularly provides a method of accurately and stably extracting a guest such as a golf club which is very difficult to extract in an image by a conventional image processing method, since it is possible to efficiently extract a golf club such as a golf club It is difficult to extract the object in the image, so it is not only a object that is shaped like a golf ball, but also can be used for image processing by using the sensing device and the sensing method according to the present invention for any unspecified object. Efficiently extract motion-based objects to accurately calculate their location information.
以下,關於所述的「運動的客體」,主要以高爾夫球桿為例子進行說明,但如上所述,不僅是高爾夫球桿,而且在難以通過影像處理提取的任意的客體的情況下,當然也能夠同樣地應用本發明。 In the following, the "moving object" is mainly described by taking a golf club as an example. However, as described above, not only a golf club but also an arbitrary object that is difficult to be extracted by image processing, of course, The present invention can be applied similarly.
如圖2所示,本發明的一個實施例所關係的感測裝置包括攝像機部100以及感測處理部200,該感測處理部200構成為包括影像處理部210和資訊計算部220。 As shown in FIG. 2, a sensing device according to an embodiment of the present invention includes a camera unit 100 and a sensing processing unit 200. The sensing processing unit 200 includes a video processing unit 210 and an information computing unit 220.
該攝像機部100構成為從觀察運動的客體的視角連續地獲取圖像,為了計算關於運動的客體的三維空間上的位置資訊,該攝像機部100優選使從相互不同的視野角針對同一物件分別獲取圖像的多個攝像機、例如如圖2所示使第一攝像機110和第二攝像機120相互同步化而以立體方式構成。 The camera unit 100 is configured to continuously acquire an image from a viewing angle of a moving object, and in order to calculate positional information on a three-dimensional space of a moving object, the camera unit 100 preferably acquires different viewing angles from each other for the same object. The plurality of cameras of the image, for example, as shown in FIG. 2, synchronize the first camera 110 and the second camera 120 to each other in a three-dimensional manner.
如上所述,攝像機部100的多個攝像機110、120相互同步化而以立體方式構成,從而能夠將在針對同一客體通過第一攝像機110獲取的圖像和通過第二攝像機120獲取的各個圖像中提取的相應客體的二維資訊變換為三維資訊。 As described above, the plurality of cameras 110, 120 of the camera unit 100 are synchronized with each other and configured in a stereoscopic manner, so that the image acquired by the first camera 110 for the same object and the respective images acquired by the second camera 120 can be acquired. The two-dimensional information of the corresponding object extracted in the object is transformed into three-dimensional information.
在圖2中,表示攝像機部100的多個攝像機110、120針對使用者U手持高爾夫球桿GC進行高爾夫揮桿的情形獲取圖像的情況,但本發明不限定於此,能夠利用於例如使用者進行棒球揮桿的情況等各種體育類比系統等。 FIG. 2 shows a case where the plurality of cameras 110 and 120 of the camera unit 100 acquire an image when the user U holds the golf club GC to perform a golf swing. However, the present invention is not limited thereto and can be utilized, for example, for use. Various sports analog systems, such as the case of a baseball swing.
該感測處理部200構成為包括:影像處理部210,從該攝像機部100的各攝像機110、120收集圖像並執行預定的影像處理來提取相應 客體;以及資訊計算部220,根據從圖像提取的客體的二維位置資訊計算三維位置資訊等。 The sensing processing unit 200 includes a video processing unit 210 that collects images from the cameras 110 and 120 of the camera unit 100 and performs predetermined image processing to extract corresponding objects; and an information calculation unit 220 according to the image The two-dimensional position information of the extracted object calculates three-dimensional position information and the like.
該感測處理部200從通過該攝像機部100的各攝像機110、120收集到的各個圖像提取運動的客體,計算相應客體的位置資訊並將該計算出的資訊傳送到用戶端300,該用戶端300執行利用該傳送的客體的位置資訊來計算新的資訊或者計算分析資訊等用戶端300固有的功能。 The sensing processing unit 200 extracts the object of the motion from the respective images collected by the cameras 110 and 120 of the camera unit 100, calculates the location information of the corresponding object, and transmits the calculated information to the user terminal 300, the user. The terminal 300 performs a function inherent to the client 300 such as calculating new information or calculating analysis information by using the location information of the transmitted object.
例如,在將用戶端300構成為在螢幕高爾夫系統中利用的模擬器的情況下,能夠從感測處理部200接受高爾夫球和高爾夫球桿的位置資訊,利用該資訊實現球在虛擬的高爾夫路線上飛行的軌跡的類比影像。 For example, when the user terminal 300 is configured as a simulator used in the screen golf system, the position information of the golf ball and the golf club can be received from the sensing processing unit 200, and the information can be used to realize the virtual golf course of the ball. Analog image of the trajectory on the flight.
此外,在將該用戶端300構成為高爾夫揮桿分析裝置的情況下,能夠從感測處理部200接受高爾夫球和高爾夫球桿的位置資訊,利用該資訊來提供關於使用者的高爾夫揮桿的分析資訊、揮桿的問題診斷以及用於解決該問題的輔導資訊等。 Further, when the user terminal 300 is configured as a golf swing analysis device, the position information of the golf ball and the golf club can be received from the sensing processing unit 200, and the information can be used to provide the golf swing for the user. Analyze information, diagnose problems with swings, and counseling information to solve the problem.
該影像處理部210構成為為了針對基準圖像相對由該攝像機部100連續地獲取的各個圖像的各個差分運算圖像,分別提取連續的兩個差分運算圖像的差分運算圖像,執行影像處理,該資訊計算部220構成為能夠根據由該影像處理部提取的該連續的兩個差分運算圖像的差分運算圖像分別計算該運動的客體的位置資訊。 The image processing unit 210 is configured to extract a difference calculation image of two consecutive difference calculation images for each difference calculation image of each image continuously acquired by the camera unit 100 with respect to the reference image, and execute the image. In the processing, the information calculation unit 220 is configured to be able to calculate the position information of the object of the motion based on the difference calculation image of the two consecutive difference operation images extracted by the image processing unit.
雖然如圖2圖示,該資訊計算部220構成為包含於感測裝置,但不限定於此,還能夠包含於用戶端300。即,感測裝置還能夠執行圖像的獲取以及通過影像處理提取運動的客體的功能,將該提取的資訊傳送到用戶端的資訊計算部,計算該提取的運動的客體的位置資訊並且利用該位置 資訊計算各種資訊。 Although the information calculation unit 220 is included in the sensing device as illustrated in FIG. 2 , the present invention is not limited thereto and may be included in the user terminal 300 . That is, the sensing device is further capable of performing image acquisition and extracting the function of the moving object by image processing, and transmitting the extracted information to the information calculation unit of the user side, calculating position information of the extracted moving object and using the position Information calculates various information.
另一方面,參照在圖3中圖示的流程圖,說明本發明的一個實施例所關係的感測裝置的感測方法。 On the other hand, a sensing method of the sensing device according to an embodiment of the present invention will be described with reference to the flowchart illustrated in FIG.
首先,攝像機部從預定的視角,對使用者用高爾夫球桿進行高爾夫揮桿來擊打球的情形進行攝影,連續地獲取圖像(S10)。 First, the camera unit photographs a situation in which the user performs a golf swing with a golf club to hit the ball from a predetermined angle of view, and continuously acquires an image (S10).
該連續地獲取的圖像被傳遞到感測處理部的影像處理部,該影像處理部在連續地獲取的圖像中,依照預先設定的事項,選定用於差分運算的基準圖像(S11)。 The continuously acquired image is transmitted to the image processing unit of the sensing processing unit, and the image processing unit selects the reference image for the difference calculation in accordance with a predetermined item in the continuously acquired image (S11). .
該基準圖像是用於去除在差值影像中利用的圖像上的背景部分的圖像,該差值影像用於在圖像中提取想要探測的運動的客體(關注物件客體)。 The reference image is an image for removing a background portion on the image utilized in the difference image, the difference image being used to extract the object of the motion to be detected (the object of interest) in the image.
該基準圖像優選選定成去除作為關注物件的運動的客體而只有背景部分的圖像,至少選定成即便作為關注物件的運動的客體在圖像中顯現但顯現於與在其他圖像上出現的客體不重疊的位置的圖像。 The reference image is preferably selected to remove the object of motion as the object of interest and only the image of the background portion, at least selected such that even if the object of motion as the object of interest appears in the image but appears on the other image An image of the location where the objects do not overlap.
例如,在ta<...<t1<t2<...<tn的時刻,分別連續地獲取關於運動的客體的圖像時,如果為了提取關注物件客體,利用在t1、t2...等時間點獲取的圖像,則作為用於針對在t1、t2...等時間點獲取的各個圖像進行差分運算的基準圖像,能夠選定ta時間點的圖像或者tn時間點的圖像。 For example, when the image of the object of the motion is continuously acquired at the time of ta<...<t1<t2<...<tn, if it is used to extract the object of interest, it is used at t1, t2, etc. The image acquired at the time point can be used as a reference image for performing differential calculation on each image acquired at time points such as t1, t2, ..., and can select an image at a time point of ta or an image at a time point at tn. .
在此,為便於說明,將利用基準圖像執行差分運算的t1、t2...等時間點的圖像還稱為「靶心圖表像」。 Here, for convenience of explanation, an image at a time point such as t1, t2, ..., which performs a difference calculation using a reference image is also referred to as a "bulb-eye chart image".
將在哪個時間點獲取的圖像選定成基準圖像能夠根據運動的客體是什麼且如何運動等而個別地決定,例如在高爾夫揮桿時想要從靶 心圖表像提取運動的高爾夫球桿時,優選使在靶心圖表像上高爾夫球桿出現的位置不與在基準圖像上高爾夫球桿出現的位置重疊,所以在所述的例子中,能夠在感測處理部中,預先設定為將作為非常遠離各靶心圖表像的時間點的tn時間點的圖像選定成基準圖像。 The image at which point in time is selected as the reference image can be individually determined depending on what the moving object is and how to move, for example, when the golf club wants to extract the moving golf club from the bull's eye chart image during the golf swing. Preferably, the position where the golf club appears on the bull's-eye chart image does not overlap with the position where the golf club appears on the reference image. Therefore, in the above-described example, the sensing processing unit can be set in advance as The image at the time point tn away from the time point of each of the bullseye chart images is selected as the reference image.
另一方面,如果如上所述選定基準圖像,則通過基準圖像相對各個靶心圖表像的差分運算,提取關注物件客體。然而,本發明所關係的該差分運算並非在前面的背景技術部分說明的一般的差分運算,而是執行兩次差分運算的利用差分運算的差分運算的方法,更具體而言,提供通過針對執行「絕對值差分運算」而形成的「絕對值差分運算圖像」重新執行「差分運算」來形成「差分-差分運算圖像」,準確地提取關注物件客體的方法。 On the other hand, if the reference image is selected as described above, the object of interest object is extracted by the difference operation of the reference image with respect to each of the bull's-eye chart images. However, the differential operation associated with the present invention is not a general differential operation explained in the background section above, but a method of performing differential operation using differential operations for performing two differential operations, and more specifically, by performing for The "absolute value difference calculation image" formed by the "absolute value difference calculation" re-executes the "difference calculation" to form a "difference-differential calculation image", and accurately extracts the object of interest.
其如在圖3的S12以及S13步驟中該,如果如上所述感測處理部的影像處理部選定基準圖像(S11),則影像處理部形成將通過基準圖像相對連續地獲取的各個圖像(靶心圖表像)的差分運算得到的值的絕對值作為圖元值的絕對值差分運算圖像(S12),通過連續的兩個絕對值差分運算圖像的差分運算,形成差分-差分運算圖像(S13)。 As described above, in the steps S12 and S13 of FIG. 3, when the image processing unit of the sensing processing unit selects the reference image as described above (S11), the image processing unit forms each map that is relatively continuously acquired by the reference image. The absolute value of the value obtained by the difference calculation of the (target image) is used as the absolute value difference calculation image of the primitive value (S12), and the differential operation of the image is performed by two consecutive absolute difference calculations to form a difference-differential operation. Image (S13).
即,該「絕對值差分運算」是獲取靶心圖表像上的圖元的圖元值(亮度值)與基準圖像上的對應的位置的圖元的圖元值(亮度值)的差的絕對值的運算,把將所述的兩個圖元值的差的絕對值作為圖元值的圖像稱為「絕對值差分運算圖像」。 In other words, the "absolute value difference calculation" is to obtain the absolute difference between the primitive value (luminance value) of the primitive on the target image image and the primitive value (luminance value) of the primitive at the corresponding position on the reference image. In the calculation of the value, an image in which the absolute value of the difference between the two primitive values is used as the primitive value is referred to as an "absolute value difference calculation image".
另外,將利用所述的絕對值差分運算圖像重新執行差分運算而形成的圖像稱為「差分-差分運算圖像」。 Further, an image formed by re-executing the difference calculation using the above-described absolute value difference calculation image is referred to as a "difference-differential calculation image".
在如上所述執行兩次差分運算而形成的差分-差分運算圖像中,干擾關注物件客體即想提取的運動的客體的要素、即干擾要素被排除,而關注物件客體非常準確地出現,所以能夠從該差分-差分運算圖像容易地提取關注物件客體。 In the difference-difference operation image formed by performing the difference operation twice as described above, the interference object object, that is, the element of the object of the motion to be extracted, that is, the interference element is excluded, and the object of interest object appears very accurately, so The object of interest object can be easily extracted from the difference-difference operation image.
如上所述形成的各差分-差分運算圖像上的與運動的客體相應的部分是利用邊緣資訊來決定輪廓的部分,能夠尋找該決定的輪廓的中心或者重心等特徵位置,將該位置計算為運動的客體的位置資訊(S14)。 The portion corresponding to the moving object on each of the differential-difference operation images formed as described above is a portion that uses the edge information to determine the contour, and can find the feature position such as the center or the center of gravity of the determined contour, and calculates the position as Location information of the moving object (S14).
能夠從以往利用的技術可知,在圖像中提取與運動的客體相應的部分最重要,如果該部分被提取,則在該提取的部分中計算位置資訊。 It can be known from the techniques used in the past that it is most important to extract a portion corresponding to the moving object in the image, and if the portion is extracted, the position information is calculated in the extracted portion.
以下,參照圖4至圖6,利用具體的例子,說明基於本發明的一個實施例所關係的感測裝置的感測方法的「絕對值差分運算圖像」的形成以及「差分-差分運算圖像」的形成。 Hereinafter, the formation of the "absolute value difference calculation image" and the "difference-difference operation map" of the sensing method of the sensing device according to one embodiment of the present invention will be described with reference to FIGS. 4 to 6 . Like the formation of.
圖4表示在本發明的一個實施例所關係的感測裝置的攝像機部中連續地獲取的圖像的例子,圖5表示利用在圖4中圖示的圖像來形成絕對值差分運算圖像的過程,圖6表示利用在圖5中圖示的絕對值差分運算圖像來形成差分-差分運算圖像的過程。 4 shows an example of an image continuously acquired in a camera portion of a sensing device according to an embodiment of the present invention, and FIG. 5 shows an image obtained by using the image illustrated in FIG. 4 to form an absolute value difference operation image. The process of FIG. 6 shows a process of forming a difference-differential operation image using the absolute value difference operation image illustrated in FIG. 5.
在圖4至圖6中表示的各圖像並非實際上通過攝影獲取的圖像,而是為了更高效地說明本發明所關係的感測裝置的感測方法而示意地製作的圖像,為了便於說明和理解,簡化了圖像的結構。 The images shown in FIGS. 4 to 6 are not actually images obtained by photographing, but are images which are schematically produced in order to more effectively explain the sensing method of the sensing device according to the present invention, in order to It is easy to explain and understand, simplifying the structure of the image.
如圖4所示,將通過攝像機部連續地獲取的各圖像設為在t1、t2、t3...tn時間點分別獲取的圖像,在圖4的(a)中,表示在t1時間點獲取的圖像(Frame_t1),在(b)中,表示在t2時間點獲取的圖像(Frame_t2), 在(c)中,表示在t3時間點獲取的圖像(Frame_t3),進而,在(d)中,表示在作為相當遠離t1、t2、t3...時間點的tn時間點獲取的圖像(Frame_tn)。 As shown in FIG. 4, each image continuously acquired by the camera section is set as an image acquired at time points t1, t2, t3, ... tn, respectively, and in (a) of FIG. 4, it is shown at time t1. The image acquired by the point (Frame_t1), in (b), represents the image acquired at time t2 (Frame_t2), and in (c), the image acquired at time t3 (Frame_t3), and further, In (d), an image (Frame_tn) acquired at a time point tn which is quite distant from t1, t2, t3, ... is indicated.
在此,在圖4的(a)至(c)中圖示的各圖像(Frame_t1、Frame_t2、Frame_t3等)用選定成靶心圖表像的圖像來定義,而且,作為在(d)中圖示的圖像的Frame_tn用選定成基準圖像的圖像來定義。 Here, each of the images (Frame_t1, Frame_t2, Frame_t3, and the like) illustrated in (a) to (c) of FIG. 4 is defined by an image selected as a target image of the bull's-eye image, and, as in (d) The Frame_tn of the illustrated image is defined by the image selected as the reference image.
如從圖4的(a)至(d)可知,在連續地獲取的圖像中,在圖像上存在同一背景部分BG。 As can be seen from (a) to (d) of FIG. 4, in the images acquired continuously, the same background portion BG exists on the image.
與想要探測的運動的客體相應的關注物件客體M1、M2、M3...Mn在獲取圖像的t1~tn時間的期間移動的同時,在各圖像上存在於相互不同的位置(雖然可能一部分稍微重複,但優選根據客體的運動速度和攝像機的攝影速度等,大體上針對每幀存在於相互不同的位置)。 The object objects M1, M2, M3, ... Mn corresponding to the object of the motion to be detected move while being in the period of t1 to tn of the acquired image, and exist in mutually different positions on each image (although It is possible that a part is slightly repeated, but it is preferably present in mutually different positions for each frame, depending on the moving speed of the object, the photographing speed of the camera, and the like.
另外,假設在各圖像上,存在雖然在獲取圖像的t1~tn時間的期間移動但不大幅移動或者搖晃或者原地運動等呈現重疊的活動變換的運動位置重疊部分V1、V2、V3...Vn。 In addition, it is assumed that there is a moving position overlapping portion V1, V2, V3 which is superimposed in the period of t1 to tn of the acquired image but does not move largely, or shakes, or moves in place, etc., in each image. ..Vn.
在圖4的(a)至(d)中,可知運動位置重疊部分V1、V2、V3...Vn呈現重疊的活動變換。 In (a) to (d) of Fig. 4, it is understood that the motion position overlapping portions V1, V2, V3, ..., Vn exhibit overlapping motion transitions.
例如,在針對使用者用高爾夫球桿進行高爾夫揮桿的情形連續地獲取圖像的情況下,使用者的身體部分、高爾夫墊部分、橡膠球座部分等與呈現重疊的活動變換的運動位置重疊部分相應,高爾夫球桿與針對每幀呈現不重疊的位置變換的關注物件客體相應。 For example, in the case where an image is continuously acquired for a case where a user performs a golf swing with a golf club, the body part of the user, the golf pad portion, the rubber ball seat portion, and the like overlap with the moving position of the motion change that exhibits overlap. In part, the golf club corresponds to a subject object of interest that exhibits a positional shift that does not overlap for each frame.
在圖5中,表示利用在圖4的(a)至(d)中表示的各靶心圖表像以及基準圖像,在圖3的S12步驟中形成相應的絕對值差分運算圖 像的過程。 In Fig. 5, a process of forming a corresponding absolute value difference operation image in the step S12 of Fig. 3 by using the respective bullseye chart images and reference images shown in (a) to (d) of Fig. 4 is shown.
圖5的(a)表示根據t1時間點的靶心圖表像(Frame_t1)相對基準圖像(Frame_tn)的絕對值差分,形成絕對值差分運算圖像AbsDiff(t1,tn)的情況,圖5的(b)表示根據t2時間點的靶心圖表像(Frame_t2)相對基準圖像(Frame_tn)的絕對值差分運算,形成絕對值差分運算圖像AbsDiff(t2,tn)的情況,圖5的(c)表示根據t3時間點的靶心圖表像(Frame_t3)相對基準圖像(Frame_tn)的絕對值差分運算,形成絕對值差分運算圖像AbsDiff(t3,tn)的情況。 (a) of FIG. 5 shows a case where the absolute value difference operation image AbsDiff(t1, tn) is formed based on the absolute value difference of the target image chart image (Frame_t1) with respect to the reference image (Frame_tn) at time t1, and FIG. 5( b) shows the case where the absolute value difference operation image AbsDiff(t2, tn) is formed based on the absolute value difference calculation of the target image chart (Frame_t2) with respect to the reference image (Frame_tn) at time t2, and (c) of FIG. The absolute value difference operation image AbsDiff(t3, tn) is formed based on the absolute value difference calculation of the bullseye chart image (Frame_t3) with respect to the reference image (Frame_tn) at the time point t3.
如果如上述說明進行絕對值差分運算,則取關於靶心圖表像和基準圖像的圖元值(亮度值)的差的絕對值,所以圖元值的差越大,在絕對值差分運算圖像中,形成越高的圖元值(亮度值)。 When the absolute value difference calculation is performed as described above, the absolute value of the difference between the pixel value (luminance value) of the target image and the reference image is taken, so that the difference between the primitive values is larger, and the difference is calculated in the absolute value. Medium, the higher the primitive value (luminance value) is formed.
於是,靶心圖表像和基準圖像的圖元值實際上相同的背景部分BG如圖5的(a)至(c)所示能夠全部去除,但各個靶心圖表像和基準圖像上的關注物件客體M1、M2、M3和運動位置重疊部分V1、V2、V3、Vn、以及不需要的位置移動客體Mn在絕對值差分運算圖像上非常完整地出現。 Therefore, the background portion BG in which the bullseye chart image and the reference image have substantially the same primitive value can be completely removed as shown in (a) to (c) of FIG. 5, but each of the bullseye chart images and the attention object on the reference image. The objects M1, M2, M3 and the moving position overlapping portions V1, V2, V3, Vn, and the unwanted position moving object Mn appear very completely on the absolute value difference operation image.
如在圖5的(a)中圖示,根據靶心圖表像Frame_t1相對基準圖像Frame_tn的絕對值差分運算,形成絕對值差分運算圖像AbsDiff(t1,tn),在該絕對值差分運算圖像AbsDiff(t1,tn)中,背景部分BG大部分被刪除,存在於相互不同的位置的關注物件客體M1和不需要的位置移動客體Mn由於差分運算的絕對值而幾乎完整地出現,且還存在運動位置重疊部分VO1(V1和Vn重疊相當部分而出現的部分)。 As illustrated in (a) of FIG. 5, an absolute value difference operation image AbsDiff(t1, tn) is formed based on the absolute value difference calculation of the target image chart image Frame_t1 with respect to the reference image Frame_tn, and the absolute value difference operation image is formed. In AbsDiff(t1, tn), the background portion BG is mostly deleted, and the object of interest object M1 and the unnecessary position moving object Mn existing at mutually different positions appear almost completely due to the absolute value of the difference operation, and still exist. The moving position overlaps the portion VO1 (the portion where V1 and Vn overlap by a considerable portion).
此時,該運動位置重疊部分VO1在V1和Vn相互重疊的部分中圖元值相同的部分的情況下,該相同的部分通過絕對值差分運算被刪除後出現。即,雖然在附圖上圖示成完整地出現,但一部分被刪除地出現的情況較多。 At this time, in the case where the moving position overlapping portion VO1 has the same pixel value in the portion where V1 and Vn overlap each other, the same portion appears after being deleted by the absolute value difference operation. That is, although it appears as a complete appearance in the drawing, a part of it appears in a deleted manner.
在圖5的(b)中圖示的根據靶心圖表像Frame_t2相對基準圖像Frame_tn的絕對值差分運算形成絕對值差分運算圖像AbsDiff(t2,tn)的形成中與關注物件客體M2、不需要的位置移動客體Mn以及運動位置重疊部分VO2有關的事項、和在圖5的(c)中圖示的根據靶心圖表像Frame_t3相對基準圖像Frame_tn的絕對值差分運算形成絕對值差分運算圖像AbsDiff(t3,tn)的形成中與關注物件客體M3、不需要的位置移動客體Mn以及運動位置重疊部分VO3有關的事項也與所述的圖5的(a)相同。 The absolute value difference operation image AbsDiff(t2, tn) formed in accordance with the absolute value difference calculation of the target image chart image Frame_t2 with respect to the reference image Frame_tn illustrated in (b) of FIG. 5 is not required to be formed with the object of interest object M2. The matter relating to the positional moving object Mn and the moving position overlapping portion VO2 and the absolute value difference operation based on the absolute value of the target image chart image Frame_t3 with respect to the reference image Frame_tn illustrated in (c) of FIG. 5 form an absolute value difference operation image AbsDiff The matters relating to the object of interest object M3, the unnecessary positional movement object Mn, and the moving position overlapping portion VO3 in the formation of (t3, tn) are also the same as those of (a) of Fig. 5 described above.
於是,通過如上述的絕對值差分運算,與一般的差分運算不同,不僅形成關注物件客體的圖元中的具有高圖元值的部分和具有低的圖元值的部分都在絕對值差分運算圖像上非常完整地出現,而且還能夠使具有低的圖元值的部分成為具有高的圖元值而明顯區別於周邊(即,能夠使在靶心圖表像中較暗地出現的部分在絕對值差分運算圖像中較亮地出現)。 Thus, by the absolute value difference operation as described above, unlike the general difference operation, not only the portion having the high primitive value and the portion having the low primitive value in the primitive of the object of interest is formed in the absolute value difference operation. The image appears very completely, and it is also possible to make a portion with a low primitive value to have a high primitive value and be clearly distinguishable from the periphery (ie, to enable a portion that appears darker in the bull's-eye chart image to be absolute The value difference operation appears brighter in the image).
在此,隨著根據絕對值差分運算刪除背景部分,除了關注物件客體M1、M2、M3、不需要的位置移動客體Mn以及運動位置重疊部分VO1、VO2、VO3以外的剩餘部分幾乎大部分成為0的圖元值而以黑色顯示,但在附圖上為了便於說明和理解,成為0的圖元值的部分全部以白色顯示。 Here, as the background portion is deleted according to the absolute value difference operation, the remaining portions other than the object object objects M1, M2, M3, the unnecessary position moving object Mn, and the motion position overlapping portions VO1, VO2, VO3 become almost 0. The primitive values are shown in black, but for ease of illustration and understanding in the drawings, portions of the primitive values that become 0 are all displayed in white.
作為在圖5的(a)至(c)中分別形成的絕對值差分運算圖 像的AbsDiff(t1,tn)、AbsDiff(t2,tn)以及AbsDiff(t3,tn)等再次通過差分運算形成差分-差分運算圖像,參照圖6對其進行說明。 AbsDiff(t1, tn), AbsDiff(t2, tn), and AbsDiff(t3, tn), which are absolute value difference operation images respectively formed in (a) to (c) of FIG. 5, are again differentially formed by differential operation. The difference calculation image will be described with reference to Fig. 6 .
如參照圖5說明,根據絕對值差分運算,形成AbsDiff(t1,tn)、AbsDiff(t2,tn)以及AbsDiff(t3,tn)等絕對值差分運算圖像,如果針對連續的兩個絕對值差分運算圖像執行差分運算(在此差分運算意味著通常的差分運算),則如圖6所示,能夠獲得差分-差分運算圖像。 As described with reference to FIG. 5, an absolute value difference operation image such as AbsDiff(t1, tn), AbsDiff(t2, tn), and AbsDiff(t3, tn) is formed according to an absolute value difference operation, if two consecutive absolute value differences are formed. When the arithmetic image performs a difference operation (where the difference operation means a normal difference operation), as shown in FIG. 6, a difference-difference operation image can be obtained.
從作為在圖5中圖示的絕對值差分運算圖像的AbsDiff(t1,tn)、AbsDiff(t2,tn)以及AbsDiff(t3,tn)等來看,關注物件客體是M1、M2、M3,運動位置重疊部分VO1、VO2、VO3以及不需要的位置移動客體Mn全部是作為干擾要素而應去除的部分。 From the viewpoints of AbsDiff(t1, tn), AbsDiff(t2, tn), and AbsDiff(t3, tn) which are the absolute value difference operation images illustrated in FIG. 5, the object objects of interest are M1, M2, M3, The moving position overlapping portions VO1, VO2, VO3 and the unnecessary position moving objects Mn are all portions that should be removed as interference elements.
這樣的各干擾要素能夠通過連續的兩個絕對值差分運算圖像的差分運算去除,如圖6的(a)所示,通過絕對值差分運算圖像AbsDiff(t1,tn)相對AbsDiff(t2,tn)的差分運算,在兩個圖像上同樣地存在的部分即位置移動客體Mn和幾乎同樣地存在的部分即運動位置重疊部分VO1以及VO2幾乎大部分被刪除,位置不重疊的關注物件客體M1被保存,由於差分運算而具有負的值的M2部分由於圖元值被0代替而在差分-差分運算圖像DOD(t1,t2|tn)上消失。 Such interference elements can be removed by the difference operation of two consecutive absolute value difference operation images. As shown in (a) of FIG. 6, the image AbsDiff(t1, tn) is calculated by absolute value difference with respect to AbsDiff(t2, The difference operation of tn), the portion that exists equally on the two images, that is, the position shifting object Mn and the portion that exists almost identically, that is, the moving position overlapping portions VO1 and VO2 are mostly deleted, and the object objects of interest that do not overlap each other M1 is saved, and the M2 portion having a negative value due to the difference operation disappears on the difference-difference operation image DOD(t1, t2|tn) because the primitive value is replaced by 0.
同樣地,如圖6的(b)所示,通過絕對值差分運算圖像AbsDiff(t2,tn)相對AbsDiff(t3,tn)的差分運算,在兩個圖像上同樣地存在的部分即位置移動客體Mn和幾乎同樣地存在的部分即運動位置重疊部分VO2以及VO3幾乎大部分被刪除,位置不重疊的關注物件客體M2被保存,由於差分運算而具有負的值的M3部分由於圖元值被0代替而在差分-差分 運算圖像DOD(t2,t3|tn)上消失。 Similarly, as shown in (b) of FIG. 6, the difference between the absolute difference value operation image AbsDiff(t2, tn) and AbsDiff(t3, tn) is the same position on the two images. The moving object Mn and the almost identical portion, that is, the moving position overlapping portions VO2 and VO3 are mostly deleted, the object object M2 whose position is not overlapped is saved, and the M3 portion having a negative value due to the difference operation is due to the primitive value Replaced by 0 and disappeared on the difference-difference operation image DOD(t2, t3|tn).
在此,通過連續的兩個絕對值差分運算圖像的差分運算,不完全重複的運動位置重疊部分VO1以及VO2、VO2以及VO3在差分-差分運算圖像DOD(t1,t2|tn)以及DOD(t2,t3|tn)中分別剩下非常小的部分而可能成為雜訊NZ。 Here, the difference between the moving position overlapping portions VO1 and VO2, VO2, and VO3 in the differential-differential operation image DOD(t1, t2|tn) and DOD is performed by the difference operation of the two consecutive absolute value difference operations. There is a very small portion left in (t2, t3|tn) and may become a noise NZ.
在此,並非通過所述的差分-差分運算,準確地只留下用NZ表示的部分而去除運動位置重疊部分VO1以及VO2或者VO2以及VO3。 Here, the moving position overlapping portions VO1 and VO2 or VO2 and VO3 are removed without accurately leaving only the portion indicated by NZ by the differential-difference operation described above.
例如,絕對值差分運算圖像AbsDiff(t1,tn)上的運動位置重疊部分VO1和絕對值差分運算圖像AbsDiff(t2,tn)上的運動位置重疊部分VO2並非除了NZ部分以外全部完全相同,在兩個運動位置重疊部分中可能存在相互對應的位置的各圖元的圖元值大小相互不同的部分,所以除了實際上在圖6中出現的各差分-差分運算圖像上出現的NZ部分以外,還可能留下無法確定的各種大小的雜訊。 For example, the motion position overlapping portion VO1 on the absolute value difference operation image AbsDiff(t1, tn) and the motion position overlapping portion VO2 on the absolute value difference operation image AbsDiff(t2, tn) are not all identical except for the NZ portion. In the overlapping portion of the two motion positions, there may be portions in which the primitive values of the respective primitives at mutually corresponding positions are different from each other, so in addition to the NZ portion appearing on each differential-difference operation image actually appearing in FIG. In addition, there may be unknown noise of various sizes.
即,在本說明中,為了簡化說明,在圖6中僅將NZ部分表示為雜訊,但實際上除了NZ部分以外,還可能存在其他雜訊,但這樣的雜訊(NZ部分以及其以外的其他雜訊部分)部分能夠應用一般利用的雜訊去除技術來去除。 That is, in the present description, in order to simplify the explanation, only the NZ portion is represented as noise in FIG. 6, but in fact, other noise may exist in addition to the NZ portion, but such noise (the NZ portion and the other portions) Some of the other noise parts can be removed using the commonly used noise removal techniques.
於是,如果如上所述通過連續的兩個絕對值差分運算圖像的差分運算,獲得差分-差分運算圖像,則在該差分-差分運算圖像上,關注物件客體非常完整地出現,所以能夠通過適合的影像處理,準確地確定關注物件客體。 Therefore, if the difference-difference operation image is obtained by the difference operation of the two absolute value difference operation images as described above, on the difference-difference operation image, the object of interest object appears very completely, so Accurately determine the object of interest by appropriate image processing.
如上所述,能夠在差分-差分運算圖像中,在分別提取關注 物件客體、即運動的客體之後,利用關於相應部分的邊緣資訊等,確定輪廓線,確定由該輪廓線決定的部分的中心或者重心等特徵點,計算為位置資訊。 As described above, in the difference-difference operation image, after separately extracting the object of interest, that is, the object of motion, using the edge information about the corresponding portion, etc., the contour line is determined, and the center of the portion determined by the contour line is determined. Or feature points such as center of gravity are calculated as position information.
於是,針對如高爾夫球桿那樣具有無法確定的形狀、大小、材質、反射度等的客體,也能夠通過如上述的本發明所關係的絕對值差分運算圖像以及差分-差分運算圖像的形成非常準確地提取,所以能夠用簡單的方法準確且穩定地計算該運動的客體的位置資訊。 Therefore, the object having the shape, the size, the material, the reflectance, and the like which cannot be determined, such as a golf club, can also be formed by the absolute value difference calculation image and the differential-difference calculation image according to the present invention described above. Very accurate extraction, so the position information of the object of the motion can be accurately and stably calculated in a simple manner.
Claims (7)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
KR1020170012196A KR101932525B1 (en) | 2017-01-25 | 2017-01-25 | Sensing device for calculating information on position of moving object and sensing method using the same |
??10-2017-0012196 | 2017-01-25 |
Publications (2)
Publication Number | Publication Date |
---|---|
TW201827788A TW201827788A (en) | 2018-08-01 |
TWI647424B true TWI647424B (en) | 2019-01-11 |
Family
ID=62978049
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW107101649A TWI647424B (en) | 2017-01-25 | 2018-01-17 | Sensing device for calculating information on position of moving object and sensing method using the same |
Country Status (3)
Country | Link |
---|---|
KR (1) | KR101932525B1 (en) |
TW (1) | TWI647424B (en) |
WO (1) | WO2018139810A1 (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR102256260B1 (en) * | 2020-02-10 | 2021-05-27 | (주) 알디텍 | Smart camera sensor for screen golf |
KR102439549B1 (en) * | 2020-09-14 | 2022-09-02 | 주식회사 골프존 | Device for sensing golf swing and method for sensing impact position on club head using the same |
US20220291139A1 (en) * | 2021-03-15 | 2022-09-15 | The Government Of The United States Of America, As Represented By The Secretary Of The Navy | Foreign object debris detection |
CN114317675B (en) * | 2022-01-06 | 2024-06-21 | 福州大学 | Method and system for detecting bacteria on different wound surfaces based on machine learning qualitative and quantitative detection |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1158270A1 (en) * | 2000-05-24 | 2001-11-28 | Seiko Epson Corporation | Mesuring system for sports events |
US20120203360A1 (en) * | 2007-02-14 | 2012-08-09 | Nike, Inc. | Collection and display of athletic information |
CN103713148A (en) * | 2013-12-13 | 2014-04-09 | 柳州市蓝海科技有限公司 | Golf simulation system ball detection device |
CN104567919A (en) * | 2013-10-12 | 2015-04-29 | 北京航天计量测试技术研究所 | Device for calibrating dynamic measurement errors of photogrammetric system and application method thereof |
TW201644272A (en) * | 2011-11-07 | 2016-12-16 | 英孚布瑞智有限私人貿易公司 | An apparatus of encoding an image in an advanced motion vector prediction mode |
Family Cites Families (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003111880A (en) * | 2001-10-04 | 2003-04-15 | Ditect:Kk | Golf swing analyzing system and image processing method |
JP3782368B2 (en) * | 2002-03-29 | 2006-06-07 | 株式会社東芝 | Object image clipping method and program, and object image clipping device |
KR101078954B1 (en) * | 2011-03-22 | 2011-11-01 | (주) 골프존 | Apparatus for virtual golf simulation, and sensing device and method used to the same |
JP2012212373A (en) * | 2011-03-31 | 2012-11-01 | Casio Comput Co Ltd | Image processing device, image processing method and program |
JP6218351B2 (en) * | 2011-12-06 | 2017-10-25 | ダンロップスポーツ株式会社 | Golf swing diagnosis method |
-
2017
- 2017-01-25 KR KR1020170012196A patent/KR101932525B1/en active IP Right Grant
-
2018
- 2018-01-17 TW TW107101649A patent/TWI647424B/en active
- 2018-01-19 WO PCT/KR2018/000897 patent/WO2018139810A1/en active Application Filing
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP1158270A1 (en) * | 2000-05-24 | 2001-11-28 | Seiko Epson Corporation | Mesuring system for sports events |
US20120203360A1 (en) * | 2007-02-14 | 2012-08-09 | Nike, Inc. | Collection and display of athletic information |
TW201644272A (en) * | 2011-11-07 | 2016-12-16 | 英孚布瑞智有限私人貿易公司 | An apparatus of encoding an image in an advanced motion vector prediction mode |
CN104567919A (en) * | 2013-10-12 | 2015-04-29 | 北京航天计量测试技术研究所 | Device for calibrating dynamic measurement errors of photogrammetric system and application method thereof |
CN103713148A (en) * | 2013-12-13 | 2014-04-09 | 柳州市蓝海科技有限公司 | Golf simulation system ball detection device |
Also Published As
Publication number | Publication date |
---|---|
KR101932525B1 (en) | 2018-12-27 |
WO2018139810A1 (en) | 2018-08-02 |
TW201827788A (en) | 2018-08-01 |
KR20180087748A (en) | 2018-08-02 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
TWI647424B (en) | Sensing device for calculating information on position of moving object and sensing method using the same | |
KR101472274B1 (en) | Device for sensing moving ball and method for the same | |
US11151787B2 (en) | Generation device, generation method and storage medium for three-dimensional model from object images and structure images | |
KR101723432B1 (en) | Device for sensing moving ball and method for the same | |
CA2717485A1 (en) | System and method for tracking three dimensional objects | |
US11798233B2 (en) | Generation device, generation method and storage medium for three-dimensional model that remove a portion of the three-dimensional model | |
US9830712B2 (en) | Method and apparatus for sensing moving ball | |
JP6102002B2 (en) | Sensing device and sensing method for moving ball, and ball image image processing method for calculating spin of moving ball | |
US20180053308A1 (en) | Spatial Alignment of Inertial Measurement Unit Captured Golf Swing and 3D Human Model For Golf Swing Analysis Using IR Reflective Marker | |
US20180053309A1 (en) | Spatial Alignment of M-Tracer and 3-D Human Model For Golf Swing Analysis Using Skeleton | |
JP5335574B2 (en) | Image processing apparatus and control method thereof | |
JP7163049B2 (en) | Information processing device, information processing method and program | |
JP2022016929A (en) | Information processing device, information processing method, and program | |
JP2005123823A (en) | Distance information attaching apparatus, attached video image generating apparatus, attached video information generating method, and distance information attaching program | |
JP7080614B2 (en) | Image processing equipment, image processing system, image processing method, and program | |
KR101461144B1 (en) | Device for sensing moving ball and method for the same | |
JP7520539B2 (en) | Image processing device, image processing method, and program | |
WO2008073563A1 (en) | Method and system for gaze estimation | |
WO2019053790A1 (en) | Position coordinate calculation method and position coordinate calculation device | |
KR100466587B1 (en) | Method of Extrating Camera Information for Authoring Tools of Synthetic Contents | |
KR101551685B1 (en) | Device for sensing moving ball and method for the same | |
JP2020035218A (en) | Image processing device, method, and program | |
Xie et al. | Object tracking method based on 3d cartoon animation in broadcast soccer videos | |
WO2022190206A1 (en) | Skeletal recognition method, skeletal recognition program, and gymnastics scoring assistance system | |
JP2021051374A (en) | Shape data generation device, shape data generation method, and program |