JP2013538067A - Sensing device for moving object, sensing processing method, and virtual golf simulation device using the same - Google Patents

Abstract

The present invention accurately extracts an image of an object on the acquired image even when a multiple exposure image for a moving object is acquired by a low-resolution low-speed camera device and strobe device, By acquiring multiple exposure images for the object to be extracted, even when multiple object images are shown superimposed, the image of the accurately superimposed object is extracted and the coordinates of the center point are extracted Therefore, the present invention is to provide a sensing device for a moving object, a sensing processing method, and a virtual golf simulation device using the same, which can realize high sensing processing capability and sensing accuracy at low cost.
[Selection] Figure 1

Description

  The present invention relates to a sensing device, a sensing processing method, and a virtual golf simulation device using the sensing method, and more specifically, by acquiring an image of a moving object such as a golf ball and analyzing the image, The present invention relates to a sensing device for a moving object that enables calculation of physical information, a sensing processing method, and a virtual golf simulation device using the sensing device.

  Recently, active developments have been made on various devices that allow popular sports competitions such as baseball, soccer, basketball, and golf to be enjoyed in the form of interactive sports games through simulations indoors or at specific locations. Has been made.

  In such an interactive sports game, physical information about the moving object, that is, the movement of the ball, is used in order to simulate a sport using a ball such as a baseball ball, a soccer ball, a basketball, and a golf ball. Research and development on various sensing systems for accurate sensing is very active.

  For example, various sensing systems such as a sensing system using an infrared sensor, a sensing system using a laser sensor, a sensing system using an acoustic sensor, and a sensing system using a camera sensor have appeared.

  The present invention accurately extracts an image of an object on the acquired image even when a multiple exposure image for a moving object is acquired by a low-resolution low-speed camera device and strobe device, By acquiring a multiple exposure image for an object to be reproduced, even when a plurality of object images are shown superimposed, the image of the superimposed object is accurately extracted so that the coordinates of the center point are extracted. The present invention is to provide a sensing device for a moving object, a sensing processing method, and a virtual golf simulation device using the same, which can realize high sensing processing capability and sensing accuracy at low cost. .

  A sensing device for a moving object according to an embodiment of the present invention includes a sensor unit that acquires a multiple exposure image for an object moving under strobe illumination; and an image of an object moving due to multiple exposure from the multiple exposure image. First object processing means for extracting an object image area shown in an overlapping manner, and generating at least two center point candidates for the extracted object image area, and adjusting the position of the center point candidate, thereby the object image A sensing processing unit including second processing means for extracting coordinates of at least two center points with respect to the region.

  In addition, a sensing device according to another embodiment of the present invention includes a sensor unit that obtains a multiple exposure image for an object moving under strobe illumination; a first movement that estimates a movement direction of the object through position information of the initial object At least one of direction estimation means, second movement direction estimation means for estimating the movement direction of an object from pixel values of each image area on the multiple exposure image, and at least one of the first movement direction estimation means and the second movement direction estimation means A sensing processing unit including object extracting means for extracting an image of the object from the image in the motion direction estimated by one.

  According to another embodiment of the present invention, a sensing device includes a sensor unit that acquires a multiple exposure image for a ball moving under strobe illumination; and a ball superimposed on the moving ball from the multiple exposure image. Object extracting means for extracting an image area, fitting means for fitting so that a semicircular curve having a preset diameter is formed on each side edge of the extracted ball image area, and the fitted ball A sensing processing unit including coordinate extracting means for extracting the coordinates of at least two center points with respect to the image region.

  Meanwhile, a sensing processing method according to an embodiment of the present invention includes a step of obtaining a multiple exposure image for an object moving under strobe illumination; and an image of an object moving due to multiple exposure is superimposed on the multiple exposure image. Extracting the object image region indicated by :; generating at least two center point candidates for the extracted object image region; adjusting the position of the center point candidate according to a predetermined condition; Extracting the coordinates of at least two center points for the image region.

  According to another embodiment of the present invention, there is provided a sensing processing method comprising: obtaining a multiple exposure image of a plurality of frames for an object moving under strobe illumination; and recognizing an initial position of the object on the multiple exposure image. Generating a line penetrating each image area on the multi-exposure image; and setting the direction of the line closest to the initial position among the generated lines as the motion direction of the object Estimating and extracting and analyzing an image relating to the object from each image on the estimated direction of motion.

  According to still another embodiment of the present invention, a sensing processing method includes: obtaining a multiple exposure image of a plurality of frames for an object moving under strobe illumination; and determining an initial position of the object on the multiple exposure image. Recognizing; and combining multiple exposure images of at least two or more frames into one; performing a line scan at a predetermined angle around the initial position in the combined images, Calculating a predetermined function value based on a pixel value of an image; estimating a direction of a scan line having a maximum value among the calculated function values as a movement direction of the object; and the estimated movement Extract images about the object from each image in the direction Including; and analyzing.

  According to still another embodiment of the present invention, a sensing processing method includes: obtaining a multiple exposure image for a ball moving under stroboscopic illumination; and from the multiple exposure image, an image of a ball moving due to multiple exposure. Extracting a ball image area shown superimposed on each other; fitting to form a semicircular curve having a preset diameter on both side edges of the extracted ball image area; and the fitting Extracting the coordinates of at least two center points for the rendered ball image region.

  Meanwhile, a virtual golf simulation apparatus according to an embodiment of the present invention includes a sensor unit that acquires a multiple exposure image for an object that moves under strobe illumination; and an image of an object that moves due to multiple exposure from the multiple exposure image. First object processing means for extracting an object image area shown in an overlapping manner, and generating at least two center point candidates for the extracted object image area, and adjusting the position of the center point candidate, thereby the object image A sensing processing unit including second processing means for extracting coordinates of at least two center points with respect to the area; calculating physical information about the moving golf ball by calculating changes in the coordinates of the center points; Simulation to simulate the trajectory of And; including.

  Although the sensing device, the sensing processing method, and the virtual golf simulation apparatus using the sensing method for a moving object according to the present invention acquire a multi-exposure image for a moving object with a low-resolution low-speed camera device and a strobe device, The image of the object on the acquired image is accurately extracted, especially when multiple object images are shown superimposed by acquiring multiple exposure images for objects moving at low speed. By accurately extracting the image of the object and extracting the coordinates of the center point, it is possible to realize a high sensing processing capability and sensing accuracy at low cost.

It is a block diagram which shows the sensing apparatus or virtual golf simulation apparatus which concerns on one Example of this invention. It is a figure which shows an example of the screen golf system to which the sensing apparatus or virtual golf simulation apparatus shown by FIG. 1 was applied. It is a figure which shows the operation signal system of the camera apparatus and strobe apparatus of the sensing apparatus which concerns on one Example of this invention. (A) And (b) is a figure which respectively shows the image pattern of the object acquired by the system shown by FIG. FIG. 5A is a diagram showing an example of an actually acquired image of the image of the pattern shown in FIG. 4A, and FIG. 5B is an image shown in FIG. It is a figure which shows the image by which the pre-processing with respect to was carried out. It is a figure which shows an example of the process which estimates the motion direction of the ball | bowl by the sensing apparatus based on one Example of this invention. It is a figure which expands and shows the A area | region of FIG. It is a figure which shows another example of the process which estimates the movement direction of the ball | bowl by the sensing apparatus based on one Example of this invention. It is a figure which shows another example of the process which estimates the movement direction of the ball | bowl by the sensing apparatus based on one Example of this invention. It is a figure which shows another example of the process which estimates the movement direction of the ball | bowl by the sensing apparatus based on one Example of this invention. FIG. 11 (a) shows an example of a ball image region shown superimposed on each other, and FIG. 11 (b) shows a pixel checked along the axis of motion direction for the image shown in FIG. 11 (a). It is a graph which shows distribution of a value. FIG. 12A shows a process of fitting to the image area of the ball along the movement direction axis, and FIG. 12B shows the movement while moving along the movement direction axis in FIG. It is a graph which shows the pixel value on a semicircle added together. FIG. 13 is an image showing a state in which a semicircle fitting to a ball image region and a center point candidate are generated by the method shown in FIGS. (A) And (b) is a figure for demonstrating the process which adjusts the position of the axis | shaft of the estimated motion direction of a ball so that the center of the fitted ball image area may be penetrated. FIG. 15A is a diagram showing a process of adjusting the position of the center point candidate generated with respect to the fitted ball image area, and FIG. 15B is a process shown in FIG. 15A. It is a figure which shows that the center point was adjusted by. It is a figure for demonstrating the process which correct | amends a center point so that it may respond | correspond to the space | interval ratio of strobe illumination with respect to the center point of a ball image. It is a flowchart which shows the process of the sensing processing method which concerns on this invention. It is a flowchart which shows the process of the sensing processing method which concerns on this invention.

  Embodiments relating to a sensing device for a moving object, a sensing processing method, and a virtual golf simulation device using the same according to the present invention will be described more specifically with reference to the drawings.

  The sensing device according to the present invention is for obtaining physical information by acquiring and analyzing an image of a moving object, where the moving object is a sport using a ball such as golf. For example, the present invention can be applied to sensing a ball that is hit and moved by a golfer, and as an example, it can be applied to a so-called screen golf system to which a virtual golf simulation apparatus is applied.

  1 and 2 show a schematic configuration of such a sensing device according to the present invention and a virtual golf simulation device using the sensing device.

  First, with reference to FIG.1 and FIG.2, the sensing apparatus which concerns on one Example of this invention, and the virtual golf simulation apparatus using the same are demonstrated.

  As shown in FIG. 1, a sensing device for a moving object according to an embodiment of the present invention is obtained by a sensor unit including camera devices 310 and 320, a strobe device 330, a signal generation unit 210, and the like, and the sensor unit. The image processing apparatus includes a sensing processing unit 220 that processes the image of the moving object and extracts the coordinates of the center point by processing the image.

  Although FIG. 1 shows a case where two camera apparatuses 310 and 320 are provided, the present invention is not limited to this, and all cases including one or more camera apparatuses can be included.

  The camera devices 310 and 320 are equipped to acquire images of a plurality of frames with respect to a state in which the object moves along the movement direction from the initial position of the object.

  The strobe device 330 is an illuminating device using an LED or the like, and is used as a photographing light source for the camera device. A multiple exposure image is acquired by the camera devices 310 and 320.

  That is, a multi-exposure image that appears after an object is photographed as many times as the number of flashes by the strobe device 330 is obtained in an image of one frame photographed by the camera device.

  Specific matters regarding the operation of the camera devices 310 and 320 and the strobe device 330 will be described later.

  On the other hand, a trigger signal for operating the camera devices 310 and 320 and the strobe device 330 is generated by a signal generator 210, and multiple exposure images by the camera devices 310 and 320 and the strobe device 330 are subjected to a sensing process. The data is processed by the unit 220 and transmitted to the simulator 100.

  The sensing processing unit 220 extracts, from the multiple exposure image, a first processing unit 230 that extracts an object image region in which images of objects moving due to multiple exposure are superimposed on each other, and at least the extracted object image region. And second processing means 240 for extracting at least two center points for the object image region by generating and adjusting two center point candidates.

  The first processing unit 230 includes a pre-processing unit 231 that removes background and noise in the multiple exposure image, a motion direction estimation unit 232 that estimates a motion direction of the object in the multiple exposure image, and the estimated motion. It is preferable to include object extraction means 233 for extracting object image regions for objects superimposed on each other from images in the direction.

  The motion direction estimation means 232 estimates the motion direction of the object from the first motion direction estimation means for estimating the motion direction of the object based on the position information of the initial object, and the pixel value of each image area on the multiple exposure image. And a second motion direction estimating means. The movement direction estimation means 232 may include only one of the first movement direction estimation means and the second movement direction estimation means, or both. When both the first motion direction estimation unit and the second motion direction estimation unit are provided, first, the first motion direction estimation unit is applied to estimate the motion direction of the object, and the first motion direction estimation unit temporarily When the estimated motion direction gives an unfavorable result, the motion direction can be estimated by applying the second motion direction estimation means.

  The object extraction unit 233 can extract the object image region by analyzing the pattern of each image in the estimated movement direction in order to effectively extract the object image region for the superimposed object. Pattern analysis means for checking, for each image on the estimated motion direction, checking pixel values along the motion direction, and analyzing the pattern for the checked pixel values, thereby the object image region And the like.

  On the other hand, the second processing unit 240 fits the extracted object image area with a value set in advance as the size of the object, and selects at least two center point candidates for the fitted object image area. Fitting means 241 to be generated and coordinate extraction means 242 for extracting at least two center points with respect to the object image area by adjusting the position of the generated center point candidate according to a predetermined condition. Is preferred.

  More specific matters regarding the first processing unit 230 and the second processing unit 240 will be described later.

  Meanwhile, the simulator 100 preferably includes a control unit M, a database 110, a video processing unit 120, a video output unit 130, and the like.

  The control unit M receives the coordinate information of the object obtained by image processing by the sensing unit 220, converts the coordinate information into three-dimensional coordinate information, and performs predetermined physical motion simulation for object motion trajectory. Information is calculated and transmitted to the video processing unit 120.

  At this time, predetermined data for simulating the object's motion trajectory is extracted from the database 110 and used. This can be done by extracting data.

  Here, a converting means for converting the coordinate information of the object transmitted from the sensing processing unit 220 into three-dimensional coordinate information may be provided separately from the control unit M.

  In addition, the first processing unit 230 and the second processing unit 240 may be embodied as one controller provided to perform the function of each unit in terms of hardware, and are separately provided for each unit. The controller may implement the corresponding function. Moreover, in terms of software, the functions may be embodied as a single program, or the corresponding functions may be embodied by separate programs for each means.

  FIG. 2 shows an example of a screen golf system to which a sensing device having the above-described device configuration or a virtual golf simulation device is applied.

  As shown in FIG. 2, a golf tee on which a golf ball 10 is placed on one side of a golf booth B and a ground (preferably including at least one of a fairway mat, a rough mat, and a bunker mat) are provided. A hitting mat 30 and a swing plate 20 provided for a golfer to perform a golf swing are provided.

  In front of the golf booth B, a screen 40 on which a virtual golf simulation video realized by the video output unit 130 is displayed is provided, and camera devices 310 and 320 and a strobe device 330 are provided on the ceiling, respectively.

  FIG. 2 shows the case where the camera devices 310 and 320 are respectively provided on the ceiling and the wall. However, the present invention is not limited to this, and it is possible to effectively acquire an image relating to the motion state of the golf ball 10 while swinging by a golfer. It can be installed at any position as long as it is a preset position that does not interfere with the golf ball hit by the golfer and does not collide with the golf ball.

  FIG. 2 shows the case where the strobe device 330 is attached to the ceiling and installed to provide strobe lighting in a direction substantially perpendicular to the striking point, but is not limited thereto. Any location that can provide effective strobe lighting is possible.

  In the system as described above, when the golf player hits the golf ball 10 on the hitting mat 30 toward the screen 40 with the swing plate 20, as shown in FIG. 2, a predetermined area where the hit is made is photographed. The camera devices 310 and 320 respectively acquire images of a plurality of frames. At this time, the strobe device 330 applies a plurality of flashes per frame, and acquires a multiple exposure image of the moving golf ball.

  FIG. 3 shows a trigger signal generation system of the camera device and strobe device by the signal generator 210 (see FIG. 1).

  As shown in FIG. 3, the trigger signal for the camera device is made at tc time intervals. That is, a trigger signal is generated at tc time intervals per frame. At this time, the time exposed per frame is te time (in this case, it is preferable to satisfy tc> te), and the data for the image acquired during tc-te time is the sensing processing unit. It is preferable to transmit to 220 (see FIG. 1).

  That is, as shown in FIG. 3, the camera device is exposed during the trigger signal generation time interval per frame of the camera device, and data for the acquired multiple exposure image is transmitted to the sensing processing unit.

Then, strobe illumination by the strobe device is generated a plurality of times during the exposure time (te time) of the camera device, and FIG. 3 shows a case where the strobe illumination trigger signal is generated three times at a time interval of ts ₁ . .

That is, during the exposure time (te time) of the camera device, the strobe illumination occurs three times at equal time intervals of ts ₁ hour. At this time, it is preferable that the signal generator 210 (see FIG. 1) synchronizes the camera device and the strobe device so that the first trigger signal is generated simultaneously.

Also, as shown in FIG. 3, the time from the last strobe illumination trigger signal to the first strobe illumination trigger signal of the next frame among the three strobe illuminations is set to an interval of ts ₂ hours. It is preferable to do. The ts ₁ and ts ₂ time intervals may be set to be the same, but as shown in FIG. 3, it is preferable to set the ts ₁ time and the ts ₂ time to be different from each other, more preferably , Ts ₂ time is preferably set to be longer than ts ₁ hour.

  As described above, since the interval between the trigger signals of the camera device and the strobe device is fixed to be constant, the interval between the object images on the acquired multiple exposure image must be constant.

  Therefore, when the multiple exposure image is processed by the sensing processing unit, the accurate object image from the object image and various noises existing on the multiple exposure image is related to the fixed interval of the trigger signal described above. Can be effectively separated based on

  In addition, since the strobe illumination interval per frame of the camera device is fixed, images of various patterns can be generated in the multiple exposure image depending on the motion speed of the object.

  Hereinafter, the moving object will be described by taking a ball (golf ball) that moves by being hit by a golf club as an example.

  In FIG. 4, two patterns (a) and (b) of the multiple exposure images acquired by the camera device and the strobe device based on the signal generation system shown in FIG. 3 are shown.

  That is, in FIG. 4, the state in which the ball moves by being hit by the golf club is shown using images of two patterns acquired by the trigger signal system shown in FIG. 3 (I1 and I2). .

  In the image (I1) shown in FIG. 4A, when the golf club images C1, C2, C3 and the ball images 11a, 11b, 11c are acquired by multiple exposure, the ball images 11a, 11b, 11c are obtained. This is a case where they are shown separated from each other at a predetermined interval.

  Also, the image 12 shown in FIG. 4B shows a case where the balls are shown as an image region 12 having a predetermined size by overlapping each other.

  That is, the image shown in FIG. 4A is a case where the image is formed at a predetermined distance when each strobe illumination is triggered because the ball moves at high speed. The image shown in FIG. 4 (b) is a case where strobe illuminations are triggered and superimposed on each other before the ball travels far because the ball moves at a low speed.

  As shown in FIG. 4B, the sensing device and the sensing processing method according to the present invention allow multiple exposure images by moving a ball at a low speed equal to or less than a predetermined speed according to a fixed period of strobe illumination. When the balls are shown superimposed on each other, the coordinates of the center point in the superimposed ball image are obtained accurately.

  Hereinafter, with reference to FIG. 5 to FIG. 18, the processing of the ball images superimposed on each other as shown in FIG. 4B by the sensing device for the moving object according to the present invention will be described.

  FIG. 5 (a) shows the original multi-exposure image containing images of balls superimposed on each other and was acquired using a 320 × 240 low resolution 75 fps camera device and a 330 Hz strobe device. It is an image.

  With respect to the original image as shown in FIG. 5A, the preprocessing unit 231 (see FIG. 1) of the first processing unit stops an image through subtraction or the like, that is, a background image or the like. And a predetermined preprocessing process such as Gaussian Blur is performed. FIG. 5B shows an image after the preprocessing process.

  However, since the balls are superimposed on each other and the resolution of the camera device is low, that is, the image is not an image having a resolution that can clearly distinguish that the balls are superimposed on each other. , The shape of the image area of a predetermined size. This is shown confused with various noises, such as an image area of a predetermined size for a part of a golfer's body and an image area of a predetermined size for a golf club, so there is such a variety of noise In this state, it is very difficult to extract only the image area related to the superimposed ball.

  As described above, as one method for extracting the superimposed ball image, the sensing device according to the present invention uses the movement direction estimation means 232 (see FIG. 1) to estimate the direction in which the ball moves, An image on the estimated motion direction can be extracted first, and a ball image area can be extracted from the extracted image.

  FIGS. 6 to 10 are diagrams illustrating a ball motion direction estimation process according to a sensing device and a sensing processing method according to an embodiment of the present invention. First, referring to FIGS. The process of estimating the motion direction of the ball by the estimating means will be described.

  First, a line L penetrating all image regions R existing in an image in which the background and noise are appropriately removed from the multiple exposure images of all frames acquired first is generated.

  That is, at least two points of all the image regions R are selected, and a line connecting them is generated as shown in FIG.

  At this time, information regarding the initial position B1 of the ball must be set in advance. That is, the camera device senses the initial position of the ball before the ball moves, and sets the initial position B1 of the ball on the multiple exposure image based on information on the detected position.

  The distance from the initial position B1 of the ball is measured for all the lines L generated as described above.

  That is, referring to FIG. 7 showing an enlarged view of the area A in FIG. 6, the linear distance from the initial position B1 of the ball is measured for a plurality of lines passing through the R1 area and a plurality of lines passing through the R2 area.

  In FIG. 7, the L1 line passing through the R1 region and the L2 line passing through the R1 region and the R2 region are closest to the initial position B1 of the ball. For this reason, since the L1 line of the two lines is closest to the initial position B1 of the ball, the L1 line is temporarily estimated as the axis of the movement direction of the ball.

Here, when the linear distance between the initial position B1 of the ball and the L1 line is D _L , if the D _L is long enough to deviate from a predetermined setting range, the motion direction estimated by the above method is Since there is a high possibility that the movement direction is erroneously estimated, it is preferable to ignore the result estimated by the first movement direction estimation means as described above and to estimate the movement direction again by the second movement direction estimation means. .

  8 to 10 show the process of estimating the motion direction by the second motion direction estimation means.

  First, as shown in FIG. 8, the images of a plurality of frames are combined into one so that the ball image area is shown in one image over time. That is, FIG. 8 shows a case where each of the frame image areas 12-1, 12-2, and 12-3 is shown in one image.

  Then, as shown in FIG. 8, the motion direction is estimated by the second motion direction estimation means based on the images combined into one. This is illustrated in FIG.

  The second motion direction estimating means performs line scanning at a predetermined angle centering on the position of the initial object, and image combining means for combining multiple exposure images of at least two or more frames into one, and on each scan line. Line scanning means for checking pixel values of the image, and axis extraction means for extracting a corresponding scan line having the largest pixel value among the scan lines by the line scanning means as an axis of the movement direction of the object, It is preferable that it is comprised so that it may contain.

  That is, the image combining means of the second motion direction estimating means combines the images of a plurality of frames as shown in FIG. 8, and the image is shown in FIG. In this way, the axis of the movement direction of the ball can be extracted.

  As shown in FIG. 9, a scan area SR having a predetermined angle is designated with reference to the initial position B1 of the ball, and each scan line X1, with reference to the initial position B1 of the ball in the corresponding scan area SR. Check pixel values for images on X2, X3,... Xi,.

  At this time, the scan area lines s1 and s2 forming the scan area SR are preferably formed to have a preset angle. Although it may be set to be 360 °, it is preferably set to about 120 ° forward as shown in FIG.

  Pixel values for the images on the scan lines X1, X2, X3,... Xi,... Xn can be checked as shown in FIG.

  As shown in FIG. 10, a value set in advance as the radius value Rs of the ball (when the radius value of the ball is known and set in advance, and the radius value is measured and set in advance from the image of the ball). The first sub-scan line UL and the second sub-scan line LL are generated on one side and the other side around the scan line Xi, respectively, and the sum of the pixel values of the image on the scan line Xi is included. (Sum1), a sum of pixel values of the image on the first sub-scan line UL (Sum2), and a sum of pixel values of the image on the second sub-scan line LL (Sum3) are calculated.

  Thereafter, the following cost function value is calculated.

  v = Sum1-Sum2-Sum3

  All the function values as described above are obtained for each scan line, and the scan line having the maximum value is estimated as the axis of the movement direction of the ball.

  Thus, by calculating the cost function value of each scan line and the sub-scan line according to the radius of the ball with respect to the corresponding scan line, the image area is too small or too large as the image area of the ball. Since the ball image area can be accurately included, it is possible to accurately extract the axis of the movement direction passing through the ball image area.

  Therefore, in FIG. 9, the Xi scan line can be estimated as the axis of the movement direction.

  On the other hand, when the axis of the movement direction of the ball is extracted by the method as described above, the ball image region on which the object extraction means 233 (see FIG. 1) is superimposed is extracted using the extracted axis of the movement direction. To do.

  That is, as shown in FIG. 11, for all images through which the extracted motion direction axis penetrates (not only the ball image region to be extracted but also noise may be included), the pixels as shown in FIG. By checking the value, noise can be effectively removed and only the ball image area can be extracted.

  FIGS. 11A and 11B show an example of a method for effectively extracting only the ball image from the image existing on the motion direction axis.

  FIG. 11A shows an image of a specific pattern that may exist on the movement direction axis, and FIG. 11B shows the movement direction axis 13 with respect to the image shown in FIG. The distribution of pixel values checked along is shown.

  As shown in FIG. 11A, the motion direction axis 13 passes through the image area 12 and its surrounding noise N1 regarding the superimposed ball, and the pixel value of the image along the motion direction axis 13 The distribution shown in FIG. 11 has a pattern as shown in FIG.

  In the superimposed image area of the ball, the interval in which the pixel value is maintained above a predetermined critical value must be approximately equal to the interval obtained by adding the diameters of at least two balls, as shown in FIG. As described above, the interval maintained above a predetermined critical value in the distribution of pixel values is divided into an a1-a2 interval and an a3-a4 interval.

  Here, in the case where the interval in which the pixel value is maintained above the critical value is an image indicating the interval a3 to a4, that is, an image referred to as 12 in FIG. In the case of an image showing a1 to a2 intervals, that is, an image referred to as N1 in FIG. 11A, it can be estimated as noise (an image related to a golf club, etc.).

  Therefore, the N1 image can be excluded and an image called 12 can be extracted as an image area of the superimposed ball.

  On the other hand, as described above, after extracting the image area related to the superimposed ball from the image existing on the axis of the movement direction, the fitting means 241 (see FIG. 1) can appropriately apply the extracted image area. A process of performing fitting (Fitting) and extracting a center point proceeds.

  As one method of fitting to the superimposed ball image region, in FIG. 12 and FIG. 13, one semicircle is fitted to one end of the ball image region 12 in which the balls are superimposed on each other. Then, a semi-circle fitting method can be used in which the opposite half is fitted to the other end.

  As shown in FIG. 12A, the ball image area 12 has a higher luminance value than the surrounding area (where the acquired image is in gray scale, each pixel has a brightness value, That is, it has a luminance value).

  The edge of the ball image region 12 is a portion where the luminance value gradually changes, and the portion where the change in the luminance value ends is the outline of the ball image region, so that fitting is preferably performed at that portion.

  Therefore, for appropriate fitting, as shown in FIG. 12A, the first semicircle CC1 and the second semicircle are formed in both directions from the center of the ball image area 12 along the axis 13 of the movement direction. While moving each of the circles CC2, the sum of the pixel values of the image on the first semicircle CC1 and the sum of the pixel values of the image on the second semicircle CC2 are respectively calculated.

  The sum of the pixel values calculated as described above has a distribution like the graph shown in FIG. FIG. 12B is a graph showing the distribution of the sum of pixel values on one semicircle.

  In the graph shown in FIG. 12 (b), the pixel value suddenly changed from the point V1 or V2 where the pixel value started to change suddenly to the point V3 where the luminance value changed completely. Part.

  It is preferable that the semicircular fitting is performed by forming semicircular curves HC1 and HC2 (see FIG. 13) at any one of the above-described V1 to V3 points.

  That is, a specific value between the V1 point and the V3 point is preset as a set value for the sum of pixel values, and the sum of the pixel values on the respective semicircles CC1 and CC2 changes to a position where the set value changes below the set value. It is preferable that the fitting is performed by forming the semicircular curves HC1 and HC2, respectively.

  Therefore, the value at the V1 point may be preset as the set value, the value at the V2 or V3 point may be preset as the set value, and an arbitrary value between the V1 point and the V3 point may be set as the set value. May be set in advance. Such setting value setting can be derived through experiments or the like to determine which value can be obtained as the setting value in advance when the most accurate result can be obtained.

  Here, as the diameters of the semicircles CC1 and CC2 or the semicircular curves HC1 and HC2, the value set in advance as the diameter of the ball may be used, or the diameter of the ball may be measured from the ball image. .

  That is, it may be a value obtained by measuring the diameter of the ball in advance and inputting this in advance. The camera device senses the ball at the initial position, and measures and sets the diameter in advance on the image. It may be a saved value.

  Further, as shown in FIG. 4A, in the image in which the balls are shown separated from each other, the diameter of the balls can be measured and used for fitting to the superimposed image of the balls.

  On the other hand, in FIG. 13, along the axis 13 of the movement direction, one of the points V1 to V3 is selected in the distribution of the sum of the pixel values on the semicircles CC1 and CC2, and the ball of a preset ball is selected. The figure shows that fitting is performed by forming semicircular curves HC1 and HC2 based on the diameter or the diameter of the measured ball at both ends of the image area 12, respectively.

  That is, a first semicircular curve HC1 is formed and fitted at one end of the ball image area 12, and a second semicircle curve HC2 is formed and fitted at the other end of the ball image area 12. It is preferable.

  The semicircle fitting as described above can be performed by one processing means. The pixel value is checked while the first semi-circle CC1 and the second semi-circle CC2 are moved to both side edges of the ball image area by the pixel checking means, and the both ends of the ball image area are checked by the fitting processing means. The fitting process can be performed so that a semicircular curve is formed in each part.

  Further, the semicircle fitting for one semicircle and the semicircle fitting for the other semicircle may be performed by different processing means.

  That is, the movement of the first semicircle CC1 and the check of the pixel values on the first semicircle CC1 are performed by the first pixel checking means, and the movement of the second semicircle CC2 and the pixels on the second semicircle CC2 are performed. The value check may be performed by a second pixel check means. Based on the result of the pixel check by the first pixel check means, the first fitting processing means performs fitting so as to form a first semicircular curve HC1 at one side end of the ball image area, and the pixel by the second pixel check means. Depending on the result of the check, the second fitting processing means can perform the fitting so as to form the second semicircular curve HC2 at the other end of the ball image area.

  On the other hand, in the graph shown in FIG. 12B, if semicircular curves HC1 and HC2 for fitting are formed at the point V3, the center point candidates EP1 and EP2 are found at the points V1 and V2. Can be generated.

  That is, the first center point candidate EP1 is generated on the side where the first semicircle curve HC1 is formed, and the second center point candidate EP2 is generated on the side where the second semicircle curve HC2 is formed. Is preferred. The center point candidates EP1 and EP2 are preferably generated on the axis 13 in the movement direction.

  Of course, the center point candidates EP1 and EP2 may be selected as arbitrary points on each semi-circle fitted side.

  The center point candidates EP1 and EP2 generated as described above need to be adjusted to the exact center point position by the coordinate extraction means 242 (see FIG. 1). Moreover, it is preferable that the position of the axis in the movement direction is also adjusted to a more accurate position.

  First, the case of adjusting the position of the axis in the movement direction will be described with reference to FIG.

  The axis of motion direction preferably passes through the center of the ball image area, but the axis of motion direction is estimated rather than accurately extracted, so the axis of motion direction estimated May not accurately penetrate the center of the ball image area.

  In such a case, it is preferable to adjust the position of the motion direction axis so that the estimated motion direction axis can accurately penetrate the center of the ball image area.

  As shown in FIG. 14 (a), the upper and lower thicknesses of the movement direction axis 13 penetrating the ball image region 12 must be substantially the same. It is preferable to shift the position of the movement direction shaft 13 by an amount corresponding to the difference between the lower thickness t1 and the upper thickness t2 of the movement direction shaft 13.

  FIG. 14B shows a state in which the axis in the movement direction has moved to the center line CL passing through the center of the ball image region 12 by the method described above.

  On the other hand, FIG. 15A shows a process of adjusting the center point candidates EP1 and EP2 generated by the method shown in FIGS.

  As shown in FIG. 15A, since the fitted semicircular curves HC1 and HC2 have a semicircular shape, the center points candidates EP1 and EP2 are accurately centered on the basis of the geometric condition. It is possible to adjust to a point.

  That is, the three-direction lines r1, r2, and r3 have the same length with respect to the first center point candidate EP1, and the three-direction lines r4, r5, with respect to the second center point candidate EP2. By adjusting the lengths of r6 to be equal to each other, adjustment is performed so that the positions of the center point candidates EP1 and EP2 are formed at correct positions.

  That is, by the method as described above, the center point candidates EP1 and EP2 are adjusted so as to be positioned at the centers of the circles with respect to the fitted semicircular curves HC1 and HC2, respectively.

  At this time, the lengths of the three-direction lines r1, r2, r3 or r4, r5, r6 may be matched with a preset value as the diameter of the ball, or matched with the diameter measured through the ball image. Also good.

  With the method described above, as shown in FIG. 15B, two center point coordinates CP1 and CP2 of the ball image region 12 are extracted.

  On the other hand, in the image area extracted by the method as described above, an image other than the image area related to the actual ball may be included.

  In order to prevent such an error in advance, as shown in FIG. 16, it is checked whether or not the extracted center point is accurate based on the period of strobe illumination for each frame, etc. It is also possible to correct the result when the center point is set.

That is, as shown in FIG. 3, the time interval of strobe illumination in one frame is ts ₁ and the time interval between the last strobe illumination of one frame and the first strobe illumination of the next frame is ts _2. The distance interval between the ball image areas 12-1, 12-2, and 12-3 of each frame in FIG. 11 must correspond with a predetermined ratio.

That is, the ratio of ts _{2 to} 2ts ₁ must correspond to the ratio of b to a or the ratio of d to c.

  If the interval with respect to the extracted center point does not correspond to the characteristics of the period of the strobe illumination, the result can be corrected to correspond to this.

  For example, if there is no center point at the position where it must be located due to the characteristics of the strobe lighting cycle, the center point can be corrected to be generated at the corresponding position. When the center point is located at a position where the point should not exist, the corresponding center point can be deleted.

  As described above, after the center point is extracted and corrected, the final center point coordinates for the superimposed ball image area can be determined.

  Therefore, in order to take advantage of the low quality images acquired by low resolution low speed camera devices and strobe devices, even if the balls are shown as image areas of a predetermined size superimposed on each other, the center point for the moving ball Coordinates can be extracted very accurately.

  As described above, the coordinates of the center point of each ball finally determined are converted into three-dimensional coordinates by converting means, and based on this, the physical information of the motion trajectory of the simulated ball is obtained. Calculated.

  Meanwhile, a sensing processing method according to the present invention will be described with reference to FIG.

  As shown in FIG. 17, first, a trigger signal is applied by the signal generation unit, and the camera device and the strobe device respectively acquire a multiple exposure image regarding the motion state of the object at a predetermined cycle (S10).

  At this time, the initial position of the ball is sensed and recognized (S20). Thereafter, various image pre-processing steps for removing the background image and various noises from the acquired multiple exposure image are performed (S30).

  As described above, a plurality of lines penetrating each image area on the preprocessed image are generated (S40), and a line having the shortest linear distance from the initial position of the ball to each generated line is extracted. (S50), the axis of the movement direction of the ball is estimated.

  At this time, if the distance from the initial position of the ball to the line forming the shortest distance is smaller than the set distance, the corresponding line is extracted as an axis in the movement direction (S61).

  If the distance from the initial position of the ball to the line forming the shortest distance is greater than the set distance, the axis of the motion direction estimated in steps S40 and S50 is considered to be erroneously estimated, and the second motion direction The process for estimation proceeds.

  That is, images for two or more frames among the preprocessed images are collected (S62), and a line scan is performed at a predetermined angle around the initial position of the ball (S63).

  At this time, a cost function (see FIG. 10) for the pixel value of each image area on each scan line is calculated (S64), and the scan line having the maximum value among the calculated cost functions (cost function). Is extracted as an axis of motion direction (S65).

  By extracting and analyzing the image relating to the ball from the image area on the axis of the motion direction extracted as described above, the coordinates of the center point of the ball can be finally obtained (S70).

  On the other hand, with reference to FIG. 18, step S70 of FIG. 17 will be described more specifically.

  As described above, the axis for the movement direction in which the ball moves is extracted by the movement direction estimation means, the ball image area is extracted from the image through which the extracted axis of the movement direction penetrates, and the extracted ball image area is stored in the extracted ball image area. For each pixel value on each semi-circle, a semi-circle based on a preset diameter or a measured diameter is moved from the center of the ball image area to both sides along the axis of motion. The sum is calculated (S71).

  Thereafter, a semicircular curve is formed at a position where the sum of the calculated pixel values has changed below the set value, and fitting to the ball image area is performed (S72).

  At least two center point candidates for the ball image region are generated on an axis of motion direction penetrating the ball image region (S73), and the generated center point candidates are positioned at the center of the circle with respect to the fitted semicircular curve. Adjustment is made so that it is possible (S74).

  Further, by correcting the position of the center point so as to correspond to the feature related to the period of the strobe illumination with respect to the center point adjusted as described above (S75), two center points for the ball image area are finally obtained. Can be acquired (S76).

By calculating the change in the three-dimensional space of the coordinates of the center point thus obtained, it is possible to calculate physical characteristic information of the moving ball.
[Item 4]
The sensing device according to claim 1, further comprising a correction unit that corrects the coordinates of the extracted center point so as to correspond to a cycle of the strobe illumination.
[Item 8]
The object extraction means includes
When the distance between the axis of the movement direction of the object extracted by the first movement direction estimation means and the initial object position is larger than a set value, the movement direction estimated by the second movement direction estimation means is applied. The sensing device according to claim 5, wherein the sensing device is configured to extract an image of the object.
[Item 11]
The fitting means includes
First semicircle fitting means for fitting so that a first semicircular curve having a preset diameter is formed on one side end of the superimposed ball image region;
2nd semicircle fitting means for fitting so that the 2nd semicircle curve of the above-mentioned diameter may be formed on the other side edge part of the above-mentioned superimposed ball image field, Item 9. The sensing device according to Item 8.
[Item 12]
The first semicircular fitting means includes:
A first pixel check means for calculating a sum of pixel values on a first semicircle of the set diameter while moving in a direction from the center of the superimposed ball image area to one side edge; First fitting processing means for fitting so that the first semicircular curve is formed at a position where the sum of the pixel values is changed below a preset value,
The second semicircular fitting means includes
Second pixel check means for calculating a sum of pixel values on the second semicircle of the set diameter while moving from the center of the superimposed ball image area toward the other end, and the calculation 11. A second fitting processing means for fitting so that the second semicircular curve is formed at a position where the sum of the pixel values is changed to a predetermined setting value or less. The sensing device described in 1.

Claims (17)

A sensor unit for acquiring a multiple exposure image for an object moving under strobe lighting;
Generating a first processing means for extracting an object image region in which images of objects moving by multiple exposure are superimposed on each other from the multiple exposure image; and generating at least two center point candidates for the extracted object image region. A sensing processing unit including a second processing unit that extracts coordinates of at least two center points with respect to the object image region by adjusting a position of the center point candidate. The first processing means includes
A movement direction estimating means for estimating a movement direction of the object in the multiple exposure image;
2. The sensing apparatus according to claim 1, further comprising: an object extracting unit that extracts object image regions for objects superimposed on each other from the image in the estimated movement direction. The second processing means includes
Fitting means for fitting to the extracted object image area to generate at least two center point candidates for the fitted object image area;
2. A coordinate extracting unit that extracts coordinates of at least two center points with respect to the object image area by adjusting a position of the generated center point candidate according to a predetermined condition. Or the sensing apparatus of 2. A sensor unit for acquiring a multiple exposure image for an object moving under strobe lighting;
First motion direction estimating means for estimating the motion direction of the object through position information of the initial object, second motion direction estimating means for estimating the motion direction of the object from pixel values of each image area on the multiple exposure image, and the first A sensing processing unit including an object extraction unit that extracts an image of the object from an image in the movement direction estimated by at least one of the first movement direction estimation unit and the second movement direction estimation unit; The first motion direction estimating means includes
Line generating means for selecting at least two points of each image area on the multi-exposure image and generating a line connecting them;
Line extraction means for extracting a line having the shortest distance from the position of the initial object among the generated lines as an axis of the movement direction of the object,
The second motion direction estimating means includes
Image combining means for combining multiple exposure images of at least two or more frames into one;
Line scan means for performing a line scan at a predetermined angle around the position of the initial object and checking the pixel value of the image on each scan line;
5. The apparatus according to claim 4, further comprising: an axis extracting unit that extracts a corresponding scan line having the largest pixel value among the scan lines by the line scanning unit as an axis in the movement direction of the object. The sensing device described. A sensor unit for acquiring a multiple exposure image of a ball moving under strobe lighting;
Object extracting means for extracting mutually overlapping ball image areas for a moving ball from the multiple exposure image, and semicircular curves having a preset diameter are formed on both end portions of the extracted ball image area, respectively. And a sensing processing unit including a fitting means for fitting so as to include a coordinate extracting means for extracting coordinates of at least two center points with respect to the fitted ball image region. The fitting means includes
Pixel check means for checking pixels on a semicircle of a preset diameter while moving from the center of the superimposed ball image area toward the ends of both sides,
The fitting processing means for forming a semicircular curve with the set diameter on each of both end portions of the ball image area according to a result of the pixel check. 6. The sensing device according to 6. Acquiring a multiple exposure image for an object moving under strobe illumination;
Extracting from the multiple exposure image an object image region in which images of objects moving by multiple exposure are shown superimposed on each other;
Generating at least two center point candidates for the object image region extracted in the extracting step;
Extracting the coordinates of at least two center points with respect to the object image region by adjusting the position of the center point candidate according to a predetermined condition. Extracting the object image region comprises:
Estimating the direction of motion of the object in the multiple exposure image;
Extracting an image in the estimated direction of motion from the multiple exposure image;
The sensing processing method according to claim 8, further comprising: extracting the object image region from each image on the motion direction estimated in the estimating step. The step of generating the center point candidate includes
Fitting so that a semicircular curve having a preset diameter is formed on both end portions of the object image region extracted in the extracting step;
The sensing processing method according to claim 8, further comprising: generating at least two center points with respect to the object image region fitted in the fitting step as the center point candidates. Acquiring multiple exposure images of multiple frames for an object moving under strobe lighting;
Recognizing an initial position of an object on the multiple exposure image;
Generating each line through each image area on the multiple exposure image;
Estimating the direction of the line closest to the initial position among the lines generated in the generating step as the movement direction of the object;
Extracting and analyzing an image related to the object from each image on the direction of motion estimated in the estimating step. The step of estimating the movement direction includes:
Of the generated lines, when the distance from the initial position of the line closest to the initial position is greater than a preset setting value,
Combining multiple exposure images of at least two or more frames into one,
Performing a line scan at a predetermined angle around the initial position to check the sum of the pixel values of the images on each scan line;
And estimating the direction of the corresponding scan line when it has the largest value among the sum of the pixel values of the image on each scan line as the movement direction of the object. Item 12. The sensing processing method according to Item 11. Acquiring multiple exposure images of multiple frames for an object moving under strobe lighting;
Recognizing an initial position of an object on the multiple exposure image;
Combining multiple exposure images of at least two or more frames into one,
Performing a line scan at a predetermined angle around the initial position in the image combined in the combining step, and calculating a predetermined function value based on a pixel value of the image on each scan line;
Estimating the direction of the scan line having the maximum value among the function values calculated in the calculating step as the movement direction of the object;
Extracting and analyzing an image related to the object from each image on the direction of motion estimated in the estimating step. The step of calculating the function value includes:
A first value for a sum of pixel values of an image on the corresponding scan line is calculated along each of the scan lines;
Calculating a second value relating to a sum of pixel values of an image on the corresponding line along a line generated on one side of the scan line at a preset interval;
Calculating a third value for a sum of pixel values of the image on the corresponding line along a line generated on the other side of the scan line at a preset interval;
The sensing processing method according to claim 13, further comprising: calculating a value obtained by subtracting the second value and the third value from the first value. Acquiring a multiple exposure image for a ball moving under strobe lighting;
Extracting, from the multiple exposure image, a ball image region in which images of balls moving due to multiple exposure are superimposed on each other;
Fitting so that a semicircular curve having a preset diameter is formed on both end portions of the ball image region extracted in the extracting step;
Extracting the coordinates of at least two center points with respect to the ball image region fitted in the fitting step. The fitting step includes
The semi-circle with the set diameter is moved from the center of the ball image area toward both end portions along the axis of the movement direction of the ball passing through the ball image area extracted in the extracting step. While calculating the sum of the pixel values on the semicircle;
Fitting so that a semicircular curve having the set diameter is formed at a position where the sum of the pixel values calculated in the calculating step changes to a preset set value or less. The sensing processing method according to claim 15, wherein: A sensor unit for acquiring a multiple exposure image for an object moving under strobe lighting;
Generating a first processing means for extracting an object image region in which images of objects moving by multiple exposure are superimposed on each other from the multiple exposure image; and generating at least two center point candidates for the extracted object image region. A sensing processing unit including second processing means for extracting the coordinates of at least two center points with respect to the object image region by adjusting the position of the center point candidate;
A virtual golf simulation apparatus comprising: a simulator that calculates physical information related to a moving golf ball by simulating a change in coordinates of the center point and simulates a trajectory of the golf ball.